Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
Compare data faster in Django
I have a model with phone_number and I'm comparing this phone number with phone_number column uploaded in csv file from user. I'm fetching all data from DB, converting Django-query to Pandas Dataframe and comparing. This works, but takes too much time - Especially fetching and conversion of data to the Dataframe takes a lot of time. Is there a way to make this processing faster? For example comparing without conversion or so? BTW.main_dataframe (user's csv) doesn't need to be strictly Dataframe. Or maybe finding values in DB instead of fetching all DB data? def process(self): main_dataframe = pd.read_csv(self.save_path + "/" + self.file_name, encoding='cp932', dtype={self.main_col: object}) # loading csv file main_dataframe = self.cleaner(main_dataframe) # cleaning data from errors all_hikanshou_db = pd.DataFrame(list(Hikanshou.objects.all().values())) # getting data from DB(postgress) and converting it to DF m1 = main_dataframe[self.main_col].isin(main_dataframe['phone_number']) # comapring main_dataframe['compared_phone_number'] = main_dataframe[self.main_col].where(m1) main_dataframe.to_csv("abc.csv", index=False, encoding="cp932") -
Django - Q lookup search is slow for over a million records in MySQL
I have a Django application which involves a search in the database table and filters out all the results and data. models.py class TableData(models.Model): field1 = models.CharField(max_length=254, unique=True, null=False) field2 = models.CharField(max_length=254) field3 = models.CharField(max_length=254) field4 = models.CharField(max_length=254) field5 = models.CharField(max_length=50) field6 = models.CharField(max_length=10) field7 = models.CharField(max_length=10, null=False, default='200') slug = models.SlugField(max_length = 254, unique=True) search_qs = models.CharField(max_length=254, default='N/A') views.py def search(request): searchquery = request.GET.get("q") if searchquery: all_queryset_list = TableData.objects.all() queryset_list = all_queryset_list.filter(Q(search_qs__icontains=searchquery)) paginator = Paginator(queryset_list, 25) page = request.GET.get('page') searchres = paginator.get_page(page) res_count = queryset_list.count() context = { 'searchres': searchres, 'res_count': res_count, } return render(request, 'results.html', context) Here everything works fine as expected, but the searched results takes too much of time for the table having over a million records. How can I optimize the query to get the results faster ? Really appreciate for anyone who can help me this. -
How to do filtering in DateTime as null in Django rest framework?
I just want to build the filtering API with Django rest framework. One field is datetime2. I want to filter the records that have NULL value as datetime2. Here is my models.py ruleend = models.DateTimeField(db_column='RuleEnd', blank=True, null=True) Here is my serializer.py class ProductPriceSerializer(serializers.ModelSerializer): class Meta: model = ProductPrice fields = ['pricingruleid', 'productkey', 'productcode', 'customerid', 'customerchain', 'productpriceusd', 'rulestart', 'ruleend', 'creatorupn', 'created', 'customername', 'productdescription'] Here is views.py class ProductPriceViewSet(viewsets.ModelViewSet): serializer_class = ProductPriceSerializer pagination_class = pagination.PageNumberPagination filter_backends = [DjangoFilterBackend, filters.OrderingFilter] filterset_fields = ['customerid', 'customername', 'productkey', 'productdescription', 'ruleend', 'rulestart'] ordering_fields = ['customerid', 'productkey', 'rulestart'] def get_queryset(self): self.pagination_class.page_size_query_param = 'page_size' return ProductPrice.objects.all() urls.py path('productprice/', ProductPriceViewSet.as_view({'get':'list','post':'create'}), name="product-price"), This makes me implement pagination, filtering and sorting very easy. But I have one problem. I need to filter the records that have the null value in ruleend. What is the valid date/time of null? Or should I add something to my backend? -
Finding available workers for each date
In my application I have to display how many workers are available on each day from the current date to a year in advance. I have trouble figuring out how to do 2 things 1. How to calculate how many people are available The way I set it up is I create a new model object each time a worker is assigned to work on a project. There are multiple projects going on at the same time and workers only assigned to one project at a time. Model looks like this -> class AssignedToProject(models.Model): worker = models.ForeignKey(Worker, on_delete=models.CASCADE) project = models.ForeignKey(Project, on_delete=models.CASCADE) start_date = models.DateField(auto_now=False, auto_now_add=False) end_date = models.DateField(auto_now=False, auto_now_add=False) def __str__(self): return str(self.worker) + " - " + str(self.project) I had a few ideas how to go about this but none of them seem quite right. To create a string with length 365 that consists of 1s and 0s depending on whether the worker is available that day. Then I would just get availability strings for all workers and add them up for each day and that would give me a number telling me how many people are available that day. Problem with this is I think I would … -
What is the django version of "flask.request.url"?
I was just converting a code from flask to django and I was wondering what should I use in django instead of flask.request.url. -
How to download a newly created csv file in django?
I have created a django application that converts txt to csv files. Now, I want to create a view function to download the newly converted csv files. Any suggestions will be highly appreciated. views.py def submit(request): # this function converts txt to csv files. For example abcd.txt is converted into abcd0000.csv and abcd0000.csv is stored in /main/temp/ folder. Now I have just created the download function which is wrong. def download(request): file_path = os.path.join(BASE_DIR, 'main/temp/??') if os.path.exists(file_path): with open(file_path, 'rb') as fh: response = HttpResponse(fh.read()) response['Content-Disposition'] = 'inline; filename=' + os.path.basename(file_path) return response raise Http404 What should be the correct way to write the newly created path for the csv file that it downloads the newly created csv file? And also please suggest what should I write in urls.py to route to download view? -
In wagtail, wanna show user profile. if already user created means wanna show profile data, otherwise wanna show profile form
I already created user profile table for storing. now how can I redirect from same option to user profile data if already user data exist in table. -
Models aren’t shown in django admin
I am currently working on a science django blog. I've finished coding it all and I realized that the models items overview and content don't appear in my django.admin. Here's an image so you can see. I know there are other threads related to it. However, I couldn't find the solution in those. Models.py: class Post(models.Model): title = models.CharField(max_length=100) overview = models.TextField() timestamp = models.DateTimeField(auto_now_add=True) content = HTMLField() author = models.ForeignKey(Author, on_delete=models.CASCADE) thumbnail = models.ImageField() categories = models.ManyToManyField(Category) featured = models.BooleanField() previous_post = models.ForeignKey( 'self', related_name='previous', on_delete=models.SET_NULL, blank=True, null=True) next_post = models.ForeignKey( 'self', related_name='next', on_delete=models.SET_NULL, blank=True, null=True) Admin.py: from django.contrib import admin from .models import Author, Category, Post, Comment, PostView # Register your models here. admin.site.register(Author) admin.site.register(Category) admin.site.register(Post) Settings.py: INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.sites', 'allauth', 'allauth.account', 'allauth.socialaccount', 'crispy_forms', 'tinymce', 'marketing', 'posts' ] And urls.py: urlpatterns = [ path('admin/', admin.site.urls), By the way, the models.py is in the app Posts. Just in case somebody needs this information. Also, the other models like author, category and featured are seen normally. -
i want to set maximum quantity but getting error
I am trying to add maximum quantity.Here is my code : def post(self, request): product = request.POST.get('product') cart = request.session.get('cart') remove = request.POST.get('remove') if cart: quantity = cart.get(product) if quantity: if remove: cart[product] = quantity - 1 else: cart[product] = quantity + 1 else: cart[product] = 1 if cart[product] < 1: cart.pop(product) else: cart = {} cart[product] = 1 request.session['cart'] = cart return redirect("cart/") here when I try if quantity <= 2 i get this error <= not supported. So I think there will be better way to set maximum quantity. Can someone please help? -
could not enter into if condition: if request.method == 'POST':
I have used Django 2.2 to build a website, however the if condition: "if request.method == 'POST': " could not be hit in the if condition. view.py: def check_discilpine(request): if request.method == 'POST': discipline_name=request.POST.get('discipline_name') dropdown_semester=request.POST.get('dropdown_semester') dropdown_year = request.POST.get('dropdown_year') print("-----------------------------------------------") print(discipline_name, dropdown_semester, dropdown_year) return render(request,'specific-semester.html', context={'discipline_name': discipline_name, 'dropdown_semester': dropdown_semester, 'dropdown_year': dropdown_year}) html: <form method="post" action="{% url 'bms:content_checklist_for_discipine_info' %}"> {% csrf_token %} <div class="row"> <div class="px-3"> <div class="d-sm-flex align-items-xl-stretch justify-content-between mb-4"> <h1 class="h4 mb-0 text-gray-800">&nbsp;&nbsp;Creating Content Checklist for:</h1> <select id="dropdown_semester_0" name="dropdown_semester_0" > <option value="">--Dropdown January or July--</option> <option value="Dropdown_Jan">January</option> <option value="Dropdown_Jul">July</option> </select> <select id="dropdown_year_0" name="dropdown_year_0" > <option value="">--Calender year since 2020--</option> <option value="Dropdown_2021">2021</option> <option value="Dropdown_2022">2022</option> <option value="Dropdown_2023">2023</option> <option value="Dropdown_2024">2024</option> <option value="Dropdown_2025">2025</option> <option value="Dropdown_2026">2026</option> <option value="Dropdown_2027">2027</option> <option value="Dropdown_2028">2028</option> </select> </div> </div> </div> <div class="row"> <div class="px-3"> <div class="d-sm-flex align-items-xl-stretch justify-content-between mb-4"> <h1 class="h4 mb-0 text-gray-800">&nbsp;&nbsp;Content Checklist(s) for Discipline:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</h1> </div> </div> <select name="discipline"> {% for discipline in query_results %} <option value="{{ discipline.name }}" name="discipline_name">{{ discipline.name }}</option> {% endfor %} </select> </div> <br> <br> <div class="col"> <label class="radio-inline"> <input type="radio" name="optradio" value="notspecific" checked>&nbsp;Latest Available Semester </label> <label class="radio-inline"> <input type="radio" name="optradio" value="specific">&nbsp;Specific Semester </label> <select id="dropdown_semester" name="dropdown_semester" > <option value="">--Dropdown January or July--</option> <option value="Dropdown_Jan">January</option> <option value="Dropdown_Jul">July</option> </select> <select id="dropdown_year" name="dropdown_year" > <option value="">--Calender year … -
Heroku Slug Size Too Large due to Keras [closed]
I'm deploying my app to heroku however their free tier only allow < 500MB packages, I'm currently using tensorflow and it takes > 500 MB, I can downgrade it to lower version or use the tensorflow-cpu instead, however, I'm also using Keras, which requires at least tensorflow=2.2.0 (so size > 500MB). I want to be able to use Keras while the size of my packages < 500MB. I have looked into these two questions but somehow downgrading Keras causes a lot of imcompatible issues. Deploy python app to Heroku "Slug Size too large" Error "Keras requires TensorFlow 2.2 or higher" -
nested annotations django with OuterRef
I want to annotate a field based on another queryset. return real_instances.annotate( submitted_count=user_enrolments.filter(course_progresses__module_progresses__module__id=OuterRef('id')).annotate( submitted_attempt=Exists( AssignmentModuleProgress.objects .filter( Q(module_progress__module__id=OuterRef(OuterRef('id'))) Q(module_progress__user=OuterRef('user')) & Q(module_progress__module_progress_course_progresses__course_progress_userenrolments__id=OuterRef("id")), is_submitted=True) )).filter(submitted_attempt=True).count(), Here i have a QuerySet by the name real_instances and i want to annotate a field submitted_count in it . user_enrolments is another queryset and im annotating another field in it by the name submitted_attemmpt to finally get my count . Here im having issue with OuterRef(OuterRef('pk')) which refers to the id of the real_instances object in iteration. Im getting the error: ValueError: This queryset contains a reference to an outer query and may only be used in a subquery. -
How to pass Django filtered object to views( xls download)?
I am trying to pass the filter object to my view function which is responsible to download data into .xls format(Earlier I am directly using the Model name so I am getting all the records.). My requirement is when the user will apply any filter then that object data should be pass to download views. Please find the below code. filter.py class Shift ChangeFilter(django_filters.FilterSet): id = django_filters.NumberFilter(label="DSID") class Meta: model = AAChange fields = ['ConnectID', 'EmailID','id','SerialNumber','Project_name','Shift_timing'] function in views.py to download DB records: ###########for Downloading ############### from django.utils import timezone now_aware = timezone.now() import xlwt,openpyxl def exportgen_data(request): model=ShiftChange allgen = ShiftChange.objects.all() gene_filter = ShiftChangeFilter(request.GET,queryset=allgen ) allgene = gene_filter.qs print('download:',allgene) response = HttpResponse(content_type='application/ms-excel') response['Content-Disposition'] = 'attachment; filename="Gene_ShiftTiming.xls"' wb = xlwt.Workbook(encoding='utf-8') ws = wb.add_sheet('GenShiftChange Data') # this will make a sheet named Users Data # Sheet header, first row row_num = 0 font_style = xlwt.XFStyle() font_style.font.bold = True columns = ['id', 'AConnectID', 'Shift_timing', 'EmailID', 'Vendor_Company', 'Project_name', 'SerialNumber', 'Reason', 'last_updated_time'] for col_num in range(len(columns)): ws.write(row_num, col_num, columns[col_num], font_style) # at 0 row 0 column # Sheet body, remaining rows font_style = xlwt.XFStyle() rows = allgene.values_list('id', 'AConnectID', 'Shift_timing', 'EmailID', 'Vendor_Company', 'Project_name', 'SerialNumber', 'Reason', 'last_updated_time') for row in rows: row_num += 1 for col_num in … -
how to take the hostname or base url
how to get a running base url? in admin.py, new function/method def your_api(self, object_id): """ Function save your api """ # print(reverse('admin:index')) parameters = Parameter.objects.filter(agregator=object_id) concate_endpoint_api = '' for parameter in parameters: if parameter.flag == 'NO': concate_endpoint_api += '{}{}={}'.format(BASE_URL, parameter.keys, parameter.values if parameter.values else 'None') concate_endpoint_api += '&' return concate_endpoint_api example javascript windows.location.origin http://example.com or http://localhost or http://127.0.0.1:8000 -
How to have a form in a DetailView page in django?
I'm building a really simple twitter clone with django. I have a TweetDetailView like this: class TweetDetailView(LoginRequiredMixin,DetailView): model = Tweet context_object_name = 'tweet' template_name = "twitter/tweet.html" What I want is a form for commenting on a tweet that appears on the TweetDetailView page. Can a django DetailView handle forms? Or should I use a function based view for this detail page? Any help is appreciated. -
Django, How to paginate a JsonResponse?
I'm working on a Django and JavaScript project. I'm aware that without JavaScript APIs the pagination would be much easier, but now I'm too far down the rabit hole :) My views.py from django.http import JsonResponse def post_list(request): posts_list = Post.objects.all() posts = [p.serialize() for p in posts_list] return JsonResponse(posts, safe=False) My JavaScript fetch('/post_list') .then(response => response.json()) .then(posts => { print_post(posts); }); Then the print_post function does its magic and finishes by appending all posts to a div. My HTML <div id="to_attach_post_1"></div> So, all posts print nicely in one page. However, if I have hundreds of posts that would not look nice, right? so I have to paginate the posts, lets say every 10 posts, while still sending a JsonResponse with the posts. Does anyone know how to paginate this? -
Django Not Found: /virtual/Scripts/dashApp/static/main.css
I have looked through SO and this seems to be a common issue among new django users however none of what I have attempted has fixed my issue. after running dev server http://127.0.0.1:8000/ I get not found error in CMD window Not Found: /virtual/Scripts/dashApp/static/main.css Not Found: /virtual/Scripts/dashApp/static/main.js the relevant html in my app as below <link href= virtual/Scripts/dashApp/static/main.css rel="stylesheet"></head> <script type="text/javascript" src= virtual/Scripts/dashApp/static/main.js ></script> both the .css and .js file live in C:\Users\James\Desktop\Scripts\django\dealerDash\virtual\Scripts\dashApp\static my settings.py: STATIC_URL = '/static/' STATICFILES_DIRS = [ os.path.join(BASE_DIR , 'static') I have tried wrapping virtual/Scripts/dashApp/static/main.css as "virtual/Scripts/dashApp/static/main.css " and /virtual/Scripts/dashApp/static/main.css but without joy.. -
Get the top n rows of each day from same table in Django
I am sure this is not a novel/new problem. I really tried to look into other solutions. However, I could not find how to solve this. I have a model like class Deal(models.Model): title = models.CharField(max_length=1500) viewCounter = models.SmallIntegerField(default=0) thumbsUpCounter = models.SmallIntegerField(default=0) createDateTime = models.DateTimeField(auto_now_add=True) Now I want to get top 10 (or less) deals ordering by thumbsUpCounter and viewCounter of every day. I tried to look into the Subquery and Outerref. However, I could not figure out how can I get the right results. ** I am using MySQL. Thanks in advance. -
Django url pattern to pass a date parameter [duplicate]
I'm passing date parameters "check_in" and check_out" from one view to another through URL. Right now, my url pattern defines them as string (ie, str:check_in), so I get something like this. http://127.0.0.1:8000/checkout/2020-12-16/2020-12-25 It works but it's not ideal; what if someone edited the url with other strings that are not dates? Is there a way for the url to throw an error if the pattern for a date is not respected or recognized? urls.py path('checkout/<str:check_in>/<str:check_out>', views.check_out, name='check_out') views.py def check_out(request, check_in, check_out): ... -
How can I let users add their email server account to a Django app?
I am making a CRM app using a Django framework and I want to make a functionality that allows users to add their email server account (IMAP/STMP) so that they can send emails to their customers, receive replies and get notifications of the replies from the app. The similar app is HubSpot which has a feature of connecting personal email. Any help would be much appreciated. -
Django bulk_create check for duplicate entry of multiple fields
I'm trying to use bulk_create for large dataset insert since i can't use get_or_create() because of N^N problem: users = User.objects.all() for user in users: seat_type = SeatType.objects.all()[random_entry] AnotherTable.objects.bulk_create( AnotherTable(user=user, seat_type=seat_type) , None, ignore_conflicts=True) The thing is i want to check for duplicate entry unique with 2 columns(user, seat_type) values For example: user |seat_type | ... ---------- | --------- | ... 1 |1 | ... 2 |2 | ... 3 |4 | ... 4 |1 | ... 1 |1 | ... 3 |4 | ... (1, 1) and (3, 4) is invalid when inserting But ignore_conflict only trigger on 1 unique column(which is not what i want which is unique value of 2 columns), i'm looking for something like defaults of get_or_create, My DataBase is MySQL -
Loaded django models dyanmically from the database
I know there is a way to generate the models classes from the database using the inspectdb command. But it need the models.py to store the generated models. How can I directly load the models to the memory and use it? -
Nginx cannot find my Django WSGI application bad gateway
I have been a while stuck trying to configure my Django rest framework WSGI API, but at the moment of configuring the SSL certificate on the nginx conf file everything stopped working, no matter what, I always receive a 502 Bad gateway right away when trying to connect to the API. I have to clarify, that if I set the config back to HTTP (No SSL), everything works just fine. I'm using gunicorn to run the WSGI Django App, and I'm running everything in docker containers, so far, I think docker configuration is OK, as everything works just fine when I config the nginx webserver for HTTP. the following is my current NGINX configuration file for HTTPS: user nginx; worker_processes 1; events { worker_connections 1024; } http { include /etc/nginx/mime.types; client_max_body_size 100M; upstream mysite_drf { server backend:8000; } server { listen 80; access_log /var/log/nginx/api_access.log; error_log /var/log/nginx/api_error.log; server_name mysite.com; location / { return 301 https://$host$request_uri; } } server { listen 443 ssl; ssl_certificate /etc/letsencrypt/live/mysite.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/mysite.com/privkey.pem; include /etc/letsencrypt/options-ssl-nginx.conf; ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; access_log /var/log/nginx/api_ssl_access.log; error_log /var/log/nginx/api_ssl_error.log; server_name mysite.com; location / { proxy_pass http://mysite_drf; proxy_ssl_session_reuse on; proxy_set_header Host $host; proxy_redirect off; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 100M; } location /static/ { alias /mysite_drf/static/; } } … -
Why does my Django app in Azure not respond to pings?
I am trying to publish my web app to Azure, but it fails when I try to connect. The app service plan I am deploying to is B1 (basic tier) with a Linux OS. I only have one app on the plan. Azure launches my app inside of a docker container. I'm using one of the prebuilt docker container images that Azure allowed me to select when I created the app. It comes with Python installed. I configured it to run my startup.sh file. I publish the app with these two commands: az login az webapp up --resource-group myresourcegroup --location mylocation --plan ASP-myresourcegroup-blah --sku B1 --name mywebsite These commands succeed, but my app fails when I connect. My startup.sh file has two commands inside: pip install -r requirements.txt python manage.py runserver The logs show the startup commands run and complete successfully, and that the website started successfully and is listening on port 8000, but then when Azure tries to ping my web app on that very same port, it says that it didn't get a response. Here are the logs (notice they are out of timestamp order... this is how they appeared in the log stream): 2020-12-16T02:20:04.585Z INFO - Waiting … -
Django tasks done associated with a project
I want to obtain the percentage of tasks performed associated with a project my model class Srv(models.Model): srv_year = models.CharField(max_length=4) slug = AutoSlugField(populate_from = 'srv_year', always_update = True) class Project(models.Model): srv = models.ForeignKey(Srv, on_delete=models.CASCADE, null = True, blank = True) project_title = models.CharField(max_length=200, unique = True) slug = AutoSlugField(populate_from = 'project_title', always_update = True) class Todo(models.Model): project = models.ForeignKey(Project, on_delete=models.CASCADE, null = True, blank = True) todo_title = models.CharField(max_length=200) slug = AutoSlugField(populate_from = 'todo_title', always_update = True, null = True) My view class index(ListView): model = Srv template_name = 'srv_list.html' ordering = ['srv_year'] def get_context_data(self, *args, **kwargs): context = super().get_context_data(**kwargs) context['get_percentage_done'] = Todo.objects.filter(state = True).count() * 100 / Todo.objects.all().count() return context But with this context['get_percentage_done'] = Todo.objects.filter(state = True).count() * 100 / Todo.objects.all().count() I get all the system tasks
