Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
The best setup for Gunicorn + Django or why workers and threads at the same time degrade performance?
On my backend I use Django + Gunicorn. I use Nginx too but that does not matter for this question. Db is Postgres. I want to improve performance of my application (i have some api points with heavy i/o operations that degrade the whole backend), so i decided to survey how gunicorn can help me with that. I have read these SOF answers, and these, and this very nice article and many more but in reality i'm getting rather strange results. I created this point to experiment with Gunicorn config (very simple one): As you can see i have created some i/o operation inside my view. I want to optimize the amount of RPS and to measure that i'm going to use WRK with basic config 2 threads with 10 connections each. Lets start gunicorn with that config (basic simplest sync without any workers): Then i'm getting these results: Ok, that correlates with reality because this api point usually response in 50-100ms. Look: Now lets increase amount of workers to 8: I'm getting: Oops: i supposed that i'll get x8 performance and 160 RPS, because as i thought, new worker works in separate parallel process. 1. So the first question … -
Traefik Django & React setup
Recently I came across server configuration using GitLab CI/CD and docker-compose, I have two separated repositories one for Django and the other for React JS on Gitlab. The Django Repo contains the following production.yml file: version: '3' volumes: production_postgres_data: {} production_postgres_data_backups: {} production_traefik: {} services: django: &django build: context: . dockerfile: ./compose/production/django/Dockerfile image: one_sell_production_django platform: linux/x86_64 expose: # new - 5000 depends_on: - postgres - redis env_file: - ./.envs/.production/.django - ./.envs/.production/.postgres command: /start labels: # new - "traefik.enable=true" - "traefik.http.routers.django.rule=Host(`core.lwe.local`)" postgres: build: context: . dockerfile: ./compose/production/postgres/Dockerfile image: one_sell_production_postgres expose: - 5432 volumes: - production_postgres_data:/var/lib/postgresql/data:Z - production_postgres_data_backups:/backups:z env_file: - ./.envs/.production/.postgres traefik: # new image: traefik:v2.2 ports: - 80:80 - 8081:8080 volumes: - "./compose/production/traefik/traefik.dev.toml:/etc/traefik/traefik.toml" - "/var/run/docker.sock:/var/run/docker.sock:ro" redis: image: redis:6 This is work perfectly using the Traefik, I have also the following code for React JS repo: version: '3.8' services: frontend: build: context: ./ dockerfile: Dockerfile expose: - 3000 labels: # new - "traefik.enable=true" - "traefik.http.routers.django.rule=Host(`lwe.local`)" restart: 'always' env_file: - .env Now I don't know how to connect both Django and React Js Repo using the Traefik and also how the CI/CD configuration should be, the following is the CI/CD configuration for Django Repo (I omitted unnecessary info and just include the deploy … -
get_throttling_function_name: could not find match for multiple
Traceback (most recent call last): File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube_main_.py", line 181, in fmt_streams extract.apply_signature(stream_manifest, self.vid_info, self.js) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\extract.py", line 409, in apply_signature cipher = Cipher(js=js) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\cipher.py", line 29, in init self.transform_plan: List[str] = get_transform_plan(js) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\cipher.py", line 198, in get_transform_plan return regex_search(pattern, js, group=1).split(";") File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\helpers.py", line 129, in regex_search raise RegexMatchError(caller="regex_search", pattern=pattern) During handling of the above exception (regex_search: could not find match for $x[0]=function(\w){[a-z=.(")];(.);(?:.+)}), another exception occurred: File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\handlers\exception.py", line 47, in inner response = get_response(request) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\handlers\base.py", line 181, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "C:\Users\Mohd Athar\Desktop\Django\New folder\youtubedownloader\downloadapp\views.py", line 13, in index stream = video.streams.get_highest_resolution() File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube_main.py", line 296, in streams return StreamQuery(self.fmt_streams) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube_main_.py", line 188, in fmt_streams extract.apply_signature(stream_manifest, self.vid_info, self.js) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\extract.py", line 409, in apply_signature cipher = Cipher(js=js) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\cipher.py", line 29, in init self.transform_plan: List[str] = get_transform_plan(js) File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\cipher.py", line 198, in get_transform_plan return regex_search(pattern, js, group=1).split(";") File "C:\Users\Mohd Athar\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\helpers.py", line 129, in regex_search raise RegexMatchError(caller="regex_search", pattern=pattern) Exception Type: RegexMatchError at / Exception Value: regex_search: could not find match for $x[0]=function(\w){[a-z=.(")];(.);(?:.+)} -
Preview of loaded CSV file with many columns
When the user drops the csv file in drag and drop section, He should be able to see a preview of that CSV file, but the problem is the csv file user can have many columns for example:- In the above image u can see due to the CSV file had many(23) columns it is now too long for my web page. Is it possible to contain it in a div tag or adding a scrollbar on the bottom of the table. I tried papaparse too but JS file of papaparse:- function Upload(){ var file = document.getElementById("fileUpload").value; data = Papa.parse(file, { worker: true, step: function(results) { Heiho(results.data); } }); } htmlfile of papaparse:- <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <link rel="stylesheet" href="heiho.css" /> <title>Document</title> </head> <body> <input type="file" id="fileUpload" /> <input type="button" id="upload" value="Upload" onclick="Upload()" /> <hr /> <div id="dvCSV"> </div> <script src="heiho.js"></script> <script src="https://cdn.jsdelivr.net/gh/mholt/PapaParse@latest/papaparse.min.js"></script> <script src="test.js"></script> </body> </html> Note:- I am not including paparse js and css file as it is too big u can download the file from the link below. What is papaparse? -
Django remember me error didn't return an HttpResponse object
I create forms.py like this. class LoginForm(forms.Form): username = forms.CharField(max_length=100,required=True) password = forms.CharField(max_length=32,required=True,widget=forms.PasswordInput) remember = forms.BooleanField(label="Remember me") In views.py I create function loginForm like this. def loginForm(request): if request.method == 'POST': form=LoginForm(request.POST) if form.is_valid(): username = form.cleaned_data.get('username') password = form.cleaned_data.get('password') remember = form.cleaned_data.get('remember') user=auth.authenticate(username=username,password=password) if user is not None: if remember == "on": request.session.set_expiry(1209600) auth.login(request, user) return redirect('/main') else: message = "User not found" messages.info(request, message) form = LoginForm() return render(request,'login.html',{'form':form}) else: form = LoginForm() return render(request,'login.html',{'form':form}) In login.html I create form like this. <form action="loginForm" method="post"> <h2>Login</h2> <p> <div class="mb-3"> {{form|crispy}} {% csrf_token %} </div> <button type="submit" class="btn btn-primary">OK</button> </p> </form> If I not check Remember me it show this message. It's not remember me button. Please check this box if you want to proceed when I login it show error. ValueError at /loginForm The view device.views.loginForm didn't return an HttpResponse object. It returned None instead. If I remove this line it have no error. if remember == "on": request.session.set_expiry(1209600) How to create login form with remember me? -
Django form login error didn't return an HttpResponse object
I create forms.py like this. class LoginForm(forms.Form): username = forms.CharField(max_length=100,required=True) password = forms.CharField(max_length=32,required=True,widget=forms.PasswordInput) remember = forms.BooleanField(label="Remember me") In views.py I create function loginForm like this. def loginForm(request): if request.method == 'POST': form=LoginForm(request.POST) if form.is_valid(): username = form.cleaned_data.get('username') password = form.cleaned_data.get('password') remember = form.cleaned_data.get('remember') user=auth.authenticate(username=username,password=password) if user is not None: if remember == "on": request.session.set_expiry(1209600) auth.login(request, user) return redirect('/main') else: message = "User not found" messages.info(request, message) form = LoginForm() return render(request,'login.html',{'form':form}) else: form = LoginForm() return render(request,'login.html',{'form':form}) In login.html I create form like this. <form action="loginForm" method="post"> <h2>Login</h2> <p> <div class="mb-3"> {{form|crispy}} {% csrf_token %} </div> <button type="submit" class="btn btn-primary">OK</button> </p> </form> If I not check Remember me it show this message. It's not remember me button. Please check this box if you want to proceed when I login it show error. ValueError at /loginForm The view device.views.loginForm didn't return an HttpResponse object. It returned None instead. How to create login form ? -
I have two tables named as intermediate_transaction and merchant_table. I have to find who is inactive user among all the users
I have to find the inactive users who is not doing any transaction in a week. If a user is doing transaction in 7 days atleast one then he is active user otherwise inactive. -
Is there a way to add Map coordinates in Django just as we add Datefield [duplicate]
I have been working on this project of address book application. I've completed the project and now I have one small task pending. Along with the address details, can I have a field in which someone can add a coordinate field. When we click on those coordinate it should redirect to Maps page(say google maps) where the place represent the coordinates should be displayed. Thank you in advance😊 -
Select radio item in edit view Django Template
I am making an edit view and I would like to have some radio fields selected when fetching the results. My issue is that I can't make my values of the forloop and the values from database comparing correctly. Let me be more clear with some code. I have these lists named mission_entry and gradings. mission_entry has some grades inside it, a vote from 1 to 5 in each value of the list. views.py mission_entry = MissionEntry.objects.filter(log_entry_id=log_entry_id) gradings = range(1,6) models.py GRADING_VALUE = ( ('1', '1'), ('2', '2'), ('3', '3'), ('4', '4'), ('5', '5'), ) class MissionEntry(models.Model): student = models.ForeignKey( Student, on_delete=models.DO_NOTHING, blank=True, null=True) mission = models.ForeignKey( Mission, on_delete=models.DO_NOTHING, null=True, blank=True) log_entry = models.ForeignKey( LogEntry, on_delete=models.CASCADE, blank=True, null=True) learning_objective = models.ForeignKey( LearningObjective, on_delete=models.DO_NOTHING, blank=True, null=True) grade = models.CharField( max_length=10, choices=GRADING_VALUE, blank=True, null=True) comment = models.TextField(null=True, blank=True) created_at = models.DateTimeField(auto_now_add=True) updated_at = models.DateTimeField(auto_now=True) template {% for lo in mission_entry %} <tr> <td id='learning_obj'>{{lo.learning_objective.name}}</td> {% for grade in gradings %} <td> <input class="form-check {{grade}}" {% if lo.grade == grade %}selected{% endif %} type="radio" value="{{grade}}" name="grade{{lo.learning_objective.name}}" id="grade" required/> </td> {% endfor %} <td><input type="text" class="form-control" name="note" id='note' value="{{lo.comment}}"></td> </tr> {% endfor %} I know that {% if lo.grade == grade %}selected{% endif %} … -
WebDriverException at / Message: Service /app/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 127
I am trying to deploy a django website to a heroku app, however , my browser is giving me an error. The error is : WebDriverException at / Message: Service /app/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 127 Can anyone help me? -
jinja django dictionary, how do I insert this dictionary key from an model id
this is probably more simple than I think it is but I am looking to insert a value from a django html template: {% for cert in list_of_certs %} <td>{{ due.cert.id }}</td> {% endfor %} due is a dictionary: due = {} for cert in list_of_certs: due.update({cert.id: str(cert.exp - datetime.date.today())}) but I cannot figure out how to due this in Jinja (in python it would be due[cert.id]) any help you can provide would be great thanks! -littlejiver -
wrong path doesn't give error anymore in django
Before if I go to a wrong path django gives an error that there is no path by that name but now I can go to any url path even if I didn't specify that path. and I noticed that before I was able to add new templates and everything was fine, now whenever I add a new template it shows an empty page with the navbar and the footer only without the new template. but I'm still able to edit the old templates. urls.py from django.urls import path from . import views urlpatterns = [ path("", views.home, name=""), path('home/',views.home,name='home'), path('privacy/',views.privacy,name='privacy'), path('return/',views.returnP,name='return'), path('search',views.search_book,name='search'), path('contact_us/',views.contact_us,name='contact_us'), path('suggestions/',views.suggestions,name='suggestions'), path('books-list', views.bookslistAjax), path('<str:cats>/',views.CategoryView, name='category'), ] -
Comparing two fields of different models without using foreign key in django
This is regarding CRM software I have two models profile_candidate and Agent, profile_candidate contains some 20 fields and Reference_Mobile is one of them. Agent is the model which contains the names of all the users who work as agents with the field name user. here what I need to do is... I need to compare the field Reference_Mobile of each profile_candidate with all the users available in Agent model, and I need to retrieve the objects of profile_candidate(model) where the Reference_Mobile is not matching with any of the users in the Agent Model. Note: Without using foreign key for Reference_Mobile field from the Agent model, because I cannot display all the users to the candidates while filling the form of profile_candidate. This is the problem I am facing from last two days, I am beginner of Django and python too. I also many other different ways but I could not found the proper output. I hope I explained clearly the problem. Please anyone help me out in achieving this.. my views.py def lead_unassigned(request): team_values = profile_candidate.objects.exclude(Reference_Mobile = Agent.user) return render(request, 'leads/lead_list_unassigned.html', {'team_values':team_values}) Thanking in you Advance, Dinesh Babu -
Process Data within WTForms Submission
I would like to reformat the data submitted via WTForms by utilizing the process_data function. I am able to successfully submit the form, however, the data is not modified in the output as expected. I've included a simplified example of my forms.py file below. The goal of this form would be to input the value of "10,000" and return the string "10000". from flask_wtf import FlaskForm from wtforms import StringField class SimpleForm(FlaskForm): price = StringField('price') def process_price(self, price): price.data = str(price).replace(',','') -
How to create a startup service of django for ubuntu server?
I am creating a project where I am using an ubuntu server for production. I need to run the server in the background even though I log out. So, as a solution I am using screen but however if I restart it won't work as we all know and I have to rewrite the screen commands. So, I want to use the startup service but somehow I am lost in that as I have no idea how to do that.here is the one way to write the script and I love it but how to add the command python manage.py runserver [IP Address] in this service. Sorry if I sound so silly but I need a quick and useful solution, so I thought to be here. Thanks if anyone can guide me on it. -
Django authentication for custom user, user object always return none if the credential is correct
this is my custom user code class CustomUser(AbstractUser): phone_number = models.CharField(max_length=12 , unique=True) Full_name = models.CharField(max_length=55) email = models.EmailField(max_length=60,null=True , blank=True) username = None first_name = None last_name= None USERNAME_FIELD = 'phone_number' REQUIRED_FIELDS = [] def __str__(self): return self.Full_name object = CustomUserManager() this code is to authenticate user in views.py user = authenticate(request,phone_number=username,password=password) -
how to separate sentences and assign each sentence to a list element on template
how do I separate the list of strings by "." and render them into separate "li" elements on the template. Currently, it just displays everything in one "li" element. I would really appreciate it if someone can help, thx! models.py class Product(models.Model): benifits = ArrayField(models.CharField(max_length=800), blank=True) @property def benifits_slicing(self): benifit = self.benifits for point in benifit: test = point.split(".") return point views.py def described_view(request, slug): products = Product.objects.get(slug=slug) context = {"products": products} HTML <div id="BENIFITS"> <li class="benifits-style">{{ products.benifits_slicing }}</li> </div> what it looks like rn: -
How to decrement a list in jinja 3
I am working on creating a hexagon grid with Django. Here is part of the code in the .html template that I am using for this specific loop: {% for item in vertical_list_hex %} {{ form.board }} {% endfor %} Where vertical_list_hex is a list such as [0, 1, 2, 3, 4] and form.board are the individual cells. At the moment it ends up like this: O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O My goal is to decrement the list with -1 in each loop so that it would end up in a hexagon shape: O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O I found some suggested solutions but they are outdated (jinja2). -
Celery jobs not running on heroku (python/django app)
I have a Django app setup with some scheduled tasks. The app is deployed on Heroku with Redis. The task runs if invoked synchronously in the console, or locally when I also have redis and celery running. However, the scheduled jobs are not running on Heroku. My task: @shared_task(name="send_emails") def send_emails(): ..... celery.py: from __future__ import absolute_import, unicode_literals import os from celery import Celery from celery.schedules import crontab # set the default Django settings module for the 'celery' program. # this is also used in manage.py os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'my_app.settings') # Get the base REDIS URL, default to redis' default BASE_REDIS_URL = os.environ.get('REDIS_URL', 'redis://localhost:6379') app = Celery('my_app') # Using a string here means the worker don't have to serialize # the configuration object to child processes. # - namespace='CELERY' means all celery-related configuration keys # should have a `CELERY_` prefix. app.config_from_object('django.conf:settings', namespace='CELERY') # Load task modules from all registered Django app configs. app.autodiscover_tasks() app.conf.broker_url = BASE_REDIS_URL # this allows you to schedule items in the Django admin. app.conf.beat_scheduler = 'django_celery_beat.schedulers.DatabaseScheduler' # These are the scheduled jobs app.conf.beat_schedule = { 'send_emails_crontab': { 'task': 'send_emails', 'schedule': crontab(hour=9, minute=0), 'args': (), } } In Procfile: worker: celery -A my_app worker --beat -S django -l info … -
Django/Bootstrap: some basic styles are not working as expected (nav-tab, button)
Using Django with Bootstrap(v5) it seems the "nav-tabs" class is not being styled as tabs, and gray buttons are not being styled as pills. Why is this? I tried using Bootstrap v4 and v5 with no change. <!DOCTYPE html> {% load static %} <html> <head> <meta charset="utf-8"> <link rel="stylesheet" href="{% static 'css/bootstrap.min.css' %}"> <script src="{% static 'js/bootstrap.min.js' %}"></script> </head> <body> <div class="text-center bg-primary text-white"> <h3>My Site!</h3> </div> <div class="container-fluid text-center"> <ul class="nav nav-tabs"> <li class="active"><a href="#">Home</a></li> <li><a href="#">Menu 1</a></li> <li><a href="#">Menu 2</a></li> <li><a href="#">Menu 3</a></li> </ul> </div> <div class="container"> <div class="row"> <div class="col-sm-4"> <h3>Filter</h3> <p>Lorem ipsum dolor sit amet, consectetur adipisicing elit...</p> <button type="btn btn-outline-primary">Reset</button> <button type="btn btn-outline-primary">Filter</button> <div class="alert alert-success alert-dismissible"> <button type="button" class="close" data-dismiss="alert">&times;</button> <strong>Success!</strong> </div> </div> <div class="col-sm-4"> <h3>Select View</h3> <p>Lorem ipsum dolor sit amet, consectetur adipisicing elit...</p> </div> <div class="col-sm-4"> <h3>Chart</h3> <p>Lorem ipsum dolor sit amet, consectetur adipisicing elit...</p> </div> </div> </div> </body> </html> -
How can I memory profile a django command?
How can I memory profile a django command? I tried using the profile decorator from memory_profiler And then I tried to invoke it with: python -m memory_profiler manage.py create_new_bhpt_records 1976020 "2022-04-14 00:00:01+00:00" -batch-size 10000 But the output logs did not include the memory profiling. Note: I added the profile decorator on my handle Command method. -
waitress-serve using local host and then using it in the procfile, But how to change the locasthost variable/dynamically before pushing to heroku?
Need help! I am stuck at the final stage of my first django app deployment on Heroku. Earlier when i used the command "waitress-serve --port=8000 project folder name.wsgi: application" This dint work stating server can't be reached. So i used "waitress-serve --listen=localhost:8000 project folder name.wsgi: application" And it worked. I then added this code inside a procfile _ " web: waitress-serve --listen=localhost:8000 project folder name.wsgi: application " Now when it's time to push to heroku ...as per the process port should be set dynamically by giving a $ sign but in my case, i have used localhost so how to set it in a Dynamic way before pushing to heroku -
How to 301 redirect django url with multiple slashes on url path?
I am trying to figure out how to redirect properly some urls that contain multiple slashes and create a permanent 301 redirect whenever the slug changes. For example: A - https://example.org/category/subcategory/article-name-A B - https://example.org/category/subcategory/article-name-B I want to redirect the A to B. I tried the default redirects app from django but did not work with multiple slashes. What is the correct approach here?! -
In django how can I run a query that (fielda, and fieldb) are in an allowed set of pairs (1, 2), (3, 4)?
In django how can I run a query that (fielda, and fieldb) are in an allowed set of pairs (1, 2), (3, 4)? It doesn't look like this is possible using filter or Q queries. Is it? Or do I need to do this with a raw sql query? -
Celery error received unregistered task of type
I am trying to add celery to my Django-Docker project environment. I have an error: received unregistered task of type 'create_task'. I was trying ideas from Celery Received unregistered task of type (run example) but didn't sucseed (was trying to specify task name, e.g. @app.task(name= ) How can I fix this error? MY project structure . ├── docker-compose.yml ├── project │ ├── core_app └── celery.py ... │ ├── nginx-conf.d │ │ └── nginx-conf.conf │ ├── parser_app ... celery.py from __future__ import absolute_import import os from celery import Celery from core_app.settings import INSTALLED_APPS os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core_app.settings') app = Celery("core_app") app.config_from_object('django.conf:settings', namespace='CELERY') app.autodiscover_tasks(lambda: INSTALLED_APPS) #@app.task(name='core_app.celery.create_task') # doesn't help @app.task(name='create_task') def create_task(task_type): period = int(task_type) * 10 print("SLEEP ", period) time.sleep(period) return True docker-compose.yml services: django: build: ./project # path to Dockerfile command: sh -c " python manage.py makemigrations && python manage.py migrate && gunicorn --bind 0.0.0.0:8000 core_app.wsgi" volumes: - ./project:/project - ./project/static:/project/static expose: - 8000 environment: - DATABASE_URL=postgres://postgres:post222@db:5432/lk_potok_4" - DEBUG=1 db: ... nginx: ... celery: build: ./project command: celery -A core_app worker --loglevel=info volumes: - ./project:/usr/src/app environment: - DEBUG=1 - DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1] - CELERY_BROKER=redis://redis:6379/0 - CELERY_BACKEND=redis://redis:6379/0 depends_on: - django - redis redis: image: redis:5-alpine
