Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
Should I use Celery to covert audio files on my server
So I have higher quality user audio submissions and I need the server to create lighter .mp3 copies and im wondering if I should use Celery with my django app or use build in Linux tools to control this? I want to control how much processing converting takes to avoid a situation where multiple people submit their audio and the processing uses so much server resources that it compromises the user experience of the app. I have an ubuntu vm in cloud. I use Django with NGINX and gunicorn. -
Django testing: how to keep only one of several test databases?
Seems like this should be simple. If I have something like this in my Django settings.py: DATABASES = { 'default': { ... }, 'second': { ... }, 'third': { ... }, } By default, when unit tests run, all three databases are rebuilt, by applying all the migrations. However, I don't want to rebuild the "expensive" database "three," but using the --keepdb option is obviously not specific enough, as that keeps all the databases. How do I only keep database "three" when running unit tests? Thank you in advance. -
upload a file with django form, request.FILES is empty no matter what I do
when I submit the file and check in the POST of the request from the browser inspector I see all the other fields but not the file field. Accessing request.FILES['attachment'] causes an error django.utils.datastructures.MultiValueDictKeyError: 'attachment' My Model class Ticket(models.Model): attachment = models.FileField(blank=True) title = models.CharField(max_length = 250,default='') My Form class NewTicketForm(forms.ModelForm): attachment = forms.FileField(required = False) My Template <form enctype="multipart/form-data" id="ticketform" method="POST" action=""> {{ form.non_field_errors }} {{ form.title }} {{ form.attachment }} <input type="submit" name="submit_new_ticket" value="Submit"> </form> My View def createNewTicket(request): if request.method == 'POST': form = NewTicketForm(request.user, request.POST) if form.is_valid(): obj = form.save() return HttpResponse(request.FILES["attachment"]) I CANNOT GET IT MANAGED AUTOMATICALLY, I NEED DIRECT ACCESS TO request.FILES -
How to solve the xmlsec Error: (100, 'lxml & xmlsec libxml2 library version mismatch')
I am facing the following error when deploying my Django (version 4.1) Backend, I have the following Dockerfile (some non-relevant parts omitted) and need to install python3-saml (which has dependencies like lxml and xmlsec). The documentation mentions the following: Hence I added the command in the Dockerfile. FROM python:3.10.6 # Install necessary packages RUN apt-get update && \ apt-get install -y s3fs pkg-config libxml2-dev libxmlsec1-dev libxmlsec1-openssl libxmlsec1 && \ apt-get clean && \ rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* # set work directory WORKDIR /usr/src/app # install dependencies RUN pip install --upgrade pip COPY ./requirements.txt . RUN pip install -r requirements.txt RUN pip install python3-saml && \ pip install --force-reinstall --no-binary lxml lxml # copy project COPY . . # create cache directory RUN mkdir -p /var/tmp/django_cache ENTRYPOINT ["./docker-entrypoint.sh"] I am able to build it without problem but upon deployment, I get the following error: File "/usr/local/lib/python3.10/site-packages/onelogin/saml2/auth.py", line 12, in <module> import xmlsec xmlsec.Error: (100, 'lxml & xmlsec libxml2 library version mismatch') I am not sure which library version is expected in this case, I have tried with pip install --no-binary lxml==4.6.3 lxml==4.6.3 --force-reinstall instead, as well as with version 4.9.3 (as seen in other recent threads) but with no success. … -
Making a Django application enterprise ready
What are the must have features over and above the standard Django deployment checklist one should consider to make the application truly enterprise ready. A few things that I can think of are: Access Logs Audit Logs MFA/SSO for user authentication I know there can be tons of things on the infrastructure side but I am focussing on the application side stuff only. Researched on the internet for suggestions but not able to find any authoritative source. Would be great to learn from the experiences of SO members. -
not able to open web page using django
I'm using the code below code for opening a webpage (my_app1) using django in Google Cloudshell, after running server showing below error in serverlink). Using the URLconf defined in demoproject.urls, Django tried these URL patterns, in this order: admin/ ^my_app1/$ The empty path didn’t match any of these. urls from django.urls import include, re_path from django.contrib import admin from .views import my_app, my_ap urlpatterns = \[ re_path(r'admin/', admin.site.urls), re_path(r'^my_app1/$', my_ap), \] views from django.shortcuts import render from django.http import HttpResponse def my_app(request): return HttpResponse("this is response") def my_ap(request): return HttpResponse("hi buddy") How do I get the response hi buddy after running server? -
KeyError: 'type' in a websocket_receive
I have two different apps in my Django project. While the first one (the one with the error raising consumers.py) works fine on its own, the second one keeps raising this: \chat\consumers.py", line 29, in websocket_receive if event['type'] == 'message': KeyError: 'type' chat\consumers.py: from channels.consumer import SyncConsumer, AsyncConsumer from channels.db import database_sync_to_async from asgiref.sync import async_to_sync, sync_to_async from django.contrib.auth.models import User from chat.models import Message, Thread, UserSetting import json from rich.console import Console console = Console(style='bold green') online_users = [] class WebConsumer(AsyncConsumer): async def websocket_connect(self, event): self.me = self.scope['user'] self.room_name = str(self.me.id) online_users.append(self.me.id) await self.channel_layer.group_add(self.room_name, self.channel_name) await self.send({ 'type': 'websocket.accept', }) console.print(f'You are connected {self.room_name}') async def websocket_receive(self, event): event = json.loads(event['text']) #console.print(f'Received message: {event["type"]}') if event['type'] == 'message': msg = await self.send_msg(event) await self.send_users(msg, [self.me.id, self.them_user.id]) elif event['type'] == 'online': msg = await self.send_online(event) console.print(online_users) await self.send_users(msg, []) elif event['type'] == 'read': msg = await sync_to_async(Message.objects.get)(id=event['id']) msg.isread = True await sync_to_async(msg.save)() msg_thread = await sync_to_async(Thread.objects.get)(message=msg) await self.unread(msg_thread, int(event['user']), -1) elif event['type'] == 'istyping': console.print(self.me, event) await self.send_istyping(event) async def websocket_message(self, event): await self.send( { 'type': 'websocket.send', 'text': event.get('text'), } ) async def websocket_disconnect(self, event): console.print(f'[{self.channel_name}] - Disconnected') event = json.loads('''{ "type": "online", "set": "false" }''') online_users.remove(self.me.id) msg = await … -
How to Connect Django 5.0.6 to MSSQL 16?
Is there any way to use MSSQL Server with Django 5.0? I know it's not the most common database for Django, but it would be very suitable for my project for various reasons. Any guidance or tips on setting this up would be really appreciated! -
Telegram mini app preview and test on localhost
I am currently working on developing a mini app, which is essentially a website built on a Django project and served locally using the development server. However, I have been facing slow loading times when trying to view this mini app on a Telegram bot, particularly when using services like ngrok. While other website URLs load quickly, the ngrok setup for my Django project is taking approximately 10 minutes to load. I have also experimented with using an SSL server from** django-sslserver** by running the command: python manage.py runsslserver 0.0.0.0:8000 I hoped that loading the app over HTTPS might improve the speed, but unfortunately, I still encountered the broken pipe error. I have not yet utilized a dedicated server for static files. Do you think incorporating one could help improve the loading speed during local development? ngrok functions smoothly and loads quickly with simple HTTP requests for webhooks or basic HTML files. Therefore, I suspect that the sluggish loading speed may be due to the volume of static files being loaded. I am seeking advice on faster and more efficient methods or tricks to preview my mini app during development using localhost or any alternative approaches. Any suggestions or recommendations … -
Unable to connect django application to Microsoft SQL Database
I have a django application that attempts to connect to an external microsoft sql database from azure. After setting up the database connection parameters in the settings file and installing the odbc driver on my local machine, running the python manage.py makemigrations throws the following error: Here is the settings.py file DATABASES = { 'default': { 'ENGINE': 'sql_server.pyodbc', 'HOST': DB_HOST, 'PORT': DB_PORT, 'NAME': DB_NAME, 'USER': DB_USER, 'PASSWORD': DB_PASSWORD, 'OPTIONS': { 'driver': 'ODBC Driver 18 for SQL Server', 'encrypt': 'yes', 'trustServerCertificate': 'no', } }, -
Django, React, Axios returning html instead of json data
I am working on a web app with Django Rest Framework and React. When I make a request to Django via http://localhost:8000/api/v1/auth/users/daniel/ for a User's profile, it's returning html instead of the JSON response of the userProfile. Reponse <!doctype html> <html lang="en"> <head> <script type="module"> import RefreshRuntime from "/@react-refresh" RefreshRuntime.injectIntoGlobalHook(window) window.$RefreshReg$ = () => {} window.$RefreshSig$ = () => (type) => type window.__vite_plugin_react_preamble_installed__ = true </script> <script type="module" src="/@vite/client"></script> <meta charset="UTF-8" /> <link rel="icon" type="image/svg+xml" href="/vite.svg" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Vite + React</title> </head> <body> <div id="root"></div> <script type="module" src="/src/main.jsx?t=1717529669140"></script> </body> </html> Here is the DRF response of what I am expecting to see in the Reac Axios response. urls.py urlpatterns = [ ....., path('api/v1/auth/users/<str:username>/', views.UserProfileByUsernameView.as_view(), name='user-profile-by-username'), ] views.py class UserProfileByUsernameView(APIView): permission_classes = [AllowAny] def get(self, request, username): user = get_object_or_404(User, username=username) profile_serializer = UserProfileSerializer(user.userprofile) profile_data = profile_serializer.data profile_data['username'] = user.username profile_data['email'] = user.email return Response(profile_data, status=status.HTTP_200_OK) serializers.py class UserProfileSerializer(serializers.ModelSerializer): username = serializers.CharField(source='user.username', read_only=True) email = serializers.EmailField(source='user.email', read_only=True) class Meta: model = UserProfile fields = '__all__' profile-page.jsx const ProfilePage = () => { const { username } = useParams(); const [userProfile, setUserProfile] = useState(null); const [isOwnProfile, setIsOwnProfile] = useState(false); const loggedInUsername = localStorage.getItem('username'); useEffect(() => { const fetchUserProfile … -
Asynchronous migrations for django/wagtail
I am working with my friends on wagtail school project, we are working at the same time on our project and making different pages at the same time. We would need a asynchronous system for migrations of pages. I have searched and found command dumpdata and loaddata but this seems that it is synchronous only, and it is not possible for two people to have 2 open Merge requests since dumped json files would override each other. Is there any solution for this problem? -
(DJango) What is the solution to "ModuleNotFoundError: No module named'rest_frameworks'" if nothing works?
I followed a tutorial on youtube and was then faced with such a problem. I tried to find a solution, but nothing worked for me. Maybe something's wrong with my IDE or anything? I checked everything and was unable to find the exact problem. Compared to the script from the video, I think everything is pretty fine ;( -
Annotate Django JSONField to convert all values to strings
I'm using Django's full text search features, and want it to search through a model called Row, which has a JSONfield .data that has a value like: [["A1", 87, 987, "blue"], ["B1", null, null, "white"]] Code to do the search: search_vector = SearchVector('data') search_query = SearchQuery(query) search_headline_data = SearchHeadline('data', search_query, highlight_all=True) results = Row.objects.all() .annotate(search=search_vector) .annotate(headline_data=search_headline_data) When query="blue", the match is a string, and highlighted correctly: [["A1", 87, 987, "<b>blue</b>"], ["B1", null, null, "white"]] But when query="87" so there's a match with an integer, the match is not highlighted. [["A1", 87, 987, "blue"], ["B1", null, null, "white"]] This makes me want to cast all values in the .data JSONField to a string, but I can't find out how to do this through annotation. -
Making WEB-Scada system
Good afternoon, I'm writing a django web scada system at school, please tell me which toolkit is better to use for polling OPCUA hardware with data output without restarting the server I tried to create a separate service in python for the survey, followed by writing data to JSON format, but it seems terribly inconvenient to me, are there any alternatives -
How to change Celery worker concurrency in Django app
Recently I noticed that celery eats up a lot of memory in the server. One instance eating like 10% of mem in processed I see quite default situation: /usr/local/bin/python -m celery -A apps worker -l INFO -c 4 --logfile celery_worker.log With threads it goes up to 14 instances My idea was to cut concurrency to 2 (from process I suggest it is set to 4) I've tried to do it in django settings CELERY_WORKER_CONCURRENCY = 2 and event put in celery file app.conf.update( task_soft_time_limit=60 * 60 * 1, # 2 hours, task_time_limit=60 * 60 * 2, # 3 hours, worker_concurrency=2 # 2 workers ) But it did not work. I've checked it with celery -A apps inspect stats "max-concurrency": 4, "max-tasks-per-child": "N/A", but for example timeout was precessed correctly can you suggest how to solve it? -
I can't open the Django admin by *.web.app domain in Django+React project in the Google Cloud Run service
First I will introduce my project structure: Frontend: React+ViteJS Backend: Django-ninja for the api stuff Admin Platform: Django original admin framework Custom Domain: Google Firebase host (integrate with google cloud run), for example: the website is https://mysite.web.app Right now I use the Google Cloud Run multicontainer service to deploy the whole project. For the frontend docker Dockerfile: FROM node:20-slim as build WORKDIR /app COPY package*.json ./ RUN npm install COPY . . RUN npm run build # Use Nginx as the production server FROM nginx:alpine COPY nginx.conf /etc/nginx/conf.d/default.conf # Copy the built React app to Nginx's web server directory COPY --from=build /app/dist /usr/share/nginx/html # Expose port 80 for the Nginx server EXPOSE 8000 # Start Nginx when the container runs CMD ["nginx", "-g", "daemon off;"] This is the nginx.conf: server { listen 8000; # listen [::]:80; # server_name localhost; location / { root /usr/share/nginx/html; index index.html index.htm; try_files $uri $uri/ /index.html; } location /api/ { proxy_pass http://localhost:8080; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } location /admin/ { proxy_pass http://localhost:8080; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; … -
How to enrich page navigator by adding field from first and last objects in Django paginator?
I am building a Django app form managing the digitalisation process of analog film rolls. I would like to add the film_roll.from_year field of the FilmRoll model to my page navigation widget in a Django template. This greatly facilitates navigating to the right page by looking at the year_from range below each page. My view is defined as follows: def index(request): film_roll_list = FilmRoll.objects.order_by( "from_year", "from_month", "title" ).annotate( scan_count=Count("myScanned_VS_NLP", distinct=True), ) paginator = Paginator(film_roll_list, 20) page_number = request.GET.get("page") try: page_obj = paginator.get_page(page_number) except PageNotAnInteger: # If the page parameter is not an integer, show the first page page_obj = paginator.get_page(1) except EmptyPage: # If the page is out of range, show the last page page_obj = paginator.page(paginator.num_pages) film_roll_count = paginator.count context = { "film_roll_list": film_roll_list, "page_obj": page_obj, "film_roll_count": film_roll_count, } return render(request, "filmrolls/index.html", context) Here's the page navigator code in the template: {# Page navigator: start #} {% if page_obj.has_other_pages %} <div class="row"> <div class="btn-group" role="group" aria-label="Item pagination"> {% if page_obj.has_previous %} <a href="?page={{ page_obj.previous_page_number }}" class="btn btn-outline-primary">&laquo;</a> {% endif %} {% for page_number in page_obj.paginator.page_range %} {% if page_obj.number == page_number %} <button class="btn btn-outline-primary active"> <span>{{ page_number }} <span class="sr-only"></span></span> # TODO - add 'year_from' range for the … -
Django session in tests
everyone! I can't figure out how the sessions work in Django. I have a shop, an anonymous user's shopping cart is bound to session_key. The session key is taken from the request object. def _check_session(self, request) -> int | str: session_key = getattr(request.session, ‘session_key’, None) if session_key: return session_key else: request.session.create() return request.session.session_key I'm writing tests, one test adds items to the basket, I take the anonymous user client and make a post request to add, everything is ok. Then in the second test, I take the same anonymous client, make a get request to get the contents of the basket. But in this request the request object has no session_key. Why in the second test, the request object does not contain a session_key? The client is the same. These two tests are written within one TestCase. -
Django constraints: unique together and accept one and only one value
I'm trying to implement a constraint for the following model: class Consumption(models.Model): simcard = models.ForeignKey(SimCard, on_delete=models.CASCADE) start_date = models.DateTimeField() duration_billed = models.FloatField(null=True) volume_billed = models.FloatField(null=True) sms_billed = models.FloatField(null=True) I would like to constraint the user to save one and only one value for duration_billed, volume_billed, sms_billed for a given simcard and start_date. I have the following Meta class: class Meta: constraints = [ UniqueConstraint( fields=['simcard', 'start_date', 'duration_billed'], name='unique_simcard_and_start_date_and_duration_billed' ), UniqueConstraint( fields=['simcard', 'start_date', 'volume_billed'], name='unique_simcard_and_start_date_and_volume_billed' ), UniqueConstraint( fields=['simcard', 'start_date', 'sms_billed'], name='unique_simcard_and_start_date_and_sms_billed' ), ] The problem I face with my constraints in that the DB accept several duration_billed, volume_billed, sms_billed for a given simcard and start_date. The result I try to achieve is if a duration_billed exists for a given simcard and start_date, the user should not be able to import a new duration_billed (even if the value is different from the one stored in DB) for the same simcard and start_date. Same goal for volume_billed and sms_billed. I hope my issue is clear. Any help or advice would be much appreciated 🙏 Many thanks in advance -
“OperationalError: Could not translate host name “db” to address in Dockerized Django App with Datadog”
I have a very complex django project that uses postgresql as database, where I have setup datadog to send traces and events. It works fine locally and I am receiving traces and events in datadog. However, if I try to run my app using docker, it won't send traces and events to datadog. Following is my docker-compose.yml file where datadog integration is borrowed from here. docker-compose.yml services: db: image: ankane/pgvector env_file: - ./docker/env.db volumes: - pgvector_db_data:/var/lib/postgresql/data app: build: context: . dockerfile: ./docker/prod.Dockerfile env_file: - ./docker/env.db - ./.env container_name: scooprank volumes: - static_volume:/opt/code/static - shm:/dev/shm - save_drivers:/opt/code/save_drivers - save_models:/opt/code/save_models command: ./docker/scripts/run-gunicorn depends_on: - db network_mode: host ddagent: image: datadog/docker-dd-agent environment: - DD_BIND_HOST=0.0.0.0 - DD_API_KEY=${DATADOG_API_KEY} - DD_APM_RECEIVER_SOCKET=/tmp/ddagent/trace.sock - DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true ports: - "8127:8126" news_scraper: image: scooprank-app env_file: - ./.env container_name: news_scraper command: python manage.py news_scraper depends_on: - app restart: always volumes: static_volume: shm: save_drivers: save_models: pgvector_db_data: I added network_mode: host as suggested by this answer. I then tried this solution on a simple dummy project having sqlite3 as database and with this docker-compose file, I am receiving traces and events. But if I try to use it in my main project, I can do docker compose build and docker compose up, but when … -
Show dd/mm/yy in django
I have a input type Date field like below. # forms.py widgets = { 'date_of_birth': forms.DateInput(attrs={'type': 'date'}) } I am using below code in HTML Template. <label class="form-label">Date of Birth</label> {{ form.date_of_birth }} I am getting mm/dd/yy format but I would like to get dd/mm/yy format. -
Using Google Cloud SDK in Django Developement and UWSGI
I am trying to access Google Cloud Service in my backend process (views.py). I have successfully run this in developement server. Here is my views.py script : from google.cloud import bigquery @login_required def update_iphone(request): user_profile = UserProfile.objects.get(user=request.user) client = bigquery.Client() if request.method == 'POST': form = IphoneUpdateForm(request.POST) if form.is_valid(): sql = f""" query script """ client.query(sql) else: pass sql = f""" query script """ corporate_list = client.query(sql).to_dataframe() iphone_form = IphoneUpdateForm() return render(request, 'myapp/home.html', {'user_name':user_profile.first_name, 'corporate_list': corporate_list['corporate_name'].to_list(), 'form': iphone_form}) This views.py script runs well in development server. But if try to run it with UWSGI, I am getting error related to the google cloud as following : google.auth.exceptions.DefaultCredentialsError: Your default credentials were not found. To set up Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information. My question is why am I getting this error if using UWSGI and not the development server ? how do I deal with this error ? Thank You -
Many one-to-one relationships pointing to the same model Django
I'm designing an application in Django where I have an Item model with numerous attributes for each item. I'm experiencing very poor performance in my application and I'm wondering if there are flaws in my setup that could be causing this issue. I am relatively new to Django, so any pointers would be greatly appreciated. Currently, I have around 200 attribute models, each with optional one-to-one fields linked to an item via a foreign key as follows: Attribute models: class AttrBase(models.Model): prev_value = models.CharField(max_length=250, null=True, blank=True) edited_by = models.ForeignKey( Account, on_delete=models.CASCADE, null=True, blank=True ) edited_on = models.DateTimeField(null=True, blank=True) options = None class Meta: abstract = True class ID(AttrBase): item = models.OneToOneField(Item, on_delete=models.CASCADE) value = models.CharField(max_length=250, null=True, blank=True) name = "ID" tooltip = "..." group = "..." class Height(AttrBase): item = models.OneToOneField(Item, on_delete=models.CASCADE) value = models.IntegerField(null=True, blank=True) name = "Height" tooltip = "..." group = "..." ...more I am also using a generic relationship to link to all of the groups, as follows: class AttributeGroup(models.Model): content_type = models.ForeignKey( ContentType, on_delete=models.CASCADE, limit_choices_to={"app_label": "attributes"}, ) name = models.CharField(max_length=250, null=True, blank=True) description = models.CharField( max_length=1000, default="TODO", null=True, blank=True ) def __str__(self): return str(self.content_type) def save(self, *args, **kwargs): self.name = self.content_type.name super(AttributeGroup, self).save(*args, **kwargs) This … -
Django CSRF cookie not set with 403 error for webhook URL
I am encountering an issue with Django's CSRF protection while trying to handle Stripe webhooks on my local host. I am receiving a 403 Forbidden error with the message "CSRF cookie not set." The error occurs when trying to access the /collect-stripe-webhook/ URL, which is intended to handle incoming webhook requests from Stripe.Also the payments go through and payment is successfull urls.py urlpatterns = [ path('subscribe/', product_list, name='product_list'), path('create-checkout-session/<int:product_id>/', create_checkout_session, name='create_checkout_session'), path('collect-stripe-webhook/', stripe_webhook, name='stripe_webhook'), # Updated path path('success/', TemplateView.as_view(template_name="subscriptions/success.html"), name='success'), path('cancel/', TemplateView.as_view(template_name="subscriptions/cancel.html"), name='cancel'), ] views.py: @csrf_exempt def stripe_webhook(request): """ View to handle Stripe webhooks. """ payload = request.body sig_header = request.META.get('HTTP_STRIPE_SIGNATURE', None) endpoint_secret = settings.STRIPE_WEBHOOK_SECRET if not sig_header: logger.error("No signature header found in the request") return JsonResponse({'status': 'invalid request'}, status=400) try: event = stripe.Webhook.construct_event( payload, sig_header, endpoint_secret ) except ValueError as e: logger.error(f"Invalid payload: {e}") return JsonResponse({'status': 'invalid payload'}, status=400) except stripe.error.SignatureVerificationError as e: logger.error(f"Invalid signature: {e}") return JsonResponse({'status': 'invalid signature'}, status=400) if event['type'] == 'checkout.session.completed': session = event['data']['object'] handle_checkout_session(session) return HttpResponse(status=200) template: <form action="{% url 'subscriptions:create_checkout_session' product.id %}" method="POST"> {% csrf_token %} <button type="submit">Subscribe</button> </form> Error Message: Forbidden (CSRF cookie not set.): /collect-stripe-webhook/ Question: What could be causing this issue with Django's CSRF protection, and how can I resolve …
