Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
How to deal with time consuming python functions in Heroku?
I have successfully deployed a Django app to Heroku using Postgres. The only problem is that some of the python functions I have written can take up to several minutes to run (data scraping many pages with selenium and generating 50 different deep learning models with keras). If it takes longer than 30 seconds the app crashes. I ultimately plan on using this Heroku app as an API that I will connect to a frontend using React on netlify. Is there a way to automatically run these functions behind the scenes somehow? If not, how can I deploy a website that runs time consuming python functions in the backend and uses React for the frontend? -
Django 100% cpu usage, can't imagine it should be like that
[htop command on webserver][1] I am experiencing 100% cpu load on our webserver that is running a django application. It's a virtualized ubuntu 20. I just cannot imagine that it should be like that, do you know how to check what the issue might be? [VM has 5 Cores of an I7-7700K][2] [1]: https://i.stack.imgur.com/U53wE.jpg [2]: https://i.stack.imgur.com/0tcVg.png -
ChannelsLiveServerTestCase throwing URL not found
I'm wrtiting tests for my app which is a websocket app; well actually I wrote those tests and they worked but after I wrote a custom adapter for allauth, for a specific url it started throwing url not found error. I've defined my urls like this: project urls.py: urlpattens = [ ... path("messenger/", include("messenger.urls", namespace="messenger")), ... ] And then in messenger/urls.py: app_name = "messenger" urlpatterns = [ ... path("", views.chats_list_view, name="home"), ... ] Now, this is my tests.py: class TestConsumers(ChannelsLiveServerTestCase): serve_static = True # === SetUp === @classmethod def _set_chrome_options(cls): chrome_options = webdriver.chrome.options.Options() chrome_options.add_argument("--headless") chrome_options.add_argument("--no-sandbox") chrome_options.add_argument("--disable-dev-shm-usage") chrome_prefs = dict() chrome_options.experimental_options["prefs"] = chrome_prefs chrome_prefs["profile.default_content_settings"] = {"images": 2} return chrome_options @classmethod def _create_driver(cls, stores_logs=False): d = DesiredCapabilities.CHROME if stores_logs: d["goog:loggingPrefs"] = {'browser': 'ALL'} return webdriver.Chrome( options=cls._set_chrome_options(), executable_path="/usr/src/app/chromedriver/chromedriver", desired_capabilities=d, ) @classmethod def setUpClass(cls): super().setUpClass() try: cls.driver = cls._create_driver() except: super().tearDownClass() raise @classmethod def tearDownClass(cls): cls.driver.close() super().tearDownClass() # === Utility === def _create_user(self, driver, user_type, username="test", password="123456789", first_name="test", last_name="test"): user = get_user_model().objects.create_user( username=username, password=password, user_type=user_type, first_name=first_name, last_name=last_name, ) force_login(user, driver, self.live_server_url) return user def _create_principal_user(self, username="test", password="123456789", school_name="test school"): user = self._create_user(self.driver, username=username, password=password, user_type="SS") School.objects.create(name=school_name, support=user) return user def _create_a_group(self, driver, name="Test Group"): """ Create a chat group in the messenger app. return: … -
Heroku reject push project
i try to publish my python project but i have some issues can you help me ? i have requirements.txt also i add my git profile this is my first time with heroku i don't understand what is wrong i add my project on git repository and main branch i use python 3 this is my python project my project files and git files are in same level i tried something is i found on internet but it not worked for me and this is heroku error : -----> Building on the Heroku-20 stack -----> Using buildpack: heroku/python -----> Python app detected -----> No Python version was specified. Using the buildpack default: python-3.10.4 To use a different version, see: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-3.10.4 -----> Installing pip 22.0.4, setuptools 60.10.0 and wheel 0.37.1 -----> Installing SQLite3 -----> Installing requirements with pip Collecting APScheduler==3.9.1 Downloading APScheduler-3.9.1-py2.py3-none-any.whl (59 kB) Collecting asgiref==3.5.0 Downloading asgiref-3.5.0-py3-none-any.whl (22 kB) Collecting beautifulsoup4==4.10.0 Downloading beautifulsoup4-4.10.0-py3-none-any.whl (97 kB) Collecting bs4==0.0.1 Downloading bs4-0.0.1.tar.gz (1.1 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting certifi==2021.10.8 Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB) Collecting cffi==1.15.0 Downloading cffi-1.15.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB) Collecting charset-normalizer==2.0.12 Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) Collecting cryptography==36.0.1 Downloading cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB) Collecting … -
Get choices values from variable text - Django
I have a model with a positivesmallinteger field that has these choices (from model_utils). PERSON_TYPE = Choices( (0, 'women', _('Women')), (1, 'men', _('Men')), (2, 'boys', _('Boys')), (3, 'girls', _('Girls')) ) Elsewhere in my code, I'm trying to read in a person type from a file, but when I go to save to the model, I only have the string version of the person type, not the numerical value I need to store in the db. I know how to use PERSON_TYPE.women to get the numerical value of a person type, but the issue is in my code the person type would be a string variable. It looks like this fileName = companyData.file.name df = pd.read_csv(fileName, delimiter=',', encoding='utf-8-sig') for index, row in df.iterrows(): custPersonType = row[int(form.cleaned_data['columnNames'][0])-1] #gives the person type string I basically want to do PERSON_TYPE.custPersonType, but that doesn't seem to be possible. Is there a way to do this without using a dictionary or a long if/else? -
TypeError: Object of type 'HttpResponseRedirect' is not JSON serializable
I'm trying to run a simple app which receives a payload from an external application and enters EntryLayerView. This view calls a method in utils.py which then redirects the payload to another view for processing. However, I keep seeing this not Json serializable error. Url.py path( "api/v2/myapp/assessments", views.EntryLayerView.as_view(), name="myapp-assessments", ), path( "api/v2/myapp/startconversation", views.startconversation.as_view(), name="myapp-start-assessment", ), Views.py The entry point to the app is the EntryLayerView class EntryLayerView(generics.GenericAPIView): permission_classes = (permissions.IsAuthenticated,) def post(self, request, *args, **kwargs): body = request.data response = get_endpoint(validated_body) #Don't worry about this line return Response(response, status=status.HTTP_200_OK) class startconversation(generics.GenericAPIView): permission_classes = (permissions.AllowAny,) def post(self, request, *args, **kwargs): print("I'm inside the view") redirect = request.GET.get('all the data') #This is the view I'm trying to pass data to utils.py def get_endpoint(payload): qs = '?'+ urlencode(payload) reverse_url = reverse('myapp-start-assessment') url = reverse_url + qs print(url) return HttpResponseRedirect(url) The output of url in utils.py is: /api/v2/myapp/startconversation?contact_uuid=67460e74-02e3-11e8-b443-00163e990bdb&choices=None&value=&cardType=&step=None&optionId=None&path=&title=&description=&message= Error: return json.dumps(*args, **kwargs) File "/usr/local/lib/python3.6/json/__init__.py", line 238, in dumps **kw).encode(obj) File "/usr/local/lib/python3.6/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/local/lib/python3.6/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/home/osboxes/ndoh-hub/venv/lib/python3.6/site-packages/rest_framework/utils/encoders.py", line 67, in default return super().default(obj) File "/usr/local/lib/python3.6/json/encoder.py", line 180, in default o.__class__.__name__) TypeError: Object of type 'HttpResponseRedirect' is not JSON serializable I'm unable … -
400 error when trying to login via social networks
I use the allauth package. I added allauth.socialaccount.providers.yandex in apps and created a yandex account. There I specified Callback URL: http://127.0.0.1/main/accounts/yandex/login/callback/. I go to http://127.0.0.1/main/accounts/yandex/login/, press Continue and get: 400 redirect_uri does not match the Callback URL defined for the client And no matter how I try to change something, I keep getting this error. What am I doing wrong? -
Assign clients to seller
In my application there are two types of users, sellers and clients. Each client must be assigned to exactly one seller but a seller can have multiple clients. I have created one usermodel, so all my users are saved in the same place. In the model there is a boolean field called "is_seller" and another boolean field called "is_client". These get set to true depending on which registration form the user use. The client must be assigned when they register but I am not able to get my head around how I should implement this logic. I would appreciate all sorts of guidance! -
Django SECURE_HSTS_SECONDS preventing access
I have a Django application deployed on a Linux virtual machine using Nginx as a reverse proxy. The website used to work very well and was accessible by users via HTTPS. However, I have set SECURE_HSTS_SECONDS = 3600 in the settings.py. This blocked access to the website, resulting in a timeout and a status code of 301. I then commented out this line of code. I expected the application to be accessible by users after an hour or so, since the variable SECURE_HSTS_SECONDS was set to 3600 seconds, but it still remains unaccessible. My relevant settings are: if not DEBUG: # Production settings SESSION_COOKIE_SECURE = True CSRF_COOKIE_SECURE = True SECURE_SSL_REDIRECT = True # SECURE_HSTS_SECONDS = 3600 # NOTICE that this line was present before but is not commented out SECURE_HSTS_PRELOAD = True SECURE_HSTS_INCLUDE_SUBDOMAINS = True Any idea as to why this might be the case? -
How would one scrape <figure> tags in bs4?
I am trying to scrape images off of https://nytimes.com, however , most of the main headlines' corresponding images on their website is stored inside a <figure> tag, not an <img> tag with a specific src attribute. How would i be able to scrape the urls for the images inside those <figure> tags so i'd then be able to aggregate them on my own website? -
django file opens locally but fails in server
What I wanted to achieve was for a user to upload a file through a form, lets say profile picture, My aim is to take that file, send it through imgbb's api and get the image url to store for use in my template later. I was able to achieve this with the piece of code below, that works locally(Windows) but as soon as I push to my production server(pythonanywhere in this case), the code fails. # CODE: def testingpage(request): image = '' url = '' if request.method == 'POST': pic = request.FILES['image'] url = upload_to_bb(request, pic) if url is not None: from django.contrib import messages messages.info(request, 'Uploaded Successfully') return render(request, 'test.html', {'image': url}) def upload_to_bb(request, pic): import os from django.conf import settings from django.core.files.storage import FileSystemStorage import base64 import requests base_url = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) fs = FileSystemStorage() filename = fs.save(pic.name, pic) file_path = f'{base_url}\media\{filename}' apikey = settings.IMGBB_API_KEY with open(file_path, "rb") as file: url = "https://api.imgbb.com/1/upload" payload = { "key": apikey, "image": base64.b64encode(file.read()), } r = requests.post(url, payload) js = r.json() if r.status_code == 200: data = js['data'] file.close() os.remove(file_path) return data['image']['url'] else: file.close() os.remove(file_path) from django.core.exceptions import ValidationError raise ValidationError("An error occurred processing the image") return None the culprit is … -
Reverse query via foreign key in Django
What query can I use to return all the Organisation objects that are members of a certain membership_id value? e.g. return all Organisation objects that have a membership_id = 101 class Organisations(model) id = models.CharField() class Memberships(model) membership_id = models.IntegerField() organisation_id = models.ForeignKey(Organisations) -
While testing a Django DRF application, how can I make the user in my tests superuser?
I currently have a Django Rest Framework application that executes different logic for superusers than regular users. I've written my tests using both rest_framework.test and pytest-django. My Tests are currently failing because of the error: TypeError: Cannot cast AnonymousUser to int. Are you trying to use it in place of User?. So because I don't have a user set (my app does not allow unauthenticated users), my code is trying to look for a profile of the user making the call and failing. Right now, I'm not looking to test the functionality of User logic. Is there a way to set the default user in test_settings.py to a superuser, or setup a superuser in the def setup(self, request) function at the start of each of my test classes? The DRF docs relating to authentication in testing seem to be geared towards testing the authentication itself as opposed to assuming authentication around other tests. Based on those docs, I think I could hack something together, but there must be a standard way of doing what I'm trying. From what I've read, I can use force_authenticate on a particular user, but it's unclear to me where that user comes from. Do I … -
How to create a table of events which are summarized daily
I have a model for user hours: class Hours(models.Model): project = models.ForeignKey(Project, related_name='project',on_delete=models.DO_NOTHING) owner = models.ForeignKey(User, related_name='owner', on_delete=models.DO_NOTHING, default=1) quantity = models.DecimalField(max_digits=10, decimal_places=2, default=1) product = models.ForeignKey(Product, related_name='product', on_delete=models.DO_NOTHING) inserted_at = models.DateTimeField(default=datetime.now, blank=True) date = models.DateTimeField(default=datetime.now, blank=True) and I want to display the hours in a django template as a table so that the hours are grouped and summed up daily. I understand that I cannot do the math in template, so I need to do it in view, but can you please give me some hints for that? Indrek -
How to filter class fields using objects.filter and the id=pk in django
Good evening I would like to ask for some assistance am trying to filter the title of a post so I can write the title into a file to track user activity. This is my code for writing in the file: ''' def cover(request,pk): text = coverposts.objects.get(id=pk) thistime = datetime.datetime.now() Name = coverposts.objects.get(title__id=pk) articlefile = open('useractivity.txt','a') articlefile.write('\n'+ str(Name) + ', '+ str(thistime)) articlefile.close() return render(request,'cover.html',{'text':text}) ''' And this is the class I wish to get information from : ''' class coverposts(models.Model): title = models.CharField(max_length=100) visual = models.ImageField(upload_to='images') subtitle = models.CharField(max_length=100) body = models.CharField(max_length=100000000) created_at = models.DateTimeField(default =datetime.now(), blank= True) ''' Kindly advise where am going wrong -
Celery Child Tasks Don't Run in Correct Queues on ECS
I have a django project that uses celery for task management and rabbitmq for the broker. The tasks are divided into multiple queues and in production these queues run in separate ECS containers. All the period tasks defined in celery.py scheduled via celery-beat run in the correct queues however when defining the queue inside a task using apply_async they always spawn in the default. 'default_task': { 'task': 'webapp.tasks.default_task', 'schedule': crontab(minute="*/1"), }, 'parent_task': { 'task': 'webapp.tasks.parent_task', 'schedule': crontab(minute="*/1"), 'options': {'queue' : 'secondary'} }, @shared_task(ignore_result=True) def parent_task(): qs = SomeModel.Objects.filter(status='in progress') task_expiration = 3600 for sm in qs: child_task.apply_async((sm.id,), expires=task_expiration, queue='secondary') I can confirm that the tasks are running just not on the specified queue which creates issues with timing due to potential noisy neighbors on the default queue container. Versions: django: 3.0.1 celery: 5.0.0 rabbitmq: 3.8.11 python: 3.8 -
Django dates calculator
this application should take a date from the user, store it in the database then count 9 days after it and show those 10 days in a list, I managed to make two views one to count the 10 days and show them in a list. The other view is to take the day from the user and store it in the database, now I want to connect both so by the end of the process I want the user to select the date then the application automatically stores the date and counts them and show the list. Models: from django.db import models from django.db.models.fields import DateField from datetime import datetime from django.db.models import CharField, Model # Create your models here class Event(models.Model): title = models.CharField(max_length=40) start = models.DateTimeField(auto_now=False, auto_now_add=False, null=True, blank=True) end = models.DateTimeField(auto_now=False, auto_now_add=False, null=True, blank=True) def __unicode__(self): return self.title class PeriodDate(models.Model): per_date = models.CharField(max_length=15) serializer: from dataclasses import fields from rest_framework import serializers from .models import PeriodDate class PeriodDateSerializer(serializers.ModelSerializer): class Meta: model = PeriodDate fields = '__all__' URLs: from django.urls import path from django.urls.resolvers import URLPattern from . import views urlpatterns = [ path('CalendarCount', views.CalendarCount, name= 'index' ), path('form', views.form), ] Views: @api_view(['GET']) def CalendarCount(request): #Taking date … -
Why does Celery task autodiscovery never trigger under Django?
I've got Celery 5.2.6 and Django 3.0.6. Everything is working except task autodiscovery under Django. If I run celery worker, the worker triggers autodiscovery, finds all my tasks, and displays them in a list as part of it's startup process. However, if I run my Django app or Django shell, this never happens. Additionally, even though the docs promise that accessing the task registry will trigger autodiscovery, it does not. I print app.tasks, I call app.tasks.keys(), and still no autodiscovery - it only shows the tasks that are built-in or were registered when the module containing them happened to be imported for other reasons. What do I need to do to trigger task autodiscovery? PS - If I try adding force=True to app.autodiscover_tasks(), it fails because the Django app registry hasn't finished loading at that time. -
Django image does not duplay
I'm new to Django, and I encounter this problem with images that I can't solve... The path is like this: Django-Project, Profiles, static, media, profileIMG. Here is my model. from django.db import models from accounts.models import NewUser class UserProfile(models.Model): user = models.OneToOneField(NewUser, on_delete=models.CASCADE) profile_pic = models.ImageField(default='Untitled.png', upload_to='profileIMG') def __str__(self): return self.user.username settings.py STATICFILES_DIRS = [ os.path.join(BASE_DIR, "static"), ] MEDIA_ROOT = os.path.join(BASE_DIR, 'media') STATIC_URL = '/static/' MEDIA_URL = '/media/' urls.py from django.contrib import admin from django.urls import path, include from django.conf import settings from django.conf.urls.static import static urlpatterns = [ path('admin/', admin.site.urls), path('', include('accounts.urls')), path('', include('profiles.urls')), ] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) form.py from django.forms import ModelForm from .models import UserProfile class ProfileForm(ModelForm): class Meta: model = UserProfile fields = '__all__' exclude = ['user'] view.py function @login_required(login_url='login_user') def profiles(request): indexID = request.user form = ProfileForm(instance=indexID) if request.method == 'POST': form = ProfileForm(request.POST, request.FILES, instance=indexID) if form.is_valid(): messages.success(request, ("The file is valid")) form.save() else: messages.success(request, ("Invalid File.")) context = {'form': form} return render(request, "profiles/profiles.html", context) And my template profiles.html. {% load static %} <div class="main"> <form method="POST" enctype="multipart/form-data"> {% csrf_token %} <img class="profile-pic" src="{{ request.user.UserProfile.profile_pic.url }}"/> <p>This is profile page </p> <span>Hello, {{request.user}} </span> <p>{{ form }}</p> <input class="imgBTN" type="submit" name="imgBTN"> <span><a href="{% url … -
warnings.warn("DateTimeField %s received a naive datetime (%s)"
I use django simple-history to get history on my models I then search the history results by date but I get the error below. How can I format the date? RuntimeWarning: DateTimeField HistoricalIssue.history_date received a naive datetime (2022-04-13 10:34:32) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" def SearchByDate(request): date_search = request.POST['date-search'] if date_search: admin_hist_search_results = Issue.history.filter(history_date=date_search) -
can you help to find a mistake?
Terminal says that i didn't defined DJANGO_SETTINGS_MODULE but i tried as i thought every method to do it and so far nothing helper (coding on windows) Can someone help me out please? I am stuck and dont know what to do Maby this will help - i run my virtual env through Anaconda - conda activate djangoenv raise ImproperlyConfigured(django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings. from faker import Faker from app.models import AccessRecrod, Webpage, Topic import random import django import os os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings') django.setup() fakegen = Faker() topics = ['Search', 'Social', 'Marketplace', 'News', 'Games'] def add_topic(): t = Topic.objects.get_or_create(top_name=random.choice(topics))[0] t.save() return t def populate(N=5): for entry in range(N): # get topic for entry top = add_topic() # create fake data for entry fake_url = fakegen.url() fake_date = fakegen.date() fake_name = fakegen.company() # create new webpage entry webpg = Webpage.objects.get_or_create( topic=top, url=fake_url, name=fake_name)[0] # create fake access record acc_rec = AccessRecrod.objects.get_or_create( name=webpg, date=fake_date)[0] if __name__ == '__main__': print('populating script!') populate(20) print('populating complete!') -
Error 500 on Websocket, deploying Django-app on Heroku
I'm having some problems with websockets that I don't have locally. The problem is that heroku is telling me that there is a 500 error. My settings on prod: ASGI_APPLICATION = 'config.asgi.application' CHANNEL_LAYERS = { "default": { "BACKEND": "channels_redis.core.RedisChannelLayer", "CONFIG": { "hosts": [("redis_url")] }, }, } My ASGI.py import django from channels.auth import AuthMiddlewareStack from channels.routing import ProtocolTypeRouter, URLRouter from django.core.asgi import get_asgi_application from my_app.chat import ( routing, ) os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_app.settings") django.setup() application = ProtocolTypeRouter({ "http": get_asgi_application(), "websocket": AuthMiddlewareStack( URLRouter( routing.websocket_urlpatterns ) ), }) I'm ready using wss instead of ws, and that not the problem. Thanks! -
How to use Django Filter with Arrayfield
Is it possible to use django-filter BaseInFilter with with django ArrayField? Here is a sample of what I am trying to do; models.py class Stays(models.Model): best_describes_house = ArrayField( models.CharField(max_length=500, blank=True, null=True), default=list, ) filterset.py class ItemInFilter(filters.BaseInFilter, filters.CharFilter): pass class StayFilter(filters.FilterSet): best_describes_house__in = ItemInFilter( field_name="best_describes_house", lookup_expr="in" ) class Meta: model = Stays fields = [ "best_describes_house__in", ] I am trying to make a request to http://127.0.0.1:8000/api/v1/stays/best_describes_house__in=villa,lake. I keep getting; operator does not exist: character varying[] = text[] LINE 1: ...ays" WHERE "lodging_stays"."best_describes_house" IN (ARRAY[... Am I doing something wrong? I have also tried doing this using IntegerField as stated from this example in the documentation - BaseInFilter, and it works fine -
Why Web server like apache/nginx cant directly server web apps made in django or flask framework?
I would like to know what's the need of this middle man- application servers like uwsgi, gunicorn. why cant a web server like Apache and Nginx directly serve a web app built in Django or flask. if your answer is that the uwsgi converts the request into a python understandable language. please elaborate on what kind of request python needs which it will understand. -
how do i get the highiest occured user in a table in Django
please I need help with this, am building an app that allows users to send out invites, and I want to get the user with the highest sent out invitations, how do I go about it. my model is as below. class Invite(models.Model): host_name = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL) invite = models.CharField(max_length=200, null=True,blank=True) def __str__(self): return self.name```
