Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
Why the session isn't delete after I log out?
Got stuck in a problem about session in Django3.2. After I added a login/register user to the project that I'm creating, the logout doesn't work. When I log out, the sessionid isn't delete, and if its not delete then the user can still have access after it is been log out. I used the request.session.flush() based on the answer I recived at my last question: Why the login_requried decorator doesn't work after I log out from the account? but its not working, the session is not deleted. For better understanding will show the code below: @login_required(login_url='login/') def logoutUser(request): logout(request) request.session.flush() return redirect('/') -
how to exclude admin user from my user list
how to exclude admin users from my user list the below image shows the problem I don't want to show the admin user in the friend list of a user. I don't want to show the main_admin user on this user list as it is my admin user of the site views.py from django.shortcuts import render, redirect, get_object_or_404 from .models import Profile from feed.models import Post from django.contrib import messages from django.contrib.auth.decorators import login_required from django.contrib.auth import get_user_model from django.conf import settings from django.http import HttpResponseRedirect from .models import Profile, FriendRequest from .forms import UserRegisterForm, UserUpdateForm, ProfileUpdateForm import random User = get_user_model() @login_required def users_list(request): users = Profile.objects.exclude(user=request.user) sent_friend_requests = FriendRequest.objects.filter(from_user=request.user) sent_to = [] friends = [] for user in users : friend = user.friends.all() for f in friend: if f in friends: friend = friend.exclude(user=f.user) friends += friend my_friends = request.user.profile.friends.all() for i in my_friends: if i in friends: friends.remove(i) if request.user.profile in friends: friends.remove(request.user.profile) random_list = random.sample(list(users), min(len(list(users)), 10)) for r in random_list: if r in friends: random_list.remove(r) friends += random_list for i in my_friends: if i in friends: friends.remove(i) for se in sent_friend_requests: sent_to.append(se.to_user) context = { 'users': friends, 'sent': sent_to } return render(request, "users/users_list.html", context) # … -
Django: Render API response to html, one page at a time
I have an API that returns a paginated response. My view (shown below) tries to get the response from this API and renders it to HTML search_result.html. However, I am only able to fetch the first page of the API response and not other pages. View.py query_url = "/api/transcript/search/?identifier_field=" + search_identifier + \ "&expand=transcript_release_set,genes,translations" response = requests.get(host_url + query_url) if response.status_code == 200: search_result = response.json() return render(request, 'search_result.html', context={'form': search_form, 'search_result': search_result, 'search_identifier': search_identifier}) The API response has been implemented through the below view: View2.py class TranscriptSearch(generics.ListAPIView): queryset = Transcript.objects.all() serializer_class = TranscriptSearchSerializer filter_backends = (TranscriptSearchFilterBackend, ) pagination_class = PageNumberPagination def get_queryset(self): queryset = Transcript.objects.order_by('pk') return queryset def get(self, request, *args, **kwargs): result = super(TranscriptSearch, self).get(request, *args, **kwargs) return result I do not want to render to HTML all API response pages at one go, that will slow down the loading of the HTML page. I want to load each API response page only when the user clicks a page number from HTML. So, how can I render to HTML each page of the API response one by one, without rendering all pages at once? -
Celery beat task is not executing when provided specific time
This question appears many times in stackoverflow, but I could not find correct answers (tried all the suggested ones, none of them worked). The code is as follows 'get-sensor-values': { 'task': 'weathers.tasks.get_fieldclimate_data_between_dates', 'schedule': crontab(hour='*/3'), }, 'Daily-Task-Start': { 'task': 'weathers.tasks.notify_task_start', 'schedule': crontab(hour=10, minute=54), }, The first one works, second one does not. -
Send a JSON through Django REST Framework
I am getting started with the Django rest framework and I encountered a problem that I am still not able to solve. Here is the problem: I want to create a tech store website. Using the API that the manufacturing company provides, I will upload the products on my website. The manufacturing company uses XML, and the result of the API is XML. Using python I converted the XML output from the API to JSON. Here is the code for that: import requests import xmltodict import json url = 'https://www.parterURLProductsInStock.asmx' headers = {'content-type': 'text/xml'} raw_data = """<?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <GetCTProducts xmlns="http://www.partnerURL.com/B2B"> <username>username</username> <password>psw</password> </GetCTProducts> </soap:Body> </soap:Envelope>""" x = requests.post(url, data=raw_data, headers=headers) js = xmltodict.parse(x.text) print(json.dumps(js)) For my website, I want to use Django and React. So in order to get the products in the frontend, I need an API, and in most cases, Django REST Framework is used. I know that with the Django REST Framework we can convert classes from modules.py into JSON and send it to the front using serializers. But how can I send the data (JSON) that is stored in my js variable? If anyone can helo me that would be … -
Django models relations: connect field to instances of itself
I am creating a network devices tracker where I want interfaces to be connected to one another only on some criteria. I am however stack at trying to connect interfaces together. I think my models logic is not right as in InterfaceConnection I am connecting an interface to a Device, not another interface. from django.db import models from django.core.exceptions import ValidationError from .constants import ROLES, FORM_FACTORS class Device(models.Model): hostname = models.CharField(max_length=200, unique=True, blank=False, null=False) ip_address = models.GenericIPAddressField(blank=False, null=False) role = models.CharField(max_length=20, choices=ROLES, blank=False, null=False) def __str__(self) -> str: return f"Device #{self.id} - hostname {self.hostname}" class Interface(models.Model): device = models.ForeignKey(to=Device, to_field="hostname", on_delete=models.CASCADE, related_name="interface") name = models.CharField(max_length=200, blank=False, null=False) form_factor = models.CharField(max_length=10, choices=FORM_FACTORS, blank=False, null=False) def __str__(self) -> str: return f"Interface #{self.id} - {self.name}" class InterfaceConnection(models.Model): interface = models.ForeignKey(to=Interface, blank=True, null=True, on_delete=models.CASCADE, related_name="interface_connection") device = models.ForeignKey(to=Device, blank=False, null=False, on_delete=models.CASCADE, related_name="interface_connection") def clean(self) -> None: if self.interface and self.interface.form_factor == 'virtual': raise ValidationError({ "interface": "virtual interface cannot be connected to any other interface" }) return super().clean() class Meta: unique_together = ("interface", "device") def __str__(self) -> str: return f"Connection #{self.id} - interface {self.interface.name} - device {self.device.hostname}" How do I express this type of relationship? I appreciate any insights. -
I am unable to identify my error in writing unit test for Django rest_farmework
I am trying to write a unit test to check my view for password reset in Django rest_framework. I can't identify where is the error and why am I getting the error. I am a beginner in django rest_framework. I would appreciated your help. my View.py from user_authentication.models import User as Client from rest_framework.response import Response from rest_framework.views import APIView class Password_Reset(APIView): def post(self, request): username = request.data.get('username') if Client.objects.filter(username = username).exists(): new_password = request.data.get('new_password') confirm_password = request.data.get('confirm_password') if new_password == confirm_password: user = Client.objects.filter(username = username).first() user.set_password(new_password) user.save() return Response({"message": "your password has been changed"}, 201) else: return Response({"message": "This email address doesn't exist"}, 404) my model.py from django.db import models from django.contrib.auth.models import AbstractUser class User(AbstractUser): created_at=models.DateTimeField(auto_now_add=True) email = models.EmailField(verbose_name="Email", max_length=60, unique=True) date_of_birth= models.DateField(verbose_name="Date of Birth", max_length=10) def __str__(self): return self.username my urls.py from django.urls import path from user_authentication.views import Login, Logout, Register, Password_Reset urlpatterns = [ path('login/', Login.as_view()), path('logout/', Logout.as_view()), path('register/', Register.as_view()), path('login/password_reset', Password_Reset.as_view()) ] my Unit Test for password reset from rest_framework.test import APITestCase from user_authentication.models import User from user_authentication.views import Password_Reset, Register, Login, Logout class ViewTest(APITestCase): def test_password_reset(self): url = 'login/password_reset' response = self.client.post(url, {'username': "developer", "new_password": "abc1234", "confirm_password": "abc1234"}) user = User.objects.get(username = 'developer') … -
Django ForeignKey, hide from user select options in dropdown menu
I have standard Django models with ForeignKey. Django docs: "ForeignKey is represented by django.forms.ModelChoiceField, which is a ChoiceField whose choices are a model QuerySet." and "If the model field has choices set, then the form field’s widget will be set to Select, with choices coming from the model field’s choices." Now I have dropdown menu with choices. I don't want dropdown menu where user can see options. I want CharField(textfield or similar) where user type, but still that must be one of the options from the database for that field. He must type a valid entry. I tried: class TransakcijeForm(forms.ModelForm): model = models.Transakcije fields = ..... labels = ..... widgets ={'subscriber':forms.TextInput()} but I receive the message: "Select a valid choice. That choice is not one of the available choices." (entry is correct and it works with dropdown menu) This is my first question here and I'm sorry if I miss the form. -
Django-filter Choice Filter does not recognize an existing field
I am trying to appply a ChoiceFilter however it as if the field I am referring to is not recongized wherehas it does exist : ModelViewsSet : class StartUpViewSet(NestedViewSetMixin, ModelViewSet): """ Class that provides List, Retrieve, Create, Update, Partial Update and Destroy actions for startups. It also include a filter by startup status """ model = Startup queryset = Startup.objects.all() serializer_class = StartupSerializer filter_backends = (filters.DjangoFilterBackend,) filterset_fields = 'status' FilterClass : class StartupFilter(filters.FilterSet): status = filters.ChoiceFilter(choices=START_UP_STATUS) class Meta: model = Startup fields = ['status'] And model : class Startup(models.Model): header = models.CharField("Header", max_length=255) title = models.CharField("Title", max_length=255) description = models.CharField("description", max_length=255) # TODO Change this to options instead of array tags = ArrayField(models.CharField(max_length=10, blank=True), size=5) # TODO Images to be stored in aws only url will be in DB card_image = models.ImageField(upload_to='media/images/cards') logo_image = models.ImageField(upload_to='media/images/logos') main_img = models.ImageField(upload_to='media/images/main', null=True) createdAt = models.DateTimeField("Created At", auto_now_add=True) status = models.IntegerField(choices=START_UP_STATUS, default=1) def __str__(self): return self.title -
I can't access to public IPv4 address not public IPv4 DNS
i opened all inbound rules, and outbound rules. i can access to public ipv4 DNS well. IPv4 Address IPv4 DNS What should i do.... -
Getting all LDAP users with django
I'm using django-auth-ldap and I'm trying to get all users from LDAP server, I didn't find a way to do so, the authenticating is working in the backend, here is my code @api_view(['GET']) def users(request): l = ldap.initialize(backend.AUTH_LDAP_SERVER_URI) l.simple_bind_s(backend.AUTH_LDAP_BIND_DN, backend.AUTH_LDAP_BIND_PASSWORD) users = LDAPBackend().ldap_to_django_username('*') #this line is returning None print (users) serializer = userSerializer(users, many=True) return Response(serializer.data) I know that line is incorrect LDAPBackend().ldap_to_django_username('*') but I really need to get all users with something like this : conn.search('ou=users, dc=example,dc=com', '(objectclass=person)', attributes=['uid', 'cn', 'givenName', 'sn', 'gidNumber']) (this is how I did with another library called ldap3) Thank you -
How to make changes to a Django website already deployed on heroku
Hello guys my name is Destiny and I really have this issue that I'm facing right now. I deployed a website on to heroku and I want to make changes to it, like add updates to the website but right now I don't know the right thing to do I don't know if I should re-deployed my new changes to heroku again, I don't just know what to do please if anybody has any idea on how I can do this any help would be greatly appreciated thanks a lot in advance -
How to load data with user id condition with Django Model?
How to load data from the existing value of the user id? class mmd1(models.Model): userid = models.ForeignKey( 'user.usid', related_name = "User2", verbose_name = 'Partner', ) ttx = models.CharField( _('ttx'), max_length = 100 ) data = models.ForeignKey( 'data', related_name = "data", verbose_name = 'data', limit_choices_to={'id': userid} ) Unable to load data based on user id and error this line: limit_choices_to={'id': userid} How to fix this issue? -
Setting up Angular & Django for frontend & backend
I had a fully-functioning Angular app that was being served on the 4200 local port. I followed a few tutorials and set it up as frontend, with Django as backend. Now, some of the images and resources do not show on the page, even though I can see that they are transferred into to the static folder, inside the assets folder. For example, my app.component.html and app.component.css are as follows with regards to a project logo: <div class="toolbar" role="banner"> <img class="headerlogo" width="150" (click)="goNavigateToPage('')" alt="Angular Logo" src="../assets/Projektlogo.png" /> <div class="spacer"></div> <div class="col-xs-12"> <app-header></app-header> </div> <div class="spacer"></div> <a href="/"></a> <img class="headerlogo" (click)="goNavigateToPage('')" src="../assets/Projektlogo.svg" alt="" height="100" width="150" /> </div> .bg { background-color: #f0abab; background-image: url(../assets/ProjektlogoBackground.png); } But the project logo does not show when I run the Django server... Should we modify all paths such as src="../assets/Projektlogo.png" manually for Angular and Django to work together, or should this be done automatically and I am missing something? -
How to check equal in template?
This is my model. class Ad_company(models.Model): idx = models.AutoField(primary_key=True) subject = models.CharField(max_length=255) memo = models.CharField(max_length=255) content = models.TextField() is_display = models.CharField(max_length=1) writer = models.CharField(max_length=255) write_date = models.DateTimeField() update_date = models.DateTimeField() delete_date = models.DateTimeField() deadline_date = models.DateTimeField() reply = models.IntegerField(blank=True) hits = models.IntegerField(blank=True) ad_apply = models.IntegerField(blank=True) ad_category1 = models.CharField(max_length=255) ad_category2 = models.CharField(max_length=255) ad_place = models.CharField(max_length=255) ad_age = models.CharField(max_length=255) ad_sex = models.CharField(max_length=255) ad_budget = models.BigIntegerField() ad_length = models.CharField(max_length=255) is_done = models.CharField(max_length=1) is_pay = models.CharField(max_length=1) ad_service = models.CharField(max_length=255) ad_object = models.CharField(max_length=255) is_file = models.CharField(max_length=1) ad_require = models.CharField(max_length=255) contract_user = models.CharField(max_length=255, blank=True) And This is my html template code {% if adlist.contract_user == user %} <span class="ad-state full-time">contract</span> {% endif %} adlist.contract_user is asdf and user is asdf same but It doesn't work. I think I need to change string or something? -
how to remove upload image validation in Django?
How can I remove this validation in django? code in Django is like this: image = models.ImageField(upload_to="data", null=True, blank=True) -
custom filtering in django rest framework
I am trying to implement custom filtering in my code and went through the documentation. But I couldnt understand the following snippets from the docs. class UserFilter(django_filters.FilterSet): class Meta: model = User fields = ['username', 'last_login'] # or class UserFilter(django_filters.FilterSet): class Meta: model = User fields = { 'username': ['exact', 'contains'], 'last_login': ['exact', 'year__gt'], } The doc says we can use either one the first or second one but what is the exact difference?? What is the meaning of contains and year_gt? -
Django Serializer is not JSON serializable
I'm trying to serialize an object details which contains ForeignKey and OneToOneField. Here is my Model: user = models.OneToOneField( "User", on_delete=models.CASCADE, null=False, blank=False, verbose_name="User", help_text="The user who subscribed.", related_name="subscription_information", unique=True, ) subscription = models.ForeignKey( Subscription, on_delete=models.CASCADE, null=False, blank=False, related_name="subscription_information", verbose_name="Subscription", help_text="This is the subscription.", ) subscription_type = models.IntegerField( choices=SUBSCRIPTION_TYPES_CHOICES, default=SubscriptionTypes.monthly, null=False, blank=False, verbose_name="Subscription Type", help_text="", ) next_payment_amount = models.FloatField( default=0.0, null=False, blank=True, verbose_name="Subscription Plan Next Payment Amount", help_text=(""), ) next_payment_date = models.DateTimeField( null=True, blank=True, default=None, verbose_name="Next Payment Date", help_text=(""), ) payment_made = models.BooleanField( null=False, blank=True, default=False, verbose_name="Is Payment Made", help_text=( "" ), ) subscription_date = models.DateTimeField( null=True, blank=True, default=None, verbose_name="Subscription Date", help_text="", ) As you see User field is OneToOneField and Subscription field is foreign key. And here is my serializer: class SubscriptionInformationDetailSerializer(serializers.ModelSerializer): class Meta: model = SubscriptionInformation fields = ( "id", "user", "subscription", "subscription_type", "next_payment_amount", "next_payment_date", "payment_made", "subscription_date", ) I want to return serialized SubscriptionInformation with this code: subscription_information = SubscriptionInformation.objects.get(user_id=user.id) serializer = SubscriptionInformationDetailSerializer(subscription_information, read_only=True) return serializer But it throws this error: Traceback (most recent call last): File "E:\Programming\project\venv\lib\site-packages\django\core\handlers\exception.py", line 42, in inner response = get_response(request) File "E:\Programming\project\venv\lib\site-packages\django\core\handlers\base.py", line 217, in _get_response response = self.process_exception_by_middleware(e, request) File "E:\Programming\project\venv\lib\site-packages\django\core\handlers\base.py", line 215, in _get_response response = response.render() File "E:\Programming\project\venv\lib\site-packages\django\template\response.py", line 109, … -
download database though sql sentence in Django view
So I have a Django view that downloads a MySQL database table depending on the button selected. The problem is that when I download the tables, the words in the CSV are changed with all commas. So what I wanted is to change the code that I have with SQL sentences. But I don't know how to export that SQL sentence to a CSV later. Help is much appreciated. This is what I have for the moment My Views.py: def download_db(request, mode): response = HttpResponse(content_type='text/csv') writer = csv.writer(response) if mode == 1: writer.writerow(['session_id', 'word1', 'word2', 'word3', 'distance_to_word12', 'distance_to_word13', 'distance_to_word23']) for word in WordDifferentTable.objects.all().values_list('session_id', 'word1', 'word2', 'word3', 'distance_to_word12', 'distance_to_word13', 'distance_to_word23'): writer.writerow(word) response['Content-Disposition'] = 'attachment; filename="ForWordDifferent.csv"' return response elif mode == 2: writer.writerow(['session_id', 'word1', 'word2', 'word3', 'distance_to_word12', 'distance_to_word13', 'distance_to_word23']) for word in LogicAnaloguiesTable.objects.all().values_list('session_id', 'word1', 'word2', 'word3', 'distance_to_word12', 'distance_to_word13', 'distance_to_word23'): writer.writerow(word) response['Content-Disposition'] = 'attachment; filename="ForLogicAnalogies.csv"' return response elif mode == 3: writer.writerow(['session_id', 'word1', 'word2', 'word3', 'distance_to_word12', 'distance_to_word13', 'distance_to_word23']) for word in SimilarWInContextTable.objects.all().values_list('session_id', 'word1', 'word2', 'word3', 'distance_to_word12', 'distance_to_word13', 'distance_to_word23'): writer.writerow(word) response['Content-Disposition'] = 'attachment; filename="ForSimilarWordInContext.csv"' return response elif mode == 0: writer.writerow(['session_id', 'word1', 'word2', 'word3']) for word in WordsTable.objects.all().values_list('session_id', 'word1', 'word2', 'word3'): writer.writerow(word) response['Content-Disposition'] = 'attachment; filename="WordsTable.csv"' return response return JsonResponse({}, status=304) One of the … -
How to link two databases together?
I have 2 databases that are supposed to be linked by a foreign key: DealershipListing and Dealers. Here are their models: class Dealer(models.Model): dealersName = models.TextField(('DealersName')) zipcode = models.CharField(("zipcodex"), max_length = 15) zipcode_2 = models.CharField(("zipCode"), max_length = 15) state = models.CharField(("state"), max_length=5) address = models.TextField(("Address")) ids = models.BigIntegerField(("ids"), primary_key=True) def __str__(self): return self.dealersName class DealershipListing(models.Model): uniqueID = models.IntegerField(("CarID"), primary_key=True) vincode = models.CharField(('vinCode'), max_length=255) price = models.CharField(('price'), max_length=30) msrp = models.CharField(('msrp'), max_length=30) mileage = models.CharField(('mileage'), max_length=9) is_new = models.BooleanField(('isNew')) first_seen = models.CharField(("first_seen"), max_length=15) last_seen = models.CharField(("last_seen"), max_length=15) model = models.CharField(("Models"), max_length= 255) make = models.CharField(("Make"), max_length=255) year = models.CharField(("Year"), max_length= 4) ids = models.ForeignKey(Dealer, on_delete=CASCADE) color = models.CharField(("ExtColor"), max_length=255, null=True, blank = True) intcolor = models.CharField(("IntColor"), max_length=255, null=True, blank=True) def __str__(self): return self.year + " " + self.make + " " + self.model But when I go check the admin database, the DealershipListing is a dropdown of the names of every dealership in the Dealer model, but the ids on the DealershipListing model is placed in the place of color. What am I doing wrong? How do I connect each car in the DealershipListing to their Dealership by the ids? -
Why the session isn't flushing?
Working on a simple project using Django 3.2. Lately I have put to the project a login/register page. The problem is the when I logout from the account the session isn't flushed. I used the code request.session.flush, and still doesn't work. For better understanding will show the code below: @login_required(login_url='login/') def logoutUser(request): logout(request) request.session.flush() return redirect("/") -
Local Host file into Cloud
I am using Django. I scraped a website and with the help of Django, I make the front-end. In Django, we run python manage.py runserver to run the localhost project. Now the next task is that I can save that file in Cloud so that everyone can access that file without using the"python manage.py runserver" command or by just clicking that file so the front-end can be shown easily. Is there any possible way to do that particular task -
Why are my static files not served when deployed to Heroku server? (Django)
So this is a common mistake for Django users, which is static files not served in production when Debug=False. I've tried many methods to overcome this issue, but still cannot figure out right solution. Below is my settings.py ... DEBUG = False MIDDLEWARE = [ 'whitenoise.middleware.WhiteNoiseMiddleware', ... ] ... STATIC_URL = '/static/' STATICFILES_DIRS = [ os.path.join(BASE_DIR, 'static'), os.path.join(BASE_DIR, 'main/static'), os.path.join(BASE_DIR, 'member/static'), os.path.join(BASE_DIR, 'register/static'), ] STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles') STATICFILES_FINDERS = [ 'django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', ] I don't see how I've done wrong. *One thing that might be a potential cause is that I've set DISABLE_COLLECTSTATIC=1 since the initial deploy to heroku server. remote: Compressing source files... done. remote: Building source: remote: remote: -----> Building on the Heroku-20 stack remote: -----> Using buildpack: heroku/python remote: -----> Python app detected remote: -----> Using Python version specified in runtime.txt remote: -----> No change in requirements detected, installing from cache remote: -----> Using cached install of python-3.7.12 remote: -----> Installing pip 21.3.1, setuptools 57.5.0 and wheel 0.37.0 remote: -----> Installing SQLite3 remote: -----> Installing requirements with pip remote: -----> Skipping Django collectstatic since the env var DISABLE_COLLECTSTATIC is set. remote: -----> Discovering process types remote: Procfile declares types -> web remote: remote: -----> Compressing... remote: … -
Media Files of a Django App deployed in Azure App Service
Like the tittle says, I have a Django App deployed to Azure App Service through GitHub the thing is, I can't access the media files. I can access the Django Admin site to upload the files to an "uploads" folder which I can find through SSH Link to SSH Image But when I try to fetch that image using https/mysite/uploads/image.jpeg in the Vue App I get a 404 Error, everything else works just fine. This my URLPatterns variable: urlpatterns = [ path('admin/', admin.site.urls), path('api/v1/', include('djoser.urls')), path('api/v1/', include('djoser.urls.authtoken')), path('api/v1/', include('commerce.urls')), path('api/v1/', include('ordenes.urls')), path('reporte/ventas/', AlmacenPDFView.as_view(), name='reporte_productos'), path('reporte/finanzas/ordenes/', OrdenesPDFView.as_view(), name='reporte_ordenes'), ] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) Question is, How can I get access to those images? or How can I serve the media files? Any help is much appreciated! -
Run scrapy crawler from DRF view
i used scrapy in my project and i want to call my spider with a URL from DRF (Django Rest Framework) View, what is the best way? one of the way i used is : from uuid import uuid4 from django.core.cache import cache from urllib.parse import urlparse from django.core.validators import URLValidator from django.core.exceptions import ValidationError from django.views.decorators.http import require_POST, require_http_methods from django.shortcuts import render from django.http import JsonResponse from django.views.decorators.csrf import csrf_exempt from scrapyd_api import ScrapydAPI from main.models import Quote from scrapy_app.scrapy_app.items import QouteItem import os from iCrawler.settings import BASE_DIR scrapyd = ScrapydAPI('http://localhost:6800') def is_valid_url(url): validate = URLValidator() try: validate(url) except ValidationError: return False return True @csrf_exempt @require_http_methods(['POST', 'GET']) def crawl(request): if request.method == 'POST': url = request.POST.get('url', None) if not url: return JsonResponse({'error': 'Missing args'}) if not is_valid_url(url): return JsonResponse({'error': 'URL is invalid'}) domain = urlparse(url).netloc unique_id = str(uuid4()) settings = { 'unique_id': unique_id, 'USER_AGENT': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' } task = scrapyd.schedule('default', 'icrawler', settings=settings, url=url, domain=domain) return JsonResponse({'task_id': task, 'unique_id': unique_id, 'status': 'started'}) elif request.method == 'GET': task_id = request.GET.get('task_id', None) unique_id = request.GET.get('unique_id', None) if not task_id or not unique_id: return JsonResponse({'error': 'Missing args'}) status = scrapyd.job_status('default', task_id) if status == 'finished': try: item = QouteItem.objects.get(unique_id=unique_id) return …
