Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
Scrap each product by filtering comments
I am trying to scrape each product page from aliexpress website. I have written this code to get number of comments, number of photos published by the custumer and also the custumer country and put it to a dataframe. But I want to get only the comments published by certain custumer in other words to apply a filter on comments by custumer country , for example get the comments published by the custum whom country is 'US', but really I don't know how to establish that filter : from selenium import webdriver from selenium.webdriver.common.by import By from webdriver_manager.chrome import ChromeDriverManager from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import pandas as pd url = 'https://www.aliexpress.com/item/1005003801507855.html?spm=a2g0o.productlist.0.0.1e951bc72xISfE&algo_pvid=6d3ed61e-f378-43d0-a429-5f6cddf3d6ad&algo_exp_id=6d3ed61e-f378-43d0-a429-5f6cddf3d6ad-8&pdp_ext_f=%7B%22sku_id%22%3A%2212000027213624098%22%7D&pdp_pi=-1%3B40.81%3B-1%3B-1%40salePrice%3BMAD%3Bsearch-mainSearch' driver = webdriver.Chrome(ChromeDriverManager().install()) driver.get(url) wait = WebDriverWait(driver, 10) driver.execute_script("arguments[0].scrollIntoView();", wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.tab-content')))) driver.get(wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '#product-evaluation'))).get_attribute('src')) data=[] number_feedbacks = driver.find_element(By.XPATH, '//*[@id="transction-feedback"]/div[1]') number_images = driver.find_element(By.XPATH, '//*[@id="transction-feedback"]//label[1]/em') print(f'number_feedbacks = {number_feedbacks}\nnumber_images = {number_images}') while True: for e in driver.find_elements(By.CSS_SELECTOR, 'div.feedback-item'): try: country = e.find_element(By.CSS_SELECTOR, '.user-country > b').text except: country = None data.append({ 'country':country, }) try: wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '#complex-pager a.ui-pagination-next'))).click() except: break pd.DataFrame(data).to_csv('filename.csv',index=False) Any help would be great. Thank you ! -
im getting "variable has been referenced before intialization" error at the last line in django framework
def result(request): check = None try: check = user_answers.objects.get(username=request.session["username"]) except: check = None if check: results,a1,a2,a3,a4,a5 = getPredictions(check) username = request.session["username"] ans = clusterNum.objects.create(username_id=request.session["username"],clusNo = results[0],extroversion=a1,neurotic=a2,agreeable=a3,conscientious=a4,openness=a5) ans.save() # results = getPredictions() return render(request,'result.html',{'name':username,'a1':a1,'a2':a2,'a3':a3,'a4':a4,'a5':a5}) i tried converting them to global but the result is not as expected,it show no values in the webpage -
Select multiple items in Django Admin
Right now filter can be done for all or none i want to make to multiple item selection. In which way I can do that. filter=(2,5,6) -
TypeError at / save() missing 1 required positional argument: 'self'
I cannot update the balance amount, when I try to save the value, it raises that error. models.py, class balance_data(models.Model): user = models.OneToOneField(User, on_delete=models.CASCADE) total_amount = models.FloatField(max_length=15) def __str__(self): return str(self.user) Views.py, def cash_in(request, amount): new_amount = request.user.balance_data.total_amount + amount balance_data.total_amount = new_amount balance_data.save(update_fields=['total_amount']) return HttpResponse(request.user.balance_data.total_amount) How can I solve this? I am expecting to update the amount. -
recognized command (autofile) does not work when I create a file
Hello friends I work through the VScode platform And also works with django, And I have a problem when I create a templates folder and within it I want to create an HTML file And when I type the html command it does not complete it for me automatically as if it does not recognize the file. But if I create a file in another folder in the project he recognizes it very nicely. Here are some pictures to illustrate. Here you can see the path when I create an html file I have the autocomplete. And here when I create a file inside a template folder it does not know to recognize. -
How to send Crypto payments in Django/Python?
I have a website that basically would categorize under gambling. So the users will be able to deposit and withdraw money/crypto. I have created the deposit part where the user is able to deposit money to the website. Now I am stuck with withdraw part. Every tutorial and post I read are about the user to website payment part. I can't find anything that would help me with the website to user process. For deposit I used PayPal and Coinbase APIs. Also I am quite new to this so sorry if my question is ridiculous or badly worded. Could someone please help me with the solution or at least lead me to the right direction? -
Using Django, send a Python Dictionary to a HTML Page and convert it to Javascript arrays in Javascript Script
I have been trying to send a Python Dictionary to a HTML Page and use that dictionary in Javascript to add a Graph on my website, using Django The form takes a Image Upload, and the code is as follows, <div class=" mx-5 font-weight-bold"> Uplaod Image </div> <input class=" " type="file" name="filePath"> <input type="submit" class="btn btn-success" value="Submit"> This image is then sent to views.py where it is processed and a resultant image, as well as a dictionary is generated from that image. And then again, a HTML page is rendered where the dictionary as well as the resultant image is sent in context variable. The code is as follows, def predictImage(request): fileObj = request.FILES['filePath'] fs = FileSystemStorage() filePathName = fs.save(fileObj.name, fileObj) filePathName = fs.url(filePathName) testimage = '.'+filePathName img_result, resultant_array, total_objects = detect_image(testimage) cv2.imwrite("media/results/" + fileObj.name, img=img_result) context = {'filePathName':"media/results/"+fileObj.name, 'predictedLabel': dict(resultant_array), 'total_objects': total_objects} #context = {} return render(request, 'index.html', context) Now, I want to convert the Dictionary items and keys into two different Javascript arrays, which are then to be used to plot a Graph on the HTML page. The template code of the Javascript is as follows, <script> // const value = JSON.parse(document.getElementById('data_values').textContent); // alert(value); var xArray = []; … -
Why running curl command using parallel needs to timeout?
I have set up daphne server with my django application and I wanted to use curl command to send parallel requests to the API that I have created. This is the command that I am using in terminal. seq 10 | parallel -j 5 --joblog pandu.log curl --connect-timeout 2 -v -F "FILE=@PATH_TO_FILE" http://127.0.0.1:8000/<wxy>/<xyz> So, I started the daphne server in one terminal and ran the curl command in another terminal. This is the view that I have created for handling API requests. @csrf_exempt def inputView(request): file = request.FILES['FILE'] call_print_primers(file) return HttpResponse("well I have recieved the file.") This is the output I am getting when I ran the curl command. Trying 127.0.0.1:8000... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0) > POST /<wxy>/<xyz> HTTP/1.1 > Host: 127.0.0.1:8000 > User-Agent: curl/7.82.0 > Accept: */* > Content-Length: 2828 > Content-Type: multipart/form-data; boundary=------------------------4e8f33ef7353b32d > } [2828 bytes data] * We are completely uploaded and fine * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Content-Type: text/html; charset=utf-8 < X-Frame-Options: DENY < Content-Length: … -
Category and subcategory for django-filter
I want to build a filter for category and subcategory. The filter works enough good, but I don't know how to build left space for children (subcategory). Also I don't know how to build by link. For example, I click a category "Nature" and this category contains different subcategory. In short, I have two question: How to build space for children (subcategory)? how to present categories and subcategories by link? I will be happy of any answer. My code which I tried to use Django-MPTT. But I have error when I use "tree info" and "recursetree" - "'BoundWidget' object has no attribute '_mptt_meta'" template.html <form method="get" class="d-flex flex-column"> {% for su,structure in filter.form.subject|tree_info %} {% if structure.new_level %}<ul> <li>{% else %}</li> <li>{% endif %} {{ su }} {% for level in structure.closed_levels %}</li> </ul>{% endfor %} {% endfor %} OR {% recursetree filter.form.subject %} <li> {{ node }} {% if not node.is_leaf_node %} <ul class="children"> {{ children }} </ul> {% endif %} </li> {% endrecursetree %} <button class="btn-primary mt-2" type="submit">Search</button> </form> models.py class Book(models.Model): title = models.CharField(max_length=255, db_index=True) author = models.ForeignKey( "users.CustomUser", on_delete=models.SET_NULL, null=True, db_index=True ) subject = TreeManyToManyField("Subject") class Subject(MPTTModel): name = models.CharField( max_length=100, unique=True, verbose_name=_("category name"), help_text=_("format: required, … -
Manage multiple accounts in different companies with django
I'm learning and working with django less than a year ago. Now I have to develop a site managing multiple users separated by many different companies. I've recently found django-organizations package which seems suitable for this application, I have read the complete documentation in readthedocs, but I can't find simple working example apps to start trying, and the activity on communities like stackoverflow on this subject seems to have decayed two years ago. Please can anybody tell me if django-organizations is an updated package, compatible with last python and django versions, and provide a source of simple “Hello World”-like examples. Or, by the other hand, please suggest me the best way/library to manage signup, login, permissions, etc,in a multiple users – multiple company Django web application. Thanks in advance. -
Loading an interactive Plotly graph at a click of a button in Django
I am working on a stock screener project using Django, Pandas and Plotly. One of the pages is the user's watchlist where certain stats get displayed for each stock (one HTML table row per stock). This HTML table gets generated in the "watchlist" view by converting a Pandas DataFrame into html using the "to_html" method (with escape=False). Each row of the HTML table has a "View graph" button which opens an interactive Plotly graph for the relevant stock. The graph is generated based on the results of the calculations made in the "watchlist" view, and is represented by an html string that gets sent to the "watchlist" template as part of the context. At the moment I am loading all the stock graphs below the HTML table each time the "watchlist" page gets loaded, but initially the graphs remain hidden with css. Each graph has an id in the dataset that allows me to "unhide" it with JavaScript when someone clicks the matching button with the same id in the table. However, loading all the graphs at once when the "watchlist" page is being loaded takes a lot of time, and I am trying to think of for a solution … -
Cannot set user.is_active to false
I have a custom user model and I am trying to deactive the user but it is not saving. There are also no error messages. My model: class User(AbstractUser): # ... just some additional attributes u = User.objects.get(email="some@email.de") u.is_active = False u.save() u.refresh_from_db() u.is_active # Still returns True but should be False -
django's urlize changing text color of entire string containing url substring
I am loading a long text from a database into a Django template. This string contains a url https://yyyyyy as part of the text. I use django's urlize in order to display the url as actual links, however adding urlize is causing the entire text including the link to change color to blue. Removing the urlize leaves the text as black. I only want the text to be black. How can I stop this color from changing to blue while using urlize? Here is my implementation : <p style="font-size: 15px; color: black" class="tm-pt-30"> {{ value.description | safe | urlize }} </p> The class tm-pt-30 is only responsible for padding. -
setUpTestData and setUp TestCase methods in Django
Can anyone explain me the differences between setUpTestData and setUp TestCase methods in Django? Documentation says. *The class-level atomic block described above allows the creation of initial data at the class level, once for the whole TestCase. This technique allows for faster tests as compared to using setUp(). Be careful not to modify any objects created in setUpTestData() in your test methods. Modifications to in-memory objects from setup work done at the class level will persist between test methods. If you do need to modify them, you could reload them in the setUp() method with refresh_from_db() * And a piece of code... from django.test import TestCase class MyTests(TestCase): @classmethod def setUpTestData(cls): # Set up data for the whole TestCase cls.foo = Foo.objects.create(bar="Test") def test1(self): # Some test -
Acces AJAX data as form data from a dictionary in Django
I am beginner in Django. I am trying to save a form data through AJAX. my code is $("#form").submit(function (e) { e.preventDefault(); var serializedData = $(this).serialize(); value = "user1"; $.ajax({ type: 'POST', url: "{% url 'create-form' %}", data: { "serializedData":serializedData, "value":value, 'csrfmiddlewaretoken': '{{ csrf_token }}', }, success: function (response) { console.log("response", response); /*window.location.replace("https://www.youtube.com/"); */ }, error: function (response) { console.log("error:", response.responseJSON.error); } }) }) And i am trying to accept data in views like this... I used class based view. I added only needed part of views.py code. def post(self, *args, **kwargs): if self.request.is_ajax and self.request.method == "POST" : form = self.form_class(self.request.POST) val = self.request.POST.get("value") if val == "user1" : if form.is_valid(): print("form is valid ") return JsonResponse({"Sucess": True}, status=200) else: print("form is not valid ") return JsonResponse({"error": form.errors} , status=400) .. My problem is when i used to save form = self.form_class(self.request.POST).. It access all values that i sended through AJAX. I want only save serialized data in form. I want to save it as form data because i want to work form with is_valid. -
TypeError: a coroutine was expected, got None
class Test(): def send(self,dela,i): time.sleep(dela) print("process",i) return async def proc(self,dela): task=list() for i in range(5): task.append(asyncio.create_task(self.send(3,i))) res = await asyncio.gather(*task) print(res) return def call(self): asyncio.run(self.proc(3)) return "123" obj=Test() obj.call() I want to run the send method in a for loop concurrently and dont want to wait for each iteration to be completed ,but in the end gather the tasks. -
How to open the file on internet and check the size
I want to open the url such as https://example.com/test.jpg and check the size of image. My idea is using subprocess with curl? However,I suspect.. it looks not good Is there any practice for this purpose? -
Weird 5 second halts for Django on Kubernetes
We have a K8 cluster to host some of our services. These services talk with each other to exchange data over HTTPS APIs. Sometimes we have seen that the execution of services halts for 5 seconds and then continues. We experience high latency during this issue and found that API calls are stalled for 5 seconds through APM traces. I have attached screenshot of one such sample. These services are running Django with gunicorn, containerised in Ubuntu images. What could be the reason of such issues? Is this related to name resolution or lookup? This even K8 related or something else totally? How can we proceed to debug this? If any additional information is required please let me know. Thank you. -
Break response when client goes from stream (StreamHttpResponse) (Django, Google Cloud Run)
I am using StreamHttpResponse to pass binary data to the client. First, I need to set the start and end times of the stream. Locally, when the client leaves the stream (I close the browser page), the close() method is triggered. From django code (from HttpResponseBase class): # These methods partially implement the file-like object interface. # See https://docs.python.org/library/io.html#io.IOBase # The WSGI server must call this method upon completion of the request. # See http://blog.dscpl.com.au/2012/10/obligations-for-calling-close-on.html def close(self): for closer in self._resource_closers: try: closer() except Exception: pass # Free resources that were still referenced. self._resource_closers.clear() self.closed = True signals.request_finished.send(sender=self._handler_class) On Google Cloud Run, this method is called only after 300 seconds (regardless of whether the client is listening to the stream or not). Is it possible to call the close() method when the client leaves the stream on the server? I expected that the response would be interrupted on the server as soon as the client leaves the stream (web page). It worked locally. -
ModuleNotFoundError: No module named 'django' while running the application with Gunicorn
I am getting error while running the application with gunicorn. Everything is working properly with manage.py but wsgi seems not to be working properly. Gunicorn command: gunicorn --bind 0.0.0.0:8000 basicprojectmanagement.wsgi wsgi.py import os from django.core.wsgi import get_wsgi_application os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'basicprojectmanagement.settings') application = get_wsgi_application() Versions: Django==4.0.2 gunicorn=20.1.0 Error (venv) root@vmi819232:/home/neotricinsights/neotricinsights# gunicorn --bind 0.0.0.0:8000 basicprojectmanagement.wsgi [2022-03-30 04:36:38 -0500] [223346] [INFO] Starting gunicorn 20.0.4 [2022-03-30 04:36:38 -0500] [223346] [INFO] Listening at: http://0.0.0.0:8000 (223346) [2022-03-30 04:36:38 -0500] [223346] [INFO] Using worker: sync [2022-03-30 04:36:38 -0500] [223348] [INFO] Booting worker with pid: 223348 [2022-03-30 04:36:38 -0500] [223348] [ERROR] Exception in worker process Traceback (most recent call last): File "/usr/lib/python3/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker worker.init_process() File "/usr/lib/python3/dist-packages/gunicorn/workers/base.py", line 119, in init_process self.load_wsgi() File "/usr/lib/python3/dist-packages/gunicorn/workers/base.py", line 144, in load_wsgi self.wsgi = self.app.wsgi() File "/usr/lib/python3/dist-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/usr/lib/python3/dist-packages/gunicorn/app/wsgiapp.py", line 49, in load return self.load_wsgiapp() File "/usr/lib/python3/dist-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp return util.import_app(self.app_uri) File "/usr/lib/python3/dist-packages/gunicorn/util.py", line 383, in import_app mod = importlib.import_module(module) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line … -
How can I serialize custom many-to-many model django?
I have 3 tables on data base: News, CustomUser(not important) and Comment: class News(models.Model): date = models.DateTimeField(auto_now_add=True) title = models.CharField(max_length=200) text = models.TextField() image = models.ImageField(upload_to='news_images/', blank=True) author = models.ForeignKey(CustomUser, on_delete=models.SET_DEFAULT, blank=True, default=1) tags = models.ManyToManyField(Tags, blank=True) published = models.BooleanField(default=False) comments = models.ManyToManyField( CustomUser, related_name='author', through='Comments.Comment', blank=True, ) class Comment(models.Model): date = models.DateTimeField(auto_now_add=True) text = models.TextField() author = models.ForeignKey(CustomUser, on_delete=models.SET_NULL, null=True) event = models.ForeignKey(News, on_delete=models.CASCADE, parent_link=True) As you can see, comment is a custom many-to-many model, that links Users and News, and adds some extra data (text of comment itself). I using DRF, so I need to serialize it. I need news serializer to include list of comments with all their fields(like author fields as you can see below on the screenshot).But when i getting one of the news I see that: {"id":2,"author":{"username":"Admin","email":"alexsiid29@gmail.com","first_name":"","last_name":""},"tags":[],"comments":[1],"date":"15.08.2021 10:10","title":"Тест2","text":"тестовая 2","image":"http://127.0.0.1:8000/news_images/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png","published":true} Comments: "comments":[1] And when I printing during the view-function execution: self.object = News.objects.filter(pk=pk).first() print(self.object.comments.all()) I getting this: <QuerySet [<CustomUser: Admin>]> So, instead of normal representation of comments I get the authors ID. There is my serializers: class CommentSerializer(serializers.ModelSerializer): author = CustomUserSerializer(source='author.id') event = NewsSerializer(source='event.id') class NewsSerializer(serializers.ModelSerializer): author = CustomUserSerializer(read_only=True) tags = TagSerializer(required=False, allow_null=True, many=True) comments = serializers.PrimaryKeyRelatedField(allow_null=True, many=True, read_only=True) So, how can I serialize this … -
Django Failed to establish a new connection <urllib3.connection.HTTPSConnection>
I use Django 2.2 and I have an api that is connected to two Esx servers. a = 10.131.171.80 b = 10.131.171.90 everything works perfectly, except that during my test when I add a third fake server, it tells me that the server does not respond because connection time out. So the error is handled and I don't have a yellow page from Django. Except when enter fourth server this time private ip like 172.16.15.15 I have an error Max retries exceeded NewConnectionError(<urllib3.connection.HTTPSConnection'> But when I use a public IP like 1.1.1.1, I don't get any error. What I would like is to handle the error better when someone enters a private IP. My full traceback Here is my api.py file that connects to Esx servers. import requests import os import json from requests.packages.urllib3.exceptions import InsecureRequestWarning requests.packages.urllib3.disable_warnings(InsecureRequestWarning) os.environ['REQUESTS_CA_BUNDLE'] = os.path.join('/etc/ssl/certs/') req = requests.Session() req.verify = False class VmwareApi: def __init__(self): self.ip = "" self.user = "" self.password = "" self.arg1 = "" self.ses = "" def session(self): try: a = req.post('https://' + self.ip + '/api/session', auth=(self.user, self.password), timeout=1, verify=False) self.ses = str(a.json()) except requests.exceptions.Timeout: return 'ConnectTimeoutError' return req def param_loader(self): if self.params: self.params = json.loads(self.params) def vapirequestget(self): try: VmwareApi.param_loader(self) myreq … -
how to use new table as a foreign key in existing table in django
Hi Everyone i have two model 1 in Dailytrip and 2nd one is Team, i need to use Team model as foreign key in Dailytrip model, while run Makemigrations getting error. i am using django-2.2 and python 3.7 Models.py class DailyTrip(BaseModel): date = models.DateField(default=None) Team=models.ForeignKey( Team, models.CASCADE, verbose_name='Team', null=True, default=None ) class Team(BaseModel): name = models.CharField(max_length=30) def __str__(self): return self.name Error class DailyTrip(BaseModel): File "E:\11-march-2022\everest_jarvis\fleet\models.py", line 595, in DailyTrip Team, NameError: name 'Team' is not defined -
Best strategy for saving profile picture using Django
This may be a very trivial question but its something I could not find a convincing answer. I am building a web application with Django as the backend. My application has a user profile which can take a profile picture. I have to decide on cloud storage solution. I wanted to ask what is best strategy for such a task. Storing on cloud, or storing on the web server itself. I want to know the pros and cons (ignoring cost). Also if I decide to go with cloud storage, what are the best for such a task. Here its profile picture (limiting single picture to say less than 1MB), so taking into account latency etc. I have worked on AWS S3 and have a solution in place to store these images on S3. But I wanted to have opinions. -
Beautifulsoup returning empty list. Not detecting div or class
I trying to scrape some data. But the beautifulsoup is not detecting the div or class. it is returning an empty list. def get_jobs(url): headers = { 'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0' } response = requests.get(url, headers=headers) print("RESPONSE IS: ", response.reason) if response.reason == "OK": soup = BeautifulSoup(response.text, 'html.parser') cards = soup.find_all('a',{'class':'job-cardstyle__JobCardComponent-sc-1mbmxes-0 eCvApI'}) print(cards) I am getting results if no class I was given. i need the data of the specific class.
