Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
Django views to run python script
i have chatbot web application in Django framework.so far everything is working but now i want to run the chatbot python script using ajax and calling the view for it from the Javascript file. i have an api using REST and the view for the python script and the ajax to call that view. view.py: from chat.chatbot1 import main_chatbot def run_python_script(request): os.system('python3 main_chatbot.py') return HttpResponse("OK") index.js: function run_chatbot_script(){ $.ajax({ type: 'GET', url: 'http://127.0.0.1:8000/chatbot/run_python_script/',}); and the python script folder is located in the chat app inside the chatbot django project. the problem is that the view can't find the file and this error appears: python3: can't open file 'main_chatbot.py': [Errno 2] No such file or directory -
Serializer results in KeyError after correct data input and save
I'm creating an API endpoint that works and saves data through its API. It works, but my concern is that it throws a KeyError as shown below. I'm not sure if this is an issue with my code. Your help is very much appreciated. Data: data = {'title': u'abc', 'career_level': 1} Serializer: JobPostSerializer(data={'title': u'abc', 'career_level': 1}): career_level = PrimaryKeyRelatedField(allow_null=True, queryset=CareerLevel.objects.all(), required=False) title = CharField(allow_blank=True, allow_null=True, max_length=240, required=False) Error Message: Internal Server Error: /api/v1/posts/ Traceback (most recent call last): File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 149, in get_response response = self.process_exception_by_middleware(e, request) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 147, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, **kwargs) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/viewsets.py", line 87, in view return self.dispatch(request, *args, **kwargs) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch response = self.handle_exception(exc) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch response = handler(request, *args, **kwargs) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/mixins.py", line 22, in create headers = self.get_success_headers(serializer.data) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 503, in data ret = super(Serializer, self).data File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 241, in data self._data = self.to_representation(self.validated_data) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 463, in to_representation attribute = field.get_attribute(instance) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/relations.py", line 157, in get_attribute return get_attribute(instance, self.source_attrs) File "/home/ubuntu/panbecopy/src/penv/local/lib/python2.7/site-packages/rest_framework/fields.py", line 78, in get_attribute instance = instance[attr] KeyError: u'career_level' -
Speed up Python xml Parsing
I need to speed up my code a factor of 2-3x, which may be asking too much. I am using celery to run the code on a schedule, but the code is not completing quickly enough and I am having CPU usage problems. I am using the Django ORM with Postgres. runTrafficData() is not the issue, it is an xml parser and is fast. The main issue is with the creating and saving of the Django objects. import sys from datetime import datetime from displaylivedata.models import StationSensor as SS from displaylivedata.models import TempLaneData as TempLD from displaylivedata.models import StationSensor from displaylivedata.models import Device from displaylivedata.models import SensorLocation from dateutil.parser import parse from django.utils import timezone import pytz from pprint import pprint from django.db import transaction from Traffic_Data import runTrafficData from displaylivedata.models import Lane approach_cnt = 0 def check(val): try: v = val return v except: return None @transaction.atomic def storeTrafficData(): ts = runTrafficData() time_zone = pytz.timezone('US/Central') timestamp = time_zone.localize(datetime.strptime(str(ts.local_date) + str(ts.local_time),'%Y%m%d%H%M%S')).astimezone(time_zone) if not Lane.objects.filter(timestamp = timestamp).exists(): for station in ts.stationList: for appr in station.approachList: lane_list = [] for laneInst in appr.lanes: lane_list.append(laneInst.lane_id) for laneInst in appr.lanes: lane = Lane() lane.lane_id = laneInst.lane_id lane.timestamp = timestamp lane.station_id_name = station.station_id lane.repeated_id = … -
Assign or `SELECT` or `OPTION` value to a django Form CharField
I need to send information form SELECT tag or tag OPTION to a specific Django form.CharFiled. need this in order to make the logic in SELECT, since some OPTION may not be available depending on the value in the Product (models.Model) Example: in Product(models.Model) item_1 = False item_2 = True in order.html <select> {% if item_1 %} <option>Item1</option> {% endif %} {% if item_2 %} <option>Item2</option> {% endif %} </select> User wil choose only item_2 and need this value was recorded in the form field like form.CharField and than send to validation. For that i need to assign or SELECT or OPTION value {{form.items_example}} Or is there another way to make the if logic in dajngo forms? -
Django attach a file to FileField encoding error
Would like to attach a file from a View to a Model in Django 1.8. It is working with a txt file but not with pdf or excel. I am getting a 'ascii' codec can't decode, exception type: UnicodeDecodeError. Do I need to set the encoding in Django or am I using "open" wrong? Exception Value: 'ascii' codec can't decode byte 0xb9 in position 11: ordinal not in range(128) Python Version: 3.5.2 views.py local_file = open('/var/www/taras/portal/portal1987/media/uploads/APP2017.xlsx', 'r') application = Application.objects.get(pk=pk) application.report_excel_doc.save('APPL2017_New.xlsx', File(local_file)) models.py report_excel_doc = models.FileField(null=True, blank=True) -
Django Admin Page: Help Text for Model Methods?
I have a model method in Django that I am displaying on an admin page just like I would a model field. With a field, I can just add a help_text argument to it to give a description of what the field is and what the user should put into it. However, with a model method, help_text does not work. Adding the attribute short_description changes the way the method name is displayed, which is sort of okay, but I'm looking for a way to add a few sentences of description beneath the method value that is displayed. Is there any way to do this natively, or would I have to resort to overriding admin templates or something? (Which I do not think is worth it for something this minor). -
How to join data with two foreign keys in django class to primary key class
I am trying to add a record of data in the Shell with two non-null Many-To-Many foreign keys (that link to the same primary field to a django class). The data entry described in the Django Many-to-Many documentation https://docs.djangoproject.com/en/1.10/topics/db/examples/many_to_many/ only works for adding a record with one M2M foreign key. p1 = Publication(title='The Python Journal') p1.save() a1 = Article(headline='Django lets you build Web apps easily') a1.save() a1.publications.add(p1) My data model uses two foriegn keys that describe relationships between entities. The form of M2M data entry doesn't work for two foreign keys. I am trying to follow the One-to-Many data entry https://docs.djangoproject.com/en/1.10/topics/db/examples/many_to_one/, which can be demonstrated like this : e1 = Entities(name='Community Board 2') e1.save() e2 =Entities(name='Land Use Committee') e2.save() r1 = Relationships(first_entity=e1, first_role='Parent', second_entity=e2, second_role='Child') I definied the M2M foreign key fields in the classes in models.py to have related names. When I try the O2M data entry, I get this error message : TypeError: 'second_entity' is an invalid keyword argument for this function I have an intermediary class that has a O2M relationship to the 'name' primary key, and data entry for that class works exactly as it should. I'm following Part 2 of the polls tutorial (which I … -
Can't sort table by model's property (accessor set)
I can't sort table by it's models property. I know that I should set accessor in the column so django-tables2 knows what field to process but it does not work. This is the table: class ScansTable(tables.Table): site = tables.columns.Column(accessor='occurence.site', verbose_name='Site') url = tables.columns.TemplateColumn("""<a href="{{ record.occurence.url }}">{{ record.occurence.url|truncatechars:20 }}</a>""", accessor='occurence.url', verbose_name='Url') price = tables.columns.TemplateColumn(u"""{{ record.price }} €""") date = tables.columns.Column(accessor='date',order_by='date') time = tables.columns.Column(accessor='time',order_by='time') class Meta: model = Scan fields = ('date', 'time', 'site', 'url', 'valid', 'price') attrs = {'id': 'cans_table', 'class': 'table',} This is the Scan model: class Scan(models.Model): occurence = models.ForeignKey('Occurence', related_name='scans') datetime = models.DateTimeField() price = models.DecimalField(max_digits=20,decimal_places=2,null=True,blank=True,verbose_name='Price') valid = models.BooleanField(default=True,verbose_name='Valid') def __unicode__(self): return u'{} {} {} {}'.format(self.occurence, self.datetime, self.price, u'OK' if self.valid else 'NOK') @property def date(self): return self.datetime.date() @property def time(self): return self.datetime.time() The view: def scans(request): ... scans = Scan.objects.filter(occurence__product=product) scans_table = ScansTable(scans) RequestConfig(request).configure(scans_table) scans_table.paginate(page=request.GET.get('page', 1), per_page=50) return render(request,"dashboard_app/scans.html",context={'scans_table':scans_table}) The table is being properly renderd when I don't want to sort it. When I click on time (for example), it returns: Cannot resolve keyword u'time' into field. Choices are: datetime, groups, id, occurence, occurence_id, price, valid Do you know where is the problem? -
Confused about how to make Django model field optional
I have an 'address' field in my Django model that I want to be optional: class Home(models.Model): address = models.CharField(max_length=256, blank=True, default=None) city = models.CharField(max_length=25) I want this to be an optional field in my database but I didn't set "null=True" because the documentation says "Avoid using null on string-based fields such as CharField and TextField because empty string values will always be stored as empty strings, not as NULL. If a string-based field has null=True, that means it has two possible values for “no data”: NULL, and the empty string. In most cases, it’s redundant to have two possible values for “no data;” the Django convention is to use the empty string, not NULL." However, when I create a Home object with this command: Home.objects.create(city='Chicago') I get an error: IntegrityError: null value in column 'address' violates not-null constraint. Shouldn't I add "null=True" to the address field despite what the docs say in order to make it optional? If not, how should it be declared so that I can create new Home objects without raising this error? -
Not getting validated_data.get() value properly
I am supposed to override the create() method on my serializer but I cannot understand why DRF cannot get me the real value of a validate_data.get('some_field', None). Here is my models. class Heat(models.Model): # Fields performer = models.CharField(max_length=25) is_bred = models.BooleanField(default=False) note = models.TextField(max_length=250, blank=True, null=True) class Breeding(models.Model): # Relationship Fields heat = models.OneToOneField( Heat, on_delete=models.CASCADE) # Fields performer = models.CharField(max_length=25) remarks = models.TextField(max_length=255, blank=True, null=True) And here is my serializer class class BreedingSerializer(serializers.ModelSerializer): def create(self, validated_data): heat_id = self.validated_data['heat'] heat = Heat.objects.get(pk=heat_id) breeding = Breeding.objects.create(**validated_data) return breeding Here is my validated_data looks like. { "heat": 1, "performer": "Some person", "remarks": "testing" } When i try to print the heat_id it yield the value as 'Heat Object' not the actual data provided as heat integer 1. but it works well in this code breeding = Breeding.objects.create(**validated_data). Can anyone explain? -
Django get choices value in lazy query
Simple question, is there a way do to something like this : Blog.objects.values('name', 'choices__values') instead of having to : blog.get_choices.display() for each object? -
Is it possible to have a a django middleware module contain its own template and static files?
I migrated my DUO 2FA to the middleware layer and it does the following: If the request is not authenticated through DUO, render the template containing the DUO login form My problem is, my middleware contains this code, but I'm pulling the template and static files from the templates folder for my frontend application and static files from my frontend application as well. Is it possible to contain both the depending template and static files within a middleware module? Here's a simplified view of how the folder structure looks like djangoproject myapp settings.py duo_middleware middlewareclass.py frontend views.py template index.html duo-login.html static someduodependency.js I'm looking for a possibility on adding a templates folder within the duo_middleware and a static folder -
How to run manage.py inside venv?
I have been given an existing project to work on and I am really struggling to get the environment set up. The project folder firstly contains manage.py server, which I use as an entry point to run the server. There is also a venv folder which contains all the modules etc. I need. So when I do runserver on manage.py, I get that "No module named sqlserver_ado.base". Even when I have activated the virtual environment and am in bash.... this module for instance is in venv folder in a venv\Lib\site-packages. I am so very confused. I have also tried copying whatever modules are said to be missing and have ran into other issues this way also. -
generic.edit.CreateView Assign Default Values and Insert Into Database
I have the following class-based view that uses my model to generate a "Create" screen: class ProjectCreateView(generic.edit.CreateView): model = Project fields = ['project_title','project_description','created_date'] template_name = 'steps/project-detail.html' success_url = reverse_lazy('index') Model: class Project(models.Model): project_title = models.CharField(max_length=200) project_description = models.CharField(max_length=200, default="") created_date = models.DateTimeField('date created') owner = models.ForeignKey(User) def __str__(self): return self.project_title Now, the problem is that if I do not want the 'created_date' to be on the actual form. Instead, I would like it to be populated automatically whenever the Submit button is pressed or whenever the record is being added into database. How can I achieve this? Lastly, is it possible to also specify the owner field by default? That is, I need to populate that field with information of the user who is creating this record. -
Aggregating Django Queries for Many-to-Many related models
I have three related models listed below and I would like to run an aggregate query to give me the total number of tasks completed by person but only those tasks that are in a specific folder. class Person(models.Model): firstName = models.CharField(max_length=100, null=True, blank=True) lastName = models.CharField(max_length=100, null=True, blank=True) class Folder(models.Model): title = models.CharField(max_length=254, null=True, blank=True) assignees = models.ManyToManyField(Person, related_name="projects") completedDate = models.DateTimeField(blank=True, null=True) class Task(models.Model): title = models.CharField(max_length=254, null=True, blank=True) assignees = models.ManyToManyField(Person, related_name="tasks") folders = models.ManyToManyField(Folder, related_name="tasks") And here is my attempted query: tasks = Contact.objects.filter(tasks__folders__id='I58343DS89ASDF').distinct().filter(tasks__completedDate__gte='2016-10-01 00:00:00').annotate(total=Count('tasks'), name=F('firstName')).values('total', 'name') This is giving me an incorrect count. So I inspected the SQL statement that is generated by this Django query. The SQL statement is: SELECT DISTINCT COUNT(`wrike_task_assignees`.`task_id`) AS `total`, `wrike_contact`.`firstName` AS `name` FROM `wrike_contact` INNER JOIN `wrike_task_assignees` ON (`wrike_contact`.`id` = `wrike_task_assignees`.`contact_id`) INNER JOIN `wrike_task` ON (`wrike_task_assignees`.`task_id` = `wrike_task`.`id`) INNER JOIN `wrike_task_folders` ON (`wrike_task`.`id` = `wrike_task_folders`.`task_id`) INNER JOIN `wrike_task_assignees` T6 ON (`wrike_contact`.`id` = T6.`contact_id`) INNER JOIN `wrike_task` T7 ON (T6.`task_id` = T7.`id`) WHERE (`wrike_task_folders`.`folder_id` = I58343DS89ASDF AND T7.`completedDate` >= 2016-10-01 00:00:00) GROUP BY `wrike_contact`.`id` ORDER BY NULL And a nicely formatted version is: SELECT DISTINCT COUNT(ta.task_id) AS `total`, c.firstName AS `name` FROM wrike_contact c INNER JOIN wrike_task_assignees ta ON (c.id … -
Python Django - Models: for loop in query/filter
so i want to do a query where a field equals a property of an object in a for loop. Here is a working sample/example using sqlAlchemy: you = session.query(Users).filter_by(id=login_session['userid']).first() friends = session.query(Friends).filter_by(user_id=you.id).all() dashboard = session.query(Markers).filter(Markers.owner.in_([f.friend_id for f in friends)]).all() How would i do this in django? -
Streaming large byte iterator over http
In my application, I need to stream large amount of data to the end user over http. Initially I was thinking about using a StreamingHttpResponse like so: def my_view(...): def my_generator(): for _ in range(10000): yield b'' return StreamingHttpResponse(my_generator) Then, I configured uwsgi with a large amount of threads to handle concurrency: processes = cpu_cores threads = 100 This is working great for this specific view but is degrading the performance of the other views which are pretty normal (fast request / response cycles). I also tried to use the async workers (with Greenlet). Same result: working great for this view but degrading the performance of the remaining of the application. I am now thinking of writing a little server using asyncio which would only handles this iterator. I could also run an other uwsgi instance with async workers only for this view but that would complexify the overall architecture. Is there a better way to do that ? Is there any way to only uses async workers for a part of an application ? -
Using mail-templated for Django, how do I NOT display a bullet at the end of my sent emails?
Using this as the file 'my_email.tpl': {% extends "mail_templated/base.tpl" %} {% block subject %} Some subject 🚀 {% endblock %} {% block html %} T {% endblock %} I send an email with this code: from mail_templated import send_mail email = 'somearbitraryemailaddress@hotmail.com' send_mail('my_email.tpl', {'user': email}, 'noreply@somecompany.com', [email], fail_silently=False) I have to have something in the HTML block or I get a character encoding error, hence the letter 'T' - However, my issue is that no matter what I put in the HTML block I always get a bullet or full stop at the end of the email. I don't want that as it looks rubbish and badly formatted. Furthermore, the bullet is huge on mobile phone email screens and looks absolutely awful. Please can someone advise as to how to remove this bullet which I didn't ask for? Thanks very much. -
Django: objects and model_set
I am learning django 1.10 official tutorial part 2 class Question(models.Model): # ...... class Choice(models.Model): question = models.ForeignKey(Question, on_delete=models.CASCADE) #....... Recent i saw the following command:- q = Question.objects.get(id=1) q.choice_set.all() My questions:- How Question instance contain choice_set, i know it for accessing related objects. Why is this not valid c = Choice.objects.get(id=1) c.question_set.all() -
Django MySQL Add ForeignKey
In Django when I change a column, regardless of its datatype, to a foreignkey reference, it doesn't seem to be working. As an example: models.py From: company_id = models.IntegerField(db_column='Company ID', blank=True, null=True) models.py To: company_id = models.ForeignKey('TblCompanies', db_column='Company ID', null=True, on_delete=models.CASCADE) When I makemigrations & migrate it shows as "OK"; no issues. But when I check the DB, it doesn't show these changes as foreign keys, but instead indexes. I'm not an SQL server guru by any means, but generally when I start apps from scratch, this is not a problem. I would see foreign keys right away. In the KEY_COLUMN_USAGE table I would see the associated reference_table _schema, table_name and column_name. In this case I am converting a web site and am bringing over the data from the old SQL server. In the above mentioned table and columns, they show as NULL. What am I missing here? Here is the output of CREATE TABLE tbl Communications which uses two foreign keys. The reference tables SHOULD be tbl Customers.Company ID and tbl Contacts.Contact ID tbl Communications | CREATE TABLE `tbl Communications` ( `Comm ID` int(11) NOT NULL AUTO_INCREMENT, `Contact ID` int(11) DEFAULT '0', `Rep` varchar(50) DEFAULT NULL, `DateTime` datetime DEFAULT … -
calling d3.json from non root-URL
I have a problem calling the json function from d3. From a page that renders a model in my Django DB (this website is called from this url '/app/function/pk') I call in JS d3.json in order to use some data using "app/dootherthing/pk" as url. E.g: d3.json("/app/dootherthing/pk"). however, the console in developer mode gives me an error since d3.json tries to get the json object from http://127.0.0.1:8000/app/function/pk/app/dootherthing/pk instead of http://127.0.0.1:8000/app/dootherthing/pk that is where this json object is. How can I redirect the d3.json to the root url "127.0.0.1:8000/" and not use the current url as root anymore? I dont know if I am explained myself but I guess it should be something stupid and easy, thanks for your answers! Best. -
Is applying temporary attributes to django models for use in views bad practice?
I'm working with a system for tracking employee hours, and need to calculate the year end information for those employees. In a separate section of our app, the previous dev iterated through the employees/employee tasks that have hours and created dictionaries to pass to the view for display. I've found that I can substantially cut the amount of code necessary to do this by applying 'temporary attributes' to the employees as I iterate through them then simply passing the employee queryset into the view, but is this bad practice and is there a better way to go about solving this problem? Heres a snippet for an example: models.py class Employee(models.Model): user = models.ForeignKey(User) class EmployeeTask(models.Model): emp = models.ForeignKey(Employee) hours = models.FloatField() views.py def employee_report(request): employees = Employees.objects.all() tasks = EmployeeTask.objects.all() for employee in employees: emp_tasks = [task for task in tasks if task.employee == employee] employee.total_hours = 0 for emp_task in emp_tasks: employee.total_hours += emp_task.hours context = { 'employees': employees } return render(request, 'app/employees.html', context) I tried placing the login in the model by creating a method that retrieve the employee tasks and did the calculation but that resulted in a massive amount of sql queries. Getting and matching the … -
Combining ChartJS options with Chartkick in Django
With Chartkick I can the height of a ChartJS chart with: {% column_chart price_data with with height='500px' %} and the padding with {% column_chart price_data with library={"layout":{"padding":{"bottom":20}}} %} How do I combine multiple options? Any help will be much appriciated. -
Django with Angular 2 - Model design
i'm trying to integrate Django (backend) and Angular 2 (frontend) frameworks, via a RESTful API based on Django Rest Framework. I'm still practicing with them and i have some design and architectural doubts i want to solve. This is what i understood: Angular 2 is a component-based framework. Components should focus on displaying data and shouldn't contain application logic. Every component has properties (which are bound to templates) and methods. Some methods populate the properties calling services. Services forward HTTP requests to the RESTful API endpoints. The RESTful API gets and serialize data which will then returned to services. My questions are: 1- The application model is backend-side (Django). In my domain, i have a "House" which is related to multiple "Person" objects. Backend-side, in "models.py", i have the python class "Person" which has a "myhouse = models.ForeignKey(House)" field (one-to-many relationship). Client-side, for displaying and other purposes, i'd like to have two classes: "Person" and "House". "House" should have a "people: Person[]" property. Is this the correct way to proceed? Or should i duplicate the model i have backend-side, client-side? For example: Having a "User" class in "models.py", should i match this class with a typescript "user.ts" class with the … -
django celery beat is not working when using existing celery instance
I have a huge project and a small web-server for it written in Django. I am using django-celery for PeriodicTasks managment through admin site. There is already configured celery instance in my project and I want this instance to be used by django-celery(I need exactly this instance, because I already use it for other purposes, and I do not want separate instances: one for project and one for django-server) Here is project structute - project - app/ - django_server/ - django_server/ - __init__.py - ... - celery_app.py In project/django_server/__init__.py I import from celery_app import celery_instance. It worked almost perfectly, now all registered project tasks are displayed in PeriodicTask admin form, but django celery beat is not working. When I run python manage.py celery beat, it just logs: celery beat v3.1.23 (Cipater) is starting. __ - ... __ - _ Configuration -> . broker -> amqp://guest:**@localhost:5672// . loader -> celery.loaders.app.AppLoader . scheduler -> celery.beat.PersistentScheduler . db -> celerybeat-schedule . logfile -> [stderr]@%DEBUG . maxinterval -> now (0s) +0000 INFO [celery.beat] beat: Starting... If I comment this import, then celery-beat is working, but there are no celery_instance tasks.
