Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
How can i set query parameter dynamically to request.GET in Django class based views
OVERVIEW A url with query parameters looks like. http://example.api.com/search/?name=jhon&age=26 and on view if i am using django-filter all parameters are automatically extracted from request and it will return a filtered query-set . views.py class SearchView(TemplateView): template_name = "search.html" filter_class = SearchFilter def get_context_data(self, **kwargs): context = super(SearchView, self).get_context_data(**kwargs) return context If i want to extract it manually from request.GET i can do. def get_queryset(self): # extracting query parameter q = self.request.GET.get('name') PROBLEM STATEMENT So my question is what if my url looks like http://example.api.com/search/jhon-26 I am doing this because i don't want to reveal keys like 'name' and 'age' to public, this is for security purpose. I am getting jhon-26 in **kwargs, I want to split it and set as query parameter to request.GET so that my filter class will work fine Is there anything to set attribute to request.GET # may be any set function available for this self.request.GET.set('name', 'jhon') How can i achieve this. -
Bootstrap form not working with django ( POST)
I'm building a simple login mechanism while learning django and I have the view code right - it works with a simple form, but it doesn't work with this ( practically copy-pasted from the documentation ) Bootstrap and I can't figure out why. Specifically both ( for username and password) request.POST.get's return none. <!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <meta http-equiv = "X-UA-Compatible" contant = "IE = edge"> <meta name = 'viewport' content = 'width = device-width, initial-scale = 1'> <title>title</title> <meta name="description" content="Minimal Blogging Platform"> <meta name="author" content="MiBlo"> {% load staticfiles %} <link rel="stylesheet" href="{% static '/css/bootstrap.min.css' %}"> <script src="{% static '/js/bootstrap.min.js' %}"></script> </head> <body> <h1 align='middle' style="margin-bottom: 80px; margin-top: 80px;">Blogging made (very) minimal</h1> <form action = "/blog/login/" class="form-horizontal col-sm-offset-3" method='post'>{% csrf_token %} <div class="form-group"> <label for="username" class="col-sm-2 control-label">Username</label> <div class="col-sm-3"> <input type="text" class="form-control" id="username" placeholder="Username" value="chuj"> </div> </div> <div class="form-group"> <label for="password" class="col-sm-2 control-label">Password</label> <div class="col-sm-3"> <input type="password" class="form-control" id="password" placeholder="Password"> </div> </div> <div class="form-group"> <div class="col-sm-offset-2 col-sm-4"> <div class="checkbox"> <label> <input type="checkbox"> Remember me </label> </div> </div> </div> <div class="form-group"> <div class="col-sm-offset-2 col-sm-4"> <input type="submit" value="OK"> </div> </div> <div class="form-group"> <div class="col-sm-offset-2 col-sm-4"> <a href="#">Register</a> </div> </div> </form> </body> </html> -
How to convert string date 11:01 PM - 11 Aug 2009 dateTime object - Python 3
Im trying to convert a string date "11:01 PM - 11 Aug 2009" into a datetime object in python. My code is: from datetime import datetime datetime.strptime('11:01 PM - 11 Aug 2009', '%I:%M %p - %d %B %Y') but I am getting an error: time data '11:01 PM - 11 Aug 2009' does not match format '%I:%M %p - %d %B %Y' Not sure what I've missed. -
Django form saving in database and send email with formspree.io
I'm looking for set up a ticket service for my company and I am getting a little problem between my Django form and Formspree.io. Users create a ticket with my Django form. Then, their tickets are storaged in my database and my application send an email to support@mycompany.fr with Formspree. When I click on submit button, the mail is well send to my email address but data are not saved in my database. Up to now I don't find a way to do this process ... My models looks like : class Ticket(models.Model): Etat = models.CharField(max_length = 30, choices = CHOIX_ETAT_TICKET, verbose_name="Etat du ticket") Nom = models.CharField(max_length=40, verbose_name="Nom du createur") Prenom = models.CharField(max_length=40, verbose_name="Prenom du createur") Telephone = models.CharField(max_length=20, verbose_name='Téléphone') Mail = models.CharField(max_length=40, verbose_name='Email') Objet = models.CharField(max_length=60, verbose_name='Objet du ticket') Description = models.CharField(max_length=250, verbose_name='Description du ticket') Creation = models.DateTimeField(auto_now_add=True) Utilisateur = models.CharField(max_length=100, verbose_name="Utilisateur") My form looks like this : class TicketFormulaire(forms.ModelForm): Utilisateur = forms.CharField(widget=forms.HiddenInput()) Etat = forms.CharField(widget=forms.HiddenInput()) class Meta : model = Ticket fields = ["Etat", "Nom", "Prenom", "Telephone", "Mail", "Objet", "Description", "Utilisateur"] My view looks like this : @login_required def Identity_Ticket(request) : if request.method == 'POST': form = TicketFormulaire(request.POST or None) if form.is_valid() : # Vérification sur la validité … -
rasing Form errors
I am creating a function in the form class to check for a condition and raise an error if it find it. i can't check it in the clean because i have to calculate the amount first. if i use forms.ValidationError' it gives me a django error not an error in the{{form.non_field_errors}} .. any ideas gow to do this .. here is my approach forms.py: class InvoiceForm(forms.ModelForm): class Meta: model = Invoices exclude = ['amount', 'remaining', 'products', 'status'] def valid_amount(self,invoice, amount): customer = invoice.customer if amount > customer.credit_limit: msg = 'the amount is above credit limit' self.add_error('customer', forms.ValidationError(msg)) return True -
Django conditional annotation: Queryset return empty if `When` condition return empty
I have two model Flat and FlatDebit class Flat(models.Model): number = models.IntegerField() class FlatDebit(models.Model): flat = models.ForeingKey('foo.Flat', related_name='debits') debit_type = models.ForeingKey('foo.DebitType') unpaid_amount = models.DecimalField() I want to get sum of unpaid_amount of some debits of each flat by debit_type. Following annotation works as expected if there is debit types which has id 1 and 2. flats = Flat.objects.all() flats.annotate( amount=Sum(Case( # If there is no debit with specified debit_type # `flats` queryset return empty When(debits__debit_type_id__in=[1,2], then='debits__unpaid_amount' ), output_field=models.DecimalField(), default=Decimal('0.00') )), ) But if there is no debits which specified debit_type, queryset returns empty. print(flats) #<QuerySet []> -
<XXX> needs to have a value for field xxx before this many-to-many relationship can be used
I currently have the following 3 models class modelToolName(models.Model): tool_name = models.CharField(max_length=250,unique=True) class modelBodyPart(models.Model): part_name = models.CharField(max_length=128,unique=True) class modelNormalBodyPartResult(models.Model): body_part = models.ForeignKey(modelBodyPart, default=None) tool_name = models.ManyToManyField(modelToolName, default=None, blank=True) result = models.TextField(blank=True, null=True) Now I am attempting to insert value into the modelNormalBodyPartResult in this way result="xxxx" bodpart = modelBodyPart.objects.get(part_name="xxx") #--->returns object fine toolqset = modelToolName.objects.get(tool_name="xxx")#--->returns object fine modelNormalBodyPartResult.objects.create(body_part=bodpart,tool_name = toolqset,result=result) --->error and I get the error <modelNormalBodyPartResult: modelNormalBodyPartResult object> needs to have a value for field "modelnormalbodypartresult" before this many-to-many relationship can be used. I looked at this post but still could not figure out the issue any suggestions in this regard would be appreciated. -
Django CMS Group/Nest Placeholders
I'm trying to group my placeholders in Django CMS so that they don't appear in one long list. I want some sort of indication that some placeholders are linked together other than just their name. Currently I want the user to be able to choose their own Header Image, Title and Subtitle so my placeholder tree looks like this: -- Header Image -- Header Title -- Header Subtitle I want to be able to group these together in my template to show a correlation like so: -- Header ---- Header Image ---- Header Title ---- Header Subtitle I found a post from 2013 that said this was possible but the plugin that was suggested that would give this functionality no longer exists. -
Django JsonResponse return no data
I'm subclassing Django UpdateView to update my model using AJAX form (based on Django 1.11 and jQuery-3.2.1). I want to change the UpdateView to return JSON data (instead of HttpResponseRedirect() or rendering form again with render_to_response()) Here's my UpdateView's subclass looks like: class MediaSetUpdateView(UpdateView): def form_valid(self, form): self.object = form.save() print("Data is saved", file=sys.stderr) return JsonResponse({'message' : 'Data is saved'}, status=204) def form_invalid(self, form): print("form invalid", file=sys.stderr) return JsonResponse(form.errors.as_json(), status=422) Here's the URL setup for my UpdateView: urlpatterns = [ ..., url(r'^mediaset/(?P<pk>[0-9]+)/$', views.MediaSetUpdateView.as_view(), name='mediaset_update'), ..., ] And here's my jQuery AJAX POST: /* The global var to store current data */ var json_storage = {"template_id":1 }; $.ajax({ url: '/esignage/mediaset/6/', type: 'post', // Performing a POST request data : json_storage, dataType: 'json', success: function(data) { console.log(data); }, error: function(data) { console.log(data); } }); The django UpdateView returns success ('Data is saved' is printed in my django console), but there's no message that is returned (console.log(data) in AJAX success() return 'undefined'). Further investigation from my browser indicates that Django didn't return any message, other than HTTP success status. Here's capture from my chrome's debugger: HTTP/1.0 204 No Content Date: Wed, 19 Jul 2017 09:28:09 GMT Server: WSGIServer/0.2 CPython/3.5.2 X-Frame-Options: SAMEORIGIN Content-Type: application/json … -
Django, get user profile using library User without call models
I try to show user profile base on PK. So, i create menu with the name is "Profile". After user click the menu, user can see the profile. i was try it: def view_profile(request, pk): profile = get_object_or_404(User, pk=pk) return render(request, 'girl/base.html', {'profile':profile}) And i was try it too : def view_profile(request, pk): profile = User.objects.filter(User, pk=pk) return render(request, 'base.html', {'profile':profile}) On the base.html {% if user.is_authenticated %} <a href='{% url "profile" pk=profile.pk%}' class="prof"><span class="pro">Profile</span></a> {% endif %} And the template, profile.html : {% extends 'base.html' %} {% block content %} {% if user.is_authenticated %} <h2><a href="{% url 'profile' pk=profile.pk %}">{{ profile.username }}</a></h2> {% endif %} {% endblock %} urls.py url(r'^cat/(?P<pk>\d+)/profile/$', views.view_profile, name='profile') The profile never show, i got the error and the error is : Reverse for 'profile' with keyword arguments '{'pk': ''}' not found. 1 pattern(s) tried: ['cat/(?P<pk>\\d+)/profile/$'] -
How to send a body in Swagger Django documentation?
I am trying to use the Swagger documentation in my Django project. However, I am not able to send the body. Furthermore, the List Operations doesn't work. Does Anyone has an idea how to fix it? I only have the default settings: urls.py: from rest_framework_swagger.views import get_swagger_view schema_view = get_swagger_view(title='Title API') urlpatterns = [ url(r'^swagger', schema_view), ] I can only list all of my URLs and log in as admin. Logout doesn't work. At this moment when I choose Expand Operations, I see something like this: -
Upon scrolling to the top of the div, ajax will be loaded
I am currently working on a django chat application which involves using ajax to load the list of chat messages and appending them to one of the divs in my template via jquery (e.g. chatmsg_div). Since the chat app will be used by many users, I am required to do pagination for the chat messages which I have been successful in doing so. Now, only the last 20 chat messages in the chatroom will be loaded for the users to see (with the latest message being at the bottom of the div). However, there is another requirement that I need to do which is to load the chat history (e.g. previous 20 messages that is in another page) upon scrolling to the top of the chatmsg_div. My questions will be: I did some research on google, but could not find any jquery function that allows me to trigger an ajax call upon reaching the top of a specific div. I am pretty new to jquery, so please pardon me even if the answer I am looking for is obvious. After loading the previous 20 chat messages, I want the div to remain at the position I last scrolled to instead … -
enable a disabled html button based in django template
I want to enable below button based on the boolean value from context i.e. {{ enable }}. if enable is True then make it enable otherwise disabled. <input type="submit" name="upload" class="btn btn-primary" value="Upload" disabled/> -
Working with List of numbers in Django
I have a loop of buttons, ranging from 1-100, if a user selects any of the button it populates an input field e.g 32, 47, 84, 69. So i have a django view that saves it to the database but it is saved in the database in this format [u'32', u'47', u'84', u'69'] which i know will be regarded as one entity, but i need the numbers to be regarded as separate entity, so that any number that is in the database, wont be selected by any other user. def game_details(request, id): template_name = 'app_agent_games/game_details.html' get_game_id = get_object_or_404(gamesModel, id=id) context = { 'game_details': get_game_id, 'range': range(1, get_game_id.people + 1) } if request.POST: guesses = request.POST.get('guessed_number', False) splited = guesses.split(',') counter = request.POST.get('count_guess', False) update_game = PlayGameForm(request.POST) obj = PlayGame() obj.guessed_number = splited obj.total_stake = int(get_game_id.amount) * int(counter) obj.game_id = get_object_or_404(gamesModel, id=id) obj.agent_id = get_object_or_404(CustomAgent, user_id=request.user.user_id) obj.save() return HttpResponseRedirect('/games/details/'+id) return render(request, template_name, context) The model structure: class PlayGame(models.Model): agent_id = models.ForeignKey(CustomAgent, related_name='agent_game') game_id = models.ForeignKey(gamesModel, related_name='agent_game_id') guessed_number = models.CharField(max_length=100) total_stake = models.IntegerField(default=0) The model field that saves the list is guessed_number -
manyToMany field not working both ways in Django
My model: class Painting(models.Model): objectNumber = models.CharField(max_length=128) class Hit(models.Model): assignmentId = models.CharField(max_length=512) Painting.add_to_class('hit', models.ManyToManyField(Hit,blank=True,related_name='hit')) I fetch one certain hit: hit = Hit.objects.get(assignmentId='3HRMW88U17T14EOXNK8S42QES890M5') Now, this hit has a m2m relationship to Painting. I fetch the first painting in hit. p = hit.painting.all()[0] Now, this p should also contain the hit, such that p.hit.all() would return the 3HRMW... hit. Instead: p.hit.all() >>>[] Why am I unable to access the hits from the painting objects? -
jquery.jscroll and removing div from main container
I'm beginning my journey with jquery (thanks to django). I'm using jquery.jscroll to lazy load next pages which work (now) flawlessly (thanks to adding doctyp html), and when I scroll down the page it work fine, the problem is when I don't scroll at all but use button to remove content I watch from page. I tried a little hack with moving window by 1px which work but after few clicks move menu out of view - I would like to change that to something that works. My code is: html: <div class="scroll" id="scroll"> {% if is_paginated %} {% if page_obj.has_next %} <center><a href="?page={{ page_obj.next_page_number }}" class="last">more</a></center> {% endif %} {% endif %} </div> <script> $('.scroll').jscroll({ autoTrigger: true, nextSelector: 'a:last', loadingHtml: '<center><img src="../../static/loading.gif" alt="Loading" /> Loading...</center>' }); </script> and my js file looks like this: function hidePost(my_id) { $.ajax({ url: "/post/hide/" + my_id, type: "POST", success:function(msg) { $('#' + my_id).remove(); $('.lazy').Lazy(); window.scrollTo(window.scrollX, window.scrollY + 1); return false; }, error:function(jqXHR, textStatus) { alert('Error Occured'); alert(textStatus); alert(jqXHR); } }); } This code work fine but window.scrollTo(window.scrollX, window.scrollY + 1); give me scroll down and if i add window.scrollTo(window.scrollX, window.scrollY - 1); to move back up then it wont trigger $('.scroll').jscroll and I … -
Combine filter results in for loop
I'm using Django to make a website. I want to combine multiple filter results (Querysets). My view.py: wanted_refund = set() for m in staff.members.all(): payment = m.PaymentHistory.filter(division="Membership") for p in payment: try: refund = RefundHistory.objects.filter(payment=p).filter(refund_date__range=[this_month_start, date]) wanted_refund.add(refund) except RefundHistory.DoesNotExist: pass context = { 'wanted_refund' : wanted_refund,} return render(request, 'refund.html', context) But, it doesn't work by using filters. It only works when I use 'get'. print(refund) shows me the result like this: < QuerySet [] > < QuerySet [] > < QuerySet [] > < QuerySet [< RefundHistory: RefundHistory object >] > I want to use only the Querysets that have the object and what I want is the below one in template: {% for refund in wanted_history %} {{ refund.refund_date }} {{ refund.refund_amount}} {% endfor %} How do I pass over the multiple filters results in for loop? -
ImportError: No module named rest_framework_mongoengine
I have installed both DjangoRest framework and Mongo Engine using pip, included rest_framework_mongoengine in INSTALLED_APPS list. Still exception is being thrown when I run server. Traceback File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/utils/autoreload.py", line 227, in wrapper fn(*args, **kwargs) File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run autoreload.raise_last_exception() File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/utils/autoreload.py", line 250, in raise_last_exception six.reraise(*_exception) File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/utils/autoreload.py", line 227, in wrapper fn(*args, **kwargs) File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/__init__.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/Users/anum/Desktop/Python/ConnectBox/env/lib/python2.7/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) ImportError: No module named rest_framework_mongoengine Here is my Settings.py. Using pip list command, following packages were listed down: Django (1.11.2) djangorestframework (3.6.3) mongoadmin (0.2) mongodbforms (0.3) mongoengine (0.13.0) pip (9.0.1) PyJWT (1.5.2) pymongo (3.4.0) pytz (2017.2) setuptools (36.0.1) six (1.10.0) wheel (0.29.0) Please help me to get in right direction.. Thanks. -
Siubmit two forms in a single submit in Django
I have two forms in template. At the moment I have two submit buttons. Would like to combine those to a single submit button. Below code is now updating only one form, that's AnswerForm. How i can update AnswerReplyForm along with that? class AnswerView(ObjectEditView): form_class = forms.AnswerReplyForm answer_form = forms.AnswerForm model = AnswerReply def get(self, request, pk): answer = get_object_or_404(Answer, pk = pk) answer_reply = AnswerReply.objects.filter(answer_id = pk).order_by('-id') self.answer_form = self.answer_form(instance=answer) return render(request, 'helpdesk/answer.html', { 'answer': answer, "answer_reply" : answer_reply, 'obj_type': 'answer reply', 'form': self.form_class, "form2":self.answer_form, "pre_reply_from" : self.predefined_reply_form }) def post(self, request, pk, *args, **kwargs): answer = get_object_or_404(Answer, id=pk) answer_reply = AnswerReply.objects.filter(answer_id = pk).order_by('-id') self.answer_form = self.answer_form(instance=answer) obj = self.model() obj = self.alter_obj(obj, request, args, kwargs) form = self.form_class(request.POST, request.FILES, instance=obj) if form.is_valid(): form.instance.answer_id = pk obj_created = not form.instance.pk obj = form.save() return render(request, 'helpdesk/answer.html', { 'answer': answer, "answer_reply" : answer_reply, 'obj_type': 'answer reply', 'form': self.form_class, "form2":self.answer_form, }) -
Django rest framework parallel execution of many=True field data
I have a serializer, in that I've overridden the validate method. class ProcessItemSerializer(serializers.Serializer): field1 = serializers.CharField(max_length=63) # Few more fields def validate(self, data): # few data processing and HTTP calls return data I have another serializer which use above serializer as many=True field class DataSerializer(serializers.Serializer): items = ProcessItemSerializer(many=True) If I pass a list of item data to DataSerializer it will process one by one each item. thats desirable! but if length of items is more than 100 it takes much time. what I want is to execute set of 20-20 items parallel using python-multiprocess so that I can reduce the overall time. how can I do this in DRF. what method do I have to override? -
Django Wagtail CMS Server Error Explorer Menu
I updated my Django Wagtail CMS release to Wagtail 1.11.1. Since then, when in the admin area the 'Explorer Menu' is now called pages and the hover affect is gone (not important right now though). I click on the 'Pages' menu item and in the pullout menu there's what I assume should be the latest pages modified, but it says server error. There's also a 'See all 0 pages' link at the bottom of the pullout that produces a 404. I did modify the home page and re-save but the server error is still there. Below is a screen shot. Any thoughts? Thank you. -
display data from mongodb database in django
I am using mongodb(mongoengine) and django.the first three field display data from sql and other one field from mongodb.and it displaying error for mongodb field in table. Does anybody please know, what is wrong with view file below? view.py for mongodb feild - class OrderListJson1(BaseDatatableView): model = Location columns = ['battery_status'] order_columns = ['battery_status'] max_display_length = 100 def __init__(self): pass def render_column(self, row, column): # We want to render user as a custom column if column == 'user': return '{0} {1}'.format(row.battery_status) else: return super(OrderListJson, self).render_column(row, column) # def get_initial_queryset(self): return Location.objects.all().order_by('battery_status') def filter_queryset(self, qs): search = self.request.GET.get(u'search[value]', None) if search: qs=qs.filter(battery_status__istartswith = search) return qs def prepare_results(self, qs): json_data = [] for item in qs: json_data.append([ item.battery_status ]) return json_data -
Ubuntu 16.04/Django - gunicorn - Worker failed to boot
I'm deploying Django project on Digital Ocean Ubuntu 16.04 VPS. I used one-click-install of Django and then replaced with my project. The problem is that server returns 502 Error. Do you know where is the problem? gunicorn.service [Unit] Description=Gunicorn daemon for Realestate Scanner Before=nginx.service After=network.target [Service] WorkingDirectory=/home/django/realestate_scanner ExecStart=/usr/bin/gunicorn --name=realestate_scanner --pythonpath=/home/django/realestate_scanner --bind unix:/home/django/gunicorn.socket --config /etc/gunicorn.d/gunicorn.py realestate_scanner.wsgi:application Restart=always SyslogIdentifier=gunicorn User=django Group=django [Install] WantedBy=multi-user.target LOGS: > sudo journalctl -u gunicorn returns this: ...skipping... Jul 19 06:45:08 django-512mb-ams2-01-beta gunicorn[20681]: self.reap_workers() Jul 19 06:45:08 django-512mb-ams2-01-beta gunicorn[20681]: File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 464, in reap_workers Jul 19 06:45:08 django-512mb-ams2-01-beta gunicorn[20681]: raise HaltServer(reason, self.WORKER_BOOT_ERROR) Jul 19 06:45:08 django-512mb-ams2-01-beta gunicorn[20681]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3> Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: gunicorn.service: Main process exited, code=exited, status=1/FAILURE Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: gunicorn.service: Unit entered failed state. Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: gunicorn.service: Failed with result 'exit-code'. Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: gunicorn.service: Service hold-off time over, scheduling restart. Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: Stopped Gunicorn daemon for Realestate Scanner. Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: gunicorn.service: Start request repeated too quickly. Jul 19 06:45:08 django-512mb-ams2-01-beta systemd[1]: Failed to start Gunicorn daemon for Realestate Scanner. ...skipping... Jul 19 06:45:08 django-512mb-ams2-01-beta gunicorn[20681]: self.reap_workers() Jul 19 06:45:08 django-512mb-ams2-01-beta gunicorn[20681]: File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line … -
How to run pyspark with Django?
I'm developing a web application retrieving data from data lake, the data is stored in HDFS and I want to use pyspark to perform some analysis. In other words we have a script within ipython notebook and we want to use it with Django. I see that pyspark is also available at pypi, so I installed it with pip and the same script is imported as .py file from notebook is running fine, when I run it as python myscript.py it works fine. Hence, it should also work fine if I import that script within Django. So, is it the correct method, or I will have to run spark-submit myscript.py? I want to use Spark in cluster mode. -
[FreeTDS][SQL Server]Write to the server failed (20006) (SQLSetConnectAttr)')
I have a Django app with a worker process which does database transactions. Under the hood, I have an Azure SQL database attached to my application. The worker code looks something like this: class Command(BaseCommand): def handle(self, *args, **kwargs): os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings') django.setup() consumer = Consumer() logging.info('Booting drive consumer') while True: messages = consumer.poll() for message in messages: ... DriveEvent.objects.create(event_id=response['Id'], drive_id=drive_payload['drive_id']) ... After about 20 mins, the create db object starts failing with the following error: [drive-consumer-1]2017-07-19T03:19:08.545128400Z Traceback (most recent call last): [drive-consumer-1]2017-07-19T03:19:08.545133000Z File "/code/miles/management/commands/drive_consumer_worker.py", line 28, in handle [drive-consumer-1]2017-07-19T03:19:08.545137900Z drive_consumer.consume(json.loads(record.value)) [drive-consumer-1]2017-07-19T03:19:08.545142500Z File "/code/miles/workers/drive_consumer.py", line 35, in consume [drive-consumer-1]2017-07-19T03:19:08.545152200Z self._on_create(message['calendar_id'], drive_payload, message['access_token']) [drive-consumer-1]2017-07-19T03:19:08.545156900Z File "/code/miles/workers/drive_consumer.py", line 46, in _on_create [drive-consumer-1]2017-07-19T03:19:08.545161500Z DriveEvent.objects.create(event_id=response['Id'], drive_id=drive_payload['drive_id']) [drive-consumer-1]2017-07-19T03:19:08.545166500Z File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method [drive-consumer-1]2017-07-19T03:19:08.545171300Z return getattr(self.get_queryset(), name)(*args, **kwargs) [drive-consumer-1]2017-07-19T03:19:08.545175900Z File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 399, in create [drive-consumer-1]2017-07-19T03:19:08.545180700Z obj.save(force_insert=True, using=self.db) [drive-consumer-1]2017-07-19T03:19:08.545185400Z File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 796, in save [drive-consumer-1]2017-07-19T03:19:08.545190200Z force_update=force_update, update_fields=update_fields) [drive-consumer-1]2017-07-19T03:19:08.545194800Z File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 821, in save_base [drive-consumer-1]2017-07-19T03:19:08.545211400Z with transaction.atomic(using=using, savepoint=False): [drive-consumer-1]2017-07-19T03:19:08.545215800Z File "/usr/local/lib/python2.7/site-packages/django/db/transaction.py", line 184, in __enter__ [drive-consumer-1]2017-07-19T03:19:08.545220400Z connection.set_autocommit(False, force_begin_transaction_with_broken_autocommit=True) [drive-consumer-1]2017-07-19T03:19:08.545224900Z File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 391, in set_autocommit [drive-consumer-1]2017-07-19T03:19:08.545229600Z self._set_autocommit(autocommit) [drive-consumer-1]2017-07-19T03:19:08.545234000Z File "/usr/local/lib/python2.7/site-packages/sql_server/pyodbc/base.py", line 453, in _set_autocommit [drive-consumer-1]2017-07-19T03:19:08.545238600Z self.connection.autocommit = autocommit [drive-consumer-1]2017-07-19T03:19:08.545243100Z File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__ [drive-consumer-1]2017-07-19T03:19:08.545247700Z six.reraise(dj_exc_type, dj_exc_value, traceback) [drive-consumer-1]2017-07-19T03:19:08.545252200Z File "/usr/local/lib/python2.7/site-packages/sql_server/pyodbc/base.py", line 453, …
