Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
django-html-validator - now locally, fast!
A couple of years ago I released a project called django-html-validator (GitHub link) and it's basically a Django library that takes the HTML generated inside Django and sends it in for HTML validation. The first option is to send the HTML payload, over HTTPS, to https://validator.nu/. Not only is this slow but it also means sending potentially revealing HTML. Ideally you don't have any passwords in your HTML and if you're doing HTML validation you're probably testing against some test data. But... it sucked. The other alternative was to download a vnu.jar file from the github.com/validator/validator project and executing it in a subprocess with java -jar vnu.jar /tmp/file.html. Problem with this is that it's really slow because java programs take such a long time to boot up. But then, at the beginning of the year some contributors breathed fresh life into the project. Python 3 support and best of all; the ability to start the vnu.jar as a local server on http://localhost:8888 and HTTP post HTML over to that. Now you don't have to pay the high cost of booting up a java program and you don't have to rely on a remote HTTP call. Now it becomes possible to … -
Package of the Week: django-import-export
As the name suggests, this is a library to handle importing and exporting data. The django-import-export library supports multiple formats, including xls, csv, json, yaml, and all other formats supported by tablib. It also have a Django admin integration, which is really convenient to use. Installation Pip is the way to go: pip install django-import-export Update your settings.py: INSTALLED_APPS = ( ... 'import_export', ) There is also an optional configuration that I usually add: IMPORT_EXPORT_USE_TRANSACTIONS = True The default value is False. It determines if the library will use database transactions on data import, just to be on the safe side. Resources The django-import-export library work with the concept of Resource, which is class definition very similar to how Django handle model forms and admin classes. In the documentation the authors suggest to put the code related to the resources inside the admin.py file. But if the implementation is not related to the Django admin, I usually prefer to create a new module named resources.py inside the app folder. models.py from django.db import models class Person(models.Model): name = models.CharField(max_length=30) email = models.EmailField(blank=True) birth_date = models.DateField() location = models.CharField(max_length=100, blank=True) resources.py from import_export import resources from .models import Person class PersonResource(resources.ModelResource): … -
Exploring Django Utils #1
Exploring the Django source code I ended up discovering some really nice utility functions that I wasn’t aware of. I thought about sharing with you guys in a form of reference-like article. There are great stuff there, so I decided to break this post into a few parts. Crypto Module: django.utils.crypto get_random_string Calling it without any parameters defaults the length to 12. from django.utils.crypto import get_random_string get_random_string() '5QxAxqyhsyJM' Your may pass the number of characteres you want: get_random_string(50) 'lrWYnyxhnXpwmjHDzmdgTFaIi1j73cKD5fPDOPwuVBmmKxITYF' And also the collection of characteres the random string will have: get_random_string(12, '0123456789') '805379737758' Date Module: django.utils.dates Basically it is a collection of commonly-used date structures. WEEKDAYS from django.utils.dates import WEEKDAYS WEEKDAYS = { 0: _('Monday'), 1: _('Tuesday'), 2: _('Wednesday'), 3: _('Thursday'), 4: _('Friday'), 5: _('Saturday'), 6: _('Sunday') } WEEKDAYS_ABBR from django.utils.dates import WEEKDAYS_ABBR WEEKDAYS_ABBR = { 0: _('Mon'), 1: _('Tue'), 2: _('Wed'), 3: _('Thu'), 4: _('Fri'), 5: _('Sat'), 6: _('Sun') } MONTHS from django.utils.dates import MONTHS MONTHS = { 1: _('January'), 2: _('February'), 3: _('March'), 4: _('April'), 5: _('May'), 6: _('June'), 7: _('July'), 8: _('August'), 9: _('September'), 10: _('October'), 11: _('November'), 12: _('December') } DateFormat Module: django.utils.dateformat The implementation of PHP date() style date formatting, which is used in the … -
Dritte Hamburger Python Unconference
Vom 09. bis 11. September 2016 findet die 3. Python Unconference Hamburg an der Technischen Universität Hamburg-Harburg (TUHH) statt. Erwartet werden über 100 Python-User bzw. Developer aus Hamburg und dem Rest der Welt. Von "Greenhorns" bis zu "proven Experts" aus den Bereichen Mathematik, Data Science, System-Administration und DevOps bis hin zu Web-Development und Python-Core-Entwicklung werden alle nur erdenklichen Facetten der Python-Welt vertreten sein. Wie in den letzten Jahren findet die Unconference am Samstag und Sonntag im Barcamp-Stil statt, mit Vorträgen und Diskussionen aus allen Bereichen der Python Welt. Am Freitag gibt es wieder Raum für Sprints, Tutorials und Workshops. Das aktuelle Freitags-Programm findet Ihr hier . Tickets gibt es hier. Mehr Informationen gibt es unter www.pyunconf.de. -
How to Deploy Django Applications on Heroku
Heroku is a cloud application platform, basically a Platform-as-a-Service (PaaS) platform. They support several programming languages, including Python. It is very easy to deploy Django applications on Heroku. They also offer a free plan, which is quite limited, but it is great to get started and to host demos of Django applications. Install Heroku Toolbelt Actually, first thing – sign up Heroku. Then install the Heroku Toolbelt. It is a command line tool to manage your Heroku apps. After installing the Heroku Toolbelt, open a terminal and login to your account: $ heroku login Enter your Heroku credentials. Email: vitor@simpleisbetterthancomplex.com Password (typing will be hidden): Authentication successful. Preparing the Application In this tutorial I will deploy an existing project, Bootcamp. It’s open-souce Django project I’ve developed a couple of years ago, and it’s also available on GitHub, so you actually can clone the repository and try it by yourself. Basically things will work better if you are already using Git. The Heroku deployment process is done through Git. Your application will be stored in a remote Git repository in the Heroku Cloud. Anyway, here is the list of things you will probably need to add to your project: Add a … -
How to Export to PDF
There are a few ways to export data to a PDF file using Django. All of them requires a third-party library so to generate the file itself. First I will show how to return a PDF response, which can also be used if you are just serving an existing PDF file. Then I will show how to use ReportLab and WeasyPrint. Writing a PDF to Response In the example below I’m using the Django’s FileSystemStorage class. It will also work if you simply use open() instead. The FileSystemStorage sets the base_url to the project’s MEDIA_ROOT. from django.core.files.storage import FileSystemStorage from django.http import HttpResponse, HttpResponseNotFound def pdf_view(request): fs = FileSystemStorage() filename = 'mypdf.pdf' if fs.exists(filename): with fs.open(filename) as pdf: response = HttpResponse(pdf, content_type='application/pdf') response['Content-Disposition'] = 'attachment; filename="mypdf.pdf"' return response else: return HttpResponseNotFound('The requested pdf was not found in our server.') This way the user will be prompted with the browser’s open/save file. If you want to display the PDF in the browser you can change the Content-Disposition to: response['Content-Disposition'] = 'inline; filename="mypdf.pdf"' Using ReportLab Installation pip install reportlab It relies on Pillow, which is a third-party Python Image Library. Sometimes it is a pain to get it installed. If you are … -
Setting up a Honey Pot with django-admin-honeypot
Security is something we often ignore until it is too late. However, there are some things you can do right now that are easy to increase your security. Using django-admin-honeypot is one of those things you can do. It is super easy and provides you with the means of tracking who is trying to access your site.Watch Now... -
Package of the Week: Flake8
Flake8 is a Python library that wraps PyFlakes, pycodestyle and Ned Batchelder’s McCabe script. It is a great toolkit for checking your code base against coding style (PEP8), programming errors (like “library imported but unused” and “Undefined name”) and to check cyclomatic complexity. If you are not familiar with the term cyclomatic complexity, it is a software metric created by Thomas J. McCabe to measure the number of independent paths through the source code. Generally speaking, the higher number of ifs inside a function, the higher number of paths it will have, thus a higher cyclomatic complexity. Of course there are other control flow operations that impact the calculus of the cyclomatic complexity. It is also referred as McCabe complexity. Speaking about coding style, I also like to follow the Django Coding Style on the Django projects I develop. This style standard is only enforced when writing code for inclusion in Django. Installation Easiest way to get started is installing it using pip: pip install flake8 Usage Inside the Django project dir, run the command: flake8 Or you can pass the path to a file/dir: flake8 bootcamp/feeds/ The output will be something like that: ./bootcamp/settings.py:96:80: E501 line too long (91 … -
Django Tips #9 How to Create a Change Password View
This is a very quick tip about how to create a change password view using the built-in PasswordChangeForm. For that matter, using a function-based view is easier, because the PasswordChangeForm is slightly different. It does not inherit from ModelForm and it takes an user argument in its constructor. The example below is a functional code to change the password for the authenticated user. views.py from django.contrib import messages from django.contrib.auth import update_session_auth_hash from django.contrib.auth.forms import PasswordChangeForm from django.shortcuts import render, redirect def change_password(request): if request.method == 'POST': form = PasswordChangeForm(request.user, request.POST) if form.is_valid(): user = form.save() update_session_auth_hash(request, user) # Important! messages.success(request, 'Your password was successfully updated!') return redirect('accounts:change_password') else: messages.error(request, 'Please correct the error below.') else: form = PasswordChangeForm(request.user) return render(request, 'accounts/change_password.html', { 'form': form }) The messages.success() and messages.error() are optional, but it is a good idea to keep your user aware about what is going on in the application. Now an important bit is to call update_session_auth_hash() after you save the form. Otherwise the user’s auth session will be invalidated and she/he will have to log in again. urls.py from django.conf.urls import url from myproject.accounts import views urlpatterns = [ url(r'^password/$', views.change_password, name='change_password'), ] change_password.html <form method="post"> {% … -
Serving a Customized Site with Moya
This SO post prompted me to think about how you would go about customizing a entire Django site (not just an app). Particularly if it is a third party project, still in development. Forking the site repository is a solution, but the more customizations you make, the more awkward it becomes to merge upstream fixes and features. The problem is compounded if you have multiple deploys, with custom templates, assets, and settings. I wanted to do a quick write-up of how this would be done in Moya, where customization is more of a first-class concept. I'll briefly go over how you would serve a Moya site with a custom template, without modifying any of the original files. The project I'll be customizing is Moya Techblog which power this blog. Install Moya If you want to follow along, then first install Moya + dependencies and Techblog with the following: pip install moya requests_oauthlib -U git clone https://github.com/moyaproject/moya-techblog.git Make a custom project Next we want to create a new directory along side the Techblog checkout, which will contain two files; moya and settings.ini. The former is an empty file that tells Moya this is a project directory and the latter contains the … -
How to Paginate with Django
As part of the Django’s common Web application tools, Django offers a few classes to manage paginated data. You can pass either a list/tuple of objects or an QuerySet. In this tutorial I will show how to paginate data using function based views and how to paginate using class-based views (ListView). The Paginator The paginator classes lives in django.core.paginator. We will be working mostly with the Paginator and Page classes. Consider the auth.User table has 53 user instances. from django.contrib.auth.models import User from django.core.paginator import Paginator user_list = User.objects.all() paginator = Paginator(user_list, 10) In the example above I’m telling Paginator to paginate the user_list QuerySet in pages of 10. This will create a 6 pages result. The first 5 pages with 10 users each and the last page with 3 users. Debugging the Paginator Object Input Output Type paginator.count 53 <type 'int'> paginator.num_pages 6 <type 'int'> paginator.page_range xrange(1, 7) <type 'xrange'> paginator.page(2) <Page 2 of 6> <class 'django.core.paginator.Page'> The Paginator.page() method will return a given page of the paginated results, which is an instance of Page. This is what we will return to the template. users = paginator.page(2) Debugging the Page Object Input Output Type users <Page 2 of 6> … -
How to Split Views Into Multiple Files
It is a good idea to keep your apps views module small. A common mistake is do much work inside the views functions. For the most part you can create separate modules to do the processing work outside the views and just import it and use the functions inside the views. Another common mistake is a single app implementing many different requirements or serving for many purposes. Sometimes you can split the work into different apps. When it makes sense of course. But sometimes, the views will get big anyway. In this case, you might want to split the views into different files. Sample Scenario Before I show you how, please consider the scenario below: App dir |∙∙core/ |∙∙__init__.py |∙∙admin.py |∙∙migrations/ |∙∙models.py |∙∙tests.py |∙∙urls.py |∙∙views.py views.py from django.shortcuts import render def view_a(request): return render(request, 'view_a.html') def view_b(request): return render(request, 'view_b.html') def view_c(request): return render(request, 'view_c.html') def view_d(request): return render(request, 'view_d.html') urls.py from django.conf.urls import url import .views urlpatterns = [ url(r'^aaa$', views.view_a, name='view_a'), url(r'^bbb$', views.view_b, name='view_b'), url(r'^ccc$', views.view_c, name='view_c'), url(r'^ddd$', views.view_d, name='view_d'), ] Splitting the Views This strategy is good if you are refactoring your code base. This way you won’t need to change the way you handle your urls.py. … -
Please Fix Your Decorators
If your Python decorator unintentionally changes the signatures of my callables or doesn’t work with class methods, it’s broken and should be fixed. Sadly most decorators are broken because the web is full of bad advice. -
How to Upload Files With Django
In this tutorial you will learn the concepts behind Django file upload and how to handle file upload using model forms. In the end of this post you will find the source code of the examples I used so you can try and explore. The Basics of File Upload With Django When files are submitted to the server, the file data ends up placed in request.FILES. It is mandatory for the HTML form to have the attribute enctype="multipart/form-data" set correctly. Otherwise the request.FILES will be empty. The form must be submitted using the POST method. Django have proper model fields to handle uploaded files: FileField and ImageField. The files uploaded to FileField or ImageField are not stored in the database but in the filesystem. FileField and ImageField are created as a string field in the database (usually VARCHAR), containing the reference to the actual file. If you delete a model instance containing FileField or ImageField, Django will not delete the physical file, but only the reference to the file. The request.FILES is a dictionary-like object. Each key in request.FILES is the name from the <input type="file" name="" />. Each value in request.FILES is an UploadedFile instance. I will talk more … -
Django + Elastic Search + haystack = Powering your website with search functionality – part 2
Search is one of the most important and powerful functionality in web applications these days. It helps users to easily find content, products etc on your website. Without search option, users will have to find their way out to reach the desired content which no one likes to do. Just imagine Amazon without search bar, you will have to navigate through various categories to find out the product, and it may take you forever to find it. Having said that, implementing search into your web app is not difficult thanks to lot of libraries built for this purpose. I have broken down this tutorial into two parts. First article explains how to setup elastic search and haystack with django, create search search indexes and how to query data. In this article, we will take one step forward and discuss about using autocomplete, spelling suggestions and creating custom backends. If you haven’t read part-1, I suggest you do it first and then jump to this article. Remember the SearchIndex we created in the part-1 of this tutorial, we are going to continue modifying the same in this article. import datetime from haystack import indexes from myapp.models import Blog class BlogIndex(indexes.SearchIndex, indexes.Indexable): … -
How to Export to Excel
Export data to excel is a common requirement on many web applications. Python makes everything easier. But, nevertheless, it is the kind of task I need to look for references whenever I have to implement. I will give you two options in this tutorial: (1) export data to a .csv file using Python’s csv module; (2) export data to a .xls file using a third-party module named xlwt. For both cases, consider we are exporting django.contrib.auth.models.User data. Export Data to CSV File Easiest way. No need to install dependencies, which is a great thing. Use it if you do not need any fancy formating. views.py import csv from django.http import HttpResponse from django.contrib.auth.models import User def export_users_csv(request): response = HttpResponse(content_type='text/csv') response['Content-Disposition'] = 'attachment; filename="users.csv"' writer = csv.writer(response) writer.writerow(['Username', 'First name', 'Last name', 'Email address']) users = User.objects.all().values_list('username', 'first_name', 'last_name', 'email') for user in users: writer.writerow(user) return response urls.py import views urlpatterns = [ ... url(r'^export/csv/$', views.export_users_csv, name='export_users_csv'), ] template.html <a href="{% url 'export_users_csv' %}">Export all users</a> In the example above I used the values_list in the QuerySet for two reasons: First to query only the fields I needed to export, and second because it will return the data in a … -
How to Export to Excel
Export data to excel is a common requirement on many web applications. Python makes everything easier. But, nevertheless, it is the kind of task I need to look for references whenever I have to implement. I will give you two options in this tutorial: (1) export data to a .csv file using Python’s csv module; (2) export data to a .xls file using a third-party module named xlwt. For both cases, consider we are exporting django.contrib.auth.models.User data. Export Data to CSV File Easiest way. No need to install dependencies, which is a great thing. Use it if you do not need any fancy formating. views.py import csv from django.http import HttpResponse from django.contrib.auth.models import User def export_users_csv(request): response = HttpResponse(content_type='text/csv') response['Content-Disposition'] = 'attachment; filename="users.csv"' writer = csv.writer(response) writer.writerow(['Username', 'First name', 'Last name', 'Email address']) users = User.objects.all().values_list('username', 'first_name', 'last_name', 'email') for user in users: writer.writerow(user) return response urls.py import views urlpatterns = [ ... url(r'^export/csv/$', views.export_users_csv, name='export_users_csv'), ] template.html <a href="{% url 'export_users_csv' %}">Export all users</a> In the example above I used the values_list in the QuerySet for two reasons: First to query only the fields I needed to export, and second because it will return the data in a … -
How to Create Django Signals
The Django Signals is a strategy to allow decoupled applications to get notified when certain events occur. Let’s say you want to invalidate a cached page everytime a given model instance is updated, but there are several places in your code base that this model can be updated. You can do that using signals, hooking some pieces of code to be executed everytime this specific model’s save method is trigged. Another common use case is when you have extended the Custom Django User by using the Profile strategy through a one-to-one relationship. What we usually do is use a “signal dispatcher” to listen for the User’s post_save event to also update the Profile instance as well. I’ve covered this case in another post, which you can read here: How to Extend Django User Model. In this tutorial I will present you the built-in signals and give you some general advices about the best practices. When Should I Use It? Before we move forward, know when you should use it: When many pieces of code may be interested in the same events; When you need to interact with a decoupled application, e.g.: A Django core model; A model defined by a … -
How to Create Django Signals
The Django Signals is a strategy to allow decoupled applications to get notified when certain events occur. Let’s say you want to invalidate a cached page everytime a given model instance is updated, but there are several places in your code base that this model can be updated. You can do that using signals, hooking some pieces of code to be executed everytime this specific model’s save method is trigged. Another common use case is when you have extended the Custom Django User by using the Profile strategy through a one-to-one relationship. What we usually do is use a “signal dispatcher” to listen for the User’s post_save event to also update the Profile instance as well. I’ve covered this case in another post, which you can read here: How to Extend Django User Model. In this tutorial I will present you the built-in signals and give you some general advices about the best practices. When Should I Use It? Before we move forward, know when you should use it: When many pieces of code may be interested in the same events; When you need to interact with a decoupled application, e.g.: A Django core model; A model defined by a … -
How to Return JSON-Encoded Response
Since version 1.7, Django counts with the built-in JsonResponse class, which is a subclass of HttpResponse. Its default Content-Type header is set to application/json, which is really convenient. It also comes with a JSON encoder, so you don’t need to serialize the data before returning the response object. See a minimal example below: from django.http import JsonResponse def profile(request): data = { 'name': 'Vitor', 'location': 'Finland', 'is_active': True, 'count': 28 } return JsonResponse(data) By default, the JsonResponse’s first parameter, data, should be a dict instance. To pass any other JSON-serializable object you must set the safe parameter to False. return JsonResponse([1, 2, 3, 4], safe=False) See below the class signature: class JsonResponse(data, encoder, safe, json_dumps_params, **kwargs) Defaults: data: (no default) encoder: django.core.serializers.json.DjangoJSONEncoder safe: True json_dumps_params: None Extra bits: If you want to return Django models as JSON, you may want to it this way: def get_users(request): users = User.objects.all().values('first_name', 'last_name') # or simply .values() to get all fields users_list = list(users) # important: convert the QuerySet to a list object return JsonResponse(users_list, safe=False) Troubleshooting Django 1.6 or below For older versions of Django, you must use an HttpResponse object. See an example below: import json from django.http import HttpResponse def … -
How to Return JSON-Encoded Response
Since version 1.7, Django counts with the built-in JsonResponse class, which is a subclass of HttpResponse. Its default Content-Type header is set to application/json, which is really convenient. It also comes with a JSON encoder, so you don’t need to serialize the data before returning the response object. See a minimal example below: from django.http import JsonResponse def profile(request): data = { 'name': 'Vitor', 'location': 'Finland', 'is_active': True, 'count': 28 } return JsonResponse(data) By default, the JsonResponse’s first parameter, data, should be a dict instance. To pass any other JSON-serializable object you must set the safe parameter to False. return JsonResponse([1, 2, 3, 4], safe=False) See below the class signature: class JsonResponse(data, encoder, safe, json_dumps_params, **kwargs) Defaults: data: (no default) encoder: django.core.serializers.json.DjangoJSONEncoder safe: True json_dumps_params: None Extra bits: If you want to return Django models as JSON, you may want to it this way: def get_users(request): users = User.objects.all().values('first_name', 'last_name') # or simply .values() to get all fields users_list = list(users) # important: convert the QuerySet to a list object return JsonResponse(users_list, safe=False) Troubleshooting Django 1.6 or below For older versions of Django, you must use an HttpResponse object. See an example below: import json from django.http import HttpResponse def … -
uWSGI Basic Django Setup
Here are two basic examples of almost the same uWSGI configuration to run a Django project; one is configured via an ini configuration file and the other is configured via a command line argument.This does not represent a production-ready example, but can be used as a starting point for the configuration.Setup for this example:# create dir for virtualenv and activatemkdir envs/virtualenv envs/runrun/. ./envs/runrun/bin/activate# create dir for project codebasemkdir proj/# install some django depspip install django uwsgi whitenose# create a new django projectcd proj/django-admin startproject runruncd runrun/# Add to or modify django settings.py to setup static file serving with Whitenoise.# Note: for prod environments, staticfiles can be served via Nginx.# settings.py MIDDLEWARE_CLASSES = [ .... 'whitenoise.middleware.WhiteNoiseMiddleware',] STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'Create the config file$ cat run.ini [uwsgi]module = runrun.wsgi:applicationvirtualenv = ../../envs/runrun/env = DJANGO_SETTINGS_MODULE=runrun.settings env = PYTHONPATH="$PYTHONPATH:./runrun/" master = trueprocesses = 5enable-threads = truemax-requests = 5000harakiri = 20vacuum = truehttp = :7777stats = 127.0.0.1:9191Execute configuwsgi --ini run.iniOr create a shell script$ cat run.sh uwsgi --module=runrun.wsgi:application \ -H ../../envs/runrun/ \ --env DJANGO_SETTINGS_MODULE=runrun.settings \ --env PYTHONPATH="$PYTHONPATH:./runrun/" \ --master \ --processes=5 \ --max-requests=5000 \ --harakiri=20 \ --vacuum \ … -
How to Reset Migrations
./manage.py migrate --fake <app-name> zero -
How to Reset Migrations
The Django migration system was developed and optmized to work with large number of migrations. Generally you shouldn’t mind to keep a big amount of models migrations in your code base. Even though sometimes it causes some undesired effects, like consuming much time while running the tests. But in scenarios like this you can easily disable the migrations (although there is no built-in option for that at the moment). Anyway, if you want to perform a clean-up, I will present a few options in this tutorial. Scenario 1: The project is still in the development environment and you want to perform a full clean up. You don’t mind throwing the whole database away. 1. Remove the all migrations files within your project Go through each of your projects apps migration folder and remove everything inside, except the __init__.py file. Or if you are using a unix-like OS you can run the following script (inside your project dir): find . -path "*/migrations/*.py" -not -name "__init__.py" -delete find . -path "*/migrations/*.pyc" -delete 2. Drop the current database, or delete the db.sqlite3 if it is your case. 3. Create the initial migrations and generate the database schema: python manage.py makemigrations python manage.py migrate … -
ShipIt Day Recap - July 2016
We finished up last week with another successful ShipIt Day. ShipIt Days are quarterly events where we put down client work for a little bit and focus on learning, stretching ourselves, and sharing. Everyone chooses to work together or individually on an itch or a project that has been on the back of their mind for the last few months. This time, we stretched ourselves by trying out new frameworks, languages, and pluggable apps. Here are some of the projects we worked on during ShipIt Day:
