Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Testing Cloudflare workers
I like Cloudflare workers. They allow you to execute arbitrary Javascript as close to your users as possible. For simple use cases your scripts probably don't need to be tested, but once they grow in size and complexity you probably want to be confident that they don't break production. Something I dislike about the serverless trend is that most platforms are hard to test and debug. Testing is poorly documented and examples are far and few between. I wrote this blog post after implementing a non-trivial Cloudflare worker script for a client. The script used the Caching API, communicated with a backend server and manipulated HTTP headers. It required correctness tests before we rolled it out in production. I dug through the Cloudflare documentation and related blog posts to find testing examples, but there wasn't enough material out there so I figured I'd consolidate the knowledge I have on testing Cloudflare workers. The starting point for testing workers is in the [Cloudflare](https://developers.cloudflare.com/workers/archive/deploying-workers/unit-testing/) documentation. However I struggled get the examples to work and it didn't cover common Node.js testing techniques such as stubbing, mocking and running debuggers. This post will cover common testing techniques and scenarios I use to test Cloudflare … -
Signals
Signals in the Django docs -
Migrating from Python 2 to 3 at EdX - David Ormsbee & Nimisha Asthagiri
EdX WebsiteEdX Github Repodjango-waffleOpen edX Proposals -
Multipage Forms in Django
Introduction Most online forms fit on a single page. Think of a "join our forum" or "contact us" form into which the user enters a name, email address, and maybe a few other pieces of information. If you're building this kind of functionality into a Django site, you can take advantage of Django's built-in form classes. These are especially handy when dealing with model forms, where the form fields correspond to the fields on a model that will be recorded in your database. But what if you need a form that spans more than one page? Like a multipage job application where your personal details are on page 1, your relevant experience is on page 2, and so on? There are third-party libraries to help with this, but if you do it yourself you can sidestep a dependency and maybe become more familiar with Django's form handling. So let's do that. In the post below we'll go step-by-step through the creation of a multipage job application form. We'll start with the simplest functionality and then make it (slightly) more sophisticated. The most important modules ("models.py", "forms.py", and "views.py") will be reproduced here, but a working, standalone project is available from … -
How To Remove A Menu Item From Wagtail Settings Menu
Let's say I want to remove the User menu item from Wagtail settings menu. All I need to do is to put this hook into wagtail_hooks.py file of my Wagtail CMS app: from wagtail.core import hooks @hooks.register('construct_settings_menu') def hide_user_menu_item(request, menu_items): menu_items[:] = [item for item ... Read now -
Working with Dates and Times in the Forms
HTML5 comes with a bunch of new types for the input fields that are rendered as rich native widgets. Browsers even restrict invalid values and validate the input immediately. Let's explore how we could make use of them in Django forms. We will be using an Exhibition model with models.DateField, models.TimeField, and models.DateTimeField: # exhibitions/models.pyfrom django.db import modelsfrom django.utils.translation import gettext_lazy as _class Exhibition(models.Model): title = models.CharField(_("Title"), max_length=200) start = models.DateField(_("Start")) end = models.DateField(_("End"), blank=True, null=True) opening = models.TimeField(_("Opening every day")) closing = models.TimeField(_("Closing every day")) vernissage = models.DateTimeField(_("Vernissage"), blank=True, null=True) finissage = models.DateTimeField(_("Finissage"), blank=True, null=True) class Meta: verbose_name = _("Exhibition") verbose_name_plural = _("Exhibitions") def __str__(self): return self.title Here is a quick model form for the Exhibition model: # exhibitions/forms.pyfrom django import formsfrom .models import Exhibitionclass ExhibitionForm(forms.ModelForm): class Meta: model = Exhibition fields = "__all__" If we now open a Django shell and create an instance of the model form with some initial values, then print the form as HTML to the console, we will notice, that all date and time fields are rendered as <input type="text" /> and the values for the dates are in a local format, not the ISO standard YYYY-MM-DD: (venv)$ python manage.py shell>>> from exhibitions.forms … -
DjangoCon 2019 Delivered Again
Above: Django Fellow Carlton Gibson gives a talk on "Your Web Framework Needs You: An Update." Again this year, DjangoCon more than delivered on its promise of something for everyone. A keynote on burnout and balance? Thoroughly entertaining! Jessica Rose’s talk left us in stitches. Who knew a talk about occupational burnout could be so fun? It was one of a wide range of subjects covered in three days of talks, bookended by a day of tutorials and two days of sprints. The conference took place in San Diego and ran from September 22 - 26. What a great week of learning and networking! Caktus was honored to sponsor again this year, our tenth in a row. Attending this year from Caktus were Karen Tracey, lead developer and technical director; Jeremy Gibson, developer; Erin Mulaney, contractor developer; and myself, Ian Huckabee, CBDO. The Talks … The sessions packed a strong lineup of speakers with a diversity of topics, and this year, every word was captured with closed captioning. There were the practical: Luan Fonseca, for example, went to great lengths to ensure you’re never left with technical debt in this era of sustainable software. He addressed how to find it, … -
Fixing anymail/mailgun "Domain not found" error
Are you swearing at mailgun now? Have you smashed your keyboard already? Do you keep receiving the Domain not found: domainname.com error although you're 100% sure you've set everything up correctly? Yeah, I've been there and probably, I have a solution for you. Chances are, your ... Read now -
My Most Used Pytest Commandline Flags
Pytest is quickly becoming the “standard” Python testing framework. However it can be overwhelming to new users. pytest --help currently outputs 275 lines of command line flags and options. Where do you even begin? I searched my ZSH history for my recent usage of Pytest. I found 184 unique invocations, out of which I split the command line flags I used. Here are the top five flags I used by frequency. 1. -v -v, or --verbose, increases the verbosity level. The default level outputs a reasonable amount of information to debug most test failures. However when there are many differences between actual and expected data, some get hidden. In such cases Pytest normally appends a message to the failure text such as: ...Full output truncated (23 lines hidden), use '-vv' to show This tells us to see the hidden lines, we need to rerun the tests with verbosity level 2. To do this we can pass -v twice as -v -v, or more easily as -vv. Another change from passing -v at least once is that the test names are output one per line as they run: $ pytest -v tests.py::test_parrot_status PASSED [100%] Sometimes I use this when looking for … -
Search
Django Search Tutorialdjango-filterMDN on sending form data and form data validationDjango Q Objectsdjango.contrib.postgres.searchPostgreSQL Full Text SearchEuroPython 2017 - Full-Text Search in Django with PostgreSQL by Paulo MelchiorreDjangoCon Europe 2018 - On The Look-Out For Your Data by Markus HoltermannDjangoCon US 2015 - Beyond the basics with Elasticsearch by Honza Kral -
Django Projects – ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to… -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Blackifying and fixing bugs
Since version 0.9 of Evennia, the MU*-creation framework, was released, work has mainly been focused on bug fixing. But there few new features also already sneaked into master branch, despite technically being changes slated for Evennia 1.0. On FrontendsContributor friarzen has chipped away at improving Evennia's HTML5 web client. It already had the ability to structure and spawn any number of nested text panes. In the future we want to extend the user's ability to save an restore its layouts and allow developers to offer pre-prepared layouts for their games. Already now though, it has gotten plugins for handling both graphics, sounds and video: Inline image by me (griatch-art.deviantart.com) A related fun development is Castlelore Studios' development of an Unreal Engine Evennia plugin (this is unaffiliated with core Evennia development and I've not tried it, but it looks pretty nifty!): Image ©Castlelore Studios On BlackEvennia's source code is extensively documented and was sort of adhering to the Python formatting standard PEP8. But many places were sort of hit-and-miss and others were formatted with slight variations due to who wrote the code. After pre-work and recommendation by Greg Taylor, Evennia has adopted the black autoformatter for its source code. I'm not really convinced that … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Ruby on Rails & Django- David Heinemeier Hansson
David Heinemeier Hansson personal siteRuby on Rails -
DjangoCon US 2019: Python & Django in San Diego!
We are back to San Diego!! Our team will be joining DjangoCon US's conference, one of the biggest Django events in the world. For this year, we'll be giving two talks: Pull Requests: Merging good practices into your project and Building effective Django queries with expressions Here is the slide from the talk we gave during the conference: Pull Re -
Deploying Django on Docker to Heroku with OpenCV
Building off [this post](https... -
Django on Docker - A Simple Introduction
This is a simple tutorial but ... -
Working around memory leaks in your Django application
Several large Django applications that I’ve worked on ended up with memory leaks at some point. The Python processes slowly increased their memory consumption until crashing. Not fun. Even with automatic restart of the process, there was still some downtime. Memory leaks in Python typically happen in module-level variables that grow unbounded. This might be an lru_cache with an infinite maxsize, or a simple list accidentally declared in the wrong scope. Leaks don’t need to happen in your own code to affect you either. For example, see this excellent write-up by Peter Karp at BuzzFeed where he found one inside Python’s standard library (since fixed!). Workarounds The below workarounds all restart worker processes after so many requests or jobs. This is a simple way to clear out any potential infinitely-accumulating Python objects. If your web server, queue worker, or similar has this ability but isn’t featured, let me know and I’d add it! Even if you don’t see any memory leaks right now, adding these will increase your application’s resilience. Gunicorn If you’re using Gunicorn as your Python web server, you can use the --max-requests setting to periodically restart workers. Pair with its sibling --max-requests-jitter to prevent all your workers … -
Simple Docker Container Tutorial
Docker is all the rage. I cove... -
Django 3.0 Preview
Django 3.0 alpha downloadASGI (Asynchronous Server Gateway Interface)PyCon AU 2019 - Just Add Await: Retrofitting Async into Django by Andrew GodwinDjangoCon Europe 2019 - Sketching out a Django Redesign by Tom Christieforum.djangoproject.com -
How "Export to Excel" Almost Killed Our System
A few weeks ago we had some trouble with an "Export to Excel" functionality in one of our systems. In the process of resolving this issue, we made some interesting discoveries and came up with original solutions. This article is inspired by the actual issue we used to track this incident over a period of two days. We go through the process of identifying the problem, experimenting and benchmarking different solutions until eventually deploying to production. These are the main takeaways described in this article: Generating xlsx files can consume significant amount of resources. Under some circumstances better performance can be gained by not using prefetch_related. pyexcelerate is a fast package for creating simple Excel files. tablib (and django-import-export) can be patched to use pyexcelerate and produce excel files faster. Table of Contents Exporting a QuerySet to Excel Using django-import-export Finding the Best File Format Improving the Query Replacing prefetch_related with Subquery and OuterRef Using an Iterator Simplifying the Query Manual Prefetch Trouble in Paradise Using a Different Excel Writer A Faster Excel Writer in Python Patching tablib Results Summary Seifa How a server must feel when asked to produce an Excel file A few weeks ago we started getting … -
Django shell_plus with Pandas and Jupyter Notebook
The Django shell provides an environment where developers can interact with the database via Django's ORM. While the shell is great for basic interactions, it quickly becomes laborious to work in, be it due to manual imports, having to scroll through shell history to repeat commands, or working with / viewing queries returning more than, say, 20 records. These issues, and more, can be remedied by interacting with the ORM in Jupyter notebooks, using Pandas. Our environment will be setup inside a virtual environment using Django 2.2. Once inside the virtual environment, install IPython, Jupyter, Pandas, django-extensions, and django-pandas. pip install ipython jupyter pandas django-extensions django-pandas django-extensions needs to be placed into INSTALLED_APPS in your settings.py file. INSTALLED_APPS = [ ... 'django_extensions', ] Once installed, then run python manage.py shell_plus --notebook shell_plus is Django shell with autoloading of the apps database models and subclasses of user-defined classes. The --notebook argument tells shell_plus to open in a Jupyter notebook. You will notice that a new Jupyter tab opens in your browser. In the upper right corner click on the New drop-down and select Django Shell-Plus. A new tab will open and you will see your new Jupyter notebook. Import … -
A Single File Asynchronous Django Application
Django 3.0 alpha 1 came out this week. It introduces ASGI support thanks to lots of hard work by Andrew Godwin. Thanks to a question from Emiliano Dalla Verde Marcozzi on the shiny new Django forums, I decided to play with it a bit. I adapted my “single file Django app” to ASGI. Here’s how to try it yourself. First, create a new virtual environment (I’m using Python 3.7.4) and install requirements: $ python -m venv venv $ source venv/bin/activate $ pip install django==3.0a1 daphne==2.3.0 Then create a new file app.py: import html import os import sys from django.conf import settings from django.core.asgi import get_asgi_application from django.http import HttpResponse from django.urls import path from django.utils.crypto import get_random_string settings.configure( DEBUG=(os.environ.get("DEBUG", "") == "1"), # Disable host header validation ALLOWED_HOSTS=["*"], # Make this module the urlconf ROOT_URLCONF=__name__, # We aren't using any security features but Django requires this setting SECRET_KEY=get_random_string(50), ) def index(request): name = request.GET.get("name", "World") return HttpResponse(f"Hello, {html.escape(name)}!") urlpatterns = [path("", index)] application = get_asgi_application() if __name__ == "__main__": from django.core.management import execute_from_command_line execute_from_command_line(sys.argv) Then run it under Daphne: $ daphne app:application Visit http://localhost:8000/?name=Django%20user in your browser to see “Hello, Django user!” Async support in Django 3.0 is the first … -
How I Import Python's datetime Module
Python’s datetime module risks a whole bunch of name confusion: It contains a class also called datetime. When you read the word datetime, you can’t be sure it’s the module or the class. It contains a time class. This can be confused with the time module, or the time module’s time() function. It contains a class called timezone. In a Django project this can be confused with the django.utils.timezone module For these reasons, I use this import idiom and recommend you do too: import datetime as dt Rather than any of: import datetime from datetime import datetime # or time, timezone Then in your code, dt.datetime, dt.time, and dt.timezone will be unambiguous. Fin Hope this helps you save time understanding code, —Adam
