Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Random, Fixed Ordering and Pagination
Consider the following situation: > We have a set of items that we want to show in a random order. > The order, however, should remain fixed for a period. > The display of items will need to be paginated, and we have over 200,000 items. If the ordering is not truly random, then we could have an expression index, and allow ordering based on that. However, that doesn't really help out with the pagination, and issues around LIMIT/OFFSET ordering of very large sets. Instead, I came up with a solution this afternoon that uses Postgres Materialised Views. Let's start with a Django model: {% highlight python %} class Post(models.Model): title = models.TextField() content = models.TextField() posted_at = models.DateTimeField() {% endhighlight %} We can build up a materialised view that associates each post with a position: {% highlight postgresql %} CREATE MATERIALISED VIEW post_ordering AS SELECT post.id AS post_id, row_number() OVER () AS position FROM ( SELECT id FROM blog_post ORDER BY random() ) post; CREATE INDEX post_ordering_id ON post_ordering(post_id); CREATE INDEX post_ordering_position ON post_ordering(position); {% endhighlight %} Because a materialised view stores a copy of the data, we need to index it if we want to get performance benefits. … -
Channels
Weekly Django Chat newsletterdjango-channelsDjangoCon 2019 - Just Add Await: Retrofitting Async Into Django by Andrew GodwinDeveloping a Real-Time Taxi App with Django Channels and Angularhttpxdeps: Django Enhancement ProposalsDjango Developers Google GroupDjango Forum -
Start Learning Web Development
There is a LOT of good information out there. Tutorials, videos, paid courses, free courses… so much good stuff. When I decided I wanted to learn to code,… -
Start Learning Web Development
Skip to tl;dr There is a LOT of good information out there. Tutorials, videos, paid courses, free courses… so much good stuff. When I decided I wanted to learn to code, I just started doing whatever looked interesting to me. It’s easy to fall into the trap of trying to pick the “optimal” syllabus for your personal learning journey. I wanted to start coding now. I actually started years ago – overnight in an airport in Bergen, Norway – doing the Django polls tutorial. This is a good one, but my goodness it was a hard time. Yes, we can attribute that partially to pulling an all-nighter. I definitely get dumber without sleep. But the real problem was that I just didn’t understand much of anything about web development. Plus, my Python was meh at best. The Django polls tutorial is good, but probably not where you want to start. First, learn Python I found that whole experience kind of frustrating. I made it through the whole app, but I didn’t know any programming and didn’t really understand how this web development framework even works. I didn’t understand any of the anatomy – the structure – of the code, so … -
To •••• With Passwords
Slides Video Notes THREAD: some marginalia and further reading for folks who attended my #nbpy talk and would like to explore further 👇🏻 — jacobian (@jacobian) November 2, 2019 -
Functions as Tables in Django and Postgres
Postgres has some excellent features. One of these is [set-returning functions](https://www.postgresql.org/docs/current/functions-srf.html). It's possible to have a function (written in SQL, or some other language) that returns a set of values. For instance, the in-built function `generate_series()` returns a set of values: {% highlight sql %} SELECT day::DATE FROM generate_series(now(), now() + INTERVAL '1 month', '1 day') day; {% endhighlight %} This uses a set returning function as a table source: in this case a single column table. You can use scalar set-returning functions from within Django relatively easily: [I blogged about it last year](https://schinckel.net/2018/11/02/set-returning-and-row-accepting-functions-in-django-and-postgres/). It is possible to [create your own set-returning functions](https://www.postgresql.org/docs/current/sql-createfunction.html). Further, the return type can be a `SETOF` any table type in your database, or even a "new" table. {% highlight postgresql %} CREATE OR REPLACE FUNCTION foo(INTEGER, INTEGER) RETURNS TABLE(id INTEGER, bar_id INTEGER, baz JSON[]) AS $$ SELECT foo.id AS id, bar.id AS bar_id, ARRAY_AGG(JSON_BUILD_OBJECT(bar.x, foo.y)) FROM foo INNER JOIN bar ON (foo.id = bar.foo_id) WHERE foo.y = $1 AND bar.x > $2 GROUP BY foo.id, bar.id $$ LANGUAGE SQL STABLE; {% endhighlight %} It's possible to have a Postgres VIEW as the data source for a Django model (you just set the `Meta.db_table` on the … -
Django Slug Tutorial
How to add slugs to a Django DetailView. -
How Django Experts Think
Carlton’s Python bugPython 3 Standard LibraryBest Python booksDjango for ProfessionalsDreyfus Model of Skill AcquisitionZen Mind, Beginner’s Mind book -
How to deploy Django project to Dokku
In this Dokku tutorial, I will talk about how to deploy Django project with Dokku, it would use Postgres db and Amazon S3 to store data and media files. -
How to make Django request.is_ajax() work with JS fetch()
Django's request object has a nifty little method is_ajax. It allows determining whether a request comes from a JS framework (aka old-school ajax). While it works fine with some JS libraries, including the venerable jQuery, it won't work out of the box with modern JS native fetch API ... Read now -
Detecting queries in Django tests
Putting this here so I can find it next time I need to know it... Django has a useful test assertion you can use to ensure you make a set number of queries. However, at times this is a bit less useful than it needs to be, because something changes and we do indeed have a different number of queries, but it's got nothing to do with the actual code under test. If you are running in `DEBUG=True` mode, then you can examine the queries that have been made to the database connection, and ensure the raw SQL of a specific query matches (and is not duplicated, for instance). This does require a little bit of trickery: {% highlight python %} from django.db import connection from django.test import TestCase, override_settings from foo.factories import FooFactory class TestTheThing(TestCase): def test_no_update_query(self): foo = FooFactory() # Our Foo instance should be smart enough to notice that nothing # has changed, and thus should not emit an UPDATE query. with override_settings(DEBUG=True): foo.save() self.assertFalse([ x for x in connection.queries if 'UPDATE' in x['sql'] ]) {% endhighlight %} This is a bit of a contrived case: in my case today it was a celery task that only … -
Things I want to remember about SSH
SSH, short for Secure Shell, is a protocol for secure network communications. It is widely used for executing commands on remote servers, and for file uploads or downloads. If you are working with Django, use Git version control, or administrate servers, you surely are using SSH. In this post, I want to share some technical details about it. Secure Shell is using private and public key pairs. You can either use automatically generated private and public keys combined with a password, or manually generated private and public keys. In the latter case, you need to keep your private key on your computer and upload the public key to the remote server. Creating a pair of SSH keys manually If you are using GitHub, Bitbucket, DigitalOcean, or some other service, you might have seen the possibility to upload public SSH keys for direct access to remote servers. Here is how you usually create the SSH keys on the computer from which you want to establish a secure connection (your local machine or one of your servers that has access to other servers or services). In the Terminal you would execute these commands: $ ssh-keygen$ ssh-agent /usr/local/bin/bash$ ssh-add ~/.ssh/id_rsa The id_rsa is … -
Middleware
Weekly Django Chat newsletterMiddleware in Django DocsCarlton's DjangoCon US 2019 talk on Django as a MicroframeworkWSGIBaseHandler class in Django source code -
Testing Cloudflare workers
I like Cloudflare workers. They allow you to execute arbitrary Javascript as close to your users as possible. For simple use cases your scripts probably don't need to be tested, but once they grow in size and complexity you probably want to be confident that they don't break production. Something I dislike about the serverless trend is that most platforms are hard to test and debug. Testing is poorly documented and examples are far and few between. I wrote this blog post after implementing a non-trivial Cloudflare worker script for a client. The script used the Caching API, communicated with a backend server and manipulated HTTP headers. It required correctness tests before we rolled it out in production. I dug through the Cloudflare documentation and related blog posts to find testing examples, but there wasn't enough material out there so I figured I'd consolidate the knowledge I have on testing Cloudflare workers. The starting point for testing workers is in the [Cloudflare](https://developers.cloudflare.com/workers/archive/deploying-workers/unit-testing/) documentation. However I struggled get the examples to work and it didn't cover common Node.js testing techniques such as stubbing, mocking and running debuggers. This post will cover common testing techniques and scenarios I use to test Cloudflare … -
Testing Cloudflare workers
I like Cloudflare workers. They allow you to execute arbitrary Javascript as close to your users as possible. For simple use cases your scripts probably don't need to be tested, but once they grow in size and complexity you probably want to be confident that they don't break production. Something I dislike about the serverless trend is that most platforms are hard to test and debug. Testing is poorly documented and examples are far and few between. I wrote this blog post after implementing a non-trivial Cloudflare worker script for a client. The script used the Caching API, communicated with a backend server and manipulated HTTP headers. It required correctness tests before we rolled it out in production. I dug through the Cloudflare documentation and related blog posts to find testing examples, but there wasn't enough material out there so I figured I'd consolidate the knowledge I have on testing Cloudflare workers. The starting point for testing workers is in the [Cloudflare](https://developers.cloudflare.com/workers/archive/deploying-workers/unit-testing/) documentation. However I struggled get the examples to work and it didn't cover common Node.js testing techniques such as stubbing, mocking and running debuggers. This post will cover common testing techniques and scenarios I use to test Cloudflare … -
Signals
Signals in the Django docs -
Migrating from Python 2 to 3 at EdX - David Ormsbee & Nimisha Asthagiri
EdX WebsiteEdX Github Repodjango-waffleOpen edX Proposals -
Multipage Forms in Django
Introduction Most online forms fit on a single page. Think of a "join our forum" or "contact us" form into which the user enters a name, email address, and maybe a few other pieces of information. If you're building this kind of functionality into a Django site, you can take advantage of Django's built-in form classes. These are especially handy when dealing with model forms, where the form fields correspond to the fields on a model that will be recorded in your database. But what if you need a form that spans more than one page? Like a multipage job application where your personal details are on page 1, your relevant experience is on page 2, and so on? There are third-party libraries to help with this, but if you do it yourself you can sidestep a dependency and maybe become more familiar with Django's form handling. So let's do that. In the post below we'll go step-by-step through the creation of a multipage job application form. We'll start with the simplest functionality and then make it (slightly) more sophisticated. The most important modules ("models.py", "forms.py", and "views.py") will be reproduced here, but a working, standalone project is available from … -
How To Remove A Menu Item From Wagtail Settings Menu
Let's say I want to remove the User menu item from Wagtail settings menu. All I need to do is to put this hook into wagtail_hooks.py file of my Wagtail CMS app: from wagtail.core import hooks @hooks.register('construct_settings_menu') def hide_user_menu_item(request, menu_items): menu_items[:] = [item for item ... Read now -
Working with Dates and Times in the Forms
HTML5 comes with a bunch of new types for the input fields that are rendered as rich native widgets. Browsers even restrict invalid values and validate the input immediately. Let's explore how we could make use of them in Django forms. We will be using an Exhibition model with models.DateField, models.TimeField, and models.DateTimeField: # exhibitions/models.pyfrom django.db import modelsfrom django.utils.translation import gettext_lazy as _class Exhibition(models.Model): title = models.CharField(_("Title"), max_length=200) start = models.DateField(_("Start")) end = models.DateField(_("End"), blank=True, null=True) opening = models.TimeField(_("Opening every day")) closing = models.TimeField(_("Closing every day")) vernissage = models.DateTimeField(_("Vernissage"), blank=True, null=True) finissage = models.DateTimeField(_("Finissage"), blank=True, null=True) class Meta: verbose_name = _("Exhibition") verbose_name_plural = _("Exhibitions") def __str__(self): return self.title Here is a quick model form for the Exhibition model: # exhibitions/forms.pyfrom django import formsfrom .models import Exhibitionclass ExhibitionForm(forms.ModelForm): class Meta: model = Exhibition fields = "__all__" If we now open a Django shell and create an instance of the model form with some initial values, then print the form as HTML to the console, we will notice, that all date and time fields are rendered as <input type="text" /> and the values for the dates are in a local format, not the ISO standard YYYY-MM-DD: (venv)$ python manage.py shell>>> from exhibitions.forms … -
Heroku Logs Tutorial
In this Heorku tutorial, I talked about how to use heroku logs and how to use Sentry to record error log for Heroku app. -
DjangoCon 2019 Delivered Again
Above: Django Fellow Carlton Gibson gives a talk on "Your Web Framework Needs You: An Update." Again this year, DjangoCon more than delivered on its promise of something for everyone. A keynote on burnout and balance? Thoroughly entertaining! Jessica Rose’s talk left us in stitches. Who knew a talk about occupational burnout could be so fun? It was one of a wide range of subjects covered in three days of talks, bookended by a day of tutorials and two days of sprints. The conference took place in San Diego and ran from September 22 - 26. What a great week of learning and networking! Caktus was honored to sponsor again this year, our tenth in a row. Attending this year from Caktus were Karen Tracey, lead developer and technical director; Jeremy Gibson, developer; Erin Mulaney, contractor developer; and myself, Ian Huckabee, CBDO. The Talks … The sessions packed a strong lineup of speakers with a diversity of topics, and this year, every word was captured with closed captioning. There were the practical: Luan Fonseca, for example, went to great lengths to ensure you’re never left with technical debt in this era of sustainable software. He addressed how to find it, … -
DjangoCon 2019 Delivered Again
Above: Django Fellow Carlton Gibson gives a talk on "Your Web Framework Needs You: An Update." Again this year, DjangoCon more than delivered on its promise of something for everyone. A keynote on burnout and balance? Thoroughly entertaining! Jessica Rose’s talk left us in stitches. Who knew a talk about occupational burnout could be so fun? It was one of a wide range of subjects covered in three days of talks, bookended by a day of tutorials and two days of sprints. The conference took place in San Diego and ran from September 22 - 26. What a great week of learning and networking! Caktus was honored to sponsor again this year, our tenth in a row. Attending this year from Caktus were Karen Tracey, lead developer and technical director; Jeremy Gibson, developer; Erin Mulaney, contractor developer; and myself, Ian Huckabee, CBDO. The Talks … The sessions packed a strong lineup of speakers with a diversity of topics, and this year, every word was captured with closed captioning. There were the practical: Luan Fonseca, for example, went to great lengths to ensure you’re never left with technical debt in this era of sustainable software. He addressed how to find it, … -
Fixing anymail/mailgun "Domain not found" error
Are you swearing at mailgun now? Have you smashed your keyboard already? Do you keep receiving the Domain not found: domainname.com error although you're 100% sure you've set everything up correctly? Yeah, I've been there and probably, I have a solution for you. Chances are, your ... Read now -
My Most Used Pytest Commandline Flags
Pytest is quickly becoming the “standard” Python testing framework. However it can be overwhelming to new users. pytest --help currently outputs 275 lines of command line flags and options. Where do you even begin? I searched my ZSH history for my recent usage of Pytest. I found 184 unique invocations, out of which I split the command line flags I used. Here are the top five flags I used by frequency. 1. -v -v, or --verbose, increases the verbosity level. The default level outputs a reasonable amount of information to debug most test failures. However when there are many differences between actual and expected data, some get hidden. In such cases Pytest normally appends a message to the failure text such as: ...Full output truncated (23 lines hidden), use '-vv' to show This tells us to see the hidden lines, we need to rerun the tests with verbosity level 2. To do this we can pass -v twice as -v -v, or more easily as -vv. Another change from passing -v at least once is that the test names are output one per line as they run: $ pytest -v tests.py::test_parrot_status PASSED [100%] Sometimes I use this when looking for …