Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
SOCCR: the framework I use for decision briefs
In my previous article, I wrote about gathering consensus before a decision. Several folks asked for more detail about how I structure those consensus-gathering and decision-making exercises. There’s a specific format I find helpful, which I remember by the acronym SOCCR. Read on for an explanation and example! -
Are Django and Flask Similar?
Maybe you’re new to web development in Python, and you’ve encountered the two most popular Python web frameworks, Django and Flask, and have questions about which one you should use. Are Django and Flask similar tools for building web applications? Yes, Django and Flask share many similarities and can both make great websites, but they have some different development philosophies which will attract different types of developers. What do I know about this? -
Recap of 2020 and Resolutions for 2021
Recap of 2020 I got a job saving the planet For years I've been worried about climate change. I've tried to live a mostly ecologically friendly life, and this year I dived in with making compost for the garden. However, seeing the lack of consistent political will to correct the problem has had me worried. So I was delighted that when I started to look for jobs in the autumn I was contacted by several firms attempting to address the problem. I've always thought that finance is a great motivator, albeit all-too-often the results are negative (the petroleum and coal industries are examples). Ultimately I went with Octopus Energy, a company that is to borrow a phrase, disrupting the legacy power industry in areas with free-er energy markets (UK, Australia, Germany, Texas, Japan). They have embraced renewable energies as a source of power, which is good because it is cheaper than fossil fuels. Even if customers don't believe in global climate change they won't be able to ignore the fact renewables are a cheaper source of energy. In any case, I can't begin to say how wonderful it is being part of a company whose mission is to address climate … -
Recap of 2020 and Resolutions for 2021
Recap of 2020 I got a job saving the planet For years I've been worried about climate change. I've tried to live a mostly ecologically friendly life, and this year I dived in with making compost for the garden. However, seeing the lack of consistent political will to correct the problem has had me worried. So I was delighted that when I started to look for jobs in the autumn I was contacted by several firms attempting to address the problem. I've always thought that finance is a great motivator, albeit all-too-often the results are negative (the petroleum and coal industries are examples). Ultimately I went with Octopus Energy, a company that is to borrow a phrase, disrupting the legacy power industry in areas with free-er energy markets (UK, Australia, Germany, Texas, Japan). They have embraced renewable energies as a source of power, which is good because it is cheaper than fossil fuels. Even if customers don't believe in global climate change they won't be able to ignore the fact renewables are a cheaper source of energy. In any case, I can't begin to say how wonderful it is being part of a company whose mission is to address climate … -
Developing a Single Page App with Flask and Vue.js
The following is a step-by-step walkthrough of how to set up a basic CRUD app with Vue and Flask. -
Django News - New Wagtail and Documentation Sprint - Jan 29th 2021
News Wagtail Documentation Sprint A Wagtail Sprint is occurring February 4th and 5th, 2021 focused on documentation: tooling, writing, editing, content reorganization, design etc. It will be run from Torchbox’s offices in the UK, where anyone is welcome to join us, if it’s safe and legal, but we expect most people to participate remotely. wagtail.io Wagtail 2.11 just released Features include internationalization and improved image and document management. wagtail.io Articles Efficient Reloading in Django’s Runserver With Watchman Watchman is an efficient file watcher open sourced by Facebook, for which Django support was added in 2.2. In this article, Adam Johnson demonstrates how to use it with runserver for improved performance. adamj.eu Reading CloudFlare headers in a Django middleware Adam breaks down how to find a client’s IP address when using a CDN like Cloudflare. adamj.eu 5 Tips to Speed up Your Docker Image Build Docker is powerful but can be very time consuming. Here are 5 good tips for speeding up your image builds. vsupalov.com Which Django Version Should I Use? Matt Layman provides some advice on thinking about which Django version to use in your projects. mattlayman.com Sponsored Link Tackle Architecture Issues on Your Complex Django Codebase. As a … -
Customer Feature - Building SaaS #89
In this episode, I show you how to take a feature idea from nothing to something. We added new UI, wrote a new view, a new form, and all the associated test code to prove that the feature works. My customer wants the ability to complete a task on any day she desires. The feature flow looks like: Click a calendar icon next to a task on a student’s course page. -
Cleaning up a required process when you quit
Some time ago, I moved a bunch of our devops tools into a Makefile. I'm not sure exactly what the benefit was then, but there have been some challenges. One of these was related to having to run multiple processes when performing a task, but cleaning up the dependencies when we finish our task. Specifically, we run a websocket server in production that uses Redis for PubSub. We can safely assume the developer is running redis already, since the rest of the platform also depends upon it. However, to collect the changes from Postgres and publish them into Redis, we use a custom django management command. All this does is listen for postgres notifications and forward them to redis. $ ./manage.py relay Mostly we don't keep this running in development, unless you need to be running the websocket code locally for testing. It is a hassle to remember to start a process and then kill it when you are done, so it would be convenient to start the relay command, then start our websocket worker, and clean up both processes when we quit the worker. This _can_ be done in a Makefile command: {% highlight make %} websocket: $(shell trap … -
Cleaning up a required process when you quit
Some time ago, I moved a bunch of our devops tools into a Makefile. I'm not sure exactly what the benefit was then, but there have been some challenges. One of these was related to having to run multiple processes when performing a task, but cleaning up the dependencies when we finish our task. Specifically, we run a websocket server in production that uses Redis for PubSub. We can safely assume the developer is running redis already, since the rest of the platform also depends upon it. However, to collect the changes from Postgres and publish them into Redis, we use a custom django management command. All this does is listen for postgres notifications and forward them to redis. $ ./manage.py relay Mostly we don't keep this running in development, unless you need to be running the websocket code locally for testing. It is a hassle to remember to start a process and then kill it when you are done, so it would be convenient to start the relay command, then start our websocket worker, and clean up both processes when we quit the worker. This _can_ be done in a Makefile command: {% highlight make %} websocket: $(shell trap … -
Generating a Static Site with Flask and Deploying it to Netlify
This tutorial shows how to leverage the JAMstack with Python and Flask by creating a static site and deploying it to Netlify -
How to Limit Test Time in Django’s Test Framework
I recently optimized a client project’s test suite, and in the process found a test whose runtime had crept up ever since it had been written. The problematic test exercised an import process from a fixed past date until the current day. The test’s runtime therefore grew every day, until it reached over a minute. The solution for that test was to add a date mock using time-machine, so the test created a fixed amount of data. But we also wanted to prevent such a problem occurring again. We came up with the idea of implementing a test time limit that would automatically fail any long running tests. This way, if any appeared again, they would be detected early. The project uses Django’s test framework, which is based on unittest. Based on my spelunking of the unittest’s internals, I figured the best place to add such a limit would be in the TestCase class. The project already had its own customized subclasses of Django’s TestCase classes, so I could add the logic there. Cutting out other details, here is the implementation I came up with: import time from django import test class SlowTestException(Exception): pass class ProjectTestCaseMixin: def _callTestMethod(self, method): start … -
Which Django Version Should I Use?
Django is a big and popular web framework for Python to help you build a web app. Which Django version should you use? If you’re starting a Django web app, use the latest version of Django available. Why listen to me? I’m a software developer with over 7 years of professional Django experience. I work at companies that use Django successfully at the core of their business. I’ll explain why you should use the latest. -
Django News - Django 3.2.1 alpha 1 released - Jan 22nd 2021
News Django 3.2 alpha 1 released Django 3.2 alpha 1 is now available. It represents the first stage in the 3.2 release cycle and is an opportunity for you to try out the changes coming in Django 3.2. djangoproject.com PyCon 2021 Call for Proposals The PyCon 2021 call for proposals is open and closes on February 12, 2021. PyCon US is a massive conference--even virtually--and in recent years the web track, specifically Django, has been underrepresented vs data science/AI/ML talks. Consider applying! pycon.org Articles 5 JavaScript things you should know/understand as a Python Developer by Žan Anderle A number of good tips on better understanding modern JavaScript if you're a Python developer. zanderle.com Testing Python Applications with Pytest [Guide] A look at the most common Pytest configurations and usage, including several Pytest plugins and external libraries. stribny.name It’s time to switch to Docker BuildKit BuildKit is a stable way to dramatically improve your Docker performance when building images. pythonspeed.com Everything you need to know about Python data classes by Josue Balandrano Coronel A very in-depth look at dataclasses, their pros/cons, and multiple use cases with plenty of code examples. rmcomplexity.com 7 concepts you should know to get a job as … -
Reading CloudFlare headers in a Django middleware
For my new Django project, DB Buddy, I’m using CloudFlare as my CDN. It has a bunch of useful features that would otherwise take extra work, such as DDoS protection, HTML minification, and analytics. Since CDN’s proxy your site, all requests are seen to be coming from their servers’ IP’s, rather than actual users’ IP’s. But sometimes you need to inspect real user IP’s, for example to implement rate limiting. To help with this, CloudFlare adds several useful headers, including CF-Connecting-IP with the client’s real IP. In order to provide an interface to get the client’s IP, whether my project is running in development or production, I added a middleware to attach the client IP as request.ip: class CloudflareMiddleware: def __init__(self, get_response): self.get_response = get_response def __call__(self, request): try: ip = request.headers["CF-Connecting-IP"] except KeyError: ip = request.META["REMOTE_ADDR"] request.ip = ip return self.get_response(request) Although the middleware is short, I still made sure to write some tests. I based these on my subclass of Django’s SimpleTestCase, since no DB queries should be made. To generate example requests, I used Django’s RequestFactory. The end result was this test case with one test for each branch in the middleware: from django.http import HttpResponse from … -
Setting up a headless Raspberry Pi
Notes to my future self on setting up a headless Raspberry Pi. -
Mercurial mirror for Django 3.1 branch
As usual when Django releases a new Alpha version, I’ve settled a new mirror. The URL (both web and clone) is https://hg.freehackers.org/mirrors/django-3.2-production As stated before, I no longer use bitbucket as they’ve dropped support for mercurial. Shame on them ! 🙂 -
Simple In-Memory Caching of Django Model Data With cachetools
A client project recently was suffering from an N+1 queries problem in a complicated Django admin page. Many measures had already been taken to prevent N+1 queries, such as use of django-auto-prefetch and some manually tuned select_related() / prefetch_related() calls. The remaining N+1 in question was a bit resistant to those methods because it came through several layers of admin code and many-to-many fields, making it harder than normal to find the place to modify the QuerySet construction. Instead of spelunking through those layers of code to fix this N+1, I took a “shortcut” with some local in-memory caching. This was possible because of the particular model. The model in question represents a county in the United States: class County(auto_prefetch.Model): state = auto_prefetch.ForeignKey(State, on_delete=models.CASCADE) name = models.CharField(max_length=30) ... def __str__(self): return f"{self.name}-{self.state.abbreviation}" The N queries came from the __str__() method, which is used to display a given County in the admin. A list of N counties was selected from a related model without using select_related() or prefetch_related() to select the related states at the same time. Therefore Django performed an extra query to select the state when calling __str__() on each county. The page in question displayed a list of … -
Customer Requests - Building SaaS #88
In this episode, I worked on some customer requests now that I’ve finished launching the product. These requests improved the usability of the application in a few spots that were lacking. The first request from the customer was to make it clear on the daily view when tasks are graded or not. Before I could change the template, I need to add a new method to the CourseTask to check if the task is graded. -
Django Girls - Rachell Calhoun
RachellCalhoun.comDjangoGirls.orgGitHub profileToptal scholarship winnerDjango Girls new leadershipDjangoCon US 2016: Git in Control, Version Control and How it Could... by Rachell CalhounSupport the ShowThis podcast is a labor of love and does not have any ads or sponsors. To support the show, consider purchasing or recommending a book from LearnDjango.com or signing up for the free weekly Django News newsletter. -
Efficient Reloading in Django’s Runserver With Watchman
If you start the development server on a Django project, it looks something like this: $ python manage.py runserver Watching for file changes with StatReloader Performing system checks... System check identified no issues (1 silenced). January 20, 2021 - 04:25:31 Django version 3.1.5, using settings 'db_buddy.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C. When you make changes to a Python file, the server automatically reloads. This is powered by a file watcher, and Django reports which one it’s using in the first line, defaulting to StatReloader. The StatReloader class is simple but reliable. It works by running a loop that checks all your files for changes every second - this is pretty inefficient, especially as your project grows! A lesser-known but better alternative is to use Django’s support for watchman. Support was added in Django 2.2, thanks to Tom Forbes in Ticket #27685. Watchman is an efficient file watcher open sourced by Facebook. It works by receiving file change notifications from your operating system and bundling them together. When nothing is changing, it doesn’t need to do any work - saving processing power and consequently laptop battery life. And when something changes, Django gets notified about it … -
Efficient Reloading in Django’s Runserver With Watchman
If you start the development server on a Django project, it looks something like this: $ python manage.py runserver Watching for file changes with StatReloader Performing system checks... System check identified no issues (1 silenced). January 20, 2021 - 04:25:31 Django version 3.1.5, using settings 'db_buddy.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C. When you make changes to a Python file, the server automatically reloads. This is powered by a file watcher, and Django reports which one it’s using in the first line, defaulting to StatReloader. The StatReloader class is simple but reliable. It works by running a loop that checks all your files for changes every second - this is pretty inefficient, especially as your project grows! A lesser-known but better alternative is to use Django’s support for watchman. Support was added in Django 2.2, thanks to Tom Forbes in Ticket #27685. Watchman is an efficient file watcher open sourced by Facebook. It works by receiving file change notifications from your operating system and bundling them together. When nothing is changing, it doesn’t need to do any work - saving processing power and consequently laptop battery life. And when something changes, Django gets notified about it … -
Scheduling Celery Tasks in the (far) future
We used to make use of the fact that a celery task can be scheduled at some time in the future to auto-punch-out staff members who failed to punch out 24 hours after their shift started. This was as simple as scheduling a task with an `eta=86400`. However, as Adam points out [here](https://adamj.eu/tech/2020/02/03/common-celery-issues-on-django-projects/) (see number 5). This is not a great idea. For one, it will bog down your celery workers (and make them use a bunch more memory). Secondly, something Adam doesn't mention is that if your queue is corrupted, then all of those future tasks will not be executed. Discussing this in IRC today, I thought of a simple mechanism for scheduling these tasks and processing them at some point after they are due. We will start with a model to store our task: {% highlight python %} class ScheduledTask(models.Model): task_name = models.TextField() task_args = models.JSONField(default=list) task_kwargs = models.JSONField(default=dict) due_date = models.DateTimeField() objects = ScheduledTaskQuerySet.as_manager() @property def task(self): module, task = task_name.rsplit('.', 1) return getattr(importlib.import_module(module), task) def execute(self): self.task.apply_async(args=self.args, kwargs=self.kwargs) {% endhighlight %} We have a custom queryset defined, that allows us to see which tasks are due using a nice queryset method: {% highlight python %} class … -
Scheduling Celery Tasks in the (far) future
We used to make use of the fact that a celery task can be scheduled at some time in the future to auto-punch-out staff members who failed to punch out 24 hours after their shift started. This was as simple as scheduling a task with an `eta=86400`. However, as Adam points out [here](https://adamj.eu/tech/2020/02/03/common-celery-issues-on-django-projects/) (see number 5). This is not a great idea. For one, it will bog down your celery workers (and make them use a bunch more memory). Secondly, something Adam doesn't mention is that if your queue is corrupted, then all of those future tasks will not be executed. Discussing this in IRC today, I thought of a simple mechanism for scheduling these tasks and processing them at some point after they are due. We will start with a model to store our task: {% highlight python %} class ScheduledTask(models.Model): task_name = models.TextField() task_args = models.JSONField(default=list) task_kwargs = models.JSONField(default=dict) due_date = models.DateTimeField() objects = ScheduledTaskQuerySet.as_manager() @property def task(self): module, task = task_name.rsplit('.', 1) return getattr(importlib.import_module(module), task) def execute(self): self.task.apply_async(args=self.args, kwargs=self.kwargs) {% endhighlight %} We have a custom queryset defined, that allows us to see which tasks are due using a nice queryset method: {% highlight python %} class … -
[Django]Composite Primary Key
０．Do Django models support composite primary key? https://docs.djangoproject.com/en/3.1/faq/models/#do-django-models-support-multiple-column-primary-keys https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys Django doesn’t officially support composite primary key. If there is no field for which “primary_key = True” is specified, “id” will be automatically added as the primary key. So if you are using the DB of the conventional system, “update” and “delete” method do not work. You have to add “id” field to your tables or to make your own SQL query. １．Solutions Proposal１：Add a surrogate key If I add a surrogate key (‘id’) to the DB and put a unique constraint (unique_together) on the composite key, I can use the Django model as it is. (Problems) DB rebuild takes time and requires planned outage. Proposal２：Implement with your own query Make my own query-write implementation without relying on ORM. I can control the performance of queries by yourself. (Problems) More code and testing. Proposal３：Extend Django Model It seems that it was also considered in the Django project, but I’m not sure recent status. https://code.djangoproject.com/ticket/373 https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys#CurrentStatus It seems that some people have tried various ways, but in the end, are they all unfinished? For the time being, by extending the model myself, create a subclass of Model so that I can use … -
Adding Social Authentication to Django
This tutorial details how to set up social auth with Django and Django Allauth.
