Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
UX Polish - Building SaaS #90
In this episode, I worked on more customer requests to refine the user interface. We added a new convenient shortcut button and included additional information and links on the course detail view. I’m continuing on my path to include the changes that my customer wants as she running a school year for the first time. The first change was to add a new button on the student’s view of a course. -
Using Django Check Constraints to Limit A Model to a Single Instance
Yet another use case for creating a database constraint with Django’s CheckConstraint class. Sometimes it’s useful to have a model with only have one instance in the database, sometimes known as a singleton. This is useful for storing a small amount of structured data that we want to share between all our project’s processes. For example, imagine a remote API that we authenticate with using a temporary access token. We have the username and password for the API in our Django settings, and use those to get a temporary access token. We then need to store that temporary access token for all operations with that API, and refresh it when it nears its expiry time. And for security reasons, we are only allowed to store the current access token. We can store the token and its expiry in a model like this: from django.db import models class RemoteAPIAccount(models.Model): access_token = models.CharField(max_length=120) access_token_expires = models.DateTimeField() This model has the right fields for holding the token, but it can also have many instances of it in our database. We can write code that always uses a single instance by always the model through get_or_create() or update_or_create(), and passing all field values through … -
Wagtail, React, & Gatsby - Dawn Wages
Dawn’s personal siteDawn on Twitter @DawnWagesSaysWagtail Documentation SprintAt the Root: Wagtail + Gatsby + GitPod - PyConline AU 2020 talkWagtail CMSDecolonizing Technology on Martin Luther King Day 2021Biometric Insecurity - Carina C. Zona @PyConline AU 2020Support the ShowThis podcast is a labor of love and does not have any ads or sponsors. To support the show, please consider recommending a book from LearnDjango.com, signing up for the free weekly Django News newsletter, or learning more about Button a simpler deployment story for Django. -
Working with Django ImageField
<![CDATA[ Working with Django ImageField Django ImageField is great solution to associate images with your models. It is also a field that I most often have some issues with (because saving images is a bit more complicated than say text or a number). For this reason I decided to write this post as a future reference to myself and hopefully it will help someone else too. To even work with the ImageField you need the Pillow package. pip install Pillow When specifying the field definition, there is an option to select the directory where the images will be stored. For example: header_image = models.ImageField(upload_to='blog-images/') This is a relative path inside the media directory. Make sure you have the MEDIA_ROOT configured in your settings file. Saving images to ImageField There are numerous ways of saving actual image inside this field. The easier option is to use ModelForm, which you can just save (assuming it is valid) in your view and done. However, you need to include the request.FILES while creating the form like this: form = NewImageForm(request.POST, request.FILES) And for a brief visit to the HTML, the <form> needs special definition too in a form of the enctype attribute. <form method="post" … -
Django and Pydantic
This article looks at how to integrate Pydantic with a Django application. -
The Unexpected Find That Freed 20GB of Unused Index Space
Every few months we get an alert from our database monitoring to warn us that we are about to run out of space. Usually we just provision more storage and forget about it, but this time we were under quarantine, and the system in question was under less load than usual. We thought this is a good opportunity to do some cleanups that would otherwise be much more challenging. To start from the end, we ended up freeing more than 70GB of un-optimized and un-utilized space without dropping a single index or deleting any data! Using conventional technics such as rebuilding indexes and tables we cleared up a lot of space, but then one surprising find helped us clear an additional ~20GB of unused indexed values! This is what the free storage chart of one of our databases looked like in the process: Free space over time (higher means more free space) Table of Contents The Usual Suspects Unused Indexes Index and Table Bloat Clearing Bloat in Indexes Activating B-Tree Index Deduplication Clearing Bloat in Tables Using pg_repack The "Find" The "Aha Moment" Utilizing Partial Indexes Bonus: Migrating with Django ORM Prevent Implicit Creation of Indexes on Foreign Keys Migrate … -
Using Django Check Constraints to Prevent the Storage of The Empty String
Here’s another use case for creating a database constraint with Django’s CheckConstraint class. By default, a model CharField will store any string that the database supports. This includes the empty string, which is often inappropriate. For example, imagine we have a Team model, representing a group that users can belong to. We started by giving it a unique name: from django.db import models class Team(models.Model): name = models.CharField(max_length=120) class Meta: constraints = [ models.UniqueConstraint( name="%(app_label)s_%(class)s_name_unique", fields=["name"], ), ] Whilst seemingly reasonable, this model definition allows the creation of teams with an empty name. Such a team object is likely to break other parts of our application. For example, links might not display or unexpected errors may occur due to name being falsey. We could try to prevent the creation of empty-named teams with form validation, but that does not prevent creation of bad data through other processes, such as bulk imports. To ensure all the data in our system is valid, we need a database constraint. Here we want to add a minimal database constraint that ensures the length of name is greater than 0. Let’s walk through that now. First, we need to register the Length function for use … -
SOCCR: the framework I use for decision briefs
In my previous article, I wrote about gathering consensus before a decision. Several folks asked for more detail about how I structure those consensus-gathering and decision-making exercises. There’s a specific format I find helpful, which I remember by the acronym SOCCR. Read on for an explanation and example! -
Are Django and Flask Similar?
Maybe you’re new to web development in Python, and you’ve encountered the two most popular Python web frameworks, Django and Flask, and have questions about which one you should use. Are Django and Flask similar tools for building web applications? Yes, Django and Flask share many similarities and can both make great websites, but they have some different development philosophies which will attract different types of developers. What do I know about this? -
Recap of 2020 and Resolutions for 2021
Recap of 2020 I got a job saving the planet For years I've been worried about climate change. I've tried to live a mostly ecologically friendly life, and this year I dived in with making compost for the garden. However, seeing the lack of consistent political will to correct the problem has had me worried. So I was delighted that when I started to look for jobs in the autumn I was contacted by several firms attempting to address the problem. I've always thought that finance is a great motivator, albeit all-too-often the results are negative (the petroleum and coal industries are examples). Ultimately I went with Octopus Energy, a company that is to borrow a phrase, disrupting the legacy power industry in areas with free-er energy markets (UK, Australia, Germany, Texas, Japan). They have embraced renewable energies as a source of power, which is good because it is cheaper than fossil fuels. Even if customers don't believe in global climate change they won't be able to ignore the fact renewables are a cheaper source of energy. In any case, I can't begin to say how wonderful it is being part of a company whose mission is to address climate … -
Recap of 2020 and Resolutions for 2021
Recap of 2020 I got a job saving the planet For years I've been worried about climate change. I've tried to live a mostly ecologically friendly life, and this year I dived in with making compost for the garden. However, seeing the lack of consistent political will to correct the problem has had me worried. So I was delighted that when I started to look for jobs in the autumn I was contacted by several firms attempting to address the problem. I've always thought that finance is a great motivator, albeit all-too-often the results are negative (the petroleum and coal industries are examples). Ultimately I went with Octopus Energy, a company that is to borrow a phrase, disrupting the legacy power industry in areas with free-er energy markets (UK, Australia, Germany, Texas, Japan). They have embraced renewable energies as a source of power, which is good because it is cheaper than fossil fuels. Even if customers don't believe in global climate change they won't be able to ignore the fact renewables are a cheaper source of energy. In any case, I can't begin to say how wonderful it is being part of a company whose mission is to address climate … -
Developing a Single Page App with Flask and Vue.js
The following is a step-by-step walkthrough of how to set up a basic CRUD app with Vue and Flask. -
Django News - New Wagtail and Documentation Sprint - Jan 29th 2021
News Wagtail Documentation Sprint A Wagtail Sprint is occurring February 4th and 5th, 2021 focused on documentation: tooling, writing, editing, content reorganization, design etc. It will be run from Torchbox’s offices in the UK, where anyone is welcome to join us, if it’s safe and legal, but we expect most people to participate remotely. wagtail.io Wagtail 2.11 just released Features include internationalization and improved image and document management. wagtail.io Articles Efficient Reloading in Django’s Runserver With Watchman Watchman is an efficient file watcher open sourced by Facebook, for which Django support was added in 2.2. In this article, Adam Johnson demonstrates how to use it with runserver for improved performance. adamj.eu Reading CloudFlare headers in a Django middleware Adam breaks down how to find a client’s IP address when using a CDN like Cloudflare. adamj.eu 5 Tips to Speed up Your Docker Image Build Docker is powerful but can be very time consuming. Here are 5 good tips for speeding up your image builds. vsupalov.com Which Django Version Should I Use? Matt Layman provides some advice on thinking about which Django version to use in your projects. mattlayman.com Sponsored Link Tackle Architecture Issues on Your Complex Django Codebase. As a … -
Customer Feature - Building SaaS #89
In this episode, I show you how to take a feature idea from nothing to something. We added new UI, wrote a new view, a new form, and all the associated test code to prove that the feature works. My customer wants the ability to complete a task on any day she desires. The feature flow looks like: Click a calendar icon next to a task on a student’s course page. -
Cleaning up a required process when you quit
Some time ago, I moved a bunch of our devops tools into a Makefile. I'm not sure exactly what the benefit was then, but there have been some challenges. One of these was related to having to run multiple processes when performing a task, but cleaning up the dependencies when we finish our task. Specifically, we run a websocket server in production that uses Redis for PubSub. We can safely assume the developer is running redis already, since the rest of the platform also depends upon it. However, to collect the changes from Postgres and publish them into Redis, we use a custom django management command. All this does is listen for postgres notifications and forward them to redis. $ ./manage.py relay Mostly we don't keep this running in development, unless you need to be running the websocket code locally for testing. It is a hassle to remember to start a process and then kill it when you are done, so it would be convenient to start the relay command, then start our websocket worker, and clean up both processes when we quit the worker. This _can_ be done in a Makefile command: {% highlight make %} websocket: $(shell trap … -
Cleaning up a required process when you quit
Some time ago, I moved a bunch of our devops tools into a Makefile. I'm not sure exactly what the benefit was then, but there have been some challenges. One of these was related to having to run multiple processes when performing a task, but cleaning up the dependencies when we finish our task. Specifically, we run a websocket server in production that uses Redis for PubSub. We can safely assume the developer is running redis already, since the rest of the platform also depends upon it. However, to collect the changes from Postgres and publish them into Redis, we use a custom django management command. All this does is listen for postgres notifications and forward them to redis. $ ./manage.py relay Mostly we don't keep this running in development, unless you need to be running the websocket code locally for testing. It is a hassle to remember to start a process and then kill it when you are done, so it would be convenient to start the relay command, then start our websocket worker, and clean up both processes when we quit the worker. This _can_ be done in a Makefile command: {% highlight make %} websocket: $(shell trap … -
Generating a Static Site with Flask and Deploying it to Netlify
This tutorial shows how to leverage the JAMstack with Python and Flask by creating a static site and deploying it to Netlify -
How to Limit Test Time in Django’s Test Framework
I recently optimized a client project’s test suite, and in the process found a test whose runtime had crept up ever since it had been written. The problematic test exercised an import process from a fixed past date until the current day. The test’s runtime therefore grew every day, until it reached over a minute. The solution for that test was to add a date mock using time-machine, so the test created a fixed amount of data. But we also wanted to prevent such a problem occurring again. We came up with the idea of implementing a test time limit that would automatically fail any long running tests. This way, if any appeared again, they would be detected early. The project uses Django’s test framework, which is based on unittest. Based on my spelunking of the unittest’s internals, I figured the best place to add such a limit would be in the TestCase class. The project already had its own customized subclasses of Django’s TestCase classes, so I could add the logic there. Cutting out other details, here is the implementation I came up with: import time from django import test class SlowTestException(Exception): pass class ProjectTestCaseMixin: def _callTestMethod(self, method): start … -
Which Django Version Should I Use?
Django is a big and popular web framework for Python to help you build a web app. Which Django version should you use? If you’re starting a Django web app, use the latest version of Django available. Why listen to me? I’m a software developer with over 7 years of professional Django experience. I work at companies that use Django successfully at the core of their business. I’ll explain why you should use the latest. -
Django News - Django 3.2.1 alpha 1 released - Jan 22nd 2021
News Django 3.2 alpha 1 released Django 3.2 alpha 1 is now available. It represents the first stage in the 3.2 release cycle and is an opportunity for you to try out the changes coming in Django 3.2. djangoproject.com PyCon 2021 Call for Proposals The PyCon 2021 call for proposals is open and closes on February 12, 2021. PyCon US is a massive conference--even virtually--and in recent years the web track, specifically Django, has been underrepresented vs data science/AI/ML talks. Consider applying! pycon.org Articles 5 JavaScript things you should know/understand as a Python Developer by Žan Anderle A number of good tips on better understanding modern JavaScript if you're a Python developer. zanderle.com Testing Python Applications with Pytest [Guide] A look at the most common Pytest configurations and usage, including several Pytest plugins and external libraries. stribny.name It’s time to switch to Docker BuildKit BuildKit is a stable way to dramatically improve your Docker performance when building images. pythonspeed.com Everything you need to know about Python data classes by Josue Balandrano Coronel A very in-depth look at dataclasses, their pros/cons, and multiple use cases with plenty of code examples. rmcomplexity.com 7 concepts you should know to get a job as … -
Reading CloudFlare headers in a Django middleware
For my new Django project, DB Buddy, I’m using CloudFlare as my CDN. It has a bunch of useful features that would otherwise take extra work, such as DDoS protection, HTML minification, and analytics. Since CDN’s proxy your site, all requests are seen to be coming from their servers’ IP’s, rather than actual users’ IP’s. But sometimes you need to inspect real user IP’s, for example to implement rate limiting. To help with this, CloudFlare adds several useful headers, including CF-Connecting-IP with the client’s real IP. In order to provide an interface to get the client’s IP, whether my project is running in development or production, I added a middleware to attach the client IP as request.ip: class CloudflareMiddleware: def __init__(self, get_response): self.get_response = get_response def __call__(self, request): try: ip = request.headers["CF-Connecting-IP"] except KeyError: ip = request.META["REMOTE_ADDR"] request.ip = ip return self.get_response(request) Although the middleware is short, I still made sure to write some tests. I based these on my subclass of Django’s SimpleTestCase, since no DB queries should be made. To generate example requests, I used Django’s RequestFactory. The end result was this test case with one test for each branch in the middleware: from django.http import HttpResponse from … -
Setting up a headless Raspberry Pi
Notes to my future self on setting up a headless Raspberry Pi. -
Mercurial mirror for Django 3.1 branch
As usual when Django releases a new Alpha version, I’ve settled a new mirror. The URL (both web and clone) is https://hg.freehackers.org/mirrors/django-3.2-production As stated before, I no longer use bitbucket as they’ve dropped support for mercurial. Shame on them ! 🙂 -
Simple In-Memory Caching of Django Model Data With cachetools
A client project recently was suffering from an N+1 queries problem in a complicated Django admin page. Many measures had already been taken to prevent N+1 queries, such as use of django-auto-prefetch and some manually tuned select_related() / prefetch_related() calls. The remaining N+1 in question was a bit resistant to those methods because it came through several layers of admin code and many-to-many fields, making it harder than normal to find the place to modify the QuerySet construction. Instead of spelunking through those layers of code to fix this N+1, I took a “shortcut” with some local in-memory caching. This was possible because of the particular model. The model in question represents a county in the United States: class County(auto_prefetch.Model): state = auto_prefetch.ForeignKey(State, on_delete=models.CASCADE) name = models.CharField(max_length=30) ... def __str__(self): return f"{self.name}-{self.state.abbreviation}" The N queries came from the __str__() method, which is used to display a given County in the admin. A list of N counties was selected from a related model without using select_related() or prefetch_related() to select the related states at the same time. Therefore Django performed an extra query to select the state when calling __str__() on each county. The page in question displayed a list of … -
Customer Requests - Building SaaS #88
In this episode, I worked on some customer requests now that I’ve finished launching the product. These requests improved the usability of the application in a few spots that were lacking. The first request from the customer was to make it clear on the daily view when tasks are graded or not. Before I could change the template, I need to add a new method to the CourseTask to check if the task is graded.
