Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
How to maintain user session across sub domains in Django
Nowadays, people are using wildcard domains to provide same user experience across different domains. Using subdomains, we can be able to host multiple sites with the same domain name with the same code in a less time. To enable wildcard subdomains for your site, you should add CNAME record like. *.testsite.com CNAME YOUR_IP_ADDRESS With this, you can be able to access your site domain with abc.testsite.com, xyz.testsite.com, etc. After enabling wildcard domains, you should update your nginx file to accept wildcard domain requests. server { listen YOUR_IP_ADDRESS; server_name testsite.com, *.testsite.com; ... } With the above nginx settings, you're able to access requests from wildcard domains like normal domain. Now you've different wildcard domains for your site(testsite.com). When user login into testsite.com, redirects to the wildcard domain(abc.testsite.com) we should maintain same user session across subdomains. By default, django application isn't maintaining same user session across subdomains. To maintain this, we should add following details to the application settings file. SESSION_COOKIE_DOMAIN = ".testsite.com" DOMAIN_NAME = "testsite.com" Here SESSION_COOKIE_DOMAIN should start with '.' character followed by DOMAIN_NAME. This will handle and enable user session/cookies across wildcard domains(abc.testsite.com, xyz.testsite.com, testsite.com) Note: In local development, your … -
Caching tricks with Cloudflare workers
*tl;dr: Cloudflare allows you to vary their cache responses by arbitrary headers, such as Cookies or User-Agent. This requires you to have an enterprise account (~$5000). You could either pay $5000 or solve the problem with $5 and some Javascript code.* ## Intro Cloudflare offers caching on their global CDN. For sites that rarely update Cloudflare handles most of the traffic for you without ever reaching your origin server. Once a user visits a specific page, Cloudflare keeps the page response in the cache and serves it for the next visitor. This reduces the load on your servers while also improving the page performance since it's served closer to the user via one of [Cloudflare's POPs](https://www.cloudflare.com/en-au/network/). While this is great in theory, it's challenging to implement for any site that requires page customization per visitor (read: most sites). For example: we may want to serve different cache responses based on the cookie (unique caching per visitor) or on the User-Agent (unique caching per device type - mobile / tablet / desktop). Luckily, Cloudflare allows you to vary the cache by HTTP headers: - `Accept-Encoding` - caches each resource by the encoding of the payload. - `Cookie` - allows the cache … -
Writing JSONField from scratch
Last week, I improved an existing Django package and implemented some SQLite lookups for its JSONField to get some sense of how Django model fields work. This week, I tried to create a unified JSONField from scratch. It’s unified in the sense that it works on all database backends supported by Django. I also set up docker containers to run Django’s test suite with all database backends with the help of django-docker-box (though I ended up making my own for Oracle since it’s a bit more complex). However, I’m just going to talk about writing the field as that’s more interesting. As before, the docs on how to write custom model fields is very handy. I started building my own JSONField and got it working in a relatively short time. Not without bugs and quirks I had to deal with on each database backend, though. First, I wrote the code with SQLite as the top priority. I overrode the db_type method so it returns json. It’s also the name of the data type used in MySQL and MariaDB for JSON values, so I guess it’s a good start. def db_type(self, connection): return 'json' This means that the SQL query for … -
Drafting a POC JSONField for SQLite
There’s this neat little python package called django-jsonfallback. It’s made by one of my mentors. To quote the description, this package allows you to Use PostgreSQL’s and MySQL’s JSONField, but fall back to a TextField implementation on other databases. Since my project is to bring a cross-DB JSONField, this project is very useful for me as a starting point. To get a better understanding of how that package works (and how Django model fields work in general), I did a good read on these wonderful Django docs about custom model fields and custom lookups. I also found this interesting Custom Database Backends presentation from Django Under the Hood 2016 event. As explained in the docs and presentation, you can create as_vendor methods such as as_mysql, as_sqlite, etc. to create different implementations of lookups and transforms (it also applies for any class that extends BaseExpression, if I’m not mistaken). I began looking at the code in django-jsonfallback, and I found that it was made with some if conditionals to check for the database backend. I tried replacing the as_sql methods with as_mysql as you can see in this pull request, and it worked pretty well. I also tried to replace the … -
Community Bonding and getting started
In the official GSoC timeline, there’s a Community Bonding period on May 7 - 27. To quote the glossary, it’s The period of time between when accepted students are announced and the time these students are expected to start coding. This time is an excellent one to introduce students to the community, get them on the right mailing lists, working with their mentors on their timeline for the summer, etc. At the start this period, we’re also given emails from Google about what we need to do. We are tasked to set up a Payoneer account to receive our stipend later. We were also invited to a mailing list for current (and past) students. Some students decided to create a Telegram group for discussions with other students all over the world. I posted on the django-developers mailing list to ask about communication channels we’ll use during the program, and one of my mentors said that he will send an email to get me (and another student) started. On May 13, my mentors (there are four) sent introductory emails on how to get started. The plans are quite similar to what I put in my proposal, so I guess I’m on … -
Python in Azure Pipelines, Step by Step
Since the acquisition of Travis CI, the future of their free offering is unclear. Azure Pipelines has a generous free tier, but the examples I found are discouragingly complex and take advantage of features like templating that most projects don’t need. To close that gap, this article shows you how to move a Python project with simple CI needs from Travis CI to Azure Pipelines. -
Build an XML sitemap of XML sitemaps
Suppose that you have so many thousands of pages that you can't just create a single /sitemap.xml file that has all the URLs (aka <loc>) listed. Then you need to make a /sitemaps.xml that points to the other sitemap files. And if you're in the thousands, you'll need to gzip these files. The blog post demonstrates how Song Search generates a sitemap file that points to 63 sitemap-{M}-{N}.xml.gz files which spans about 1,000,000 URLs. The context here is Python and the getting of the data is from Django. Python is pretty key here but if you have something other than Django, you can squint and mentally replace that with your own data mapper. Generate the sitemap .xml.gz file(s) Here's the core of the work. A generator function that takes a Django QuerySet instance (that is ordered and filtered!) and then starts generating etree trees and dumps them to disk with gzip. import gzip from lxml import etree outfile = "sitemap-{start}-{end}.xml" batchsize = 40_000 def generate(self, qs, base_url, outfile, batchsize): # Use `.values` to make the query much faster qs = qs.values("name", "id", "artist_id", "language") def start(): return etree.Element( "urlset", xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" ) def close(root, filename): with gzip.open(filename, "wb") as f: f.write(b'<?xml … -
Build an XML sitemap of XML sitemaps
Suppose that you have so many thousands of pages that you can't just create a single /sitemap.xml file that has all the URLs (aka <loc>) listed. Then you need to make a /sitemaps.xml that points to the other sitemap files. And if you're in the thousands, you'll need to gzip these files. The blog post demonstrates how Song Search generates a sitemap file that points to 63 sitemap-{M}-{N}.xml.gz files which spans about 1,000,000 URLs. The context here is Python and the getting of the data is from Django. Python is pretty key here but if you have something other than Django, you can squint and mentally replace that with your own data mapper. Generate the sitemap .xml.gz file(s) Here's the core of the work. A generator function that takes a Django QuerySet instance (that is ordered and filtered!) and then starts generating etree trees and dumps them to disk with gzip. import gzip from lxml import etree outfile = "sitemap-{start}-{end}.xml" batchsize = 40_000 def generate(self, qs, base_url, outfile, batchsize): # Use `.values` to make the query much faster qs = qs.values("name", "id", "artist_id", "language") def start(): return etree.Element( "urlset", xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" ) def close(root, filename): with gzip.open(filename, "wb") as f: f.write(b'<?xml … -
Generate a random IP address in Python
A fun trick to generate random, but seeded, IPv4 addresses in a Django app. -
Generate a random IP address in Python
A fun trick to generate random, but seeded, IPv4 addresses in a Django app. -
Bitbucket Pipelines and Ansible: Continuous delivery for your Django project
In a previous post I described how to deploy a Django project using Ansible. Here I explain how to make a step further and deploy a Django project using Bitbucket Pipelines and Ansible. Bitbucket Pipelines are basically docker containers, hosted in the Bitbucket infrastructure, that you can run to build or deploy your code, attaching them to “events” happening in your repository. Usually a push on one branch. You can clone or fork this Bitbucket repository to follow along with this tutorial. First thing first, add a file named bitbucket-pipelines.yml to the root of your repository. The content of the file should be similar to this: # use the official Python 2.7.16 docker image image: python:2.7.16 pipelines: branches: # deploy only when pushing to the master branch master: - step: name: Deploy to prod # this is the name of the Bitbucket Deployment deployment: Production caches: # cache the Ansible installation - pip script: # install Ansible - pip install ansible==2.8.0 # go into the ansible directory # in this same repository - cd ansible # perform the actual deploy - ansible-playbook -i ./hosts deploy.yaml Let’s examine the configuration line by line: On line 2 you define the docker image … -
Celery Once
Within our audit logs, which happen at the database level, we also want to be able to show the user a representation of what the object "looks like": basically, what `str(instance)` is. This is shown as a supplementary thing, and often makes it easier to identify which object has been edited. Otherwise, the audit logs only show the things that have changed on the object in that event. However, because this happens in python/django, and not in the database, we need some mechanism for fetching audit logs that don't have this string representation, and then update those. This is the perfect use case for celery: we can have a periodic task that looks for audit logs missing this related object, and then creates them. There are a few things that can cause problems: * If we run the task infrequently, then there can at times be a large number of new audit logs, which can cause this task to take a long time to run: blocking other tasks, or perhaps even timing out. We should limit the number of objects that may be handled in a given task. * If we run the task infrequently, there can be a big … -
Django Tips: Recovering Gracefully From ORM Errors
Django views are the glue between your users and the underlying database. When a user visits an url Django can map that url with a view. And most of the times the view is also responsible for fetching some data from the database. Consider the following example. There is a model named Workshop which among the others has a slug field: """ models.py """ from django.db import models class Workshop(models.Model): """ ... some other field here """ slug = models.SlugField(max_length=20) """ ... some other field here """ def __str__(self): return f'{self.title}' You may want to use that slug for fetching the appropriate entity from the database when the user visits someurl/slugname/. That means you will have an url declaration like so: """ urls.py of your Django app """ from django.urls import path from .views import workshop_detail urlpatterns = [ path('someurl/<slug:slug>/', workshop_detail) ] At this point you're ready to create a Django view for displaying the model detail. I'll use a function view for keeping things easy. A naive implementation for the view could be the following: """ views.py of your Django app """ from django.shortcuts import render from .models import Workshop def workshop_detail(request, slug): workshop = Workshop.objects.get(slug=slug) context = { … -
The Simplest WSGI Middleware
My library apig-wsgi bridges between AWS API Gateway’s JSON format for HTTP requests and Python WSGI applications. Recently Théophile Chevalier opened an issue requesting the library add an extra WSGI environ variable. I closed it by pointing out that it’s not much code to add a custom WSGI middleware to do so (plus the exact key is a bit out of scope for the library). I guess most Python web developers don’t touch WSGI day to day, so I figured I’d write this short post to share the knowledge. A Quick Shot of WSGI The WSGI specification defines an application as a callable takes two positional parameters. These parameters are, with their conventional names: environ, a dict of variables that describe the HTTP request. start_response, a function to call to call to start generation of the HTTP response. The application is called for each request, should call start_response and then return an iterable representing the contents of the HTTP response body. There’s a lot of specification around this, but to implement the simplest middleware we don’t need to dive into that. We want to only change environ and pass everything on to the proxied application. If we have original_app pointing … -
Creating Evscaperoom, part 2
The Jester, your 'adversary'This is part two of my post-mortem dev-blog about Evscaperoom, the multiplayer, text-based 'escape room' I wrote in Python and Evennia. You can read the first part of the dev blog here. This was a game-jam entry I created in a month for the Mud Coder's guild's Game Jam. The theme was One Room. You can play the game for free in your browser or with a traditional MUD client. There are no spoilers in these blog posts. The first part dealt with the overall game-design aspects. This second, final part will go into details of the code and the systems I built to quickly create all the content. The code referenced here is released under the BSD license and is available on github.At the time of this post, players have played Evscaperoom for a little more than a week. At the end I'll share some observations and things I learned along the way.Ease of buildingOver the one-month game jam, I spent about four days making the game's 'engine' and toolset with Evennia. The rest of the time was spent using those tools to actually create game content (the story, puzzles etc).An important thing was that I … -
Postgres Generated Columns
A little while ago, I wrote about creating a nice way to have a [Django ComputedField](https://schinckel.net/2018/11/12/django-computedfield%28%29/). It is pretty neat, except it needs to do some black magic to sniff up the stack to work around a limitation in the way a Ref/Col works in Django. The way it works is that you define the expression in Python, and it evaluates it in the database, allowing you to query based on this, and have it automatically annotated on. What it _doesn't_ do, however, is actually store that value in the database. Indeed, if you are actually querying on this column, you'd probably want to have a functional index that uses the same expression, so that the database can do a reasonable job of improving query times on that column. New in Postgres 12 is a feature that really piqued my interest: [Generated Columns](https://www.postgresql.org/docs/12/ddl-generated-columns.html). These are basically what the ComputedField does, but at the database level. And, instead of it being an expression that is evaluated at query time, it is instead an expression that is evaluated at write time, and stored in an actual column (that could then have an index applied to it). Let's have a look at an … -
The Best Animation and Prototyping Tools for Designers in 2019
Prototyping is a great way to showcase your product before embarking on a long development process. It gives you the opportunity to make changes prior to development. It also saves you significant time and money. The post The Best Animation and Prototyping Tools for Designers in 2019 appeared first on Distillery. -
Creating Evscaperoom, part 1
Over the last month (April-May 2019) I have taken part in the Mud Coder's Guild Game Jam "Enter the (Multi-User) Dungeon". This year the theme for the jam was One Room.The result was Evscaperoom, an text-based multi-player "escape-room" written in Python using the Evennia MU* creation system. You can play it from that link in your browser or MU*-client of choice. If you are so inclined, you can also vote for it here in the jam (don't forget to check out the other entries while you're at it).This little series of (likely two) dev-blog entries will try to recount the planning and technical aspects of the Evscaperoom. This is also for myself - I'd better write stuff down now while it's still fresh in my mind!Inception When I first heard about the upcoming game-jam's theme of One Room, an 'escape room' was the first thing that came to mind, not the least because I just recently got to solve my my own first real-world escape-room as a gift on my birthday. If you are not familiar with escape-rooms, the premise is simple - you are locked into a room and have to figure out a way to get out of it by solving practical … -
New features planned for Python 4.0
With the release of Python 3.8 coming soon, the core development team has asked me to summarize our latest discussions on the new features planned for Python 4.0, codename "ouroboros: the snake will eat itself". This will be an exciting release and a significant milestone, many thanks to the hard work of over 100 contributors. After heated debate on the mailing list, the 79-character line limit prescribed by PEP8 will be updated. IDE users all over the world will now be able to take advantage of their 30" ultra-wide 4K monitors, as the recommended line length will be increased to 89.5 characters (this was a compromise with the 100-character lobby, the decision being to split the difference). All new libraries and standard lib modules must include the phrase "for humans" somewhere in their title, and have a splashy documentation page with lots of fonts typography and testimonials. Finally, a new string-type for the masses, Python 4.0 will feature "z-strings": C-style NULL terminated bytestrings. Just prefix your string with z'my string' and Python will automatically ensure it is NULL-terminated. Note: the new z-strings cannot be used with any of the existing APIs that take string arguments - they must first be … -
New features planned for Python 4.0
With the release of Python 3.8 coming soon, the core development team has asked me to summarize our latest discussions on the new features planned for Python 4.0, codename "ouroboros: the snake will eat itself". This will be an exciting release and a significant milestone, many thanks to the hard work of over 100 contributors. After heated debate on the mailing list, the 79-character line limit prescribed by PEP8 will be updated. IDE users all over the world will now be able to take advantage of their 30" ultra-wide 4K monitors, as the recommended line length will be increased to 89.5 characters (this was a compromise with the 100-character lobby, the decision being to split the difference). All new libraries and standard lib modules must include the phrase "for humans" somewhere in their title. Finally, a new string-type for the masses, Python 4.0 will feature "z-strings": C-style NULL terminated bytestrings. Just prefix your string like so, z'my string' and Python will automatically ensure it is NULL-terminated. Note: the new z-strings cannot be used with any of the existing APIs that take string arguments - they must first be decoded to unicode strings or cast to bytes. Type-hinting has been extended … -
<p>django-debug-toolbar 2.0a1 is now <a target="_blank" rel="nofollow" href="https://pypi.org/project/django-debug-toolbar/2.0a1/">available on PyPI</a>. Please help with testing or with success stori
django-debug-toolbar 2.0a1 is now available on PyPI. Please help with testing or with success stories or with bug reporting & squashing. -
The Price of the Hallway Track
There are many good reasons to not go to every talk possible when attending conferences. However increasingly it became hip to boast with not going to talks at all – encouraging others to follow suit. As a speaker, that rubs me the wrong way and I’ll try to explain why. -
5 Ways Software Outsourcing Helps FinTech Startups Compete and Win
As many FinTech startups have already learned, the right development partner is a massive competitive advantage. Don’t get left behind. Instead, let outsourcing help you accelerate. The post 5 Ways Software Outsourcing Helps FinTech Startups Compete and Win appeared first on Distillery. -
Django + Tailwind CSS = ❤️
If you use Tailwind CSS and want to integrate it easily with Django, check out a package I created about a month ago. Surprisingly it's called Django-Tailwind. You can find it here on GitHub: https://github.com/timonweb/django-tailwind. If you don't know what Tailwind CSS is, then ... Read now -
Query Zen is no queries at all
Performing no queries is always going to be faster than performing a query. Today I had two instances of the same problem: I have two tables, one of which essentially stores calculated data based on other data (and data in other tables, or involving a process that uses application code, and cannot be purely determined within the database). In one case, we have an audit logging table (which is purely handled within postgres) and another related table that stores a string representation of what the audited object looked like according to the application at that point in time, which needs to be calculated after the fact in Django. The other case stores some cached values that _can_ be calculated in the database: basically some metadata about a shift according to the location that the shift is at. Changes to the shift table will cause this value to automatically be updated, however we have several million shifts that do not currently have this value, but we need to create items for all shifts that currently don't have the annotation. In both cases, we have a celery task that will create a (relatively small, to prevent locks and other performance issues) number …
