Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
5 Communication Mistakes That Are Hurting Your IT Outsourcing Relationship
When done right, IT outsourcing can provide businesses with a massive competitive advantage, powering innovation, sustainable scaling, cost savings, budget/staffing flexibility, and countless other benefits. It makes IT leaders’ lives easier, making them look good while driving value for the business. Again, however, that’s when it’s done right. So how can you avoid doing it wrong? Much of the success of any IT outsourcing relationship boils down to a single factor: effective communication. Communication mistakes are incredibly common in outsourcing. Below, we’ve broken down five of the most common mistakes, explaining the problems, their potential impact, and how you can avoid them. Communication Mistake #1: Micromanaging The Problem It’s your business and therefore your baby. It’s not surprising that it can be difficult to give up control over how things get done. So you become overly prescriptive in telling your IT outsourcing provider how they should do your work. After all, you don’t want to risk they’ll do things wrong. The Damage Reduced productivity, efficiency, and quality. Your IT outsourcing provider is increasingly hamstrung in how they can serve you. You’ve narrowed the definitions of what’s acceptable so much that it’s causing an entirely new set of problems. As they … -
Optimizing Postgres full text search in Django
Postgres provides great search capability out of the box. For the majority of Django apps there is no need to run and maintain an ElasticSearch cluster unless you need the advanced features ElasticSearch offers. Django integrates nicely with the Postgres search through the built in [Postgres module](https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/). For small data sets the default configuration performs well, however when your data grows in size the default search configuration becomes painfully slow and we need to enable certain optimizations to keep our queries fast. This page will walk you through setting up Django and Postgres, indexing sample data and performing and optimizing full text search. The examples go through a Django + Postgres setup, however the advice is generally applicable to any programming language or framework as long as it uses Postgres. **If you're already a Django veteran you can skip the first steps and jump right into the optimization.** # Table of Contents * [Project setup](#project-setup) * [Creating models and indexing sample data](#creating-models) * [Optimizing search](#optimizing-search) * [Specialized search column and gin indexes](#specialized-column) * [Postgres triggers](postgres-triggers) * [Measuring the performance improvement](#measuring) * [Drawbacks](#drawbacks) * [Conclusion](#conclusion) ## Project setup Create the directories and setup the Django project. ```bash mkdir django_postgres cd django_postgres … -
Django Favicon Tutorial
How to add a favicon to any Django website. -
Types of Outsourcing: How to Choose the Best Solution for Your Business
To stay competitive, more businesses than ever are turning to outsourcing. These businesses know that leveraging one or more types of outsourcing helps them: Execute and innovate faster. From 2016 to 2018, Deloitte saw an increase from 20% to 49% in the number of organizations moving to outsourced providers as they innovate. Source top talent with the right expertise. Gartner’s 2017 and 2016 CIO Agenda Reports identified talent as the “single biggest issue standing in the way of CIOs achieving their objectives.” Make smarter use of their resources. As the Harvard Business Review states, “Full-time employees are the most expensive and least flexible source of labor.” That’s why smart businesses are making strategic use of outsourced, contract workers to fill gaps in expertise, meet fluctuating demands, and manage certain projects. Did you know that Microsoft and Google both have contract workforces that either equal or exceed their in-house headcounts? Done correctly, outsourcing can be your secret weapon to ongoing innovation, rapid releases, and sustainable business success. This guide will help you gain a stronger grasp on how different types of outsourcing models really work, so you can accurately assess which model is the right fit for your needs. We’ll start … -
How to maintain user session across sub domains in Django
Nowadays, people are using wildcard domains to provide same user experience across different domains. Using subdomains, we can be able to host multiple sites with the same domain name with the same code in a less time. To enable wildcard subdomains for your site, you should add CNAME record like. *.testsite.com CNAME YOUR_IP_ADDRESS With this, you can be able to access your site domain with abc.testsite.com, xyz.testsite.com, etc. After enabling wildcard domains, you should update your nginx file to accept wildcard domain requests. server { listen YOUR_IP_ADDRESS; server_name testsite.com, *.testsite.com; ... } With the above nginx settings, you're able to access requests from wildcard domains like normal domain. Now you've different wildcard domains for your site(testsite.com). When user login into testsite.com, redirects to the wildcard domain(abc.testsite.com) we should maintain same user session across subdomains. By default, django application isn't maintaining same user session across subdomains. To maintain this, we should add following details to the application settings file. SESSION_COOKIE_DOMAIN = ".testsite.com" DOMAIN_NAME = "testsite.com" Here SESSION_COOKIE_DOMAIN should start with '.' character followed by DOMAIN_NAME. This will handle and enable user session/cookies across wildcard domains(abc.testsite.com, xyz.testsite.com, testsite.com) Note: In local development, your … -
Caching tricks with Cloudflare workers
*tl;dr: Cloudflare allows you to vary their cache responses by arbitrary headers, such as Cookies or User-Agent. This requires you to have an enterprise account (~$5000). You could either pay $5000 or solve the problem with $5 and some Javascript code.* ## Intro Cloudflare offers caching on their global CDN. For sites that rarely update Cloudflare handles most of the traffic for you without ever reaching your origin server. Once a user visits a specific page, Cloudflare keeps the page response in the cache and serves it for the next visitor. This reduces the load on your servers while also improving the page performance since it's served closer to the user via one of [Cloudflare's POPs](https://www.cloudflare.com/en-au/network/). While this is great in theory, it's challenging to implement for any site that requires page customization per visitor (read: most sites). For example: we may want to serve different cache responses based on the cookie (unique caching per visitor) or on the User-Agent (unique caching per device type - mobile / tablet / desktop). Luckily, Cloudflare allows you to vary the cache by HTTP headers: - `Accept-Encoding` - caches each resource by the encoding of the payload. - `Cookie` - allows the cache … -
Writing JSONField from scratch
Last week, I improved an existing Django package and implemented some SQLite lookups for its JSONField to get some sense of how Django model fields work. This week, I tried to create a unified JSONField from scratch. It’s unified in the sense that it works on all database backends supported by Django. I also set up docker containers to run Django’s test suite with all database backends with the help of django-docker-box (though I ended up making my own for Oracle since it’s a bit more complex). However, I’m just going to talk about writing the field as that’s more interesting. As before, the docs on how to write custom model fields is very handy. I started building my own JSONField and got it working in a relatively short time. Not without bugs and quirks I had to deal with on each database backend, though. First, I wrote the code with SQLite as the top priority. I overrode the db_type method so it returns json. It’s also the name of the data type used in MySQL and MariaDB for JSON values, so I guess it’s a good start. def db_type(self, connection): return 'json' This means that the SQL query for … -
Drafting a POC JSONField for SQLite
There’s this neat little python package called django-jsonfallback. It’s made by one of my mentors. To quote the description, this package allows you to Use PostgreSQL’s and MySQL’s JSONField, but fall back to a TextField implementation on other databases. Since my project is to bring a cross-DB JSONField, this project is very useful for me as a starting point. To get a better understanding of how that package works (and how Django model fields work in general), I did a good read on these wonderful Django docs about custom model fields and custom lookups. I also found this interesting Custom Database Backends presentation from Django Under the Hood 2016 event. As explained in the docs and presentation, you can create as_vendor methods such as as_mysql, as_sqlite, etc. to create different implementations of lookups and transforms (it also applies for any class that extends BaseExpression, if I’m not mistaken). I began looking at the code in django-jsonfallback, and I found that it was made with some if conditionals to check for the database backend. I tried replacing the as_sql methods with as_mysql as you can see in this pull request, and it worked pretty well. I also tried to replace the … -
Community Bonding and getting started
In the official GSoC timeline, there’s a Community Bonding period on May 7 - 27. To quote the glossary, it’s The period of time between when accepted students are announced and the time these students are expected to start coding. This time is an excellent one to introduce students to the community, get them on the right mailing lists, working with their mentors on their timeline for the summer, etc. At the start this period, we’re also given emails from Google about what we need to do. We are tasked to set up a Payoneer account to receive our stipend later. We were also invited to a mailing list for current (and past) students. Some students decided to create a Telegram group for discussions with other students all over the world. I posted on the django-developers mailing list to ask about communication channels we’ll use during the program, and one of my mentors said that he will send an email to get me (and another student) started. On May 13, my mentors (there are four) sent introductory emails on how to get started. The plans are quite similar to what I put in my proposal, so I guess I’m on … -
Python in Azure Pipelines, Step by Step
Since the acquisition of Travis CI, the future of their free offering is unclear. Azure Pipelines has a generous free tier, but the examples I found are discouragingly complex and take advantage of features like templating that most projects don’t need. To close that gap, this article shows you how to move a Python project with simple CI needs from Travis CI to Azure Pipelines. -
Build an XML sitemap of XML sitemaps
Suppose that you have so many thousands of pages that you can't just create a single /sitemap.xml file that has all the URLs (aka <loc>) listed. Then you need to make a /sitemaps.xml that points to the other sitemap files. And if you're in the thousands, you'll need to gzip these files. The blog post demonstrates how Song Search generates a sitemap file that points to 63 sitemap-{M}-{N}.xml.gz files which spans about 1,000,000 URLs. The context here is Python and the getting of the data is from Django. Python is pretty key here but if you have something other than Django, you can squint and mentally replace that with your own data mapper. Generate the sitemap .xml.gz file(s) Here's the core of the work. A generator function that takes a Django QuerySet instance (that is ordered and filtered!) and then starts generating etree trees and dumps them to disk with gzip. import gzip from lxml import etree outfile = "sitemap-{start}-{end}.xml" batchsize = 40_000 def generate(self, qs, base_url, outfile, batchsize): # Use `.values` to make the query much faster qs = qs.values("name", "id", "artist_id", "language") def start(): return etree.Element( "urlset", xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" ) def close(root, filename): with gzip.open(filename, "wb") as f: f.write(b'<?xml … -
Build an XML sitemap of XML sitemaps
Suppose that you have so many thousands of pages that you can't just create a single /sitemap.xml file that has all the URLs (aka <loc>) listed. Then you need to make a /sitemaps.xml that points to the other sitemap files. And if you're in the thousands, you'll need to gzip these files. The blog post demonstrates how Song Search generates a sitemap file that points to 63 sitemap-{M}-{N}.xml.gz files which spans about 1,000,000 URLs. The context here is Python and the getting of the data is from Django. Python is pretty key here but if you have something other than Django, you can squint and mentally replace that with your own data mapper. Generate the sitemap .xml.gz file(s) Here's the core of the work. A generator function that takes a Django QuerySet instance (that is ordered and filtered!) and then starts generating etree trees and dumps them to disk with gzip. import gzip from lxml import etree outfile = "sitemap-{start}-{end}.xml" batchsize = 40_000 def generate(self, qs, base_url, outfile, batchsize): # Use `.values` to make the query much faster qs = qs.values("name", "id", "artist_id", "language") def start(): return etree.Element( "urlset", xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" ) def close(root, filename): with gzip.open(filename, "wb") as f: f.write(b'<?xml … -
Generate a random IP address in Python
A fun trick to generate random, but seeded, IPv4 addresses in a Django app. -
Generate a random IP address in Python
A fun trick to generate random, but seeded, IPv4 addresses in a Django app. -
Bitbucket Pipelines and Ansible: Continuous delivery for your Django project
In a previous post I described how to deploy a Django project using Ansible. Here I explain how to make a step further and deploy a Django project using Bitbucket Pipelines and Ansible. Bitbucket Pipelines are basically docker containers, hosted in the Bitbucket infrastructure, that you can run to build or deploy your code, attaching them to “events” happening in your repository. Usually a push on one branch. You can clone or fork this Bitbucket repository to follow along with this tutorial. First thing first, add a file named bitbucket-pipelines.yml to the root of your repository. The content of the file should be similar to this: # use the official Python 2.7.16 docker image image: python:2.7.16 pipelines: branches: # deploy only when pushing to the master branch master: - step: name: Deploy to prod # this is the name of the Bitbucket Deployment deployment: Production caches: # cache the Ansible installation - pip script: # install Ansible - pip install ansible==2.8.0 # go into the ansible directory # in this same repository - cd ansible # perform the actual deploy - ansible-playbook -i ./hosts deploy.yaml Let’s examine the configuration line by line: On line 2 you define the docker image … -
Celery Once
Within our audit logs, which happen at the database level, we also want to be able to show the user a representation of what the object "looks like": basically, what `str(instance)` is. This is shown as a supplementary thing, and often makes it easier to identify which object has been edited. Otherwise, the audit logs only show the things that have changed on the object in that event. However, because this happens in python/django, and not in the database, we need some mechanism for fetching audit logs that don't have this string representation, and then update those. This is the perfect use case for celery: we can have a periodic task that looks for audit logs missing this related object, and then creates them. There are a few things that can cause problems: * If we run the task infrequently, then there can at times be a large number of new audit logs, which can cause this task to take a long time to run: blocking other tasks, or perhaps even timing out. We should limit the number of objects that may be handled in a given task. * If we run the task infrequently, there can be a big … -
Django Tips: Recovering Gracefully From ORM Errors
Django views are the glue between your users and the underlying database. When a user visits an url Django can map that url with a view. And most of the times the view is also responsible for fetching some data from the database. Consider the following example. There is a model named Workshop which among the others has a slug field: """ models.py """ from django.db import models class Workshop(models.Model): """ ... some other field here """ slug = models.SlugField(max_length=20) """ ... some other field here """ def __str__(self): return f'{self.title}' You may want to use that slug for fetching the appropriate entity from the database when the user visits someurl/slugname/. That means you will have an url declaration like so: """ urls.py of your Django app """ from django.urls import path from .views import workshop_detail urlpatterns = [ path('someurl/<slug:slug>/', workshop_detail) ] At this point you're ready to create a Django view for displaying the model detail. I'll use a function view for keeping things easy. A naive implementation for the view could be the following: """ views.py of your Django app """ from django.shortcuts import render from .models import Workshop def workshop_detail(request, slug): workshop = Workshop.objects.get(slug=slug) context = { … -
The Simplest WSGI Middleware
My library apig-wsgi bridges between AWS API Gateway’s JSON format for HTTP requests and Python WSGI applications. Recently Théophile Chevalier opened an issue requesting the library add an extra WSGI environ variable. I closed it by pointing out that it’s not much code to add a custom WSGI middleware to do so (plus the exact key is a bit out of scope for the library). I guess most Python web developers don’t touch WSGI day to day, so I figured I’d write this short post to share the knowledge. A Quick Shot of WSGI The WSGI specification defines an application as a callable takes two positional parameters. These parameters are, with their conventional names: environ, a dict of variables that describe the HTTP request. start_response, a function to call to call to start generation of the HTTP response. The application is called for each request, should call start_response and then return an iterable representing the contents of the HTTP response body. There’s a lot of specification around this, but to implement the simplest middleware we don’t need to dive into that. We want to only change environ and pass everything on to the proxied application. If we have original_app pointing … -
Creating Evscaperoom, part 2
The Jester, your 'adversary'This is part two of my post-mortem dev-blog about Evscaperoom, the multiplayer, text-based 'escape room' I wrote in Python and Evennia. You can read the first part of the dev blog here. This was a game-jam entry I created in a month for the Mud Coder's guild's Game Jam. The theme was One Room. You can play the game for free in your browser or with a traditional MUD client. There are no spoilers in these blog posts. The first part dealt with the overall game-design aspects. This second, final part will go into details of the code and the systems I built to quickly create all the content. The code referenced here is released under the BSD license and is available on github.At the time of this post, players have played Evscaperoom for a little more than a week. At the end I'll share some observations and things I learned along the way.Ease of buildingOver the one-month game jam, I spent about four days making the game's 'engine' and toolset with Evennia. The rest of the time was spent using those tools to actually create game content (the story, puzzles etc).An important thing was that I … -
Postgres Generated Columns
A little while ago, I wrote about creating a nice way to have a [Django ComputedField](https://schinckel.net/2018/11/12/django-computedfield%28%29/). It is pretty neat, except it needs to do some black magic to sniff up the stack to work around a limitation in the way a Ref/Col works in Django. The way it works is that you define the expression in Python, and it evaluates it in the database, allowing you to query based on this, and have it automatically annotated on. What it _doesn't_ do, however, is actually store that value in the database. Indeed, if you are actually querying on this column, you'd probably want to have a functional index that uses the same expression, so that the database can do a reasonable job of improving query times on that column. New in Postgres 12 is a feature that really piqued my interest: [Generated Columns](https://www.postgresql.org/docs/12/ddl-generated-columns.html). These are basically what the ComputedField does, but at the database level. And, instead of it being an expression that is evaluated at query time, it is instead an expression that is evaluated at write time, and stored in an actual column (that could then have an index applied to it). Let's have a look at an … -
The Best Animation and Prototyping Tools for Designers in 2019
Prototyping is a great way to showcase your product before embarking on a long development process. It gives you the opportunity to make changes prior to development. It also saves you significant time and money. The post The Best Animation and Prototyping Tools for Designers in 2019 appeared first on Distillery. -
Creating Evscaperoom, part 1
Over the last month (April-May 2019) I have taken part in the Mud Coder's Guild Game Jam "Enter the (Multi-User) Dungeon". This year the theme for the jam was One Room.The result was Evscaperoom, an text-based multi-player "escape-room" written in Python using the Evennia MU* creation system. You can play it from that link in your browser or MU*-client of choice. If you are so inclined, you can also vote for it here in the jam (don't forget to check out the other entries while you're at it).This little series of (likely two) dev-blog entries will try to recount the planning and technical aspects of the Evscaperoom. This is also for myself - I'd better write stuff down now while it's still fresh in my mind!Inception When I first heard about the upcoming game-jam's theme of One Room, an 'escape room' was the first thing that came to mind, not the least because I just recently got to solve my my own first real-world escape-room as a gift on my birthday. If you are not familiar with escape-rooms, the premise is simple - you are locked into a room and have to figure out a way to get out of it by solving practical … -
New features planned for Python 4.0
With the release of Python 3.8 coming soon, the core development team has asked me to summarize our latest discussions on the new features planned for Python 4.0, codename "ouroboros: the snake will eat itself". This will be an exciting release and a significant milestone, many thanks to the hard work of over 100 contributors. After heated debate on the mailing list, the 79-character line limit prescribed by PEP8 will be updated. IDE users all over the world will now be able to take advantage of their 30" ultra-wide 4K monitors, as the recommended line length will be increased to 89.5 characters (this was a compromise with the 100-character lobby, the decision being to split the difference). All new libraries and standard lib modules must include the phrase "for humans" somewhere in their title, and have a splashy documentation page with lots of fonts typography and testimonials. Finally, a new string-type for the masses, Python 4.0 will feature "z-strings": C-style NULL terminated bytestrings. Just prefix your string with z'my string' and Python will automatically ensure it is NULL-terminated. Note: the new z-strings cannot be used with any of the existing APIs that take string arguments - they must first be … -
New features planned for Python 4.0
With the release of Python 3.8 coming soon, the core development team has asked me to summarize our latest discussions on the new features planned for Python 4.0, codename "ouroboros: the snake will eat itself". This will be an exciting release and a significant milestone, many thanks to the hard work of over 100 contributors. After heated debate on the mailing list, the 79-character line limit prescribed by PEP8 will be updated. IDE users all over the world will now be able to take advantage of their 30" ultra-wide 4K monitors, as the recommended line length will be increased to 89.5 characters (this was a compromise with the 100-character lobby, the decision being to split the difference). All new libraries and standard lib modules must include the phrase "for humans" somewhere in their title. Finally, a new string-type for the masses, Python 4.0 will feature "z-strings": C-style NULL terminated bytestrings. Just prefix your string like so, z'my string' and Python will automatically ensure it is NULL-terminated. Note: the new z-strings cannot be used with any of the existing APIs that take string arguments - they must first be decoded to unicode strings or cast to bytes. Type-hinting has been extended … -
<p>django-debug-toolbar 2.0a1 is now <a target="_blank" rel="nofollow" href="https://pypi.org/project/django-debug-toolbar/2.0a1/">available on PyPI</a>. Please help with testing or with success stori
django-debug-toolbar 2.0a1 is now available on PyPI. Please help with testing or with success stories or with bug reporting & squashing.
