Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Deploying Django on Docker to Heroku with OpenCV
Building off [this post](https... -
Django on Docker - A Simple Introduction
This is a simple tutorial but ... -
Working around memory leaks in your Django application
Several large Django applications that I’ve worked on ended up with memory leaks at some point. The Python processes slowly increased their memory consumption until crashing. Not fun. Even with automatic restart of the process, there was still some downtime. Memory leaks in Python typically happen in module-level variables that grow unbounded. This might be an lru_cache with an infinite maxsize, or a simple list accidentally declared in the wrong scope. Leaks don’t need to happen in your own code to affect you either. For example, see this excellent write-up by Peter Karp at BuzzFeed where he found one inside Python’s standard library (since fixed!). Workarounds The below workarounds all restart worker processes after so many requests or jobs. This is a simple way to clear out any potential infinitely-accumulating Python objects. If your web server, queue worker, or similar has this ability but isn’t featured, let me know and I’d add it! Even if you don’t see any memory leaks right now, adding these will increase your application’s resilience. Gunicorn If you’re using Gunicorn as your Python web server, you can use the --max-requests setting to periodically restart workers. Pair with its sibling --max-requests-jitter to prevent all your workers … -
Simple Docker Container Tutorial
Docker is all the rage. I cove... -
Django 3.0 Preview
Django 3.0 alpha downloadASGI (Asynchronous Server Gateway Interface)PyCon AU 2019 - Just Add Await: Retrofitting Async into Django by Andrew GodwinDjangoCon Europe 2019 - Sketching out a Django Redesign by Tom Christieforum.djangoproject.com -
How "Export to Excel" Almost Killed Our System
A few weeks ago we had some trouble with an "Export to Excel" functionality in one of our systems. In the process of resolving this issue, we made some interesting discoveries and came up with original solutions. This article is inspired by the actual issue we used to track this incident over a period of two days. We go through the process of identifying the problem, experimenting and benchmarking different solutions until eventually deploying to production. These are the main takeaways described in this article: Generating xlsx files can consume significant amount of resources. Under some circumstances better performance can be gained by not using prefetch_related. pyexcelerate is a fast package for creating simple Excel files. tablib (and django-import-export) can be patched to use pyexcelerate and produce excel files faster. Table of Contents Exporting a QuerySet to Excel Using django-import-export Finding the Best File Format Improving the Query Replacing prefetch_related with Subquery and OuterRef Using an Iterator Simplifying the Query Manual Prefetch Trouble in Paradise Using a Different Excel Writer A Faster Excel Writer in Python Patching tablib Results Summary Seifa How a server must feel when asked to produce an Excel file A few weeks ago we started getting … -
Django shell_plus with Pandas and Jupyter Notebook
The Django shell provides an environment where developers can interact with the database via Django's ORM. While the shell is great for basic interactions, it quickly becomes laborious to work in, be it due to manual imports, having to scroll through shell history to repeat commands, or working with / viewing queries returning more than, say, 20 records. These issues, and more, can be remedied by interacting with the ORM in Jupyter notebooks, using Pandas. Our environment will be setup inside a virtual environment using Django 2.2. Once inside the virtual environment, install IPython, Jupyter, Pandas, django-extensions, and django-pandas. pip install ipython jupyter pandas django-extensions django-pandas django-extensions needs to be placed into INSTALLED_APPS in your settings.py file. INSTALLED_APPS = [ ... 'django_extensions', ] Once installed, then run python manage.py shell_plus --notebook shell_plus is Django shell with autoloading of the apps database models and subclasses of user-defined classes. The --notebook argument tells shell_plus to open in a Jupyter notebook. You will notice that a new Jupyter tab opens in your browser. In the upper right corner click on the New drop-down and select Django Shell-Plus. A new tab will open and you will see your new Jupyter notebook. Import … -
A Single File Asynchronous Django Application
Django 3.0 alpha 1 came out this week. It introduces ASGI support thanks to lots of hard work by Andrew Godwin. Thanks to a question from Emiliano Dalla Verde Marcozzi on the shiny new Django forums, I decided to play with it a bit. I adapted my “single file Django app” to ASGI. Here’s how to try it yourself. First, create a new virtual environment (I’m using Python 3.7.4) and install requirements: $ python -m venv venv $ source venv/bin/activate $ pip install django==3.0a1 daphne==2.3.0 Then create a new file app.py: import html import os import sys from django.conf import settings from django.core.asgi import get_asgi_application from django.http import HttpResponse from django.urls import path from django.utils.crypto import get_random_string settings.configure( DEBUG=(os.environ.get("DEBUG", "") == "1"), # Disable host header validation ALLOWED_HOSTS=["*"], # Make this module the urlconf ROOT_URLCONF=__name__, # We aren't using any security features but Django requires this setting SECRET_KEY=get_random_string(50), ) def index(request): name = request.GET.get("name", "World") return HttpResponse(f"Hello, {html.escape(name)}!") urlpatterns = [path("", index)] application = get_asgi_application() if __name__ == "__main__": from django.core.management import execute_from_command_line execute_from_command_line(sys.argv) Then run it under Daphne: $ daphne app:application Visit http://localhost:8000/?name=Django%20user in your browser to see “Hello, Django user!” Async support in Django 3.0 is the first … -
How I Import Python's datetime Module
Python’s datetime module risks a whole bunch of name confusion: It contains a class also called datetime. When you read the word datetime, you can’t be sure it’s the module or the class. It contains a time class. This can be confused with the time module, or the time module’s time() function. It contains a class called timezone. In a Django project this can be confused with the django.utils.timezone module For these reasons, I use this import idiom and recommend you do too: import datetime as dt Rather than any of: import datetime from datetime import datetime # or time, timezone Then in your code, dt.datetime, dt.time, and dt.timezone will be unambiguous. Fin Hope this helps you save time understanding code, —Adam -
Learning to Love Django Tests - Lacey Williams Henschel
@laceynwilliams on TwitterLacey Williams Henschel personal siteGoodby Print Statement, Hello Debugger! - Nina Zakharenko @ Pycon AU 2019OpenSource.com articlesDjango Admin/Templates courses on TreehouseHow to Add Django Models to the Wagtail AdminSHAMELESS PLUGSWilliam's books on DjangoCarlton's website Noumenal -
Django shell_plus with Pandas and Jupyter Notebook
The Django shell provides an environment where developers can interact with the database via Django's ORM. While the shell is great for basic interactions, it quickly becomes laborious to work in, be it due to manual imports, having to scroll through shell history to repeat commands, or working with / viewing queries returning more than, say, 20 records. These issues, and more, can be remedied by interacting with the ORM in Jupyter notebooks, using Pandas. Our environment will be setup inside a virtual environment using Django 2.2. Once inside the virtual environment, install IPython, Jupyter, Pandas, django-extensions, and django-pandas. pip install ipython jupyter pandas django-extensions django-pandas django-extensions needs to be placed into INSTALLED_APPS in your settings.py file. INSTALLED_APPS = [ ... 'django_extensions', ] Once installed, then run python manage.py shell_plus --notebook shell_plus is Django shell with autoloading of the apps database models and subclasses of user-defined classes. The --notebook argument tells shell_plus to open in a Jupyter notebook. You will notice that a new Jupyter tab opens in your browser. In the upper right corner click on the New drop-down and select Django Shell-Plus. A new tab will open and you will see your new Jupyter notebook. Import … -
Mercurial mirrors updates
Time to clean up my mercurial mirrors as Django 3.0 has just had its first alpha released. Keeping an eye on the supported versions, I did the following changes: Added mirror for 3.0 branch : https://bitbucket.org/orzel/django-3.0-production/ Removed mirror for 1.8 branch Removed mirror for 1.9 branch Removed mirror for 1.10 branch Branch 2.0 is officially […] -
Custom Application Metrics with Django, Prometheus, and Kubernetes
Why are custom metrics important?While there are volumes of discourse on the topic, it can't be overstated how important custom application metrics are. Unlike the core service metrics you'll want to collect for your Django application (application and web server stats, key DB and cache operational metrics), custom metrics are data points unique to your domain with bounds and thresholds known only by you. In other words, it's the fun stuff.How might these metrics be useful? Consider:You run an ecomm website and track average order size. Suddenly that order size isn't so average. With solid application metrics and monitoring you can catch the bug before it breaks the bank.You're writing a scraper that pulls the most recent articles from a news website every hour. Suddenly the most recent articles aren't so recent. Solid metrics and monitoring will reveal the breakage earlier.I 👏 Think 👏 You 👏 Get 👏 The 👏 Point 👏Setting up the Django ApplicationBesides the obvious dependencies (looking at you pip install Django), we'll need some additional packages for our pet project. Go ahead and pip install django-prometheus-client. This will give us a Python Prometheus client to play with, as well as some helpful Django hooks including middleware … -
Django Fellow - Mariusz Felisiak
Django IRC channel is #django irc.freednote.net Django Developers List Django Fellows Report - August 2019 django-docker-box SHAMELESS PLUGS William's books on Django Carlton's website Noumenal -
Profiling & Optimizing Bottlenecks In Django
In the previous article, we have learnt where to start with performance optimization in django application and find out which APIs to optimize first. In this article, we will learn how to optimize those selected APIs from the application. Profling APIs With django-silk django-silk provides silk_profile function which can be used to profile selected view or a snippet of code. Let's take a slow view to profile and see the results. from silk.profiling.profiler import silk_profile @silk_profile() def slow_api(request): time.sleep(2) return JsonResponse({'data': 'slow_response'}) We need to add relevant silk settings to django settings so that required profile data files are generated and stored in specified locations. SILKY_PYTHON_PROFILER = True SILKY_PYTHON_PROFILER_BINARY = True SILKY_PYTHON_PROFILER_RESULT_PATH = '/tmp/' Once the above view is loaded, we can see the profile information in silk profiling page. In profile page, silk shows a profile graph and highlights the path where more time is taken. It also shows cprofile stats in the same page. This profile data file can be downloaded and used with other visualization tools like snakeviz. By looking at the above data, we can see most of the time is spent is time.sleep in our view. Profling APIs With django-extensions If you don't want to … -
From Django 0.9 to Present - Russell Keith-Magee
PyCon Australia 2015: Money, Money, Money - Writing software, in a rich (wo)man’s world BeeWare Django 2008 Keynote Keynote: Why I Hate Django by Cal Henderson PyCon Australia 2017 - Red User, Blue User, MyUser, auth.User by Russell Keith-Magee django-improved-user PyCon 2019 Keynote Django for Beginners book DjangoX Starter Project PyCon Australia 2019 - Just Add Await: Retrofitting Async Into Django by Andrew Godwin SHAMELESS PLUGS William's books on Django Carlton's website Noumenal -
Revisions
Mistakes happen, and that’s part of a learning process. In a large project like Django, it can be hard to spot a mistake. Thanks to it being open source, anyone can see the code and fix the mistakes they see. In this post, I’ll explain how I found a vulnerability in contrib.postgres.fields.JSONField that allowed SQL injections to be performed. If you’re familiar with Python’s DB-API, you may have used the . -
Build a Python Jupyter Notebook Server with Docker & Heroku
Jupyter notebooks have become ... -
MySQL and Security - Adam Johnson
Adam Johnson personal site django-mysql MariaDB Django Hosting Options Django’s Test Case Classes and a Three Times Speed-Up DjangoCon 2019 Europe - Django and Web Security Headers SHAMELESS PLUGS William's books on Django Carlton's website Noumenal -
Django Tutorial - ManyToManyField via a Comma Separated String in Admin & Forms
The `ManyToManyField` is super... -
My Appearance on DjangoChat
A few weeks ago I had the pleasure of talking over the internet with Will Vincent and Carlton Gibson about lots of Django-related topics. They somewhat informed me it was being recorded for a podcast. Now the day I anticipated and feared is here. I discovered through this tweet that my voice is online: Episode 27 - MySQL & Security with Adam Johnson (@AdamChainz) is now live! Adam is a Django core developer responsible for the popular django-mysql package. We discuss why MySQL still makes sense with Django, security, hosting on AWS, and more. https://djangochat.com We talked about all kinds of things including: How I got started with Django Why MySQL and MariaDB are still a competitive databases for Django apps—despite their relative unpopularity in the community and ecosystem fragmentation. Ways to host Django My security talk at DjangoCon Europe this year Testing Django applications, coverage and speeding up tests If you’re interested in Django, head on over to the website and listen to Episode 027: MySQL & Security. Enjoy! —Adam (P.S. Don’t get mixed up and go to the wrong Twitter account, @DjangoChat, unless you want to see an adorable French cat.) -
DjangoCon, Here We Come!
We’re looking forward to the international gathering at DjangoCon 2019, in San Diego, CA. The six-day conference, from September 22 - 27, is focused on the Django web framework, and we’re proud to attend as sponsors for the tenth year! We’re also hosting the second annual Caktus Mini Golf event. ⛳ If you’re attending DjangoCon, come play a round of mini golf with us. Look for our insert in your conference tote bag. It includes a free pass to Tiki Town Adventure Golf on Wednesday, September 25, at 7:00 p.m. (please RSVP online). The first round of golf is on us! And whoever shoots the lowest score will win a $100 Amazon gift card.* Talk(s) of the Town Among this year’s talented speakers is one of our own, Erin Mullaney (pictured). Erin has been with Caktus since 2015, and has worked as a contractor for us since July 2017. On Monday, September 23, she’ll share her experiences going from a full-time developer to a contractor in her talk, “Roll Your Own Tech Job: Starting a Business or Side Hustle from Scratch.” The talk will cover her first two years as a consultant, including how she legally set up her business … -
Python Tutorial - Google Geocoding API
In this short tutorial, we're ... -
Django Foreign Keys: Limiting Choices in Forms
I'm going to assume you're not... -
Django x Docker to Production on Heroku
Manually deploying a [project ...
