Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Django Projects – ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to… -
Django Projects - 99 ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to plan out a project and how to research what you don’t know. We believe in learn by doing. And that there’s no better time to start than now. It doesn’t matter if it already exists. Your not building software to change the world, you’re building software to learn web development. Here are some ideas to get you started. 99 Beginner Django Projects Track sleep Track customized habit performance Track customized habits of users Send daily reminders in email Send daily reminders in SMS Store a list of current projects and log entries Freelance time tracking on current projects Freelance project tracking with view ability for project owners Freelance project tracking with comment ability for project owners Send daily inspirational quote to your email Send daily inspirational quote in SMS Make a blog like WordPress Make an e-commerce store like Gumroad Make someone else a blog Make someone else an e-commerce store Collect your thoughts on books Make an e-commerce store with Amazon Affiliate links Make an e-commerce store that automatically pulls Amazon … -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Blackifying and fixing bugs
Since version 0.9 of Evennia, the MU*-creation framework, was released, work has mainly been focused on bug fixing. But there few new features also already sneaked into master branch, despite technically being changes slated for Evennia 1.0. On FrontendsContributor friarzen has chipped away at improving Evennia's HTML5 web client. It already had the ability to structure and spawn any number of nested text panes. In the future we want to extend the user's ability to save an restore its layouts and allow developers to offer pre-prepared layouts for their games. Already now though, it has gotten plugins for handling both graphics, sounds and video: Inline image by me (griatch-art.deviantart.com) A related fun development is Castlelore Studios' development of an Unreal Engine Evennia plugin (this is unaffiliated with core Evennia development and I've not tried it, but it looks pretty nifty!): Image ©Castlelore Studios On BlackEvennia's source code is extensively documented and was sort of adhering to the Python formatting standard PEP8. But many places were sort of hit-and-miss and others were formatted with slight variations due to who wrote the code. After pre-work and recommendation by Greg Taylor, Evennia has adopted the black autoformatter for its source code. I'm not really convinced that … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Speeding up Postgres using a RAM disk?
I happened to come across [Speeding up Django unit tests with SQLite, keepdb and /dev/shm](http://www.obeythetestinggoat.com/speeding-up-django-unit-tests-with-sqlite-keepdb-and-devshm.html) today. I can't quite do that, because we use a bunch of Postgres specific things in our project, but it did make me think "Can we run Postgres on a RAM disk?" Following up on this, it turns out that using a TABLESPACE in postgres that is on a RAM disk is a really bad idea, but it should be possible to run a seperate database cluster (on a different port) that could have the whole cluster on the database. It's possible to create a RAM disk on macOS, and init a cluster there: {% highlight bash %} #! /bin/sh PG_CTL=/Applications/Postgres.app/Contents/Versions/10/bin/pg_ctl new_disk=$(hdid -nomount ram://1280000) newfs_hfs $new_disk mkdir /tmp/ramdisk mount -t hfs $new_disk /tmp/ramdisk mkdir /tmp/ramdisk/pg PG_CTL initdb -D /tmp/ramdisk/pg {% endhighlight %} This needs to be run once, after bootup (and probably before Postgres.app starts it's servers). You'd also need to do the setup in Postgres.app to show where it is (alternatively, you could just start it on the port you want in the script above). But, is it faster? Let's look at the results. The "Initial run" is the first run of tests after … -
Ruby on Rails & Django- David Heinemeier Hansson
David Heinemeier Hansson personal siteRuby on Rails -
How to use Heroku Pipeline
In this Heroku tutorial, I will talk about how to setup Heroku pipeline and how to use it. -
Embedding Videos Into Django Project
Have you ever tried to embed a video into your Django website so find that it wouldn’t work? Well, I have. Let’s go over the solution I found to this problem. The problem I’ve been making an app for some other people that I mentor. I decided it would be easier on me and more beneficial to my mentees if I answered their questions in video instead of writing; speeds things along and I can increase my output while conveying my non-verbal communication. I expected to just hold a Vimeo URL in the database and plug it into a set embed code in my template. Long story short: that didn’t work. So I went searching for a new answer. Answer: django-embed-video I found the solution in a lovely plugin package, django-embed-video. Install the package pip install django-embed-video Change settings.py INSTALLED_APPS = ( ... 'embed_video', ) Add the field to your model # app_name/models.py from django.db import models from embed_video.fields import EmbedVideoField class ModelName(models.Model): video = EmbedVideoField(blank=True) # an empty field is fine for me Load the tags at the top of your template {# for me, this was app_name/model_name_detail.html #} {% load embed_video_tags %} Call the video in the template … -
DjangoCon US 2019: Python & Django in San Diego!
We are back to San Diego!! Our team will be joining DjangoCon US's conference, one of the biggest Django events in the world. For this year, we'll be giving two talks: Pull Requests: Merging good practices into your project and Building effective Django queries with expressions Here is the slide from the talk we gave during the conference: Pull Re -
Deploying Django on Docker to Heroku with OpenCV
Building off [this post](https... -
Learning Django Past the Documentation
After you’ve read documentation from a bunch of different projects, you’ll come to realize the Django Documentation is fantastic. It’s concise and contains lots of examples and suggested use cases, which I have found invaluable as a n00b. But sometimes it still leaves you with wtf feels. When you get to a point where something you think you need to know is far outside your scope, what do you do? Realize you probably don’t need it Most of the books, well-written documentation, and tutorials you’ve read and watched cover the parts of Django that are most useful or best practice. If you haven’t heard about something, it’s because most people aren’t using it. And there’s usually a reason for that. You may want to ask a professional’s opinion. Google search There may be a rando somewhere who truly DOES think this is useful to know, and they may have written something about it somewhere. There are two main types of search results in my head: Private sites, such as this one Stack Overflow questions A third, less likely scenario is that your google search actually points you to a “hidden-to-you” part of the documentation that actually answers your question. Take … -
Django on Docker - A Simple Introduction
This is a simple tutorial but ... -
Making Your Own Email Templates in Django
I’m looking to make my app notify the user when their object is updated by an admin. Problem is that I had no idea how to use my own email templates. But I got it working! Here’s how you can put it into your Django project. I’m using Django 2.2.5. Email settings in settings.py There are two useful ways that I know of to send emails: Logging directly to the console window Sending via SMTP The first is much quicker, but doesn’t actually send emails. It’s good for testing. The second is what you need if you need to actually send emails to a user’s inbox. Send to console To log emails to the console, you just have to set EMAIL_BACKEND in settings.py. EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend' Here’s the documentation for reference. Send via SMTP There are a few settings you’ll need to take care of if you want to send live SMTP emails: EMAIL_BACKEND EMAIL_HOST EMAIL_PORT EMAIL_HOST_USER EMAIL_HOST_PASSWORD EMAIL_USE_TLS or EMAIL_USE_SSL If you’re ready to give this a try, I would recommend making an account on SendGrid. Even when your free trial expires, you’ll have 100 free emails to send every day. Not enough to run a big project, but … -
Working around memory leaks in your Django application
Several large Django applications that I’ve worked on ended up with memory leaks at some point. The Python processes slowly increased their memory consumption until crashing. Not fun. Even with automatic restart of the process, there was still some downtime. Memory leaks in Python typically happen in module-level variables that grow unbounded. This might be an lru_cache with an infinite maxsize, or a simple list accidentally declared in the wrong scope. Leaks don’t need to happen in your own code to affect you either. For example, see this excellent write-up by Peter Karp at BuzzFeed where he found one inside Python’s standard library (since fixed!). Workarounds The below workarounds all restart worker processes after so many requests or jobs. This is a simple way to clear out any potential infinitely-accumulating Python objects. If your web server, queue worker, or similar has this ability but isn’t featured, let me know and I’d add it! Even if you don’t see any memory leaks right now, adding these will increase your application’s resilience. Gunicorn If you’re using Gunicorn as your Python web server, you can use the --max-requests setting to periodically restart workers. Pair with its sibling --max-requests-jitter to prevent all your workers … -
Simple Docker Container Tutorial
Docker is all the rage. I cove... -
Django 3.0 Preview
Django 3.0 alpha downloadASGI (Asynchronous Server Gateway Interface)PyCon AU 2019 - Just Add Await: Retrofitting Async into Django by Andrew GodwinDjangoCon Europe 2019 - Sketching out a Django Redesign by Tom Christieforum.djangoproject.com -
Postgres ENUM types in Django
Postgres has the ability to create custom types. There are several kinds of `CREATE TYPE` statement: * composite types * domain types * range types * base types * enumerated types I've used a metaclass that is based on Django's Model classes to do Composite Types in the past, and it's been working fairly well. The current stuff I have been working on made sense to use an Enumerated Type, because there are four possible values, and having a human readable version of them is going to be nicer than using a lookup table. In the first iteration, I used just a TEXT column to store the data. However, when I then started to use an `enum.Enum` class for handling the values in python, I discovered that it was actually storing `str(value)` in the database, rather than `value.value`. So, I thought I would implement something similar to my Composite Type class. Not long after starting, I realised that I could make a cleaner implementation (and easier to declare) using a decorator: {% highlight python %} @register_enum(db_type='change_type') class ChangeType(enum.Enum): ADDED = 'added' CHANGED = 'changed' REMOVED = 'removed' CANCELLED = 'cancelled' ChangeType.choices = [ (ChangeType.ADDED, _('hours added')), (ChangeType.REMOVED, _('hours subtracted')), (ChangeType.CHANGED, … -
Django 3.0 Alpha Released
Big news! The alpha version of Django 3.0 has been released. This is not suitable for production! BUT, this is suitable for sandboxing. Feel free to install it and see if it breaks anything. And if you notice something, make sure to submit a bug! Or you can try to get up and running with ASGI. How exciting! Here’s a link to the official release statement. And here’s a link to the Django 3.0 release notes. -
Limit Access to Only Superusers in Django
So in my other life as a strength and conditioning coach, I have a few online mentees. They basically email me questions and I answer them. I’m using Django to make an interface for them to submit questions and see all the other questions that other members have asked. Plus adding some searching so the old stuff doesn’t just die. I am using the generic UpdateView to answer the questions. This view is marking the question “published” and storing my answer in the database. My issue is that I don’t want the users to be able to see this because then they could be adding answers to questions and people are paying for ME to answer the question. This entails two major parts: Adding a link to the UpdateView in the DetailView Blocking permissions to access the form with UserPassesTestMixin Let’s walk through it. Add link to UpdateView Order doesn’t particularly matter here. You’ll have to do each step. I like to start with the URL, so first, we need to add the URL to our UpdateView to our urls.py file. # app_name/urls.py from .views import AppNameUpdateView # make sure to import your view urlpatterns = [ # other URL … -
Django Has Changed Over Time
Repetition is king for learning. Working with models, views, and templates over and over gets repetitive, but that’s a good thing. Have you ever seen a tutorial online and thought, “That code doesn’t look very familiar.” No repetition, no knowledge. Usually these differences are not WAY off. The similarities give you sense for what you have to do. But after you give it a shot, you get an error. One of the biggest reasons to start your web development journey by learning Django is because it is OLD. Old things are set in their ways. They won’t change so much on you as you slog along. In fact, this was one of the biggest reasons I decided to pick Django as my learning language. You can be sure that you won’t lose much on your investment because what you learn will still be relevant in the future. But still, things have changed in Django. And this makes it difficult to learn on the internet. There’s a lot of wasted time. And time is more valuable than money. To help save you time, I want to outline some of the major differences I’ve run into while learning Django. Note When reading … -
How "Export to Excel" Almost Killed Our System
A few weeks ago we had some trouble with an "Export to Excel" functionality in one of our systems. In the process of resolving this issue, we made some interesting discoveries and came up with original solutions. This article is inspired by the actual issue we used to track this incident over a period of two days. We go through the process of identifying the problem, experimenting and benchmarking different solutions until eventually deploying to production. These are the main takeaways described in this article: Generating xlsx files can consume significant amount of resources. Under some circumstances better performance can be gained by not using prefetch_related. pyexcelerate is a fast package for creating simple Excel files. tablib (and django-import-export) can be patched to use pyexcelerate and produce excel files faster. Table of Contents Exporting a QuerySet to Excel Using django-import-export Finding the Best File Format Improving the Query Replacing prefetch_related with Subquery and OuterRef Using an Iterator Simplifying the Query Manual Prefetch Trouble in Paradise Using a Different Excel Writer A Faster Excel Writer in Python Patching tablib Results Summary Seifa How a server must feel when asked to produce an Excel file A few weeks ago we started getting … -
Django shell_plus with Pandas and Jupyter Notebook
The Django shell provides an environment where developers can interact with the database via Django's ORM. While the shell is great for basic interactions, it quickly becomes laborious to work in, be it due to manual imports, having to scroll through shell history to repeat commands, or working with / viewing queries returning more than, say, 20 records. These issues, and more, can be remedied by interacting with the ORM in Jupyter notebooks, using Pandas. Our environment will be setup inside a virtual environment using Django 2.2. Once inside the virtual environment, install IPython, Jupyter, Pandas, django-extensions, and django-pandas. pip install ipython jupyter pandas django-extensions django-pandas django-extensions needs to be placed into INSTALLED_APPS in your settings.py file. INSTALLED_APPS = [ ... 'django_extensions', ] Once installed, then run python manage.py shell_plus --notebook shell_plus is Django shell with autoloading of the apps database models and subclasses of user-defined classes. The --notebook argument tells shell_plus to open in a Jupyter notebook. You will notice that a new Jupyter tab opens in your browser. In the upper right corner click on the New drop-down and select Django Shell-Plus. A new tab will open and you will see your new Jupyter notebook. Import …
