Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
My Most Used Pytest Commandline Flags
Pytest is quickly becoming the “standard” Python testing framework. However it can be overwhelming to new users. pytest --help currently outputs 275 lines of command line flags and options. Where do you even begin? I searched my ZSH history for my recent usage of Pytest. I found 184 unique invocations, out of which I split the command line flags I used. Here are the top five flags I used by frequency. 1. -v -v, or --verbose, increases the verbosity level. The default level outputs a reasonable amount of information to debug most test failures. However when there are many differences between actual and expected data, some get hidden. In such cases Pytest normally appends a message to the failure text such as: ...Full output truncated (23 lines hidden), use '-vv' to show This tells us to see the hidden lines, we need to rerun the tests with verbosity level 2. To do this we can pass -v twice as -v -v, or more easily as -vv. Another change from passing -v at least once is that the test names are output one per line as they run: $ pytest -v tests.py::test_parrot_status PASSED [100%] Sometimes I use this when looking for … -
Search
Django Search Tutorialdjango-filterMDN on sending form data and form data validationDjango Q Objectsdjango.contrib.postgres.searchPostgreSQL Full Text SearchEuroPython 2017 - Full-Text Search in Django with PostgreSQL by Paulo MelchiorreDjangoCon Europe 2018 - On The Look-Out For Your Data by Markus HoltermannDjangoCon US 2015 - Beyond the basics with Elasticsearch by Honza Kral -
Django Projects – ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to… -
Django Projects - 99 ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to plan out a project and how to research what you don’t know. We believe in learn by doing. And that there’s no better time to start than now. It doesn’t matter if it already exists. Your not building software to change the world, you’re building software to learn web development. Here are some ideas to get you started. 99 Beginner Django Projects Track sleep Track customized habit performance Track customized habits of users Send daily reminders in email Send daily reminders in SMS Store a list of current projects and log entries Freelance time tracking on current projects Freelance project tracking with view ability for project owners Freelance project tracking with comment ability for project owners Send daily inspirational quote to your email Send daily inspirational quote in SMS Make a blog like WordPress Make an e-commerce store like Gumroad Make someone else a blog Make someone else an e-commerce store Collect your thoughts on books Make an e-commerce store with Amazon Affiliate links Make an e-commerce store that automatically pulls Amazon … -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Blackifying and fixing bugs
Since version 0.9 of Evennia, the MU*-creation framework, was released, work has mainly been focused on bug fixing. But there few new features also already sneaked into master branch, despite technically being changes slated for Evennia 1.0. On FrontendsContributor friarzen has chipped away at improving Evennia's HTML5 web client. It already had the ability to structure and spawn any number of nested text panes. In the future we want to extend the user's ability to save an restore its layouts and allow developers to offer pre-prepared layouts for their games. Already now though, it has gotten plugins for handling both graphics, sounds and video: Inline image by me (griatch-art.deviantart.com) A related fun development is Castlelore Studios' development of an Unreal Engine Evennia plugin (this is unaffiliated with core Evennia development and I've not tried it, but it looks pretty nifty!): Image ©Castlelore Studios On BlackEvennia's source code is extensively documented and was sort of adhering to the Python formatting standard PEP8. But many places were sort of hit-and-miss and others were formatted with slight variations due to who wrote the code. After pre-work and recommendation by Greg Taylor, Evennia has adopted the black autoformatter for its source code. I'm not really convinced that … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Speeding up Postgres using a RAM disk?
I happened to come across [Speeding up Django unit tests with SQLite, keepdb and /dev/shm](http://www.obeythetestinggoat.com/speeding-up-django-unit-tests-with-sqlite-keepdb-and-devshm.html) today. I can't quite do that, because we use a bunch of Postgres specific things in our project, but it did make me think "Can we run Postgres on a RAM disk?" Following up on this, it turns out that using a TABLESPACE in postgres that is on a RAM disk is a really bad idea, but it should be possible to run a seperate database cluster (on a different port) that could have the whole cluster on the database. It's possible to create a RAM disk on macOS, and init a cluster there: {% highlight bash %} #! /bin/sh PG_CTL=/Applications/Postgres.app/Contents/Versions/10/bin/pg_ctl new_disk=$(hdid -nomount ram://1280000) newfs_hfs $new_disk mkdir /tmp/ramdisk mount -t hfs $new_disk /tmp/ramdisk mkdir /tmp/ramdisk/pg PG_CTL initdb -D /tmp/ramdisk/pg {% endhighlight %} This needs to be run once, after bootup (and probably before Postgres.app starts it's servers). You'd also need to do the setup in Postgres.app to show where it is (alternatively, you could just start it on the port you want in the script above). But, is it faster? Let's look at the results. The "Initial run" is the first run of tests after … -
Ruby on Rails & Django- David Heinemeier Hansson
David Heinemeier Hansson personal siteRuby on Rails -
How to use Heroku Pipeline
In this Heroku tutorial, I will talk about how to setup Heroku pipeline and how to use it. -
Embedding Videos Into Django Project
Have you ever tried to embed a video into your Django website so find that it wouldn’t work? Well, I have. Let’s go over the solution I found to this problem. The problem I’ve been making an app for some other people that I mentor. I decided it would be easier on me and more beneficial to my mentees if I answered their questions in video instead of writing; speeds things along and I can increase my output while conveying my non-verbal communication. I expected to just hold a Vimeo URL in the database and plug it into a set embed code in my template. Long story short: that didn’t work. So I went searching for a new answer. Answer: django-embed-video I found the solution in a lovely plugin package, django-embed-video. Install the package pip install django-embed-video Change settings.py INSTALLED_APPS = ( ... 'embed_video', ) Add the field to your model # app_name/models.py from django.db import models from embed_video.fields import EmbedVideoField class ModelName(models.Model): video = EmbedVideoField(blank=True) # an empty field is fine for me Load the tags at the top of your template {# for me, this was app_name/model_name_detail.html #} {% load embed_video_tags %} Call the video in the template … -
DjangoCon US 2019: Python & Django in San Diego!
We are back to San Diego!! Our team will be joining DjangoCon US's conference, one of the biggest Django events in the world. For this year, we'll be giving two talks: Pull Requests: Merging good practices into your project and Building effective Django queries with expressions Here is the slide from the talk we gave during the conference: Pull Re -
Deploying Django on Docker to Heroku with OpenCV
Building off [this post](https... -
Learning Django Past the Documentation
After you’ve read documentation from a bunch of different projects, you’ll come to realize the Django Documentation is fantastic. It’s concise and contains lots of examples and suggested use cases, which I have found invaluable as a n00b. But sometimes it still leaves you with wtf feels. When you get to a point where something you think you need to know is far outside your scope, what do you do? Realize you probably don’t need it Most of the books, well-written documentation, and tutorials you’ve read and watched cover the parts of Django that are most useful or best practice. If you haven’t heard about something, it’s because most people aren’t using it. And there’s usually a reason for that. You may want to ask a professional’s opinion. Google search There may be a rando somewhere who truly DOES think this is useful to know, and they may have written something about it somewhere. There are two main types of search results in my head: Private sites, such as this one Stack Overflow questions A third, less likely scenario is that your google search actually points you to a “hidden-to-you” part of the documentation that actually answers your question. Take … -
Django on Docker - A Simple Introduction
This is a simple tutorial but ... -
Making Your Own Email Templates in Django
I’m looking to make my app notify the user when their object is updated by an admin. Problem is that I had no idea how to use my own email templates. But I got it working! Here’s how you can put it into your Django project. I’m using Django 2.2.5. Email settings in settings.py There are two useful ways that I know of to send emails: Logging directly to the console window Sending via SMTP The first is much quicker, but doesn’t actually send emails. It’s good for testing. The second is what you need if you need to actually send emails to a user’s inbox. Send to console To log emails to the console, you just have to set EMAIL_BACKEND in settings.py. EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend' Here’s the documentation for reference. Send via SMTP There are a few settings you’ll need to take care of if you want to send live SMTP emails: EMAIL_BACKEND EMAIL_HOST EMAIL_PORT EMAIL_HOST_USER EMAIL_HOST_PASSWORD EMAIL_USE_TLS or EMAIL_USE_SSL If you’re ready to give this a try, I would recommend making an account on SendGrid. Even when your free trial expires, you’ll have 100 free emails to send every day. Not enough to run a big project, but … -
Working around memory leaks in your Django application
Several large Django applications that I’ve worked on ended up with memory leaks at some point. The Python processes slowly increased their memory consumption until crashing. Not fun. Even with automatic restart of the process, there was still some downtime. Memory leaks in Python typically happen in module-level variables that grow unbounded. This might be an lru_cache with an infinite maxsize, or a simple list accidentally declared in the wrong scope. Leaks don’t need to happen in your own code to affect you either. For example, see this excellent write-up by Peter Karp at BuzzFeed where he found one inside Python’s standard library (since fixed!). Workarounds The below workarounds all restart worker processes after so many requests or jobs. This is a simple way to clear out any potential infinitely-accumulating Python objects. If your web server, queue worker, or similar has this ability but isn’t featured, let me know and I’d add it! Even if you don’t see any memory leaks right now, adding these will increase your application’s resilience. Gunicorn If you’re using Gunicorn as your Python web server, you can use the --max-requests setting to periodically restart workers. Pair with its sibling --max-requests-jitter to prevent all your workers … -
Simple Docker Container Tutorial
Docker is all the rage. I cove... -
Django 3.0 Preview
Django 3.0 alpha downloadASGI (Asynchronous Server Gateway Interface)PyCon AU 2019 - Just Add Await: Retrofitting Async into Django by Andrew GodwinDjangoCon Europe 2019 - Sketching out a Django Redesign by Tom Christieforum.djangoproject.com -
Postgres ENUM types in Django
Postgres has the ability to create custom types. There are several kinds of `CREATE TYPE` statement: * composite types * domain types * range types * base types * enumerated types I've used a metaclass that is based on Django's Model classes to do Composite Types in the past, and it's been working fairly well. The current stuff I have been working on made sense to use an Enumerated Type, because there are four possible values, and having a human readable version of them is going to be nicer than using a lookup table. In the first iteration, I used just a TEXT column to store the data. However, when I then started to use an `enum.Enum` class for handling the values in python, I discovered that it was actually storing `str(value)` in the database, rather than `value.value`. So, I thought I would implement something similar to my Composite Type class. Not long after starting, I realised that I could make a cleaner implementation (and easier to declare) using a decorator: {% highlight python %} @register_enum(db_type='change_type') class ChangeType(enum.Enum): ADDED = 'added' CHANGED = 'changed' REMOVED = 'removed' CANCELLED = 'cancelled' ChangeType.choices = [ (ChangeType.ADDED, _('hours added')), (ChangeType.REMOVED, _('hours subtracted')), (ChangeType.CHANGED, … -
Django 3.0 Alpha Released
Big news! The alpha version of Django 3.0 has been released. This is not suitable for production! BUT, this is suitable for sandboxing. Feel free to install it and see if it breaks anything. And if you notice something, make sure to submit a bug! Or you can try to get up and running with ASGI. How exciting! Here’s a link to the official release statement. And here’s a link to the Django 3.0 release notes. -
Limit Access to Only Superusers in Django
So in my other life as a strength and conditioning coach, I have a few online mentees. They basically email me questions and I answer them. I’m using Django to make an interface for them to submit questions and see all the other questions that other members have asked. Plus adding some searching so the old stuff doesn’t just die. I am using the generic UpdateView to answer the questions. This view is marking the question “published” and storing my answer in the database. My issue is that I don’t want the users to be able to see this because then they could be adding answers to questions and people are paying for ME to answer the question. This entails two major parts: Adding a link to the UpdateView in the DetailView Blocking permissions to access the form with UserPassesTestMixin Let’s walk through it. Add link to UpdateView Order doesn’t particularly matter here. You’ll have to do each step. I like to start with the URL, so first, we need to add the URL to our UpdateView to our urls.py file. # app_name/urls.py from .views import AppNameUpdateView # make sure to import your view urlpatterns = [ # other URL … -
Django Has Changed Over Time
Repetition is king for learning. Working with models, views, and templates over and over gets repetitive, but that’s a good thing. Have you ever seen a tutorial online and thought, “That code doesn’t look very familiar.” No repetition, no knowledge. Usually these differences are not WAY off. The similarities give you sense for what you have to do. But after you give it a shot, you get an error. One of the biggest reasons to start your web development journey by learning Django is because it is OLD. Old things are set in their ways. They won’t change so much on you as you slog along. In fact, this was one of the biggest reasons I decided to pick Django as my learning language. You can be sure that you won’t lose much on your investment because what you learn will still be relevant in the future. But still, things have changed in Django. And this makes it difficult to learn on the internet. There’s a lot of wasted time. And time is more valuable than money. To help save you time, I want to outline some of the major differences I’ve run into while learning Django. Note When reading …