Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Things I want to remember about SSH
SSH, short for Secure Shell, is a protocol for secure network communications. It is widely used for executing commands on remote servers, and for file uploads or downloads. If you are working with Django, use Git version control, or administrate servers, you surely are using SSH. In this post, I want to share some technical details about it. Secure Shell is using private and public key pairs. You can either use automatically generated private and public keys combined with a password, or manually generated private and public keys. In the latter case, you need to keep your private key on your computer and upload the public key to the remote server. Creating a pair of SSH keys manually If you are using GitHub, Bitbucket, DigitalOcean, or some other service, you might have seen the possibility to upload public SSH keys for direct access to remote servers. Here is how you usually create the SSH keys on the computer from which you want to establish a secure connection (your local machine or one of your servers that has access to other servers or services). In the Terminal you would execute these commands: $ ssh-keygen $ ssh-agent /usr/local/bin/bash $ ssh-add ~/.ssh/id_rsa The … -
Things I want to remember about SSH
SSH, short for Secure Shell, is a protocol for secure network communications. It is widely used for executing commands on remote servers, and for file uploads or downloads. If you are working with Django, use Git version control, or administrate servers, you surely are using SSH. In this post, I want to share some technical details about it. Secure Shell is using private and public key pairs. You can either use automatically generated private and public keys combined with a password, or manually generated private and public keys. In the latter case, you need to keep your private key on your computer and upload the public key to the remote server. Creating a pair of SSH keys manually If you are using GitHub, Bitbucket, DigitalOcean, or some other service, you might have seen the possibility to upload public SSH keys for direct access to remote servers. Here is how you usually create the SSH keys on the computer from which you want to establish a secure connection (your local machine or one of your servers that has access to other servers or services). In the Terminal you would execute these commands: $ ssh-keygen$ ssh-agent /usr/local/bin/bash$ ssh-add ~/.ssh/id_rsa The id_rsa is … -
Middleware
Weekly Django Chat newsletterMiddleware in Django DocsCarlton's DjangoCon US 2019 talk on Django as a MicroframeworkWSGIBaseHandler class in Django source code -
Testing Cloudflare workers
I like Cloudflare workers. They allow you to execute arbitrary Javascript as close to your users as possible. For simple use cases your scripts probably don't need to be tested, but once they grow in size and complexity you probably want to be confident that they don't break production. Something I dislike about the serverless trend is that most platforms are hard to test and debug. Testing is poorly documented and examples are far and few between. I wrote this blog post after implementing a non-trivial Cloudflare worker script for a client. The script used the Caching API, communicated with a backend server and manipulated HTTP headers. It required correctness tests before we rolled it out in production. I dug through the Cloudflare documentation and related blog posts to find testing examples, but there wasn't enough material out there so I figured I'd consolidate the knowledge I have on testing Cloudflare workers. The starting point for testing workers is in the [Cloudflare](https://developers.cloudflare.com/workers/archive/deploying-workers/unit-testing/) documentation. However I struggled get the examples to work and it didn't cover common Node.js testing techniques such as stubbing, mocking and running debuggers. This post will cover common testing techniques and scenarios I use to test Cloudflare … -
Testing Cloudflare workers
I like Cloudflare workers. They allow you to execute arbitrary Javascript as close to your users as possible. For simple use cases your scripts probably don't need to be tested, but once they grow in size and complexity you probably want to be confident that they don't break production. Something I dislike about the serverless trend is that most platforms are hard to test and debug. Testing is poorly documented and examples are far and few between. I wrote this blog post after implementing a non-trivial Cloudflare worker script for a client. The script used the Caching API, communicated with a backend server and manipulated HTTP headers. It required correctness tests before we rolled it out in production. I dug through the Cloudflare documentation and related blog posts to find testing examples, but there wasn't enough material out there so I figured I'd consolidate the knowledge I have on testing Cloudflare workers. The starting point for testing workers is in the [Cloudflare](https://developers.cloudflare.com/workers/archive/deploying-workers/unit-testing/) documentation. However I struggled get the examples to work and it didn't cover common Node.js testing techniques such as stubbing, mocking and running debuggers. This post will cover common testing techniques and scenarios I use to test Cloudflare … -
Signals
Signals in the Django docs -
Migrating from Python 2 to 3 at EdX - David Ormsbee & Nimisha Asthagiri
EdX WebsiteEdX Github Repodjango-waffleOpen edX Proposals -
Multipage Forms in Django
Introduction Most online forms fit on a single page. Think of a "join our forum" or "contact us" form into which the user enters a name, email address, and maybe a few other pieces of information. If you're building this kind of functionality into a Django site, you can take advantage of Django's built-in form classes. These are especially handy when dealing with model forms, where the form fields correspond to the fields on a model that will be recorded in your database. But what if you need a form that spans more than one page? Like a multipage job application where your personal details are on page 1, your relevant experience is on page 2, and so on? There are third-party libraries to help with this, but if you do it yourself you can sidestep a dependency and maybe become more familiar with Django's form handling. So let's do that. In the post below we'll go step-by-step through the creation of a multipage job application form. We'll start with the simplest functionality and then make it (slightly) more sophisticated. The most important modules ("models.py", "forms.py", and "views.py") will be reproduced here, but a working, standalone project is available from … -
How To Remove A Menu Item From Wagtail Settings Menu
Let's say I want to remove the User menu item from Wagtail settings menu. All I need to do is to put this hook into wagtail_hooks.py file of my Wagtail CMS app: from wagtail.core import hooks @hooks.register('construct_settings_menu') def hide_user_menu_item(request, menu_items): menu_items[:] = [item for item ... Read now -
Working with Dates and Times in the Forms
HTML5 comes with a bunch of new types for the input fields that are rendered as rich native widgets. Browsers even restrict invalid values and validate the input immediately. Let's explore how we could make use of them in Django forms. We will be using an Exhibition model with models.DateField, models.TimeField, and models.DateTimeField: # exhibitions/models.pyfrom django.db import modelsfrom django.utils.translation import gettext_lazy as _class Exhibition(models.Model): title = models.CharField(_("Title"), max_length=200) start = models.DateField(_("Start")) end = models.DateField(_("End"), blank=True, null=True) opening = models.TimeField(_("Opening every day")) closing = models.TimeField(_("Closing every day")) vernissage = models.DateTimeField(_("Vernissage"), blank=True, null=True) finissage = models.DateTimeField(_("Finissage"), blank=True, null=True) class Meta: verbose_name = _("Exhibition") verbose_name_plural = _("Exhibitions") def __str__(self): return self.title Here is a quick model form for the Exhibition model: # exhibitions/forms.pyfrom django import formsfrom .models import Exhibitionclass ExhibitionForm(forms.ModelForm): class Meta: model = Exhibition fields = "__all__" If we now open a Django shell and create an instance of the model form with some initial values, then print the form as HTML to the console, we will notice, that all date and time fields are rendered as <input type="text" /> and the values for the dates are in a local format, not the ISO standard YYYY-MM-DD: (venv)$ python manage.py shell>>> from exhibitions.forms … -
Heroku Logs Tutorial
In this Heorku tutorial, I talked about how to use heroku logs and how to use Sentry to record error log for Heroku app. -
Working with Dates and Times in the Forms
HTML5 comes with a bunch of new types for the input fields that are rendered as rich native widgets. Browsers even restrict invalid values and validate the input immediately. Let's explore how we could make use of them in Django forms. We will be using an Exhibition model with models.DateField, models.TimeField, and models.DateTimeField: # exhibitions/models.py from django.db import models from django.utils.translation import gettext_lazy as _ class Exhibition(models.Model): title = models.CharField(_("Title"), max_length=200) start = models.DateField(_("Start")) end = models.DateField(_("End"), blank=True, null=True) opening = models.TimeField(_("Opening every day")) closing = models.TimeField(_("Closing every day")) vernissage = models.DateTimeField(_("Vernissage"), blank=True, null=True) finissage = models.DateTimeField(_("Finissage"), blank=True, null=True) class Meta: verbose_name = _("Exhibition") verbose_name_plural = _("Exhibitions") def __str__(self): return self.title Here is a quick model form for the Exhibition model: # exhibitions/forms.py from django import forms from .models import Exhibition class ExhibitionForm(forms.ModelForm): class Meta: model = Exhibition fields = "__all__" If we now open a Django shell and create an instance of the model form with some initial values, then print the form as HTML to the console, we will notice, that all date and time fields are rendered as <input type="text" /> and the values for the dates are in a local format, not the ISO standard YYYY-MM-DD: … -
DjangoCon 2019 Delivered Again
Above: Django Fellow Carlton Gibson gives a talk on "Your Web Framework Needs You: An Update." Again this year, DjangoCon more than delivered on its promise of something for everyone. A keynote on burnout and balance? Thoroughly entertaining! Jessica Rose’s talk left us in stitches. Who knew a talk about occupational burnout could be so fun? It was one of a wide range of subjects covered in three days of talks, bookended by a day of tutorials and two days of sprints. The conference took place in San Diego and ran from September 22 - 26. What a great week of learning and networking! Caktus was honored to sponsor again this year, our tenth in a row. Attending this year from Caktus were Karen Tracey, lead developer and technical director; Jeremy Gibson, developer; Erin Mulaney, contractor developer; and myself, Ian Huckabee, CBDO. The Talks … The sessions packed a strong lineup of speakers with a diversity of topics, and this year, every word was captured with closed captioning. There were the practical: Luan Fonseca, for example, went to great lengths to ensure you’re never left with technical debt in this era of sustainable software. He addressed how to find it, … -
DjangoCon 2019 Delivered Again
Above: Django Fellow Carlton Gibson gives a talk on "Your Web Framework Needs You: An Update." Again this year, DjangoCon more than delivered on its promise of something for everyone. A keynote on burnout and balance? Thoroughly entertaining! Jessica Rose’s talk left us in stitches. Who knew a talk about occupational burnout could be so fun? It was one of a wide range of subjects covered in three days of talks, bookended by a day of tutorials and two days of sprints. The conference took place in San Diego and ran from September 22 - 26. What a great week of learning and networking! Caktus was honored to sponsor again this year, our tenth in a row. Attending this year from Caktus were Karen Tracey, lead developer and technical director; Jeremy Gibson, developer; Erin Mulaney, contractor developer; and myself, Ian Huckabee, CBDO. The Talks … The sessions packed a strong lineup of speakers with a diversity of topics, and this year, every word was captured with closed captioning. There were the practical: Luan Fonseca, for example, went to great lengths to ensure you’re never left with technical debt in this era of sustainable software. He addressed how to find it, … -
Fixing anymail/mailgun "Domain not found" error
Are you swearing at mailgun now? Have you smashed your keyboard already? Do you keep receiving the Domain not found: domainname.com error although you're 100% sure you've set everything up correctly? Yeah, I've been there and probably, I have a solution for you. Chances are, your ... Read now -
My Most Used Pytest Commandline Flags
Pytest is quickly becoming the “standard” Python testing framework. However it can be overwhelming to new users. pytest --help currently outputs 275 lines of command line flags and options. Where do you even begin? I searched my ZSH history for my recent usage of Pytest. I found 184 unique invocations, out of which I split the command line flags I used. Here are the top five flags I used by frequency. 1. -v -v, or --verbose, increases the verbosity level. The default level outputs a reasonable amount of information to debug most test failures. However when there are many differences between actual and expected data, some get hidden. In such cases Pytest normally appends a message to the failure text such as: ...Full output truncated (23 lines hidden), use '-vv' to show This tells us to see the hidden lines, we need to rerun the tests with verbosity level 2. To do this we can pass -v twice as -v -v, or more easily as -vv. Another change from passing -v at least once is that the test names are output one per line as they run: $ pytest -v tests.py::test_parrot_status PASSED [100%] Sometimes I use this when looking for … -
Search
Django Search Tutorialdjango-filterMDN on sending form data and form data validationDjango Q Objectsdjango.contrib.postgres.searchPostgreSQL Full Text SearchEuroPython 2017 - Full-Text Search in Django with PostgreSQL by Paulo MelchiorreDjangoCon Europe 2018 - On The Look-Out For Your Data by Markus HoltermannDjangoCon US 2015 - Beyond the basics with Elasticsearch by Honza Kral -
Django Projects – ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to… -
Django Projects - 99 ideas for beginners
Learning Django, I found it really helpful to go through some beginner tutorials. But eventually, you have to fly on your own. You have to learn how to plan out a project and how to research what you don’t know. We believe in learn by doing. And that there’s no better time to start than now. It doesn’t matter if it already exists. Your not building software to change the world, you’re building software to learn web development. Here are some ideas to get you started. 99 Beginner Django Projects Track sleep Track customized habit performance Track customized habits of users Send daily reminders in email Send daily reminders in SMS Store a list of current projects and log entries Freelance time tracking on current projects Freelance project tracking with view ability for project owners Freelance project tracking with comment ability for project owners Send daily inspirational quote to your email Send daily inspirational quote in SMS Make a blog like WordPress Make an e-commerce store like Gumroad Make someone else a blog Make someone else an e-commerce store Collect your thoughts on books Make an e-commerce store with Amazon Affiliate links Make an e-commerce store that automatically pulls Amazon … -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Tips On Writing Data Migrations in Django Application
Introduction In a Django application, when schema changes Django automatically generates a migration file for the schema changes. We can write additional migrations to change data. In this article, we will learn some tips on writing data migrations in Django applications. Use Management Commands Applications can register custom actions with manage.py by creating a file in management/commands directory of the application. This makes it easy to (re)run and test data migrations. Here is a management command which migrates the status column of a Task model. from django.core.management.base import BaseCommand from library.tasks import Task class Command(BaseCommand): def handle(self, *args, **options): status_map = { 'valid': 'ACTIVE', 'invalid': 'ERROR', 'unknown': 'UKNOWN', } tasks = Task.objects.all() for tasks in tasks: task.status = status_map[task.status] task.save() If the migration is included in Django migration files directly, we have to rollback and re-apply the entire migration which becomes cubersome. Link Data Migrations & Schema Migrations If a data migration needs to happen before/after a specific schema migration, include the migration command using RunPython in the same schema migration or create seperate schema migration file and add schema migration as a dependency. def run_migrate_task_status(apps, schema_editor): from library.core.management.commands import migrate_task_status cmd = migrate_task_status.Command() cmd.handle() class Migration(migrations.Migration): dependencies = [ … -
Blackifying and fixing bugs
Since version 0.9 of Evennia, the MU*-creation framework, was released, work has mainly been focused on bug fixing. But there few new features also already sneaked into master branch, despite technically being changes slated for Evennia 1.0. On FrontendsContributor friarzen has chipped away at improving Evennia's HTML5 web client. It already had the ability to structure and spawn any number of nested text panes. In the future we want to extend the user's ability to save an restore its layouts and allow developers to offer pre-prepared layouts for their games. Already now though, it has gotten plugins for handling both graphics, sounds and video: Inline image by me (griatch-art.deviantart.com) A related fun development is Castlelore Studios' development of an Unreal Engine Evennia plugin (this is unaffiliated with core Evennia development and I've not tried it, but it looks pretty nifty!): Image ©Castlelore Studios On BlackEvennia's source code is extensively documented and was sort of adhering to the Python formatting standard PEP8. But many places were sort of hit-and-miss and others were formatted with slight variations due to who wrote the code. After pre-work and recommendation by Greg Taylor, Evennia has adopted the black autoformatter for its source code. I'm not really convinced that … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Update to speed comparison for Redis vs PostgreSQL storing blobs of JSON
Last week, I blogged about "How much faster is Redis at storing a blob of JSON compared to PostgreSQL?". Judging from a lot of comments, people misinterpreted this. (By the way, Redis is persistent). It's no surprise that Redis is faster. However, it's a fact that I have do have a lot of blobs stored and need to present them via the web API as fast as possible. It's rare that I want to do relational or batch operations on the data. But Redis isn't a slam dunk for simple retrieval because I don't know if I trust its integrity with the 3GB worth of data that I both don't want to lose and don't want to load all into RAM. But is it entirely wrong to look at WHICH database to get the best speed? Reviewing this corner of Song Search helped me rethink this. PostgreSQL is, in my view, a better database for storing stuff. Redis is faster for individual lookups. But you know what's even faster? Nginx Nginx?? The way the application works is that a React web app is requesting the Amazon product data for the sake of presenting an appropriate affiliate link. This is done … -
Speeding up Postgres using a RAM disk?
I happened to come across [Speeding up Django unit tests with SQLite, keepdb and /dev/shm](http://www.obeythetestinggoat.com/speeding-up-django-unit-tests-with-sqlite-keepdb-and-devshm.html) today. I can't quite do that, because we use a bunch of Postgres specific things in our project, but it did make me think "Can we run Postgres on a RAM disk?" Following up on this, it turns out that using a TABLESPACE in postgres that is on a RAM disk is a really bad idea, but it should be possible to run a seperate database cluster (on a different port) that could have the whole cluster on the database. It's possible to create a RAM disk on macOS, and init a cluster there: {% highlight bash %} #! /bin/sh PG_CTL=/Applications/Postgres.app/Contents/Versions/10/bin/pg_ctl new_disk=$(hdid -nomount ram://1280000) newfs_hfs $new_disk mkdir /tmp/ramdisk mount -t hfs $new_disk /tmp/ramdisk mkdir /tmp/ramdisk/pg PG_CTL initdb -D /tmp/ramdisk/pg {% endhighlight %} This needs to be run once, after bootup (and probably before Postgres.app starts it's servers). You'd also need to do the setup in Postgres.app to show where it is (alternatively, you could just start it on the port you want in the script above). But, is it faster? Let's look at the results. The "Initial run" is the first run of tests after …
