Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Python Tutorial - Google Geocoding API
In this short tutorial, we're ... -
Django Foreign Keys: Limiting Choices in Forms
I'm going to assume you're not... -
Heroku vs AWS Which is Best for Your Django project
In this Django Heroku tutorial, I will talk about the relationship and difference between Heroku and AWS. -
Django x Docker to Production on Heroku
Manually deploying a [project ... -
SEO friendly urls in Django
Examples sites built with Django usually use primary keys as the URL identifier for a resource. While this is the simplest and cleanest approach to resource urls, it's not optimized for SEO. It's [debatable](https://webmasters.stackexchange.com/questions/47342/are-keywords-in-urls-good-seo-or-needlessly-redundant) on whether or not keywords in urls matter for Google, but it's obvious that most of the major players have adopted this technique. Below are five examples of popular sites that all use keywords in urls to improve their SEO. - Stackoverflow - [https://stackoverflow.com/questions/8318911/why-does-html-think-chucknorris-is-a-color](https://stackoverflow.com/questions/8318911/why-does-html-think-chucknorris-is-a-color) - Wikipedia - [https://en.wikipedia.org/wiki/Python_(programming_language)](https://en.wikipedia.org/wiki/Python_(programming_language)) - Amazon.com - [https://www.amazon.com/Django-Unchained-Jamie-Foxx/dp/B00BJS7104/](https://www.amazon.com/Django-Unchained-Jamie-Foxx/dp/B00BJS7104/) - Reddit - [https://www.reddit.com/r/soccer/comments/c22vuf/referee_trolls_player/](https://www.reddit.com/r/soccer/comments/c22vuf/referee_trolls_player/) - Dev.to - [https://dev.to/danihodovic/optimizing-postgres-full-text-search-with-django-42hg](https://dev.to/danihodovic/optimizing-postgres-full-text-search-with-django-42hg) If you poke around at the top 100 sites as ranked by [Alexa](https://www.alexa.com/topsites) you will notice that most sites with public facing content use some sort of keywords approach in their urls. Besides being a possible SEO optimization strategy it also improves the chances that a user will click on [your link over other links.](https://stackoverflow.com/a/910741/2966951). Both Chrome and Firefox will match and highlight parts of urls that match the keywords if the user has visited the site previously.  ## Show me the code Django doesn't have views that use keywords in urls out of the box, but it's trivial to add. I'll go … -
Key, index, and path transforms
It’s time for one of the most exciting important parts: transforms! This is probably one of the last few posts on this blog (at least for this year), since GSoC 2019 is almost finished and I’m wrapping up my project soon. In this post, I’ll explain how I implemented key, index, and path transforms to allow querying with JSON values. Actually, they aren’t made into different transforms, but a transform (called KeyTransform) that can be chained to form a JSON path. -
Multiple Category Classification with Keras, Tensorflow, Pandas, Numpy, & Python
In this one, we'll be creating... -
Adding charts to Django admin
## Introduction Django offers a functional admin UI out of the box with CRUD interface for db management. This covers most use cases for basic content and user management systems. However, it doesn't have exploratory views that show summaries or historical trends which is something you'd expect from an admin dashboard. Luckily, the django admin app is extensible and with a few tweaks we can add interactive Javascript charts to the admin. ## Problem I wanted to get a graph overview of the email subscribers over time on [findwork.dev](https://findwork.dev). Is the site growing or stagnating in terms of email subscribers? How many subscribers did we have last month? What week did we gain most subscribers? Are all subscribers verifying their emails? Using exploratory charts we can get a historical overview to how our site is performing. I initially explored the land of ready-made Django admin apps and dashboards. The requirements were that it included charting abilities, was well documented and looked good. While all of the apps I experimented with looked better that the default admin in terms of styling, they were either lacking in documentation or were unmaintained. - [xadmin](https://github.com/sshwsfc/xadmin) - no documentation in English - [django-jet](https://github.com/geex-arts/django-jet) - unmaintained … -
This Is How We Promote a Strong Culture With Distributed Teams
It’s no secret that building and maintaining a robust company culture is challenging for any organization. Is it any more difficult for distributed agile teams, in which an engineering organization is spread across multiple offices — or even across different countries and time zones? I’m Distillery’s General Manager. I’ve been with the company since 2012. As a fast-growing company with 150+ people distributed across 5 offices, 2 countries, and time zones 10 hours apart, I’ve got a strong perspective on this question.  What’s my answer? It’s not more difficult. It’s just different in certain ways. The bottom line is that there’s no “easy” way to build a strong company culture, whether your team is in one office or one hundred. Since it’s a consequence of how a company actually functions, it can’t be manufactured. It’s something you need to pay deliberate, daily attention to.  You can’t manufacture culture, but you can bring it to the surface. You can work to define it, demonstrate it, and protect it. By making our workplace culture a central focus for our people, we create a conversation and working environment that strengthens that culture. Here’s how we do it at Distillery.  Defining Our Company’s Culture  … -
How to Add Database Modifications Beyond Migrations to Your Django Project
On several Django projects I’ve worked on, there has been a requirement for performing database modifications beyond Django migrations. For example: Managing stored procedures Managing check constraints, which weren’t supported before Django 2.2 Importing static data from a file Recording migration operations in a log Let’s look at three approaches to extending Django to do this as neatly as possible. 1. Use Django migrations Often I find developers have only been taught how to use Django migrations for model operations. They might know some SQL, but since they haven’t used it within Django, they assume migrations can’t use SQL directly. They can! Many of the uses of custom migration-style SQL code that I’ve seen could be better implemented within migrations. Let’s take a look at an example we’ll use for the rest of the article. Imagine you’re running a Django version before 2.2 that doesn’t support database check constraints. You might add one to a database table with this SQL: ALTER TABLE myapp_book ADD CONSTRAINT percent_lovers_haters_sum CHECK ( (percent_lovers + percent_haters) = 100 ); This constraint will make the database raise an error for any rows added or updated in the myapp_book table that have a percentage of lovers and … -
API Calls within a Pandas Dataframe API using Pandas Apply
Whenever you're doing data ana... -
Feature-Policy updates - now required for an A+ on SecurityHeaders.com
In my blog post and DjangoCon Europe talk earlier this year How to Score A+ for Security Headers on Your Django Website, I covered that Feature-Policy was a “bonus header”. In a recent update, Scott Helme wrote that an A+ on SecurityHeaders.com now requires Feature-Policy. Also it no longer requires X-Xss-Protection (though it’s still a good idea). Chrome also has some Feature-Policy support enabled by default, so most users will be protected when it’s set. Previously it was hidden behind the “experimental web features” flag, but this is now only used for enabling Feature-Policy support for certain features. Opening the JavaScript console and querying for the list of features on Chrome 75 without the experimental flag on, I see 18 features allowed: > document.featurePolicy.allowedFeatures() < [ "accelerometer", "ambient-light-sensor", "autoplay", "camera", "encrypted-media", "focus-without-user-activation", "fullscreen", "geolocation", "gyroscope", "magnetometer", "microphone", "midi", "payment", "picture-in-picture", "speaker", "sync-xhr", "usb", "vr", ] I added some updates to my how-to on these changes. If you haven’t set the header previously, consider adding it for that sweet A+ score! You can add it on your Django apps with my django-feature-policy package. I updated django-feature-policy this morning to version 3.0.0 to include the latest set of features. Since I don’t … -
10 Important Cloud Migration Case Studies You Need to Know
For most businesses considering cloud migration, the move is filled with promise and potential. Scalability, flexibility, reliability, cost-effectiveness, improved performance and disaster recovery, and simpler, faster deployment — what’s not to like?  It’s important to understand that cloud platform benefits come alongside considerable challenges, including the need to improve availability and latency, auto-scale orchestration, manage tricky connections, scale the development process effectively, and address cloud security challenges. While advancements in virtualization and containerization (e.g., Docker, Kubernetes) are helping many businesses solve these challenges, cloud migration is no simple matter.  That’s why, when considering your organization’s cloud migration strategy, it’s beneficial to look at case studies and examples from other companies’ cloud migration experiences. Why did they do it? How did they go about it? What happened? What benefits did they see, and what are the advantages and disadvantages of cloud computing for these businesses? Most importantly, what lessons did they learn — and what can you learn from them?  With that in mind, Distillery has put together 10 cloud migration case studies your business can learn from. While most of the case studies feature companies moving from on-premise, bare metal data centers to cloud, we also look at companies moving … -
Subquery and Subclasses
Being able to use correlated subqueries in the Django ORM arrived in 1.11, and I also backported it to 1.8. Quite commonly, I am asked questions about how to use these, so here is an attempt to document them further. There are three classes that are supplied with Django, but it's easy to write extensions using subclassing. Let's first look at an example of how you might want to use the included classes. We'll consider a set of temperature sensors, each with a name and a code, both of which are unique. These sensors will log their current temperature at some sort of interval: maybe it's regular, maybe it varies between devices. We want to keep every reading, but want to only allow one reading for a given sensor+timestamp. {% highlight python %} class Sensor(models.Model): location = models.TextField(unique=True) code = models.TextField(unique=True) class Reading(models.Model): sensor = models.ForeignKey(Sensor, related_name='readings') timestamp = models.DateTimeField() temperature = models.DecimalField(max_digits=6, decimal_places=3) class Meta: unique_together = (('sensor', 'timestamp'),) {% endhighlight %} Some of the things we might want to do for a given sensor: * Get the most recent temperature * Get the average temperature over a given period * Get the maximum temperature over a given period … -
Django SASS/SCSS Tutorial Series
I will teach you how to use SASS/SCSS in your Django projects. -
How to use SCSS/SASS in your Django project (NPM Way)
In this blog post, I will talk about how to use SCSS/SASS in your Django project with NPM. -
Run Django tests in the main database
Recently I started working on a Django project, loaded the prod database into it, and ran the tests. Then the data was gone. I know some people intentionally want to run tests on the "main database", and there is a very simple way to achieve this. Simply inherit from a test class that doesn't wrap your db interactions in transactions or puts them on a different test database. Then the database in the settings will be used. Raw import unittest class YourDjangoTest(unittest.TestCase): pass -
Run Django tests in the main database
Recently I started working on a Django project, loaded the prod database into it, and ran the tests. Then the data was gone. I know some people intentionally want to run tests on the "main database", and there is a very simple way to achieve this. Simply inherit from a test class that doesn't wrap your db interactions in transactions or puts them on a different test database. Then the database in the settings will be used. Raw import unittest class YourDjangoTest(unittest.TestCase): pass -
The Right Agile Foundations Will Boost Your Success
Agile project management has strong foundations in open source software development and mobile application development for iOS & Android. In the course of the last decade, Agile project management techniques have grown in popularity. Agile methodologies first arose in the Web 2.0 era to speed up innovation in software development and get to market at lower costs, allowing startups and SMEs to better compete with enterprise corporations. In today’s Cloud era, Agile has become an industry standard and companies integrate it with DevOps techniques. Businesses of any size can use Agile techniques to implement their unique goals and requirements with cost-efficiency and innovation. To leverage Agile project management methodologies successfully, businesses must lay the proper foundations. In order to lay these foundations, it is important to understand the history and evolution of Agile project management. A Brief History of How Agile Project Management Evolved How Agile Evolved: The DotCom Era “The traditional Project Management (waterfall) approach is linear where all the phases of a process occur in sequence. Its concept depends on predictable tools and predictable experience. Each and every project follows the same life cycle which includes the stages such as feasibility, plan, design, build, test, production, (&) support…. … -
Web maps: the Mer et Demeures project
Since ancient times, maps have always allowed to seek and find information in “spatial mode”. Today it is very common for them to be inserted into websites, as we did in the case of Mer et Demeures. -
How Serverless Is Changing Security: The Good, Bad, Ugly, and How to Fix It
Serverless fundamentally changes security. Sometimes for the better. Sometimes for the worse. Sometimes a little in the middle. To keep data and applications secure in our increasingly serverless world, startups and enterprises of all sizes need to understand what’s different, why it matters, and what they should do to protect themselves. In his talk at O’Reilly’s 2017 Velocity Conference, “Serverless Security: What’s Left to Protect?”, Guy Podjarny, co-founder and CEO of open source security platform Snyk, effectively boiled down the basics. In this article, we’ve summarized his 40-minute presentation and bookended it with a bit of helpful context. (Podjarny’s full talk can be found here.) Podjarny focuses on the ways in which serverless is better, neutral, and worse for security. Where possible, he offers up tips and tricks. We’ve added background on what serverless is, why businesses are adopting it so quickly, and what you should do next. What Is Serverless? Serverless architectures are cloud-based and service-based. This means that, instead of provisioning and managing their own servers, organizations buy cloud computing services such as the following: Software as a Service, or SaaS, delivers centrally hosted and managed software on demand to end users over the internet (e.g., Gmail, Slack, … -
Django's Test Case Classes and a Three Times Speed-Up
This is a story about how I sped up a client’s Django test suite to be three times faster, through swapping the test case class in use. Speeding up test runs is rarely a bad thing. Even small teams can repeat their test run hundreds of times per week, so time saved there is time won back. This keeps developers fast, productive, and happy. Django’s Test Case Classes A quick refresher on how the Django’s three basic test case classes affect the database: SimpleTestCase is the simplest one. It provides the basic features of unittest.TestCase plus some Django extras. It blocks database access by default, because it doesn’t do anything to isolate changes you would make there. You should use it for testing components that don’t need the database. TransactionTestCase extends SimpleTestCase to allow database modifications. It resets the database at the end by removing all rows from all database tables. This is slow, but reliable. You should use it when you need the database, but you can’t use TestCase… TestCase is the class you should normally use. It extends TransactionTestCase and replaces its database reset process with one that uses transactions. This is much faster as the database only … -
SongSearch autocomplete rate now 2+ per second
By analyzing my Nginx logs, I've concluded that SongSearch's autocomplete JSON API now gets about 2.2 requests per second. I.e. these are XHR requests to /api/search/autocomplete?q=.... Roughly, 1.8 requests per second goes back to the Django/Elasticsearch backend. That's a hit ratio of 16%. These Django/Elasticsearch requests take roughly 200ms on average. I suspect about 150-180ms of that time is spent querying Elasticsearch, the rest being Python request/response and JSON "paperwork". Caching strategy Caching is hard because the queries are so vastly different over time. Had I put a Redis cache decorator on the autocomplete Django view function I'd quickly bloat Redis memory and cause lots of evictions. What I used to do was something like this: def search_autocomplete(request): q = request.GET.get('q') cache_key = None if len(q) < 10: cache_key = 'autocomplete:' + q results = cache.get(cache_key) if results is not None: return http.JsonResponse(results) results = _do_elastisearch_query(q) if cache_key: cache.set(cache_key, results, 60 * 60) return http.JsonResponse(results) However, after some simple benchmarking it was clear that using Nginx' uwsgi_cache it was much faster to let the cacheable queries terminate already at Nginx. So I changed the code to something like this: def search_autocomplete(request): q = request.GET.get('q') results = _do_elastisearch_query(q) response = http.JsonResponse(results) … -
SongSearch autocomplete rate now 2+ per second
By analyzing my Nginx logs, I've concluded that SongSearch's autocomplete JSON API now gets about 2.2 requests per second. I.e. these are XHR requests to /api/search/autocomplete?q=.... Roughly, 1.8 requests per second goes back to the Django/Elasticsearch backend. That's a hit ratio of 16%. These Django/Elasticsearch requests take roughly 200ms on average. I suspect about 150-180ms of that time is spent querying Elasticsearch, the rest being Python request/response and JSON "paperwork". Caching strategy Caching is hard because the queries are so vastly different over time. Had I put a Redis cache decorator on the autocomplete Django view function I'd quickly bloat Redis memory and cause lots of evictions. What I used to do was something like this: def search_autocomplete(request): q = request.GET.get('q') cache_key = None if len(q) < 10: cache_key = 'autocomplete:' + q results = cache.get(cache_key) if results is not None: return http.JsonResponse(results) results = _do_elastisearch_query(q) if cache_key: cache.set(cache_key, results, 60 * 60) return http.JsonResponse(results) However, after some simple benchmarking it was clear that using Nginx' uwsgi_cache it was much faster to let the cacheable queries terminate already at Nginx. So I changed the code to something like this: def search_autocomplete(request): q = request.GET.get('q') results = _do_elastisearch_query(q) response = http.JsonResponse(results) … -
Single-file Python/Django Deployments
This post covers portions of my talk, Containerless Django, from DjangoCon US 2018. Deploying Python has improved significantly since I started working with it over a decade ago. We have virtualenv, pip, wheels, package hash verification, and lock files. Despite all the improvements, it still feels harder than it needs to be. Installing a typical large project has many steps, each one easy to trip up on: Install Python Install build tools (pip/virtualenv, pipenv, poetry, etc.) Install build dependencies (C compiler, development libraries, etc.) Download the code Run the build tools If you're using Node to build client-side files for a website, repeat steps 1-5 for that It's normal for ops teams to repeat this process as part of the automated testing and then again on every server the project is deployed to. It's no wonder Docker has become so popular because of the ease in which you can build-once and deploy-everywhere. But Docker is a heavy-handed solution and doesn't fit for every project. I envy the simplicity of languages like Go where you can compile your project down to a single binary that runs without any external dependencies. Even Java's JAR file format which requires Java to be preinstalled, …
