Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Fastest Redis configuration for Django
I have an app that does a lot of Redis queries. It all runs in AWS with ElastiCache Redis. Due to the nature of the app, it stores really large hash tables in Redis. The application then depends on querying Redis for these. The question is; What is the best configuration possible for the fastest service possible? Note! Last month I wrote Fastest cache backend possible for Django which looked at comparing Redis against Memcache. Might be an interesting read too if you're not sold on Redis. Options All options are variations on the compressor, serializer and parser which are things you can override in django-redis. All have an effect on the performance. Even compression, for if the number of bytes between Redis and the application is smaller, then it should have better network throughput. Without further ado, here are the variations: CACHES = { "default": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', 'redis://127.0.0.1:6379') + '/0', "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", } }, "json": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', 'redis://127.0.0.1:6379') + '/1', "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", "SERIALIZER": "django_redis.serializers.json.JSONSerializer", } }, "ujson": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', 'redis://127.0.0.1:6379') + '/2', "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", "SERIALIZER": "fastestcache.ujson_serializer.UJSONSerializer", } }, "msgpack": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', … -
Distillery QA Team Makes the Finals in the 2nd Open EU Championship as They Prepare for the Grand Final in Poland
The Distillery Quality Assurance team competed in the 2nd Open European Championship for Web and Mobile Specialists and made it to the finals! Anna Varetsa, QA Manager, and Ekaterina Golubeva, QA Specialist, excelled at solving the different challenges presented by Jury Board, a selection of highly qualified and experienced industry professionals. The results were revealed on April 28th, after two weeks of nervously waiting for the outcome. The post Distillery QA Team Makes the Finals in the 2nd Open EU Championship as They Prepare for the Grand Final in Poland appeared first on Distillery. -
Django Version Viewer Announcement
Presenting a new open source plugin: The Django Version Viewer! Imaginary Landscape is pleased to present a helpful new plugin for the Django Web Framework - The Django Version Viewer! If you find yourself frequently switching between projects or perhaps looking at a codebase you haven’t seen in some time, it’s often useful to know the contents of the project’s virtual environment. Typically this information is only accessible either by examining your project’s requirements.txt (which may or may not accurately reflect what is installed in your environment) or by accessing your app’s server directly to examine the app’s virtual environment. The Django Version Viewer is a handy tool that exposes this information in a handful of ways: Through the Django Admin panel The plugin conveniently exposes a link at the top of your admin index pane to expose a simple drop down listing all of your project dependencies. Through a DjangoCMS Toolbar Menu Item If Django CMS is installed, a new menu item will be added to the CMS Toolbar Page Menu that will allow opening the version viewer popup. Through an API call Alternatively, you can make a request to a URL at www.yourAppsDomain/django_version_viewer to retrieve a list of … -
Django and working with large database tables
The slides from my presentation for the Django Stockholm Meetup group. Contains small comparison between MySQL and PostgreSQL in terms of performance when modigying the tables structure. Django and working with large database tables from Ilian Iliev -
A simple Python email gateway
Say you have a Django web application that you want to integrate with emails to make it possibile to send data and files to your web application over SMTP. The good news is that Python has a simple SMTP daemon in the standard library, together with modules to parse emails. Let’s see how to create a simple email gateway on top of these tools. Using the following Python module you can setup a simple pure Python email server on port 25, parse the incoming emails and make a multipart/form-data POST HTTP request to your web application. You’ll need the awesome Requests library for the HTTP request to work. In a production environment you can setup supervisor to launch your simple SMTP server. With a simple configuration like this: [program:smtp_server] command = /home/ubuntu/.virtualenvs/venv/bin/python /home/ubuntu/mail_gateway/smtp_server.py user = root Then try to send an email to your server, if the server public IP has been associated to a DNS record yourserver.example.com you can use something like test@yourserver.example.com as a recipient, and you should see the incoming email in the logs. Have fun with emails! -
A simple Python email gateway
Say you have a Django web application that you want to integrate with emails to make it possibile to send data and files to your web application over SMTP. The good news is that Python has a simple SMTP daemon in the standard library, together with modules to parse emails. Let’s see how to create a simple email gateway on top of these tools. Using the following Python module you can setup a simple pure Python email server on port 25, parse the incoming emails and make a multipart/form-data POST HTTP request to your web application. You’ll need the awesome Requests library for the HTTP request to work. https://gist.github.com/baxeico/d5d6f855774ba9f72e11909648a46cae In a production environment you can setup supervisor to launch your simple SMTP server. With a simple configuration like this: [program:smtp_server] command = /home/ubuntu/.virtualenvs/venv/bin/python /home/ubuntu/mail_gateway/smtp_server.py user = root Then try to send an email to your server, if the server public IP has been associated to a DNS record yourserver.example.com you can use something like test@yourserver.example.com as a recipient, and you should see the incoming email in the logs. Have fun with emails! The post A simple Python email gateway appeared first on Augusto Destrero - Freelance developer and sysadmin. -
A simple Python email gateway
Say you have a Django web application that you want to integrate with emails to make it possibile to send data and files to your web application over SMTP. The good news is that Python has a simple SMTP daemon in the standard library, together with modules to parse emails. Let’s see how to create a simple email gateway on top of these tools. Using the following Python module you can setup a simple pure Python email server on port 25, parse the incoming emails and make a multipart/form-data POST HTTP request to your web application. You’ll need the awesome Requests library for the HTTP request to work. import smtpd import asyncore import logging import email from email.header import decode_header import requests logger = logging.getLogger(__name__) class CustomSMTPServer(smtpd.SMTPServer): def process_message(self, peer, mailfrom, rcpttos, data): try: logger.debug('Receiving message from: %s:%d' % peer) logger.debug('Message addressed from: %s' % mailfrom) logger.debug('Message addressed to: %s' % str(rcpttos)) msg = email.message_from_string(data) subject = '' for encoded_string, charset in decode_header(msg.get('Subject')): try: if charset is not None: subject += encoded_string.decode(charset) else: subject += encoded_string except: logger.exception('Error reading part of subject: %s charset %s' % (encoded_string, charset)) logger.debug('Subject: %s' % subject) text_parts = [] attachments = {} logger.debug('Headers: %s' … -
Django webpacker - A compression tool to bundles css, js files
In production, it's challenging to developers to improve the performance of an application. One of the technique to improve performance is compressing the CSS, js files into a single file. Django webpacker provides you a single compressed CSS, js file to the page. Instead of generating compressed files for every request in production, we're generating compressed css, js files, updates to html file in the development only. Django webpacker really helps you to generate these compressed files with webpack Django webpacker is a django compressor tool which bundles css, js files to a single css, js file with webpack and updates your html files with respective css, js file path with a single management command.. Django webpacker uses webpack, npm, aws s3 services to provide its features in an effective way. Webpack is a system for bundling the front-end assets of a web application into a few static resources that represent the bulk of the application's CSS, Javascript, and images. npm makes it easy for JavaScript developers to share the code that they've created to develop applications. With the help of packages(A reusable code) will generate compressed css , js files. Django webpacker also supports django-storages to load compressed static files from AWS … -
How to use nested formsets in django
Django Formsets manage the complexity of multiple copies of a form in a view. By using formsets you can know how many forms were their initially, which ones have been changed, and which ones should be deleted. Similar to Forms and Model Forms, Django offers Model Formsets, which simplify the task of creating a formset for a form that handles multiple instances of a model. Django also provides inline formsets which can be used to handle set of objects belong to common foreign key. In the below example models, we can write a inline-formset for handling all children for a parent or all addresses of a child. # models.py class Parent(models.Model): name = models.CharField(max_length=255) class Child(models.Model): parent = models.ForeignKey(Parent) name = models.CharField(max_length=255) class Address(models.Model): child = models.ForeignKey(Child) country = models.CharField(max_length=255) state = models.CharField(max_length=255) address = models.CharField(max_length=255) # forms.py from django.forms.models import inlineformset_factory ChildrenFormset = inlineformset_factory(models.Parent, models.Child, extra=1) AddressFormset = inlineformset_factory(models.Child, models.Address, extra=1) By using above formsets you can handle all children for a parent in one page and can handle all addresses of a child in other page. But if you want to allow the users to add/edit all children along with addresses, all in a single page, then … -
What's great about Django girls to inspire women into programming
Now a days programming is a hot topic across the globe. Django girls are inspiring more women into programming. Which is an international community to teach programming stuff and to getting more women into tech.They run free programming workshops all over the world and offering several opportunities for women to develop coding skills and to make the industry more balanced. It's important to encourage beginners/non-programmers that programming is not hard as they think. Where does it start? It started in 2014 by two women, Ola Sitarska and Ola Sendecka. Their main aim is to fall women into the programming world. Django girls workshops are free and till now they have conducted the events Over 67 countries around the world, where women can learn extensive programming stuff and they will get an awesome programming experience at the first time. It was not only a programming session but something as exciting. What They Do? They will guide all the participants to learn how to code using Django. They will start from the ground level. So, no one left behind. There are so many mentors & volunteers to guide everyone in each step. A participant can move as fast or as slow as … -
How to convert xml content into json using xmltodict
xmltodict is a python package which is used to convert XML data into JSON and vice versa. Within a less span of time, we can also process large amounts of data and get the desired results. Installation: pip install xmltodict It supports both python2.7, 3.4 versions Advantages over other packages: 1. No need to process each node in XML document to get the results 2. Easy to convert complex XML data into JSON 3. It's user-friendly Converting XML to a dictionary: >>> import xmltodict >>> import json >>> json.loads(json.dumps(xmltodict.parse(''' <root> <persons city="hyderabad"> <person name="abc"> <name age="50" mobile="789" /> </person> </persons> <persons city="vizag"> <username></username> <person name="xyz"> <name age="70" mobile="123" /> </person> </persons> </root> '''))) {u'root': {u'persons': [{u'@city': u'hyderabad', u'person': {u'@name': u'abc', u'name': {u'@mobile': u'789', u'@age': u'50'}}}, {u'@city': u'vizag', u'person': {u'@name': u'xyz', u'name': {u'@mobile': u'123', u'@age': u'70'}}}]}} Converting dictionary to XML: >>> xmltodict.unparse({u'root': {u'persons': [{u'@city': u'hyderabad', u'person': {u'@name': u'abc', … -
Django deployments without downtime
This post describes a deployment and maintenance method for Django projects that was designed with scalability in mind. The goal is to push new releases into production at any moment with minimal or no downtime. Upgrades can be performed with unprivileged access to the production server, and rollbacks are possible. I use Gunicorn, Fabric and Supervisord in the examples. -
Stable Django deployments without downtime
This post describes a deployment and maintenance method for Django projects that was designed with scalability in mind. The goal is to push new releases into production at any moment with minimal or no downtime. Upgrades can be performed with unprivileged access to the production server, and rollbacks are possible. I use Gunicorn, Fabric and Supervisord in the examples. -
Machine learning empowered blackkiwi
A blackkiwi which tells your mood. Blackkiwi is a django powered website which uses machine learning to tell if you wrote happy or sad things on your latest Facebook status. And it tends to be wayyy positive. But we will get there… The genesis of blackkiwi There were two main things combined with the genesis of blackkiwi. The first it was this curiosity about Natural Text processing and classificaiton techniques. In particular I wanted to write some classifers to see how well they were performing and I also wanted to try to do and test something new. But I needed some kind of application. This is usually a good trick in programming in general. If you build towards something, it is always easier to stay motivated and actually getting it done, instead of giving up the hobby and end up playing World of tanks on the play :). The second ingredient was to try to test the release process via gitlab, using automatic push via CI to a server. As stack I wanted to use a classic dokku stack which I’m very happy to use, beccause it basically brings the nice and easy way to deploy similar to heroku/gondor style … -
Django REST framework (DRF) with Angular 2+ tutorial (Part 1)
This is the first part of a tutorial series about getting started with Django framework and Angular 2 . The final objective is to create a restful web application which has two separate parts .The first part is the back end which is created with Django and the Django REST framework ,the second part is the front end which is built using Angular 2+ framework . Django REST framework (DRF) with Angular 2+ tutorial (Part 1) Django REST framework (DRF) with Angular 2+ tutorial (Part 2) Django REST framework (DRF) with Angular 2+ tutorial (Part 3) We are going to use Django 1.11 ,Django REST framework 3.6.2 and Angular 2+ . Before starting ,this tutorial has a bunch of development requirements : You need to have Python installed and configured PIP and Virtualenv installed Node.js and Angular 2 CLI installed Objectives of the part 1 In the first part of these tutorial series ,we are going to cover basic introductions to the two server side used frameworks Django and DRF ,and their initial setup . Getting started with Django and DRF . The custom project anatomy and structure (front end and back end) Getting started with Django and DRF Django … -
Django REST framework (DRF) and Angular 2 tutorial (Part 1)
This is the first part of a tutorial series about getting started with Django framework and Angular 2 . The final objective is to create a restful web application which has two separate parts .The first part is the back end which is created with Django and the Django REST framework ,the second part is the front end which is built using Angular 2 framework . We are going to use Django 1.11 ,Django REST framework 3.6.2 and Angular 2 . Before starting ,this tutorial has a bunch of development requirements : You need to have Python installed and configured PIP and Virtualenv installed Node.js and Angular 2 CLI installed Objectives of the part 1 In the first part of these tutorial series ,we are going to cover basic introductions to the two server side used frameworks Django and DRF ,and their initial setup . Getting started with Django and DRF . The custom project anatomy and structure (front end and back end) Getting started with Django and DRF Django is a web framework based on Python which allows developers to quickly prototype and build web applications .It has a powerful ORM that abstracts how you can interact with popular … -
Make More Room: How to Resolve Database Record Limitations
Sometimes in the process of designing a database, developers can underestimate the scope of a project. If that project were to become successful, it would result in significant loads and put undue strain on the system and it’s developers down the line. Eventually, people realize that they are using an INT identity key with a maximum value of 2,147,483,647 as the primary key of a huge table. The post Make More Room: How to Resolve Database Record Limitations appeared first on Distillery. -
How To Write Custom migrations in Django
Migrations are mainly for keeping the data model of your database up-to-date. Django will never run same migration file more than once on the same database. All this is managed by table named "django_migrations" which gets created when we first apply the migrations. For each migration(file), a new record(row) is inserted into this table. There are also a few “special” operations of Migration class which can be used to add our custom migrations. RunSQL This allows us to execute raw SQL. Example: from __future__ import unicode_literals from django.db import migrations, models class Migration(migrations.Migration): dependencies = [ ... dependencies goes here ] operations = [ migrations.RunSQL("INSERT INTO product (title) VALUES ('Product1');"), ] RunPython This can be used to run custom Python code. Here code should be separate function that accepts two arguments - first is an instance of django.apps.registry.Apps containing historical models that match the operation’s place in the project history, and the second is an instance of SchemaEditor. Example: from __future__ import unicode_literals from django.db import migrations def create_countires(apps, schema_editor): Country = apps.get_model("app_name", "Country") Country.objects.bulk_create([ Country(name="India"), Country(name="USA"), ]) class Migration(migrations.Migration): dependencies = [ ... dependencies goes here ] operations = [ migrations.RunPython(create_countires), ] Once we have the function we can … -
Django efficient implementation of Amazon s3 and Cloufront CDN for faster loading.
Django by default to store the files in your local file system. To make your files load quickly and secure we need to go for any third party storage systems. AWS s3 is one of the storage service for the Internet. It is designed to make web-scale computing easier for developers. django has a package called django-storages which can be used to store the files in the amazon s3. and serve them from its cloud front service. Implementing django-storages with amazon s3 as storage service in your django application: To implment django-storages in your django application you need to follow the following steps. 1. Install django-storages: We can install django-storages with the following command. pip install django-storages 2. Keep storages in your installed apps. INSTALLED_APPS = ( ... 'storages', ... ) 3. Change your DEFAULT_FILE_STORAGE setting in your settings.py file: Django by default comes with a setting called DEFAULT_FILE_STORAGE which deals with file storage. By default its value is 'django.core.files.storage.FileSystemStorage'. By keeping this value to DEFAULT_FILE_STORAGE, django, by default tries to store the files on your local machine. To store your files in amazon s3 just change that value to 'storages.backends.s3boto3.S3Boto3Storage' DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' 4. Configure your AWS ACCESS_KEY … -
How To Add A Custom Managers In Django
Django Custom Managers A Manager is used to query database operations which will be provided to Django models. Every model in Django application consists at least one Manager by default. Regarding the Manager names: By default, Django adds a Manager with the name "objects" to every Django model class. However, if you want to use "objects" as a field name (or) if you want to use a name instead of "objects" for the Manager, you can rename it on a per-model basis. To rename the Manager for a given class, define a class attribute of type models.Manager() on that model. For example: from django.db import models class Employee(models.Model): gender_choices = ( ("M", "Male"), ("F", "Female") ) roles_choices = ( ("J", "Junior"), ("S", "Senior"), ) first_name = models.CharField(max_length=200) last_name = models.CharField(max_length=200) email = models.CharField(max_length=250) gender = models.CharField(max_length=1, choices=gender_choices) role = models.CharField(max_length=120, choices=roles_choices, default="J") active = models.BooleanField(default=True) # custom manager replaces objects manger all_employees = models.Manager() def __str__(self): return str(self.first_name) + str(self.last_name) Using the above example model, Employee.objects will throw you an AttributeError exception, but Employee.all_employees.all() will provide a list of all Employee objects. Now, here we go for the Custom Managers: You can use a custom Manager in a particular model … -
Custom validations for serializer fields Django Rest Framework
why we use validators ? Validators are used to validate the data whether it is semantically valid or not. Validation simplifies the data processing Validation avoids the data redundancy Custom Validation for serializer fields Django REST supports both serializers and model serializers. Serializers provides basic validation for fields, In some cases we need to write custom validations for fields . Let's take an example for validation for serializer fields. Case: A company want's recruit for the position of "django developer". Company set age restriction on applicant that his/her age should be greater than twenty and less than thirty years. Method-1: from datetime import date from rest_framework import serializers def age_restriction(dob): today = date.today() age = today.year - dob.year - ((today.month, today.day) < (dob.month, dob.day)) if (not(20 < age < 30)): raise serializers.ValidationError("You are no eligible for the job") return dob class EligibilitySerializer(serializers.Serializer): email = serializers.EmailField() name = serializers.CharField(max_length=200) date_of_birth = serializers.DateField(validators=[age_restriction]) "serializers.Serializer" apply primary validations on the field when we call method "is_valid". Serializer converts the value of the field into python object. After it checks for the attribute "validate_<field_name>" if it has the attribute it will the attribute(method). After this validation it will check for "validators" attribute for the field. … -
Python Memcached Implementation for Django project
A fundamental trade-off in dynamic websites is they’re dynamic. Every time a user requests a page, the Web server fetches data from database then applies logic and renders in templates. This will affect page load time due to time taken by server to fetch data and apply the business logic, That’s where caching comes in. To cache something is to save the result of an expensive calculation so that you don’t have to perform the calculation next time. In This Blog Post, Lets see how to use Memcached for server-side application caching. Memcached is an in-memory key-value pair store, that helps in caching dynamic websites. Django uses python-memcached binding for communication between our web application and Memcached Server. apt-get install memcached #to install memcached-server pip install python-memcached Add the following settings to settings.py or django settings CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'LOCATION': 'unix:/tmp/memcached.sock', } } Now you're all set use caching. from django.core.cache import cache cache.set(cache_key, result, cache_time) # to cache a value cache.set_many({'a': 1, 'b': 2, 'c': 3}) # to cache many keys cache.get(cache_key) # to retive a key from cache cache.get_many(['a', 'b', 'c']) # to retrive many keys from cache To use as Decorator: from django.views.decorators.cache import … -
ShipIt Day Recap Q2 2017
Once per quarter, Caktus employees have the opportunity to take a day away from client work to focus on learning or refreshing skills, testing out ideas, or working on open source contributions. The Q2 2017 ShipIt Day work included building apps, updating open source projects, trying out new tools, and more. Keep reading for the details. -
How to deploy a Django project in 15 minutes with Ansible
In this tutorial I will assume that you are a Django developer and you have built and tested a project locally. It’s time to deploy the project on a public server to let users access your awesome application. So you need a VPS with an SSH access, then you will access the server, install and configure all necessary software (web server, application server, database server), create a database user, configure Django to use it, copy your Django project on the server, migrate the database, collect static files, trial and error, fix, trial and error, … All this boring stuff will take some good hours that you should definitely spend in a more profitable way, don’t you think? The good news is that you can automate almost all the work needed to go from a vanilla VPS to a fully deployed server hosting your Django project. Follow this tutorial and I’ll show you how to leverage the power of Ansible to automate all the needed steps in 15 minutes. Are you ready? Check the time on your clock and follow me! 1. Setup the SSH access to your VPS There are plenty of good VPS providers out there, and the choice … -
How to deploy a Django project in 15 minutes with Ansible
In this tutorial I will assume that you are a Django developer and you have built and tested a project locally. It’s time to deploy the project on a public server to let users access your awesome application. So you need a VPS with an SSH access, then you will access the server, install and configure all necessary software (web server, application server, database server), create a database user, configure Django to use it, copy your Django project on the server, migrate the database, collect static files, trial and error, fix, trial and error, … All this boring stuff will take some good hours that you should definitely spend in a more profitable way, don’t you think? The good news is that you can automate almost all the work needed to go from a vanilla VPS to a fully deployed server hosting your Django project. Follow this tutorial and I’ll show you how to leverage the power of Ansible to automate all the needed steps in 15 minutes. Are you ready? Check the time on your clock and follow me! 1. Setup the SSH access to your VPS There are plenty of good VPS providers out there, and the choice …
