Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
DjangoCon EU 2013
DjangoCon EU has always been something special. The very first European DjangoCon back in 2009 in Prague was a great start and over the years with Berlin, Amsterstam and Zurich in between in grew and grew and got better and better. This year, DjangoCon made a stop in Warsaw and, to be honest, it was the greatest conference I’ve ever been to. Back when the first plans for this year’s event started to leak I was all “what on Earth???”, but the local team and all the helpers including Ola Sitarska, Ola Sendecka, Kuba Kucharski, Tomek Paczkowski and Jarek Piotrowski made it happen: A tech conference in a park, in a tent, that worked!!! The conference Over the years, conferences did evolved into those perfectly organized events where usually only one thing breaks: the WiFi. They usually take place in perfectly AC’d hotels or conference centers and depending on the price you pay you get served hot and cold beverages by people in black & white & bow-ties. Well, DjangoCon EU has always been different being a conference by the community for the community, but this year was like off a whole different planet with the organizers taking a huge ... -
Django: Stop Writing Settings files
At this year's DjangoCon Europe I gave a Lightning talk titled "Stop Writing Settings Files". You can see the slides on SpeakerDeck. After the talk I got some interesting feedback. Some people disagree on some points for very valid reasons so I tought I'd write in more details what I was talking about. Settings across multiple environments, how do they work? A historically famous pattern is the "local_settings trick": it simply consists in adding the following lines at the end of the project's settings: try: from local_settings import * except ImportError: pass This works combined with a local_settings.py that's kept out of source control and managed manually for development-specific or production-specific settings. One issue with this technique is that it's hard to extend the base settings. If you want to add something to the value defined in the base settings, you need to copy the initial value completely. There is no way to add something to INSTALLED_APPS or MIDDLEWARE_CLASSES without redefining the value completely. To solve this problem, another pattern has emerged. Coined by Jacob Kaplan-Moss as The One True Way, it consists in reversing the import flow. Instead of importing local settings from the base settings, just import the ... -
Django 1.5 Cheat Sheet
At Mercurytide, we know all too well the difficulties of memorizing shortcuts when you work in different frameworks. Our skilled developers have created a solid, quick-start cheat sheet with an easy to reference layout. -
Django 1.5 Cheat Sheet
At Mercurytide, we know all too well the difficulties of memorizing shortcuts when you work in different frameworks. Our skilled developers have created a solid, quick-start cheat sheet with an easy to reference layout. -
Django 1.5 Cheat Sheet
At Mercurytide, we know all too well the difficulties of memorizing shortcuts when you work in different frameworks. Our skilled developers have created a solid, quick-start cheat sheet with an easy to reference layout. -
Prehistorical Python: patterns past their prime - Lennart Regebro
Dicts This works now: >>> from collections import defaultdict >>> data = defaultdict(list) >>> data['key'].add(42) It was added in python 2.5. Previously you'd do a manual check whether the key exists and create it if it misses. Sets Sets are very useful. Sets contain unique values. Lookups are fast. Before you'd use a dictionary: >>> d = {} >>> for each in list_of_things: ... d[each] = None >>> list_of_things = d.keys() Now you'd use: >>> list_of_things = set(list_of_things) Sorting You don't need to turn a set into a list before sorting it. This works: >>> something = set(...) >>> nicely_sorted = sorted(something) Previously you'd do some_list.sort() and then turn it into a set. Sorting with cmp This one is old:: >>> def compare(x, y): ... return cmp(x.something, y.something) >>> sorted(xxxx, cmp=compare) New is to use a key. That gets you one call per item. The comparison function takes two items, so you get a whole lot of calls. Here's the new: >>> def get_key(x): ... return x.something >>> sorted(xxxx, key=get_key) Conditional expressions This one is very common! This old one is hard to debug if blank_choice also evaluates to None: >>> first_choice = include_blank and blank_choice or [] There's a ... -
Dynamic models in Django - Juergen Schackmann
The classical approach in django is: Code development You create models. Deployment Tables and columns are created with syncdb. Runtime Models and db tables are populated. This means that models are pretty much static. There is no way to modify them at runtime based on user interactions. You can get something working with for instance hstore in postgresql (see the postgresql talk). His usecase is for medical forms. The contents of those forms should be able to be defined inside the system. There are strict processes for installing medical software, so you cannot just release a new version with a new field. So you must get it to work at runtime. The solution could be to use dynamic models, models created at runtime. Sometimes configuration by subject matter experts is better than code customization by developers. Also, dynamic models reduce the number of deployment cycles. He has some criteria: Performance. Querability, which means the standard django query stuff should work. Django standard tool integration (admin, cache, and so). Supported DB backends. If possible, support all django DB backends. Complexity/maintainability. There are a couple of possible solutions: Entity attribute value (EAV) Colums are stored in separate table rows. Instead of a ... -
The path to continuous deployment - Òscar Vilaplana
If you've got continuous deployment, you've got stable servers. You make big changes in small increments. Continuous deployment forces you to do many good things: Good tests. Repeatable build. Well-configured identical machines. Automated deployment. Migrations and rollbacks Etc. Lots of good things. But let's compare it with lion taming. Originally, lions were beaten into submission, confused and kept in line with whips. Likewise you'll be beaten if you dare to touch the production machine as it might break. Now lions are understood better. Conditioning, behavior/signal mapping, reward and trust are the methods now. We understand that deployment is hard. We have behaviour/signal mapping with code/test/green/deploy. Etc. Continuous deployment: everyone is responsible. Everyone deploys. You automatically learn. Everybody uses the same environment locally for test deployments. The same as on the server. Testing is core. Slow tests are killing. Fast tests. And all types of tests: unit, functional and acceptance tests. Also automatic code checkers. The light must stay green. Quality must stay high, also test quality. You need a repeatable build. And it should include not just code, but also configuration and infrastructure. And... always follow the pipeline. Even in emergencies, follow the pipeline. Peer review, tests, and then the ... -
Class based views: untangling the mess - Russell Keith-Magee
Russell is a Django core dev. Class based views were introduced two years ago, but they weren't greeted with universal acclaim. So he's here to clear up the mess and hopefully make it all more clear for everyone. History In the beginning of Django, there were only views. Function-based views. No generic views. Next, because of DRY, don't repeat yourself, several generic views were added. Listing objects, editing an object, for instance. Editing something happens so often that a generic view inside Django seemed like a good idea. There are some problems here, though. The configuration you can do is limited by the arguments you can give in your URL configuration. No control over the logic view. You can't pass in an alternative view. There's no re-use between views. You could "fix" this by adding more and more arguments and allow passing in callables and so, but in the end you're almost building what you'd already get with object oriented class inheritance... So... Next: class based views. It landed in Django 1.3 after it didn't work out to get it in 1.1 or 1.2. What went wrong? Then the wheels fell off. What went wrong? Fundamental confusion over purpose. There ... -
Lightning talks day 3 - Djangocon.eu
html5lib Browsers are terribly forgiving. Python's parsers don't deal with everything, even valid html5 docs. html5lib was a problem. Google code and so and not python 3 compatible. The new html5lib supports python 3. Github, readthedocs, works fine! See https://pypi.python.org/pypi/html5lib Real time web - Aymeric Augustin He looked at web sockets in django. He played with tulip, Guido's library for async python. He had 1000 processes calculating a 'game of life' screen and django connected with them just fine and pushed the result to the browser. PyWaw PyWaw is a python community in Warsaw. They have now had 24 meetings with about 55 attendees. At the last meeting they even had 100 people attending. They are not alone in Poland, there are other user groups. So... go back to your cities and start user groups! Scrapy Screen scraping is when you need to get structured information from the web, quickly and with no hassle. Scrapy takes the hassle out of screen scaping. It takes away the pain of parsing horrible html. It has perfect documentation and a helpful community. You can even scrape from amazon, even including logging in. What can you do? Convert SVG to VML. Stock checker for ... -
Transactions for web developers - Aymeric Augustin
Initially he didn't know a lot about transactions, so he researched them in depth. A quote by Christophe Pettus: "transaction management tools are often made to seem like a black art". He moves from the database (postgres and sqlite) to the interface (psycopg2 and sqlite3) to the framework (django). Database A definition: an SQL transaction is a sequence of SQL statements that is atomic with respect to recovery. In SQL 92, a transaction begins with a transaction-initiating statement (almost everything can start a transaction) and it ends with a commit, an explicit rollback (ROLLBACK) or an implicit rollback. SQL 1999 changed this a bit. It has savepoints. After a savepoint, you can rollback to that savepoint, to a previous savepoint or you can set a new savepoint. Oh, and there is an explicit transaction start statement (START TRANSACTION). Key findings: Statements always run in transactions. Transactions are opened automatically. Transactions are advanced technology. Remember the dreaded "current transaction is aborted, commands ignored until end of transaction block" postgresql fault? What it actually means is "a previous statement failed, the application must perform a rollback". You cannot let postgres do any auto-recovery, that would break transactional integrity. It is your application ... -
Keynote - Daniel Greenfeld
Django conferences have a tradition: there's an external luminary that gets to give a critical talk on Django. His talk won't be that. He's not external either: he wrote two scoops of Django together with Audrey (see also my review). Being critical is sometimes easy. Just bash class based views, for instance. Bashing is easy. A rant like Zed Shaw's is fun, but he's not asked because of his rants, but because of his contributions (like books). Similarly, Django delivers working stuff and that working stuff makes a lot of our work possible. So here are some good points about Django: Django is everywhere. So many people and companies use it. Django is powered by Python. Pep8, python is beautiful. And there's the import this zen of python that we use all the time to steer others and ourselves in the right direction. Django's API wins. It is understandable. No weird names: templates, views, logging, sessions. Django projects also have understandable structures. If there's no views.py or models.py or templates/ directory, you know someone messed something up. Fat models are great. Just put your business logic all on your models. They do get big this way, however. You can make ... -
Does your stuff scale? - Steven Holmes
They grew from a two-person company to a 70-person one in two years. Central to that growth were Django and google app engine. Scalability means both load scalability and functional scalability. You also have to deal with organizational scalability and geographical scalability if you want to grow your organization. 1: Running Django on app engine It is easy to get confused. Is app engine real? Is it a joke? How to run your django stuff on it? Their reasons to use it: Auto-scaling. They build high-profile stuff and it needs to scale. They had a valentine day site that got a lot of attention on that day and it automatically scaled up without a change in the app. The day after it scaled down automatically, too. Services and APIs. No sysadmin needed. Some caveats with app engine: it is a sandbox. You you cannot do "pip install". The filesystem isn't there in the traditional sense; there is a blob storage instead. And it is lock-in, mostly; portability is an issue. They could work arounds these issues and ended up with a better application as a result. There are three ways (that they use) of running Django on app engine: Django ... -
The advantages of diversity - Steve Holden
Open source is great. It is absolutely amazing. We live in a multi-dimensional world, though it is often presented otherwise. Some present a simple line-based worldview. Bad-Good for instance. Where do you want to be on the line? Republican-democrat? Ruby-Python? Foreigner-native? Once you think along those lines (...) you tend to start thinking in opposites. This is the basis for many invalid world views. Just draw a line, cluster according to your preference and you're ready. Linear concepts are not useful. The issue is polariation. In a one-dimensional world, there is no room for complexity. What about a Venn-diagram based worldview? It allows for a bit more subtlety, but there's still a line on the outside... The open source world has a lot to teach the rest of the world. It is focused, mostly, on outcomes and results. But it is not representative. It is not even representative of the tech industry generally. In tech, 20% are women, in open source it is more like 2%, for instance. And... we need diversity! The biggest resource in open source is people. So you'd rather not exclude many people. The most common diversity areas, to give you an idea: Ethnicity Religion Gender ... -
Principle philosophy - Swift
Principle philosophy: a way to discuss our rules and beliefs that govern our actions. He tells it from his personal experience. His parents wanted to raise him as a good person. So they thought him good principles (like don't be a quitter, don't steal, etc). This is quite black/white though. We are all more gray/gray. What about the question "how can I be a good programmer"? Programmers use logic, which sounds black/white again: write tests, don't repeat yourself. Sigh. Talking about things like this is impossible without Immanuel Kant. He differentiates between reason and instinct. If "be happy" were our life goal, we'd just follow our instincts. So what is reason for, then, apart for doing good? Reason has to do with moral. There are three ways of looking at "doing good": Duty. Good things can come from duty. Duty can also lead to non-good things, though. Hm, so this is not it. Make a difference between the goal and the outcome. The outcome might be bad even though the goal could be worthy. Universal lawfullness. Only do something if you know that everybody thinks it is a good idea. Does this help with a question like "is testing good"? ... -
Get Django to play with old friends - Lynn Root
She works for Red Hat on http://freeipa.org, on identity stuff for Linux. Note: see her website for instructions and code examples. Say that your pointy haired boss (or customer) asks you to make an internal web app with all the buzzwords. So you can't use regular django auth, you'll need single sign on. Luckily since Django 1.5 you can have custom user models, so it'll fit with all your external requirements. One or two pieces of MIDDLEWARE_CLASSES and AUTHENTICATION_BACKENDS later and you play nice with the external single sign on. Django can be a team player. Webserver? You'll probably have to use apache. So the environment can be kerberos+apache. Add mod_auth_kerb for kerberos support. Add a "keytab" (making sure it is chown'ed to apache). There's a difference between authentication and authorization. Authentication is "just" logging in, authorization is what you're allowed to do. You'll have to connect to LDAP for that to ask which group(s) the user is a member of. Setting up your own kerberos environment (for testing) is a pain. Unless you use a ready made vagrant box for it. Instructions are on her website. -
Splitting up settings in Django
Splitting up settings in Django By default all Django settings are in one monolithic settings.py file. A single big file is hard to read and hard to maintain. Django users have found many ways to split up the settings into multiple files, all of them with their pros and cons. In our latest projects, we have developed yet another way, which uses file inclusion instead of importing Python scripts. The main features of django-split-settings are: Settings can be split into files and directories. Files later in the list can modify configurations, for example add or remove apps from INSTALLED_APPS. The main settings file lists the files that make up the project’s settings. Files can be marked optional. Optional files can be used to override settings per instance. Wildcards can be used in file paths. Maintains support for Django’s runserver auto-reloading. Read more... -
Taming multiple databases with Django - Marek Stępniowski
Marek works at SetJam: "We came to Django for the views, but stayed for the ORM". Django's ORM is pretty much in the sweet spot. SQLalchemy in comparison is less nice, having to learn a non-sql, non-pythonic language. At SetJam, they have what they call a backend and frontend. The backend collects data and stores it in the database, the frontend spits it out, mostly via feeds. They started out with one single big database, but that was hard to optimize. Many backend servers would write to the same database and the frontend server would read from it. Hard to optimize. Next they added a database slave for reading. That was before Django's multi-db support, so they had if/elses in their settings files based on environment variables. After Django's multi-db support, they could really support two databases and refer to them in the code with 'DEFAULT' and 'SLAVE'. Later on they splitted up the database even more. What goes where is handled by two custom database routers: a "MasterSlaveRouter" for the master/slave distinction and an "AppRouter" for shuffling some apps' data to certain databases. Tip: look at https://github.com/jbalogh/django-multidb-router, especially for the handy decorators (@use_master, for instance) it provides. At a ... -
Growing open source seeds - Kenneth Reitz
He shows us three kinds of (more or less) open source projects. Type 1: public source Once upon a time there was an "open source project" called the facebook SDK. Basically it just stopped working one day and nobody could help, despite offers for help on the issue tracker. Hacker news got wind of it and it was on the front page for a while. Facebook's reaction? Disabling the issue tracker... (Later on they fixed it). That's not open source, that's public source. Often it is abandoned due to loack of interest, change of focus or so. The motivation for having it as open source simply is not clear. Type 2: shared investment A different example: gittip. They aim to be the world's first open company. There's a github issue for everything, even the company name. Major decisions are voted for on github. The code is open source, of course. All interviews with journalists are filmed and live-streamed. And all otherwise-often-backdoor-cooperation-agreements are fully open. Projects like gittip are shared investment projects. Shared ownership, extreme transparency. There is very little questioning of motivations. The motivation is clear and public. There's a documented process for new contributers. The advantage? It is low ... -
Bleed for speed - Rob Spectre
He started with a little history lesson. The sea battle of mobile bay. The admiral (Faragut) ordered the ships straight through the minefield (called "torpedoes" at the time). "Damn the torpedoes, full speed ahead". And it worked. What does this to have to do with Django? Well, "damn the torpedoes, full speed ahead" feels a bit like how rapid prototyping feels afterwards. He's often involved with hackathons. Lot of quick coding in limited time with a lot of people. He learned a lot about his tools that way (and he often used Django). There's a time to make a distinction between production and prototype. Sometimes it is better to just try something with a prototype. Throw-away code. Aaargh! Throw-away code?!? We never throw code away. But it is something we must learn. It is good to let go once in a while. Let your code go. It isn't yourself, it is just some code. The danger is that prototype code is put into action as production code. With some work, this danger can be prevented. What about Django? Django is the best for prototyping. For rapid prototyping, Django is better than micro-frameworks like Flask that might seem better at first ... -
The web of stuff - Zack Voase
A plane flew over (noisily) at the start of his presentation. He put our work in perspective by saying that that was a 80 ton plane and that we're just building websites :-) Possibilities Computers used to take up whole rooms, now you have a smartphone. Big data is really big data now. Moore's lawworks both ways, though, so you have really small computers now. An arduino for instance. He often makes comparison to the human body. All over our body, sensors give off signals that go into the central nervous system. The brain processes it and gives signals back to muscles if necessary. Sensing, feedback, understanding, reaction. Stuff can talk to the cloud. Like a sensor in your body talks to your mind, stuff can treat the cloud as a brain. The cloud is what allows small tools to be smart. Stuff does often need a human to interact with it. Like a smartphone. There's all sorts of people thinking about how to "liberate the computers from their human overlords". Why cannot computers sense and act on their own account? So how do you bridge the gap betwen sensing and acting of stuff? How do you use Django for ... -
Advanced Python through Django: metaclasses - Peter Inglesby
Metaclasses are a handy feature of Python and Django makes good use of them. When you create certain kinds of classes in Django, a metaclass will do something to the class before it is created. For forms, the various attributes of the class are converted into a base_fields dictionary on the class. Similarly, a subclass of Model also fires up a metaclass that does some registering. A foreignkey to another model adds a relation back on that other model, for instance. As a recap, a class is something that can be instantiated into an object. It can have an __init__() method that does something upon instantiation. type(your_instance) will return the class. Did you know that you can create classes dynamically? See for yourself: >>> name = 'ExampleClass' >>> bases = (object,) >>> attrs = {'__init__': lambda self: print('Hello from __init__')} >>> ExampleClass = type(name, bases, attrs) >>> example = ExampleClass() Hello From __init__ >>> type(example) <class '__console__.ExampleClass'> So... we can actually control how classes are created! You could create a create_class() method that calls type but that modifies, for instance, the name. Or we could take all the attributes and add them to a base_fields dictionary on the instance. Hey, ... -
Fractal architectures - Laurens van Houtven
He worked twisted on. twisted And people tend to talk about subjects that are almost antithetical to how Django does things. The thing that he does different from Django is that he's not using a single data source... Once a database gets really really too big, putting multiple databaseservers next to eachother doesn't really work. You slowly start to get into expensive Oracle territory. How he set it up now is what he calls a fractal architecture. The whole accepts requests. The parts of the whole acccept requests. The parts of the parts accept requests. That's why he calls it fractal. You could also call it sharded, but that has a bad name: it is something you do when nothing else works. The way he looks at the architecture is SMTP. Email. Simple. He prefers SQLite. Simple and included in the python standard library. Sure, you can use postgres but you'll need a VM to re-create the same environment locally as on your production machine. SQLite is the same everywhere. In fact, he uses Axiom: an object store on top of SQLite. (Note: he is trying to write documentation for it at https://github.com/lvh/axiombook). Another advantage of sqlite: it is easy ... -
Mock your database - Marc Tamlyn
Your database is slow. Especially for testing. Nobody likes waiting, so SPEED is essential. Speed of iteration. Faster unit tests mean more tests. No one likes waiting. Once you create a couple of model objects in your tests, you're going to hit your database. He did a quick test with some 8 database queries. 5 seconds for 1000 runs on postgresql. 4 seconds for in-memory sqlite, so that's a bit better. Why not use factory_boy? That could help, right? More efficient instantiation, perhaps, all in one go (as the test was with a bunch of related objects)? 6 seconds on postgres, sigh. Higher. He tried a dirty dirty dirty crazy hack, using Django internals. (I looked to my right and the #1 Django database core committer was grinning from ear to ear...). No database queries. 1 second. Better: mocking. So he did some still-quite-hacky mocking with the mock library. 1.2 seconds. Ok, but the code is a bit yucky. Summary till now: avoiding the database is quicker. The difference is bigger than the difference between postgresql and in-memory sqlite. So... write your code in such a way to make avoiding the database easier. But... is there a better way? He ... -
Apps for advanced plans, pricings, billings and payments - Krzysztof Dorosz
He runs multiple sites with a common business model: accounts with plan subscriptions. So there's an obvious need for a generic account billing application. The app should not be too specific, as that limits your business flexibility. Also it should not be too generic: you'll end up with an architecture from hell that way. And there's the billing as such: you need to pay close attention to security and so. Hard problem! What he's making is django-plans for keeping track of the billing data, the plans, etc. And django-getpaid as payment processing app. django-getpaid Some challenges for the actual payment integration: He wants it to be generic and lightweight. He doesn't want to pull in half of pypi for a payment processing app. He wants a single API so that he can switch payment brokers if needed. He wants it to be asynchronous. Synchronous processing blocks too long. Multiple currency support. None of the existing apps were good enough, so he made django-getpaid. It is stable and supports a lot of (Polish) payment systems and is pluggable if you need to add another one. Pluggability is achieved with special backends you can enable in your Django settings. This way you ...
