Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Djangocon: The whys and hows of using django formsets - Shena Priscilla Makini
(One of the summaries of a talk at the 2014 djangocon.eu). Many people don't use or know about django formsets. She google for articles and found "formsets: the worst thing ever" and "formset gotcha: or how to lose your sanity". Ouch. What are form sets for? When you want to use the same form multiple times. For instance for a data grid. A table with one form per row, for instance. Django formsets help you put several of those forms on one page. The example she showed was of changing your "mailman" settings for a couple of mailinglists on one page. It is used in postorius, a new web UI (django) for mailman. It works something like this: SomeFormset = formsets.formset_factory(SomeForm, extra=5) formset = SomeFormset() And in the template it is just: {% formset %} This gives it to you as a table. You could also iterate over it and get the individual forms: {% for form in formset %} ... {% endfor %} (Personal note: oh blast, I didn't know about this feature! I know just the spot where I desperately need to use it. That one app where I do some manual field generation and mess about with … -
Djangocon: open data on the sea shore - Richard Moch
(One of the summaries of a talk at the 2014 djangocon.eu). Richard Moch shows us open sea map, open street map for the sea! Light houses, shipping langes, etc. They're also trying to build up a sea depth map consisting of open data. See http://depth.openseamap.org/ . There's more: http://openweathermap.org/ . Yes, open weather data for current weather and forecasts. There is an open meteo foundation that tries to get it off the ground. Based on this open weather data, you can build apps, for instance http://meteo-parapente.com for showing good locations for doing paragliding. Open weather data? Then you need to measure it. They are working on an open source autonomous weather station. Modern French diesel unit in Kruth in 2007 -
Djangocon: pytest helps you write better django apps - Andreas Pelme
(One of the summaries of a talk at the 2014 djangocon.eu). Andreas Pelme's slides are at http://speakerdeck.com/pelme . Django already provides some nice tools to make testing easier. The testing experience can become even better, for instance by using pytest. Pytest is quite popular. The ones using it don't want to use anything else. (He showed a couple of tweets of well-known django developers to prove it). Pytest is exensible. Andreas is the maintainer of the pytest-django plugin. The main advantage of pytest is that it allows you to write pythonic tests, without boilerplate. No more self.assertEquals(a, b) but simply assert a == b. Advantage: the error message when it doesn't match is much clearer than the regular test exception. You don't need to subclass from TestCase anymore. Just a function is enough. Django's TestCase would normally give you self.client. The function will look something like def test_something(client): .... The django extension for pytest recognizes "client" and passes in the correct object automatically. That auto-recognized "client" attribute? Pytest calls that a "fixture". No, that's something else than django's fixtures. Test fixtures are very handy for test dependencies. You run the tests with py.test. For this, you need to have your … -
Djangocon: challenges when building high profile news sites - Yann Malet
(One of the summaries of a talk at the 2014 djangocon.eu). Yann Malet talks about editorial sites (=newspapers). And especially high profile ones, so ones that see a lot of visitors. Multi layer cache to protect your database Django is web scale. No problem. Django won't be in your way regarding performance. Which means: if you use something else, you'll run into the very same problems. A news item on one of their sites got mentioned a lot on twitter and other sites. Their servers even didn't break a sweat. Caching for the win! You should add caching. You don't need to re-calculate the very same page in the very same way for every new visitor. That takes way too much time. Luckily, adding caching for a news site is relatively easy. Varnish is the first layer of defence. A "web application accelerator", also known as "a caching HTTP reverse proxy". A page coming out of the varnish cache will be 10-1000 times faster than calculating it fresh inside django. Note: building something that caches reliably is very hard. Don't build it yourself, don't re-invent it. Just use varnish and configure it. Some varnish tips: Strip the cookies. Increasing the … -
Djangocon: good schema design and why it matters - Andrew Godwin
(One of the summaries of a talk at the 2014 djangocon.eu). Andrew Godwin is a django core developer and he's the author of South and its successor, the database migrations inside django itself. Schemas can be explicit or implicit. Postgresql, mysql and so have explicit schemas, they're regular . Redis, ZODB, couchDB: implicit, you basically "just" put data in (which in probably does have some kind of structure, of course). A problem with implicit schemas is that you can get silent failures: a weight might be "85kg" instead of 74, so a string instead of an int. Schemas inform storage, which might help with performance. And they enforce structure upon the data. The drawback? It enforces structure upon the data. It is less flexible. When you add columns to your table or if you add an index, postgresql often behaves much better than mysql. It is faster and does much less locking. In case you use oracle or mssql, learn their strengths. Well. Schemas. They will need to change. You can put your schema into a vcs, but there's no guarantee that your data will survive the modification. Django has migrations build-in now. But migrations aren't enough: you can't automate … -
Djangocon: introduction to docker - Amjith Ramanujam
(One of the summaries of a talk at the 2014 djangocon.eu). Amjith Ramanujam Why docker? Your site consists of a full stack. Django, database, apache, celery, etc. If you're a full stack deveoper, you need to be able to look at all of them. And ideally you'd have exactly the same environment locally as on the server. Docker calls itself "an easy, lightweight virtualized environment for portable applications". A portable, self-sufficient container. It is a sandboxing technique that is a bit similar to BSD chroots. It uses Linux containers, AUFS, Git-like versioning and a REST API. What is the difference with a normal virtual machine? A VM needs: Host OS Hypervisor Guest OS Libs The app Docker needs: Host Libs. Your app. The libs and the app are part of a docker image. It is faster and much more lightweight. It reuses the host OS (but it is completely isolated!). On OSX: use boot2docker plus an OSX docker client. It uses a virtualbox with linux, the docker client still runs on OSX, but talks to docker inside the virtualbox. Some terminology: an image is a base that contains libs, binaries, environment variables, files and so. When you instantiate ("run") it, … -
Djangocon: Making angularjs play nice with django - Jacob Rief
(One of the summaries of a talk at 2014 djangocon.eu) Remark by Jacob beforehand: there's a bit of trend to put a lot of the UI in javascript. Fine for games and single page apps, but don't do it for ecommerce or CMS sites that you want to have indexed by google. Jacob Rief loves django. The three best parts according to him: Single source of truth. Model view control. Separation of concern. How about doing MVC on the client? A problem is that the browser UI is event-driven: you cannot do a simple one-way data binding from a model + a template to the html code. What you need is a two-way data binding. The model and the view are contiously updated both ways. The model is the single source of thruth. Changes to the view update the model; changes to the model update the view. This way of working is used by many javascript libraries. Angularjs is one of them. The view itself is compiled from a template inside the html code. You mark part of the html structure with angular directives (<div ng-something="bla">) and angular then turns that into the proper view (html DOM) that's bound in … -
Djangocon keynote: devops, it is much more than just automation - Kevin van Wilder
(One of the summaries of a talk at the 2014 djangocon.eu.) Kevin van Wilder talks about devops. (Note: I changed the title from "your product is more than the application" to "devops, it is much more than just automation" as that covers the main message much better). Nothing good happens after 14:00 on friday. So don't do a friday deploy. He remembered a quick fix they needed to deploy one friday evening. It actually went OK and it even worked during the press conference demo. 10 days later the site died, though. Who's problem is it? "It worked in development". So, "it is an operations problem now?". Is devops the solution? You cannot really be a "devops person". It is more a movement, a philosophy: lets work together as dev and ops. Partially it is just a name for something that people have been doing for a long time. A useful acronym is CLAMS: Culture. How can we influence co-workers? You cannot directly change culture. But you can change behavior. And behaviour becomes culture. If you do it right, you'll start influencing others to do right, too. We've been influenced ourselves at this conference, right? Haven't we been shown The … -
Djangocon: from icontains to search - Honza Kral
(One of the summaries of a talk at the 2014 djangocon.eu.) Honza Kral talks about searching, which is a big topic. He'll focus on unstructured search: fulltext search. Search is an interface to your data. Textual search: $ grep -ri django * or: >>> Documents.objects.filter( ... Q(title__icontains='django') | ... Q(body__icontains='django')) Don't do this. None of them are going to scale. And you're going to miss things. If you search for "running" you'll miss "runs", for instance. The first index in world history was the 1230 bible concordance. Just a simple list of biblical words with pointers to the passages in which the words occurs. We can do the same. Technically it is an inverted index. We can do more than just looking up individual words. We can also look at phrases. In those cases you look both at the files the words occur in, but also at their place in those documents. We can do more things when storing words in our index: Leave out common words like 'a' and 'the'. Lowercase everything. Normalize words (normalize various verb variants to just one, normalize single/plural). Look for synonyms. (Fast is almost the same as rapid, for instance). All this happens at … -
Djangocon: introduction to user experience design - Meghan Reilly
(One of the summaries of a talk at the 2014 djangocon.eu.) Meghan Reilly is a user experience strategist. In the USA, they're slowly starting to cycle a bit more. And they're still learning how to share the road as cyclists and car drivers. When she's cycling, everybody else seems to be a bad car driver: they're all over the place. When she drives a car, the cyclists seem to be novices that are all over the road. (Personal comment: if you want to have more cyclists on your road, the cycling infrastructure needs to be good. Look for instance at Dutch junction design) She asked "who has been frustrated by bad websites". And afterwards "who ever implemented something in a website in the simplest way possible, thinking 'the user will figure it out or read the documentation'"... What is missing is empathy. Empathy is the ability to understand and share the feelings of the other. In user experience design we're putting the customer at the center. Not ourselves. For many, user experience design will be a totally new territory (just like she's now in new territory, never having visited Europe before). What is user experience? The experience you have, good … -
Djangocon: wednesday lightning talks
(One of the summaries of a talk at the 2014 djangocon.eu.) Lazy devops' guide to SSO with Kerberos - Aymeric Augustin The goal is to provide single sign on (SSO) to an internal application within your company network. You can use ksktutil to create a keytab. You then need to configure apache to look at that keytab. And you need to tell it to require a valid user. Django has support for such an externally set user that's send as a REMOTE_USER in the header. See https://docs.djangoproject.com/en/1.7/howto/auth-remote-user/ You next need to check if your browser works out-of-the-box or whether you need to do some more configuration There's more data in his notes: https://gist.github.com/aaugustin/10715655 A pycon in an African nation - Daniele Procida He hopes to get a pycon off the ground in sub-saharan Africa (not South Africa, as there's already a conference there). The goal: sustainability. They have to be able to build up skills locally. Skills shouldn't have to be imported. The only way forward is to use open source products. Through his university, he has some contacts. Nothing is fixed. There is much work to be done. And you cannot rush it: if you blow it, you could … -
Djangocon: gamers do REST - Angel Ramboi
(One of the summaries of a talk at the 2014 djangocon.eu.) Angel Ramboi works for a company, demonware that provides services for games, like leaderboards and so. They mostly use REST for communication. Interoperability and scalability are great. The whole web is one big REST API in a way, actually, so using REST is almost a guarantee that you can scale your service and that you can get it robust. Their stack: django 1.6, python 2.7, mysql 5.6 (sharded), CentOS and apache+mod_wsgi. So nothing really fancy and exotic. They take reliability very seriously. REST API design? Read Roy Fielding's thesis. Afterwards you'll be surprised at the elegance of it all. GET/POST/PUT/DELETE verbs for a CRUD API. HTTP for the communication protocol. JSON is handy for representation (they even use some light-weight schema validation). They use a pragmatic approach: "good enough" is better than "perfect". For development, they use github. Development is on branches. When everything is ready, they package it up as an RPM and deploy it. They do app configuration with YAML. It works better for them than python settings files as you can also use it for other applications. And in their case they can now also run … -
Djangocon: taming complexity with django - Ustun Ozgur
(One of the summaries of a talk at the 2014 djangocon.eu.) Ustun Ozgur tells about someone's talk about simplicity, simpleness and complexity. What does "complexity" tell us as django developers? He quotes Richard Gabriel who talks about "habitability" as a characteristic of source code that enables developers that see a piece of code to understand it and to work on it, even if they haven't seen the code before. This is so for django. Complexity should be fought head-on. Look at domain driven design. Keep your architecture close to the actual domain. That helps. Django has a couple of layers of defense against complexity. URLs. Django routes various parts of the url to various applications. The division into seperate applications helps a lot to fight complexity. Models and managers. Django uses several well-known design patters that help structure your code in the right way. But... watch out for "fat" classes that do too much. You could look at mixins and compositions. Or you can move some of the fat class's methods to separate functions. Sometimes no code is the best code. You could put state machine state in a variable (a dict or so) and use that. Data is easier … -
Djangocon: really, really fast django - Christophe Pettus
(One of the summaries of a talk at the 2014 djangocon.eu.) Christophe Pettus works for http://pgexperts.com . So... postgresql expert! I've got two other talks of him postgresql when it is not your job and advanced postgresql in django. How fast django anyway? You hear things like "the orm is slow" or "the template engine is old". So... when in doubt, measure. How high-overhead are django's components? You'll need to have a good test. Just returning an empty response is nonsense. He did a bunch of tests and django holds up pretty well. The overhead is very low. Most ORM operations are O(N) on number of fields. So on average, a 1-field model will be 10 times faster than one with 10 fields. Updating an update? Use the ORM update method instead of looping over items. Much faster. Using manual SQL can be a little bit faster than the ORM, but it is neglegible. Django's basic request loop is plenty fast. Request/response cycles to the database generally swamp everything else. Always do bulk and batch operations instead of iterating. Don't use components you don't need. Often you load a lot of libraries. Do you really need a full REST library … -
Djangocon: typography and the web, the state of the art - Idan Gazit
(One of the summaries of a talk at the 2014 djangocon.eu.) Idan Gazit is someone I really like to hear at django conferences. I'll simply point at two earlier talks. Design for developers (2010) and compass and less (2011). There's a nice book "thinking fast and slow". A general principle is that anything you can to to reduce cognitive strain is worthwhile. One of them is making your text readable. This is mentioned explicitly. The printing press was invented 600 years ago and we did a lot of writing beforehand, so we have a lot of experience. There are two kinds of factors: Micro. Typeface, kerning. Macro: measure ("width"), leading ("spacing"), flow of type on a page. There's a lot of common wisdom for books. How to lay it out, the ideal size of margins. But on the web we have a huge problem: we do not have control. There's no fixed page size, we have many screen sizes and screen resolutions and screen quality. But still... there's a lot that we can learn from the existing body of knowledge. Type size. 16 pixels is the default size in browsers. In books it is often smaller. But it is 16 … -
Djangocon: django minus django - Jacob Burch and Jacob Kaplan-Moss
(One of the summaries of a talk at the 2014 djangocon.eu.) Jacob Burch and Jacob Kaplan-Moss say Django is awesome. It comes with basically everything you need to get your site started. But it doesn't come with everything. And sometimes you might want to rip something out and replace it with something else. You could rip out the template engine and put jinja2 in. Or remove the views and only use REST and lots and lots of javascript. And you add redis or so to make communication between the frontend and backend. And slowly it isn't so recognizable as a Django project anymore. Django's batteries-included nature is great, but sometimes those batteries hem us in. So they're going to talk about ripping things out of django. You can see what goes well and in which places django fights back. Templates Something that is often replaced is django's template language. You might want something more powerful or you might want a completely different template language. Jinja2 is a common candidate. More expressive and it is often quicker. You do need to pass a regular python dictionary as context to jinja, though, instead of a django context. You can use https://github.com/jbalogh/jingo to … -
Djangocon: the power of natural language - Ilja Bauer
(One of the summaries of a talk at the 2014 djangocon.eu.) Ilja Bauer's talk has as subtitle "from ATDD to lean modeling". ATDD means Acceptance Test Driven Development. Constantine the African was an ancient physician and one of the first ones to use test driven development: he tested people's morning urine manually and could check whether they had diabetes or not. This is however a very manual process. In our test driven development, we can luckily automate a lot. Software development is a communication problem. He showed a picture of the biblical Tower of Babel where people couldn't understand each other anymore because they suddenly spoke different languages. It sometimes seems the same with the way our customers' requirements somehow doesn't quite end up in the software in the right way. Do we really understand each other? A solution can be acceptance test driven development: you write the acceptance tests together with the programmer. The client has a request. You write down the requirements as textual scenarios. You need to somehow convert the text into an automated acceptance test. He currently uses natspec, a plugin for Eclipse that is free for open source projects. You write the software and continuously … -
High Performance Django Kickstarter Project
I'm really excited to (finally) announce that we are writing a book! We've been working with Django professionally for a long time (over 7 years now). During that time, we've learned a lot about how to use the framework to build fast and scalable websites. We're bundling up all that knowledge into an e-book called High Performance Django and it is up on Kickstarter today. We're already well into the writing process, so we can tell you a little about what you'll get. The book is heavy on Django tips, but also reaches far beyond Python, showing you how to architect and tune the rest of your stack. It gives you an opinionated battle-tested blueprint utilizing many of the same techniques as high-profile Django sites like Disqus, Instagram, and Pintrest. We hope to get the first edition out in July, but we need your help to make that happen. Writing and editing the book is going to be a massive undertaking and you can help support our effort. Give us your vote of confidence by backing the project today! -
High Performance Django Kickstarter Project
I'm really excited to (finally) announce that we are writing a book! We've been working with Django professionally for a long time (over 7 years now). During that time, we've learned a lot about how to use the framework to build fast and scalable websites. We're bundling up all that knowledge into an e-book called High Performance Django and it is up on Kickstarter today. We're already well into the writing process, so we can tell you a little about what you'll get. The book is heavy on Django tips, but also reaches far beyond Python, showing you how to architect and tune the rest of your stack. It gives you an opinionated battle-tested blueprint utilizing many of the same techniques as high-profile Django sites like Disqus, Instagram, and Pintrest. We hope to get the first edition out in July, but we need your help to make that happen. Writing and editing the book is going to be a massive undertaking and you can help support our effort. Give us your vote of confidence by backing the project today! -
A break from the tutorial: Intuitive stuff
Hey guys! Let's take a break from the bombarding of information, and let's do something interesting. Let's talk about some of the features that are taken for granted in all the giant websites. You may already have noticed them. If not, double back and check them out. We'll also see how to implement them in your website.The "Keep me logged in":Many websites like Facebook and Gmail offer this option at the time of login. What it does is even if you close the browser/tab/window, the next time you open it it opens on your homepage. Even more fascinating, if you try to log in from some other device it you're not logged in, and you even get an alert that someone tried to access your account from an unknown location. Let's see how this is done in steps:When you log in, cookies with your username are stored in your browser. If you close it and reopen it, and access that site again, it checks for that-site-specific cookies. If they are there, they take you to your homepage. Else, you're redirected to the login page.To prevent multiple people accessing the account at the same type, the session is logged in to … -
Djangocon: two short talks about django-oscar and about performance gains
(One of the summaries of a talk at 2014 djangocon.eu) An introduction to django-oscar - David Winterbottom David Winterbottom is, which I just now noticed, the originator of the very useful www.commandlinefu.com website that I've had in in my RSS reader for the past few years. Recommended: you get a useful tip out of it every month or so. More related to django: he's the author of the e-commerce framework 'django-oscar'. In e-commerce, especially B2B (business to business), you get loads and loads of exceptions. He originally worked on a PHP system that basically broke down under all those exceptions to the standard rules. He now works on django-oscar, which is used by a number of very large customers. Ecommerce: products, baskets, orders. Overridable apps. It could be added to your site without modifying anything: it won't take over your whole site. It doesn't use the django admin though, but a custom edit site. Philosophically, oscar isn't an out-of-the box product. It is a generic product, so it cannot know how your tax system works. Which payment provider you use, How you've handled shipping. So you'll have to add that yourself. Oscar provides a base platform you can build upon. … -
Djangocon keynote: where the wild things are - Aymeric Augustin
(One of the summaries of a talk at 2014 djangocon.eu) Aymeric Augustin talks about app loading. App loading is a step in django's initialization sequence: import all models and populate a cache. It is also a project that improves this step. The trac ticket for this feature is seven years old... Many many people have worked on it and provided patches and branches. The final implementation looks like this: from django.apps import AppConfig class MyConfig(AppConfig): name = ... label = ... verbose_name = ... # and in the settings file: INSTALLED_APPS = ( 'some_app', # Old style. 'yourapp.apps.MyConfig', # Points to the config class. ... All nice, but in practice there were crashes when using the implementation in production with gunicorn. The root cause was that django doesn't have a proper signal that says that it is fully configured and ready. So he started digging into the code. And found out that actually reading all the models and setting everything up occurs at various points in time, depending on how you call it. "runserver" is different from "shell" which is different from "running it from wsgi". At the core is the AppCache that django uses behind the scenes. This looks … -
Djangocon: tuesday lightning talks
(One of the summaries of a talk at 2014 djangocon.eu) Migrate an existing web application to django - Samuel Goldszmidt What to do when you have a PHP/mysql app and want to turn it into a django site? Set up a django site with the latest version (1.7) with migrations included. Add your database settings. Create an app and use manage.py inspectdb > yourapp/models.py to inspect your database and generate models. Then create the default core django tables (just run manage.py migrate). Start the initial migration manage.py makemigrations yourapp. Start renaming tables and fixing things up and turn those steps into migrations. At the end you have a nice clean set of models including automatic migrations. You could use the "squashmigrations" command to get a new clean baseline migration after you're ready. Harness the speed of the wheel - Xavier Fernandez Wheel is python's new (binary) package format. It is a zip format archive, you can basically extract it and it is ready. It has a specially formatted file name that includes information for the target architecture (as it can contained compiled code). Installing from wheels is 4 or 5 times faster for Django. For something that is heavy in … -
Djangocon: the future of postgresql in django - Marc Tamlyn
(One of the summaries of a talk at 2014 djangocon.eu) Marc Tamlyn is well-known for his succesful kickstarter campaign to improve postgresql support in django. He asked for a show of hands: some 90% of the attendees use postgresql. It is the most popular database for django projects. Most of the core team favours it. It has a wide feature set and it is a proper active (non-oracle-owned) open source project. Wide feature set... but not all of the features are supported by django. You can use several add-ons, but they're not that nicely integrated. So he proposed a kickstarter project for creating django.contrib.postgres. 14k (in UK pounds) was raised by the kickstarter campaign! Wow. He thanked everyone that contributed (a couple of big contributions by companies by name). The core of the project is to add support for a couple of data types, like array, json and hstore. Array can list just about any type. Nested structure is OK. No foreign keys, though. This one is almost ready for django 1.8. HStore is a key-value store. The most requested feature! It is quite similar to django-hstore, which is already a very good add-on. They'll help him with getting HStore … -
Djangocon: web components in django - Xavier Dutreilh
(One of the summaries of a talk at 2014 djangocon.eu) Xavier Dutreilh talks about web development. You probably use libraries like jquery and backbone. And css frameworks like bootrap. And tools like bower, grunt, yeoman and jshint. One of the things you should care about is performance. Especially for mobile devices, you don't want to send over too much css/js. You don't want too exotic and hard technologies, as you as a web developer probably will lack the specialized knowledge. Accessibility is important. Cross-browser functionality, too. But... we often disregard the wishes we enumerated above and simply look for ready-to-use framework and pile them on top of each other. And we hope everything works as we hope and expect. Which it doesn't and we spend lots of time debugging. We inherit technical dept by piling everything up like this. What can we do? Well, we first have to stop making such a mess in the first place and have to start looking at the requirements we figured out. The best tip is to think about individual elements. We could also try to do it lean and mean: custom code. Start by looking at a web page. What does it consist …
