Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Developing Reusable Django Apps: Signals
I wrote “signals provide a great way to propagate the events generated from your app” earlier. I think reusable apps should avoid hardcoding any kind of event handling and send signals instead. App consumers might prefer an email over an on-screen notification. They may even choose to ignore the event silently. A reusable app should [...] -
Django query set iterator – for really large, querysets
When you try to iterate over a query set with about 0.5 million items (a few hundred megs of db storage), the memory usage can become somewhat problematic. Adding .iterator to your query set helps somewhat, but still loads the entire query result into memory. Cronjobs at YouTellMe.nl where unfortunately starting to fail. My colleague [...] -
Faster, Export and the Shop!
This week saw a boost in speed, the ability to export your recipes and the addition of The Shop, along with the usual smattering of bugfixes. Performance We heard the pleas for quicker pageloads and made some performance enhancements to Forkinit. For the curious, we were previously running with next to no caching on a small server. We upped the size on the server and dropped a little bit of caching in several places on the site. We've already seen 50-75% drops in page load time, which was encouraging. Export Newly on everyone's dashboard is an "export" link. Clicking on that link will allow you to download a file of all of your recipes on Forkinit. Each person only has access to their own recipes. The export file is comma-delimited, which Excel can read just fine. It's important to us that your recipes be your own, and that includes not locking you in to Forkinit. Further, it's a great way to make sure that no matter what happens to us, you've got your recipes. The Shop We've also added The Shop to Forkinit. Currently, it's strictly product recommendations, things we own and find useful. They should be a little intelligent … -
Updates on djangoappengine
This post is a short update on new features we've added to our App Engine backend djangoappengine. So let's plunge in at the deep end. :) New Features First we added support for ListFields. You can use them in combination with any Django field type. Let's say you want to add a ListField for strings to one of your models: class Post(models.Model): words = ListField(models.CharField(max_length=500)) title = models.CharField(max_length=200) content = models.TextField(blank=True) It's as easy as that. Validation for ListFields is done on each entry in the list using the field type specified. The example above uses CharField's validation. ListFields now allows us to write many applications we couldn't write before. One example is a geolocation app. It should be possible to port geomodel or mutiny using native Django only now. We've put ListField into djangotoolbox.fields (see djangotoolbox repository). The next feature added is QuerySet.values() support though it's only efficient on primary keys. Let's see an example using the Post model from above: android_posts = Post.objects.filter(words='android').values('pk') This will get only the primary keys of all posts including the word "android" without fetching the entities from the database itself. It's possible to use QuerySet.values() on other fields than the primary key too … -
Django Up In Your CRON
For one off scripts for a particular project: #!/usr/bin/env python from django.core.management import setup_environ from myapp import settings setup_environ(settings) # do some stuff Related posts:Setup Python 2.5, mod_wsgi, and Django 1.0 on CentOS 5 (cPanel) Getting Started with Django and Python – First Impressions Adventures in Django and Python – Part III Related posts:Setup Python 2.5, mod_wsgi, and Django 1.0 on CentOS 5 (cPanel) Getting Started with Django and Python – First Impressions Adventures in Django and Python – Part III -
Is johnny-cache for you?
Is johnny-cache for you?. “Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus. -
'self' ForeignKeys always result in a JOIN
'self' ForeignKeys always result in a JOIN -
twod.wsgi Presentation at the Django User Group in London
Last week I had the pleasure to present twod.wsgi, a library to improve WSGI support in Django, at the Django User Group in London. The slides are now available in OpenDocument and PDF formats.That day I used a demo application which we are going to publish, possibly on Bitbucket. We should be releasing twod.wsgi very soon, as soon as the documentation is finished. So stay tuned! -
jmoiron.net: Johnny Cache
jmoiron.net: Johnny Cache. The blog entry announcing Johnny Cache (“a drop-in caching library/framework for Django that will cache all of your querysets forever in a consistent and safe manner”) to the world. -
Pycon 2010 report II
The first half day report for formal conference activities on February 21.Plenary: IntroductionI helped in registration so I arrived late to listen to Van Lindburgh start the conference. Which was a shame because I like to listen to him speak. Nevertheless, I consoled myself with the knowledge that I had contributed my time and service to help him launch what turned out to be an amazing conference.Keynote: Steve HoldenIn December Steve had presented to the NOVA-Django a much earlier draft of his speech. As much as his stuff back in December was good, what he did at Pycon was right on the money. He was in fine form, and the conclusion was very much Steve Holden at his best.The next night Steve was in rare fine form, but that is a story for another day...Keynote: Guido Van RossumBeing that I am Guido's biggest fan, and have trouble breathing in his presence, you might think I'm a bit biased. Alas, in this case, Guido's talk was not my favorite of the conference. If memory serves, at last year's Pycon he mentioned a desire to remove the "For Life" from BDFL and this year I think that showed a little bit. It … -
Django template2pdf
Django template2pdf -
Johnny Cache
Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. -
Django Management.tmbundle
Did some work on my [Django Management.tmbundle][1] last night. It now handles running tests when (a) The apps are not directly in the project root, but inside another folder, for instance; and (b) the app/tests.py file has been split into seperate files. The main reason I made this was so that I could run tests and have clickable links in the results window for the location of failing tests. There is still much to do on this. I am considering re-writing it in python rather than ruby, so I can programmatically find the app name, rather than guess it. I also want to refactor the hell out of it and make it much nicer. Anyway, if you are interested, you can find the most recent version at [http://github.com/schinckel/Django-Management.tmbundle][1] - and I think it also appears in TextMate's getBundles bundle. [1]: http://github.com/schinckel/Django-Management.tmbundle -
Dynamic form generation
I had the pleasure of being on a forms panel at PyCon 2010 chaired by Brandon Craig Rhodes. To get a stable baseline, Brandon asked each of us to provide code showing how each forms toolkit might tackle a problem: Imagine that someone has already written a form with your forms library. The form looks something like this: New username: __________ Password: __________ Repeat password: __________ [Submit] Now, someone from Marketing comes along and announces that the developers must add some additional questions to the form - and the number of extra questions is not determined until runtime! -
The darker side of Pinax
I really like the concept of Pinax, but I’m beginning to see the darker sides of the project. Here’s the fundamental issue: Pinax does a completely kick ass job of making a combined and base project that pulls together a … Continue reading → -
Django Advent: Scaling Django
Django Advent: Scaling Django. Mike Malone’s advice on scaling Django applications, including taking advantage of new features in 1.2. -
Open vs Closed Source
I have heard open source compared a lot to socialism, or maybe better as something that is not capitalism. Equally frustrating, I have been around the crowd that jeers the people that sell closed source software. There is a lot of misperceptions that exist about both commercial software and open source. I have been involved with both and have worked around people who have held views on opposite ends of the spectrum. There is room for both closed source, proprietary software as well as open source software. Bill Gates is Evil. Linux is for Communists. Software should be free. Patents stifle innovation. If you have worked in technology around other people at all, you have heard these types of statements before. Are we really supposed to believe these types of statements? Furthermore, what good does it accomplish? My Background I started my career in closed source and couldn't believe how or why people would "work for free" or give their code away. Didn't they know that not many people can write software -- it was valuable, they shouldn't just give it away. Later, I learned Python and then Django. I started using some open source libraries to build solutions in … -
Open vs Closed Source
I have heard open source compared a lot to socialism, or maybe better as something that is not capitalism. Equally frustrating, I have been around the crowd that jeers the people that sell closed source software. There is a lot of misperceptions that exist about both commercial software and open source. I have been involved with both and have worked around people who have held views on opposite ends of the spectrum. There is room for both closed source, proprietary software as well as open source software. Bill Gates is Evil. Linux is for Communists. Software should be free. Patents stifle innovation. If you have worked in technology around other people at all, you have heard these types of statements before. Are we really supposed to believe these types of statements? Furthermore, what good does it accomplish? My Background I started my career in closed source and couldn't believe how or why people would "work for free" or give their code away. Didn't they know that not many people can write software -- it was valuable, they shouldn't just give it away. Later, I learned Python and then Django. I started using some open source libraries to build solutions in … -
New Python Library: Bleach
We’ve released another Python library: Bleach, for sanitizing HTML and automatically linking URLs in text, that we’re using on addons.mozilla.org and support.mozilla.com. It’s based on html5lib, which we chose because of its foundation in web standards and its active development community. Bleach is available on Github and PyPI. Bleach uses a tree-walking approach to automatically [...] -
New Python Library: Bleach
We’ve released another Python library: Bleach, for sanitizing HTML and automatically linking URLs in text, that we’re using on addons.mozilla.org and support.mozilla.com. It’s based on html5lib, which we chose because of its foundation in web standards and its active development community. Bleach is available on Github and PyPI. Bleach uses a tree-walking approach to automatically [...] -
How to Store Arbitrary Data in a Django Model
I have a number of different places where I have wanted to store arbitrary data along with structured data in a Django model. This is data that I wouldn't necessarily care to use a value to retrieve data but when displaying or working with a record or records of this data, being able to have this data available in a manner that I didn't have to parse was nice. My solution was to serialize/deserialize in and out of JSON using simplejson and a Field class that derives from a TextField. i think it is easier to just read the code and the example of how to use it in the gist below than for me to continue with my rambles. The Code UPDATE: It wasn't clear in this post. This wasn't my code, as I had previously pointed out. Just wanted to be clear that I have found this solution useful and have used the snippet so much that I feel like it's mine -- but it's not. :) -
How to Store Arbitrary Data in a Django Model
I have a number of different places where I have wanted to store arbitrary data along with structured data in a Django model. This is data that I wouldn't necessarily care to use a value to retrieve data but when displaying or working with a record or records of this data, being able to have this data available in a manner that I didn't have to parse was nice. My solution was to serialize/deserialize in and out of JSON using simplejson and a Field class that derives from a TextField. i think it is easier to just read the code and the example of how to use it in the gist below than for me to continue with my rambles. The Code from django.db import models from django.utils import simplejson as json from django.conf import settings from datetime import datetime class JSONEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime): return obj.strftime('%Y-%m-%d %H:%M:%S') elif isinstance(obj, datetime.date): return obj.strftime('%Y-%m-%d') elif isinstance(obj, datetime.time): return obj.strftime('%H:%M:%S') return json.JSONEncoder.default(self, obj) class JSONField(models.TextField): def _dumps(self, data): return JSONEncoder().encode(data) def _loads(self, str): return json.loads(str, encoding=settings.DEFAULT_CHARSET) def db_type(self): return 'text' def pre_save(self, model_instance, add): value = getattr(model_instance, self.attname, None) return self._dumps(value) def contribute_to_class(self, cls, name): self.class_name = cls super(JSONField, … -
PyCon 2010 Recap
The videos from PyCon 2010 appear to be coming online now and I highly recommend checking them out if you weren't able to attend or missed some of the sessions you wanted to see while there at other sessions. There was a ton of great stuff to soak in through osmosis. In addition, it was great to meet others in the community that I know through IRC / Twitter / Email / Projects. Here is my recap of things that impressed me and that I saw as directly applicable to what I am working on. redis / pyres / celery redis, pyres, celery I need to use Redis with pyres or celery. Could replace a lot of my crontabs Could replace local queuing (versus distributed queuing running in the cloud). Could replace adhoc email queue infrastructure Simplifies stuff done asynchronously with crontabs - queue up the fact that something needs to be done at the moment instead of running queries repeatedly looking for conditions to trigger action. Becomes super simple to mark things as something to execute asynchronously so more functionality can be added without slowing down website (e.g. logging, near real-time data warehousing, etc.) picloud picloud Looks promising, however, … -
PyCon 2010 Recap
The videos from PyCon 2010 appear to be coming online now and I highly recommend checking them out if you weren't able to attend or missed some of the sessions you wanted to see while there at other sessions. There was a ton of great stuff to soak in through osmosis. In addition, it was great to meet others in the community that I know through IRC / Twitter / Email / Projects. Here is my recap of things that impressed me and that I saw as directly applicable to what I am working on. redis / pyres / celery redis, pyres, celery I need to use Redis with pyres or celery. Could replace a lot of my crontabs Could replace local queuing (versus distributed queuing running in the cloud). Could replace adhoc email queue infrastructure Simplifies stuff done asynchronously with crontabs - queue up the fact that something needs to be done at the moment instead of running queries repeatedly looking for conditions to trigger action. Becomes super simple to mark things as something to execute asynchronously so more functionality can be added without slowing down website (e.g. logging, near real-time data warehousing, etc.) picloud picloud Looks promising, however, … -
Pycon 2010 report I
Pycon was an incredible learning experience and networking opportunity. I met many good friends again and made just as many new ones. In addition, this was the first time I presented and did so on Pinax two times. Furthermore, in the name of diversity, this instance of Pycon saw the premiere of the Financial Assistance Grant for Women. We also had a dedicated talk on Diversity as a Dependency. The benefit this focus on diversity was that...um...yeah...like...uh...um...Diversity RocksDid I learn a lot at pycon? Heck yeah. Networking was life changing. And unlike previous conferences, I'm in a position to take advantages of opportunities offered. The next few weeks and months will see a lot of changes and challenges for me. Note: I've got to keep some things under wraps for now so I'm going to have to aggressively moderate comments. Feel free to comment, just don't take comment rejections personally.
