Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Custom queries I: Comparison between SQLite and MySQL
I've already said in earlier posts that you can use SQLite or MySQL as a database for your website. Recently when I was working on one I had to move to MySQL (I'll tell you why later in this article). So I thought I'll tell you guys how to transition.But first, let's compare.SQLite:It is a lightweight DBMSIt is mainly used for testing and for small websites that do not generate too much data.The biggest advantage is that it handles all the data through a .db file which can be transported elsewhere and used in the same wayIt supports up to 2^64 rows (wow!)On the downside, for the kind of data that giants like Facebook and Google generate, SQLite falls laughably shortIt also has low concurrency control, which means that it crashes if too many people are trying to access the same dataThe last point, the concurrency control, was the main reason I decided to switch to MySQL. The database of the website was crashing again and again. Let's look at MySQL now.MySQL:It is tough, inflexible and an extremely powerful DBMSIt is used by Facebook, Google, Twitter, pretty much all the web giants that exist todayIt has very high concurrency controlIt … -
Migrating Django projects with fixtures
When migrating a Django project it's often useful to work with the built-in dumpdata and loaddata management commands. However, this can easily lead to errors like django.db.utils.IntegrityError: Problem installing fixture Could not load foo.Bar(pk=ba): (1062, "Duplicate entry for key 'app_label'"). Such errors indictate that a similar object already exists in the database, and that you can't insert the duplicate from your fixture. There are basically two approaches to dump data in a way that solves this problem: Specify exactly which models to dump Exclude the conflicting models from the dump For the first approach you would use a command like: ./manage.py dumpdata --natural --indent=2 app1 app2 app3 auth.User > dump.json I include the auth data as well in this example, because it's what I usually do. You want to include all third party apps that create their own database tables and store user generated content. You want to avoid apps that auto-populate the database, like thumbnail generators, search indexers, etc. The second approach is to exclude models. You can do this like: ./manage.py dumpdata --natural --indent=2 --exclude=contenttypes --exclude=sessions.Session --exclude=south.Migrationhistory > dump.json If you googled the error django.db.utils.IntegrityError: Problem installing fixture Could not load contenttypes.ContentType(pk=X): (1062, "Duplicate entry for key 'app_label'"). … -
Migrating Django projects with fixtures
When migrating a Django project it's often useful to work with the built-in dumpdata and loaddata management commands. However, this can easily lead to errors like django.db.utils.IntegrityError: Problem installing fixture Could not load foo.Bar(pk=ba): (1062, "Duplicate entry for key 'app_label'"). Such errors indictate that a similar object already exists in the database, and that you can't insert the duplicate from your fixture. There are basically two approaches to dump data in a way that solves this problem: Specify exactly which models to dump Exclude the conflicting models from the dump For the first approach you would use a command like: ./manage.py dumpdata --natural --indent=2 app1 app2 app3 auth.User > dump.json I include the auth data as well in this example, because it's what I usually do. You want to include all third party apps that create their own database tables and store user generated content. You want to avoid apps that auto-populate the database, like thumbnail generators, search indexers, etc. The second approach is to exclude models. You can do this like: ./manage.py dumpdata --natural --indent=2 --exclude=contenttypes --exclude=sessions.Session --exclude=south.Migrationhistory > dump.json If you googled the error django.db.utils.IntegrityError: Problem installing fixture Could not load contenttypes.ContentType(pk=X): (1062, "Duplicate entry for key 'app_label'"). … -
Migrating Django projects with fixtures
When migrating a Django project it's often useful to work with the built-in dumpdata and loaddata management commands. However, this can easily lead to errors like django.db.utils.IntegrityError: Problem installing fixture Could not load foo.Bar(pk=ba): (1062, "Duplicate entry for key 'app_label'"). Such errors indictate that a similar object already exists in the database, and that you can't insert the duplicate from your fixture. There are basically two approaches to dump data in a way that solves this problem: Specify exactly which models to dump Exclude the conflicting models from the dump For the first approach you would use a command like: ./manage.py dumpdata --natural --indent=2 app1 app2 app3 auth.User > dump.json I include the auth data as well in this example, because it's what I usually do. You want to include all third party apps that create their own database tables and store user generated content. You want to avoid apps that auto-populate the database, like thumbnail generators, search indexers, etc. The second approach is to exclude models. You can do this like: ./manage.py dumpdata --natural --indent=2 --exclude=contenttypes --exclude=sessions.Session --exclude=south.Migrationhistory > dump.json If you googled the error django.db.utils.IntegrityError: Problem installing fixture Could not load contenttypes.ContentType(pk=X): (1062, "Duplicate entry for key 'app_label'"). … -
Sending emails
If you've noticed, on many sites, once you sign up, it sends you an automated confirmation email. This is another function your site should be able to perform. So today let's take a look at that.Let's assume you have a Gmail account (if you don't, create one today, it's great!). We'll be using Gmail's email host (wow that rhymed).Go to the settings.py file and add the following code anywhere in between.EMAIL_USE_TLS = TrueEMAIL_HOST = "smtp.gmail.com"EMAIL_HOST_USER = "your_name@example.com"EMAIL_HOST_PASSWORD = "your_password"EMAIL_PORT = 587The port number 587 is pretty standard. Also, it is very important to set TLS to True. We'll go into why later.You have to enter your Gmail password there, but I assure you it is secure. Only the programmer(s) can see it. You may also create another common account and use those credentials instead.Ok, now we'll need to write the Python code to send emails.Open the views.py file in the appropriate app and write the following code:At the top, import the Django's send_mail:from django.core.mail import send_mailAlso import settings.py:from django.conf import settingsLater, create this function:def foo (request): title = "some title" message = "some message" host = settings.EMAIL_HOST_USER receiver = ["someone@example.com"] send_mail (title, message, host, receiver, fail_silently = … -
Github
Hey guys! For those of you who don't know, Github is a code sharing site on which you can review and edit other people's code and post your own. I have shown you a lot of code on this blog. Some of the code like file upload, login sessions etc is the same and can be reused. So I'm going to post such pieces of code on Github for you guys to review and edit. Please do so, I would be very happy if someone improved upon the code I've written.Also, my favorite language is Python and I do a lot of other things like software development, hacking in it. I've posted repos for these also. Please feel free to review and edit those too.Here's the link:https://github.com/AgentK1729Cheers! -
Intermittent renewal demonstrated through github graph
I've got a number of books at home in the "personal development" genre. One of the more practical and better ones is The Power of Full Engagement: Managing Energy, Not Time, Is the Key to High Performance and Personal Renewal. Core idea 1 A core idea in this book is that you've got four kinds of energy that you need to watch: Physical, health. Emotional, happiness. Mental, focus. Spiritual, purpose. As an exercise, rate yourself on a scale of 1 (bad) to 10 (good) and multiply the numbers. Should be a pretty good measurement of how well you're doing! The wake-up call is that you shouldn't disregard any of the four. If you're physically drained because of 4 hours of sleep and a sedentary lifestyle, you can be focused all you want, but you won't get too much out of yourself. And when emotionally drained because of some family mishap or huge stress, sleeping 8 hours a night won't help all that much. So... try to get all of them to a reasonable level. Core idea 2 The second core idea is that each of these four is trainable. Just like you can train your body, you can train the … -
Making a specific Django app faster
This is just a simple tale of making a specific Django app faster. Just a tale of the steps I took to give you an idea of what you can do to make your app faster. This isn't the definitive guide, it just aims at giving ideas and giving a feel for the process. tl;dr summary: First measure, then optimize. Django's caching helps. There's a nice algorithm to speed up javascript graphs. Background The app is lizard-fewsjdbc. It is on github, though we're basically the only ones using it. The basic setup is that we've got an external special-kind-of-database that talks xml-rpc to the outside world. Over the xml-rpc we can send SQL queries to the database that in turn uses a jdbc connection to execute the query. Don't ask. So... we talk to some xml-rpc and have to return information, mostly as graphs. The big performance problem is that the data is time-based. A measurement every minute. And the customers wanted to see a year of data in a graph. The graph is a javascript flot chart because of it provides nice interaction like zooming in and out. And... javascript, or rather Flot, or rather the browser, doesn't take … -
Writing a README
Good documentation is important. I just heard two colleagues talking and what one of them said sounded like a README: - Install the shit. - Run it. - Look for error messages. It triggered me to write a blog post as I was just busy writing a README for a small script. pip install is currently compiling numpy and matplotlib, so time enough for me to gather my thoughts. "Good documentation is important", yes. But there are many things that we can all agree on "yeah, that's important, too". "Being important" isn't the same as "actually getting written". Many of the small tools and django apps that we write aren't actually going to see widespread use. Many are company-internal, effectively. Is read-the-docs level documentation useful/necessary? In some cases, yes. One of the company-internal django apps I wrote has extensive documentation and 100% test coverage. That's what you get with something named "lizard-security" which uses django's row-level security and some special settings and other tricks. Especially when it is used in almost all our sites. In most cases, nobody is going to bother looking up the official documentation, even if you wrote it. All they'll glance at is the README. And … -
Profiling Django apps and Python scripts with cProfile and Kcachegrind
Some time ago I wrote about profiling Django with hotshot with mod_python that is now dead. To profile execution time of Python scripts as well as Django views, tests, or management commands we use cProfile and its wrappers. There are also converters that provide data for KCachegrind. Let us see what we have available to use. -
Uploading files on Google Cloud Storage with Django
Intro On of the features of Django Appengine Toolkit is simplifying the configurations neede to make Google Cloud Storage a static files storage for Django applications running on Google App Engine. Infact all you have to do to use GCS as the destination of your uploads is writing something like this in your settings.py module: APPENGINE_TOOLKIT = { 'APP_YAML': os.path.join(BASE_DIR, 'app.yaml'), 'BUCKET_NAME': 'media-uploads', } DEFAULT_FILE_STORAGE = 'appengine_toolkit.storage.GoogleCloudStorage' STATICFILE_STORAGE = 'appengine_toolkit.storage.GoogleCloudStorage' A complete example This repo contains a minimalistic Django project implementing a file storage application that permits users to upload files, listing and delete them. The project has just one app implementing all the logic, defining the model and exposing the views. For detailed instructions on how to setup a Django project on App Engine with django-appengine-toolkit please check out this blog post before playing with this project. Now let’s take a look at the code. The Model class Document(models.Model): docfile = models.FileField(upload_to='documents/%Y/%m/%d') def delete(self, *args, **kwargs): storage, path = self.docfile.storage, self.docfile.path super(Document, self).delete(*args, **kwargs) storage.delete(path) Pretty easy, we have just one field containing the file. Notice the delete method we’re going to use so that once deleted an instance, the same will happen to corresponding file on Cloud Storage. … -
Django Extensions 1.3.4
We are happy to release: Django-Extensions Version 1.3.4 This brings the usual tons of fixes and improvements Get it at: https://pypi.python.org/pypi/django-extensions/1.3.4 Changelog: Feature: Start maintaining a CHANGELOG file in the repository Feature: ActivatorModelManager now has an ActivatorQuerySet Feature: Add a deconstruct() method for future Django 1.7 migration compatibility Feature: show_urls, now support --language for i18n_patterns Feature: show_urls, now shows the decoraters set on a view function Feature: graph_models, now support --include-models to restrict the graph to specified models Feature: print_settings, allow to specify the settings you want to see Improvement: graph_models, use '//' instead of '#' as comment character in dot files Improvement: graph_models, added error message for abstract models without explicit relations Improvement: JSONField, use python's buildin json support by default with fallback on django.utils.simplejson Improvement: PostgreSQLUUIDField, parse value into UUID type before sending it to the database Improvement: Use django.JQuery in autocomplete.js if available Improvement: use "a not in b" instead of "not a in b" in the entire codebase Removed: clean_pyc command since it does not work correctly in many cases Removed: sync_media_s3 command in favor of sync_s3 BugFix: syncdata, use pk instead of id for identifying primary key of objects BugFix: sync_s3, use safer content type … -
Continuous integration with travis and coveralls.io for Django apps
Maintaining a solid rapport with the ongoing software development process always turns out to be a walk on air. Ensuring a software build integrity and quality in every single commit makes it much more exciting. If the current software bulid is constantly available for testing, demo or release isn't it a developer's paradise on earth? Giving a cold shoulder to "Integration hell" the 'Continuous integration' process stands out to deliver all the above assets. What's Continuous integration? Continuous integration is a development practice that requires a developer to integrate code into a shared repository in which isolated changes are immediately tested and reported on when they are added to a larger code base. Continuous integration (CI) originated from within the extreme programming paradigm, but the principles can be applied to any iterative programming model, such as agile programming. As critics noted several potential drawbacks on extreme programming many organizations have adopted CI without all of the extreme programming concepts. Principles of CI: Maintain a single source repository Automate the build Make the build self-testing Keep the build fast Everyone can see the results of the latest build Automate deployment Keywords: Build: All steps necessary to compile and create deliverables. Commit: … -
Absolute last chance for tickets!
We have been able to make a few more tickets available for DjangoCon Europe 2014. This last batch of sales must close at the very latest 1700 CET today, 29th April, and may close earlier. If you would like a ticket and don't have one, get it now. Please don't be disappointed! You can tell us about accommodation preferences, but please note that there are no more single rooms available. -
Working with JSON and Django
In this day and age of web development we work with json, a lot, especially with using web services. Unfortunately, sometimes you can forget what does what, when, and where when dealing with json. I know I have to look up the same thing over and over at times. This blog post is everything you need to deal with json "day-to-day" with Django. JSON to Dict One of the first things we, usually, do is convert json data that we get back from a web service call, or from a webhook, into a python datastructure. import json json_data = '{"hello": "world", "foo": "bar"}' data = json.loads(json_data) The key element from this is the json.loads which loads a string of json and converts the data to a python dict. From there you get to use it like normal. Dict to JSON import json data = {'baz': 'goo', 'foo': 'bar'} json_data = json.dumps(data) Normally you want to get your data into json from your dict, or model, which is just as easy using json.dumps. It converts your dict to a string of json so you can return it to in a response. HttpResponse with JSON There are a couple of key parts … -
Writing styled forms with absolutely minimal HTML (Part-1)
When controlling the form rendering, we all know that we can use the three pre-baked output format of the forms namely: as_u, as_p and as_table. For example: if we use: {{ registration_form.as_ul}} The output template would be: <li> <label for="id_uname">Userame:</label> <input type="text" name="username" id="id_uname"> <li> <li> <label for="id_email">Email:</lable> <input type="text" name="email" id="id_email"> <li> And so on... Here are some questions that come into the picture: What if we to want deal with divs? What if we want to reorder the fields? and what if I have 100 such fields? And what if I want to deal with model forms? How about field.errors and form.non_field_errors? To answer all those significant questions Django-crispy-forms steps into the arena. What are Django-crispy-forms? It's an app that brings up the best way to get the DRY Django forms, providing the tag and filter that lets us quickly render forms in a div format. This also provides an enormous amount of capability to configure and control the rendered HTML. As an added asset Django-crispy-forms support several frontend frameworks like Twitter Bootstrap(versions- 2 and 3), Uniform and Foundation. Using Django-crispy-forms is very simple and effortless. Let's dive in to learn to use the Django-crispy-forms with Twitter Bootstrap … -
Django backward relationship lookup
I often limit the lookup to fields of the model and forget about backward relations. Consider the following relationship: class Group(models.Model): name = models.CharField(max_length=100) class Student(models.Model): name = models.CharField(max_length=100) group = models.ForeignKey(Group) A group can have many students. We want to get the groups based on certain conditions on model Student. An example is getting the groups which contain a student named 'stud1'. If you can get it using model Group and without using Student, you can skip this post. In such a scenario, we tend to use Student model. It's more intuitive because the Foreign Key relationship exists from Student to Group. Let's load the following data to test our queries: group1 = Group.objects.create(name='Group 1') group1 = Group.objects.create(name='Group 2') Student.objects.create(name='stud1', group=group1) Student.objects.create(name='stud2', group=group1) Student.objects.create(name='stud3', group=group1) Student.objects.create(name='stud1', group=group2) Get the groups which contain student named stud1. Intuitive way: We first try to get all the students which satisfy the criteria. Query for that would be: Student.objects.filter(name='stud1') And then we try to get the groups of those students. If we use the Student model, we can't get a queryset of Group. So our approach would be to first get the ids of desired groups and then get a queryset of Group … -
Writing styled forms with absolutely minimal HTML (Part-2)
In the earlier post we have cruised through the fundamentals of using crispy forms for writing styled forms with minimal HTML. At this time, let us dip into getting a clear understanding on how to customize the crispy forms. Before diving into the crux of this post we will become aware with the following key words: FormHelper Layout FormHelper Django-crispy-forms implements a class called FormHelper to define the form rendering behavior. Helpers give us a way to control the form attributes and its layout. They control global form rendering behavior. Layout It is an another powerful class of Django-crispy-forms which allow us to change the way the form fields are rendered. This allows us to set the order of the fields, wrap them in divs or other structure, etc. Custom output Flexible and highly reususable In the first place we will get to know how to render a model form and then we will be customizing the divs and fields in the template. Handling ModelForm with a Django-crispy-form. As we know ModelForms render Model fields as HTML, let us do this using the crispy-forms now. Now lets render an Address form containing the fields like name, street, landmark and so … -
Webhooks, Django, and ngrok
Working with webhooks can be a pain. In this video learn more of what they are and how to easily work with them in your local environment using ngrok.Watch Now... -
Re: What Can Men Do?
TL;DR: Ignore Jeff; read Shanley. Jeff Atwood: If you see any behavior that gives you pause, behavior that makes you wonder “is that OK?” […] speak up. Honestly, as one man to another. OK, you got it. Jeff Atwood’s blog post today asks an important question: “what can men do?” (I’m not linking to Jeff’s post directly; it doesn’t need the signal boosting.) But it fails to answer that question effectively, reinforces several myths about gender in tech, and, worst of all, structurally reinforces male dominance rather than breaking it down. -
Two Scoops of Django 1.6 India Edition Available!
Newsletter #2 The India Edition of Two Scoops of Django 1.6 is now available at the following outlets: Within the next few weeks we hope to add Infibeam to the list of supported Indian retailers. Personal Note From Audrey The India Edition has special significance to us because my father is from India, and some of my family is still there. On my father's side, my grandfather was a civil engineer, and my grandmother was a mathematician. My grandmother passed away when I was very young, and my grandfather passed away a few years ago. I wish that they could have been around to see this. I grew up visiting my family in West Bengal often during the holidays. Information About the India Edition The India Edition, available internationally, is described in detail on our FAQ. -
Two Scoops of Django 1.6 India Edition Available!
Newsletter #2 The India Edition of Two Scoops of Django 1.6 is now available at the following outlets: Within the next few weeks we hope to add Infibeam to the list of supported Indian retailers. Personal Note From Audrey The India Edition has special significance to us because my father is from India, and some of my family is still there. On my father's side, my grandfather was a civil engineer, and my grandmother was a mathematician. My grandmother passed away when I was very young, and my grandfather passed away a few years ago. I wish that they could have been around to see this. I grew up visiting my family in West Bengal often during the holidays. Information About the India Edition The India Edition, available internationally, is described in detail on our FAQ. -
Two Scoops of Django 1.6 India Edition Available!
Newsletter #2 The India Edition of Two Scoops of Django 1.6 is now available at the following outlets: Within the next few weeks we hope to add Infibeam to the list of supported Indian retailers. Personal Note From Audrey The India Edition has special significance to us because my father is from India, and some of my family is still there. On my father's side, my grandfather was a civil engineer, and my grandmother was a mathematician. My grandmother passed away when I was very young, and my grandfather passed away a few years ago. I wish that they could have been around to see this. I grew up visiting my family in West Bengal often during the holidays. Information About the India Edition The India Edition, available internationally, is described in detail on our FAQ. -
DjangoCon Europe accommodation preferences
Our Django Island hotel offers accommodation in various different kinds of rooms (single, twin or double) in various different kinds of apartments (of one, two or three rooms). We need to know your preferences - tell us! Rooms The standard room is a twin (two single beds), which will be shared with another Django Islander of the same sex. If you would prefer to share with a particular person, you can let us know. There are 60 single rooms available. If you need - you don't need to tell us why - or strongly prefer a single room, just say so. If you are travelling with a partner (i.e. you have purchased a "companion" ticket) or you and your partner are both conference attendees, you can ask for a double room. Each room has its own private bathroom. Apartments You can also let us know whether you prefer to be in an apartment with only one room, or amongst others, and with whom. If you do have a preference about apartments, please tell us whether: you'd like to be in an apartment on your own you'd like to share the apartment with friends (tell us whom) you'd prefer to be … -
Assisted places for students
Thanks to the generosity of our sponsors and the attendees who bought a "Contribution" ticket we've been able to support a number of our speakers who were on tight budgets, and who would otherwise have been unable to attend. We are delighted to announce that the same generosity means that we're able to make an additional 7 conference places available to students at a discount of 270 € - that is, for 400 € each. The 400 € covers the conference, accommodation (Tuesday and Wednesday night) and all meals. These tickets are only available, on a first-come first-served basis, to students in full-time education. We'll require some proof of status. We're truly grateful to the community for providing the support that has made this possible. Student tickets
