Empty Thoughts

This Week in Django is a weekly podcast about all things Django.

This week we discuss the NewForms-Admin merge into Trunk, DjangoCon, a few source commits, some cool projects from the community, and the Tip of the Week.

Please see the Show Notes below for all the pertinent information and links

Downloads

AAC Enhanced Podcast (41.8 MB, 49:31, AAC)

MP3 Edition (34.1 MB, 49:31, MP3)

OGG Edition (27.9 MB, 49:31, Vorbis)

The Enhanced Podcast version contains screenshots and easy access links to all of the items we discuss throughout the podcast.

Feeds Available

iTunes Feeds are available. By subscribing using the iTunes feeds the podcasts will automatically be downloaded for you when we release them.

iTunes Feeds

This Week in Django – AAC Edition

This Week in Django – MP3 Edition

Regular RSS Feeds

This Week in Django – AAC Edition

This Week in Django – MP3 Edition

This Week in Django – OGG Edition

Give Us Feedback

Want to give us some feedback on the show? We’re always looking for ideas or suggestions that will help improve each episode. Please contact us at feedback __at__ thisweekindjango.com.

Show Notes

Big News (0:47)

• NewForms-Admin Branch Merges to Trunk – A huge step towards Version 1.0.
  • Version 1.0 Sprint Schedule
  • NewForms Admin Documentation
  • Updated NewForms Documentation
  • newforms-admin Migration and Screencast – Great way to get started and make the migration to NewForms-Admin.
  • Pre-NFA Merge Tag

• DjangoCon Web Site and DjangoCon Schedule Released – It’s quite a lineup. We hope to see everyone there.

Tracking Trunk (15:04)

• Support for pre-1900 dates (7946)
  • Missing Files From 7946 (7950)

• Removed the mysql_old backend (7949) – According to Malcolm Tredinnick, it smells bad and has no friends.

• updated generic create_update views to use newforms (7952)

• newforms -> forms switch (7971) ** django.newforms is now django.forms. Updated your code.

• Added support for optgroups in models and form fields (7977)
  • Documentation on new optgroups support

• Performance Improvements for urlize and urlizetrunc filters (7985) – This one has bit us on the Django Logger.

• Exception on 400 Response Fallback to 500 Response (7988)

Community Catchup (26:16)

• Kevin Fricovsky – joins the This Week in Django team as Community Evangelist. Kevin will be working to produce the show, contacting guests, gathering news items, coming up with ideas. Kevin has been doing this work regularly anyway so it’s great of him to team up with us to help out the program:
  • Monty Lounge Industries – Kevin’s web strategy, design, and development company.
  • How I Work Daily – Kevin’s blog. You should have this one in your feedreader.

• PyWorks conference – to be held in Atlanta, GA on November 12-14, 2008.

• PyOhio – Reminder that this free one day conference is in Columbus, OH on July 26, 2008.

• Twitter Search – via Kevin Fricovsky using the new Twitter Search capability to track all tweets about django.

• Jinja2 Final aka Jinjavitus Released – Armin Ronacher and the rest of the Pocoo team announced the release of this wicked-fast Django inspired template engine.

• OSCON Python BoF – Tuesday, July 22nd 7pm – 10pm from Jax Bar and Restaurant. Via Jason Kirtland’s excellent blog discorporate.

Tip of the Week (38:32)

This tip comes to us via Ben Jao Ming in his post Django auto-translation of field values.

If you need to translate content in a field then gettext is not going to help you out. Since you can create your own custom fields it’s easy to wrap a CharField with the translation behavior:

from django.db import models
from django.utils.translation import gettext_lazy as _

class AutoTranslateField(models.CharField):
    __metaclass__ = models.SubfieldBase
    def to_python(self, value):
        return str(_(value))

Just add whatever translations you know of to the locale file and run compilemessages.

Thank You! (42:45)

• 7 Habits For Effective Text Editing 2.0 – Awesome video by Bram Moolenaar
  • Vim Cheatsheet PDF

• Brian Rosner
  • Brian Rosner Blog
  • Brian on Twitter
  • TWiD Live

In the ongoing march to Version 1.0, the Django project just tagged and bagged the 1.0 Alpha version with Changeset 8037. I encourage everyone to grab the tarball or svn up and start banging on the latest and greatest code.

update: Django 1.0 alpha release notes

update: Django 1.0 alpha released! blog post

Brian Rosner just released a new screencast and blog post, newforms-admin Migration and Screencast, that walks you through how to get started with migrating to NewForms-Admin from your existing trunk based code. The screencast and accompanying blog post are very well done. It’s great to get this kind of helpful information, especially from someone that knows so much about the internals of the code.

I highly encourage everyone to check it out, especially if you’re on the fence on whether or not the conversion is worth it. Brian shows how easy it is to get started.

Robert Lofthouse just announced that the DjangoCon schedule is available. There are a ton of exciting speakers and topics being presented. It’s going to be a great time and I’m looking forward to seeing everyone there. I am curious if people will be extending the conference into the following week and if sprints are being planned. I’d like to make the most of it.

Brian Rosner just now, with Changeset 7967, merged the NewForms-Admin branch of Django into Trunk. This is a huge step forward in the push to version 1.0. I want to personally thank all of the numerous contributors that have made NewForms-Admin possible. It’s been an enormous amount of work, and I am so appreciative. Thanks!

update: Read the release post.

I’m pleased to announce the release of GitPython 0.1.4. I appreciate all of the work from contributors on this release, especially from Florian Apolloner who has really taken the lead and managed everything.

DOWNLOAD

You can get it directly from cheeseshop at: http://pypi.python.org/pypi/GitPython/

Or checkout the tag with:

$ git fetch --tags 
$ git checkout -b 0.1.4 0.1.4 

CHANGES

• renamed git_python to git. Be sure to delete all pyc files before testing.

Commit

• Fixed problem with commit stats not working under all conditions.

Git

• Renamed module to cmd.
• Removed shell escaping completely.
• Added support for stderr, stdin, and with_status.
• git_dir is now optional in the constructor for git.Git. Git now falls back to os.getcwd() when git_dir is not specified.
• add a with_exceptions keyword argument to git commands. GitCommandError is raised when the exit status is non-zero.
• add support for a GIT_PYTHON_TRACE environment variable. GIT_PYTHON_TRACE allows us to debug GitPython’s usage of git through the use of an environment variable.

Tree

• Fixed up problem where name doesn’t exist on root of tree.

Repo

• Corrected problem with creating bare repo. Added Repo.create alias.

Please let us know if you find problems.

We’re at a point where we have to decide where to go with the library, so if you have ideas, we’d like to know that as well.

This Week in Django is a weekly podcast about all things Django.

This week we have a special guest, Kevin Fricovsky, that joins us as we talk about DjangoCon, EuroPython, a few source commits, some cool projects from the community, and the Tip of the Week.

Please see the Show Notes below for all the pertinent information and links

Downloads

AAC Enhanced Podcast (74.2 MB, 1:31:20, AAC)

MP3 Edition (62.8 MB, 1:31:20, MP3)

OGG Edition (51.5 MB, 1:31:20, Vorbis)

The Enhanced Podcast version contains screenshots and easy access links to all of the items we discuss throughout the podcast.

Feeds Available

iTunes Feeds are available. By subscribing using the iTunes feeds the podcasts will automatically be downloaded for you when we release them.

iTunes Feeds

This Week in Django – AAC Edition

This Week in Django – MP3 Edition

Regular RSS Feeds

This Week in Django – AAC Edition

This Week in Django – MP3 Edition

This Week in Django – OGG Edition

Give Us Feedback

Want to give us some feedback on the show? We’re always looking for ideas or suggestions that will help improve each episode. Please contact us at feedback __at__ thisweekindjango.com.

Show Notes

Big News (3:38)

• EuroPython 2008 has come and gone.
  • EuroPython Talk Materials

• DjangoCon – Jacob Kaplan-Moss let the cat out of the bag and let us know that a DjangoCon is being planned for the 1.0 release timeframe for September 6th and 7th. More details to follow. We’ll keep you posted.
  • Official Announcement
  • A Quote from Robert Lofthouse via Simon Willison’s blog.

“DjangoCon 2008. Venue: Googleplex, San Francisco Bay Area. Dates: 6th and 7th Sept. Official post will be on djangoproject.com soon.”

• Callcast – Jacob Kaplan-Moss and DjangoCon 2008

Tracking Trunk (14:27)

• Added a ping_google management command to the sitemaps contrib (7813) – Makes it easy to ping Google once your Sitemap is up and running.

• Fixed #2070: refactored Django’s file upload capabilities. (7814)
  • Django now handles large files nicely in uploads with the new streaming file upload handling.
  • New uploading documentation
  • There have been a ton of fixes since this landed.
  • Multi-part MIME parsing fix (7905)
  • Support pluggable backends for FileField (5361)

• Moved make-messages, compile-messages and daily-cleanup into django-admin.py (7844) – This makes them easily accessible as management commands and reduces the number of binaries and man pages.
  • django-command-extensions – This is a repository for collecting global custom management extensions.
  • Jannis Leidel -

• Rolled QuerySetPaginator into the Paginator class, to simplify things. QuerySetPaginator still exists as an alias, for backwards compatibility. (7865)

Branching and Merging (34:18)

• Added autodiscover functionality to django.contrib.admin. (7872)
  • This makes the admin aware of per-app admin.py modules and does an import on them when explicitly called.

• Added some documentation about media defintions in ModelAdmin classes. (7873)
  • Subtle backward incompatible change

• Learning NewForms-Admin
  • NewForms-Admin Wiki
  • NewForms-Admin HowTo Wiki
  • Django Screencasts 7 – NewForms Admin I

Community Catchup (47:27)

• How to build Django web apps – Looks to be a great workshop with one of Django’s superstars, Jeff Croft.

• How To Build A Web App in Four Days For $10,000

• django-treemenus – Very cool application by Julien Phalip that makes it easy to add tree menus to your Django app and manage the structure dynamically from the customized admin interface. View the 0.4 release post for more information.

• First Two Django Screencasts – Eric Florenzano never sleeps. His first foray into screencasts are quite impressive. He covers: 1. Setting up a Django Environment 2. Using Django-Pagination

• Interview with Jacob Kaplan-Moss – Great interview by Brazilian Marinho Brandão.

• Yahoo’s Build your Own Search Service in Django – Will Larson offers two informative articles on integrating BOSS with Django.
  • Polishing Up Our Django & BOSS Search Service

Tip of the Week (1:10:41)

Mike Axiak offers the following tip for getting started with contributing to Django source code.

1. Find a closed ticket that interests you. 2. Take a look at a closed ticket, but not the patch. 3. Check out the revision prior to the one that closed the ticket. 4. Write a patch for Django with your own implementation. 5. Compare your patch with the committed patch and try to learn from the differences.

Once you’re comfortable do it for real.

• Framework and tests to check for the correct operation of django-admin and manage.py

Thank You! (1:23:24)

• Django-Chunks – Simple keyed content that you can insert into your templates.
• Cloud27 – A social network built on Pinax.
• This Week in Django 24 – Interview with Pinax / Cloud27 founder James Tauber.
• Django-Hotclub Channel

• Kevin Fricovsky
  • How I Work Daily
  • Django-NYC – Discussion group for Django-NYC attendees. If you’re in the NY area, be sure to check it out.

• Brian Rosner
  • Brian Rosner Blog
  • Brian on Twitter
  • TWiD Live

There’s been lots of discussion and hints over the past several days about a forthcoming Django based conference called DjangoCon. The official announcement was just released. I’ve always been fond of small intimate group settings and this one will certainly be that with only 200 attendees allowed to participate. I only hope that I get one of the coveted spots.

Although some people may be aware that SQLite supports Full-Text Search (FTS) I would guess that not many have much experience with getting it setup and using it. I only base this information on the fact that the documentation on the SQLite site as well as what I could find on the Googles has been pretty slim. As a result I thought it might would to provide a walkthrough on getting SQLite FTS setup and working. That said I must preface this by saying that the instructions here apply specifically to Mac OS X.

Too Many Options

There are a lot of downloads available for SQLite, and that alone can cause some confusion. The thing to keep in mind is that with SQLite you’re really dealing with two parts: the engine and the client. The client is what you use to interact with a SQLite database on the command-line. The engine is the functionality itself and can operate independently of the command-line application.

When it comes to building from source code, there’s a couple of different options as well. First there’s the raw source. But you can also get an amalgamation of the source. This comes in two different flavors: with and without configure scripts and makefiles. You might be curious as to why there are so many options, and it really comes down to being able to support the many different ways people use SQLite. Some folks are only interested in using SQLite in an embedded environment where they might be linking to the engine directly. On the other spectrum some people are using SQLite as a nice testing platform for doing web development. This often requires the SQLite engine, a language-engine interface (dbapi) library, and the client application.

To support FTS within SQLite we have a couple options. SQLite supports the idea of loadable plugin modules. In other words technically we could build the FTS plugin as a dynamically loadable module and then load it before use. The second option is to statically link the FTS plugin into the SQLite source code at build time. This is the approach I will take here.

Compiling and Installing

The source code bundle that we need is the sqlite-amalgamation package. The amalgamation is a single large source code file that contains the entire SQLite implementation, and (as of version 3.5.3) the Full-text search engine. Therefore we want to be sure we get at least release version 3.5.3, just to make things simple. Since there’s two amalgamation packages be sure to select the one that contains the configure script and makefile for building it.

First let’s download and extract the source code:

$ curl -O http://www.sqlite.org/sqlite-amalgamation-3.5.9.tar.gz
$ tar -xvzf sqlite-amalgamation-3.5.9.tar.gz
$ cd sqlite-amalgamation-3.5.9

Now that we have the source we just need to follow the standard process for configuring, compiling, and installing the package. The most important piece is setting configuration parameters appropriately for our platform. Since we want to statically include FTS we need to let the compile know that’s what we plan to do. Additionally since I’ll be installing on Mac OS X I need to be sure to tell the Makefile that I’ll be changing the default location for the installation. To do this let’s issue the following command:

$ CFLAGS="-DSQLITE_ENABLE_FTS3=1" ./configure --prefix=/usr

Note that we’ve enabled FTS3, the latest version, plus we’ve also modified the installation prefix to /usr from the default /usr/local. This last piece is important because if you don’t do that you’ll actually end up with two versions of SQLite, one that you’ll access through the command-line and one that Python itself will be accessing. This tripped me up for a bit, because it appears that you’re using the latest version.

Along these lines, it is important to not get confused by the actual _sqlite3.so library used by Python. On my Mac this is located in the following directory:

/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/

It appears that this loadable module simply calls out to the installation in /usr/lib/sqlite3.

The next thing we need to do is actually compile the source and install it. So to do that issue the following two statements:

$ make
$ sudo make install

At this point the latest SQLite should be installed and ready to go on your system. We can verify this two different ways. First let’s verify that the command-line client is in fact updated:

$ sqlite3 --version
3.5.9

If you got something other than that you might want to re-check the steps. Secondly we want to be sure that Python is using the correct version.

Note: I assume Python here because that’s what I’m using but if you’re using a different language you’ll want to modify the checks accordingly. If you are still seeing the older version then you might want to investigate how SQLite is being used in your system. Since it has such a small footprint it’s not uncommon for it to be just bundled in with the program. I know that a search on my system found at least 4 or 5 different installations of SQLite.

$ python
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.5.9'

The above assumes that you’re using Python 2.5+. If you’re using an older version of Python then you will likely be using the pysqlite module.

Usage

Once we have SQLite with Full-Text Search capability built in the next step is to just get familiar with the how it all works. I’m not going to go into too much detail here, because there is pretty good usage documentation on the SQLite website.

The key piece is that when you want to utilize FTS on SQLite you need to create your table as a virtual table that utilizes the FTS engine. To do that we need to modify the table creation syntax slightly:

CREATE VIRTUAL TABLE posts using FTS3(title, body);

After we’ve created the appropriate virtual table we can interact with it just like any other table. The difference is we have access to a new keyword syntax: the MATCH operator. The MATCH operator is used to access the FTS functionality.

For instance if we want to find all posts that contain the word python in them we can issue the following SQL:

SELECT * FROM posts WHERE posts MATCH 'python';

This will actually find all posts where the title or body columns contain the text python. In addition to this simple syntax the FTS engine supports things like OR and not (-) types of statements, as well as more complex items like prefix searching:

SELECT * FROM posts WHERE posts MATCH 'py*';

Note: one of the limitations (there’s many) of SQLite’s FTS is that you can’t use more than one MATCH operator in a query. To counteract this just be sure to combine search terms into a single MATCH operator statement and take advantage of the search syntax to do and, or, not, etc…

Have Fun

Well I hope this is enough to get you on your way to using Full-Text Search within SQLite. If you have any special tips, please be sure to leave me a comment.

This morning I got my first glimpse of what will become Cloud27, featuring a sporty design by Greg Newman, and I already like what I see. Cloud27 is a new social network based on Pinax, a Django reference application for reusable applications.

So if there’s Pinax, why Cloud27? Well Pinax is where all development and new ideas happen. Pinax is project intended to provide a starting point for websites. Cloud27 will actually be focussed on building out on the base of Pinax and expanding it into a kick-ass social network application.

To find out a whole lot more about reusable applications and Pinax, I highly recommend a listen to This Week in Django 24, featuring Pinax / Cloud27 founder James Tauber.

I’ll be keeping an eye on Cloud27.