en205

My Mac does a great job of providing me with the tools I need to make data management efficient. Mail.app, Eclipse, Address Book, iCal, iPhoto, iTunes etc take care of their respective data types with style. I normally use my text editor to keep track of arbitrary structured data that isn’t covered by a tool I like. I’ve got backup records, financial data, minutes and notes from various types of meetings, lists of books/movies/CDs, passwords (encrypted and plaintext) and more. What I’ve realised I want when dealing with these, but don’t get from the combination of my filesystem and the text editor, is:

• Input cues. I want forms to fill in, rather than a blank page that I’ll mess up or have to think about laying out.
  
• Data validation. I want the system to tell me when I forget data or input something insane.
  
• Derived data fields. I want summary statistics to be displayed as I enter data and search for it.
  
• Structured search. Spotlight (or even CTRL+F) is great, but it’s not the most convenient way to list action items from project meetings last month or check when I last backed up my iPhoto library. It certainly won’t let me pipe the results into another tool.
  
• Versioning. I want change metadata to let me search and backtrack. 
  
• Synchronisation to my ipod/phone. Some data needs to be with me at all times. I should be able to mark it for synchronisation to my mobile devices.
  

Since “personal database” seemed a reasonable description of what I had in mind, downloading the Bento trial was my first attempt to give myself and TextMate a rest and satisfy my urge to systematise. Unfortunately Bento only made me more aware of what I was lacking. Only simple type-based validation is possible, calculated fields have very limited functonality, there is no programmatic interface to the data, and the forms interface is restrictive and surprisingly unattractive.

Other personal database applications I tried were evidently built by people trying to solve different problems. Instead I’ve adopted Django as a flexible solution that gives me interface richness and complete control over my data-models at the cost of some programming time. Django’s “admin” tool automatically provides nice CRUD forms for my models. Fortunately I like programming and consider learning a new web framework a worthwhile time investment, though I’ll probably prefer to wait for someone else to solve the mobile device synchronisation problem. I still think there’s a niche for an application to help structure-focussed folks who aren’t necessarily programmers. Maybe Filemaker will get closer with the next version of Bento.

I recently needed a Support Vector Machine (SVM) classifier for a document classification experiment written in Python. I chose to use Thorsten Joachims’ SVMlight. I couldn’t find a Python interface, so I wrote a simple shell-based one.

class SVMLight:
    """
    An interface class for U{SVM light<http://svmlight.joachims.org/>}
    
    This class currently supports classification with default options 
    only. It calls the SVMLight binaries as external programs.
    
    Future versions should add a SWIG interface and support for use of 
    non-default SVMlight options.
    
    C{SVMLight} reads sparse feature vectors - dictionaries with 
    numeric keys, representing features, and arbitrary numeric values.
    """

svmlight.py is free for any use.

Seasons 1-5 of Curb Your Enthusiasm are $15 each at JB Hifi in Bourke St (Melbourne). Gold.