Announcing the Brighton and Hove Python User Group Google Group

January 29th, 2009

After managing to organise two previous Python meetups and failing to organise many more it seemed like a good time to set-up mailing list of some sort so that things can self-organise a bit more. So please sign-up for the Brighton and Hove Python User Group Google Group (BHPUGGG) and hopefully there will be a few more meetups more often.

For any Pythonistas out there you might want to have a look at the 5K app competition. The idea is to write apps that are less than 5120 bytes in size. It’d be great to see some small Python apps entered and with Python (and a bit of self-extracting script fun) they should pack quite a punch!

Converting a pair of trousers into a laptop sleeve

January 18th, 2009

I ordered one of the new metal Macbooks on the night they came out. So far it’s been a great machine, but there has been one minor problem – it’s a different size to my old 12″ Powerbook. So obviously my old crumpler laptop case was going to be no use.

After purchasing some green cotton thread, grabbing an old pair of trousers and a couple of hours sewing (by hand) I ended up with a perfectly workable laptop sleeve for my Macbook:

A 5K Java Guitar Tuner

January 17th, 2009

So after writing a 5K Twitter Client, a 5K TODO app and a 5K Fullscreen Text Editor I thought that’d be enough example apps to get things started. However while adding a list of app ideas to the 5K App page I suddenly decide that one of the ideas really appealed. The idea was to write a guitar tuner, that would listen on the computers mic and show the user what pitch it thought the note played was. I’m not much of a guitar player, but when I do play it’s always nice to have a guitar that’s in tune. As I’m unable to tune a guitar by ear I rely on guitar tuners and I always seem to be out of the 9V batteries my electronic tuner needs. Given my laptop is usually nearby, being able to use it as a guitar tuner would be most useful.

Check out the video below or download 5KTuner for yourself (requires Java):



The app is more or less non-interactive. It simply starts up tries to work out what note is being played. It shows a simple slider that moves according to what note is being played and indicates how close to that note you are. In addition the raw frequency (in hz) is shown, as well a chart that shows the autocorrelation function of the input sound. A red line is drawn on this chart when a “valid” note is heard and indicates the wavelength of the overall frequency.

The vital statistics are:

  • Final (obfuscated) app size: 3538 bytes
  • Final (un-obfuscated) app size: 4649 bytes
  • Final Source-code size: 8532 bytes
  • Lines of code: 245

This was my first foray into audio programming and I’m grateful that in the end it wasn’t too tricky. Initially I’d been looking into using the Fast Fourier Transform to work out the frequencies, but the autocorrelation function was much easier. It’s essentially just two for loops and a bit of maths. You listen for a certain amount of audio and then compare the first half of the sample with itself, but shifted. By working out the amount of shift where the difference is smallest you can find the overall frequency of the sample.

It could probably be made more robust, but if you want to see how it works (or improve it) you can download the source code.

Client-side sql databases in Javascript

January 13th, 2009

It occurred to me that I didn’t really go into much detail about the local storage part of my 5K TODO app. It’s quite a neat piece of tech and because it works in Safari it also works on the iPhone. That obviously means you can use javascript+local storage to create apps for your iPhone without needing to touch the iPhone SDK or learn objective-c. As everything is client-side you also don’t need server-side code to store your app state. The only downside is that local storage is at the mercy of the user. If they decide to reset Safari or generally clear out their user data the local storage goes too! Then again this would be true of files on a Mac’s file-system anyway…

So for any “serious” stuff you might want to look elsewhere, but in the meantime you can still build interesting client-side apps that use local-storage and run on your iPhone.

At the time of writing database storage in html5 is still a work in progress and is only available in Safari/Webkit. In the future it should appear in Firefox etc. Though there’s a HTML5 Wrapper for Google Gears that uses Gears to provide the html5 local storage functions (e.g. openDatabase).

Opening the database

The main function we’re concerned with is openDatabase which let’s us open/create a database that can only be accessed by the current domain (in a similar manner to cookies). In the TODO app I use this code to see if openDatabase is available and output a message if not and stop any further processing:


    if ( !window.openDatabase ) {
        $('add_note').innerHTML = "Browser does not support local storage."
        return;
    }

(note $ is a shortcut to document.getElementById in the TODO app code)

Assuming we find the openDatabase function we can then get a database object thus:


        var dbName = '5KTodo',
             db = openDatabase(dbName, '1.0' /*version*/, dbName, 65536 /*max size*/);

In this case we create/open a database called 5KTodo. We provide a version number (not really used at this stage, but is intended for allowing upgrading database schemas) and a max size. The max size is provided as a hint to the browser as to how big the database will grow. The upper limit for this is defined by the browser (and is about 5M in Safari).

Now once we’ve got our database object we can start running some SQL. We do this using the database object’s transaction method and the transaction’s (tx) executeSql function:


    db.transaction(
        function(tx) {
            tx.executeSql(sql, params, callback);
        }
    );

That will run the sql using the given params (replacing ? in the sql string with the param values to avoid sql-injection attacks) and then calls the callback with the results (if it succeeded).

Creating the tables

The first time we run the app the database will not exist and will contains no database tables, so we need to run some SQL to create the tables:


    function dbTransaction(fn) {
        db.transaction(fn);
    }
    
    /*create todo table*/
    dbTransaction(
        function (tx) {
            tx.executeSql('CREATE TABLE todo (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, description TEXT NOT NULL DEFAULT "", todo_order INTEGER NOT NULL DEFAULT 0, done INTEGER NOT NULL DEFAULT 0 );');
        }
    );

On the next run of the application that will fail silently, which is fine as it only needs to run successfully once (to create the tables). It’s key that the create table statement runs in it’s own transaction though, so when it fails it doesn’t stop anything else running. Note that dbTransaction used here is only so that YUICompressor can better compress the code, as this code is from an example for the 5K App.

Reading from the database

After creating the tables (or doing nothing if they are already there) the next step is to see what’s already in the database. Unlike the create table statement we expect to get some data back, so we provide a callback to get the results:


    dbTransaction(
        function (tx) {
            listTODOs(tx, 'SELECT * FROM todo ORDER BY todo_order ASC', []);
        }
    );
    
    /*run the sql to read the notes and insert them into the DOM*/
    function listTODOs(tx, sql, params) {
        tx.executeSql(
            sql, params,
            function(tx, results) {
                var rows = results.rows;
                for ( var i = 0; i < rows.length; i++ ) {
                    var row = rows.item(i),
                        li = createElement('li'),
                        checkbox = createElement('input'),
                        label = createElement('label');
                    setAttribute(checkbox, 'type','checkbox');
                    setAttribute(checkbox, 'id', 'checkbox_'+row['id']);
                    setAttribute(label, 'for', 'checkbox_'+row['id']);
                    appendChild(label, createTextNode(row['description']));
                    appendChild(li,checkbox);
                    appendChild(li,label);
                    setAttribute(li,'id', 'note_'+row['id']);
                    
                    if ( !row['done'] ) {
                        // not done, so insert above text field
                        notesList.insertBefore(li,$('add_note'));
                    }
                    else {
                        // done so put below text field
                        setAttribute(checkbox,'checked','checked');
                        li.className = 'done';
                        appendChild(notesList, li);
                    }
                    
                    attachTODOHandlers(row['id'],li,checkbox);
                }
            });
    }

(note there are a lot of shortcut functions in there, but hopefully you should get the idea)

What that basically does is execute 'SELECT * FROM todo ORDER BY todo_order ASC' on the database in a transaction (tx) and then takes the results and inserts them into the DOM.

Inserting into the database

Inserting into the database is pretty straightforward - we just use an SQL insert statement. Though in the case of the 5K TODO app we need to do a couple of other things. The first is figuring out the next highest todo_order, then we insert the TODO item and next we re-select the TODO items from the database (with the id of the newly inserted item - results.insertId). The 2nd part is a bit overly obtuse in some respects, but was done so as to help cut down on code size (to fit under the 5K limit):


    function insertTODO(description) {
        dbTransaction(
            function(tx) {
                // set order to be one after items that aren't done
                tx.executeSql('SELECT MAX(todo_order) AS next_order FROM todo WHERE done=0', [],
                    function(tx,results) {
                        var order = results.rows.item(0)['next_order'] + 1;
                        tx.executeSql('INSERT INTO todo (description,todo_order) VALUES(?,?)', [description,order], 
                            function(tx,results) { 
                                listTODOs(tx, 'SELECT * FROM todo WHERE id=?', [results.insertId]);
                                updateTODOOrder(tx);
                            }
                        );
                    }
                );
            }
        );
    }

As the later SQL statements depend on results from the earlier statement(s) they actually get run in the statement callback functions.

It's fairly straightforward to access a database this way. For those familiar with databases it should all be pretty familiar territory. There's plenty of more info on Apple's site about using the Javascript database. You may also want to look at the source code the the 5K TODO app, though be warned a little bit of readability has been sacrificed for code size.

A 5K Python Fullscreen Text Editor

January 5th, 2009

After a 5K Java Twitter client and a 5K Javascript TODO list app, the next example 5K App is a wxPython fullscreen text editor. Much like Writeroom and Duncan’s JDarkroom (both of which are more fully featured) it lets you edit text files in “distraction-free” fullscreen. Ideal for creative writing or similar where you want to just get on with writing without the distraction of the outside world. So at the risk of invoking the apoplectic rage of Mark Pilgrim a fullscreen text editor seemed quite feasible in 5K of Python. In fact after using my self-extracting script code I actually had to add more features as I was way below budget!

Check out the video below or download 5KEdit for yourself.



Features:

  • Fullscreen text editing
  • Open/Save files
  • Undo/Redo
  • Retro looks
  • Resizable fonts
  • Configurable colours
  • Word count
  • Goto line number dialog
  • Find text dialog
  • Help page listing commands
  • About box

It’s actually a bit more feature-full than I’d thought it would be. Part of this was that wxPython’s wx.StyledTextCtrl and wxPython in general provided quite a lot of what I needed (e.g. undo/redo) out of the box. Plus the self-extracting script technique really did a good job of shrinking the code down – allowing me to add more.

To run 5KEdit your need Python (2.4+) and wxPython (2.8+) installed. If you are running OS X Leopard you already have those installed, so you can simple switch to a terminal and run that 5KEdit script using the command:

pythonw 5KEdit.pyw

Make sure you run that from the directory 5KEdit.pyw is in. On a Windows system you may find that you can simply double-click on the .pyw file to run it. There’s a small repaint bug under Windows when you change colors, but it’s not a show stopper.

Here are the vital statistics:

  • 17284 bytes/485 lines of original source-code
  • 13767 bytes/379 lines of stripped source-code
  • 5069 bytes of compressed/packed final version

As you can see the final version is less than a third the size of the original source code. A large part of this is due to the compression, but to help push it a bit further the source was also “stripped”. Blank-lines and comments were removed and the indentation was changed from my usual 4 spaces to a single space. The stripping doesn’t make a huge contribution, but without it the final version would be 5421 bytes in size. So for 400 bytes or so it’s probably worth it. Plus it means that you don’t have to worry about leaving yourself comments or spacing out your code nicely.