Category Archives: Application

“Ultimate” Arduino Doorbell – part 2 (Software)

As mentioned in the previous post about my arduino doorbell I wanted to get the doorbell and my Chumby talking.

As the Chumby runs Linux, is on the network and is able to run a recent version of Python (2.6) it seemed like it would be pretty easy to get it to send Growl notifications to the local network.

This did indeed prove fairly easy and involved three main activities:

  • Listening to the serial port for the doorbell to ring
  • Using Bonjour (formally Rendezvous)/Zeroconf to find computers on the network
  • Sending network Growl notifications to those computers

Getting Python and PySerial running on the Chumby is pretty easy. The Chumby simply listens for the doorbell to send the string ‘DING DONG’ and can then react as needed.


def listen_on_serial_port(port, network):
    ser = serial.Serial(port, 9600, timeout=1)
    try:
        while True:
            line = ser.readline()
            if line is not None:
                line = line.strip()
            if line == 'DING DONG':
                network.send_notification()
    finally:
        if ser:
            ser.close()

port is the string representing the serial port (e.g. /dev/ttyUSB0). network is an object that handles the network notifications (see below).

The network object (an instance of Network) has two jobs. Firstly to find computers on the network (using PicoRendezvous – now called PicoBonjour) and secondly to send network growl notifications to those computers.

A background thread periodically calls Network.find, which uses multicast DNS (Bonjour/Zeroconf) to find the IP addresses of computers to notify:


class Network(object):
    title = 'Ding-Dong'
    description = 'Someone is at the door'
    password = None
    
    def __init__(self):
        self.growl_ips = []
        self.gntp_ips = []

    def _rendezvous(self, service):
        pr = PicoRendezvous()
        pr.replies = []
        return pr.query(service)
    
    def find(self):
        self.growl_ips = self._rendezvous('_growl._tcp.local.')
        self.gntp_ips  = self._rendezvous('_gntp._tcp.local.')
        def start(self):
        import threading
        t = threading.Thread(target=self._run)
        t.setDaemon(True)
        t.start()
    
    def _run(self):
        while True:
            self.find()
            time.sleep(30.0)

_growl._tcp.local. are computers that can handle Growl UDP packets and _gntp._tcp.local. those that can handle the newer GNTP protocol. Currently the former will be Mac OS X computers (running Growl) and the latter Windows computers (running Growl for Windows).

I had to tweak PicoRendezvous slightly to work round a bug on the Chumby version of Python, where socket.gethostname was returning the string '(None)', but otherwise this worked ok.

When the doorbell activates netgrowl is used to send Growl UDP packets to the IP addresses in growl_ips and the Python gntp library to notify those IP addresses in gntp_ips (but not those duplicated in growl_ips). For some reason my Macbook was appearing in both lists of IP addresses, so I made sure that the growl_ips took precedence.


    def send_growl_notification(self):
        growl_ips = self.growl_ips
        
        reg = GrowlRegistrationPacket(password=self.password)
        reg.addNotification()
    
        notify = GrowlNotificationPacket(title=self.title,
                    description=self.description,
                    sticky=True, password=self.password)
        for ip in growl_ips:
            addr = (ip, GROWL_UDP_PORT)
            s = socket(AF_INET, SOCK_DGRAM)
            s.sendto(reg.payload(), addr)
            s.sendto(notify.payload(), addr)

    def send_gntp_notification(self):
        growl_ips = self.growl_ips
        gntp_ips  = self.gntp_ips
        
        # don't send to gntp if we can use growl
        gntp_ips = [ip for ip in gntp_ips if (ip not in growl_ips)]
        
        for ip in gntp_ips:
            growl = GrowlNotifier(
                applicationName = 'Doorbell',
                notifications = ['doorbell'],
                defaultNotifications = ['doorbell'],
                hostname = ip,
                password = self.password,
            )
            result = growl.register()
            if not result:
                continue
            result = growl.notify(
                noteType = 'doorbell',
                title = self.title,
                description = self.description,
                sticky = True,
            )

    def send_notification(self):
        '''
        send notification over the network
        '''
        self.send_growl_notification()
        self.send_gntp_notification()

The code is pretty simple and will need some more tweaking. The notification has failed for some reason a couple of times, but does usually seem to work. I’ll need to start logging the doorbell activity to see what’s going on, but I suspect the Bonjour/Zeroconf is sometimes finding no computers. If so I’ll want to keep the previous found addresses hanging around for a little longer – just in case it’s a temporary glitch.

You can see the full code in my doorbell repository.

Kite Physics

I’ve finally managed to finish the Kite Physics Demo I started back in October:



Kite Physics Demo

It’s more of a plaything than anything else. It did give me a chance to try out writing some physics code again, something I don’t get much of a chance to do during web development. Though of course advances in Javascript speed and the Canvas tag may change that in the future.

Originally I made use of the keyboard to control the figure and the kite, but I switched to using the mouse as it makes interacting with the scene more pleasing. There’s a certain joy to being able to grab the kite to move it around or shift the wind direction by nudging a cloud.

It’s all written in Java, using maven to build and package. The code is available in my kite repo on guthub.

There’s about one third of a physics engine in the code. There’s no real collision detection for example. Most of the physics code is for handling constraints and basic forces. For the constraints I made use of the Gauss-Seidel technique, as outlined in Advanced Character Physics by Thomas Jakobsen. This was quite a revelation to me. Previously I’d experimented with rigid body dynamics (see my 2D racing game circa 2001), but the Gauss-Seidel method seemed much more intuitive. I found it much easier to think of bodies made up of points connected by lines. The “physics” naturally emerges from the way the bodies are then constructed.

The core physics code (in the com.psychicorigami.physics.* package) is fairly well structured. There a bit of compiler placation to (theoretically) allow for the same code to work for 2D and 3D physics. I’ve only actually implemented the 2D side of things, but have tried to use generics to make room for 3D too. When everything is points and lines, moving from 2D to 3D is not that tricky. Whereas for rigid body dynamics this can mean quite a big change in representation.

The code in the default package is used for the specifics of the demo itself, so is a bit messy really. Lots of tweaking and prodding was needed to get things how I wanted.

The physics for the kite and wind interaction were aided massively by this guide to forces on a kite. Particularly useful for understanding how to tweak the rigging, centre of gravity and centre of pressure of the kite to make it behave in a pleasing – if not entirely realistic – manner.

At some point I hope to revisit the physics code here. I’ve always had a wish to create something like Karl Sims “blocky creatures”, but lacked the specific technical knowledge. Know that I have a technique for simulating complex constraints in an intuitive manner maybe I’ll manage it yet.

watch-cp.py

During software development it’s key to minimize the time between editing code and seeing the results of a change. During web-development the constant tweaking of CSS/HTML/Javascript etc means you’re always reloading the browser to see changes.

At work I do a lot of Java web development, which normally involves compiling code, packaging into a .war file and deploying it to Tomcat (running locally). I make use of the maven tomcat plugin, so it’s just a case of calling maven tomcat:redeploy. However it still takes tens of seconds (or more if there are tests to run). For quick tweaks of css it’s nice to be able to short-circuit this process.

Tomcat unpacks the .war file to another directory after the app has been deployed. All the .jsp pages, css and javascript files can be edited in this directory and changes can be seen immediately. However getting into the habit of editing these files within this directory is usually a bad idea, as the files will get overwritten at the next deployment.

We’ve been using compass lately for css and it has a handy command:


    compass watch

That monitors .sass files for changes, then re-compiles them to css as they change, so you can see changes quickly (without needing to manually run compass each time).

So I thought I could do something similar for the editable files within the war file. So I created watch-cp.py. It simply monitors a set of files and/or directories for changes and copies over files that have changed to a source file or directory. To provide a bit of feedback it prints out when it spots a changed file, but beyond that it’s pretty brute-force in it’s approach.

watch-cp.py works in a very similar way to the cp command, but without as many options. For example:


    # recursively copy files from src/main/webapp to tomcat
    watch-cp.py -r src/main/webapp/* ~/tomcat/webapps/mywebapp

This is great as it means I can edit my sass files, have them compiled to css by compass, then copied over to tomcat without needing to intervene. It takes a second sometimes, but it’s fast enough for the most part.

Feel free to fork the gist of watch-cp.py on github.

iPhone Brighton Buses webapp/javascript front-end

Here in Brighton a good number of the bus stops have electronic boards, that tell you when the next buses are due. The data for these boards is available online and even provides data for stops that don’t have the electronic boards. In fact there’s an free iPhone app available, so you can get the data easily on the move.


Brighton Station (Stop B)

After playing around with iPhone javascript apps, I thought it would be interesting to try and create a javascript app for the bus times. I had some specific ideas about how I wanted the app to work. I was also keen on trying out some more “HTML 5” tech. Plus I do like things to be (at least in principle) cross-platform, so this has a chance of working ok on Android phones (though I’ve not tested it on one).

There are still a few rough edges, but you can try out the app yourself at:

It will prompt you to add a stop. If you type in the text field, you’ll get a list of stops that match. If your browser supports geo-location, you’ll also see a “Find nearby” button, which will list the nearest few stops.

I had the pleasure of demoing this app at the Async Javascript show’n’tell last month. I quickly outlined how it worked, which involves the following:

Getting the data

The buses.co.uk site has an API, which I used to scrape all of the bus stops in Brighton. I chose to scrape the update data from the website itself though, as I couldn’t get the API to give me the combined services for each stop (so you can see when the next 7 and 81 are coming for example). There are typically two stops for each name – one either side of the road. The updates are for both directions, so I actually merged the stops together – which makes searching easier.

All of the stop data was stored in a single SQLite database – mostly for convenience. The database is used in a read-only method in the actual running app. I’m sure I could use something a bit better, particularly when it comes to the distance queries. Currently, as there are a relatively small number of stops I’m using a brute force method to find the nearest stops, but I’m sure with the use of a proper spatial index this could be done a lot more efficiently. If this was scaled up to work for all stops in the UK, then that would be a necessity.

Geo-location

Initially the geo-location was pretty straightforward – I simply used the getCurrentPosition function and I got a latitude and longitude back. Testing in Firefox and Safari on the mac gave fairly reasonable results. However on the iPhone itself I started to notice it wasn’t very accurate. Sometimes it was out by several 100 metres, meaning that stops I was standing next to were sometimes not showing up at all! I had noticed, in the past, that the built-in map app sometimes has a bit of “lag” in getting an accurate position. So I switched to using watchPosition and polling for the position for several seconds. This worked pretty well, as the initial result would only be vaguely in the right place and then after a few seconds the position would become more accurate and the listing of nearby stops would start to look more sensible.

Look and feel

Originally I’d planned to mimic the look and feel of the iPhone’s built-in weather app. The app is built around a similar concept – you search for bus stops and add them to a list of stops that you can flick through. I tried to get a nice animated sliding action working, but kept on running into trouble on the device itself. In Safari and Firefox the animation all worked ok, but I suspect there’s an issue on the iPhone when you are dragging an area that limits redrawing. In the end I had to ditch the flip/swipe gesture too – interfering with the touch events again seemed to cause some rendering issues on occasion. So instead simply clicking on the left or right of the timetable moves between pages. It’s a slightly less complicated way of doing things, so there’s less that can go wrong really.


Old Brighton No. 5 Bus

Deployment

As this was a nice simple app (on the back-end at least), I decided to take a little time to create a Fabric file to control deployment. The app is hosted on Webfaction, where I have already setup key-based login for ssh. The app is deployed using mod_wsgi with a daemon process, so that one merely needs to copy over the new code, then “touch” the .wsgi file restart the daemon process. Not exactly tricky to do, but making it a one-button process saves making any mistakes.

To trigger the deployment I run my fab file like this:


fab --user=lilspikey --hosts=lilspikey.webfactional.com deploy:webapps/buses/app/

I actually have that in a shell script, that isn’t checked into git, so that the deployment specific username, host name and remote directory aren’t checked into source control. If I wasn’t using key-based login, I’d need to specify a password and I definitely wouldn’t want to check that into git!

Source code

You can get the source code on github in my buses repository.

Chumby Podcast Client Source Code

Ok so I blogged about a podcatcher/podcast client for the Chumby quite a while ago (August 2008 to be precise). At the time I said I’d tidy things up and release the source code etc. Well that didn’t quite happen, but I figured I might as well release the source code. So seeing as I’ve been using git a lot lately and github is easy to use I thought that’d I’d best just put online what I had.

So you can now find the hybrid Python, Javascript and Flash Chumby postcast client I dubbed chumbycast, on github. I’m providing the code just for the curious – I’ve had at least one request for it, but I’m not really going to detail too much of how it works. Though some of this is outlined in the original post where I mentioned the project.

I am tempted to revive this project a bit, but probably by trying a different approach. I mostly gave up on this as I was creating a UI on the Chumby in Flash and my Flash skills aren’t exactly great, plus I was trying to use mtasc which hampered things further. Which is not to belittle mtasc – it’s just with limited time and no previous Flash experience I wasn’t exactly helping myself by not using some nice friendly Flash IDE.

I’ve since realised that if I ditched the Flash UI I could probably get quite a bit done. The Python httpserver side of things was pretty easy. So if I focussed on that I would then have a few of ways of providing a UI by hijacking existing functionality on the Chumby.

  • The Chumby runs a httpserver on port 8080 when an ipod is plugged in, which the UI uses to access playlists etc. I could mimic this approach and effectively make the podcast client look like an iPod as far as the Chumby’s UI was concerned. By plugging in a USB stick loaded up with the podcast client everything would behave the same as if you had plugged in an iPod.
  • It’s possible to create playlist files and edit the “My Streams” section programmatically. Each podcast subscribed to would create a matching pls or m3u file and be added to the “My Streams” section.
  • Create a javascript/web UI for controlling the playback and subscriptions to podcasts and other podcast management tasks (like removing old files, manually downloading older episodes etc) from another computer. Possibly adding bonjour/zero-conf support so the Chumby can be browsed to easily

I would need to see whether those first two could be made to work for my purpose, but it would make sense to just use the existing UI on the Chumby – rather than creating a new one. The existing chumbycast code already provides a javascript/web UI for controlling the playback on the Chumby. This was originally done so I could remote control the chumby, but also so that I could easily create and debug an API for the Flash UI to use.

The other major missing feature is allowing the podcasts to be paused part way through. The current client does not support this, as the audio files are streamed and not downloaded. So that would be the first change to make. Now that I’ve dug out the code maybe I’ll be inspired to play around some more. I’ve also been listening to a lot more podcasts (thanks to Huffduffer) so it might happen yet.

An iPhone friendly, local storage backed, offline TODO list webapp

A while back I had a go at using the local storage features being added to javascript to create a simple TODO list app. The main focus of the app was getting it all to fit under 5K (5120 bytes), but it was a good test of using a client-side sql database in javascript.

As the app was written for size it didn’t have many frills. It did however work on the iPhone, as it’s version of Safari had support for the openDatabase call needed. However it didn’t look so good and although the TODO list items were stored locally the phone still had to be online to access the host webpage – which kind of undermined some of it’s utility.

With my recent acquisition of an iPhone I thought I’d revisit this TODO app and make it play nicely on the iPhone. In addition to using a client-side SQL database this new version features:

These features mean that if you have add a bookmark to your homescreen for this app, you might as well be running a native app. It looks pretty native, stores data locally, doesn’t require a net connection and even features standard app UI mechanisms. The main giveaway is of course the chrome associated with the Safari browser. Still not bad for some html, css and javascript. Not a total replacement for native Cocoa apps, but it does put the creation of client-side apps for the iPhone into the hands of even more people.

If you are on an iPhone or are running Safari 4.0 you can try out jTODO yourself or watch the video of it in action below:



I won’t rehash the sql-side of things in this post – instead I’d refer you to my original post on the matter.

The use of iui was also fairly straightforward – I’m only using the style-sheets and images. This means that there are no animations involved, but at this stage it seemed excessive to add them in. Perhaps I’ll add some in a future version.

This leaves the offline application cache and getting drag and drop to work.

Offline application cache

The offline application cache is really simple to implement. It’s currently supported by Safari on the iPhone, Safari 4.0 and Firefox 3+.

In my case I want all files to be cached, so I simple specify them in a “manifest” file:


CACHE MANIFEST

# version 1.0.2.37

iui/backButton.png
iui/blueButton.png
iui/cancel.png
iui/grayButton.png
iui/iui-logo-touch-icon.png
iui/iui.css
iui/iui.js
iui/iuix.css
iui/iuix.js
iui/listArrow.png
iui/listArrowSel.png
iui/listGroup.png
iui/loading.gif
iui/pinstripes.png
iui/selection.png
iui/thumb.png
iui/toggle.png
iui/toggleOn.png
iui/toolButton.png
iui/toolbar.png
iui/whiteButton.png

css/todo.css

js/jquery-1.3.2.min.js
js/jquery-ui-1.7.2.custom.min.js
js/todo.js

img/delete.png
img/deleting.png
img/redButton.png
img/handle.png
img/todo-touch-icon.png

This manifest file is then linked to the main html file thus:


<html manifest="todo.manifest">

That’s it. We now have an offline capable web-app. The main issue I had when doing this, was that Safari was very strict about the manifest – any file not mentioned in the manifest would not be loaded (even if it was normally accessible). The other issue of course is that we’ve now introduced another level of caching, so developing can be a bit annoying – as you think you’ve made a change, but then nothing shows up. I often ended up disabling the manifest for a while when debugging and then re-enabled it after things were working again. There’s also a version number in the manifest file, so that it well register as changed – to help refresh the caches after we’ve made a change.

With all this in place the app can be used when no net connection is available (e.g. in Airplane mode).

Drag and drop

The last job when developing this app was to enable drag and drop for re-ordering items. My first go at this worked pretty easily using jQuery UI’s sortable plugin – when running on my mac in Safari:


            page.sortable({
                axis: 'y',
                handle: '.handle',
                update: function(){
                    db.transaction(function(tx) {
                        $('.todo_item').each(function(i,item) {
                            var id = $(item).find(':input[type=checkbox]').attr('id');
                            id = Number(id.substring('todo_checkbox_'.length));
                            tx.executeSql('UPDATE todo SET todo_order=? WHERE id=?', [i, id]);
                        });
                    });
                }
            });

Essentially this is just setup to enabling drag and drop sorting only along the y axis (up and down), using the element with class handle to start the drag. Once the dragging has finished update gets called and I inspect the DOM to work out the current order of the todo_items and update the database accordingly.

That was pretty easy and working really well on the mac. Then of course I thought I’d test it on the iPhone. At that point I realised I’d forgotten that drag and drop doesn’t normally work in Safari in the iPhone. Holding and dragging normally scrolls the entire page – so drag and drop was a bit useless here.

However after a little digging I found someone had figured out how to get drag and drop working on the iPhone, by hijacking the various “touch” events that the phone generates. These work a little differently to the normal mouse events, but with some work can made to do our bidding:


    function handleTouchEvent(event) {
        /* from http://jasonkuhn.net/mobile/jqui/js/jquery.iphoneui.js
         but changed a bit*/
        
        var touches = event.changedTouches;
        var first = touches[0];
        var type = '';
        
        
        // only want to do this for the drag handles
        if ( !first.target || !first.target.className || first.target.className.indexOf("handle") == -1 ) {
            return;
        }
        
        switch(event.type) {
            case 'touchstart':
                type = 'mousedown';
                break;
                
            case 'touchmove':
                type = 'mousemove';
                break;        
                
            case 'touchend':
                type = 'mouseup';
                break;
            
            default:
                return;
        }
        
        var simulatedEvent = document.createEvent('MouseEvent');
        simulatedEvent.initMouseEvent(type, true, true, window, 1, first.screenX, first.screenY, first.clientX, first.clientY, false, false, false, false, 0/*left*/, null);
        
        first.target.dispatchEvent(simulatedEvent);
        
        if ( event.type == 'touchmove' ) {
            event.preventDefault();
        }
    }
    document.addEventListener("touchstart", handleTouchEvent, false);
    document.addEventListener("touchmove", handleTouchEvent, false);
    document.addEventListener("touchend", handleTouchEvent, false);

This registers listeners for the touch events, but only does any extra work if the target of the event has the class “handle”. If that’s the case a simulated mouse event is sent for the first touched item. To stop the page from scrolling when we want to drag instead event.preventDefault() is called just for the touchmove event. This is sufficient to let jquery UI do it’s job and enable drag and drop sorting of the TODO items.

Introducing NoteComb

I’ve been writing an application for my other half in recent months to help her organise observations of the children in her class. I’ve actually written two versions of the app. The first was dubbed “Observertron” and in retrospect was overly complex. After a bit of thought I was able to vastly simplify the app, whilst also making it more useful for other purposes beyond making observations.

The app is called NoteComb and is written in Python using wxPython. It’s essentially a specialised text-editor. The core feature is a grep-like search functionality coupled with the ability to edit text in-place during a search. Lines that don’t contain the search terms get hidden, leaving you free to edit the remaining lines as you want:



I’ve decided to take the time and try to package it up “properly”. So NoteComb is available as Mac and Windows apps, complete with icons etc. In fact to “regular” users it should appear that NoteComb is an app like any other – the fact it’s written using Python is largely incidental.

This packaging works pretty well overall and is pretty seamless once the app is downloaded, but it does tend to make for rather large apps. The Windows version when packaged as an installer runs in at about 4Mb, which isn’t too crazy, but the OS X app packed into a compressed DMG file weighs in at 15Mb! At the end of the day the app does include Python + wxPython + various libraries, so it’s not a surprise really, but I guess I was kind of hoping that I might develop in Python and get everything for free…

This is an early version of NoteComb (version 0.2.1), but it’s core functionality is there. Most of the extra work I’ve done has been on adding those little extra bits that aren’t core to the app, but are generally just expected (e.g. remembering previous window positions, copy/paste, undo/redo etc).

So feel free to download NoteComb and give it a go.

A 5K Java Guitar Tuner

So after writing a 5K Twitter Client, a 5K TODO app and a 5K Fullscreen Text Editor I thought that’d be enough example apps to get things started. However while adding a list of app ideas to the 5K App page I suddenly decide that one of the ideas really appealed. The idea was to write a guitar tuner, that would listen on the computers mic and show the user what pitch it thought the note played was. I’m not much of a guitar player, but when I do play it’s always nice to have a guitar that’s in tune. As I’m unable to tune a guitar by ear I rely on guitar tuners and I always seem to be out of the 9V batteries my electronic tuner needs. Given my laptop is usually nearby, being able to use it as a guitar tuner would be most useful.

Check out the video below or download 5KTuner for yourself (requires Java):



The app is more or less non-interactive. It simply starts up tries to work out what note is being played. It shows a simple slider that moves according to what note is being played and indicates how close to that note you are. In addition the raw frequency (in hz) is shown, as well a chart that shows the autocorrelation function of the input sound. A red line is drawn on this chart when a “valid” note is heard and indicates the wavelength of the overall frequency.

The vital statistics are:

  • Final (obfuscated) app size: 3538 bytes
  • Final (un-obfuscated) app size: 4649 bytes
  • Final Source-code size: 8532 bytes
  • Lines of code: 245

This was my first foray into audio programming and I’m grateful that in the end it wasn’t too tricky. Initially I’d been looking into using the Fast Fourier Transform to work out the frequencies, but the autocorrelation function was much easier. It’s essentially just two for loops and a bit of maths. You listen for a certain amount of audio and then compare the first half of the sample with itself, but shifted. By working out the amount of shift where the difference is smallest you can find the overall frequency of the sample.

It could probably be made more robust, but if you want to see how it works (or improve it) you can download the source code.

A 5K Twitter Client

So as part of preparations for the 5K App I’m writing a few example apps. The first one so far is a 5K Twitter Client, which makes it smaller than the Twitter logo itself (at 5.4K).

Check out the demo video to see it in action or download 5KTwit (requires Java):



The features are:

  • Update twitter status
  • View friends timeline (new tweets checked for every 3 minutes)
  • Easily change logged-in user
  • Count of characters remaining when typing status message

It doesn’t handle viewing direct messages, but I don’t tend to use them. Plus I wanted to make sure the features I had were done fairly well.

Anyway, I thought it would be a good idea to walk through making a 5K app, as there’s a fair bit of stuff just needs to be done to make it feasible. It’s the sort of stuff you don’t normally encounter daily when working on “real” software – unless you’ve been working on mobile or embedded apps. Nowadays most programmers don’t really need to worry about the size of the code they write, so how to make something that’s less that 5K in size isn’t immediately obvious.

The Twitter app (which I’ve lovingly christened “5KTwit”) is written in Java and runs as a standard client-side application. For some good pointers on how to make a Java app that’s really tiny, but packs a punch you should probably start with these Java 4K tips.

Now for a few numbers:

  • Final (obfuscated) app size: 5119 bytes
  • Final (un-obfuscated) app size: 6400 bytes
  • Final Source-code size: 10398 bytes
  • Lines of code: 310

As you can see the difference between the obfuscated and un-obfuscated versions is about 1K, so obviously obfuscation is pretty key. So it’s worth starting of with a good obfuscator. For those going “what’s an obfuscator?” it’s basically a tool to optimise, shrink and scramble your code. Originally they were intended to make code harder to decompile (hence “obfuscate”), but also have the added benefit of shrinking the compiled code to. I used Proguard and that’s probably your best bet too.

Proguard does a pretty good job of shrinking your code, but it’s worth reading up on how it works, so you can get an idea of what code changes you can make to get the most effect. For example Proguard will inline (and remove) small private functions for you (so they effectively have no overhead), but only if they are only called in one place. In general you probably want to use a source-control system/backup changes so you can rollback when a change doesn’t result in smaller obfuscated code.

Here’s the config file I used for Proguard (running on OS X):


-injars      dist/5KTwit.jar
-outjars     obfuscated/5KTwit.jar
-libraryjars <java.home>/../Classes/classes.jar
-optimizationpasses 2
-allowaccessmodification
-keep public class T  {
    public static void main(java.lang.String[]);
}

Proguard does a great job, but it won’t stop you writing code that’s bigger than it needs to be. In order to get the app under 5K I only used two classes (one for the main window and one for the username/password dialog) and one resource file. Each additional file has some overhead associated with it. Not only will each file tend to have some standard header info etc, but even if it was empty it would increase the size of the packaged jar file. So really cutting down on the number of files used makes a big difference. During testing I made a simple HelloWorld application and that was 700 bytes or so in size with a single class file and no real code. Most of that size would have been taken up with this sort of overhead.

As this app used swing for the GUI it also meant I had to be a bit creative about event handling. The norm is to have small event handling classes, but that would have incurred too much overhead. Instead I resorted to tricks like this:


    public void actionPerformed(ActionEvent e) {
        if ( e.getSource() == prefs ) {
            start();
        }
        else {
            String status = this.status.getText();
            if ( status.length() <= MAX_CHAR ) {
                tweetsDoc = loadDocument("http://twitter.com/statuses/update.xml", status);
                loadTweets();
                this.status.setText("");
            }
        }
    }

Where I checked the source of the event to see which button had been pressed, rather than having a separate event listener for each button.

I also had to deal with listening for window resize events, as there was some manual intervention required to make tweets wrap inside the scroll pane. So for that I circumvented the listener approach entirely and overrode processComponentEvent. In this code I had to try to estimate how big the panel containing the tweets would be, when I knew the width I wanted. So I resorted to a bit of nasty hackery and repeatedly set/unset the preferred size a few times so it would settle on the right height…

    protected void processComponentEvent(ComponentEvent e) {
        if ( e != null ) { super.processComponentEvent(e); }
        
        // force width of tweets to match width of window
        for ( int i = 0; i < 3; i++ ) {
            tweets.invalidate();
            tweets.setPreferredSize(null);
            int height = tweets.getPreferredSize().height;
            tweets.setPreferredSize(new Dimension(scrollPane.getViewportBorderBounds().width,height));
            scrollPane.validate();
        }
    }

Whereas in "real" code this would probably be done with a bit more finesse (and a few more classes).

Another tricky spot involved the background thread used to check for new tweets. This ran on another thread (rather than the main GUI thread), so as not to lock up the GUI when it runs (once every three minutes). However updating the GUI had to occur on the GUI thread. Normally this would simply be done by creating a few Runnable tasks, but that would have meant more classes. Instead I end up checking to see whether we're running on the GUI thread and change behaviour if that's the case:

    public void run() {
        if ( SwingUtilities.isEventDispatchThread() ) {
            // update UI as running on the right thread
            loadTweets();
        }
        else {
            while( currentThread == Thread.currentThread() ) {
                String friendsTimelineURL = "http://twitter.com/statuses/friends_timeline.xml?count=200";
                if ( lastTweetID != null ) {
                    friendsTimelineURL += "&since_id=" + lastTweetID;
                }
                tweetsDoc = loadDocument(friendsTimelineURL, null);
                try {
                    // invoke and wait, to run loadTweets()
                    // on awt thread
                    SwingUtilities.invokeAndWait(this);
                    Thread.sleep(1000*60*3); // 3 mins
                }
                catch( Exception e ) {
                    // break loop
                    break;
                }
            }
        }
    }

There are probably quite a few other places where I had to make this sort of "design" choice. It's feasible to do for this little code, but I wouldn't want to do this kind of thing in a big project. Though I guess if it was really needed you could write something that would take nice well written code and convert it to use these sorts of tricks...

Apart from that, most of the rest of the work came down to carefully weighing up which features I wanted and carefully tweaking and re-organising the code to make it smaller. I also found that the single image (for the preference button) was a lot smaller when saved as a gif rather than a png (90 bytes vs ~200 bytes).

Anyway, for 5K it's not a bad little client. I've actually been using it for the past couple of days (as I've been writing it). I doubt it would really survive under the hands of a more active user, but it does the job.