Kite Physics Demo

It’s more of a plaything than anything else. It did give me a chance to try out writing some physics code again, something I don’t get much of a chance to do during web development. Though of course advances in Javascript speed and the Canvas tag may change that in the future.

Originally I made use of the keyboard to control the figure and the kite, but I switched to using the mouse as it makes interacting with the scene more pleasing. There’s a certain joy to being able to grab the kite to move it around or shift the wind direction by nudging a cloud.

It’s all written in Java, using maven to build and package. The code is available in my kite repo on guthub.

There’s about one third of a physics engine in the code. There’s no real collision detection for example. Most of the physics code is for handling constraints and basic forces. For the constraints I made use of the Gauss-Seidel technique, as outlined in Advanced Character Physics by Thomas Jakobsen. This was quite a revelation to me. Previously I’d experimented with rigid body dynamics (see my 2D racing game circa 2001), but the Gauss-Seidel method seemed much more intuitive. I found it much easier to think of bodies made up of points connected by lines. The “physics” naturally emerges from the way the bodies are then constructed.

The core physics code (in the com.psychicorigami.physics.* package) is fairly well structured. There a bit of compiler placation to (theoretically) allow for the same code to work for 2D and 3D physics. I’ve only actually implemented the 2D side of things, but have tried to use generics to make room for 3D too. When everything is points and lines, moving from 2D to 3D is not that tricky. Whereas for rigid body dynamics this can mean quite a big change in representation.

The code in the default package is used for the specifics of the demo itself, so is a bit messy really. Lots of tweaking and prodding was needed to get things how I wanted.

The physics for the kite and wind interaction were aided massively by this guide to forces on a kite. Particularly useful for understanding how to tweak the rigging, centre of gravity and centre of pressure of the kite to make it behave in a pleasing – if not entirely realistic – manner.

At some point I hope to revisit the physics code here. I’ve always had a wish to create something like Karl Sims “blocky creatures”, but lacked the specific technical knowledge. Know that I have a technique for simulating complex constraints in an intuitive manner maybe I’ll manage it yet.

At work I do a lot of Java web development, which normally involves compiling code, packaging into a .war file and deploying it to Tomcat (running locally). I make use of the maven tomcat plugin, so it’s just a case of calling maven tomcat:redeploy. However it still takes tens of seconds (or more if there are tests to run). For quick tweaks of css it’s nice to be able to short-circuit this process.

Tomcat unpacks the .war file to another directory after the app has been deployed. All the .jsp pages, css and javascript files can be edited in this directory and changes can be seen immediately. However getting into the habit of editing these files within this directory is usually a bad idea, as the files will get overwritten at the next deployment.

We’ve been using compass lately for css and it has a handy command:

    compass watch

That monitors .sass files for changes, then re-compiles them to css as they change, so you can see changes quickly (without needing to manually run compass each time).

So I thought I could do something similar for the editable files within the war file. So I created watch-cp.py. It simply monitors a set of files and/or directories for changes and copies over files that have changed to a source file or directory. To provide a bit of feedback it prints out when it spots a changed file, but beyond that it’s pretty brute-force in it’s approach.

watch-cp.py works in a very similar way to the cp command, but without as many options. For example:

    # recursively copy files from src/main/webapp to tomcat
    watch-cp.py -r src/main/webapp/* ~/tomcat/webapps/mywebapp

This is great as it means I can edit my sass files, have them compiled to css by compass, then copied over to tomcat without needing to intervene. It takes a second sometimes, but it’s fast enough for the most part.

Feel free to fork the gist of watch-cp.py on github.

After playing around with iPhone javascript apps, I thought it would be interesting to try and create a javascript app for the bus times. I had some specific ideas about how I wanted the app to work. I was also keen on trying out some more “HTML 5″ tech. Plus I do like things to be (at least in principle) cross-platform, so this has a chance of working ok on Android phones (though I’ve not tested it on one).

There are still a few rough edges, but you can try out the app yourself at:

• http://buses.psychicorigami.com/

It will prompt you to add a stop. If you type in the text field, you’ll get a list of stops that match. If your browser supports geo-location, you’ll also see a “Find nearby” button, which will list the nearest few stops.

I had the pleasure of demoing this app at the Async Javascript show’n'tell last month. I quickly outlined how it worked, which involves the following:

• Offline App Cache – to make startup quick (key when you don’t want to miss your bus)
• Client-side Local Storage – to record which bus stops you add
• Geo-location – so you can find nearby stops
• JQuery – for front end
• Bottle.py – for back end
• BeautifulSoup – for scraping bus times

Getting the data

The buses.co.uk site has an API, which I used to scrape all of the bus stops in Brighton. I chose to scrape the update data from the website itself though, as I couldn’t get the API to give me the combined services for each stop (so you can see when the next 7 and 81 are coming for example). There are typically two stops for each name – one either side of the road. The updates are for both directions, so I actually merged the stops together – which makes searching easier.

All of the stop data was stored in a single SQLite database – mostly for convenience. The database is used in a read-only method in the actual running app. I’m sure I could use something a bit better, particularly when it comes to the distance queries. Currently, as there are a relatively small number of stops I’m using a brute force method to find the nearest stops, but I’m sure with the use of a proper spatial index this could be done a lot more efficiently. If this was scaled up to work for all stops in the UK, then that would be a necessity.

Geo-location

Initially the geo-location was pretty straightforward – I simply used the getCurrentPosition function and I got a latitude and longitude back. Testing in Firefox and Safari on the mac gave fairly reasonable results. However on the iPhone itself I started to notice it wasn’t very accurate. Sometimes it was out by several 100 metres, meaning that stops I was standing next to were sometimes not showing up at all! I had noticed, in the past, that the built-in map app sometimes has a bit of “lag” in getting an accurate position. So I switched to using watchPosition and polling for the position for several seconds. This worked pretty well, as the initial result would only be vaguely in the right place and then after a few seconds the position would become more accurate and the listing of nearby stops would start to look more sensible.

Look and feel

Originally I’d planned to mimic the look and feel of the iPhone’s built-in weather app. The app is built around a similar concept – you search for bus stops and add them to a list of stops that you can flick through. I tried to get a nice animated sliding action working, but kept on running into trouble on the device itself. In Safari and Firefox the animation all worked ok, but I suspect there’s an issue on the iPhone when you are dragging an area that limits redrawing. In the end I had to ditch the flip/swipe gesture too – interfering with the touch events again seemed to cause some rendering issues on occasion. So instead simply clicking on the left or right of the timetable moves between pages. It’s a slightly less complicated way of doing things, so there’s less that can go wrong really.

Deployment

As this was a nice simple app (on the back-end at least), I decided to take a little time to create a Fabric file to control deployment. The app is hosted on Webfaction, where I have already setup key-based login for ssh. The app is deployed using mod_wsgi with a daemon process, so that one merely needs to copy over the new code, then “touch” the .wsgi file restart the daemon process. Not exactly tricky to do, but making it a one-button process saves making any mistakes.

To trigger the deployment I run my fab file like this:

fab --user=lilspikey --hosts=lilspikey.webfactional.com deploy:webapps/buses/app/

I actually have that in a shell script, that isn’t checked into git, so that the deployment specific username, host name and remote directory aren’t checked into source control. If I wasn’t using key-based login, I’d need to specify a password and I definitely wouldn’t want to check that into git!

Source code

You can get the source code on github in my buses repository.

First you’ll want Python on the Chumby. At the time the latest version already compiled for the Chumby is Python 2.6, so it’s pretty up-to-date.

Once you’ve got Python installed and on a USB stick you’ll also want to download PySerial – I picked the latest version PySerial-2.5-rc2.

With PySerial expanded and on the USB stick (alongside Python) you’ll want to put the USB stick in the Chumby and connect to it via SSH.

Change to the directory for the USB stick (e.g. cd /mnt/usb).

You should see (at least) two directories, one for Python and one for PySerial:

chumby:/mnt/usb-EC5C-3D0A# ls -l
drwxr-xr-x    6 root     root         4096 Jun 26 21:15 pyserial-2.5-rc2
drwxrwxrwx    4 root     root         4096 Jan 10 13:51 python2.6-chumby

You’re first instinct may be to try and install PySerial via the usual call to python setup.py install. However this appears not to work.

All is not lost though – just manually copy the relevant directory (serial) from PySerial to the python site-packages directory, e.g.:

cp -r pyserial-2.5-rc2/serial python2.6-chumby/lib/python2.6/site-packages/

Know to check that’s worked open a python prompt and try to import the serial library:

chumby:/mnt/usb-EC5C-3D0A# python2.6-chumby/bin/python
Python 2.6.2 (r262:71600, May 23 2009, 22:28:43)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>>

If you don’t see any errors (as above) then everything has installed ok.

Next step is to try out connecting an Arduino.

So first off you’ll want to ensure the Arduino has a program that can read from the serial port – to show that everything is working ok. In my case I picked my Morse Code program. This will read bytes from the serial port and toggle the built-in LED (on pin 13), so it’s handy for verifying the serial port is working.

So now you have a program loaded on the Arduino, connect it to the Chumby. The Arduino should get enough power from the Chumby to start up ok.

You need to work out which serial port the Arduino is using on the Chumby. List the tty’s in /dev and pick the most likely looking one (should have USB in it’s name):

chumby:/mnt/usb-EC5C-3D0A# ls /dev/tty*
/dev/tty      /dev/ttyS00   /dev/ttyS02   /dev/ttyS1    /dev/ttyS3
/dev/ttyS0    /dev/ttyS01   /dev/ttyS03   /dev/ttyS2    /dev/ttyUSB0

On my Chumby the serial port was /dev/ttyUSB0.

Now open up a python prompt again and try talking to the Arduino. You’ll need to configure the baud-rate of the serial port to match the program on the Chumby. In my case this was 9600.

chumby:/mnt/usb-EC5C-3D0A# python2.6-chumby/bin/python
Python 2.6.2 (r262:71600, May 23 2009, 22:28:43)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>> ser = serial.Serial('/dev/ttyUSB0', 9600, timeout=1)
>>> ser.write('sos')
3

If you are using the same morse code program you should see the LED blink out ‘DOT-DOT-DOT DASH-DASH-DASH DOT-DOT-DOT’ – confirming that the serial port works.

This is really quite handy, as the Chumby is a small low-power Linux server. Coupled with Python this means it’s really easy to get any Arduino based project online, without needing a “normal” computer constantly running. Not quite as self-contained as using an Ethernet Shield, but the Chumby is fairly small, so it and an Arduino will easily sit on a window ledge (for example).

The voltage from the potentiometer is read via analogRead, then is mapped to a freqency to output on the buzzer. To make things sound nicer the frequencies are limited to the twelve “well tempered” notes. As the potentiometer is turned the note played changes. There’s a little bit of smoothing (legato) in between note transitions too (so it sounds a bit more theremin-like).

All code for this Arduino “theremin” is available on github, as part of my Arduino Sketches project.

Here’s a demo of the tone generation in action:

So what to make first? Well morse code seemed like a good starting point. I’d written an app to output morse on the capslock LED before and the starter kit had LEDs, so it seeded like a good starting point. On closer inspection there was also a piezoelectric buzzer – so even better it was possible to generate a tone too!

Getting the LED to turn on and off was pretty easy and it forms the standard “hello world” program for the Arduino. Getting the buzzer to generate a tone was marginally trickier, but not that hard and basically just consisted of turning it on and off quick enough to cause a vibration and therefore make a sound. With the help of this guide to making the Arduino play the Can-Can I also had details of how to wire the circuit correctly (in particular adding in a suitable resistor to avoid overloading the AVR chip).

Using the starter kits breadboard, jumper wires, a 330 Ohm resistor, an LED and the piezoelectric buzzer I put together this:

The buzzer had it’s positive pin wired in to pin 4 on the Arduino and the LED was in turn wired in to pin 13. In the picture above, I arbitrarily used yellow jumper leads for “ground” and green for positive. So one green lead goes to each pin (4 and 13) and only one yellow lead goes to the “ground” on the Arduino.

Initially I hard-coded the message the Arduino was going to output, but having to re-compile each time to change the message seemed a little tedious. So I decided to get the Arduino to read from the serial port, store the characters it read in, then output the relevant morse. To achieve this the program has a 512 byte buffer that it fills up with data read from the serial port. If the buffer is full it signals this by writing back a -1 down the serial port – otherwise it echos back what it has read in. As it plays the morse it moves along the buffer looking for the next character. Once all characters are output it resets the buffer. The program is actually able to read from the serial port whilst it is turning the LED on/off and generating tones. It’s only limited by the size of the buffer it is using and the fact that it can read faster from the serial port than it can output morse code.

Most of the work in the code revolved around trying to both read from the serial port and generate tones. If the morse was known ahead of time, it would be pretty easy to right several for-loops to run through each character generating tones until we were finished. However the program would not start reading from the serial port until it had finished outputting the morse. So apart from using some basic state-machine style antics to keep track of which character we were on and whether we were currently during a dot, dash or silent period the major a-ha moment for me was realizing how to easily generate a consistent tones without using delays in the code. In previous code I’d seen for generating tones, the code looked like:

for (long i=0; i < numCycles; i++){
    digitalWrite(targetPin,HIGH); // write the buzzer pin high to push out the diaphram
    delayMicroseconds(delayValue); // wait for the calculated delay value
    digitalWrite(targetPin,LOW); // write the buzzer pin low to pull back the diaphram
    delayMicroseconds(delayValue); // wait againf or the calculated delay value
}

This is nice and easy to follow, but all those calls to delayMicroseconds mean you are hanging around waiting for the air to move to make a sound.

As I wasn't too concerned with the particular fidelity of the sound from the buzzer I opted for the following instead:

void morse_pulse_on() {
  digitalWrite(LED_PIN, HIGH);
  unsigned long period = micros() % buzzer_delay;
  digitalWrite(BUZZER_PIN, period < (buzzer_delay>>1)? HIGH : LOW);
}

This was called every time through the loop() function if a sound needed to be played. As there is no delay in calling it, one could then immediately perform other tasks, such as checking for new serial data or turning the LED on and off. It's quite simple real - the modulus operator (%) is used to get the sub-divide the current time in microseconds into a number between 0 and buzzer_delay (a value pre-calculated to let us generate the right frequency) representing the current "period". If we are halfway through this period then we set the buzzer to HIGH, otherwise we set it to LOW. By repeatedly calling morse_pulse_on we end up with (in this case) a glorious 700Hz tone.

All code for this Arduino morse code program is available on github, as part of my Arduino Sketches project. Hopefully I'll add more code to that project as I get a chance to delve deeper into what the Arduino can do.

Here's a demo of the whole thing in action:

UPDATE: not long after finishing this, I discovered the tone and noTone functions. These produce a better tone than the technique outlined above. Though they do involve use of one of the interrupts on the arduino, that's normally used for PWM on pins 3 and 11 - not that it's a problem for this project.

So you can now find the hybrid Python, Javascript and Flash Chumby postcast client I dubbed chumbycast, on github. I’m providing the code just for the curious – I’ve had at least one request for it, but I’m not really going to detail too much of how it works. Though some of this is outlined in the original post where I mentioned the project.

I am tempted to revive this project a bit, but probably by trying a different approach. I mostly gave up on this as I was creating a UI on the Chumby in Flash and my Flash skills aren’t exactly great, plus I was trying to use mtasc which hampered things further. Which is not to belittle mtasc – it’s just with limited time and no previous Flash experience I wasn’t exactly helping myself by not using some nice friendly Flash IDE.

I’ve since realised that if I ditched the Flash UI I could probably get quite a bit done. The Python httpserver side of things was pretty easy. So if I focussed on that I would then have a few of ways of providing a UI by hijacking existing functionality on the Chumby.

• The Chumby runs a httpserver on port 8080 when an ipod is plugged in, which the UI uses to access playlists etc. I could mimic this approach and effectively make the podcast client look like an iPod as far as the Chumby’s UI was concerned. By plugging in a USB stick loaded up with the podcast client everything would behave the same as if you had plugged in an iPod.
• It’s possible to create playlist files and edit the “My Streams” section programmatically. Each podcast subscribed to would create a matching pls or m3u file and be added to the “My Streams” section.
• Create a javascript/web UI for controlling the playback and subscriptions to podcasts and other podcast management tasks (like removing old files, manually downloading older episodes etc) from another computer. Possibly adding bonjour/zero-conf support so the Chumby can be browsed to easily

I would need to see whether those first two could be made to work for my purpose, but it would make sense to just use the existing UI on the Chumby – rather than creating a new one. The existing chumbycast code already provides a javascript/web UI for controlling the playback on the Chumby. This was originally done so I could remote control the chumby, but also so that I could easily create and debug an API for the Flash UI to use.

The other major missing feature is allowing the podcasts to be paused part way through. The current client does not support this, as the audio files are streamed and not downloaded. So that would be the first change to make. Now that I’ve dug out the code maybe I’ll be inspired to play around some more. I’ve also been listening to a lot more podcasts (thanks to Huffduffer) so it might happen yet.

I thought I’d have a go at writing up the instructions for how I did this – complete with template for making your own:

How to sew a felt horse

It’s pretty easy really. As long as you can sew some backstitch you should be able to make this easily. If not then it might not be the worst first project to try either!

A liberal dose of select_related helped. However that was only useful in the cases where a ForeignKey needed to be pre-fetched.

In the case however there was a page that was selecting a set of objects that had tags. The tags for each object were being displayed along side a link to the main object. Given that the initial query returned over three hundred objects, this meant the page was performing another three hundred (plus) queries to fetch the individual tags for each object! Now we could cache the page and that’s was indeed what was being done. The trouble however was when the cache expired. It also made things painful when developing – as I’d typically want to disable caching whilst I’m making changes to pages frequently.

I came up with a specific solution for this project – to perform the initial query, then a second query to fetch *all* of the other tags in one go. The results of the second query could then be stitched into the original results, to make them available in a handy manor within the page’s template.

I took the original idea and made it re-usable and am now releasing that code as Django Batch Select. There are examples of how to use it over at github, but it works a bit like this:

>>> entries = Entry.objects.batch_select('tags').all()
>>> entry = entries[0]
>>> print entry.tags_all
[<Tag: Tag object>]

It’s a very early release – after the result of only a few hours coding, so use with care. It does have 100% test code coverage at the moment and I’m reasonable confident about it working. Please try it out and let me know whether it works for you.

From there I moved on to creating a pair of robins to sit on top of the wedding cake:

The robins are made out of felt and stuffed using bits of cut-up cotton sheet (as that was all I had to hand at the time – hence also the green cotton). The legs are simply silver wire stitched into place inside, with the red-breast covering up the extra stitches needed. The eyes are just small black beads (from Beads Unlimited in Brighton, as was the silver wire).

I felt they turned out really well, so seeing as they didn’t take too long (a couple of hours each) we decided it’d be nice to make more felt creatures (following an autumn theme) for table centrepieces. Initially we planned on one creature per table, then two and so with some extra help from my bride-to-be we ended up with twenty eight felt hedgehogs:

Again the hedgehogs were made of felt. This time they were stuffed with wadding which made things easier. The spines were glued on as “fringed” bits of felt, with a special widows-peak shaped piece for the head. This time the eyes and nose were just small bits of black felt, glued in place – so as to be marginally more child-friendly. Each hedgehog body consists of three pieces of felt – two sides and a smaller bottom piece. The bottom piece makes them stand fairly levelly and also gives them a more rounded look.

Somewhere during this time I’ve also had several friends children’s first birthdays to contend with. So seeing as I was in a felt creature production mood I turned my hand to three felt toys:

I realised the other day that all thirty three of these were made during a roughly two month period – meaning one felt creation every two days! Not a bad rate of productivity I guess.