Python ctypes, "Illegal instruction" and garbage collected callbacks


I am currently using ctypes (lets you call native code using only Python - no C) for a project on OS X and ran into a little problem today:

"Illegal instruction"

Not exactly very specific.

Initially I thought this was due to some threading issues, but after ensuring I only had one thread running I tracked it down a bit further. The code was dying somewhere inside a call to a native function. Hmmm.

That native function (CFRunLoopRunInMode from OS X's CoreFoundation Library) happens to invoke a call-back function I defined earlier on. To pass a Python function to a native function you must wrap it in a CFUNCTYPE object. I realised that I was being a bit lazy and doing the wrapping, as I passed the Python function to a native function (setInterruptReportHandlerCallback). I therefore was not keeping a reference to the wrapped function. So it was free to be garbage collected!

The underlying Python function was obviously ok, but the CFUNCTYPE wrapper was getting garbage collected, leading to the (occasional) "Illegal Instruction".

# what I was doing (this resulted in "Illegal instruction" on occasion)
nativeFunction(MYCFUNC_WRAPPER(mycallback))

# changed to (to keep reference to wrapper)
wrapper=MYCFUNC_WRAPPER(mycallback)
nativeFunction(wrapper)

Thankfully for me it was an easy fix. Annoying that the code had been working ok initially - the problem only showed up as I started stress-testing the program a bit.

All in all I am finding ctypes to be a wonderful library, but it (obviously) can involve some of the same issues you would encounter when coding in C/C++. In fact I think I would find ctypes a bit hard going if I hadn't already had several years of experience in C/C++.