I finally got around to finishing GEB. I think I bought a copy 10-15 years ago and either gave up or got distracted during a couple of attempts to finish it; at my birthday last year I vowed to complete the thing, and I kept my promise a few months back.

Many of Hofstadter’s then-novel ideas (e.g. perception as the basis of cognition) seem more mainstream nowadays, but I got interested in some projects from his lab (FARG, the Fluid Analogies Research Group) around cognitively plausible architectures operating in microdomains.

A follow-up book, Fluid Concepts and Creative Analogies, goes into these in more detail, and I thought I’d have a go at reimplementing of one of their projects, initially for giggles but then to play with some of the underlying ideas a bit more.

Numbo is a simple number puzzle solver (UK readers: it’s the number puzzle from Countdown): given a set of 5 integers, apply mathematical operations to them to make a 6th target value. Classically a computer scientist might treat this as a search problem, but FARG was more interested in cognitive plausibility. You can find Daniel Defays’ original paper on Numbo here. I’m sure I’ve seen an OCR scan of the original source code at some point, but can’t find it now.

Numbo uses an architecture they called Copycat (from another project, called err Copycat, which investigated analogies like “abc is to pqr as bcd is to…?”). Numbo’s version of the Copycat architecture has 3 parts:

1. The Pnet, a network of static information: numbers, operations and basic calculations of a kind a child might learn by rote e.g. 2 + 2 = 4, 7 * 10 = 70.
2. The coderack, a store of priority-weighted processes (called codelets) which is sampled from probabilistically, and thus run in a slightly random order, effectively emulating parallelism even when run serially. Codelets operate on the cytoplasm and can create other codelets. They are independent processes and do not communicate with one another directly.
3. The cytoplasm, a working memory which is operated on by codelets sampled from the coderack, and stores current theories and constituent numbers.

My implementation is written in Clojure, partly because the original was Common Lisp but more because I enjoy writing Clojure. You can find it here; it can be run from the command-line, or there’s a GUI to help visualize a single run, plus a script which tries to solve each of 10 puzzles 100 times, which I’m using to track the effectiveness of changes I make. Because there’s a lot of random processes underlying Numbo, different runs can and do produce different results - or sometimes no results.

I’ve /just/ managed to get Numbo to a point where it solves 7/10 of the original sample puzzles, which in the spirit of “launching early enough to be embarrassed by my first version” seems like the right time for a push to GitHub. I have some ideas about the remaining 3 and other improvements - I suspect that speedy puzzle-solving is influenced by some of the decisions around decay rates of nodes in the Pnet and cytoplasm. There are a few other jumping-off points for future work:

• The contents of the Pnet are important; I’m interested in working out how you might construct a viable Pnet from e.g. educational materials;
• I wonder how codelets might be evolved rather than hand-coded;
• I wonder how transferable the architecture might be to a new micro domain.

Lots to think about! The process of writing this was also quite fun. It’s the largest program I’ve written since my Master’s dissertation on superoptimization (code, dissertation, paper). A few observations:

• I wrote the cytoplasm 3 times: the first one was too simple, the second too complex (but I got to learn about Clojure zippers) and the third one OK so far - but I think I’ve just found a bug which points to a error in my data model (around how secondary targets are treated)
• Writing a GUI to examine the state of runs was a good move: it’s saved me so a ton of time in debugging, as I can visually run through the evolving state of the cytoplasm, identify odd points by eye and dive in to debug them. Swing is still a PITA, even with Seesaw wrapping it.
• After starting to read Paul Graham’s On Lisp, I regret not having pulled key abstractions beyond library functions into macros. My codelets contain a ton of boilerplate and whilst I pulled out some functions to handle probabilistic sampling, I feel like this could be made into a part of the language a bit more.
• In a lovely demonstration of Conway’s Law, the needs of real life meant I had to write Numbo in tiny non-cooperating chunks which made individual sense. I left messages to future-me on what to do next but otherwise frequently lost state when I returned to work on it. Given all this, I became a set of tiny discrete processes acting towards a greater goal…

My attempt to resurrect this site in 2017 didn’t go very far. I had intended to start writing more about my experiments with the Donkey car, but they hit a bit of a wall when I started in X and had real work to do there, also parenting. I spent a year in X working for what’s now been revealed as the Everyday Robot project, and then returned to my previous team in Google Research, where I’m still working today.

In the last 3 years I think I resurrected the Donkey car 2 or 3 times, and even managed a short habit of heading to one of the DeepRacing get-togethers on a Saturday, or to DIY Robocars races; but it turns out that working full time and then taking half a day out at weekends to go play with cars isn’t totally compatible with Life. After a short attempt to get a track set up in my attic (tl;dr it’s big but needs to be bigger to be useful), I threw in the towel… for now.

Apologies for the spare look. I wanted to get something up fast, if I get around to it I’ll play with look and feel later…

So it’s been a little while since I posted here - nearly 5 years. I’ve had very little to say in public, work and life have been a lot of fun, and I’ve not missed writing.

A lot has changed in that time: I joined Google in London, turned 40, got married, relocated to San Francisco, and at the end of last year our daughter arrived.

A lot has stayed the same too. When I look at my writing I’m happy to see some continuity with my work at Google. There I ended up tugging on a thread of “user interfaces for local intelligence” which manifested across many products I worked on: Auto Awesome Movies, the Android launcher, Smart Text Selection, and Federated Learning (not an exhaustive list). I’ve particularly enjoyed working with a mix of designers, engineers and researchers.

And to hammer home the point about continuity, my last post here touched on the tortoises of W. Grey Walter; fittingly, in August I’m moving to X to work in robotics. More interfaces, more intelligence, a new domain.

This is why I’ve resurrected this site: to get some hands-on experience I’ve been playing at the edges of the DIY Robocars movement, building a Donkey and trying to get it running reliably around The Panhandle, a nearby park. I want somewhere to share what I’m doing; this site will do nicely, thank you.

The Science Museum in London is currently showing an exhibition about Alan Turing. Kate and I wandered up there on Saturday. I found the exhibition itself a little superficial - which isn't so surprising given the breadth of material the curators had to draw on between his personal life and death, contributions to computing, the war effort and Bletchley park, and his work on morphogenesis.

But there were two little gems in there which I focused on: the first, one of the tortoises of W. Grey Walter: beautiful and tremendously simplistic devices which exhibit eerily animalistic behaviours. And secondly, a bombsight computer from a Lancaster bomber, on which I was chuffed to discover the manufacturer's mark of Sperry. For it was a Sperry machine which D P Henry used to create his spirograph...

So, down the right hand side of this page you'll see a block, marked "AdSense". That'll be me playing with advertising; I'm interesting in learning a bit more about how online advertising works from a practical perspective, and I like learning about stuff by doing stuff.

So there'll be a small ad - or it might be a large ad in future, who knows? - running there for a little while at least. I'll be donating all income from it to the WWF, to one or several of their efforts around preserving big cats.