Home Research Publications Software Robots Personal Stuff


Software available for download under various open source licences:

The Pinball Domain

The Pinball domain is a fairly challenging 4-dimensional continuous and dynamic reinforcement learning domain. The goal is to maneuver the blue ball into the red hole, while avoiding (or using, since the ball is dynamic and collisions are elastic) the obstacles. The dynamics of the ball and the presence of obstacles result in a domain with sharp discontinuities, and the location and shape of the obstacles can be specified so you can make the domain as hard or as easy as you want.

The source code is in Java, and includes full documentation, an RL-Glue interface, and GUI programs for editing obstacle configurations, viewing saved trajectories, etc. (GPL)


The Fourier Basis

I have contributed source code for an RL-Glue-compatible epsilon-greedy Sarsa(λ) agent that uses the Fourier Basis to the RL-Library project.

The source code and project summary are available on the RL Library community wiki. The code is in Java and is fully documented. (Apache License)

The Fourier Basis is described in:
gdk at cs dot umass dot edu