The PinBall domain is a 4-dimensional continuous
test domain for Reinforcement Learning (RL) algorithms.
A small blue ball is placed in an arena where it must be manouvered
into a red hole. The ball is dynamic (drag coefficient 0.995), so its
state is described by four variables: x, y, x velocity and y velocity.
Collisions
with obstacles are fully elastic and cause the ball to
bounce, so rather than merely avoiding obstacles the agent
may choose to use them to efficiently reach the hole.
There
are five primitive actions: adding or subtracting a small
force to x velocity or y velocity (which incurs a reward of -5 per
action), or
leaving them unchanged (which incurs a reward of -1 per
timestep); reaching the goal obtains a reward of 10,000.
The PinBall domain is an interesting test domain for RL
algorithms
because its dynamic aspects,
sharp discontinuities, and extended dynamic control characteristics
make it difficult for control and for function
approximation -- much more difficult than a simple navigation
task, or other common benchmarks tasks (e.g., Mountain Car and Acrobot).
The domain is written in Java (full source code and JavaDoc
is available under the GPL) and reads in
a configuration file
that specifies the size and locations of the ball and target, and
descriptions of the obstacles. Users can therefore create their
own customized versions of the domain to suit their own needs,
for which a GUI configuration editor is provided. Below
are two example configurations which are available from the
downloads section.
|
|
|
A simple configuration
|
A slightly harder configuration
|
Features: