In WaterWorld the agent, a blue circle, must navigate around the world capturing green circles while avoiding red ones.

After capture a circle it will respawn in a random location as either red or green. The game is over if all the green circles have been captured.

Valid Actions

Up, down, left and right apply thrusters to the agent. It adds velocity to the agent which decays over time.

Terminal states (game_over)

The game ends when all the green circles have been captured by the agent.


For each green circle captured the agent receives a positive reward of +1; while hitting a red circle causes a negative reward of -1.

class ple.games.waterworld.WaterWorld(width=48, height=48, num_creeps=3)[source]

Based Karpthy’s WaterWorld in REINFORCEjs.


width : int

Screen width.

height : int

Screen height, recommended to be same dimension as width.

num_creeps : int (default: 3)

The number of creeps on the screen at once.



  • player x position.
  • player y position.
  • player x velocity.
  • player y velocity.
  • player distance to each creep