Solved atari games

The list of environments from OpenAI that we have already played so far is steadily growing, so I had to made a list to keep track of them. Here we keep and share this growing list, along with it scorings and how it compares with the "second-best" algorithm in the OpenAI gym.

For the interested people: We base all on our work on the "Causal Entropic Forces" by Alexander Wissner-Gross and apply the ideas outlined in the G.A.S. algorithm. We are not actually learning in any way, so all games are independent from each other and first-ever played game is as godd as the 100th game.

First, we list the already finished environments. They include 100 games played and an official scoring in OpenAI gym as the average of those 100 games:

1. Atari - MsPacman-ram-v0: average score of 11.5k vs 9.1k (x1.2)

This was the first env. to be finished and uploaded, so it represent our first official record. We decided to use the "ram" version (instead of the image version) because it is irrelevant for our algorithm but not for a more standard approach, so we had an extra punch.

The main issue here was a dead Pacman takes about 15 frames to be noticeable on screen (there is a short animation) so you need to think in advance at least those 15 frames (ticks) in order to start detecting death.

OpenAI first record!

We have just submitted our first official scoring on OpenAI gym for the atari game "MsPackMan-ram-v0" based on RAM (so you do not see the screen image, instead you "see" a 128 KB RAM dump).

Our just submitted algorithm "Fractal AI" played 100 consecutive games -the minimum allowed for an official scoring- and get an average score for the best 10 games of 11543 +/- 492, well above previous record of  9106 +/- 143, so we are actually #1 on this particular atari game:

Fractal optimising, a first paper

The fractal "family" of algorithms actually started as a very naïve optimising algorithm: after all, intelligence is just about maximising a certain "utility function", so they are quite related.

Once the fractal AI was done, the optimisation facet was again re-visited with a much more promissing results to later abandon it again.

And finally, with the help of our friend José María Amigó from the Miguel Hernandez University, we wrote an article about this fractal algorithm we named "GAS" (namely for "General Algorithmic Search" but it was actually for the capitals on our names, Guillem, Amigó and Sergio) and compared it against other similar ones out there (Basin hopping, Descent evolution and Cuckoo search).

Fractal VS Pack-Man

Last week my friend Guillem adapted the fractal AI for the OpenAI Atari games (OpenAI is a "gym" for AIs), in particular he focused on "Ms Pack Man", an environment labeled as "unsolved" as I write this.

Yesterday the work was almost done and the first videos came out of the pipeline and, to be honest, the results have stonished me, it worked out far beyond my always-optimistic high spectations.

So here is the video that made me so happy yesterday: