Thursday, 15 February 2018

Fractal AI goes public!

Today we are glad to announce that we are finally releasing the "Fractal AI" artificial intelligence framework to the public!

This first release includes:

We have modified Fractal AI a little so now it is more powerful than ever, in fact we have been able to beat a lot of the actual records (from state-of-the-art neural network like DQN or A3C) with about 1000 samples per strep (and remember, no learning, no adaptation to any game, same code for all of them).

We are specially proud of beating even some absolute human world records, but hey, it was going to happen anyhow!


This Ice Hockey game scored 64... one goal every 3 seconds or more! Human absolute record is 36, and previous AIs scored 10.5 the best, and I have an inmortal centipede running on my PC for a week, it is scoring 1,241,988 right now and has 6 extra lifes... human record is 1,301.000 and best AIs scored 25.000, so tomorrow I will kill the game if it is still alive.



If you decide to test it by yourself, we kindly ask you that, should you test a new game or get a specially high score, please report back as asked in the readme.

The parameters are few and well explained in the hit-and-run notebook example included: Name of the game, and 3 simple numbers to adjust time horizon, reaction time, and number of paths. The CPU needed at any moment auto adjusts -but you can and should hard-limit it- so you can beat some games like boxing in real time.

The idea is making truly open science by asking the people not only to reproduce but to generate our results, and to send a video of the record so it can get certified as being legit. The score board will cite the users so you can be part of it if you feel like.

Ah! The base code of the fractal is generic, it is not atari only, it works in any other problem you can adapt it to, continuous or discrete...

I copy-paste here the abstract and a very preliminary score table from the great intro Guillem has done for the release:

Abstract

Fractal AI is a theory for general artificial intelligence. It allows to derive new mathematical tools that constitute the foundations for a new kind of stochastic calculus, by modelling information using cellular automaton-like structures instead of smooth functions.
In this repository we are presenting a new Agent, derived from the first principles of the theory, which is capable of solving Atari games several orders of magnitude more efficiently than other similar techniques, like Monte Carlo Tree Search [1].
The code provided shows how it is now possible to beat some of the current state of the art benchmarks on Atari games, using less than 1000 samples to calculate each one of the actions when MCTS uses 3 Million samples. Among other things, Fractal AI makes it possible to generate a huge database of top performing examples with very little amount of computation required, transforming Reinforcement Learning into a supervised problem.
The algorithm presented is capable of solving the exploration vs exploitation dilemma, while maintaining control over any aspect of the behavior of the Agent. From a general approach, new techniques presented here have direct applications to other areas such as: Non-equilibrium thermodynamics, chemistry, quantum physics, economics, information theory, and non-linear control theory.


SoTA = State of the art, the best AI to date.
Human = score by a game tester after 2h of playing.
Absolute record = human world record.











2 comments:

  1. Excelente trabajo. Muchas gracias.

    ReplyDelete
    Replies
    1. Las gracias me tocaría dartelas yo a tí pero... de nada!

      Delete