These future scorings are made up by adding the effects of a set of different "motivations" or goals the agent has, like in "I love speeding", "I care about energy" or "I care about health", measured over the future's path, step by step, like in a path integral aproximation.
Being able to define the right motivations set for an agent, along with a proper way to calculate the different effects those motivation could have on every step the agent takes, and mix them together to get the correct future's score, is ultimately what I am looking for and the life motiv of this blog.
I have used quite a big number of simple goal schemas to simulate some interesting intelligent behaviours, like the ones previously presented on this blog, but I am far from happy with them.
Basically they failed to show me the real optimum behaviour I was specting from them. Some had weird problems on limit situations, like if you are running out of fuel and out of energy at the same time. But there was also an ugly limitation on the length of the futures it was able to handle that really made them not so generally usable.
In the try-and-error process, I could found some interesting patterns on the goal scoring schemas that worked better: they always avoided the possibility of negative scores, and most of them defined a real metric on the space of states of the system (when you assign a score to a future connecting the initial and final states of the system, you are inducing a metric on the state space, one that tells you how interesting a future looks for your agent).
In some point of the procces, I felt that the key idea for going from a goal less inteligence, based on a pure physical principle of "future entropy production maximization" (as described in my first posts or in the Alexander Wisner Gross' paper) to a stable and rich human-like intelligence was trying with some realistic modeling of the "feelings" themselves and how they affect our own internal scoring systems, and then try to base everything else on them.
Plase note that, when I name some different parts of the involved equations like actual feelings, they are representing perfectly defined mathematical functions, not any kind of pseudo scientific concepts. It just seemed to me that the parts I needed to use in the mathematical mix were incredible similar (in the form and the effects on the resulting behaviours) with concepts usually related to human psicology. Over the time I have naturally turned into naming them as "enjoy", "fear" or "moods". Take this as a pure mnemotecnic trick or as a clue of some deeper connection, anyhow it will help you to better visualize the algorithm.
The introduction of those "feeling" models supposes a bost in the different "motivations" I am able to directly simulate now. It is quite easy now to model all basic goals I was using before and they work much better, but it also allowed me to model new kind of interesting and useful motivations.
Before going on, here you have a video showing the full potential of the emotional intelligence in streesing situations, like 50 asteroids falling over your head:
In the following posts I will recapitulate the actual working of this new "emotional" version 2.0 of the entropic intelligence algorithm in full detail. I will not mention things that didn't work in the past (they have been deleted from the V2.0 code to make it clearer) and follow the straight line between my first simplier model and the actual "emotional" one.
The topics of those posts are:
Common sense
We will examine the initial form of the AI that corresponds to Alex Wisner Gross' paper. The psicology it represent is no psicology at all, just pure "common sense" in action, and correspond with the case where all futures score just one.
Enjoy feelings
We will jump to a much better version of the AI where the distance raced on a future was the key. We will define it as a "enjoy" feeling and discuss the correct from it should be calculated to have an "entropic sense". I will then comment on some other possible examples of "enjoy feelings" and how to mix up several "enjoy feelings" on the future's final score.
Fear/Hate feelings
They correspond to negative enjoy feelings and represent the irrational fears as opposed to the ones based on real risks. It is generally a bad idea to add them into the mix, as the resulting psicology will include hidden suicide tendecies and, on some situations, the AI will panic and block. They will also negatively affect the quality of the resulting metric on the state space of the system, so I have actually banned them from my implementation, even if they are correctly used if you define such a goal.
The mood feelings
Mood feelings change the "enjoy" scoring you calculated from the exisiting mix of "enjoy feelings" by multiplying it by a factor. If it is lower that one it will correspond to "negative reinforcement" like when you are walking quite fast but you can't fully enjoy the speed because you have a peeble on one shoe or you are really exahusted. In the other hand, when it is bigger than one, it is modeling a "positive reinforcement", like when you are an intrepid explorer walking into the unknow or you are in love and prefer walking along with your beloved one.
Loose feelings
Losing your life can be scaring and must be avoided some how, but if it will happend 100 years in your future you don't really have to worry about it. Loose feelings are weird, their effects fade out in the future and apparently break some laws about entropy and metrics (all fears do after all), but they are really needed -by now- if you are serious concerned about the agent's IQs.
Gain feelings
They are opposed to the loose feelings and correspond to great hings that could happend to you at some point in the future. Like in the loose feelings, the importance of the gain tend to zero as it happend in a more distant point in the future. They can simulate the effect of landing a damaged rocket to have it repaired and fill its health up to 100%, or model a rewarding feeling when you avoid other player's dead, for instance.
This will close my presentation of the version 2 of the entropic intelligence, the emotional entropic intelligence. In some point I will release a version 2.0 (note from the future: I did it on 1st december 2014) of the app and its source code, internally revamped to reflect this emotional model and the new namings, with examples of motivations based on all those new feelings.
There will be a future version 3 in witch the three motivations' parameters (the "strength" of the feeling, for instance, I will discuss them on the "loose feeling" post) will be automatically adjusted to the optimum values for each situation in real time, boosting perfomance in a completely new way (at a computational cost) if they behave as I spect.
No comments:
Post a Comment