Introduction
As commented in Introducing Emotional Itelligence post, the goals, when are defined usign "feelings" toy models, have only three scoring parameters, three kind of emotional "outputs".The first of them correspond to things you "enjoy" experiencing, like speeding where you enjoyed velocity. Enjoy feelings are basically added together into a general "enjoy feeling" score after each movement the agent does.
But all the three components of a goal, each kind of basic feeling (enjoy, moods and gains) has a reverse, a negative counterpart you need to know and properly manage in your algorithm.
Fear feelings
The dark side of the enjoy feelings are the "fear feelings" or "hate feelings" (they are basically the same) and matematically correspond to goals with a negative fixed value for "enjoy feeling". You can read more on them in this old post about negative goals experiments.
Whatch this video of agents with stronger or lighter negative scorings (crashing and lossing any amount of health is simulated by stoping the agent after a crash -so it doesn't change its ending point- and modeled as a negative enjoy feeling in the actual "emotional" languaje):
Plase note karts are difficult to "suicide" agents. This same case, when applied to rockets, can make them decide to actively crash into the ground in stressing situations as when energy is depleting and it is modeled with negative feelings.
Thecnically, the first thing to note about negative enjoy or "fears" is that they totally break the logic of the algorithm in a couple of basic senses:
1) If we are measuring some kind of entropy gain, having a negative value means your entropy can decrease some times. It is not an entropy of a system, or your system is quantic, and this method can apply to quantum systems bacause second law of thermodinamics can't be applied neither.
2) We totally meshed up the metric we had in the state space. Negative distances are not allowed anywhere.
The results are worst than you could foresee. As in a real psicology, two efects can be easily detected:
A) Being driven by fears means that, in some cases, the fear will make the agent to stop, being unable to decide what to do.
In the algortihm, the agent can find out that moving to anywhere is just too "scaring". The fear feelings are ginving you big negative scores that, when added with the positive ones that could exists, still gives a negative global enjoy feeling. So all options will score negatively.
As staying still, away from the scaring futures you can envision, is not as bad (negative) as moving, finally the option "do nothing" scores the most and is predominant.
B) If the previous case happends when you have a way to suicide, you will do.
When an agent is surrounded by things that score negative, turning around and crashing with the ground is the best option by far, as it always scores zero (you die), so the intelligence will happily suicide.
Even small negative feelings must be avoided. When facing the sufficient desperate situation, this negativiness will be almost all it has, and it means limit situations will make the fear to win and it will commit an nice and "intelligently planed" suicide.
Many videos showing silly ways to crash where produced and then discarded as erroneus changes in the code, but indeed they were only fears being introduced into the scoring feelings. I don't use negative enjoy feelings any more, only for "educational purpouses", as they produce neurotic and suicide intelligences.
How to code them
If you still want to use them, you need to deal with negatives. I do like this in my code (but don't use it, so may be is not the right solution).
Remember we used before as the global enjoy feeling the squeare root of the sum of the squared enjoys, like in the euclidean distance you use d = sqrt(dx²+dy²), but now dx² can be negative.
Mathematically there is very suggestive way to think of it (call it a joke if you want): if the enjoy feeling dx² is negative, then the quantity being measuring, dx, must be and imaginary quantity!
Being just a mathematical joke, entropically and psicologically accurately reflects the situation: you are imaging something with a negative growth of entropy, so it is only possible in your imagination, not in real life. And psicologically, you are irrationally running away from a danger that doesn't really exists, it is purely imaginary. Take it as only a naming curiosity.
So we need to operate those negative enjoys as being pure imaginary numbers, so when squared, they turn into negative numbers.
It means (dx²+dy²) in the distance exampe, can be negative, and squared root of negatives are not possible. Well, only if you consider this sum as being, again, an pure imaginary number instead of a real negative one.
Using code and assuming sqrt() is the squared root, abs() the absolute value and sign() is -1 for negatives and 1 for positives, then you should first sum for all goals:
Sum = sign(enjoy) * sqr(abs(enjoy))
And finally define the global enjoy feeling again as:
GlobalEnjoy:= sign(Sum) * sqrt(Sum)
Why do the negative feelings even exists then?
I suppose they where neccesary in natural evolution of life, when the intelligences the agents had was too limited. If you can not foresee what will happend if you do that, then a simple strategy is adding a fear felling associated with unknow things.
Think of cocroaches: they can't harm you, but you desperately hate having one near. The "fear" is profund, you can't scape it, because it is an ancient instinc: avoid them because they run fast and randomly, and you can not know if they will be on your leg in half a second. Fear for uncertainly is the simpliest solution.
So may be they where neccesary once, and may be they still saves lives, but for an artificial intelligence it is much better if we discard this early beta of feelings and focus on the nice working ones.
No comments:
Post a Comment