Another AI-Dev Question for Sorian - 'hive mind'?

Making the AI too smart would probably stop being fun I guess, but I just watched the video Sorian linked to in the other AI discussion. In it, he mentions that the neural nets are trained once, for an hour or so, and then learn no more once they're in-game.

I'm thinking to myself:

a) Since the server will have thousands of rounds stored in full detail, could these be used to continually train and improve the AI?

b) Each player has a different style; building on the training each time it plays against a specific player would be interesting. If I always turtle and build lots of levelers, it could expand a lot, build appropriate defences etc...

c) There was discussion of 'tasks' vs 'orders' in an earlier thread, where an order might involve the task of heading over to the destination first, then beginning a build task. I'm wondering if there is another layer of abstraction there such as a 'mission' for a group of units. Perhaps the mission is to blow up the catapult that has been causing grief, or to prepare x bombers for a commander snipe. If this higher level chunking exists, it'd be great to see some of it made visible in replays or to spectators (or commentators for screencasts). Eg 'sending 50 ants to destroy 3x power generators' ... 'enemy force too large - retreating!!'

d) Do influence maps also differentiate by land, sea etc? If there is lots of AA I figure the air unit's influence map would be looking very dangerous but this shouldn't bother tanks.

I've always thought AI programming would be the most fun of all, so you're a lucky man Sorian

Different difficulties.

That would be really cool.

The ability to train the AI locally would be nice.

IN another AI thread, Sorian mentioned this interesting titbit:

I suspect (and hope) that as well as some pre-set AI's of different difficulty and personality, Sorian might expose some of the parameters of the AI in a small "custom AI" panel. Thus, players may have the option to alter a few general values such as numbers relating to how risk-averse the AI may be, how much it can cheat with intel/resources, and how aggressive the AI is. Of course, with a lot of machine learning techniques, it's possible to end up with a series of interconnected parameters where it's unclear what any one parameter does by itself. Either way, I look forwards to seeing what Sorian has in store for us.

@deadmg
Uh, training does not have the effect you think it has. It's just a back propagating neural network, it's NOT an evolutionary algorithm.

Training a neural network means letting the network discover how the input relates to the output. The way Sorian uses the network, you have the output of various analysis methods as inputs, and the outputs are "weightings" for the various options.
Given a meaningful input (this is the most important thing!!!), the so "trained" network will already give you a universal solution which is (should) be valid for every possible scenario. Once the training is complete, it already "knows" everything it CAN know.
You can't teach it new tricks or make it adapt better, because if it had flaws, then these would be either in the network itself or in the input, but not in the "knowledge" of the neural network.

That means, if the network acts improperly, than this is the fault of a broken implementation. This CAN NOT be fixed with additional training as the additional training would still yield a nonfunctional network.

If sorian actually designs a learning AI for an RTS, I will love him forever.

"Difficulty levels" at that stage would imply the AI deliberately making mistakes and a subroutine that makes the AI forget about things.

+1!
If you watch the vid Sorian talks about tweeking the input values, etc to adjust difficulty, and not hamstringing the process of learning itself. Might be splitting hairs but it sounds like a way better method. And one that leaves the possibility of a constantly learning ai as quite possible, imo.

But that's what's cool about that! The core learning ability is still there, intact!

Thats still not "learning". He just tweaks the recommendations made by the neural network by hand to make the AI favor a certain more or less agressive / defensive / smart option, despite this option being not the optimal solution, based on the experience of the neural network.

That means, he can make the AI worse on purpose to simulate sub-par tendencies in the behavior, but he can't make it better this easily as the default setting is the best the AI could ever do.

I agree, but the core learning ABILITY is untouched.