Neural network AI

Hello Sorian,
I just watched your video presentation about the SC2 neural networks and thought it was pretty cool! As I understood it, you train 3-layer classifiers to determine actions for platoons during the game, given a range of predetermined parameters.
Do you also use regularization or some form of feature selection on these parameters during training?
For longer time-scale overall strategies of the AI, have you ever looked into the possibility of using architectures like Echo State Networks/Liquid State Machines to have networks with memory, learning time series statistics instead of "merely" classification problems and instant input-output mappings? I would imagine that type of learning may be useful if you already have symbolic representations of actions and would like to perform forecasts of player actions or decide on longer time-scale strategies that take into account a history of previous actions.
The training procedure is simple linear regression, since you only learn a linear read-out layer. Memory and nonlinear feature expansion of the input statistics are performed in a fixed but randomly initialized "reservoir" of nonlinear nodes.
There are also extensions of this framework using more analytic reservoirs like Volterra-series operators or single delay-coupled nodes (a Mackey-Glass system, for example). A lot of research in this field is available for a period of about 10 years (for example http://minds.jacobs-university.de/pubs, publications about Echo State Networks by Herbert Jäger).
I imagine this type of network as statistical model could also be combined with a Bayesian learning paradigm to account for lack of information, given that the AI only has information available that it scouted (I assume, since it is not cheating?).
I do not know how feasible such ideas are for an actual implementation in a game since I have absolutely no idea in what way the in-game dynamics are formalized, but I'd be very interested to hear to what extend you guys (or gaming industry in general) are thinking about/already using these concepts. To me it sounds like a super interesting area of application for this type of statistical modeling?

I'm greatly looking forward to the game, you guys are awesome!

Johannes

Problem with all you are saying (I'm not an expert in the field, so I do understand it all only barely, btw) that it's probably not much needed for level of "dumb" AI (that do not trying to predict player actions and do not building complex strategies into far future).

And Sorian said in video that they probably won't be looking into that level of AI, at least for release version. Reason is simple - everything that you've told could be very handly in solving given problems, but that require a special research - is it useful? How it could be implemented? How fast it could be? How much resources it will take? How much better game will be with such smart AI (you can't tell it without trying actually)? Does it worth it? How to embed all this logic into game and not mess the code up?

It's a lot of work for feature that is not much needed actually. Given limited Uber's resources they probably don't have time for such academic research. But I'm fairly sure that Sorian will make his AI replaceable and moddable, so you may try to!

I don't know most of the answers You are probably right about the limited resources, that's always a problem, I guess.
But I got the impression that many people find the idea of having an AI that can somehow adapt to the user across individual games quite fascinating. In the machine learning community such techniques are well-known and I find it likely that game developers are also looking into the matter, so I thought I'd ask...
I think enough academic research has been done in this direction and Bayesian learning paradigms to produce some robust and efficient methods that could sift through the "veil of abstract theory" into industry. I know that postal services make use of such things for automatic recognition of hand-written characters, for example. And I was told Toyota has been using this kind of stuff to control fuel intake (?) dynamically. And since PA is going to feature other academically inspired things like flow fields and neural networks already...

In terms of implementation and computational resources it would not differ much from the standard neural networks that are already used. The training procedure may even be simpler than a backpropagation algorithm. I think the main problem is conceptual, deciding on what sort of information to act, or how to categorize opponent actions. At least for me, i have not really thought about these things before, but PA is incredibly inspiring in this regard.

So then, how does a modern RTS AI work on a more long-term and strategic level? in terms of deciding what to produce and what units to combine and where to attack and expand? You are saying there is not much happening here? Is an AI mostly running scripted global actions and then uses e.g. neural networks only to make local decisions? There must be some information it collects about the opponent that influences its behavior, no?

I think current AIs appear "dumb" in a sense because they feature a rigid pattern which you can exploit once you know it. However, maybe it is not that hard to make it only slightly adaptive, so the exploits become less effective after a while. A very minor adaptation may lead to a great improvement in gaming experience and prolong the fun, in particular when combined with a random scenario generator idea like the Galactic War. And such a minor adaptation may well be feasible to implement with some machine learning technique.

I don't know about you guys, but I would find such prospects super exciting.

It's pretty much a direct cultural translation of the same idea, but as it is used in Germany.

(Well im from Sweden, and we use hold thumbs here too.)