Optimization wonderland extravaganza

Only what the client doesn't know. Here check out this blog post about chronocam from forrestthewoods.
http://forrestthewoods.ghost.io/the-tech-of-planetary-annihilation-chronocam/

Ugh dear god lol

That's nothing! Server motherboards support up to four processors. Put a 12-core Opteron in each slot and weep!

Note why I said desktop

IBM has had 5GHz server CPUs for years now, only it costs much more than my parents cars.

Edit: besides, you can get coprocessor hotswap blades, adding a nearly infinite number of CPUs.

Not really. Blade servers are usually a single, separate system per blade with the exception of some exotic NUMA architecture machines.

Fun factoid: the largest core count on a single, conventional architecture system currently available that I know of would be Oracle's SPARC T5-8 server, which contains up to 8 T5 processors with 16 cores each (512 cores total) and 4 TB of RAM.

You'd love OMERESA's servers in Ohio. They are absurd really. And I didn't know anybody actually used that much RAM.

Thanks for this amazing update!! This is one of those highly specialized postings that novices (like me) have to read incredibly slow to understand 60% of what is said. From what I understood, I will anxiously await this implementation in future updates. Thanks for everything Varrak, you da' man!!!

This is absolutely great! Really interesting read and the current optimisation (client and server) is really my only gripe with the game.

Thanks Varrak! Always enjoy reading these!

Well... looks like you've been busy indeed. This will help many things . But I don't believe it will help the sever lag. Is that being addressed?

varrak, u may not relies this, but the community loves these kinda nitty gritty techy updates. It shows u guys are awesome and worth the attention O.O

You allready did a great job with the current PTE

Really hoping the devs will be making full use of cpu's such as the 8 core AMD FX 8320 in the near future. Looking forward to seeing the performance that the game will gain by using all 8 cores.

On the topic of many core CPUs, Intel released the Xeon Phi which is a 50 core cpu in a single die. It can't (currently) replace a "normal" CPU and instead acts as a co-processor, but all 50 cores are essentially x86 cpus. Uber has some past experience working with it when it was known as the Intel Larabee as was originally intended as a fully programmable graphics card.

Going that direction more, graphics cards have for a number of years now being trying to leverage their designs for general computing. OpenCL and CUDA are C like languages created for writing highly parallel tasks on those GPUs. The NVidia Tesla GPUs are specifically built for that purpose, with the latest ones having 2880 cores(!) and no video out. AMD had a similar family of GPUs named Firestream that were discontinued 2 years ago, but their latest high end consumer video card, the AMD Radeon R9 290, has 44 compute cores.

Many modern CPUs now have graphics chips embedded in them, like almost the entire line of consumer Intel CPUs, and the AMD APUs. AMD likes to boast that it's APUs have 12 compute cores, in reference to 4 CPU cores and 8 GPU cores. But recent Intel Core i7s can claim 44 compute cores from 4 CPU cores and 40 GPU cores. The PS4 and Xbox One are custom AMD APUs with 8 CPU cores and 18 GPU cores on the PS4 and 12 GPU cores on the Xbox One.

Xbox 360 was 3 cores and the PS4 was 8, so games have had some experience doing multi-threading. However the PS4's extra cores were extremely limited and you had to program for them explicitly, and PC CPUs so far outclassed the consoles once multi-core became common that just running code on a single thread was often faster than the hyper optimized multithreaded console code. Now that the new consoles are 8 modern CPU cores and several GPU compute cores I expect more and more games on the PC to take better advantage of many cores.

Sim performance (the actual game server as most people think about it) and server performance (what the client connects to and handles sending the game history data) are both being worked on. The latest build actually has some of those improvement implemented, though some other changes have also slowed down the sim.

Hey varrak, I know many parameters are involved in getting the game handle more and more units.
As far as you're concerned, and without revealing what's considered as being classified by Uber )), do you have any idea about how many units could be handled by the client, provided there are no limitations coming from server/network side ? Or do you have any idea about what's the current target for middle class gaming computer ? will the current multi threading model be able to scale over years, provided the client has more available core/threads (CPU/GPU) and memory ? Or are there inherent bottlenecks that will finally prevent the game from scaling infinetly (again considering server and bandwidth are non limiting).

By the way, I know you're involved on the client side. But regarding the server side, isn't the multi threading model going to be a bottleneck ? Isn't the one planet/one thread model going to finally be the true bottleneck ?
I mean, if people keep playing on a single planet, then, even if you're able to scale a lot on the client side, if the server is not able to sustain the same workload on a single thread (same number of units), we'll have a bottleneck. As we use to say in the rdbms performance world, "serialization is death". And here, having a single thread for a single planet is definitly likely to limit the client side.

Last question : I know you've done tons of great stuff with bgolus regarding client optimization, but who is in charge of server optimizations ?

finally, and this is for all Uber staff, I do really appreciate that even before final release, the game performance is so much good, compared to what Supcom used to be. It obviously makes all the difference when the same team is in charge of the same project from beginning to the end, and without pressure from an editor.

Ah you've updated, so i will ask same question to you : Isn't the one planet/one thread multi threading model going to be the real bottleneck ? What's the current target in terms of units/planet ? Can we imagine that the one thead/one planet model change over years ? I guess this would be a huge re-work

There are a number of bottlenecks for the sim performance, but here are three big ones:
AI, Pathing, and Recon.

A game without any AI can handle way more units than a game with AI. No hard numbers, but someone in the office recently equated the AI to "several average players' worth of units by mid game", which is an exceedingly nebulous number if there ever was one.
Next up is pathing, and this feeds somewhat into what makes AI slow. It's not the actual neural network that's really slow, it's the requests it makes about the world from the pathing system. That and major changes being done to the flow field generation and high level A* pathing that are now slower than they used to be because most of it was rewritten from scratch after @Elitron got tired of falling down the rabbit hole of "edge case" issues caused by the original implementation that kept turning into the common case. There's likely a ton of optimization work to do there again now, and I know @linike860 and @forrestthewoods have both been working on assisting with that.
Recon isn't the only remaining thing that takes up time, but it's definitely one that causes problems for planets with trees, as well as a major bottleneck for extreme numbers of units. An average sized earth type planet might have tens if not hundreds of thousands of trees, and right now everything that has vision has to find out if they can see each one, just as each unit has to check to see if it can see each other unit, structure, or projectile. There's already been a lot of work here on multiple fronts to try to reduce the overhead. You only need to know if any one unit can see another, not how many units can see it, so once a thing is seen by an army, it's removed from the list of things to check against for that army.

As for who is working on server perf, the short answer is almost everyone not @varrak and myself.

this is where the theory of OpenGL is "just as good" and "can do all the same things as" DirectX and Mantle falls apart :/ .

I'm stocked about the multithreading, especially since I run a 6-core, 12-thread CPU. If i understand correctly though : not being able to split up the work onto multiple GPU threads means being able to split CPU tasks a slight bit less.

Is it truly impossible to add a library to OpenGL that would allow multi-threading on GPU? (I know I'm talking crazy talk here but I only suggest this because I am in the presence of the Uber, the best group of coders on the planet )

Is the server side stuff as hard to parallelize as the client side stuff?