Game Server LAG Issues (Tested on High Spec Server Machine)

Hello,

So for the last couple of months i've been playing around with the ability to have local servers and noticed a big performance increase when this is used which was great. However there is still the issue with regards to "Slow Mo" even while hosting it on my desktop which is a pretty high end machine. I looked online and there are a million and a half threads complaining about slow games.

In the end I decided to really find out what the problem was and test it on our companies server that I use for basically crunching numbers for research data. Keep in mind this thing has about 128GB of Ram and a 6 core Intel Xeon E5-2630V2 processor with a bunch of drives on RAID0 which can read at about 800MB/s and write at around 600MB/s so resources really shouldn't be a problem! I also shutdown all the various tasks it was running so the machine was essentially idle.

I then ran a local server off this machine and hooked 3 of us up to it over LAN (1 Gbps of course). We started a 3 V 6 Absurd Bots match to see how far we could push it on a system with about 5ish planets (2 big ones and 3 small). It all started off with 0 lag however roughly 10ish minutes in the game fell of a cliff and became horendously slow. I checked the resource usage on the server and it seemed to be only using about 5GB ish and refused to use anymore despite the fact they were available?! We stopped the game and restarted but a similar thing happened again.

I looked everywhere and I couldn't seem to find any sort of bottleneck with anything running running anywhere near it's capacity. Unless someone can suggest a reason why this would be I can only surmise that the problem is somewhere in the game engine itself?

Any Comments or solutions?

Many Thanks

DoctorZeus

If your server doesn't crunch numbers in <1ms then you should know that available resources will always be a problem

Cola_Colin did some server testing, the results of which you may be interested in.

As far as I know, the main bottleneck is the lack of multithreading. The main game thread only uses a single core, though some of the easier tasks are delegated to other threads. You'll probably see one core at 100% usage when you hit the limit.

Haha luckily the data isn't on demanded for use as soon as it's processed yet.

Hmm interesting, seems that no one has managed to actually get a large game working at a decent speed.

Take a look at your CPU usage per core and you'll find it's always CPU bound.

https://www.cpubenchmark.net/singleThread.html

Ahh ok: Hmm that's annoying. I'll try and do another test this weekend and take a look. Is there any plans in the works by the devs to fix this or is it unlikely? I'm guessing it would be a fairly major re-write of the game!

Under any normal game I would say fair enough but unfortunately part of the joy of PA is being able to do battle on great scale!

It would be not just a re-write, but a full research project with unknown success-chance. Multithreading games like PA in a way that scales really well is relatively uncharted territory.
No way Uber will do that. Too high costs, too little reward.

It's already handling multithreading relatively well. If my memory serves me well it's using something around 4-6 full cpu cores, depending on how many players you have. So they have multiple threads working on various things.
However the bottleneck is the game simulation, which is one of those threads.

Yeah the only successfully multi threaded RTS simulation I've seen is Zervers port of the Spring RTS engine. It is much faster than the default and scales well with multiple cores. The issue however is that it's based on some unknown binary blob and isn't open source, so it was removed from the main project on licencing grounds.

I think PA is fairly well threaded overall- it uses around 3 threads for rendering, 1 thread for simulations, and multiple threads to handle network traffic and so on. It can easily fully utilize an 8 thread machine like an i7. That said the sim limitation does kick in eventually.What is also worth keeping in mind is that PA can handle as many if not more independent units than any of the previous games in the genre.

The most likely method of multi threading the simulation would be what was initially suggested- running a simulation per planet instead of per game. That helps avoid many pitfalls, although it isn't without it's own edge cases that would need addressing (what happened when units move between planets either by teleporter or orbital? What happens when one planet destroys another and so on). In the end, Uber made this game on a really tight budget and the work involved in getting this up and running was too much I guess.