I guess PA Stats never saw this many viewers on such big games before. It didn't like it. This is what the server thought of your games from 4:25 onwards: Ran out of memory retrieving query results. java.lang.OutOfMemoryError: GC overhead limit exceeded java.lang.OutOfMemoryError: Java heap space APPARENT DEADLOCK!!! java.lang.NullPointerException: null 05:52:45.336 [pool-5-thread-10] ERROR net.liftweb.actor.ActorLogger - Actor threw an exception java.lang.OutOfMemoryError: Java heap space around 6:00 it seems to recover, probably from lower load. Will do a restart just in case. Not entirely sure why even loading the static files failed. Will do some tests, maybe I can do something to make it more stable in case of server troubles. My guess is that for some requests to the mod files the server went "okay I got this .................................................................. ah yeah erm........... <never answers again>" and PA was like "don't worry buddy I'll wait for you." Great job doing this between 4:30 and 6:00 my time.
16GB. However the jvm is only allocated 850mb. I thought that was plenty. I think I am gonna rethink this a bit. I never had many of these large games to test how the server reacts and what it needs for them.
Hmm now that you mention it, the server in fact wrote such a file, I added the parameter long ago already. Didn't think of looking for it, as the server managed to stay alive somehow. Guess only some threads died or something. Will have a look at it. EDIT: Hmm nothing unexpected in it. The mechanism that transmits the data update to live viewers probably was overloaded by the extreme amount of json data it had to transmit to viewers, especially viewers who opened the page after the game had run for a while already. Then the server run into issues where it had to garbage collect too much and started to fall behind. EDIT 2: lol the game in question transmits 26 MEGABYTES of json for all the "unit created/destroyed events". So whenever somebody opens the page the server creates a 26mb string. Actually probably it builds it up and modifies it, so there may be copies around. I guess that explains the overloaded garbage collectors. The 38 players of that game transmitted 194254 events to the game that make up this data.