PTE Stream: 73687, now with 100% more offline mode

Well, the offline server has been in the works for quite a while and was pretty near to completion when they announced HR. By the time they start work on HR it could be expected to be done, so no harm in announcing it with the Kickstarter, but they didn't. So either HR doesn't get off-line server for reasons I don't fully comprehend (DRM) or it will be announced later for the sake of making a big announcement which will get more backers. (I'm hoping it's the latter of course along with Linux support. )

Oh right, our invalid token is applying to a single page instead of a whole texture layer. The current limit is 16.

TL;DR: Not easily. It's somewhere on our backlog of features we would like to implement someday, first to increase the limit to 32, and eventually getting rid of it entirely.

Full version:
The virtual texturing operates by processing a feedback buffer to figure out which virtual texture pages (and therefore which virtual textures) are visible. The way that data is packed, it has 12 bits each for the x & y components, 4 bits for the mip level, and 4 bits for the virtual texturing id for each pixel in the feedback buffer. 12 + 12 + 8 = 32bits, which is just the right size for GPU frame buffers. Unfortunately, you can only fit 16 VT ids in 4 bits.

There are a couple possible solutions here. One is that we could reduce/shuffle some bits around to give a little more space to the VT id. Since the time when it was initially implemented, we dropped off the small mips of the virtual texturing anyway, so it's likely we could get up to 32 by taking that new limit into account, and using 3 bits for the mip and 5 bits for the id.

Getting over 32 would require processing the feedback buffer in multiple passes. That would slow frame rates down a bit when more planets than the limit are on-screen, and also increase how complicated that code is. (And considering it's already quite complicated, that's not something to be taken lightly.)

Even if we do increase that limit to 32 or get rid of it entirely, however, there are still other issues. Once you change the limit, you also run into UI issues. Everything has been laid out assuming that limit, so we would need to make a UI pass for handling an arbitrary number of planets.

And each planet takes up memory. Right now, that is a very large amount of memory. Eventually that number will be smaller, but we are not there yet. Some people's machines have plenty of available memory, but others' do not, and we would really like to provide a good experience for those users.

(The 15 thing came from the fact that we need a buffer value for "not VT", so we don't try to virtual texture the skybox & etc. We're using the ID 0xFFFFFFFF for that, which means the bottom-right page of the 16th texture is never going to get mapped. Initially, I wrote it to consider the entire texture ID as invalid, and set the maximum to 15. In practice, that is very hard to ever encounter, so I updated it to 16, and then forgot that I had done so over the last year or so.)

I tested this PTE today. Worked pretty well. I have just one minor issue. Offline was disabled by default. My specs are i5 2500, 8Gb ram, nvidia 760, Ubuntu. I think offline was disabled because Ubuntu sais I have 7.8Gb ram. Maybe some ram is assigned to the integrated gpu even if it is not used. Anyway, if this is the case, maybe it is an idea to lower the ram specs by a few hundred megs? This is a very minor issue for me, since I simply changed the settings.

on the question of cpu usage:
on the linux based pastats.com server I can run a 8500 dox spread over 4 planets on area patrol with 1 sim fps. top shows 130% cpu on the process, so it does use a little more than 1 core. Though not really much more. The server has an i5, so even with the slow crawling server on it the webpage is unaffected

Ah, yeah. I had the same thing with an i7 (also 8GB RAM, NVidia and Ubuntu).

Okay, I cannot get the server.exe to run on my windows 8.1.
Even trying to run it from console produces a crash, but at lest it gives me a bit more information.

At the same time trying to connect to the server running on another computer (I got a old win7x64 laptop) does not work either. So I guess I am the 1% that has issues with the offline mode.

Here is what the commandline gives me (running under full admin rights):
C:\Spiele\Steam\SteamApps\common\Planetary Annihilation\bin_x64>server.exe --all
ow-game-config --allow-lan --headless
[19:27:31.000] INFO Mounting C:\Spiele\Steam\SteamApps\common\Planetary Annihila
tion\media\ as /
[19:27:31.000] INFO SDL: Built w/ v2.0.3, linked w/ v2.0.3
[19:27:31.000] INFO Loaded buildID 2014100873687
[19:27:31.016] WARN Crash reporting disabled.
[19:27:31.016] INFO Starting background thread pool with 3 threads
[19:27:31.016] INFO GameServerImpl::resetModUpdateAuthToken: Auth token reset to
"0f9da9de-6e78-aa41-ac5d-6ff3ba9d87a4"

uncaught C++ exception of type "class std::runtime_error"
what(): "listen() failed"

0x1403da6a3
0x140326855
UnhandledExceptionFilter+0x142
memset+0xbbf7
_C_specific_handler+0x87
_chkstk+0x9d
RtlRaiseException+0xedb
RtlRaiseException+0x18d
RaiseException+0x6c
CxxThrowException+0xd4
0x1402e6520
0x1400399c8
0x14001ad1b
0x1402c8bf7
0x1402c4246
0x140673e1e
0x1406736bb
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 2632:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x1403092b0
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 3060:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x1403092b0
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 3328:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x140334ef7
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 3368:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x140334ef7
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 3896:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x140334ef7
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 3952:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x1403092b0
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

Thread id 5344:
NtWaitForSingleObject+0xa
WaitForSingleObjectEx+0x98
CRT_RTC_INITW+0x3d4
Concurrency::details::_Condition_variable::wait+0x70
Call_onceEx+0xad
0x140334ef7
0x14020d52d
std::_Pad::_Release+0x6c
beginthreadex+0x107
endthreadex+0x192
BaseThreadInitThunk+0xd
RtlUserThreadStart+0x1d

C:\Spiele\Steam\SteamApps\common\Planetary Annihilation\bin_x64>

First of all thanks for explanation! Nice to know this limit wasn't enforced with no reason. Still would be cool if limits like this one won't be hardcoded so advanced users would able to experiment with game in some crazy ways.

This is true, but it's possible to solve it by modding. Even 14-16 planets don't work good on small screens anyway.

I totally understand and support idea to improve experience for players with low RAM, but I don't think it's valid argument case of planet limit and this is why:

• Game with just one large earth planet can use 3.5-4.5GB RAM.
  1300 meters, 0 water level, 60 height range.
• Game with 16 moons of 200 meters only use 1.3GB.

Considering PA client itself use about 800MB RAM for assets all these 16 planets use 500-600MB while one big earth use 2.8-3.5GB.

Can you check your windows firewall settings, and make sure server.exe is not getting blocked? (And try disabling antivirus software, as well.)

Firewall was off for another test. Disabeled Virus scanner now, but no improovement.
I found out about the " --allow-crash-api" switch, but it does not appear to do anything. Or at least I do not know where it is supposed to write the logs.

When PA recommends that I have 4 cores and 8gb of RAM for offline, does that mean 2 physical cores or does it include the 4 logical cores on my i7 4700MQ?

This option is NOT something you need for sure. It's only valid if you use HTTP API (--http option).

To be clear, we definitely want to remove the limit, and we want to encourage experimentation. In general, we avoid hard limits. At one point, however, overflowing that hard limit would have caused a crash elsewhere, and it saves a lot of debugging time to know we were leaking virtual textures, or overlapping our planet generation. Since that time, the virtual texturing system has gotten a lot more robust, and it doesn't crash any more, but it still helps us make changes to other systems and know that we didn't break stuff in the process.

We'll get to it, but it's going to take time.

BTW, if you do work around that limit using a debugger, the output from the virtual texturing will be completely wrong. The moon system picture posted before may look correct in the zoomed out view, but if you look closer, you will find that the texturing is broken on 10 of the moons.

I just don't like hardcoded stuff in general because it's always make me sad when I trying to mod something.

Yeah it's true, I noticed that once I tried to look closer.

Has anybody figured out what kind of js environment the server uses? I want it to post data somewhere, but I fail to do so because none of the typical tools for that are defined and I have no idea if I can add them or how.
Also does the thing have a debugger or something similar? So far it's pretty hard to use.

I saw some V8 stuff when scanning strings.

I think figuring out an interactive console will be priority when I start digging into things. Surely someone has set something like that up for Node, but I don't know if their environment is Node-like enough to use it.

Do you know if it is possible to do HTTP posts with it?

Not yet.

I like the yet part.
So there is currently no way to get the beacon of a game out of the lobby.js to my own application?

EDIT:
I guess I'll build a little nodejs script that takes control of the server and parses the log output, so I can do:
console.log(<json>); to "leak" data to the outside world.

Not from within the server. You can get it from a PA client. Or you could write an app to connect to the server directly. The beacon listener is on UDP port 8192.