PTE Stream: 73687, now with 100% more offline mode

Discussion in 'Support!' started by garat, October 9, 2014.

  1. DeathByDenim

    DeathByDenim Post Master General

    Messages:
    4,326
    Likes Received:
    2,118
    Well, the offline server has been in the works for quite a while and was pretty near to completion when they announced HR. By the time they start work on HR it could be expected to be done, so no harm in announcing it with the Kickstarter, but they didn't. So either HR doesn't get off-line server for reasons I don't fully comprehend (DRM) or it will be announced later for the sake of making a big announcement which will get more backers. (I'm hoping it's the latter of course along with Linux support. :))
    gerii likes this.
  2. pinbender

    pinbender Active Member

    Messages:
    78
    Likes Received:
    137
    Oh right, our invalid token is applying to a single page instead of a whole texture layer. The current limit is 16.

    TL;DR: Not easily. It's somewhere on our backlog of features we would like to implement someday, first to increase the limit to 32, and eventually getting rid of it entirely.

    Full version:
    The virtual texturing operates by processing a feedback buffer to figure out which virtual texture pages (and therefore which virtual textures) are visible. The way that data is packed, it has 12 bits each for the x & y components, 4 bits for the mip level, and 4 bits for the virtual texturing id for each pixel in the feedback buffer. 12 + 12 + 8 = 32bits, which is just the right size for GPU frame buffers. Unfortunately, you can only fit 16 VT ids in 4 bits.

    There are a couple possible solutions here. One is that we could reduce/shuffle some bits around to give a little more space to the VT id. Since the time when it was initially implemented, we dropped off the small mips of the virtual texturing anyway, so it's likely we could get up to 32 by taking that new limit into account, and using 3 bits for the mip and 5 bits for the id.

    Getting over 32 would require processing the feedback buffer in multiple passes. That would slow frame rates down a bit when more planets than the limit are on-screen, and also increase how complicated that code is. (And considering it's already quite complicated, that's not something to be taken lightly.)

    Even if we do increase that limit to 32 or get rid of it entirely, however, there are still other issues. Once you change the limit, you also run into UI issues. Everything has been laid out assuming that limit, so we would need to make a UI pass for handling an arbitrary number of planets.

    And each planet takes up memory. Right now, that is a very large amount of memory. Eventually that number will be smaller, but we are not there yet. Some people's machines have plenty of available memory, but others' do not, and we would really like to provide a good experience for those users.

    (The 15 thing came from the fact that we need a buffer value for "not VT", so we don't try to virtual texture the skybox & etc. We're using the ID 0xFFFFFFFF for that, which means the bottom-right page of the 16th texture is never going to get mapped. Initially, I wrote it to consider the entire texture ID as invalid, and set the maximum to 15. In practice, that is very hard to ever encounter, so I updated it to 16, and then forgot that I had done so over the last year or so.)
  3. spdr870

    spdr870 New Member

    Messages:
    18
    Likes Received:
    3
    I tested this PTE today. Worked pretty well. I have just one minor issue. Offline was disabled by default. My specs are i5 2500, 8Gb ram, nvidia 760, Ubuntu. I think offline was disabled because Ubuntu sais I have 7.8Gb ram. Maybe some ram is assigned to the integrated gpu even if it is not used. Anyway, if this is the case, maybe it is an idea to lower the ram specs by a few hundred megs? This is a very minor issue for me, since I simply changed the settings.
  4. cola_colin

    cola_colin Moderator Alumni

    Messages:
    12,074
    Likes Received:
    16,221
    on the question of cpu usage:
    on the linux based pastats.com server I can run a 8500 dox spread over 4 planets on area patrol with 1 sim fps. top shows 130% cpu on the process, so it does use a little more than 1 core. Though not really much more. The server has an i5, so even with the slow crawling server on it the webpage is unaffected
  5. DeathByDenim

    DeathByDenim Post Master General

    Messages:
    4,326
    Likes Received:
    2,118
    Ah, yeah. I had the same thing with an i7 (also 8GB RAM, NVidia and Ubuntu).
    indy802 likes this.
  6. zgrssd

    zgrssd Active Member

    Messages:
    658
    Likes Received:
    185
    Okay, I cannot get the server.exe to run on my windows 8.1.
    Even trying to run it from console produces a crash, but at lest it gives me a bit more information.

    At the same time trying to connect to the server running on another computer (I got a old win7x64 laptop) does not work either. So I guess I am the 1% that has issues with the offline mode.

    Here is what the commandline gives me (running under full admin rights):
    C:\Spiele\Steam\SteamApps\common\Planetary Annihilation\bin_x64>server.exe --all
    ow-game-config --allow-lan --headless
    [19:27:31.000] INFO Mounting C:\Spiele\Steam\SteamApps\common\Planetary Annihila
    tion\media\ as /
    [19:27:31.000] INFO SDL: Built w/ v2.0.3, linked w/ v2.0.3
    [19:27:31.000] INFO Loaded buildID 2014100873687
    [19:27:31.016] WARN Crash reporting disabled.
    [19:27:31.016] INFO Starting background thread pool with 3 threads
    [19:27:31.016] INFO GameServerImpl::resetModUpdateAuthToken: Auth token reset to
    "0f9da9de-6e78-aa41-ac5d-6ff3ba9d87a4"

    uncaught C++ exception of type "class std::runtime_error"
    what(): "listen() failed"

    0x1403da6a3
    0x140326855
    UnhandledExceptionFilter+0x142
    memset+0xbbf7
    _C_specific_handler+0x87
    _chkstk+0x9d
    RtlRaiseException+0xedb
    RtlRaiseException+0x18d
    RaiseException+0x6c
    CxxThrowException+0xd4
    0x1402e6520
    0x1400399c8
    0x14001ad1b
    0x1402c8bf7
    0x1402c4246
    0x140673e1e
    0x1406736bb
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 2632:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x1403092b0
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 3060:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x1403092b0
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 3328:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x140334ef7
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 3368:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x140334ef7
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 3896:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x140334ef7
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 3952:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x1403092b0
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    Thread id 5344:
    NtWaitForSingleObject+0xa
    WaitForSingleObjectEx+0x98
    CRT_RTC_INITW+0x3d4
    Concurrency::details::_Condition_variable::wait+0x70
    Call_onceEx+0xad
    0x140334ef7
    0x14020d52d
    std::_Pad::_Release+0x6c
    beginthreadex+0x107
    endthreadex+0x192
    BaseThreadInitThunk+0xd
    RtlUserThreadStart+0x1d

    C:\Spiele\Steam\SteamApps\common\Planetary Annihilation\bin_x64>
  7. SXX

    SXX Post Master General

    Messages:
    6,897
    Likes Received:
    1,811
    First of all thanks for explanation! Nice to know this limit wasn't enforced with no reason. Still would be cool if limits like this one won't be hardcoded so advanced users would able to experiment with game in some crazy ways. :rolleyes:

    This is true, but it's possible to solve it by modding. Even 14-16 planets don't work good on small screens anyway. :(

    I totally understand and support idea to improve experience for players with low RAM, but I don't think it's valid argument case of planet limit and this is why:
    • Game with just one large earth planet can use 3.5-4.5GB RAM.
      1300 meters, 0 water level, 60 height range.
    • Game with 16 moons of 200 meters only use 1.3GB.
    Considering PA client itself use about 800MB RAM for assets all these 16 planets use 500-600MB while one big earth use 2.8-3.5GB.
    Remy561 and tunsel11 like this.
  8. pinbender

    pinbender Active Member

    Messages:
    78
    Likes Received:
    137
    Can you check your windows firewall settings, and make sure server.exe is not getting blocked? (And try disabling antivirus software, as well.)
  9. zgrssd

    zgrssd Active Member

    Messages:
    658
    Likes Received:
    185
    Firewall was off for another test. Disabeled Virus scanner now, but no improovement.
    I found out about the " --allow-crash-api" switch, but it does not appear to do anything. Or at least I do not know where it is supposed to write the logs.
  10. fredegar1

    fredegar1 Member

    Messages:
    55
    Likes Received:
    65
    When PA recommends that I have 4 cores and 8gb of RAM for offline, does that mean 2 physical cores or does it include the 4 logical cores on my i7 4700MQ?
  11. SXX

    SXX Post Master General

    Messages:
    6,897
    Likes Received:
    1,811
    This option is NOT something you need for sure. It's only valid if you use HTTP API (--http option).
  12. pinbender

    pinbender Active Member

    Messages:
    78
    Likes Received:
    137
    To be clear, we definitely want to remove the limit, and we want to encourage experimentation. In general, we avoid hard limits. At one point, however, overflowing that hard limit would have caused a crash elsewhere, and it saves a lot of debugging time to know we were leaking virtual textures, or overlapping our planet generation. Since that time, the virtual texturing system has gotten a lot more robust, and it doesn't crash any more, but it still helps us make changes to other systems and know that we didn't break stuff in the process.

    We'll get to it, but it's going to take time.

    BTW, if you do work around that limit using a debugger, the output from the virtual texturing will be completely wrong. The moon system picture posted before may look correct in the zoomed out view, but if you look closer, you will find that the texturing is broken on 10 of the moons.
    Remy561 and SXX like this.
  13. SXX

    SXX Post Master General

    Messages:
    6,897
    Likes Received:
    1,811
    I just don't like hardcoded stuff in general because it's always make me sad when I trying to mod something. :(

    Yeah it's true, I noticed that once I tried to look closer.
  14. cola_colin

    cola_colin Moderator Alumni

    Messages:
    12,074
    Likes Received:
    16,221
    Has anybody figured out what kind of js environment the server uses? I want it to post data somewhere, but I fail to do so because none of the typical tools for that are defined and I have no idea if I can add them or how.
    Also does the thing have a debugger or something similar? So far it's pretty hard to use.
    proeleert likes this.
  15. wondible

    wondible Post Master General

    Messages:
    3,279
    Likes Received:
    2,084
    I saw some V8 stuff when scanning strings.

    I think figuring out an interactive console will be priority when I start digging into things. Surely someone has set something like that up for Node, but I don't know if their environment is Node-like enough to use it.
    proeleert likes this.
  16. masterdigital

    masterdigital Uber Alumni

    Messages:
    438
    Likes Received:
    833
    The server uses a custom integration of V8... there is no debugger and it is very difficult to use. You can use console.log to write messages and that is about it. While this is less than ideal, it would be very difficult to change at this point.
    lokiCML, Remy561, proeleert and 2 others like this.
  17. cola_colin

    cola_colin Moderator Alumni

    Messages:
    12,074
    Likes Received:
    16,221
    Do you know if it is possible to do HTTP posts with it?
  18. pinbender

    pinbender Active Member

    Messages:
    78
    Likes Received:
    137
    Not yet.
    proeleert and cola_colin like this.
  19. cola_colin

    cola_colin Moderator Alumni

    Messages:
    12,074
    Likes Received:
    16,221
    I like the yet part.
    So there is currently no way to get the beacon of a game out of the lobby.js to my own application?

    EDIT:
    I guess I'll build a little nodejs script that takes control of the server and parses the log output, so I can do:
    console.log(<json>); to "leak" data to the outside world.
    Last edited: October 9, 2014
    proeleert likes this.
  20. pinbender

    pinbender Active Member

    Messages:
    78
    Likes Received:
    137
    Not from within the server. You can get it from a PA client. Or you could write an app to connect to the server directly. The beacon listener is on UDP port 8192.
    proeleert and cola_colin like this.

Share This Page