Graphics Update

Discussion in 'Planetary Annihilation General Discussion' started by varrak, December 17, 2013.

  1. varrak

    varrak Official PA

    Messages:
    169
    Likes Received:
    1,237
    I wanted to share a little bit about what I’ve been working on over the past month, since it will probably have some impact on anyone playing the game… and we like to share with you guys, of course.

    Specifically, I’m talking about graphics.

    It’s not entirely surprising that’s what I am going to talk about. That’s pretty much all I do here at Uber… that, and drink a lot of coffee. Oh, and play the game a bunch, too.

    Anyway, it became clear a little while ago we’d accumulated quite a bit of technical debt in the graphics engine, specifically because the engine evolved really, really quickly. To make progress, and get performance to where it should be, I decided it was time to pay back some of that debt. This involved a pretty extensive low-level re-write of the core graphics engine.

    The best analogy I can think of (for the non-graphics programmers out there) is this: imagine you are building a house. The foundation is in place, the walls are up, the roof is on, the windows are in, and you’re about to put finishing touches in. Then you realize… hey, we actually want 3 floors in this house, not 2. Darn. The foundation won’t take the weight of 3 floors…. So, to stretch this analogy to breaking point, I basically had to go in and rebuild the foundation of our graphics engine/house, so it would fit the existing structure, but support future growth. And allow it to go really, really fast (OK, we’re not talking about houses any more. The only time houses go fast is in a tornado.)

    Hopefully that made some kind of sense.

    Anyway, this is done, and you’ll see the results in the next release. Now, bear in mind, you will not see miraculous improvements in performance. I readily admit that there’s still a long way to go. But you should see some (we’ve seen some good improvements in our internal testing).

    I’ve also been talking a bunch to folks from NVidia (and more recently have engaged in a dialog with AMD). They’ve given me a bunch of pointers on (some of) the internals of how the drivers work, and the kinds of things we’ve been doing that the driver finds upsetting. I’ve only scratched the surface of improvements we need to make here; expect to see some big gains over the coming months.

    The upshot here is, things will be a little better in the next release. We’ve tested quite a bit, and it looks pretty solid, but it’s more than possible some issues might come up (this is a major rewrite of some pretty core systems, after all). If you *do* have issues with graphics, I apologize in advance. But let us know, and we’ll fix them. We’re always working hard to make sure the game is as awesome as possibly for absolutely everyone.

    For the terminally curious out there, here’s some down-and-dirty details of what’s changed:

    · I added a “hardware abstraction” concept, that basically provides an ultra-thin layer between the graphics API (in this case, openGL) and the game. In doing so, I improved the way we do some things, that massively reduces the number of API calls we make, which in turn improves performance.

    · The above hardware abstraction also provides more flexibility in how we do our rendering. From this, it will be possible to make our code even more efficient.

    · The “effects” system (the system we use to manage materials and shaders) is now entirely high-level on top of the hardware abstraction. It also pulls in data from JSON files, meaning the effects as well as the shaders will be entirely customizable.

    · I’ve added code-paths to support more advanced API features in OpenGL. Up to now, we’ve only had code-paths that support the lowest common denominator that we support (which was OpenGL 2.1). This has been limiting. By providing optional codepaths to leverage more advanced features, this helps us get more from the hardware. I’ve done this whilst still providing the fallback code-paths, so we’ll still run on older systems (albeit with less awesome than on newer hardware).

    · I made the HDR luminance phase live entirely on the GPU (instead of round-tripping from the GPU to the CPU to perform the tone-mapping calculations). This makes life better on certain video cards. However, this wasn’t ready in time for the upcoming release, so you won’t get the benefit of that quite yet.

    · And lots more. I could go on, but I have code to write…
  2. feyder

    feyder Member

    Messages:
    88
    Likes Received:
    13
    Looking forward to seeing the changes :D
    EDIT: From a technical standpoint this was a enjoyable read :D Keep them coming Uber :)
    LavaSnake, BulletMagnet and Terrasque like this.
  3. xnavigator

    xnavigator Member

    Messages:
    122
    Likes Received:
    4
    I hope the graphic will be somewhat more closer to the kickstarter trailer graphic... that would be awesome.

    Also I would like to point out that in 2013, for a company that needs to deliver a product with a close deadline, writing a new graphic engine is a very bad choice. You guys could use one of the hunders of graphic engine already available and focus on the gameplay instead.

    IMO
  4. Arachnis

    Arachnis Well-Known Member

    Messages:
    938
    Likes Received:
    442
    So basically, we won't be seeing much change in how the game looks and performs initially, but the fundamental work you've done will make it possible to improve the game more easily?

    Anyway, thanks for sharing.
    albanuche likes this.
  5. varrak

    varrak Official PA

    Messages:
    169
    Likes Received:
    1,237
    Basically... yes. It's not going to be a stratospheric improvement in performance - but it will be an improvement. It sets the stage for bigger and better things to come.
    doud, maxpowerz and Arachnis like this.
  6. cptconundrum

    cptconundrum Post Master General

    Messages:
    4,186
    Likes Received:
    4,900
    People often confuse graphics libraries and graphics engines. A graphics library is a very well defined set of operations that a program can call. This is what OpenGL is, and it does it very well. Graphics engine is a somewhat less well defined and it can mean a lot of things to a lot of different people. Generally, this would be more specific to the type of game being made and it tends to be more of a full "game engine" rather than just handling graphics. There are tons of FPS game engines out there, but I doubt many of them are built from the ground up to handle tens of thousands of NPCs across several planets. In any case, Uber has already done this work so there is no reason to do that over again.
    varrak likes this.
  7. SXX

    SXX Post Master General

    Messages:
    6,896
    Likes Received:
    1,812
    So no more GL_EXT_gpu_shader4 for newer GPUs? Great! ;)
  8. varrak

    varrak Official PA

    Messages:
    169
    Likes Received:
    1,237
    ... eventually... :) It's a work in progress, but yes, we'll be cleaning up the shader code quite a bit. We didn't get time to make all the changes we wanted to for this release, but we're paving the way. :)
    doud and SXX like this.
  9. brianpurkiss

    brianpurkiss Post Master General

    Messages:
    7,879
    Likes Received:
    7,438
    Very exciting stuff!

    Keep up the great work guys!
  10. stormingkiwi

    stormingkiwi Post Master General

    Messages:
    3,266
    Likes Received:
    1,355
    So the next release is coming soon?
    cwarner7264 likes this.
  11. Gorbles

    Gorbles Post Master General

    Messages:
    1,832
    Likes Received:
    1,421
    I love everything in the OP's post.

    Great to see some graphics work going on.
  12. neutrino

    neutrino low mass particle Uber Employee

    Messages:
    3,123
    Likes Received:
    2,687
    Yes.
    LavaSnake, Corang, drz1 and 5 others like this.
  13. doud

    doud Well-Known Member

    Messages:
    922
    Likes Received:
    568
    Thanks a lot for this very detailed update. It's not so common to have very skilled people also being able to communicate and share their knowledge. Really, much appreciated !
    stormblast, stormingkiwi and varrak like this.
  14. iceDrop

    iceDrop Active Member

    Messages:
    143
    Likes Received:
    99
    All sounds great! In particular I'd highlight this:

    Those of us running on OS X can update to 10.9 Mavericks at no cost, which supports up to OpenGL 4.1 (and my GT 650M apparently supports 4.1 quite well). Very promising news for the long haul, Thanks!

    A question (perhaps for anyone)... I recall hearing that the game was (feel like this was within the past month or so) currently more CPU-bound than GPU-bound. In my experience so far, only occasionally watching the CPU blinkenlights, I don't see much evidence of CPU load over 30% or so, yet things can bog down and I'm mildly interested in a way to see real-time GPU performance monitoring. Any recommendations out there? (Mac OS X -focused, please)
  15. doud

    doud Well-Known Member

    Messages:
    922
    Likes Received:
    568
    Was just wondering what you think about Mantle (well you don't have to tell me the truth if it's at the expense of your relations with AMD:)). I'm especially interested about Oxide presentation in which they state that mantle could be very helpfull especially for STRs where there are lots of units to be rendered. They also state that obviously current APIs are not able to match with hardware changes overtime and that this is a major root cause of performance issue and Application (game) ability to scale well with underlying hardware.
    Do you think this is true for Both DirectX and OpenGL ? In the mean time Mantle is proprietary, so i wonder how this can be adopted by software developpers if it leaves on the edge of the road, players who do not buy AMD gfx cards. Do you think that at some point RTS are not very well server by current APIs (DirectX & OPenGL) ?

    [​IMG]
    For those who are interested, you will find at the bottom of this french page all the pictures from Oxide presentation :
    http://www.hardware.fr/news/13450/a...n=Feed: hardware/fr/news (HardWare.fr - News)
  16. mot9001

    mot9001 Well-Known Member

    Messages:
    833
    Likes Received:
    650
    Lift roof, try to put more steel in it, make 3th floor then put the roof back on.

    Edit: doesn't work everywhere, dont try it at/with home
  17. paulusss

    paulusss Active Member

    Messages:
    271
    Likes Received:
    144
    Thanks a bunch Varrak lovely post! Hope your hard work will pay off allot of great stuff :D
    varrak likes this.
  18. varrak

    varrak Official PA

    Messages:
    169
    Likes Received:
    1,237
    Yes - Mantle is interesting, although I've not had much time to do more than glance at it. Having spent some time going deep in Xbox One (I worked on that before I came to Uber), I can say that the APIs can get in the way more than they help - even on a dedicated platform like Xbox (or PS4... or whatever).

    That said, PA is targeted at multiple platforms, and multiple devices, so a general API is the only way to go. In theory we could add another HAL (my code-name for the hardware abstraction) to sit on top of another API, and just go for it. Practically, it would still be more work and testing than we could realistically do at this point. OpenGL is actually not too bad, once you get used to it, and some of the insights I've gotten from the hardware vendors have given me some pretty good ideas on how to boost performance and reduce our CPU load.
    LavaSnake, Terrasque, Quitch and 2 others like this.
  19. doud

    doud Well-Known Member

    Messages:
    922
    Likes Received:
    568
    Thanks for taking time to answer :) I even did not have remembered that Xbox one & PS4 were both running AMD gpu and that, at least for NextGen consoles, it could make sense to have such common mantle API. So you've worked on Xbox one ... then definitly no one will be able to say that PC game developpers at Uber do not know what optimization means :) - A urban legend tells that PC game developpers never optimize their code as opposed to Console game developpers ;)
  20. neutrino

    neutrino low mass particle Uber Employee

    Messages:
    3,123
    Likes Received:
    2,687
    Most of our engineering staff has done console work, some going all the way back to SNES (secret of evermore). Varrak also worked on the directx team for many years.

    The reason that PC games don't seem optimized is that you basically can't optimize them in the same was as a console game. On consoles you can do things like writing directly into the command buffers to make things insanely fast. You can also optimize to the specific hardware capabilities. That makes it much easier to optimize.
    Grimseff, shootall, LavaSnake and 2 others like this.

Share This Page