The Performance Update: Titans Build 99377

Summary of Changes

• Added Multi-threading support to both client and local servers
• Tweaked AI
• LZ4 compression for server mods
• Major rework of the System Editor
• Lots of bug fixes and polish

Multi-threading support

• Added option to server settings tab to enable multi-threading for local servers.
• Added server command line option "--mt-enabled" which enables server multi-threading if present.
• The Nav, Physics, and AI updates are now run in parallel when enabled.
• These changes do not affect Uber hosted games, only local and private servers (if configured)

AI changes

• Retrained all AI neural networks.
• Made changes to reduce AI baiting
• Support for unit restrictions in new game lobby (work in progress)
• Added multi-thread safety to the AI neural networks

LZ4 compression for server mods

• Enable LZ4 server mod compression on servers with replay version >= 2, to help reduce client load times on servers with large mods
• Added check to enable LZ4 server mod compression only on new servers that can support compression based on replay version sent in login acceptance message.
• Currently this also changes replays. Older builds cannot load replays created by newer builds with LZ4 compression of server mods, but newer builds can load older replays
• Server version and buildid also sent in login acceptance message will be used for custom server checks in future.

System Editor changes
Refactored system_editor UI, system_editor_view and system_editor_planet

• Fixed issues where CSG would be lost
• Fixed lost advanced edits due to events not firing
• Fixed resolution scaling not set causing incorrect air zoom calculation
• Fixed unnecessary rebuilding when changing planet
• Fixed metal spots disappearing when finished editing (invalid placement)

Camera, camera_controller, camera_contoller_planet and camera_controller_space changes:

• Added mouseLocationAsJson
• Added toLatLng (was planet private)

System_editor_view and system_editor_planet changes:

• Fixed encodeCSGBrushListJson to encode preview brushes or builder brushes in planet
• Fixed addMetalSpot to validate metal spot and any mirror when placed
• Renamed senedSelectedPlanetToUI to selectedPlanetChanged and cleaned up usage in view
• Renamed sendSystemSpecToUI to systemChanged and cleaned up usage in view
• Renamed updateUIPlanetMetalSpots to selectedPlanetMetalSpotsChanged and cleaned up usage in view
• Renamed updateUIPlanetLandingZones to selectedPlanetLandingZonesChanged and cleaned up usage in view
• Added selectedPlanetCsgChanged to view
• Moved terrain editing status from view into planet (it's per planet which fixes lost CSG and unncecessary rebuilds)
• Moved UI interaction out of planet world into view
• Moved UI planets ready from planet world to view and renamed to updatePlanetsReady
• Added arePlanetsReady to planet world
• Added basic camera lookAt support for planet mode to view (TODO: space camera)
• Added systemEditor.convertSelectedPlanetToAdvanced for advanced editing in UI to view and planet
• Added selectedPlanetEditingChanged and selected_planet_editing message for advanced editing in UI to view
• Added selectedPlanetEditingAsJsonMessage to view
• Added setLandingZoneSize to planet for advanced editing in UI to view and planet
• Added camera_movement message for UI to view
• Added planent_csg updates for UI with selected planet index and current CSG selection for every change when editing terrain to view
• Added selected_csg message for UI when selectected CSG changes to view
• Added editing mode and planet status to selected_planet_index message for UI to view
• Added mouse location to time message for UI (to avoid spamming another UI message) to view
• Refactored state handling to use new engine messages
• Added handling to restore editing state when changing planets

System Editor Advanced mode changes

• Changed advanced edit of no preview placeholder to convert with no csg, metal spots or landing zones
• Changed advanced edit of terrain preview to convert csg only with no metal spots or landing zones
• Disabled advanced edit mode changes while planets are building
• Added landing zone rules editor with camera support to look at landing zone when selected
• Added landing zone size
• Added display of delete key bindings when editing metal spots and landing zones
• Added camera location
• Added mouse location

System_editor_view / system_editor_planet, csg_brush_preview changes

• Added resetSystem for loading entire system
• Added previewSelectedPlanet and updateSelectedPlanet
• Added releaseCsg and changed grabSelectedCsg to save original position
• Added cameraLookAt with space camera to focus on sun
• Added cancel check to release grabbed CSG
• Added saveUndoPosition and restoreUndoPosition to CSG brush preview
• Renamed buildAllPlanets to previwAllPlanets
• Tweaked setLandingZoneSize
• Tweaked systemChanged and selectedPlanetChanged from first pass
• Tweaked updatePlanetSpec

Additonal changes

• Added basic camera support to planet view in system editor via camera_movement handler and api.camera.lookAt
• Added planet selector with sun zoom to celestial (no more accidentally changing orbits trying to select a planet)
• Added thrusters required for attack to advanced edit
• Added clickable CSG command bar showing key bindings when CSG selected (was key bindings help)
• Added api.terrain_editor.releaseCsg with ESC handling to cancel grabbed CSG and restore original position
• Added sandbox back into biomes
• Cleaned up handling of planetCSG, metal_spots and landing_zones conversion to source > brushes, metal_spots, landing_zones
• Cleaned up initial loading of system
• Added buildAllPlanets to replace convertAllToTerrainPlanets with checks for existing status to system_editor_view
• Added resize handling to system editor when editing controls section overflows screen height eg long landing zone lists
• Added key bindings help when editing csg to system editor
• Tweaked advanced landing zone size and selected csg updates in system editor
• Added api.settings.title
• Added systemEditor.set_flooded_csg and systemEditor.set_no_features_csg
• Tweaked handling of release CSG
• Added hasUndoPosition and checks for newly placed CSG with no undo position
• Fixed message spam when planet or CSG grabbed
• Fixed csg selection not cleared when ending CSG editing
• Fixed selected csg not updated when placing new CSG
• Fixed flooded and weight params not preserved in preview CSG

Bug fixes/improvements
Client_connection changes

• Added CONNECT_TIMEOUT of 5 seconds and STATE_INITIALIZED
• Added RECONENCT_DELAY of 0.5 seconds and STATE_RECONNECTING
• Changed MAX_RETRIES to 2 ( best handled in UI where we can provide feedback)

• Fix for change from list to vector in ThreadPool and TaskBatch where mTasks size is 1 (macos crash fix).

Connect_to_game scene

• Added cancel button visible for non Ladder1v1 games after first failure which cancels immediately if no penidng engine connection
• Changed DEFAULT_CONNECTION_ATTEMPTS to 5 (now less in engine)
• Added DEFAULT_CONNECT_DELAY of 2 seconds when connecting to a new local server (reduces chance of first time fail while server is still starting)
• Added DEFAULT_RETRY_DELAY of 5 seconds
• Added connection attempts of 15 with retry delay of 10 seconds when loading local saved games (server is non responsive when loading big saved games)

Other fixes

• Updated to FMOD 4.44.64
• Attempted fix for AMD green planet bug
• Fixed missing build bar images for Enderstryke71 and Nefelpitou commanders.
• Fixed incorrect economy rate when new player joins an empty slot cleared by another player leaving / kicked / spectating (or an AI removed) that was an economy rate not 1.0
• Fixed long timeouts with many retries when connecting to non existent or stealh blocked servers.
• Fixed loading of big saved games
• Added cancel button while connecting to server
• Fix for planet wide patrol task not getting enough initial patrol points, causing the task to constantly reset itself and cause lag.
  
• Patrol task, auto repair task, and anti entity task now filter possible targets faster. This represents a decent perf boost for later game.
• Fixed bounty claimed by ally voice over playing in group for nuclear missle ready.
• Removed the ?_? AI name, since it was not displaying properly on at least one platform.
• Fixed lobbyId typo in start rejoinGame, lobbyId not set in matchmaking, ko db extender breaking systems and player guide not scrollable
• Fixed rejoinGame not settings mods, etc due to consistent use of lobbId vs lobby_id
• Fixed lobbyId not saved in matchmaking
• Fixed broken systems when not initialised correctly with default value of empty array by ko db extender
• Fixed player guide not scrollable due to incomplete white flash fix
• Fixed server mods not uploaded during host connect for manually started local server
• Changed community mods CDN URL
• Removed the Mutex from swizzletree. Instead mEnumerationCount is a std::atomic.
• Fix for an uninitialized variable that was preventing users from being able to change the playback rate of replays
• Removed a couple of very spammy log messages
• Fix for burnables attempting to write out to the burning curve during the shutdown call when the curve was already updated by the call to incinerate on the same tick.
• We no longer query the order's unit's position for every pass of (is my fellow unit close enough to me). For area patrols with lots of units, this is a huge perf win.
• Fixed the army patrol target cache to work more like it was initially intended.
• Added sim performance percentage next to game time in live_game_options_bar.
• Fix for agent looking at wrong move type value when determining if it should just steer towards its slot position.
• Fix for formation looking at wrong move type value when determining grid and cushion size.
• Fix for group moving an agents goal to its slot position when the slot position is not valid.
• Fixed nav agent logic regarding when to steer towards their group slot position and when to use their own ground path.
• Fix for a nav agent variable getting modified on when it wasn't supposed to be.
• Adjusted the logic controlling what nav search spaces get updated per tick.
• Minor adjustment to nav to prevent units from remaining stuck due to not having a flow field for an extended amount of time.
• Adjusted nav group logic so that slots are not treated as ahead of the group when the group is not moving.
• Added gNoMods to startupJS for better handling of --nomods by community mods.
• Added cores, memory and nomods to getSetupInfo and UI start scene.
• More multi-threading protection for the unit type db.
• Fix for Collision Geometry potentially rebuilding its BVH in one thread while another thread is walking the tree. Would only occur if there were multiple AIs and multi threading was enabled.
• Fix for uninitialized variable in nav agent.
• Adjusted AsyncParallelForDriver to split work more evenly over threads.
• Added getVersion to AudioWrapper, NullAudioWrapper, FmodAudioWrapper with decoding and logging of FMOD version.
• Added getVersion / getRenderer / getShaderVersion to GLRasterizer, NullRasterizer and Rasterizer
• Added fmod_version, opengl_version, opengl_renderer and opengl_shader_version in getSetupInfo for UI
• Moved where feature observables are removed to prevent features from mutating the sim history during the retire entities step.
• Fix for octree::walk deleting the FindState we are working before we were really done with it.
• Fix for failing to parse a unit type string if one of the unit types returned an empty spec set. This can happen if a spec string includes one of the Custom unit types, but there are no mods installed that makes use of that unit type.

• Added defeated status and AI personality to army replay info
• We are no longer allocating a string eveytime we validate a build order. We do this often enough that this is a decent perf improvement.
• Added checkboxes for selected CSG pathable, mergable, no features and flooded flags
• Added display of selected CSG JSON (temporary until full parameter editing is finished)
• Added clearMergableCsg, setFloodedCsg, clearFloodedCsg, setNoFeaturesCsg and clearNoFeaturesCsg to api.terrain_editor
• Improved and renamed respondToResize to checkResize with cleaner handling of landing zone and CSG json overflow
• Fixed uninitialised weight, weightHard and weightScale
• Fixed revert to standard mode while advanced editing with zero CSG, metal spots and landing zones
• Fixed remote saved games not showing last saved time and incorrectly sorting to bottom of saved games list
• Fixed minor scroll overflow in frame for player guide article
• Fixed --nomods handling using new gNoMods in connect to game, new game and replay loading
• Fixed api.net.joinGame never returning for invalid lobbyId
• Added cancel to join game after waiting 10 seconds

woah cool! especially the multi-threading support on both server and client! this looks like an optimization focused patch

Great work by great people.

It's yuuge.

WTF

Verty nice to see updates still coming in! Thank you!

Audio seems to be broken on Ubuntu 16.10. All I get is weird static, unless I kill pulseaudio first before loading the game.

Sounds seems to work fine here on my debian.

Sadly, I've always my "azerty keyboard linux bug" (with some keys not working).
(I've tried some workarounds but none of them where satisfying)

Anyway great jobs guys, I'll test this more deeply soon with some friends !

*crawls out of a hole*

Hmm? What?! A performance update?!

So does this mean multiplayer games with like 16+ people will run smoother compared to back in the day, due to these multi-threading improvements?

Audio works fine on my 16.04 install. Do you see any error in the logs relating to FMOD? Did you have audio troubles in the previous stable or is this new?

Also do you have special settings in Steam for starting PA? Such as custom LD_LIBRARY_PATH settings?

Yes... provided you run on a custom server with multi threaded mode enabled. also the multi threading is done *per planet* so it boosts multi planet systems but not really on single planet (although there are other improvements in there that should help). in my testing, I completed a pax prime game vs 5 insane AI (also threaded now) with no sim slow down at all in over 30 mins of game time. it's definitely much faster for that type of game...

You can also use the Public Local Server mod to run a multi-threaded server on your PC. I ran a number of 10 player 10 planet games this way.

This is great. Congratulations to all!

*rubs hands frantically*

Will your servers be updated to support multithreading in the future?

No errors I could see, here's a log: http://pastebin.com/a0PJXydg

I have nothing custom set, other games that use FMOD also have this issue. I think it's a bug within FMOD directly.

Well thanks to those industrious modders, we can already host private servers for public matches (as I understand it only the person *hosting* the server needs the mod)- so even if not mt servers are available. For example, I've got a (now pretty old) FX 8320, and it works a treat with the multi threaded server mode enabled, so I'd just host the server myself...

Could you try running PA with PULSE_LATENCY_MSEC=30 or even 60?
I think setting it in the launch parameters in Steam might work using "PULSE_LATENCY_MSEC=30 %command%", but just might want to try it outside of Steam too as I'm not really sure how environment variables propagate in Steam.

>Added RECONENCT_DELAY of 0.5 seconds and STATE_RECONNECTING

Can I assume you spelled RECONNECT_DELAY correctly in the code??