Read the OP completely it is all on github since ages The one hack I found was fixed by Uber a few weeks ago. The magic mainly happens in https://github.com/pamods/Minimap/b...va/info/nanodesu/reader/PaClientAccessor.java with a version specific config file, for example https://github.com/pamods/Minimap/b...nfo/nanodesu/reader/patches/B82098Config.java There certainly is a chance I'll fix it again, for example when I want to play more. Atm I have other non-PA things keeping my busy.
with the new PTE there is an API that will mostly make this mod possible without any hackery. Well unit positions. Health and orders are missing, actually no I think at least health is also there.... unit positions = icons with correct positions = the most important feature anyway.
so generating this still takes 30-60 seconds due to the fact that I only can generate one puppet per one native call. I also can't show what kind of terrain it is. Hopefully build placement test call in the upcoming PTE will make this even better. The first one to guess the map correctly gets a cookie And the person who can explain to me how the *** the clipping of d3 works gets the rest of the cookie box. I am planning to probably do away with d3, since rendering data pixel by pixel isn't really it's point anymore. That means I have to emulate the clipping that d3 somehow does myself. All I know is that it sets a svg clippath based on a sphere. Now what I don't understand is how can I effectively tell if a pixel should be clipped or not. The only way I can imagine is to go backwards: Iterate over a sphere and project lots of points from it. Then check if the position to test is "close" to one of those dots. But that seems.... horrible. the winkel triple projection I am using is part of d3 geo and has a nice invert function but it as little to no indication that it should be clipped. The really bad areas are rather easy to detect as they yield really funny numbers, but there are some areas that "wrap around" as well. Bigger version of the images above. Generating that took a few minutes and a LOT of poor puppets that were disposed of right after the snapped to some location. And some random metal planet in extra large:
It sure is forge, I dunno why you would think that is meso. Also a pretty moon imho. Lower quality now, but this can be generated in less than 10 seconds per planet, which is probably what it'll be like.
So this is, give or take a little quality in terms of resolution, what the final maps will probably look like. A bunch of things: - Since there is no API to get mex directly I so a planet wide "place a lot of mex" test. The placement test snaps to them, so I get results for all mex, however since atm the snapping doesn't correctly tell me to where it snapped the mex spots are displayed a bit larger, resulting in the green areas. That's basically the area where an extractor will snap to the mex spot. - The random black spots on some shore areas are related to this: https://forums.uberent.com/threads/layers-for-build-placement.69409/ - Generating these images will take 5-10 seconds (hopefully there will be some API fixes that push this to 2-5 seconds instead) the first time you play a map and 2 second or less once you play it again. Especially during the initial preparation it will lag PA pretty hard. Once the image is there however it'll not create any more cpu load. Planets below a radius of 700 will be faster to generate, planets above that radius will have a lower image quality but won't generate slower. If the API improves and I further optimize the code I might push this quality border up a bit. Generally generation will be fast enough to be done within the commander spawn phase very easily. - You can still rotate the map, while rotating it will draw a low resolution version to keep up with your movement. 5 seconds after the rotation has ended it will redraw the high quality version. - There are 4 layers: Land, Sea, Mex and Blocked. For each of these a color range can be defined per biome. It will scale within this ranged based on the height of the specific location. So the color set will somewhat match the planet in question by biome at least. On this picture it uses yellow-brownish colors for land, blue for water, greenish for mex and black to gray for blocked areas. I want to thank @pinbender for sneaking in that build placement test API, it's the main source of information behind this picture.
moved to the titans forums, as this mod only works in titans. (actually after doing it... maybe it also works in the base game after the patch the base game had yesterday).
Updated the Ubermap: - Now makes strategic icons a bit brighters, especially when not selected. This shadows the icons shader. However it does so with a low priority. If you use a mod with some other shader, that mod most likely will be put on top. Notice the Ubermaps don't use the shader, so they won't follow other icon mods. The intent is to make that stupid brown color I get all the time more visible on the maps... - Implemented caching of map data. This should speed up UI loading times after you played a map for the first time. - Reduced FPS of the rendering of the maps. Should not matter much in how it looks, but saves some CPU. @netpyxa You might like this Also notice the "it will eat your computer" problem wasn't meant as being THAT bad. Please give it a try everyone and report how much of a hit it really is on your system.
Nope there are no "Units died here". Though there are half transparent "you saw enemy units here, but then lost vision" ghosts of enemies. Kinda the reverse thing, but with a similar result. Fun use case: Fly a scout over an enemy base. Don't look at it directly. A few seconds later open the ubermap and study all the stuff your scout saw via the icons. Or directly know that a group of 2 dox killed your fabber when you see a ghost of 2 dox where your fabber was before. I can look into units died here markers, but am not sure how much sense they make with the ghosts of enemy units.
I might not be totally overstating the "eat your computer part". Basically it seems once you reach ~1k units this creates severe game lags. It seems the handling of the API calls to get updates on all the unit positions gets pretty expensive and it seems that the API calls are not handled async to rendering. Meaning while it takes care of an API call it doesn't render anything. The whole game seems to stop. @pinbender can you clarify if this is the case and the API should behave like that? Once I reach these unit numbers the ingame API in fact seems to have substantially worse performance compared to my previous external memory reader. That reader really did eat one cpu core at full load, but it wasn't the core that took care of rendering the game. EDIT: So I am actually getting heavy performance drops, if I query for units states that seem to have changed. So when I have 700 dox idle it's not super great but playable. When they start to move it dies pretty hard, blocking the client for 500ms+ Interesting enough this even prevails when I give them an order move and then pause the game. It keeps massively lagging, even though, since it is paused, there are no changes. Any explanation for this?! EDIT 2: Okay testing scenario: 1.) Start a --nomods game on clutch vs idle AI with sandbox on. Make sure you are in the first slot (or you will need to change the army index in my example code) 2.) Use a bunch of avatar factories to make 1k dox. Place them somewhere and let them idle. 3.) Run this benchmark code: Code: var world = api.getWorldView(0); world.setServerCulling(false); // just so the test isn't influenced, I don't think this has any effects for this problem, apart from potentially creating false test results var tester = function() { console.time("getArmyUnits"); world.getArmyUnits(0, 0).then(function(data){ console.timeEnd("getArmyUnits"); console.log("getArmyUnits Result", data); var unitIds = []; _.forEach(data, function(elem, key) { _.forEach(elem, function(unitId) { unitIds.push(unitId); }); }); console.time("getUnitState"); world.getUnitState(unitIds).then(function(states) { console.timeEnd("getUnitState"); console.log("unit states results", states); setTimeout(tester, 3000); }); }); }; tester(); When all the dox are idle the expected result is that you'll see 10 to 30 ms for getArmyUnits and ~50 to 70ms for getUnitState. The game doesn't appear to lag, the fps might suffer a little, but clearly playable imho. Now move the dox. Expected result: getArmyUnits will still only use 10 to 30 ms. getUnitState however will use 1400 to 1500ms. Everytime this call is made the game lags for that time, no frames are rendered. Also notice that if you move the dox and pause the game the behavior seems to stay on "super slow" What is going on?
Okay another find: The performance issue seems related to large groups of units that have one single order given to them. So: select 500 dox and right click somewhere -> performance implodes select 500 dox and use cover the line to give a line command, this means each dox gets a "private" command -> performance doesn't care at all. @pinbender So I have a guess actually. The unit info API in the info about orders of the units gives that list of unit IDs that "have the order together with this unit". That sounds like a good trap for quadratic runtime: For each of the 500 dox it will iterate the other 499 dox to add their ID into that list. Bam! Quadratic runtime, apparently in a place that locks out the rendering thread from doing it's rendering. Also fits very well with the fact that pausing the game doesn't matter: It's not about state changes. I'd say remove that list of unit IDs (part of the order objects per unit. Do not remove the full order info, just that array of IDs inside the orders) until an API is in that has orders as first class entities, so you query a list of orders and not a list of units that has orders attached. once that is in the units could simply reference the order id and interested modders could query the order. I would guess nobody uses the list of IDs at the moment, so nobody should miss it and it makes the API painful to use in some scenarios atm. I guess to make the runtime of the collection of IDs way better it would be possible to put all units that will be returned in a unit id -> unit state object map and then iterate the orders and their units. For each unit lookup the unit state object and link the full list of ids that the order probably holds in there. That should be O(n log n). The resulting size of the messages would still be pretty much quadratic and I don't think anybody uses those IDs, so yeah.... don't think this is necessary. Just removing that list of IDs inside the order should be as much as commenting out some code for now :>
released an update that sets back the aggressivity with which the ubermap queries data. In case you have too big orders (see performance bug above) the map data may only update once every 1-2 seconds, but your fps should stay in the range of 20-30 with a micro lagging. Much better than what happened before. Hopefully the performance bug can be fixed soon, as it will improve performance by an order of magnitudes or two in the problem scenarios.
It's possible this is happening due to each unit having the list of all other units on the order, but I'm not in a good place to check right now. (I'd consider that somewhat unlikely, due to only increasing the number of items in the order array.) I'll add investigating that to the ever-growing todo list. The API will probably have to change to making a separate call for the order descriptions and hooking them up afterward on the JS side. It's better for memory use that way anyway.
I am talking about this: A result of the unit state query with one unit opened to see the orders. All other units have the same order, each with a copy of this array: Each single unit has that full array of all other units. So n^2 elements. Make n 500 or more and it'll take a while to process multiple hundred thousand elements. It's only IDs, but from how the performance reacts I am pretty sure there is some code in a perf critical position that iterates all these arrays one by one, effective iterating n^2 elements. When n is above 200-300 it starts to become problematic. Not to mention the player might give multiple orders to the group using shift. Each of them again will result in n^2 elements added. Thanks!
Yes, I caught what you were talking about. I wouldn't expect serialization of 250,000 numbers to take more than about 10-20ms, though. That's definitely more than 0ms, so if you're doing it more than 1Hz, it will definitely cause some problems. (It's also possible that I'm being too optimistic about the performance of our serialization layers, of course.)
