Coming up soon... Server Mods!

I wonder how true that is. What exactly do you mean by a per unit update? For many things I think some handlers for certain events should be completely fine. I can't think of many scenarios that really would require to directly sit in the sim, unless you plan to really drastically write different game, but in that case you might as well use some other engine. So let's take an example and ask based on that: DOTA.

I'd call that a total conversion. Now let's have a look what we need for this:

- The ability to spawn units. Cheatmode shows us that this obviously exists
- The ability to give a few commands to these units. Can't see why that would not be possible.
- The ability to react to the event of a unit being destroyed. I can't imagine there is any issue here. PA Stats basically gets this kind of event even inside the client. Why would it not exist server side? The death of the commander is part of this as well.
- The ability to limit what units a player controls. Can be done using teams.
- The ability to control the resources of a player. Can't see why sending metal to a player from a script should not be possible.
- The ability to lock the camera to a single unit. The tracking camera does this.
- The ability to define custom secondary weapons for abilities. Dunno how well that is possible, but at least the uber cannon exists.
- The ability to place some buildings using scripts (towers, etc)

So I can't imagine there is a big issue here. Yes adding in some "onUpdate" callback for every unit that can take in unit data, process it and move it back to the sim is completely insane performance wise. But handlers for "onUnitDeath", "onUnitBuild" and similar events that are executed asynchronous in some separate JS-thread and can send messages back to the sim that will be processed in whatever next simstep they will end up can't be that problematic and I'd expect they will exist.
About the unit attributes: If you want to store info for units you in fact can do that even without adding attributes. Units have an ID. That ID should be part of all events for that units. You can build up a map that contains data per ID. PA Stats and Alerts Manager do this inside the client right now. If you want to send this kind of data to the client... I think it was said in the past that there should be ways to send arbitrary data from the server to the client via a mod when there was a discussion that server mods may be used to send extra data to the client. Dunno how true that is by now, but I guess there is a way to send some data to the client, the game does that right now i.e. for some attributes of the army data. I'd expect a generic system behind that. Maybe something like a pulse curve that sends json data blobs.

EDIT:

The first mod I would want to try is a server side version of PA Stats. Basically it would gather the data from inside the server instead of the client.
Questions:

- Can the server js layer do the usual http stuff? GET and POST data to my server basically?
- Can the js layer access this data:
- Players and their teams
- All resources information of all players/teams
- Unit constructed/death events for everything
- Full system json definition of the system being played

Ok, I guess I exaggerated. The only feature which still seems really impossible right now, would be to replace the FOW / ghosting mechanic with the circular radius since that system is pretty much locked down.

When you think about, it's also one of the system we have the least information about how it works internally, e.g. which datastructures are used to accelerate that stuff.

As for extending unit attributes / storing data in embedded form:
It would have been useful for a very simple reason, and that is the performance of arrays in JS. Unless the compiler is able to detect static indizes, all engines I know of perform some type of hashtable lookup which - tested with array sizes of up to 10000 - is as bad as ~500-700 CPU ops per lookup. Well, thinking about it, then this is also true about looking up properties in the object passed from the native part, unless of course the JIT compiler manages to detect static offsets in which case performance goes up by far. Nonetheless - holding data structures with seemingly random access in JS is bad for performance since this denies the compiler from optimizing properly, enforcing slow fallback methods. (Don't be fooled by benchmarks like this one: http://jsperf.com/array-vs-object-performance/60 - V8 was able to fully eliminate the lookup.)

I don't think something like a hashmap lookup will overload any real cpu these days.
Ofc depends on what exactly you want to do, but I think for 99.9% of all scenarios lookups for object properties turn out to be pretty fast or at least fast enough. What kind of mod do you have in mind that would require tens of thousands of lookups every second?

EDIT:
About the FOW:
Again the question is what you are looking for. Using the current system it might i.e. be possible to spawn invisible units that give a certain FOW to a player. So you could in fact script some sort "show me that area of the map now".
In turn rewriting the whole FOW system to i.e. use real lines of sight was decided to be too expensive, even from the point of view of the developers who write native code.

Not real line of sight, but still a system which tracks visibility on a per unit base rather than on a per area base. That requires at least frequent iteration over the full unit set to update the visibility, even if that is sped up by storing the intelligence coverage as a heat map (which is why I could really use additional layers in the map) so that only a single map lookup per unit and iteration would be necessary. Still pretty fast in concept, most likely even doable with JS callbacks, but requires access to suitable, native datastructures since otherwise the performance penalty is off the limits.
Requirements in short:

• Embedded custom properties
• Texture support with optimized draw procedures, per planet
• At least deferred iterations over the entire unit set

And as for the hashmap lookups, don't underestimate it. You are capped at a mere 3-5 million hashmap lookups per second with the implementation most JS engines use - on a modern Intel CPU, not even considering L2 misses. So there's effectively a hard limit of somewhere around 500.000-1.000.000 updates (read+write) per second if going the JS way and the JIT manages to mostly eliminate lookups inside each transaction (not entirely possible). That's close to hitting the limit of touching every unit in the game once per cycle at mere 50.000 units in game.

And if the JIT does not by chance understand that an array has a homogenous structure, then this limit is even far lower since every single attribute accessed on a dynamically retrieved object requires yet another hashmap lookup so I assume a reasonable number would be at most 10-20k unit updates per second before the load gets critical.

Still quite a number, but no longer sufficient for all possible tasks.

I wonder what exactly you have in mind that actually would require to track custom attributes on every single unit with that kind of update frequency. What attributes do you have in mind that update that often? Scenarios?
Also the FOW modifications, do you have a mod concept in mind that would require that?

I do. An advanced iteration of my original concept for unified intelligence: https://forums.uberent.com/threads/discussion-about-vision-if-at-all.44469/#post-687594
Extended by accumulating intelligence on units over time instead of comparing them to a constant threshold. With the final goal of introducing actual low profile / stealth mechanics to the game as well as doing away with the FOW shaders on the client end.

I know how to do it, within certain limitations even in a rather efficient manner, but in order to do it, I need access to suitable datastructures. I can't do it entirely in JS, there's just no way to build efficient quad trees or textures (I need one) and mapping them to the planet. Barely possible when using the 2d context (commonly known as Canvas), but performance is still pretty bad compared to what you are used to when operating on a plain bitmap in C or C++.

I'm getting the impression that custom economy mechanics are out.

Does the engine support asm.js?

That's indeed a pretty advanced modification of a rather performance dependent system you want to do. Looks awesome, but certainly hard to implement in a scripting system. I doubt many RTS would allow that kind of mod or do you think other games have scripting systems that are fast enough? Then again, if you were to do it in for example SC2, that has far less units, so for that even a full js datastructure would be enough to deal with it.

EDIT:
wondible, what do you mean by custom economy? Some economy numbers don't seem like a very performance eating thing?

It would be close to impossible to do it in SupCom2, since the Lua runtime had no accelerated / native graphics library packaged. If it was using a recent JS interpreter it would have been possible though, mostly since there are less overall units and since the maps themselves are flat too, so no expensive coordinate transformations are required and therefor plain canvas could be used.
For PA, it's getting really tricky due to the required cube mapping and the fact that there is no acceleration available for this specific coordinate transformation.

Essentially for total conversions, where you have a different number or type of resources. So for instance, in Sacrifice, souls had perfect conservation (there were a fixed number in each game), and mana was generated at control points, and attenuated with distance.

I meant SC2 = Starcraft 2, because Starcraft2 definitely has a quite powerful modding system.
But it's kinda the same as in SupCom2: As long as there is API to control the FOW it wont matter much how slow the scripts are since the game only has very few overall units.

I wonder if the "this factory can now build XYZ" can be run inside of js if required. That way you could implement any resource system you want in js.

Ok, in Starcraft 2 it would be trivial. I wouldn't even require textures or any accelerated data structure, with the hard cap being 1600 units. I could redo the entire calculation every single frame without cache structures and it wouldn't hurt - at least not much.

Nice to see people getting into the gritty requests.

Understand that with modding stuff, there is always a wall where people will want access to stuff they don't have, and features/APIs added under the hood to make more stuff possible. That's the case with any game that allows any kind of modding, and it's an unbounded problem. We could spend all the time in the world on this and still never hit the point where there isn't some additional way to improve it.

With that in mind, given that it's an unbounded problem, and given that our resources most definitely are bounded, it's going to cut off somewhere. But since we have the benefit of an ongoing release process, test streams and an active community, we don't have to figure it all out at once. We can roll out some modding features, see how well stuff works and how far off the mark we are, and then refine and add more stuff accordingly. So that's what we're doing.

Naturally, the features that dovetail with actual game deliverables (like Galactic War) are going to get first dibs. After that, it's going to be features that give a lot of benefit to a lot of modders relative to the cost of implementation, and/or features that are prerequisites of other things we want to do. This is how the "transient server mod" thing came about, for example. Even when we get around to more persistent/published mods, we would still want a facility for people to iterate on their mods while they're under construction, without all the overhead of fully publishing them. So even in a persistent/published mod ecosystem, the transient mod support would still be a useful mod development and testing environment, and that's why we started with it.

All of this is several leaps away from what's being talked about in the posts above, surrounding native code vs. script code interactions and mods that have high performance requirements, and potential ties into deep and complicated systems like history, visibility, and navigation. Those systems are native code because they need to be fast and the tech is actually quite hard. I understand why modders want access to that stuff since it has so many gameplay implications, but in terms of complexity it is a world away from the kinds of lighter weight stuff that is feasible to delegate to script code right now.

I say that so that you understand, we want you folks to have access to as much as we can reasonably give you given the resources we have available, to whatever extent makes everybody happy and doesn't negatively impact our core business (which is one reason we're not giving the server executable away yet). So we're not limiting the modding APIs for unnecessary and artificial reasons. But APIs that tie into performance-critical systems take a lot of time and are extremely difficult to get right, and that puts them further out than other APIs that are easier to pull off in a reasonable time frame. That's just the reality of things.

The best possible solution is to take the modding features as they're rolled out, and use them to make cool stuff within the limits of what's available, letting us know of walls you hit so we can see where the low hanging fruit is for improvements. Then hopefully your mods get popular and bring more people into the game, who will fund us enough to continue giving you even more mod support.

The changes sound awesome!! I'm glad to hear that you are taking into consideration things like chrono cam and replays

Can't wait to start modding once this game is released. Loving all the features you are adding, including the galactic war. That was awesome fun. I hope to see more of everything.

Awesome work everyone!

Okay! The transient server mod support is going up on the PTE stream Extremely Soon (like, maybe tonight; the code is checked in but we have some publishing steps we have to do before it can deploy). Coming in hot! We have no idea if this works in the field yet; the benefit of the PTE stream is that we can play it a little more fast and loose. You're on the bleeding edge! (let's see if I can find some more cliches to use here)

Anyway, I'm writing this post ahead of that so give you folks the head start on this stuff, so you're ready when it gets deployed. Here's how it works:

First, go and refresh your memory on how the client mods have worked up until now, from this post I made a while back:

https://forums.uberent.com/threads/beta-mod-changes.51520/page-3#post-836401

From that post, make note of two things in particular:

1) The existence of a "mods" folder underneath your local app data folder for the game, where the client mods go.
2) The layout of the modinfo.json file providing metadata about each mod.

Here's the example modinfo.json file that I gave in that post:

Code:

{
    "context" : "client",
    "identifier" : "com.uitestmods.paui.testlobby",
    "display_name" : "My super sweet lobby replacement",
    "description" : "LOBBYYYYYYYY",
    "author" : "Zeus Hammer",
    "version" : "1.0",
    "signature" : "(not yet implemented)"
}

I want to call out a particular line at the top there:

Code:

    "context" : "client"

Until now, all released mods have been client mods, hence the client context. As of now, a new value for that "context" field is supported too:

Code:

    "context" : "server"

The mods that use this feature go in a new folder underneath the local app data folder for the game, the "server_mods" folder, which is a sibling to the "mods" folder.

(local app data)\Uber Entertainment\Planetary Annihilation\server_mods\...

The existing "mods" folder is now just a fallback for a "client_mods" folder, so we'll now have "server_mods" and "client_mods" going forward. If you have a "client_mods" folder, it'll use that; if you don't, that's okay, client mods will still load up from the "mods" folder as they do now.

Regardless, those client mods are completely separate from this new "server_mods" folder. Client mods and server mods exist independently, and have separate mods.json files with their own mount orders. The mods.json file for server_mods looks just like the existing one used for client mods, it just applies to the mods in the server_mods folder instead.

So that's the setup. Now let's talk about the usage.

Client mods (from the new "client_mods" or old fallback "mods" folder) will do what they've previously done, and mount onto the game's filesystem immediately at game startup. Your game client will continue to use them, just as before.

Server mods (from the new "server_mods" folder) will not mount at all though, unless you are the host client for a game by hitting Create Game in the server browser. Instead, all the mounted server mods (from the server_mods mods.json file) will be bundled up into a binary blob and sent to the server when the game is created. Other players who join your game will get that blob downloaded to them too, but it'll stay entirely in memory and only for the life of that particular game. The server, and all clients, will mount the files from that binary blob into memory and use them for your game.

This means that server mods are not strictly server mods. They can affect clients too, and in fact server mods for a specific game will mount higher in priority than the client mods installed ambiently for any game, which means that a server mod can* make UI replacements corresponding to their content changes (* "can" = "should"; I haven't actually tried this, I'm just speaking from how the infrastructure is supposed to work, bugs notwithstanding).

Server mods can then work with content that needs to be synchronized between the server and client, which previous client-only mods could not do. For example,

server_mods\energy_test_mod\pa\units\land\energy_plant\energy_plant.json (modified from original version)
server_mods\energy_test_mod\modinfo.json (formatted appropriately)

would allow the mod to shadow/replace the energy_plant unit with its own variation.

I have to run and do some family stuff for a bit, but I'll post more later tonight; hopefully this will get you going until then. More to come!

Okay, now let's talk a bit about the flow of things, and a few short-term technical issues that you'll want to watch out for (read: bugs I haven't fixed yet )

If you have your server_mods folder, and you have a mods.json file in it with your mount_order array of mod identifiers (just like you do with the client mods), then you can test whether your mods work at a basic level by going to the server browser and hitting Create Game. You don't even have to Start the game yet for this first test; just getting into the lobby should be enough.

You'll know you're past the first gate if you wait a few seconds* and then a box pops up in the lower left of the lobby window with a list of Server Mods (* may be longer than a few seconds if your mod is big or the network is slow; the mods have to upload to the server and acknowledge back to you before the box will show up).

If you see that box, it means the server has accepted your mods and mounted them, and responded to you that your client should do the same. So that's good.

I will edit this post with a screenshot of the server mods box once the PTE build is posted.

Similarly, any other player that joins into your game will want to see that box in their lobby window as well; if they don't, it means they haven't received & mounted the mods yet.

I mention all of this because you will want to wait until you see that Server Mods box in the lobby before you Ready into the game. Temporary issue, I just didn't get time to fix it yet; the lobby will ultimately need to prevent you from hitting Ready until the mods are there, but I won't get to that until next week probably. I'll update you when that's no longer an issue.

If everyone has the mods and hits Ready, your game should start as normal, but with your modified files applied.

One other temporary bug: During modded games, do not try to reconnect to a game if you get disconnected. Mods don't currently transfer back to reconnecting clients; that's on my list for early next week. In the meantime, stay connected.

Mod Development and Iteration

For some kinds of server mods, iterating on them may not be a big deal; they could be simple enough that all you need to do is start up a quick AI game with them applied, make sure they do what you want, and then start up another game later with other real players. Although the iteration times for that approach may be tedious, it's probably the most reliable way for major changes.

That said, if your changes are minor (like tweaking small balance numbers), it is possible to iterate your mods on the fly while the game is in progress, if you're willing to tolerate that A) Chronocam'ing before the update position may strange behavior, and B) you won't get a replay file written for that game.

The mechanism for doing this is a cheat mod, which enables publishing mod data updates from the host client mid-game. You make your changes to your server_mods files, then hit a key combination that's bound to an API function to publish the mods to the server (just like it does when you first created your game). If the cheat is not enabled, nothing will happen, but if it is enabled, then it'll send the updated mods to the other clients, and you'll get a generate-planets page refresh type of thing happen after the changes have been mounted.

I'll talk more about using this cheat, and the other cheat mods, in an upcoming post once the PTE build is deployed (because it'll be easier to explain with screenshots and I only want to make those with a real build).

Q: What kinds of content can we currently mod?

Units first and foremost, which thankfully covers a lot of ground (balance mods etc.). You can probably also mod effects, other entities like tools and ammo, and so on; most of my initial tests for that stuff have worked fine.

There is a temporary limitation where we still have a unit list json file with all the units that are possible to build; this means that while multiple mods that replace units can stack on top of each other, only one mod can actually add new ones. I've got a fix for this on my development machine (it'll scrape all the mounted unit directories to produce the list at runtime rather than use a single file for it), so that'll go in soon, but not quite yet; will update you when that's no longer an issue.

You can create new papa meshes too, if you're into that kind of thing; those should get deployed too. Watch out for the file sizes though folks (keep it small for now, remember these mods aren't persistent yet, so bandwidth matters).

One thing I would absolutely not mod yet though is anything related to planet generation. I don't know the details of all of that stuff yet anyway (that's more neutrino and varrak's specialty), but I do know that some of that data is currently not very mod-friendly as it's not expecting to reload dynamically, and there could be some crashes as a result. If you decide to step into those waters, be extremely careful and patient; I'd avoid it for now until we figure out some of these issues.

As for other content, I dunno... try it and let us know what happens.

Be Good. Make Good Decisions. Play Nice.

Please remember that we're still running these servers, and that puts us at a bit of a risk taking this kind of modding approach. Do not make your mods so big that the bandwidth cost is astronomical. Do not make your mods spawn so many units that any game involving them will bring the server to its knees (affecting any other games active on that machine). Do not do anything that even gives the slightest hint of trying to hijack the machine or those of other players connecting to your modded game. The host client of a game is responsible for the mod content of that game, and if we detect foul play we will be not only within our rights to take action against your UberNet account, but it may provoke us to dial back what mod capabilities we allow going forward.

That's not to say we're going to punish you for making simple slip-ups; heck you may end up exposing real bugs that we want to fix. That's all fine. But that's very different from making a mod that is deliberately evil. Use good judgment.

If you all play badly, then that wouldn't be fun for anyone. But if you play nicely, then know that I am going to be working on the modding ecosystem almost exclusively for at least the next couple months, which can mean a lot of fun times ahead. So let's all stay on the right side of the fence and enjoy what lies in store.

Okay, this post is long enough and the build isn't even up yet, so I don't want to jump too far ahead. More soon!

its a shame we can't voluntarily cut down on some bandwidth, and everyone move server mods they keep to their folder before entering a server, and the server not send them if they already exist. If multiple games use the same mod, the players could just keep the mod thru multiple games.

fingers crossed this goes well. If people stress the system too much or intentional malice, maybe you should do manual signups to access builds that allow a server mods page, it sounds like clients can recieve even if they don't have access to the server mods folder loading version.

Speaking of Papa files, any thoughts on if Uber will put out a converter for the Papa format?

Admittedly I've not been paying as much attention to this as I should have, my understanding was that ignoing the overall in-ability to mod the server the next biggest hurdle for new units was the papa format?

Mike

This is mainly due to an issue with the blend -> papa exporter being broken along with the blend -> fbx exporter being unusable for our purposes. I heard that the latter issue might have been fixed in a blender beta build, but never heard of any confirmation.

Another thing to do as soon as I can access my computer again.