Terrain 2.0

Erm, I am going to interject here and bring the subject back to height maps.

I think @exterminans mentioned that planets like the metal planet do not use the noise function at all and (not 100% sure) they can be made entirely from CSG.
How can we make our own CSG, I heard this might be possible, and if so, we can make rudimentary shapes like wedges to at least create some sort of terracing, though I do . This seems like the most immediate way to get one step closer to custom terrain.

From what I have read, when PA creates a planet, the noise function is called, and then it is converted into geometry, an actual mesh, in order for CSG operations to occur. I am assuming the boolean operations are using mesh data rather than working with volume /noise functions.

You can make amazing heightmaps using various software, or even paper and watercolour if you want to show off. But as we have read so far, using a heightmap may be inefficient, but I challenge that, most players have tonnes of ram in their gfx cards and in their mobos, alas, since I am not a coder I cannot argue beyond that.

However, what if we could just take our beautiful extremely high resolution map and retopologize it to result with a mesh which can have as little as 16k points but keep all the feature detail since they are adaptively distributed.
This mesh could then, in theory, have CSG added to it and be textured by PA.

So what I would personally like to see, is a feature where you can import a custom base mesh, which replaces the first stage of planet generation. So;
instead of:
noise function --> mesh -->features (biome, water, met etc)
you can have:
your custom mesh -->features (biome, water, met etc)

Up until now I have just assumed that the part where I say "features" procedurally textures your mesh and does other biome stuff to it, but I just did a bit of messing about in system editor, and now I am thinking that the terrain is textured using values from the noise function rather than analysing mesh point height values. And CSGs aren't procedurally textured, they are models with textures pre-set made for specific biomes, so that puts a bit of a downer on how easy I thought this could be, I don't know I'm going to have to stop writing now.

@piebaron
That's just what I was trying to do originally, using only a very limited number of points. But as it turned out for me, interpolation is hard, extremely hard. If the grid of support points was uniform (which means that all points were aligned in a regular rectangular or evenly triagulated grid), bicubic interpolation would have been no only easy, but also the solution to be favored.

Problem is, as soon as you try to use the common interpolation schemes on not uniform grids, you are loosing certain properties. Specifically, if you were to interpolate a certain point in a uniform mesh using the bicubic filter, you would only need to sample the surrounding 4x4=16 points for a smooth interpolation as the uniform properties of the grid guarantee that any more distant point can not have any impact on the continuity of the interpolation when differencing. With non-uniform grids, that is no longer true, and if you want to achieve the same mathematical properties of the result, you have to take EVERY point into consideration. That wouldn't be so bad yet, but you actually have to solve a n by n matrix whereby n is the number of points.

You can use a natural neighbor interpolation scheme to reduce the complexity, but it's still slightly beyond reason. With Bézier surface like interpolation schemes the computational complexity becomes reasonable, but have you ever tried to create arbitrarily steep borders with a Bézier surface? Hint: It's just not working.

My last (not published) attempt was to just go with 3D Voronoi regions for steep borders, and the option to declare pairs of neighbors in the Voronoi diagrams where I did then apply a sigmoid like interpolation function. This gave an acceptable amount of control while maintaining a reasonable computational complexity for doing the interpolation. The results are still not satisfying though, as I currently have no control over the slope in the actual location of the support point, it's always horizontal.

As for the memory usage: I don't think the height map is actually ever "stored" as such. It's a procedure which yields a constant value when called for a specific point of the planets surface, and it's the same procedure used for both CSG placement and elevation, just with different octaves and different seeds for the random number generator it is based on.

Most of the finer details aren't even part of the planet generation, but only part of the texture generation process which - of course - uses the same procedures as the planet generation process, but happens deferred during runtime as this is where most of the VRAM is consumed in PA so virtual texturing is used to discard unused regions of the texture. I think for texture generation, they actually have used a uniform grid + noise based index, offset, scale and rotation approach. Plus decals which have been added during runtime - of course.

I didn't mean to create a uniform mesh which sampled a heightmap then interpolated, I mean, what if we could just use our own mesh as the base planet surface, optimised using third party software, or whatever.
for example, a quick mesh I made just now:

start with 4097x4097 heightmap, map to sphere, optimise points and end up with a 32mb mesh containing around 230k points which are positioned where detail is required, this mesh isnt a great example as the starting heightmap I used had too many small features where it should have just been flat, but I wanted to show you what I mean.

Ok, I get what you mean. But there is a tiny little problem: Optimizing the mesh is much more difficult than it sounds, especially when you want to get the map size down to more reasonable sizes, like less than 1MB in total for the entire map, also including CSG and other.

Also when you start to optimize the mesh, you are basically forced to discard any smooth curvature - and you have a lot of that in every location where you want to allow traversion between two areas. You are also suffering from all the tiny, cosmetic features which have been embedded into the mesh and which are now basically incompressible - and which an overlaid noise function could have recreated quite easily as well.

So even while designing the map, you would want to work with a low resolution mesh, which is being tessellated and being transformed by noise in real time while you are adding more points to your base mesh. And then there is the problem that you want at least C1 continuity in the interpolated mesh, that means the first derivate on the terrain is continuous in every point of the planet, or else you will get visible artifacts in the interpolated planet geometry.

Achieving C1 or even C2 continuity when interpolating is the hardest part. Downsampling / optimization respectively "reconstruction" of small features using a noise function is the easier part, even though I still wouldn't recommend to ever directly modify the full resolution mesh, but only ever work on feature points right away.

Why one mb? If I recall correctly Spring maps are bigger than 1mb.

I think everything below 100mb is a totally reasonable size for a handcrafted map. Sizes of default supreme commander maps:

The biggest map is 217mb with 81x81. 5x5 maps are in the region of ~3mb.
Nobody ever really had issues with map file sizes in SupCom.
All we need is a better caching system. Currently we have none at all.

There are various programs out there which have very powerful re-topology features, and people make terrain meshes for various games using third party 3D software. Like some people have said, why do the maps have to be less than 1mb?

You are right here 100%, just making a random terrain heightmap and optimising it isn't going to yield a game friendly mesh, for a start, optimising could mean a variety of things and chances are most people who try to make their own mesh will fail at even making it work in PA unless there are very specific guidelines on how it should be formatted etc.

But what about a mesh which has been carefully hand crafted like a metal planet, where its all hard surface geometry with hardly any points, how could we go about being able to create something like this? right now it seems like the only avenue left for making maps with strategic height levels and ramps etc.

I don't know what C1 or C2 means.
I think the technique you are proposing sounds awesome, extremely efficient, compatible with the PA texturing system and very user friendly if it gets developed properly. What do we need to ask Uber to make this possible?
All I want is maps with strategic elevations and the freedom to draw them out, by any means possible, what do you think our request to Uber should be?

you have to take into account that maps in SC can contain an endless number of mods folders.

to play a certain survival map that comes with it's own game rules and enemy ai spawns and behaviors as well as units you see nowhere else or to play on zone control, a whole other game, you just need the map, nothing else.

so i think we can knock of the topmost ones and say all maps, if talking about map data alone in SC are roughly 3MB

http://en.wikipedia.org/wiki/Smoothness Have a look at that Wiki article for a definition of these terms.

Well, my prototype came already close, but achieving a smooth interpolation turned out to be much more complicated than I anticipated. In order for Uber being able to implement it, an interpolation scheme is required which provides at least C¹ continuity in order to avoid visual artifacts, otherwise there would be visible edges in the terrain wherever the curvature changes, and you also want to avoid to create plateaus at every single feature point. Maybe for my next try, I should attempt to use a derivate of the natural neighbor interpolation scheme

There is a nice list of possible interpolation schemes on Wikipedia, but they still need to be adapted, as maps in PA are not even supposed to be fully continuous, but rather to contain both passable ramps as well as impassable cliffs.

As for the file size, there are various good reasons to keep it as simple as possible. Keep random generated maps in mind, for them you want to be able to generate maps which are fun to play with a minimal amount of overhead. Crafting a map of several MB of size, transmitting it over the network, so that players can just have a look at the preview doesn't sound reasonable at all. So you want the information which describes the gameplay on a specific map to be as compact and accurate as possible, and to be just as cheap to preview. SupCom maps were not random generated, whatever map the host chose, you either already had it, or there was still a good reason the host chose this specific map.

With random generated maps, this is just not the case, you have no guarantee that the generated map will offer the features you were looking for. That's also the reason why we actually don't have any random generated maps in competitive play right now, as the current random map generator isn't even capable of guaranteeing maps which are even remotely fun to play.

For hand crafted, or manually fine tuned maps, a larger file size is acceptable, after all, such a map is supposed to be proven worthy of being used and it is likely to be reused. The file size increases naturally as you use additional CSG or additional feature points to fine tune certain parameters, but this increase in size is not desired for random generated maps where you cycle through a comparably large list of possible options to find one which suits you best.

At some point, I had drafted a sequence for generating random maps from rules which define a certain gameplay, rather than letting gameplay be determined by the output of a random noise generator.

Where can I find these maps? I'd like to check them out. How were they made?
You mentioned hand crafted maps and how a larger size for them is acceptable, that's what I am rooting for.

Also I was just looking around the mods and found this http://pamatches.com/mod/custom-planet-type-cliffs/ has anyone else tried it out? I'm gonna check it out now.

those are the default maps. They basically have no mods at all. The large parts are the map files itself.

Obviously random maps should be nothing but a group of seeds. So far they are just that. I am only thinking of hand made maps. For those I don't see the need to spent a lot of work on inventing a complex compression system.

Even random maps should be MORE than just a group of seeds. You always have a tradeoff between replica determinism, that means having the ability to recreate a state from a initial set of data, and the ability to use efficient, but not necessarily deterministic algorithms. When you have an comparably expensive algorithm which fine tunes a map to achieve certain properties in an iterative process, you don't necessarily want to run the same process on all machines. You want to find a sane balance between the amount of data transmitted, and the computational overhead to recreate the full map with the intermediate goal to be able to provide a meaningful preview as early as possible.

As for hand made maps: Don't think of it as a compression system. You are not even supposed to place all features by hand, but only to state your intention and leave the generation of details to the engine. And it's not like much additional work was spent into creating such a system. It's just an intermediate representation which is required for a normal generation process, either way.


If you wanted to go full custom map, regardless of any file size constraints, you can even do so right now. Just bake your entire, hand modelld map into two massive CSG brushes, one for pathable, the other one for non-pathable terrain, plus several thousand instances of reclaimable feature, and you have perfect control over every possible aspect. Just bear in mind, that this thing isn't going to be editable ingame (fixed texture), and the file size is going to be comparably huge, even when compressed.

You are also loosing all benefits the engine is giving you right now, such as eliminating the necessity to define all decals and cosmetic features by hand. Remember, the engine does not support procedural texture generation on top of CSG brushes, so you are even loosing procedural virtual texturing support which also increases the VRAM requirements, so you are forced to tune down texture resolution as well.

That's obviously a rather hacky workaround. A simple way to set the heightmap would be better.

I've asked here and elsewhere before, and searched many times, how do you create a CSG brush? I must have missed some important thread.

You figure out how they work and do it. There is no official support for custom brushes, though it certainly is possible to make your own. Basically you will need to create a 3d model and export it into some specific file formats.

so nobody has ever made one and if they have they haven't talked about how they did it? Unfortunately I am not a coder, just a 3D artist so I need a little bit of help.

I was under the impression that @proeleert tried and somebody else whose name I forgot succeeded.

Someone has mode one, but iirc it didn't have textures because unlike unit models the textures for the brushes are already intertwined to the model... Something about the textures was different I can't remember.

I think the person with the name I don't remember fixed just that. It has to be fixable.

It's precisely what you asked for, nothing less. These brushes are allowed to contain the raw, uncompressed planet geometry and a custom texture. With all the advantages and disadvantages.