[REL] ReScalePA: Converter to create smaller units

well I'll get you a beer basket, then.

@raevn:
I think I understand what it is missing, but I can't correctly express it with the 101 template.
Basically the PapaVerticesInformation for the bot_factory seems to actually look like this:

Code:

struct PapaVerticesInformation
{
    enum <int> {Position3Normal3Color4TexCoord4 = 7, Position3Weights4bBones4bNormal3TexCoord2 = 8, Position3Normal3Tan3Bin3TexCoord4 = 10} VertexFormat;
    int NumberOfVertices;
    int64 SizeVerticesBlock;
    int64 OffsetVerticesBlock;

    int VertexFormat2;
    int NumberOfVertices2;
    int64 SizeVerticesBlock2;
    int64 OffsetVerticesBlock2;

    int VertexFormat3;
    int NumberOfVertices3;
    int64 SizeVerticesBlock3;
    int64 OffsetVerticesBlock3;

    FSeek(OffsetVerticesBlock);

    //Position3Normal3Color4TexCoord4
    if (VertexFormat == 7) {
        PapaVertex7 Vertices[NumberOfVertices];
    }

    //Position3Weights4bBones4bNormal3TexCoord2
    if (VertexFormat == 8) {
        PapaVertex8 Vertices[NumberOfVertices];
    }

    //Position3Normal3Tan3Bin3TexCoord4
    if (VertexFormat == 10) {
        PapaVertex10 Vertices[NumberOfVertices];
    }



    FSeek(OffsetVerticesBlock2);

    //Position3Normal3Color4TexCoord4
    if (VertexFormat2 == 7) {
        PapaVertex7 Vertices2[NumberOfVertices2];
    }

    //Position3Weights4bBones4bNormal3TexCoord2
    if (VertexFormat2 == 8) {
        PapaVertex8 Vertices2[NumberOfVertices2];
    }

    //Position3Normal3Tan3Bin3TexCoord4
    if (VertexFormat2 == 10) {
        PapaVertex10 Vertices2[NumberOfVertices2];
    }



    FSeek(OffsetVerticesBlock3);

    //Position3Normal3Color4TexCoord4
    if (VertexFormat3 == 7) {
        PapaVertex7 Vertices3[NumberOfVertices3];
    }

    //Position3Weights4bBones4bNormal3TexCoord2
    if (VertexFormat3 == 8) {
        PapaVertex8 Vertices3[NumberOfVertices3];
    }

    //Position3Normal3Tan3Bin3TexCoord4
    if (VertexFormat3 == 10) {
        PapaVertex10 Vertices3[NumberOfVertices3];
    }

};

So there are 3 blocks of vertices. I guess that is the amount of models in the file. The same might be true for bones and similar, have not checked yet.
So how do I define these 3 blocks and then create 3 structs of PapaVertexN arrays afterwards?
I can't figure out how to define a new variable in a loop using FSeek (so can't just define an array) without creating duplicate variables.

EDIT:
@LennardF1989
I think you know these template files pretty well as well, don't you?

EDIT 2:
Though this template isn't actually that vital to me, I am gonna write code to handle this anyway. :>

EDIT 3:
Scaling down all 3 meshes inside the bot factory works out:


Now to why those little factory arms are so horribly out of place...

Maybe animations have their own absolute tranformation for bones?

Makes perfect sense that there would be multiple PapaVerticesInformation, blocks. There's even an unused "NumberOfVertexBuffers" . In struct PapaFile you need to modify the call to PapaVerticiesInformation (line 76) as follows:

Code:

    if(OffsetVerticesInformation > -1)
    {
        FSeek(OffsetVerticesInformation);
        PapaVerticesInformation Vertices[NumberOfVertexBuffers] <optimize=false>;
    }

This will call PapaVerticiesInformation multiple times.

Finally, under PapaVerticiesInformation, add the following before the FSeek(OffsetVerticesBlock):

Code:

    local int _Restore = FTell();

This saves the position of the pointer. After reading the Verticies, restore the pointer with:

Code:

    FSeek(_Restore);

The same is probably true for both PapaIndicesInformation (use NumberOfIndexBuffers) and PapaSkeletonInformation (NumberOfSkeletons). Maybe even PapaBonesHeader, but that works a bit different.

Edit: Looks like PapaSkeletonInformation will need a bit more modifying; It reads PapaSkeletonSegment immediately, but I'm guessing there should actually first be a seek to Offset first. I'm surprised this didn't entirely break, unless it actually is the case that the SkeletonSegments are always written immediately afterwards.

Note: Not tested, my 010 editor trial just ran out. Need to buy it, it's a great program.

I've not seen any model with more than one Skeleton according to the counter at the start at least.

That's very likely to be the case, do we know anything about those anim.papa's? All my current 101 template tells me is that it has animation data. Nothing is filled out. I have no idea whatsoever what format animation data even takes.

EDIT:
papadump actually prints every single frame. If there is no ready made information available on the format of animations I guess I can actually work from there to get the format.

Oh wow. I think that's new? Definitely should be able to reverse engineer the animation from this.

But I don't think animations are designed around specific coordinates; I still think the answer is in the model file.

Are you scaling the translation, Bind2Bone and BoneOffsetPositions of the bones?

Only translation and the BoneOffsets. I tried to scale all matrices as well, but that looked just broken. I guessed they do not have a scale and only define transformations.
What does Bind2Bone do? Looking at the numbers most of them are tiny tiny values, so I figured it was just a transformation. I guess I can try to scale Translation, Bind2Bone and BoneOffsetPosition but not the shearscale. Will try when I get home this evening.
The numbers printed for the animation in turn look like numbers of positions within the model.

From my experience, I don't think that's the case.

I made a model and put that into the game (before I had even started any animations). I then went on to add an animation. The animation played in game without the model having ever come in contact with animations.

Unfortunately it didn't work completely, but I believe that is due to a different issue (during fbx -> papa all bone turned to point right).

Looking at Papadump, the translations are in the final column of bind2bone, and are the result of the parent bind2bone value, minus the current bone translation: -5.914487 - 1.5086 = -7.423087

You'll need to check in the 010 editor whether the matrix is stored column-major or row-major, I can't remember off the top of my head. Should be easy to spot though, they'll be the only numbers that aren't 0, 1 or -1.

That doesn't seem to be quite right. Here an example from the bot_factory:

First the unmodified model, look at bone 7:

The formula of parent bind2bone - translation of current bone doesn't work out for the red marked values.
Also in the 101 template those offsets are not part of the bind2bone matrices, looks like the matrices are offset by 4x floats.

Now my current program scales down TranslationX/Y/Z as well as BoneOffsetPositionX/Y/Z

This is how the model looks ingame after the changes. Idle and without animations it looks fine.
Note how I marked one of the arms.


Through playing around with the TranslationX and BoneOffsetPositionX of the respetive bones I think I can verify that the position of those elements depends indeed on the TranslationX and BoneOffsetPositionX, with the offset affected the following bone.

Changing the position of A makes B follow. Changing the translation of B only moves B relative to A.

So this happens when the factory starts playing the build animation. I have left the positions of A and B unchanged compared to the prev screenshot. So they should be especially out of place here:


They magically move up to the "correct" location and start animating. To me this looks like the bone positions in the mesh itself are not used by the animation at all. Compare this screen of the same animation, but without the "broken" bones A and B. I can't tell any difference to the screenshot above during the animation:


So from this I conclude the animations work by setting the positions of bones to absolute values that are stored per frame in the animation papa file. Looking at the output of papadump for the build animation:


These translations to me look like they are meant to basically replace the translation of the respective bone for that frame of the animation. Looking at the "idle" translation values for the elbow01 shows this:


That's exactly that 6.89 I am scaling. Z has a tiny change. Dunno how much such a tiny change actually means for the animation. But this looks like the translations numbers in the animation file are scaled according to the mesh.

EDIT:
Scaling the translation values in the animation works, notice the correct position of the build arms:


EDIT 2:
THIS IS REAL LEGO Duplo

Nice !

[REL] !!!

@raevn:
This is the final 101 template I have, it has some info on the animations:

Code:

struct PapaFile;
struct PapaModelInformation;
struct PapaTextureInformation;
struct PapaMeshBinding;
struct PapaBoneMapping;
struct PapaMeshInformation;
struct PapaMaterialGroup;
struct PapaMaterialInformation;
struct PapaMaterialVectorParameter;
struct PapaMatrix3;
struct PapaMatrix4;
struct PapaMatrixRow3;
struct PapaMatrixRow4;
struct PapaVerticesInformation;
struct PapaVertex7;
struct PapaVertex8;
struct PapaVertex10;
struct PapaIndicesInformation;
struct PapaPrimitive0;
struct PapaBonesHeader;
struct PapaSkeletonInformation;
struct PapaSkeletonSegment;
struct PapaBone;
struct PapaRGBA;
struct AnimationInfo;
struct AnimFrame;
struct PapaMatrix4;
struct BoneTransform;

struct PapaFile
{
    char Identification[4];
    short Unknown1[2];
    short NumberOfBones;
    short NumberOfTextures;
    short NumberOfVertexBuffers;
    short NumberOfIndexBuffers;
    short NumberOfMaterials;
    short NumberOfMeshes;
    short NumberOfSkeletons;
    short NumberOfModels;
    short Unknown2[4];
    int64 OffsetBonesHeader;
    int64 OffsetTextureInformation;
    int64 OffsetVerticesInformation;
    int64 OffsetIndicesInformation;
    int64 OffsetMaterialInformation;
    int64 OffsetMeshInformation;
    int64 OffsetSkeletonInformation;
    local int64 _OffsetSkeletonHeader = OffsetSkeletonInformation;
    int64 OffsetModelInformation;
    int64 OffsetAnimationInformation;
   
    if(OffsetModelInformation > -1)
    {
        FSeek(OffsetModelInformation);
        PapaModelInformation Model[NumberOfModels] <optimize=false>;
    };

    if(OffsetTextureInformation > -1)
    {
        FSeek(OffsetTextureInformation);
        PapaTextureInformation TextureInformation[NumberOfTextures] <optimize=false>;
    };

    if(OffsetMeshInformation > -1)
    {
        FSeek(OffsetMeshInformation);
        PapaMeshInformation MeshInformation[NumberOfMeshes] <optimize=false>;
    };

    if(OffsetMaterialInformation > -1)
    {
        FSeek(OffsetMaterialInformation);
        PapaMaterialInformation MaterialInformation[NumberOfMaterials] <optimize=false>;
    };

    if(OffsetVerticesInformation > -1)
    {
        FSeek(OffsetVerticesInformation);
        PapaVerticesInformation Vertices[NumberOfVertexBuffers] <optimize=false>;
    }

    if(OffsetIndicesInformation > -1)
    {
        FSeek(OffsetIndicesInformation);
        PapaIndicesInformation Indices;
    }

    if (OffsetSkeletonInformation > -1)
    {
        FSeek(OffsetSkeletonInformation);
        PapaSkeletonInformation SkeletonInformation;
    };

    if(OffsetBonesHeader > -1)
    {
        FSeek(OffsetBonesHeader);
        PapaBonesHeader Bones(NumberOfBones);
    };

    if (OffsetAnimationInformation > -1)
    {
        FSeek(OffsetAnimationInformation);
        AnimationInfo AnimInfo;
    }
};

struct AnimationInfo
{
    int UnknownData;
    int FrameCount;
    int FPSNominator;
    int FPSDenominator;
    int64 MoreUnknown;
    int FramesOffset;
   
    local int _Restore = FTell();
    FSeek(FramesOffset);
    AnimFrame frames[FrameCount] <optimize=false>;
    FSeek(_Restore);
};

struct AnimFrame
{
    BoneTransform bonesTransforms[NumberOfBones] <optimize=false>;
};

struct BoneTransform 
{
    float TranslationX;
    float TranslationY;
    float TranslationZ;
    float Rotation[4];
};

struct PapaRGBA
{
    unsigned char r;
    unsigned char g;
    unsigned char b;
    unsigned char a;
};

struct PapaTextureInformation()
{
    char Unknown1[2];
    enum <char> {TF_DXT1 = 4, TF_DXT5 = 6} TextureFormat;
    char MIPS:4; //(TF_DXT1 only)
    char Unknown2:3;
    char SRGB:1 ;
    short Width;
    short Height;
    int64 Length;
    int64 Unknown3; //Always 128
    PapaRGBA RGBAData[Length / 4];
};

struct PapaModelInformation()
{
    int SkeletonData;
    int NumberOfMeshBindings;
    PapaMatrix4 Model2Scene;
    int64 OffsetMeshBinding;

    local int _Restore = FTell();
    FSeek(OffsetMeshBinding);
    PapaMeshBinding MeshBinding[NumberOfMeshBindings] <optimize=false>;
    FSeek(_Restore);
};

struct PapaMeshBinding()
{
    int Unknown1;
    int SkeletonSegments;
    PapaMatrix4 Mesh2Model;
    int64 OffsetBoneMappings;

    //Counts from 0 to SkeletonSegments
    if (OffsetBoneMappings > -1) {
        FSeek(OffsetBoneMappings);
        PapaBoneMapping BoneMapping[SkeletonSegments];
    };
};

struct PapaBoneMapping
{
    short SegmentID;
};

struct PapaMeshInformation
{
    int Unknown1; //possibly 2x shorts
    int NumberOfMaterialGroups;
    int64 OffsetMaterialGroup;

    local int _Restore = FTell();
    FSeek(OffsetMaterialGroup);
    PapaMaterialGroup MaterialGroup[NumberOfMaterialGroups] <optimize=false>;
    FSeek(_Restore);
};

struct PapaMaterialGroup
{
    short Unknown1;
    short MaterialIndex;
    short FirstIndex;
    short Unknown4;
    int NumberOfPrimitives;
    enum <int> {PRIM_Triangles = 2} PrimitiveType;
};

struct PapaMaterialInformation
{
    short Unknown1;
    short VectorParameters;
    short TexturedParameters;
    short MatrixParameters;
    int64 OffsetMaterialVectorParameter;
    int64 OffsetMaterialTextureParameters;
    int64 OffsetMaterialMatrixParameters;

    local int _Restore = FTell();
    if (OffsetMaterialVectorParameter > 0) {
        FSeek(OffsetMaterialVectorParameter);
        PapaMaterialVectorParameter MaterialVectorParameter[VectorParameters] <optimize=false>;
    };
    FSeek(_Restore);
};

struct PapaMaterialVectorParameter
{
    int Unknown1;
    float Vector[4];
    float Unknown6;
};

struct PapaMatrix3
{
    PapaMatrixRow3 Row1;
    PapaMatrixRow3 Row2;
    PapaMatrixRow3 Row3;
};

struct PapaMatrix4
{
    PapaMatrixRow4 Row1;
    PapaMatrixRow4 Row2;
    PapaMatrixRow4 Row3;
    PapaMatrixRow4 Row4;
};

struct PapaMatrixRow3
{
    float Column1;
    float Column2;
    float Column3;
};

struct PapaMatrixRow4
{
    float Column1;
    float Column2;
    float Column3;
    float Column4;
};

struct PapaVerticesInformation
{
    enum <int> {Position3Normal3Color4TexCoord4 = 7, Position3Weights4bBones4bNormal3TexCoord2 = 8, Position3Normal3Tan3Bin3TexCoord4 = 10} VertexFormat;
    int NumberOfVertices;
    int64 SizeVerticesBlock;
    int64 OffsetVerticesBlock;
   
    local int _Restore = FTell();
    FSeek(OffsetVerticesBlock);

    //Position3Normal3Color4TexCoord4
    if (VertexFormat == 7) {
        PapaVertex7 Vertices[NumberOfVertices];
    }

    //Position3Weights4bBones4bNormal3TexCoord2
    if (VertexFormat == 8) {
        PapaVertex8 Vertices[NumberOfVertices];
    }

    //Position3Normal3Tan3Bin3TexCoord4
    if (VertexFormat == 10) {
        PapaVertex10 Vertices[NumberOfVertices];
    }
    FSeek(_Restore);
};

//For VertexFormat 7
//Position3Normal3Color4TexCoord4
struct PapaVertex7
{
    float PositionX;
    float PositionY;
    float PositionZ;
    float NormalX;
    float NormalY;
    float NormalZ;
    byte Colour[4];
    float U;
    float V;
    float X;
    float Y;
};

//For VertexFormat 8
//Position3Weights4bBones4bNormal3TexCoord2
struct PapaVertex8
{
    float PositionX;
    float PositionY;
    float PositionZ;
    byte Weights[4];
    int BoneSegmentIndex; //should be 4 x bytes?
    float NormalX;
    float NormalY;
    float NormalZ;
    float TexCoord1;
    float TexCoord2;
};

//For VertexFormat 10
//Position3Normal3Tan3Bin3TexCoord4
struct PapaVertex10
{
    float PositionX;
    float PositionY;
    float PositionZ;
    float NormalX;
    float NormalY;
    float NormalZ;
    float TanX;
    float TanY;
    float TanZ;
    float BinX;
    float BinY;
    float BinZ;
    float X;
    float Y;
    float U;
    float V;
};

struct PapaIndicesInformation
{
    enum <int> {IF_UInt16 = 0} IndexFormat;
    int NumberOfIndices;
    int64 SizeIndicesBlock;
    int64 OffsetIndicesBlock;

    //IF_UInt16
    if (IndexFormat == 0) {
        FSeek(OffsetIndicesBlock);
        PapaPrimitive0 Primitives[NumberOfIndices/3];
    };
};

struct PapaPrimitive0
{
    short VertexA;
    short VertexB;
    short VertexC;
};

struct PapaSkeletonInformation
{
    short NumberOfBones;
    short Unknown[3];
    int64 Offset;
    PapaSkeletonSegment SkeletonSegment[NumberOfBones];
};

struct PapaSkeletonSegment
{
    short BoneIndex;
    short ParentSegmentIndex;
    float TranslationX;
    float TranslationY;
    float TranslationZ;
    PapaMatrix3 ShearScale;
    PapaMatrix4 Bind2Bone;
    float BoneOffsetPositionX;  //?
    float BoneOffsetPositionY;  //?
    float BoneOffsetPositionZ;  //?
    float Unknown;              //always 1
};

struct PapaBonesHeader(int NumberOfBones)
{
    PapaBone Bones[NumberOfBones] <optimize=false>;
};

struct PapaBone
{
    int64 LengthOfBoneName;
    int64 OffsetBoneName;

    local int _Restore = FTell();
    FSeek(OffsetBoneName);
    char BoneName[LengthOfBoneName];

    FSeek(_Restore);
};

PapaFile file;

Excellent work

And that turret is truly terrifying!

Only one addition; I'm thinking one of the short Unknown2 vars at the start might be NumberOfAnimations, and
AnimationInfo AnimInfo should then be "AnimationInfo AnimInfo[NumberOfAnimations] <optimize=false>"

That is actually a grenadier
Yeah a number of animations might make sense. I wonder if there are units that have more than one, they should look wrong in that case.

I thought it was the new turret from RCBM (or was it Stratera. I'm so confused ).

It's RCBM that has the dual barreled high damage anti-air tower.

@cola_colin when I use this mod it always get stuck on planet generation in lobby :

Code:

[23:35:41.097] INFO Starting background thread pool with 10 threads
15261 21:35:41.381254 W | PID: 15277 | 15297 21:35:41.364676 Could not recognize the mime type of coui://pa/terrain/lava.json
[23:35:45.447] INFO connection_GameConfig
[23:35:45.448] INFO connection_SimCreated
[23:35:45.610] INFO Building planet of radius 454 detail 2
[23:35:45.797] INFO 8239 decals generated.
[23:35:46.336] INFO 44428 total planet faces.
[23:35:46.579] INFO build complete: 969. msec
  PlanetBuilder::buildAsync: 968. msec
  BuildMeshes: 220. msec
  Final: 46.8 msec
  SetupVT: 173. msec
  Water: 1.05 usec
  other: 6.22 usec
  Csg: 562. msec
  UpdateBVH: 22.2 msec
  other: 539. msec
  GenBrushes: 91.7 msec
  PostCopyDecals: 547. usec
  other: 91.1 msec
  GenSurface: 443. usec
  PreCsg: 31.4 msec
  Create Surface Meshes: 23.8 msec
  Connectivity: 19.3 msec
  other: 4.46 msec
  other: 7.67 msec
  UpdateBVH: 3.01 msec
  other: 60.9 msec
  other: 890. usec

How many minutes did you wait until you gave up? It uploads 90megabyte when you create the lobby. So it may take minutes till it continues.

EDIT:
That logs looks exactly like my log during the upload progress. For me it takes roughly 90 seconds because I have 1 megabyte/s upload. You can check your taskmanager to see if your internet is active:

Since big mods like this are not exactly intended to be used like this the upload progress has no progressbar or the like. You just wait for minutes. You can make this much quicker by copying the models and animations into your PA and removing them from the mod, but then all players that join the game absolutely need to do that as well.

EDIT 2:
Notice the timestamps in my log there:

Code:

[23:54:46.985] INFO build complete: 1.78 sec
    PlanetBuilder::buildAsync: 1.78 sec
        BuildMeshes: 246. msec
            Final: 59.3 msec
            SetupVT: 185. msec
            Water: 1.48 msec
            other: 5.29 usec
        Csg: 1.38 sec
            UpdateBVH: 41.1 msec
            other: 1.34 sec
        GenBrushes: 56.7 msec
            PostCopyDecals: 401. usec
            other: 56.3 msec
        GenSurface: 403. usec
        GenWater: 1.23 msec
        PreCsg: 45.9 msec
            Create Surface Meshes: 36.3 msec
                Connectivity: 28.8 msec
                other: 7.48 msec
            other: 9.63 msec
        UpdateBVH: 4.42 msec
        other: 44.3 msec
    other: 1.33 msec
[23:56:22.004] INFO ClientPlanet::finishPlanet - Nav/Physics: 3.392730

As far as i can remember we had been told by Uber that doing such stuff would require lots of additional stuff and finally it gets modded ..... So either original stuff was absolutly not down-scalable and they modified it so that anyone can mod it, or that was b.........