Unofficial PAPA file format

Discussion in 'Mod Support' started by LennardF1989, June 4, 2013.

  1. LennardF1989

    LennardF1989 Uber Contractor

    Messages:
    798
    Likes Received:
    323
    This is the unofficial Papa file format specifications, I only partially reversed the format so models can at least be rendered. I'm working on bone and animation support, which I have almost figured out.

    This should do until Uber decides the Papa-file format in itself is mature enough to release the actual C++ headers.

    The code below is a valid 010 Editor Template, but can also be considered pseudo-code.
    Code:
    struct PapaFile;
    struct PapaFrameHeader;
    struct PapaFrame;
    struct PapaVerticesHeader;
    struct PapaVertex;
    struct PapaIndicesHeader;
    struct PapaTriangle;
    struct PapaBonesHeader;
    struct PapaBone;
    
    struct PapaFile
    {
        char Identification[4];
        byte Unknown1[4];
        int NumberOfBones;
        byte Unknown2[20];
        int64 OffsetBonesHeader;
        byte Unknown3[8]; 
        int64 OffsetVerticesHeader;
        int64 OffsetIndicesHeader;
        byte Unknown4[32]; 
        int64 OffsetFramesHeader;
    
        if(OffsetFramesHeader > -1)
        {
            FSeek(OffsetFramesHeader);
            PapaFrameHeader Frames;
        }
    
        if(OffsetVerticesHeader > -1)
        {
            FSeek(OffsetVerticesHeader);
            PapaVerticesHeader Vertices;
        }
    
        if(OffsetIndicesHeader > -1)
        {
            FSeek(OffsetIndicesHeader);
            PapaIndicesHeader Indices;
        }
    
        if(OffsetBonesHeader > -1)
        {
            FSeek(OffsetBonesHeader);
            PapaBonesHeader Bones(NumberOfBones);
        }
    };
    
    struct PapaFrameHeader
    {
        int Identification;
        int NumberOfFrames;
        int Unknown1;
        int Unknown2;
        int64 OffsetSomething;
        int64 OffsetFrames;
    
        FSeek(OffsetSomething);
        short Something[(OffsetFrames - OffsetSomething) / 2];
    
        FSeek(OffsetFrames);
        PapaFrame Frames[NumberOfFrames];
    };
    
    struct PapaFrame
    {
        byte Data[336];
    };
    
    struct PapaVerticesHeader
    {
        int Identification;
        int NumberOfVertices;
        int64 SizeVerticesBlock;
        int64 OffsetVerticesBlock;
    
        FSeek(OffsetVerticesBlock);
        PapaVertex Vertices[NumberOfVertices];
    };
    
    struct PapaVertex
    {
        float PositionX;
        float PositionY;
        float PositionZ;
        float PositionW;
        float NormalX;
        float NormalY;
        float NormalZ;
        float NormalW;
        float U;
        float V;
    };
    
    struct PapaIndicesHeader
    {
        int Identification;
        int NumberOfIndices;
        int64 SizeIndicesBlock;
        int64 OffsetIndicesBlock;
    
        FSeek(OffsetIndicesBlock);
        PapaTriangle Triangles[NumberOfIndices/3];
    };
    
    struct PapaTriangle
    {
        short A;
        short B;
        short C;
    };
    
    struct PapaBonesHeader(int numberOfBones)
    {
        PapaBone Bones[numberOfBones];
    };
    
    struct PapaBone
    {
        local int offset = FTell();
    
        int64 Size;
        int64 Offset;
    
        FSeek(Offset);
        char Name[Size];
    
        FSeek(offset);
    };
    
    PapaFile file;
    I implemented the format in C#:Changelog
    June 1 2013: Initial version

    Attached Files:

    Quitch likes this.
  2. theseeker2

    theseeker2 Well-Known Member

    Messages:
    1,613
    Likes Received:
    469
    I don't understand any of it, which is bad if I'm going to release a modding tool.
  3. brandonpotter

    brandonpotter Well-Known Member

    Messages:
    966
    Likes Received:
    389
    The sooner the tool is compete, and bug free, and usable in programs like 3DS Max, the better <3
  4. gunshin

    gunshin Well-Known Member

    Messages:
    790
    Likes Received:
    417
    I guess im new to these 'Papa' files, can anyone explain to me what they are/for? Im assuming models due to graphics structs butttt...

    Whats with the 'Unknown' names for those bytes and ints? am i missing something?
  5. numptyscrub

    numptyscrub Member

    Messages:
    325
    Likes Received:
    2
    The .papa files are the unit's physical definitions, including (but not necessarily limited to) models and bones. See also raevn's Unit Blueprints thread to see how they fit into the full definition.

    Regarding the Unknowns, those are the bits of the file Lennard hasn't figured out yet. Neutrino will release the proper headers once Uber are happy that the format is mature enough (i.e. won't change without notice). Until then, this should get you enough usable info out of the files to at least play around with ;)
  6. LennardF1989

    LennardF1989 Uber Contractor

    Messages:
    798
    Likes Received:
    323
    Neutrino provided me the header files since my initial "exercise" was over, so I can turn it into a usable open-source library. There is a lot more to it than the initial things I reversed engineered above. I'll keep you guys posted!
  7. brandonpotter

    brandonpotter Well-Known Member

    Messages:
    966
    Likes Received:
    389
    Awesome! I cant wait tto get my actual models started XD
  8. ozonexo3

    ozonexo3 Active Member

    Messages:
    418
    Likes Received:
    196
    Is posible to import models now? Im making delta commander statuette and this model will help me a lot.
  9. Gorbles

    Gorbles Post Master General

    Messages:
    1,832
    Likes Received:
    1,421
    Just wanted to give my thanks. Modelling scripts are the most integral part of any modding experience and I'm always thankful to people who step up and create stuff like this at the start of a game's lifecycle.
  10. chirmaya

    chirmaya Member

    Messages:
    30
    Likes Received:
    1
    I love and respect you people who can do all this sciency hard work! I just like to poke and play around, and you let me do that!
  11. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
    Needed to turn off optimization on the Bones Array for it to read it right, since the names are different lengths:
    Code:
    PapaBone Bones[numberOfBones] <optimize=false>;
    And the FTell in PapaBone needs to be moved, or it reads the same bone over and over again:

    Code:
    struct PapaBone
    {
        int64 Size;
        int64 Offset;
    
        local int offset = FTell();
        FSeek(Offset);
        char Name[Size];
    
        FSeek(offset);
    };
    Edit: Ok, so I've written an exporter for .papa and got it to a point where the console no longer gives errors just loading the mesh. However, it appears invisible (I replaced the ant's model with this one).


    I've also tried to expand the template, here's what I've got so far:

    Code:
    struct PapaFile;
    struct PapaFrameHeader;
    struct PapaFrame;
    struct PapaSomething;
    struct PapaVerticesHeader;
    struct PapaVertex;
    struct PapaIndicesHeader;
    struct PapaTriangle;
    struct PapaBonesHeader;
    struct PapaBone;
    
    struct PapaFile
    {
        char Identification[4];
        byte Unknown1[4];
        int NumberOfBones;
        byte Unknown2[20];
        int64 OffsetBonesHeader;
        byte Unknown3[8];
        int64 OffsetVerticesHeader;
        int64 OffsetIndicesHeader;
        int64 Offset1;
        int64 Offset2;
        int64 Offset3;
        int64 OffsetFrame;
        int64 OffsetFramesHeader;
    
        FSeek(OffsetFrame);
        PapaFrame Frame(Offset3);
    
        if(OffsetFramesHeader > -1)
        {
            FSeek(OffsetFramesHeader);
            PapaFrameHeader Frames;
        }
    
        if(OffsetFramesHeader > -1)
        {
            FSeek(OffsetFramesHeader);
            PapaFrameHeader Frames;
        }
    
        if(OffsetVerticesHeader > -1)
        {
            FSeek(OffsetVerticesHeader);
            PapaVerticesHeader Vertices;
        }
    
        if(OffsetIndicesHeader > -1)
        {
            FSeek(OffsetIndicesHeader);
            PapaIndicesHeader Indices;
        }
    
        if(OffsetBonesHeader > -1)
        {
            FSeek(OffsetBonesHeader);
            PapaBonesHeader Bones(NumberOfBones);
        }
    };
    
    struct PapaFrame(int64 Offset3)
    {
        byte Unknown1[200];
        int NumberOfTriangles;    
        int64 Unknown2;
        int NumberOfIndices;
        byte Unknown3[16];
        int LengthA; 
        int Unknown4;
        int64 Unknown5;
        int64 Unknown6;
        byte Unknown7[8];
        int LengthB; 
        int Unknown8;
        int64 Unknown9;
        int64 Unknown10;
        byte Unknown11[48];
        if (Offset3 > -1)
        {
            if (LengthA > -1) {
                if (LengthB > -1) {
                    FSeek(Offset3);
                    PapaSomething Something(LengthA, LengthB);
                };
            };
        };
    };
    
    struct PapaSomething(int A,int B)
    {
        char Data1[A];
        char Data2[B];
    };
    
    struct PapaFrameHeader
    {
        int Identification;
        int NumberOfFrames;
        int Unknown1;
        int Unknown2;
        int64 OffsetSomething;
        int64 OffsetFrames;
    
        FSeek(OffsetSomething);
        short Something[(OffsetFrames - OffsetSomething) / 2];
    
        FSeek(OffsetFrames);
        PapaFrame Frames[NumberOfFrames];
    };
    
    struct PapaVerticesHeader
    {
        int Identification;
        int NumberOfVertices;
        int64 SizeVerticesBlock;
        int64 OffsetVerticesBlock;
    
        FSeek(OffsetVerticesBlock);
        PapaVertex Vertices[NumberOfVertices];
    };
    
    struct PapaVertex
    {
        float PositionX;
        float PositionY;
        float PositionZ;
        float PositionW;
        float NormalX;
        float NormalY;
        float NormalZ;
        float NormalW;
        float U;
        float V;
    };
    
    struct PapaIndicesHeader
    {
        int Identification;
        int NumberOfIndices;
        int64 SizeIndicesBlock;
        int64 OffsetIndicesBlock;
    
        FSeek(OffsetIndicesBlock);
        PapaTriangle Triangles[NumberOfIndices/3];
    };
    
    struct PapaTriangle
    {
        short A;
        short B;
        short C;
    };
    
    struct PapaBonesHeader(int numberOfBones)
    {
        PapaBone Bones[numberOfBones] <optimize=false>;
    };
    
    struct PapaBone
    {
        int64 Size;
        int64 Offset;
    
        local int offset = FTell();
        FSeek(Offset);
        char Name[Size];
    
        FSeek(offset);
    };
    
    PapaFile file;

    Attached Files:

    Quitch likes this.
  12. LennardF1989

    LennardF1989 Uber Contractor

    Messages:
    798
    Likes Received:
    323
    Keep in mind the format constantly changes, as well as there is a lot more to the bones than you might initially think.

    I did some work on the tool today, currently working on making it easy too use, will go public soon-ish :)
  13. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
    I've got most of the bone data figured out; how they connect to form a skeleton and their positions, but I can't for the life of me seem to find how the vertices/indeces are assigned to a bone.
    Awesome! I know it's coming up, but I've kept working on this anyway for two reasons:
    1. I've never reverse engineered a file format like this before, and it's fun :)
    2. I'm writing a converter for a more archaic model format (3do).
    I think I've got the structure more or less completed, just a few areas left to figure out.

    Attached Files:

  14. LennardF1989

    LennardF1989 Uber Contractor

    Messages:
    798
    Likes Received:
    323
    It is fun to reverse engineer formats, I've been doing it for a longer period of time and I'm getting faster and faster with it, recognizing offsets, etcetera.

    010 Editor is also the most amazing tool I've come across to assist you in this with it's template system. I started out with tools like HxD and Hex Workshop.
    maxpowerz likes this.
  15. mrkroket

    mrkroket Member

    Messages:
    55
    Likes Received:
    8
    But it is just for fun, because on mod tools it's better to have the official FBX->PAPA exporter. In fact I don't know why Uber made new format, and not use the standard FBX or UDK's.
    It has something else that FBX doesn't have?
  16. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
    Quitch likes this.
  17. KNight

    KNight Post Master General

    Messages:
    7,681
    Likes Received:
    3,268
    You gonna share? :3

    Mike
  18. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
    At the moment PA is crashing when I try to build it, so something's not right :? . I'll post what I've got as soon as it works again.
  19. KNight

    KNight Post Master General

    Messages:
    7,681
    Likes Received:
    3,268
    I mean the Process, not the mod ;p

    Mike
  20. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
    I've written my own program that takes TA 3do models and spits out a papa file using the data structures posted above. I'll do a full tutorial soon.

Share This Page