Unofficial PAPA file format

Discussion in 'Mod Support' started by LennardF1989, June 4, 2013.

  1. KNight

    KNight Post Master General

    Messages:
    7,681
    Likes Received:
    3,268
    Okay, so knowing that, it's cool, but not really suitable of my needs! xD

    But it'll make your TA mod that much easier ;p

    Mike
  2. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,323
    Heres a c++ header for the papa file format, and the latest revision of the 010 Editor template.

    Attached Files:

    • papa.h
      File size:
      5.8 KB
      Views:
      34
    • PAPA.zip
      File size:
      1.2 KB
      Views:
      29
  3. ozonexo3

    ozonexo3 Active Member

    Messages:
    418
    Likes Received:
    196
    i will try create from this importer/exporter for Maya ;)
  4. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,323
    Any chance you could also pitch in with this for those who don't have very expensive software? :)
  5. ozonexo3

    ozonexo3 Active Member

    Messages:
    418
    Likes Received:
    196
    i dont know blender, only max and maya

    but first i need to know how your code works :D
  6. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,323
  7. LennardF1989

    LennardF1989 Uber Contractor

    Messages:
    798
    Likes Received:
    323
    Have a screenshot of all units.

    Attached Files:

  8. brandonpotter

    brandonpotter Well-Known Member

    Messages:
    966
    Likes Received:
    389
    Durrr, you guys are making the wait for a 3ds max plugin for Import/Export alot harder XD
  9. ozonexo3

    ozonexo3 Active Member

    Messages:
    418
    Likes Received:
    196
    today at morning i writed importer for Maya, but its not fully ready yet. I need to add UVs import. I will post it here soon(tm)
  10. neutrino

    neutrino low mass particle Uber Employee

    Messages:
    3,123
    Likes Received:
    2,687
    Don't write a 3ds plugin to import/export.

    Instead save your stuff as .FBX and then write an import/export using the FBX SDK. At least that's what we do. It's also possible that .papa may go away and we may just use FBX files directly. Either way you want an FBX centric pipeline.
    maxpowerz likes this.
  11. gunshin

    gunshin Well-Known Member

    Messages:
    790
    Likes Received:
    417
    have these header files changed at all from the ones raevn posted?
  12. kosmosprime

    kosmosprime Member

    Messages:
    97
    Likes Received:
    6
    I'm interested in that too. Why did every thread about these things die in the middle of june?
    Also, how "mature" is the PAPA format now and will it get replaced by FBX?
    element27 likes this.
  13. Culverin

    Culverin Post Master General

    Messages:
    1,069
    Likes Received:
    582
    Say what?!

    You're going to have a mode that imports all the TA models into this game?!?!
  14. LennardF1989

    LennardF1989 Uber Contractor

    Messages:
    798
    Likes Received:
    323
    Interest, time and the news FBX might be the new format. I'll see if I can finish my library this weekend and open-source it as I intended.
    kosmosprime likes this.
  15. kosmosprime

    kosmosprime Member

    Messages:
    97
    Likes Received:
    6
    Thank you.
  16. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,323
    Updating this as I try to work out the texture format. I'm not looking to work out how to read the content of the texture file inside, just how to determine the total size of it so I can extract it (I have the start offset).

    Here's the full editor template so far.

    Code:
    struct PapaFile;
    struct PapaModelInformation;
    struct PapaTextureInformation;
    struct PapaMeshBinding;
    struct PapaBoneMapping;
    struct PapaMeshInformation;
    struct PapaMaterialGroup;
    struct PapaMaterialInformation;
    struct PapaMaterialVectorParameter;
    struct PapaMatrix3;
    struct PapaMatrix4;
    struct PapaMatrixRow3;
    struct PapaMatrixRow4;
    struct PapaVerticesInformation;
    struct PapaVertex7;
    struct PapaVertex8;
    struct PapaVertex10;
    struct PapaIndicesInformation;
    struct PapaPrimitive0;
    struct PapaBonesHeader;
    struct PapaSkeletonInformation;
    struct PapaSkeletonSegment;
    struct PapaBone;
    
    struct PapaMatrix4;
    
    struct PapaFile
    {
        char Identification[4];
        short Unknown1[2];
        short NumberOfBones;
        short NumberOfTextures;
        short NumberOfVertexBuffers;
        short NumberOfIndexBuffers;
        short NumberOfMaterials;
        short NumberOfMeshes;
        short NumberOfSkeletons;
        short NumberOfModels;
        short Unknown2[4];
        int64 OffsetBonesHeader;
        int64 OffsetTextureInformation;
        int64 OffsetVerticesInformation;
        int64 OffsetIndicesInformation;
        int64 OffsetMaterialInformation;
        int64 OffsetMeshInformation;
        int64 OffsetSkeletonInformation;
        local int64 _OffsetSkeletonHeader = OffsetSkeletonInformation;
        int64 OffsetModelInformation;
        int64 OffsetAnimationInformation;
    
        if(OffsetModelInformation > -1)
        {
            FSeek(OffsetModelInformation);
            PapaModelInformation Model[NumberOfModels] <optimize=false>;
        };
    
        if(OffsetTextureInformation > -1)
        {
            FSeek(OffsetTextureInformation);
            PapaTextureInformation TextureInformation[NumberOfTextures] <optimize=false>;
        };
    
        if(OffsetMeshInformation > -1)
        {
            FSeek(OffsetMeshInformation);
            PapaMeshInformation MeshInformation[NumberOfMeshes] <optimize=false>;
        };
    
        if(OffsetMaterialInformation > -1)
        {
            FSeek(OffsetMaterialInformation);
            PapaMaterialInformation MaterialInformation[NumberOfMaterials] <optimize=false>;
        };
    
        if(OffsetVerticesInformation > -1)
        {
            FSeek(OffsetVerticesInformation);
            PapaVerticesInformation Vertices;
        }
    
        if(OffsetIndicesInformation > -1)
        {
            FSeek(OffsetIndicesInformation);
            PapaIndicesInformation Indices;
        }
    
        if (OffsetSkeletonInformation > -1)
        {
            FSeek(OffsetSkeletonInformation);
            PapaSkeletonInformation SkeletonInformation;
        };
    
        if(OffsetBonesHeader > -1)
        {
            FSeek(OffsetBonesHeader);
            PapaBonesHeader Bones(NumberOfBones);
        };
    };
    
    struct PapaTextureInformation()
    {
      char Unknown1[2];
      enum <char> {TF_DXT1 = 4, TF_DXT5 = 6} TextureFormat;
      char MIPS:4; //(TF_DXT1 only)
      char Unknown2:3;
      char SRGB:1 ;
      short Width;
      short Height;
      int64 Length;
      int64 Unknown3; //Always 128
      char Unknown4[Length];
    };
    
    struct PapaModelInformation()
    {
        int SkeletonData;
        int NumberOfMeshBindings;
        PapaMatrix4 Model2Scene;
        int64 OffsetMeshBinding;
    
        FSeek(OffsetMeshBinding);
        PapaMeshBinding MeshBinding[NumberOfMeshBindings] <optimize=false>;
    };
    
    struct PapaMeshBinding()
    {
        int Unknown1;
        int SkeletonSegments;
        PapaMatrix4 Mesh2Model;
        int64 OffsetBoneMappings;
    
        //Counts from 0 to SkeletonSegments
        if (OffsetBoneMappings > -1) {
            FSeek(OffsetBoneMappings);
            PapaBoneMapping BoneMapping[SkeletonSegments];
        };
    };
    
    struct PapaBoneMapping
    {
        short SegmentID;
    };
    
    struct PapaMeshInformation
    {
        int Unknown1; //possibly 2x shorts
        int NumberOfMaterialGroups;
        int64 OffsetMaterialGroup;
    
        FSeek(OffsetMaterialGroup);
        PapaMaterialGroup MaterialGroup[NumberOfMaterialGroups] <optimize=false>;
    };
    
    struct PapaMaterialGroup
    {
        short Unknown1;
        short MaterialIndex;
        short FirstIndex;
        short Unknown4;
        int NumberOfPrimitives;
        int PrimitiveType; //2=PRIM_Triangles
    };
    
    struct PapaMaterialInformation
    {
        short Unknown1;
        short VectorParameters;
        short TexturedParameters;
        short MatrixParameters;
        int64 OffsetMaterialVectorParameter;
        int64 OffsetMaterialTextureParameters;
        int64 OffsetMaterialMatrixParameters;
    
        local int _Restore = FTell();
        if (OffsetMaterialVectorParameter > 0) {
            FSeek(OffsetMaterialVectorParameter);
            PapaMaterialVectorParameter MaterialVectorParameter[VectorParameters] <optimize=false>;
        };
        FSeek(_Restore);
    };
    
    struct PapaMaterialVectorParameter
    {
        int Unknown1;
        float Vector[4];
        float Unknown6;
    };
    
    struct PapaMatrix3
    {
        PapaMatrixRow3 Row1;
        PapaMatrixRow3 Row2;
        PapaMatrixRow3 Row3;
    };
    
    struct PapaMatrix4
    {
        PapaMatrixRow4 Row1;
        PapaMatrixRow4 Row2;
        PapaMatrixRow4 Row3;
        PapaMatrixRow4 Row4;
    };
    
    struct PapaMatrixRow3
    {
        float Column1;
        float Column2;
        float Column3;
    };
    
    struct PapaMatrixRow4
    {
        float Column1;
        float Column2;
        float Column3;
        float Column4;
    };
    
    struct PapaVerticesInformation
    {
        int VertexFormat;
        int NumberOfVertices;
        int64 SizeVerticesBlock;
        int64 OffsetVerticesBlock;
    
        FSeek(OffsetVerticesBlock);
    
        //Position3Normal3Color4TexCoord4
        if (VertexFormat == 7) {
            PapaVertex7 Vertices[NumberOfVertices];
        }
    
        //Position3Weights4bBones4bNormal3TexCoord2
        if (VertexFormat == 8) {
            PapaVertex8 Vertices[NumberOfVertices];
        }
    
        //Position3Normal3Tan3Bin3TexCoord4
        if (VertexFormat == 10) {
            PapaVertex10 Vertices[NumberOfVertices];
        }
    };
    
    //For VertexFormat 7
    //Position3Normal3Color4TexCoord4
    struct PapaVertex7
    {
        float PositionX;
        float PositionY;
        float PositionZ;
        float NormalX;
        float NormalY;
        float NormalZ;
        byte Colour[4];
        float U;
        float V;
        float X;
        float Y;
    };
    
    //For VertexFormat 8
    //Position3Weights4bBones4bNormal3TexCoord2
    struct PapaVertex8
    {
        float PositionX;
        float PositionY;
        float PositionZ;
        byte Weights[4];
        int BoneSegmentIndex; //should be 4 x bytes?
        float NormalX;
        float NormalY;
        float NormalZ;
        float TexCoord1;
        float TexCoord2;
    };
    
    //For VertexFormat 10
    //Position3Normal3Tan3Bin3TexCoord4
    struct PapaVertex10
    {
        float PositionX;
        float PositionY;
        float PositionZ;
        float NormalX;
        float NormalY;
        float NormalZ;
        float TanX;
        float TanY;
        float TanZ;
        float BinX;
        float BinY;
        float BinZ;
        float X;
        float Y;
        float U;
        float V;
    };
    
    struct PapaIndicesInformation
    {
        int IndexFormat;
        int NumberOfIndices;
        int64 SizeIndicesBlock;
        int64 OffsetIndicesBlock;
    
        //IF_UInt16
        if (IndexFormat == 0) {
            FSeek(OffsetIndicesBlock);
            PapaPrimitive0 Primitives[NumberOfIndices/3];
        };
    };
    
    struct PapaPrimitive0
    {
        short VertexA;
        short VertexB;
        short VertexC;
    };
    
    struct PapaSkeletonInformation
    {
        short NumberOfBones;
        short Unknown[3];
        int64 Offset;
        PapaSkeletonSegment SkeletonSegment[NumberOfBones];
    };
    
    struct PapaSkeletonSegment
    {
        short BoneIndex;
        short ParentSegmentIndex;
        float TranslationX;
        float TranslationY;
        float TranslationZ;
        PapaMatrix3 ShearScale;
        PapaMatrix4 Bind2Bone;
        float BoneOffsetPositionX;  //?
        float BoneOffsetPositionY;  //?
        float BoneOffsetPositionZ;  //?
        float Unknown;              //always 1
    };
    
    struct PapaBonesHeader(int NumberOfBones)
    {
        PapaBone Bones[NumberOfBones] <optimize=false>;
    };
    
    struct PapaBone
    {
        int64 LengthOfBoneName;
        int64 OffsetBoneName;
    
        local int _Restore = FTell();
        FSeek(OffsetBoneName);
        char BoneName[LengthOfBoneName];
    
        FSeek(_Restore);
    };
    
    PapaFile file;
    The texture information is stored in a block:

    Code:
    struct PapaTextureInformation()
    {
      char Unknown1[2];
      enum <char> {TF_DXT1 = 4, TF_DXT5 = 6} TextureFormat;
      char MIPS:4; //(TF_DXT1 only)
      char Unknown2:3;
      char SRGB:1 ;
      short Width;
      short Height;
      int64 Length;
      int64 Unknown3; //Always 128
      char Unknown4[Length];
    };
    Using papadump on a texture .papa file gives the following:
    Code:
    c:\Program Files (x86)\Planetary Annihilation\PA\bin_x64\tools>papadump.exe -v aa_missile_vehicle_diffuse.papa
    1 texture:
      0:
      name: "/pa/units/land/aa_missile_vehicle/aa_missile_vehicle_diffuse.png"
      format: TF_DXT5
      width: 128
      height: 128
      mips: 7
      srgb: 1
    0 vertex buffers.
    0 index buffers.
    0 materials.
    0 meshes.
    0 skeletons.
    0 models.
    0 animations.
    Although this says 0 skeletons, the texture .papa files do have 1 bone specified, which has the name of the texture file.

    Edit: Mask & material textures are TF_DXT1 format, with srgb=0.
    Edit2: Updated texture info; length is now known but when I copy out the block it doesn't appear to be a valid png file.
    Edit3: More updates to texture info. MIPS & srgb are now included.
    Last edited: December 3, 2013
    cwarner7264 likes this.
  17. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,323
  18. proeleert

    proeleert Post Master General

    Messages:
    1,681
    Likes Received:
    1,656
    Is there some info on how to use papatran ?
    I want to create a papa file of a png for texture.
    Code:
    C:\Program Files (x86)\Steam\SteamApps\common\Planetary Annihilation\bin_x64\too
    ls>papatran evergreen_01.png -o evergreen_01.papa --texture-mode include
    evergreen_01.png: could not find resource-root setting
    1 error
  19. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,323
    I get the same error. I wanted to try and convert a known image file to papa so I could then look at the result and try and determine the format from that, but no luck.
  20. DeathByDenim

    DeathByDenim Post Master General

    Messages:
    4,326
    Likes Received:
    2,118
    Ok, I figured it out. It's not stored as a PNG as papadump would have you believe. It's simply rgba. I've included some sample code that reads out "/pa/terrain/sky/textures/skybox_01_back.papa".
    skybox_01_back.png
    The offset in the program I wrote is hard coded, but it can be found using the headers you already posted of course. I've also hardcoded the location of the file, since this is just proof of concept. I'll make a nicer version soon.

    Anyway, basically:
    Code:
    void papatexturereader::readImage(QImage &image)
    {
      image.fill(0);
    
      QFile papaFile("/home/jarno/Games/PA/media/pa/terrain/sky/textures/skybox_01_back.papa");
      if(!papaFile.open(QIODevice::ReadOnly))
        return;
    
      papaFile.seek(0x80);
      QByteArray data = papaFile.read(4*1024*1024);
      for(int i = 0; i < 4 * 1024 * 1024; i += 4)
      {
        int pixelindex = i / 4;
        image.setPixel(pixelindex / 1024, pixelindex % 1024, qRgba(data[i], data[i+1], data[i+2], data[i+3]));
      }
    }

    Attached Files:

    trialq and LavaSnake like this.

Share This Page