Blueprints Format (Unity)

    Good day, this question is a bit "technical", but how blueprints data structure looks like? I mean their storage format. I'm interested in this as I want to experiment with blueprint editing using external tools (I mean making tools for blueprint editing/creation)

    Basically the blueprint files contain various meta data, followed by the preview image (thumbnail and screenshot, both in JPG format), followed by the actual building data (which is compressed) ;) The first bytes of the blueprint (basically the header) contains information about where the data is located in the buffer.


    Blueprint data endianness in the new version is always Little Endian.


    The very first byte contains the blueprint version, currently (as of 0.4.5) it is version 6. We use the version for compatibility checks in future updates (possibly the format may slightly change in a future update, so that would be indicated by the version number).


    Java blueprints use a different format btw. Since they use the same file ending (".blueprint"), you can check if it's a Java blueprint by having a look at the first three bytes: In the Java version, the whole blueprint data was GZIP compressed, so if the first byte is always "31", the second byte is "139", and the third byte is less or equal to "8", it's most likely a Java blueprint. Java blueprints use Big Endian btw, but that's a different story. Once a Java blueprint is loaded in the new version, it gets converted to the new version format automatically.


    The whole data structure of a blueprint in the new version looks like this:


    HEADER


    Number of bytes
    		 Data
    1 blueprint version
    4 total number of bytes in blueprint data
    4 start position of meta data in buffer
    4 length of meta data (in bytes)
    4 start position of thumbnail image in buffer
    4 length of thumbnail data (in bytes)
    4 start position of screenshot data in buffer
    4 length of screenshot data (in bytes)
    4 start position of actual blueprint data in buffer
    4 length of actual blueprint data (in bytes)



    META DATA


    Number of bytes
    		 Data
    1 is legacy blueprint (from Java version)?
    8 blueprint creation date (unix timestamp)
    4 relative x coordinate of the blueprint center (float)
    4 relative y coordinate of the blueprint center(float)
    4 relative z coordinate of the blueprint center(float)
    4 extent/size of the blueprint along x axis(float)
    4 extent/size of the blueprint along y axis(float)
    4 extent/size of the blueprint along z axis(float)
    2 + xx
    		 blueprint name (string*)
    2 + xx
    		 creator name (string*)
    2 + xx
    		 creator uid (string*)
    2 + xx
    		 name of world where blueprint was created (string*)



    THUMBNAIL DATA


    Number of bytes Data
    2 thumbnail width (pixels)
    2 thumbnail height (pixels)
    2 + xx
    		 thumnail format (string*), currently always "JPG"
    4 number of bytes of thumbnail image data
    xx thumbnail image data



    SCREENSHOT DATA


    Number of bytes Data
    2 screenshot width (pixels)
    2 screenshot height (pixels)
    2 + xx
    		 screenshot format (string*), currently always "JPG"
    4 number of bytes of screenshot image data
    xx screenshot image data



    BLUEPRINT CONTENT


    Number of bytes Data
    4 compressed blueprint data length (+ 4 bytes for uncompressed len)
    4 uncompressed data length
    xx LZ4 compressed blueprint data



    UNCOMPRESSED BLUEPRINT DATA (LZ4)


    Number of bytes Data
    4 number of construction elements in blueprint
    xx serialized construction data (see below)
    4 number of object elements in blueprint
    xx serialized object data (see below)
    4 number of plants in blueprint
    xx serialized plant data (see below)
    4 terrain array length in blueprint
    xx serialized terrain data



    SERIALIZED CONSTRUCTION DATA (per element)


    Number of bytes Data
    4 relative x position (float)
    4 relative y position (float)
    4 relative z position (float)
    4 rotation x value (quaternion)
    4 rotation y value (quaternion)
    4 rotation z value (quaternion)
    4 rotation w value (quaternion)
    4 size along x (float)
    4 size along y (float)
    4 size along z (float)
    1 construction type id
    4 texture
    4 RGBA color (int) - currently unused
    4 RGBA color (int) - currently unused
    4 RGBA color (int) - currently unused
    4 RGBA color (int) - currently unused
    4 RGBA color (int) - currently unused
    4 RGBA color (int)
    4 surface x offset (float)
    4 surface y offset (float)
    4 surface z offset (float)
    4 surface x scale (float)
    4 surface y scale (float)
    4 surface z scale (float)
    4 optional thickness (float)
    4 texture scale factor (float)
    4 optional additional flags (int bitmask)



    SERIALIZED OBJECT DATA (per element)


    Number of bytes Data
    4 relative x position (float)
    4 relative y position (float)
    4 relative z position (float)
    4 rotation x value (quaternion)
    4 rotation y value (quaternion)
    4 rotation z value (quaternion)
    4 rotation w value (quaternion)
    4 size along x (float)
    4 size along y (float)
    4 size along z (float)
    2 object type id
    4 variation
    4 RGBA color (int)
    1 status
    8
    		 additional object info (long)
    4 optional additional flags (int bitmask)



    SERIALIZED PLANT DATA (per element)


    Number of bytes Data
    4 relative x position (float)
    4 relative y position (float)
    4 relative z position (float)
    4 rotation x value (quaternion)
    4 rotation y value (quaternion)
    4 rotation z value (quaternion)
    4 rotation w value (quaternion)
    4 size along x (float)
    4 size along y (float)
    4 size along z (float)
    2 plant type id
    1
    		 is plant cut?
    4 optional additional flags (int bitmask)



    SERIALIZED TERRAIN DATA


    Number of bytes Data
    4 terrain array size x
    4 terrain array size y
    4 terrain array size z
    2 * size x * size y * size z
    		 terrain data array (short[])


    Terrain data is a flattened 3d array consisting of shorts, where the first 8 bits contain the optional terrain "strength" or density between 0 and 100 (used to apply additional smoothing to the terrain) and the last 8 bits contain the actual terrain ID (0 is air).


    To get the index in the terrain array from an x, y and z value, you can use this code: x + y * sizeX + z * sizeX * sizeY




    I hope I didn't miss anything :D


    * when it comes to strings, the buffer first stores the number of characters as 16 bit int (short), then the individual characters (2 bytes per char). If the string is null, a "-1" is stored instead (again as 16 bit int)

    Looks like I can read files in that format, but there is something that looks wrong. Count of compressed bytes stored in blueprint (LZ4 section) is larger than amount of remaining bytes in loaded data (it is smaller on 4 bytes). I'm not sure why, I don't know, probably this is mistake on my side, but count of loaded bytes is equal to stored count of bytes, so looks like it should be fine... :thinking:. I decided just to lower my buffer array on 4 bytes (if I'm not reading no-element data this should work fine), and I'm not sure if this will have effect on saving files


    Edit: I tested that on 10 blueprints, the difference always exists and is always 4 bytes

    Sorry for the confusion, the int which holds the compressed buffer length includes the 4 next bytes (the uncompressed data length). The reason is that we need the uncompressed length to decompress the data, so this always gets stored in the compressed data buffer (more precisely, we consider the 4 bytes indicating the uncompressed length being part of the compressed data). I've updated my previous post to make this a bit clearer :saint:


    So if the blueprint data is 512 bytes, for example, and it would be 128 bytes compressed, the "uncompressed data length" would be "512" (4 bytes), and the "compressed blueprint data length" would be "132" (128 compressed bytes + 4 bytes for the uncompressed length int).

    FYI: I had made this one a while back ( Blueprint Texture Editor version 0.2 ) but unfortunately haven't got around to updating it as other projects have taken priority for me in the past few years.


    It is written in Java like the old version of the game and the full code is available on my github here: https://github.com/Minotorious/BlueprintTextureEditor (GPL 3.0 Licenced)

    Looks like it works :)


    This is test of fully procedural blueprint with simple primitives.


    Addition: looks like "unused byte" section in elements is filled with RGBA value of element, if this section will be not filled - emissive elements will not have color or glow. Filling it with repeating RGBA values fix issue

    This looks awesome, well done! 8):thumbup:


    Quote from paulevs

    Addition: looks like "unused byte" section in elements is filled with RGBA value of element, if this section will be not filled - emissive elements will not have color or glow. Filling it with repeating RGBA values fix issue

    The "unused" bytes are basically placeholders for individual colors per face (which might get implemented in the future), so right now they're all filled with the main element color value. However, after taking another look at it, the game currently uses the last of these color values - I've updated my previous post accordingly (sorry again for the confusion) :saint:

    After dozens experiments and some math - polygon fill. Something is wrong with normals and UVs on my models, I think this can be a result of my "hacky" triangle fill (I set element size to zero on two cardinal axis and manipulate offsets)

    Thank you very much :)

    I have a small note - terrain data stores array capacity between 3 array dimensions and actual terrain data, so most my attempts to write data looked like this before I found the reason:


    So, the terrain data looks like this:

    4 bytesInt size X
    4 bytesInt size Y
    4 bytesInt size Z
    4 bytesArray Capacity
    2 * X * Y * Z bytesData


    Also looks like terrain data is stored with [id-strength] order instead of [strength-id] order (which I noticed during test output), probably this is related to stream order


    Now the last thing that I need to configure is array index order ;)

Participate now!

Don’t have an account yet? Create a new account now and be part of our community!

Create Account Login