pitbottom

.rsv Format

Tags:

Motivation

Roody:2d has infinite procedural generation, so the world is split up into 64x64 tile chunks.

Before version 0.7.0 alpha, the data for each chunk was saved into a .sac file. Each tile was 4 bytes, so the files were each 16 KB (4 * 64 * 64 + some metadata).

version 0.6.1 alpha
main menu background world

16K     c-1_-1.sac
16K     c-2_-1.sac
16K     c-3_-1.sac
16K     c0_-1.sac
16K     c1_-1.sac
16K     c2_-1.sac
16K     c3_-1.sac
4K      world.ssd

This was very simple to implement, but had 2 problems:

  • World saves quickly became hundreds of individual files, constantly burdening the OS file system
  • No compression whatsoever, even for chunks that were just empty air
  • Can’t save dynamic sizes of data, like monsters spawning and dying, or a tile that includes a string.

This is why I created the .rsv format, introduced in version 0.7.0 alpha.

A single .rsv file can hold a 32x32 region of compressed chunks. This is large enough that many Roody:2d save files now only contain a few individual files for many more chunks. Since the player always spawns near 0,0. it is common for a save folder to only contain 4 rsv files, one for each quadrant.

version 0.10.15 beta
main menu background world

8.0K    r0_0.rsv
24K     r0_-1.rsv
8.0K    r-1_0.rsv
12K     r-1_-1.rsv
4.0K    world.smp

Format

Common Types

struct Tile {
    uint8_t byteA; // tile type, like stone, copper_bar, actuator_head, or air
    uint8_t byteB; // welding and movement
    uint8_t byteC; // tile-type defined
    uint8_t byteD; // tile-type defined
    uint8_t byteE; // either stack-count, OR 7 bits of light level and a 1 bit flag for actively being extracted/heated
//tiles have 3 more bytes for various gameplay algorithms, but since these always get reset to 0 at the end of the algorithm, they aren't saved.
};
struct regionV2D {  // region-scale "vector 2 dimensional"
    int64_t x;      // one unit of regionV2D is 32 chunkV2D units
    int64_t y;      // the file names of .rsv files use these coordinates
};
struct chunkV2D {   // chunk-scale "vector 2 dimensional"
    int64_t x;      // one unit of chunkV2D is 64 tileV2D units
    int64_t y;
};
struct tileV2D {    // tile-scale "vector 2 dimensional"
    int64_t x;      // one unit of tileV2d is 16 pixels
    int64_t y;      // tileV2D is the most commonly used scale. It is the gameplay grid of Roody:2d
};
// other smaller scale V2D units exist for entities and pixels. One unit of pixelV2D is 16 entityV2D units. These aren't important for the .rsv format

Region Data

The header for a .rsv

struct Region_Header {
    int32_t version;            // 1

    //region offsets are relative to here

    uint32_t file_size;         // total region file size, including this header
    char rsv[4] = {'.', 'r','s','v'}; // padding

    regionV2D position;         // absolute coordinates of region

    uint32_t locations_offset;  // start of the chunk-location table in the file, relative to the start of the region data, including this header, but NOT including the 4 byte version
    uint32_t locations_size;    // size  of the chunk-location table in the file

    uint32_t chunks_offset;     // start of chunk data pool, relative to the start of the region data, including this header, but NOT including the 4 byte version
    uint32_t chunks_size;       // size  of chunk data pool
};

The chunk-location table is an array of chunk coordinates and where that chunk’s data is in the chunk data pool.

Accessing region_data + static_cast<Region_Header*>(region_data)->locations_offset gives the start of this contiguous Region_Chunk_location array.

struct Region_Chunk_location {
    chunkV2D position; // relative to the region position
    uint32_t offset;   // relative to the region chunks_offset
};
static_assert(sizeof(Region_Chunk_location) == 24);

There can be anywhere from 1 to 32*32 chunks in a region. The number can be found with (locations_size - locations_offset) / sizeof(Region_Chunk_location).

Chunk Data

Accessing region_data + static_cast<Region_Header*>(region_data)->chunks_offset + region_chunk_location.offset gives the start of chunk data, begining with the Chunk_Header.

struct Chunk_Header {
    uint32_t size;                      // total chunk data size, including this header
    char chnk[4] = {'c','h','n','k'};   // padding

    chunkV2D position;                  // absolute coordinates of chunk. This is redundant when using Region_Header.position and Region_Chunk_location.position

    uint32_t generation_stage;          // Chunks are only saved if they reach the final generation stage, so this should always be the same value.
    uint32_t last_randomTick_timestamp; // used to make plants instantly grow if a chunk has been unloaded while the rest of the world has been loaded for some time

    uint32_t tilegrid_offset;           // start of the tilegrid data, relative to the start of the region data, including this header
    uint32_t tilegrid_size;             // size  of the tilegrid data

    uint32_t tile_dynamic_data_offset;  // start of the tile_dynamic data, relative to the start of the region data, including this header
    uint32_t tile_dynamic_data_size;    // size  of the tile_dynamic data. This dynamic tile data is a single human-readable askii-string, with individual tile_dynamic inside [] brackets

    uint32_t entities_offset;           // start of the entity data, relative to the start of the region data, including this header
    uint32_t entities_size;             // size  of the entity data. This dynamic tile data is a single human-readable askii-string, with individual entities inside [] brackets
};

TileGrid Data

Accessing chunk_data + static_cast<Chunk_Header*>(chunk_data)->tilegrid_offset gives the start of the TileGrid data. The TileGrid is a compressed form of a 2d array of 5 byte tiles.

struct TileGrid_Header {
    uint32_t size;                      // total tilegrid data size, including this header
    char tile[4] = {'t','i','l','e'};   // padding
    tileV2D bounds;                     // always 64, 64

    uint32_t byteA_offset;              // offset is relative to this header
    uint32_t byteA_size;
    uint32_t byteB_offset;
    uint32_t byteB_size;
    uint32_t byteC_offset;
    uint32_t byteC_size;
    uint32_t byteD_offset;
    uint32_t byteD_size;
    uint32_t byteE_offset;
    uint32_t byteE_size;
    uint32_t byteF_offset_spacer = 0;   // unused, should always be 0
    uint32_t byteF_size_spacer = 0;
    uint32_t byteG_offset_spacer = 0;
    uint32_t byteG_size_spacer = 0;
    uint32_t byteH_offset_spacer = 0;
    uint32_t byteH_size_spacer = 0;
    uint32_t byteI_offset_spacer = 0;
    uint32_t byteI_size_spacer = 0;
};

Since the different bytes of a tile define different aspects, such as type, welded state, or light levels, it is more likely to get repeating bytes when each byte is separated into byte-layers. byteA_size is not 64x64, because each of these byte-layers is ran through a RLE compression. I have open sourced this specific RLE compression.