GX2 Format

From ModdingWiki
Jump to: navigation, search
GX2 Format
GX2 Format.png
Format typeImage
HardwareVGA
Colour depth8-bit (VGA)
Minimum size (pixels)0×0
Maximum size (pixels)65535×65535
PaletteInternal
Plane count1
Transparent pixels?No
Hitmap pixels?No
Games

GX2 is an image format used in the Interactive Girls Club series of adult games. It is typically found inside archives of the SLB/M3 format.

File format

The format has a header followed by image data. The image data uses two compression methods: a rather straightforward Code-based RLE, and a bit masks system that copies pixels from the previous row.

Header

The file starts with the following header. Note that all known images of this format are 8-bit and 320x200 in size, so without further research, no assumptions can be made on the effect of changing the values that determine that.

Strangely, the palette in this format is in 8-bit VGA format, whereas the palettes embedded in these same games' DMP Format are 6-bit.

Offset Data type Name Description
0x00 UINT32LE Magic1 Magic number: the string "GX2" followed by byte 0x01. As a single UInt32, the value is 0x01325847
0x04 UINT16LE HeaderSize Always 0x19. Possibly related to the header size, though the header is 0x1B with the magic number, and 0x17 without it.
0x06 BYTE Bpp Bits per pixel.
0x07 UINT16LE Width Image width.
0x09 UINT16LE Height Image height.
0x0B UINT16LE AspectX These values are always respectively 4 and 3, possibly indicating a 4:3 aspect ratio.
0x0D UINT16LE AspectY
0x0F BYTE Unknown1 Unknown. Always 0x00.
0x10 UINT16LE Subhsize Unknown. Always 0x09. Might be a sub-header size.
0x12 UINT32LE Magic2 Magic number: the string "SPFX". As a single UInt32, the value is 0x58465053
0x16 UINT16LE Unknown2 Unknown. Always 0x0F.
0x18 BYTE Unknown3 Unknown. Always 0x00.
0x19 UINT16LE Unknown4 Unknown. Always 0x02.
0x1B BYTE[256*3] Palette Color palette, in 8-bit RGB VGA format.

Image Data

The image data comes after the header, at offset 0x31B. The image data is compressed twice: first with a bit-mask system that removes pixels that are duplicated vertically, and then with a run-length encoding. So, to decompress, these two operations need to be applied in opposite order.

The end result of the whole decompression operation is a simple byte array to be interpreted as 8-bit indexed image.

Compression

Run-Length Encoding

The data in the file is compressed with a classic Code-based RLE where the high bit indicates a Repeat command, the amount to copy or repeat is the Code byte with the highest bit removed, and the value to repeat is a single byte behind the Code.

Bit-Mask Encoding

The data you get after decompressing the RLE data has a rather peculiar format. The first row of pixels can be read normally, but after that, each row is preceded by a bit mask (1/8th of the image stride) that stores for each byte in the next row of pixels whether the byte should be read from the RLE-uncompressed data, or copied from the previous row. Indices for which the corresponding bit value is 1 will read one byte from the RLE-uncompressed data, advancing the read pointer. Those with a bit value of 0 will not advance the read pointer, and should instead take their data from the same index on the previous row of the already-decoded image data.

The reason for this second compression seems to be that it greatly reduces the uniform noise of colour dithering in the image data, and transfers that noise to another place. The operation very often reduces the actual data of a new line to pixels which all have the same value, and the dithering itself is comprised of repeating patterns that are often divisible by eight pixels, meaning the mask bytes generated from such dithered content tend to also be repeating values. Both of these factors result in much better RLE compression rates.

To indicate the amount of copied data, this is the title image with all pixels with a 0-bit filled in with red:

GX2 Format without fill.png

The detailed view of the dithering seen below shows that in many rows, the remaining pixels on a single line (indicated by the red box) are all of a single colour. The left-hand ruler shows that this applies to 28 of the shown 40 rows (without the top row; it's not part of the repeating pattern), which means that if the image were to be made up of only this dithered pattern, 70% of the lines in the image can be compressed to just one or two (given the 127-repeat limit) RLE commands for the whole image width. Only the pieces that form checkerboard patterns remain uncompressable.

As can be seen from the ruler at the bottom, the pattern repeats in blocks of eight pixels, meaning that the bit-mask bytes will be the same for the repeating pattern across the whole row.

GX2 Format without fill detail.png

Code

Compression and decompression code for the RLE algorithm can be found in the RLE article; the RleCompressionHighBitRepeat covers this case exactly.

The bit-masks handling should be done something like this:

Decompression (C#)

This code was written by Nyerguds for the Engie File Converter, and is released under the WTF Public License.

/// <summary>
/// Decodes the bit-mask based compression of the Interactive Girls Club images.
/// </summary>
/// <param name="bitMaskData">Image data with bit masks.</param>
/// <param name="stride">Amount of bytes in one pixel row in the image.</param>
/// <param name="height">Height of the image.</param>
/// <returns>The uncompressed stride*height image data.</returns>
public static Byte[] BitMaskDecompress(Byte[] bitMaskData, Int32 stride, Int32 height)
{
    Int32 inputLen = bitMaskData.Length;
    if (inputLen < stride)
        throw new ArgumentException("Not enough data to decompress image.", "bitMaskData");
    Int32 outputLen = stride * height;
    Byte[] imageData = new Byte[outputLen];
    Int32 maskLength = (stride + 7) / 8;
    // Copy first row to imageData
    Array.Copy(bitMaskData, 0, imageData, 0, stride);
    // Set pointers to initial values after the first row.
    Int32 prevRowPtr = 0;
    Int32 writePtr = stride;
    Int32 inPtr = stride;
    for (Int32 y = 1; y < height; ++y)
    {
        if (inputLen < inPtr + maskLength)
            throw new ArgumentException("Error decompressing image.", "bitMaskData");
        // Set start of mask.
        Int32 bitmaskPtr = inPtr;
        // Set start of data.
        inPtr += maskLength;
        for (Int32 x = 0; x < stride; ++x)
        {
            // Check bit in bit mask. Upshift and check 0x80 because the bits are in big-endian order.
            if (((bitMaskData[bitmaskPtr + x / 8] << (x & 7)) & 0x80) != 0)
            {
                if (inPtr >= inputLen)
                    throw new ArgumentException("Error decompressing image.", "bitMaskData");
                // Copy from RLE-uncompressed data
                imageData[writePtr] = bitMaskData[inPtr++];
            }
            else
            {
                // Copy from previous row.
                imageData[writePtr] = imageData[prevRowPtr + x];
            }
            writePtr++;
        }
        prevRowPtr += stride;
    }
    return imageData;
}

Compression (C#)

This code was written by Nyerguds for the Engie File Converter, and is released under the WTF Public License.

/// <summary>
/// Encodes to the bit-mask based compression of the Interactive Girls Club images.
/// </summary>
/// <param name="imageData">Image data.</param>
/// <param name="stride">Amount of bytes in one pixel row in the image.</param>
/// <param name="height">Height of the image.</param>
/// <returns>The compressed image data with added bit masks.</returns>
public static Byte[] BitMaskCompress(Byte[] imageData, Int32 stride, Int32 height)
{
    Int32 inputLen = stride * height;
    if (inputLen > imageData.Length)
        throw new NotSupportedException("Error compressing image: array too small to contain an image of the given dimensions!");
    Int32 maskLength = (stride + 7) / 8;
    // Worst case: no duplicate pixels at all, means original size plus (height - 1) masks.
    Int32 outputLen = inputLen + maskLength * (height - 1);
    Byte[] imageDataCompr = new Byte[outputLen];
    // Copy first row to imageData
    Array.Copy(imageData, 0, imageDataCompr, 0, stride);
    // Set pointers to initial values after the first row.
    Int32 prevRowPtr = 0;
    Int32 inPtr = stride;
    Int32 writePtr = stride;
    for (Int32 y = 1; y < height; ++y)
    {
        // Set start of mask.
        Int32 bitmaskPtr = writePtr;
        // Set start of data.
        writePtr += maskLength;
        for (Int32 x = 0; x < stride; ++x)
        {
            Byte val = imageData[inPtr + x];
            // If identical, do nothing; mask is left on 0, data is not added.
            if (imageData[prevRowPtr + x] == val)
                continue;
            // If new data, set mask bit, and write value. Downshift 0x80 because the bits are in big-endian order.
            imageDataCompr[bitmaskPtr + x / 8] |= (Byte) (0x80 >> (x & 7));
            imageDataCompr[writePtr++] = val;
        }
        prevRowPtr += stride;
        inPtr += stride;
    }
    Byte[] finalData = new Byte[writePtr];
    Array.Copy(imageDataCompr, 0, finalData, 0, writePtr);
    return finalData;
}

Tools

The following tools are able to work with files in this format.

Name PlatformView images in this format? Convert/export to another file/format? Import from another file/format? Access hidden data? Edit metadata? Notes
Engie File Converter WindowsYesYesYesN/AN/A Can also load SLB/M3 files and show the GX2 images inside them.