LZ4 Compression

Wraps the LZ4 compression library using c2py23. Demonstrates uint8 buffer handling with dynamic output sizing.

Lz4 Wrap

Interface

# Python dict format equivalent of lz4.c2py
{
    "module": "lz4mod",
    "source": [
        "../lz4/lib/lz4.c",
        "lz4_thin.c",
    ],
    "headers": ["../lz4/lib/lz4.h"],
    "functions": [
        {
            "py_sig": "compress(src: buffer, dst: buffer) -> int",
            "checks": [
                "src.format == 'B'",
                "dst.format == 'B'",
                "src.ndim == 1",
                "dst.ndim == 1",
            ],
            "c_overloads": [
                {
                    "sig": "int lz4_compress(const uint8_t *src, uint8_t *dst, int srcSize, int dstCapacity)",
                    "map": {
                        "src": "src.ptr",
                        "dst": "dst.ptr",
                        "srcSize": "src.len",
                        "dstCapacity": "dst.len",
                    },
                },
            ],
        },
        {
            "py_sig": "decompress(src: buffer, dst: buffer) -> int",
            "checks": [
                "src.format == 'B'",
                "dst.format == 'B'",
                "src.ndim == 1",
                "dst.ndim == 1",
            ],
            "c_overloads": [
                {
                    "sig": "int lz4_decompress(const uint8_t *src, uint8_t *dst, int compressedSize, int dstCapacity)",
                    "map": {
                        "src": "src.ptr",
                        "dst": "dst.ptr",
                        "compressedSize": "src.len",
                        "dstCapacity": "dst.len",
                    },
                },
            ],
        },
    ],
}

C Source

#include <stdint.h>
#include "../lz4/lib/lz4.h"

int lz4_compress(const uint8_t *src, uint8_t *dst, int srcSize, int dstCapacity) {
    return LZ4_compress_default((const char *)src, (char *)dst, srcSize, dstCapacity);
}

int lz4_decompress(const uint8_t *src, uint8_t *dst, int compressedSize, int dstCapacity) {
    return LZ4_decompress_safe((const char *)src, (char *)dst, compressedSize, dstCapacity);
}

Build

$ c2py23 lz4.c2py -o lz4mod_wrapper.c

Compile:

$ cc -shared -fPIC c2py23/runtime/c2py_runtime.c lz4mod_wrapper.c lz4_thin.c -I c2py23/runtime -o lz4mod.so -ldl -lm

See docs/building for cmake, meson, and setuptools options.

Run

$ python example.py
Traceback (most recent call last):
  File "/home/runner/work/c2py23/c2py23/examples/lz4_wrap/example.py", line 13, in <module>
    import lz4mod
ModuleNotFoundError: No module named 'lz4mod'

How It Works

uint8 buffers

LZ4 operates on char* data. The thin wrapper casts const uint8_t* to const char* and back. The format check is 'B' (uint8).

Dynamic output size

compress returns the actual compressed byte count. The Python caller reads this and slices the destination buffer to the actual size before passing to decompress.