Skip to content

dtchunker

Chunked buffer export and import with reassembly.

This group of functions provides chunking and reassembly for byte buffers. It is intended for streaming fixed-size payloads across transports with size limits. It preserves ordering and total length across sessions. The implementation does simple state tracking and zero-copy slices where possible.

Mini-guide

  • Initialize once per session by calling dtchunker_init with a valid maximum chunk size.
  • Choose a mode implicitly by calling either the export or import path first.
  • Emit a header chunk first during export, then consume payload chunks sequentially.
  • Provide chunks in order during import until completion is reported.
  • Reset or dispose between independent sessions to clear internal state.

Example

dtchunker_t chunker;
dtbuffer_t* chunk = NULL;
dtbuffer_t* final = NULL;

dtchunker_init(&chunker, 256);

while (dtchunker_export(&chunker, source, &chunk) == NULL && chunk != NULL)
{
    send_chunk(chunk);
}

dtchunker_reset(&chunker);

while (receive_chunk(&chunk))
{
    dtchunker_import(&chunker, chunk, &final);
    if (final != NULL)
    {
        use_buffer(final);
        dtbuffer_dispose(final);
        break;
    }
}

dtchunker_dispose(&chunker);

Data structures

dtchunker_header_t

Describes the serialized header for a chunked stream.

Members:

int32_t magic Magic value identifying the wire format.

int32_t source_buffer_length Total length of the original source buffer in bytes.

dtchunker_mode_t

Defines the current operating mode of a chunker instance.

Members:

DTCHUNKER_MODE_EXPORT Producing chunks from a source buffer.

DTCHUNKER_MODE_IMPORT Reassembling chunks into a destination buffer.

DTCHUNKER_MODE_UNKNOWN Mode not yet selected.

dtchunker_t

Holds internal state for chunk export or import sessions.

Members:

dtbuffer_t* working_buffer Internal heap buffer for payload slices and reassembly.

dtbuffer_t wrapped_buffer Wrapper pointing into internal storage for zero-copy returns.

int32_t working_index Current byte index within the source or target buffer.

dtchunker_mode_t mode Current operating mode.

int32_t max_chunk_size Maximum payload bytes per exported chunk.

int32_t header_length Serialized header length in bytes.

Macros

DTCHUNKER_MAGIC

Identifies the expected wire format for chunked streams.

Functions

dtchunker_dispose

Releases internal resources and clears the instance state.

Params:

dtchunker_t* self Chunker instance to dispose.

Return: void No return value.

dtchunker_export

Produces the next header or payload chunk from a source buffer.

Params:

dtchunker_t* self Chunker instance managing the export session.

dtbuffer_t* source_buffer Source buffer to split into chunks.

dtbuffer_t** chunk_buffer Output pointer receiving the current chunk or NULL when complete.

Return: dterr_t* Error status or NULL on success.

dtchunker_import

Consumes a header or payload chunk and assembles the final buffer.

Params:

dtchunker_t* self Chunker instance managing the import session.

dtbuffer_t* chunk_buffer Incoming chunk buffer to process.

dtbuffer_t** final_buffer Output pointer receiving the completed buffer when finished.

Return: dterr_t* Error status or NULL on success.

dtchunker_init

Initializes a chunker instance with a maximum chunk size.

Params:

dtchunker_t* self Chunker instance to initialize.

int32_t max_chunk_size Maximum payload size per exported chunk in bytes.

Return: dterr_t* Error status or NULL on success.

dtchunker_reset

Resets internal state while keeping allocated buffers for reuse.

Params:

dtchunker_t* self Chunker instance to reset.

Return: dterr_t* Error status or NULL on success.