Zlib Compress

Decode

Synopsis

Compresses string data using zlib algorithm and returns base64-encoded output compatible with Kusto's zlib_compress_to_base64_string() function.

Schema

- zlib_compress:
    field: <ident>
    target_field: <ident>
    level: <integer>
    description: <text>
    disabled: <boolean>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y		Source field containing data to compress
target_field	N	Same as field	Target field to store compressed result
level	N	-1	Compression level: -1 (default), 0 (no compression), 1 (best speed) to 9 (best compression)
description	N		Explanatory note
disabled	N	false	Disable processor execution
if	N		Condition to run
ignore_failure	N	false	Continue processing if compression fails
ignore_missing	N	false	Skip processing if referenced field doesn't exist
on_failure	N		See Handling Failures
on_success	N		See Handling Success
tag	N		Identifier

Details

Compresses field values using zlib compression algorithm and encodes the result to base64 format. The output is compatible with Azure Data Explorer (Kusto) zlib_compress_to_base64_string() function for cross-platform data processing.

The processor converts non-string values to string representation before compression. This includes numbers, booleans, and other data types that are automatically formatted as strings.

Compression level parameter controls the trade-off between compression speed and compression ratio. Level -1 uses default compression (equivalent to level 6). Level 0 disables compression while preserving the zlib format. Levels 1-9 provide increasing compression ratios at the cost of processing time.

The processor uses Go's standard compress/zlib package which implements RFC 1950 (ZLIB Compressed Data Format Specification). Compressed output includes zlib headers and checksums for data integrity verification.

Empty strings produce valid compressed output containing only zlib headers. The processor validates compression level range and returns an error if the level is outside the valid -1 to 9 range.

Default target field behavior: if not specified, overwrites the source field with compressed result.

Examples

Basic String Compression

Compressing message field to separate target...	{ "message": "1234567890qwertyuiop" } - zlib_compress: field: message target_field: compressed_message
Creates base64-encoded compressed output...	{ "message": "1234567890qwertyuiop", "compressed_message": "eJwzMjI2MTU1szAyNjE1NbewMDKxMDQwMDYxAADG/wl9" }

In-Place Compression

Compressing field without specifying target...	{ "message": "Hello World" } - zlib_compress: field: message
Overwrites source field with compressed value...	{ "message": "eJzzSM3JyddRKMlIVQAAE0kDMQ==" }

Compression Level Control

Using maximum compression for repetitive data...	{ "data": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" } - zlib_compress: field: data target_field: compressed level: 9
Best compression achieves smallest output...	{ "data": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", "compressed": "eJxLSkoCABIuAiU=" }

Fast Compression

Using fastest compression for high-throughput processing...	{ "log_message": "Error occurred in module authentication" } - zlib_compress: field: log_message target_field: compressed_log level: 1
Produces larger output but faster processing...	{ "log_message": "Error occurred in module authentication", "compressed_log": "eJxLzs8rSc0rUUjOyCxWSEksSVVIK8rPVUjMS1EoLinKzEvnAgCqmw6W" }

Unicode Content

Compressing text with special characters...	{ "user_message": "Hello 世界 🌍" } - zlib_compress: field: user_message target_field: compressed
Handles Unicode text correctly...	{ "user_message": "Hello 世界 🌍", "compressed": "eJzzSM3JyddRCEpNziwuUQIArXIHAA==" }

JSON Payload Compression

Compressing JSON strings for storage efficiency...	{ "payload": "{\"name\":\"John\",\"age\":30,\"city\":\"New York\"}" } - zlib_compress: field: payload target_field: compressed_payload
Reduces JSON payload size...	{ "payload": "{\"name\":\"John\",\"age\":30,\"city\":\"New York\"}", "compressed_payload": "eJyqVkpUslJQKkktLlGyUlAqSi0uzszPU9JRSCwuAQoZGgJRDVC1RkC1iUA1JUCFsQC8nBSg" }