Syslog

Synopsis

Creates a Syslog server that accepts log messages over UDP or TCP connections. Supports both plain and TLS-encrypted connections, with configurable framing and buffering options.

For details, see Appendix.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: syslog
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    protocol: <string>
    address: <string>
    port: <numeric>
    framing: <string>
    pattern: <string>
    line_delimiter: <string>
    framing_rules:
      - name: <string>
        condition: <string>
        pattern: <string>
        max_event_bytes: <numeric>
        min_raw_length: <numeric>
    max_connections: <numeric>
    timeout: <numeric>
    tls:
      status: <boolean>
      cert_name: <string>
      key_name: <string>
      min_version: <string>
      insecure_skip_verify: <boolean>
    reuse: <boolean>
    workers: <numeric>
    buffer_size: <numeric>
    max_message_size: <numeric>
    batch_size: <numeric>
    forwarding:
      - address: <string>
        port: <numeric>
        type: <string>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y		Unique identifier
name	Y		Device name
description	N	-	Optional description
type	Y		Must be syslog
tags	N	-	Optional tags
pipelines	N	-	Optional pre-processor pipelines
status	N	true	Enable/disable the device

Protocol

Field	Required	Default	Description
protocol	N	"udp"	Transport protocol (udp or tcp).
address	N	"0.0.0.0"	Listen address
port	Y		Listen port

TCP

The following are only applicable when protocol is set to tcp.

Field	Required	Default	Description
framing	N	"delimiter"	Framing mode for TCP (delimiter, octet, regex, or advanced)
pattern	Y*	-	Event-breaker regex pattern; required when framing is regex
line_delimiter	N	"\n"	Line separator for TCP delimiter framing

* = Required when framing is regex

TLS

The following are only applicable when protocol is set to tcp.

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption
tls.cert_name	Y*		TLS certificate. Accepts a file name (resolved relative to the service root directory) or inline PEM content (when the value starts with -----BEGIN).
tls.key_name	Y*		TLS private key. Same value semantics as tls.cert_name.
tls.min_version	N	"1.2"	Minimum TLS version ("1.0", "1.1", "1.2", "1.3")
tls.insecure_skip_verify	N	false	Skip peer certificate verification. Use only for testing.

* = Required when tls.status is true.

Advanced Configuration

To enhance performance and achieve better data handling, the following settings are used.

Performance

Field	Required	Default	Description
reuse	N	true	Enable socket address reuse
workers	N	CPU count	Number of worker processes when reuse is enabled. Capped at the number of physical cores.
max_connections	N	10000	Maximum concurrent TCP connections
max_message_size	N	20971520	Maximum message size in bytes (20MB)
timeout	N	300	Connection timeout in seconds
buffer_size	N	9000	Network read buffer size in bytes
batch_size	N	10000	Number of messages to batch before flushing

note

flush_interval and queue.interval are Director service-level settings configured in vmetric.yml and cannot be overridden per device.

Forwarding

Field	Required	Default	Description
forwarding[].address	Y		Forward destination address
forwarding[].port	N	514	Forward destination port
forwarding[].type	N	"udp"	Forward protocol (udp or tcp)

Framing Rules

Ordered event-breaking rules for TCP connections, used when framing is set to advanced.

At connection open, the first min_raw_length bytes are buffered. The first rule whose condition matches the buffered bytes is selected for the lifetime of that connection. The last rule should have an empty condition to act as the unconditional fallback. All rules use regex-based event breaking.

Field	Required	Default	Description
framing_rules[].name	N	"rule-N"	Descriptive rule name for logs
framing_rules[].condition	N	-	Regex matched against initial bytes to select this rule; empty = unconditional
framing_rules[].pattern	Y	-	Event-breaker regex marking the start of each event
framing_rules[].max_event_bytes	N	max_message_size	Per-rule event size cap in bytes; falls back to device-level max_message_size
framing_rules[].min_raw_length	N	256	Minimum bytes to buffer before evaluating condition

note

Framing rules only apply when protocol is tcp. Regex framing is event-start oriented: each regex match marks the beginning of a new event. Everything between consecutive matches is one complete event. The pattern must not match the empty string.

Examples

The following are commonly used configuration types.

Basic

Creating a simple UDP syslog server...

devices: - id: 1 name: basic_syslog type: syslog properties: port: 514

Checkpoint

The basic UDP Server can be configured to use a checkpoint pre-processing pipeline. This is a pre-processing pipeline that extracts Checkpoint firewall logs from syslog messages:

Creating a simple UDP syslog server with checkpoint...

devices: - id: 2 name: basic_syslog type: syslog tags: - "network_device" pipelines: - checkpoint properties: address: "10.0.0.1" protocol: "udp" port: 1514

note

If the device is a Checkpoint firewall, this pipeline will parse the logs and extract relevant fields for further processing. Otherwise, the pipeline will have no effect on the incoming messages.

High-Volume

Tuning a UDP server for high message volumes...

devices: - id: 3 name: performant_syslog type: syslog properties: protocol: udp port: 514 reuse: true workers: 4 buffer_size: 32768 batch_size: 5000

note

The worker count is automatically capped at the maximum number of physical cores available on the system.

Framing

TCP server with custom message framing, connection limits, and an idle timeout...

devices: - id: 4 name: tcp_syslog type: syslog properties: protocol: tcp port: 1514 framing: delimiter line_delimiter: "\r\n" max_connections: 5000 timeout: 60

warning

When using TCP with delimiter framing, ensure the line_delimiter matches the client side.

Advanced Framing

TCP syslog server with per-connection event-breaking rules, selected from the initial bytes...

devices: - id: 5 name: advanced_syslog type: syslog properties: protocol: tcp port: 1514 framing: advanced framing_rules: - name: multiline-timestamp condition: "^\\d{4}-" pattern: "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}" - name: fallback pattern: "\\n"

Security

Securing the server with TLS encryption and forwarding to mixed destinations...

devices: - id: 6 name: secure_syslog type: syslog properties: protocol: tcp port: 6514 tls: status: true cert_name: cert.pem key_name: key.pem forwarding: - address: "10.0.0.1" port: 514 type: udp - address: "10.0.0.2" port: 6514 type: tcp

Forwarding

Forwarding replicates incoming messages unmodified to all configured destinations. This is useful for network devices that can only send syslog data to a single destination.

Fan-out: UDP messages received on port 514 are replicated to one UDP and two TCP destinations...

devices: - id: 7 name: forwarder_syslog type: syslog properties: protocol: udp port: 514 forwarding: - address: "10.0.0.50" port: 514 type: udp - address: "10.0.0.51" port: 1514 type: tcp - address: "syslog.example.com" port: 6514 type: tcp

warning

When using TCP forwarding, ensure the destination servers can handle the connection load as each connection is persistent.