Virtual Tables
A pipeline entry normally carries its fields in an untyped map — flexible, but every read, write, and marshal pays for that flexibility with hashing, boxing, and reflection. Virtual tables give a pipeline branch an alternative: a fixed, strongly typed struct that a handful of processors write into explicitly, and that the entry can marshal from instead of the map. On a hot fan-out leaf — parse once, set a known set of fields, ship to several destinations — switching from map writes to a virtual table measures ~1.6x faster end to end and ~3.2x faster at the marshal step alone. Marshal allocations drop 12x (2 vs 24 per event). Timing ratios move with machine load. The allocation counts are exact and reproduce identically on any machine, because they come from removing reflection and hash-map growth, not CPU speed.
Virtual tables are strictly opt-in, and nothing about them is automatic. There is no migration from map to table in either direction. There is no "active schema" that bare field names silently start routing into once a table exists. There is no switch that quietly discards a table's contents behind the pipeline author's back. The engine does exactly what the pipeline writes, and nothing else:
- A table exists on an entry only after a
create_tableprocessor runs on it. - A field lands in a table only when a processor writes it there through an explicit
$<table>.Fieldpath. - An entry marshals from a table only after a
use_tableprocessor selects it as the output source.
Adopting virtual tables in one branch of one pipeline never affects any other branch, any other pipeline, or the untyped map fields already in use everywhere else, because every one of those three steps is explicit.
The Model
Four rules describe the whole feature:
- Tables don't exist until created.
create_tableallocates a table's backing struct on the entry. Before that,$<table>.Fieldpaths targeting it are the same as any other uncreated resource — reads resolve to nothing, writes fail loudly. - Fields land only through explicit
$<table>.Fieldwrites. A bare field name —HostName,log.syslog.hostname— always targets the untyped map, exactly as it did before virtual tables existed, whether or not a table happens to be alive on the entry and whether or not the table has a same-named column. - Output switches only through
use_table. Nothing marshals from a table until ause_tableprocessor says so. Selecting a table doesn't touch the map's contents — it only changes what the entry serializes at output. - The map always coexists. Creating, writing to, and even selecting a table never clears the map. Multiple tables can also be alive on the same entry simultaneously (rare, but supported) — each addressed independently by its own
$<table>prefix.
The lifecycle in one table:
| Stage | Triggered by | Effect |
|---|---|---|
| Created | create_table | Backing struct allocated on the entry; every column starts absent |
| Written | set / rename / syslog with table: / etc. targeting $table.Field | Value coerced into the column's type and stored; presence bit set |
| Selected | use_table | The entry's MarshalJSON (and reroute's staged payload) switches to serializing this table |
| Cleared | truncate_table | Every column value and presence bit reset; the table stays allocated for further writes |
| Removed | drop_table | Backing struct discarded; if it was selected, output reverts to the map |
A table addressed in an if: condition or a get-style read does not need to be the selected output table — it only needs to have been created. Selection (use_table) is purely about what ships at marshal time, not about what is readable.
Because a leading $ in a field path is reserved for table addressing, a literal $-prefixed key in the map is reachable only through the condition engine's bracket syntax ($env["$weird_key"]), never through a plain dotted field path.
Supported Tables
Twelve tables are registered, all mirroring Microsoft Sentinel schemas — the Syslog and CommonSecurityLog ingestion tables, plus ten Advanced Security Information Model (ASIM) normalized tables. These are the same normalized shapes discussed in Normalization and monitored by Schema Drift Detection — virtual tables give you a typed, high-throughput way to produce payloads in those shapes, on top of the untyped normalization path that already exists.
| Table | Columns |
|---|---|
Syslog | 13 |
CommonSecurityLog | 156 |
ASimAuditEventLogs | 123 |
ASimAuthenticationEventLogs | 126 |
ASimDhcpEventLogs | 95 |
ASimDnsActivityLogs | 135 |
ASimFileEventLogs | 122 |
ASimNetworkSessionLogs | 142 |
ASimProcessEventLogs | 136 |
ASimRegistryEventLogs | 81 |
ASimUserManagementActivityLogs | 108 |
ASimWebSessionLogs | 145 |
Table names and column names are both case-insensitive everywhere they're referenced — in create_table/use_table/drop_table/truncate_table's name: field, in $table.Field write paths, and in if: conditions. $Syslog.computer, $syslog.Computer, and $SYSLOG.COMPUTER all resolve to the same column. Exact spelling is matched first, with a case-folded lookup as the fallback, so pick one spelling and use it consistently for readability even though the engine doesn't require it.
Syslog is the smallest table and the one used throughout this page as the worked example. Its 13 columns:
| Column | Type |
|---|---|
Computer | string |
EventTime | string |
Facility | string |
HostIP | string |
HostName | string |
ManagementGroupName | string |
ProcessID | int |
ProcessName | string |
SeverityLevel | string |
SourceSystem | string |
SyslogMessage | string |
TimeCollected | string |
TimeGenerated | string |
ProcessID is the only non-string column on Syslog; every other field, including the two timestamp-shaped columns EventTime and TimeGenerated, is stored as a string. CommonSecurityLog and the ASIM tables follow the same shape — a fixed set of typed columns, one storage kind per column — just with far more of them.
The Four Processors
Four processors form the entire lifecycle. All four share the same scaffold — name, description, disabled, if, ignore_failure, on_failure, on_success, tag — and none of them has an ignore_missing field, since there's no notion of a "missing" table separate from an unknown one.
Create Table
Allocates the named table's backing struct on the entry so $<table>.Field paths can address it.
- create_table:
name: <ident>
description: <text>
disabled: <boolean>
if: <script>
ignore_failure: <boolean>
on_failure: <processor[]>
on_success: <processor[]>
tag: <string>
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | - | Table to allocate; one of the Supported Tables (case-insensitive); templated |
description | N | - | Explanatory note |
disabled | N | false | Disable the processor without removing it from the pipeline |
if | N | - | Condition to run |
ignore_failure | N | false | See Handling Failures |
on_failure | N | - | See Handling Failures |
on_success | N | - | See Handling Success |
tag | N | - | Identifier |
- create_table:
name: Syslog
- Idempotent. Running it again against a table that's already allocated is a no-op — existing values and presence bits are left completely untouched.
- Failure mode. If
nameisn't one of the 12 registered tables, the processor fails withunknown virtual table: "<name>". Addignore_failure: trueto treat an unrecognized name as a soft skip instead. nameis templated, so it can be built from the event, for examplename: "{{{target_table}}}".
Use Table
Selects which table, if any, the entry marshals from at output. This is the switch — nothing an entry has written into a table reaches a target until use_table runs.
- use_table:
name: <ident>
description: <text>
disabled: <boolean>
if: <script>
ignore_failure: <boolean>
on_failure: <processor[]>
on_success: <processor[]>
tag: <string>
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | - | Table to select as the output source; templated |
description | N | - | Explanatory note |
disabled | N | false | Disable the processor without removing it from the pipeline |
if | N | - | Condition to run |
ignore_failure | N | false | See Handling Failures |
on_failure | N | - | See Handling Failures |
on_success | N | - | See Handling Success |
tag | N | - | Identifier |
- use_table:
name: Syslog
name: "",name: "map", andname: "none"(case-insensitive, whitespace-trimmed) all mean "marshal from the map" — the default state before anyuse_tablehas run, and a valid way to switch back to the map mid-pipeline.- Failure mode. Any other name must resolve to a table that was already
create_table-d on this entry, or the processor fails withcannot use virtual table "<name>": unknown or not created. There is no silent fallback to the map for a name that looks like a table but wasn't created — that would defeat the purpose of an explicit switch. - Selecting a table doesn't clear or otherwise touch the map; it only changes what the entry's output marshal reads from. Calling
use_tableagain later — with a different table, or withmap/none— switches the output source again.
Truncate Table
Clears every value and presence bit on the named table while keeping it allocated for further writes.
- truncate_table:
name: <ident>
description: <text>
disabled: <boolean>
if: <script>
ignore_failure: <boolean>
on_failure: <processor[]>
on_success: <processor[]>
tag: <string>
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | - | Table to clear; templated |
description | N | - | Explanatory note |
disabled | N | false | Disable the processor without removing it from the pipeline |
if | N | - | Condition to run |
ignore_failure | N | false | See Handling Failures |
on_failure | N | - | See Handling Failures |
on_success | N | - | See Handling Success |
tag | N | - | Identifier |
- truncate_table:
name: Syslog
- Equivalent to zeroing the underlying struct and its presence bitmap in one step — every column goes back to absent, not just to its zero value. A truncated
ProcessIDreads as not-present, not as0. - Truncating a table that's a known table name but was never
create_table-d is a no-op that still succeeds. - Failure mode. An unrecognized name fails with
unknown virtual table: "<name>", same ascreate_table. - Useful inside a loop (for example a
foreach) where the same table shape needs to be rebuilt cleanly for each iteration without adrop_table/create_tableround trip.
Drop Table
Removes the named table from the entry entirely.
- drop_table:
name: <ident>
description: <text>
disabled: <boolean>
if: <script>
ignore_failure: <boolean>
on_failure: <processor[]>
on_success: <processor[]>
tag: <string>
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | - | Table to remove; templated |
description | N | - | Explanatory note |
disabled | N | false | Disable the processor without removing it from the pipeline |
if | N | - | Condition to run |
ignore_failure | N | false | See Handling Failures |
on_failure | N | - | See Handling Failures |
on_success | N | - | See Handling Success |
tag | N | - | Identifier |
- drop_table:
name: Syslog
- If the dropped table was the entry's currently selected output table, output reverts to the map automatically — the same as running
use_tablewithname: "map". - Dropping a table that was never created is a no-op that still succeeds; only an unrecognized name is a failure.
- Failure mode. An unrecognized name fails with
unknown virtual table: "<name>", same ascreate_tableandtruncate_table.
Addressing Fields: $table.Field
Any processor whose field path goes through the shared field resolver understands $<table>.Field paths — that covers set, rename (a get, set, and delete under the hood), remove, date's target_field, and if: conditions. The routing lives once, in the shared resolver, not in each processor individually — so if a new processor gains field-path support in the future, $table.Field addressing comes with it for free.
- set:
field: $syslog.Facility
value: "{{{log.syslog.facility.name}}}"
ignore_missing: true
- rename:
if: "$syslog.TimeGenerated == nil"
field: "@timestamp"
target_field: $syslog.TimeGenerated
ignore_missing: true
- remove:
field: $syslog.ProcessName
Four rules govern this addressing, and all of them are intentional rather than incidental:
- Bare names are pure map.
Facilityand$syslog.Facilityare two completely independent locations. Writing one never becomes readable through the other, even while both a table and the map are alive on the same entry. - The
$namespace never falls back to the map. Aset/rename/removeagainst$<table>.Fieldfails the processor — subject to that processor's ownignore_failure/ignore_missing— instead of silently landing in the map, in every one of these cases:- the named table was never
create_table-d (virtual table "<name>" not created), - the field name isn't a column of that table, or the value can't coerce into the column's type (
field "<name>" is not a valid column of virtual table "<table>"— both failure reasons produce the same error text, since a column-name miss and a coercion miss are indistinguishable from the caller's side), - the path nests deeper than
$table.Field— tables are flat, so$syslog.a.bis always an error, never a partial write into a nested structure.
- the named table was never
- Reads are quiet. Getting
$table.Field, or checking it in anif:condition, when the table doesn't exist or the field was never set, resolves to not-found (nil) — exactly like reading a missing map key. Only writes and deletes are loud. This asymmetry is deliberate: it lets guard conditions likeif: "$syslog.TimeGenerated == nil"be written safely before the table even exists, while still catching a genuine typo in asettarget immediately. $tablealone resolves to the table object. With no field segment,$syslogon its own is readable (useful in debugging or anif: $syslog != nilexistence check) but is not a validsetorremovetarget — there's no single value a whole table coerces down to.
Using Tables in Conditions
Every pipeline if: condition compiles against VirtualMetric's native condition engine, in Elastic-compatible mode with nil-safe navigation — see Conditional Running for the full condition language. Two consequences matter specifically for tables:
- Event field names are case-sensitive, but
$tablenames and their columns fold case, as covered above. - Navigating through a missing or
nilvalue with a plain dot doesn't error — it resolves tonil— so$syslog.TimeGenerated == nilis safe to write whetherSysloghas been created yet or not.
Reading a table's field in a condition is presence-aware: an unset column reports absent (nil) before any struct field is even examined, and a present empty string is correctly not nil. That distinction matters — a plain map-shaped nil check can't tell "never set" apart from "explicitly set to empty", but a table's presence bitmap can:
- set:
if: "$syslog.TimeGenerated == nil"
field: $syslog.TimeGenerated
value: "{{{parsed_timestamp}}}"
This runs only when TimeGenerated has genuinely never been set — an earlier explicit write, even to an empty-looking value, is respected and never silently overwritten.
A table is readable in a condition as soon as it's created — it doesn't need to also be the selected output table. use_table only controls what marshals to a target; it has no bearing on what a subsequent if: can see. Both of these resolve identically for an unset TimeGenerated, whether or not use_table has run yet:
if: "$syslog.TimeGenerated == nil"
if: "$syslog?.TimeGenerated == nil"
Optional chaining (?.) remains valid on a $table reference and behaves identically to a plain dot under the default nil-safe navigation — it's optional, not required. If a table was never created at all, $syslog.Computer (with or without ?.) still resolves to nil rather than erroring, exactly the same way navigating into a missing map field does.
Guard a table write with a presence check the same way you'd guard a map write: if: "$syslog.HostName == nil" before a set that shouldn't overwrite a value a parser already populated.
Type Coercion
Every column has a fixed storage kind, decided once when the table was generated. Writing to a column coerces the incoming value into that kind and fails the write when it can't:
| Kind | Accepts | Rejects |
|---|---|---|
| string | string, bool, every signed/unsigned int width, both float widths, and time.Time (formatted as RFC3339Nano, byte-identical to how encoding/json marshals a time.Time) | maps, slices, structs, nil |
| int | any numeric type (floats truncate, no rounding) and numeric strings | non-numeric strings, nil, other types; uint/uint64 values above the maximum signed 64-bit value |
| int64 | same as int, widened to 64 bits | same as int |
| float64 | any numeric type and numeric strings | non-numeric strings, nil, other types |
| bool | bool and parseable strings ("true", "false", "1", "0", "t", "f", and similar) | numbers — a 1 or 0 value does not coerce to true/false |
The bool rule is deliberately the strictest of the five: numbers never coerce to bool, so a stray integer landing in a boolean column fails loudly instead of being silently reinterpreted as truthy or falsy. Every other kind is comparatively permissive about numeric-shaped input.
The time.Time special case on string columns is what lets a parsed-timestamp value hand itself to a table column without an intermediate formatting step: if a value arriving at a string column happens to be a native time.Time (rather than an already-formatted string), it renders with the same RFC3339Nano layout the map path's own JSON marshal would have produced for that value — so the table and map paths agree on timestamp formatting even when the write route differs.
Setting a string column to an empty string clears its presence bit — an empty string is treated as "absent," not "present but empty." This is specific to string columns; a set of 0 on an int/int64/float64 column, or false on a bool column, still marks that column present. For example, set: { field: $syslog.HostName, value: "" } doesn't create a present-but-empty HostName — it leaves (or makes) the field absent, so it's omitted from the marshaled output and reads back as nil in an if: guard.
Presence itself is the other half of this model: Get (and therefore every if: read) only returns a value when the column's presence bit is set. A never-written column is never readable as its Go zero value — an unset ProcessID reads as absent, not as 0; an unset HostName reads as absent, not as "". This is what makes if: "$syslog.TimeGenerated == nil" a reliable "has this been set yet" check rather than a coincidental match on a default value.
Parsing Straight into a Table: the syslog Processor's table Parameter
The syslog processor accepts an optional table: parameter. When it's absent, the processor's behavior is unchanged from ordinary map mode — it parses into the intermediate map object at target_field (default log.syslog), exactly as documented for that processor.
When table: is set, the parser writes straight into the named table's columns instead of building an intermediate map. The table must already exist — create_table has to run first, or the processor fails with unknown virtual table "<name>" (unregistered name) or virtual table "<name>" not created (registered but never allocated on this entry).
- create_table:
name: Syslog
- syslog:
field: message
table: Syslog
The field mapping, with the gating condition for each:
| Table field | Source | Condition |
|---|---|---|
Facility | Parsed facility name | Always set |
SeverityLevel | Parsed severity name | Always set |
HostName | Parsed hostname | Only if non-empty |
ProcessName | Parsed application name | Only if non-empty |
ProcessID | Parsed process ID | Only if non-zero |
SyslogMessage | Parsed message body, falling back to the raw line if the body is empty | Always set (one or the other) |
TimeGenerated | Reparsed date from the message, formatted as RFC3339Nano | Only if TimeGenerated isn't already present |
That last row mirrors the same if: "$syslog.TimeGenerated == nil" guard a map-mode date/rename chain would apply by hand — so an earlier explicit set of TimeGenerated always wins over the parser's own reparse. Columns the parser doesn't populate at all — Computer, HostIP, EventTime, SourceSystem, ManagementGroupName — are the pipeline author's job through ordinary set/rename steps targeting $syslog.*. That mirrors how a set/rename chain reads log.syslog.* in map mode.
The parser also still publishes the detected envelope format to _vmetric.syslog_format — this always lands in system metadata, independent of map or table mode, so downstream routing like if: "_vmetric?.syslog_format == 'CEF'" keeps working with no intermediate map to read it from.
Any value the parser tries to write that fails to coerce into its target column fails the whole syslog step: virtual table "<table>" rejected <field>=<value> (unknown column or non-coercible value). This is rare, since the parser writes its own well-formed values, but it can surface if a table's generated schema and the parser's expected columns ever diverge.
What Gets Sent to Targets
Every output path an entry can take — the final pipeline result, reroute's staged payload, debug output — reads from the exact same switch: has a table been selected for this entry, or not?
- No
use_tablehas run (or the selected table was later dropped, oruse_tableselectedmap/none/""): the payload is the map, exactly as it was before virtual tables existed. Any table's contents — even a fully populated one — are simply invisible to this output. use_tableselected a table: the payload is only that table's present columns, serialized by its own generated, reflection-free marshal function.
When a table is selected, the map is dropped in its entirety — not merged, not filtered, dropped. Any field left in the map ships nowhere: an enrichment key a reader added upstream, an intermediate field a step forgot to move, _vmetric itself. If a field needs to reach the target after use_table has run, it must be written through $<table>.Field before that point, not left as a bare-name map write. This is the single most common mistake when converting a branch to virtual tables — a field that worked fine in map mode silently disappears from the output the moment use_table starts pointing elsewhere.
Fan-Out with Clones and Reroute
Two different mechanisms both use the word "clone" here, and they behave differently with respect to tables:
- Entry cloning — the
pipelineprocessor's ownclone: trueoption, and other internal entry-carrying paths — copies every table the source entry has allocated by value into a genuinely new, independent entry, and repoints that new entry's output-table selection at its own copy if the source had one selected. From that point on, the clone has its own independent copy of every table: mutating a clone's table never touches the source's, and the clone is free tocreate_table, write, oruse_tablea completely different selection afterward with zero effect on the source. reroute's ownclone: trueis a lighter-weight, different thing: it does not create a new entry or copy any table. It marshals whatever the entry's current selected table (or map) is at that moment into a parked payload for the given destination, without persisting that reroute's routing overrides onto the entry itself — so a later step, or a laterreroute, still sees the entry exactly as it was before that clone-mode reroute ran.
This pairing is what makes per-branch table selection a clean fan-out pattern. A single map-mode parse can stay untyped at the parent level while spawning one typed entry clone that builds and selects its own table shape, then fans that one already-selected table out to several destinations with a sequence of reroute steps:
pipelines:
- name: parse_syslog
processors:
# Parent stays in map mode
- syslog:
field: message
target_field: log.syslog
- pipeline:
name: typed_fanout
clone: true
- name: typed_fanout
processors:
- create_table:
name: Syslog
- set:
field: $syslog.Facility
value: "{{{log.syslog.facility.name}}}"
ignore_missing: true
- set:
field: $syslog.HostName
value: "{{{log.syslog.hostname}}}"
ignore_missing: true
- set:
field: $syslog.SyslogMessage
value: "{{{log.syslog.message}}}"
ignore_missing: true
- use_table:
name: Syslog
# Every reroute downstream of use_table marshals the typed struct
- reroute:
destination: archive
clone: true
- reroute:
destination: sentinel
Because typed_fanout selected Syslog with use_table before its reroute steps run, every reroute in that pipeline — clone-mode or not — checks the same outputTable switch at send time and serializes the typed struct for its destination. No per-destination change is needed once use_table has run, and the parent pipeline's own map-mode entry is never touched by any of it. The reroute steps themselves are completely ordinary; see reroute and Multi-Tier Pipelines for the staging and commit mechanics that pattern builds on.
Converting a Pipeline: Recipe
Converting an existing map-mode branch to a virtual table is a mechanical, five-step rewrite, plus verification:
-
Confirm the branch is worth converting. Virtual tables pay off on a branch that writes a small, fixed, known set of fields and then marshals — a fan-out
rerouteleaf is the classic shape. A branch that does open-ended, dynamic-key enrichment gains nothing, since there's no table column for an arbitrary key the branch doesn't know about ahead of time. -
Pick the table that matches the destination shape and
create_tableit at the top of the branch:- create_table:name: Syslog -
If the branch might receive an entry that hasn't been parsed yet, feed the parser straight into the table with
table:instead of building the intermediate map:- syslog:if: "log == nil"field: messagetable: Syslog -
Rewrite every
set/rename/datestep that wrote a bare field into the equivalent$<table>.Fieldpath, keeping the same value expressions:# Before (map mode)- set:field: Facilityvalue: "{{{log.syslog.facility.name}}}"ignore_missing: true# After (typed)- set:field: $syslog.Facilityvalue: "{{{log.syslog.facility.name}}}"ignore_missing: trueUpdate every
if:guard the same way:if: "TimeGenerated == nil"becomesif: "$syslog.TimeGenerated == nil". -
Select the table once every field is written, right before the fan-out:
- use_table:name: SyslogThe
reroutesteps themselves need no changes — aclone: truefan-out already picks up the selected table automatically onceuse_tablehas run.
Before treating the conversion as done, verify byte-for-byte parity against the old map-mode output: run the pipeline's existing tests, and diff a captured payload from the new typed branch against the old map-mode payload for the same input. A field-by-field comparison catches a missed rename or a coercion mismatch that a simple byte-count check would not. Converting one branch doesn't obligate converting anything else — a sibling branch of the same pipeline, or the shared parser pipeline other branches also call, can stay in map mode indefinitely.
Naming Collision: reroute's table and schema Fields
The reroute processor has its own, long-standing table: and schema: fields, and the pipeline-wide _vmetric.table metadata field — these are destination-routing metadata (for example, "which Azure Data Explorer table should this land in"), set as ordinary strings, with no connection whatsoever to create_table/use_table/$table.Field. A reroute step with table: "Syslog" is not creating or referencing the Syslog virtual table — it's telling the destination target what table name to write into on its end. The two features happen to share the word "table" because both predate the other's naming, not because they interact.
This distinction matters because the two can appear in the same pipeline, addressing genuinely different things:
- create_table:
name: Syslog # virtual-table engine: allocates a typed struct
- set:
field: $syslog.HostName
value: "{{{hostname}}}"
- use_table:
name: Syslog # virtual-table engine: selects the typed struct for marshal
- reroute:
destination: sentinel
table: "Syslog" # destination metadata: unrelated string, read by the target
If a pipeline needs both — a typed table selected for its own marshal efficiency, and a destination-side table name for the target to route by — both fields are set independently, with no implied relationship between their values. They can even use different casing or entirely different names without causing any conflict, precisely because neither one reads the other.
Performance
The headline numbers come from VirtualMetric's benchmark suite, comparing the same field mappings on the same input with only the storage and marshal path changed — map writes and reflection-based JSON marshaling on one side, a Syslog virtual table and its generated marshal function on the other:
| Comparison | Typed vs. map |
|---|---|
| End-to-end branch (parse, set fields, marshal) | ~1.6x faster, 54 vs 69 allocations per event |
| Marshal step alone | ~3.2x faster, 2 vs 24 allocations per event |
Marshal is where the gap is widest: a table's generated marshal function walks its own presence bitmap and appends bytes directly, with no reflection over an untyped map. End to end, the intermediate map object and its later cleanup steps are simply gone — use_table drops the map at output for free.
Exact timing ratios vary with hardware and machine load; the allocation counts do not — they measure heap behavior rather than clock speed, and reproduce identically everywhere. On a high-volume branch, the allocation savings are the number that compounds.
When to Use Virtual Tables
| Signal | Use a virtual table | Stay with the map |
|---|---|---|
| Field set | Small, fixed, known ahead of time | Open-ended, dynamic keys added by enrichment |
| Branch shape | Hot fan-out leaf: parse, set fields, marshal, ship | Exploratory or one-off processing |
| Destination schema | Matches one of the 12 registered tables | Doesn't correspond to any registered schema |
| Volume | High — allocation savings compound per event | Low — the fixed cost of adopting the pattern outweighs the gain |