Map fields

This page explains what map fields are and how to query them.

Map fields are a special type of field that can hold a collection of nested key-value pairs within a single field. You can think of the content of a map field as a JSON object.

Currently, Axiom automatically creates map fields in datasets that use OpenTelemetry. You cannot create map fields yourself.

Support for creating your own map fields is coming in early 2025. To express interest in the feature, contact Axiom.

Benefits and drawbacks of map fields

Map fields help you manage high-cardinality data by storing multiple key-value pairs within a single field. One of the benefits of map fields is that you can store additional attributes without adding more fields. This is particularly useful when the shape of your data is unpredictable (for example, additional attributes added by OpenTelemetry instrumentation). Using map fields means that you can avoid reaching the field limit of a dataset.

Use map fields in the following cases:

• You approach the dataset field limit.
• The shape of your data is unpredictable. For example, an OpenTelemetry instrumentation or another SDK creates objects with many keys.
• You work with feature flags or custom attributes that generate many fields.

Map fields reduce impact on field limits, but involve trade-offs in query efficiency and compression. The drawbacks of map fields are the following:

• Querying map fields uses more query-hours than querying conventional fields.
• Map fields don’t compress as well as conventional fields. This means datasets with map fields use more storage.
• You don’t have visibility into map fields from the schema. For example, autocomplete doesn’t know the properties inside the map field.

Custom attributes in tracing datasets

If you use OpenTelemetry to send data to Axiom, you find some attributes in the attributes.custom map field. The reason is that instrumentation libraries can add hundreds or even thousands of arbitrary attributes to spans. Storing each custom attribute in a separate field would significantly increase the number of fields in your dataset. To keep the number of fields in your dataset under control, Axiom places all custom attributes in the single attributes.custom map field.

Use map fields in queries

The example query below uses the http.protocol property inside the attributes.custom map field to filter results:

['otel-demo-traces']
| where ['attributes.custom']['http.protocol'] == 'HTTP/1.1'

Run in playground

Access properties of nested maps

To access the properties of nested maps, use dot notation, index notation, or a mix of the two. If you use index notation for an entity, enclose the entity name in quotation marks (' or ") and square brackets ([]). For example:

• where map_field.property1.property2 == 14
• where ['map_field'].property1.property2 == 14
• where ['map_field']['property1']['property2'] == 14

If an entity name has spaces ( ), dots (.), or dashes (-), you can only use index notation for that entity. You can use dot notation for the other entities. For example:

• where ['map.field']['property.name1']['property.name2'] == 14
• where ['map.field'].property1.property2 == 14

For more information, see Entity names.

Create map fields

To create a map field:

1. Go to the Datasets tab.
2. Select the dataset where you want to create the map field.
3. In the top right of the fields list, click More > Create map field.
4. In Field name, enter the full name of the field, including parent fields, if any. For example, map_field_name. For more information on syntax, see Access properties of nested maps
5. Click Create map field.

View map fields

To view map fields:

1. Go to the Datasets tab.
2. Select a dataset where you want to view map fields.
3. Map fields are labelled in the following way:

• MAPPED means that the field was previously an ordinary field but at some point its parent was changed to a map field. Axiom adds new events to the field as an attribute of the parent map field. Events you ingested before the change retain the ordinary structure.
• UNUSED means that the field is configured as a map field but you haven’t yet ingested data into it. Once ingested, data within this field won’t count toward your field limit.
• REMOVED means that the field was configured as a map field but at some point it was changed to an ordinary field. Axiom adds new events to the field as usual. Events you ingested before the change retain the map structure. To fully remove this field, first trim your dataset to remove the time period when map data was ingested, and then vacuum the fields.