Elastic Security Integration

Add compliance control mappings to events flowing into Elastic Security. Logstash (the Elastic Stack's native pipeline tool) handles enrichment by calling /v1/map inline — see the Logstash integration guide for pipeline setup. This guide covers the Elastic side: index mappings that store compliance fields efficiently, and KQL/EQL examples for using those fields in detection rules, alert triage, and dashboards.

Architecture

Security Events
      │
      ▼
Logstash Pipeline
  └─ http filter calls POST /v1/map
  └─ ruby block extracts compliance_controls
      │
      ▼
Elasticsearch Index
  (compliance fields mapped as nested/keyword)
      │
      ▼
Elastic Security
  ├─ KQL detection rules
  ├─ EQL event correlations
  └─ Kibana dashboards

Step 1 — Discover Available Framework IDs

Retrieve the framework IDs you want to map against before configuring anything:

curl -s -H "authorization: $SECBERUS_API_KEY" \
  https://compliance.secberus.ai/v1/frameworks \
  | jq '.[] | {id, name, region}'

Sample output:

{"id": "pci_dss_v4",     "name": "PCI DSS v4.0",                       "region": "Global"}
{"id": "nist_800_53_r5", "name": "NIST SP 800-53 Rev 5",               "region": "US"}
{"id": "nist_csf_v2",    "name": "NIST Cybersecurity Framework v2.0",  "region": "US"}
{"id": "iso_27001",      "name": "ISO/IEC 27001:2022",                 "region": "Global"}
{"id": "soc2",           "name": "SOC 2 Type II",                      "region": "US"}
{"id": "aicpa_tsc",      "name": "AICPA Trust Services Criteria",      "region": "US"}

You will reference these in your Logstash pipeline config and in KQL filters.

Step 2 — Set Up Logstash Enrichment

Follow the Logstash integration guide to configure your pipeline. The guide covers installing the logstash-filter-http plugin, storing the API key in the Logstash keystore, and building the filter block that calls /v1/map and writes a compliance_controls array to each event.

Your Logstash output block should target the index you configure in the next step:

output {
  elasticsearch {
    hosts    => ["https://your-es-host:9200"]
    index    => "logs-security-%{+YYYY.MM.dd}"
    user     => "${ES_USER}"
    password => "${ES_PASSWORD}"
  }
}

Step 3 — Create the Index Template

Create an index template that maps compliance_controls as a nested type so Elastic Security can query individual array elements independently. Without a nested mapping, array-of-object fields cannot be filtered accurately.

PUT _index_template/logs-security
{
  "index_patterns": ["logs-security-*"],
  "template": {
    "mappings": {
      "properties": {
        "compliance_controls": {
          "type": "nested",
          "properties": {
            "framework_id": { "type": "keyword" },
            "control_id":   { "type": "keyword" },
            "family":       { "type": "keyword" },
            "similarity":   { "type": "float"   },
            "confidence":   { "type": "keyword" }
          }
        },
        "compliance_error": { "type": "keyword" }
      }
    }
  }
}

Apply the template before the first document is indexed — it will not retroactively remap existing indices.

Step 4 — KQL Detection Rules

Use these KQL queries when creating detection rules in Elastic Security → Rules → Create new rule → Custom query.

Alert on any High-confidence PCI DSS match:

compliance_controls.framework_id : "pci_dss_v4"
  and compliance_controls.confidence : "High"

Alert on a specific control across any framework:

compliance_controls.control_id : "8.4.2"

Alert when an event maps to multiple frameworks simultaneously (broad compliance exposure):

compliance_controls.framework_id : ("pci_dss_v4" or "nist_800_53_r5" or "iso_27001")

Alert only on High or Medium confidence matches, excluding low-signal results:

compliance_controls.confidence : ("High" or "Medium")
  and not compliance_controls.confidence : ("Low" or "Very Low")

Surface enrichment failures for monitoring:

compliance_error : *

Step 5 — EQL Event Correlations

Use EQL in Elastic Security → Rules → Create new rule → Event correlation to detect sequences involving compliance-mapped events.

Privileged action following a High-confidence access control match:

sequence with maxspan=30m
  [any where compliance_controls.framework_id == "pci_dss_v4"
           and compliance_controls.confidence == "High"]
  [process where process.name == "net.exe" and process.args : ("localgroup", "administrators")]

Data exfiltration after a High-similarity encryption control match:

sequence with maxspan=1h
  [any where compliance_controls.control_id == "3.5.1"
           and compliance_controls.similarity >= 0.8]
  [network where network.bytes_sent > 50000000]

Step 6 — Kibana Dashboard

Create a dashboard in Kibana → Dashboards → Create dashboard to visualize compliance coverage across your event stream.

Recommended panels

Panel type	Aggregation	Description
Pie chart	Terms on `compliance_controls.framework_id`	Share of events per framework
Bar chart	Terms on `compliance_controls.control_id`	Top 10 most-matched controls
Data table	Terms on `compliance_controls.confidence`	Event counts by confidence level
Metric	Filter `compliance_error : *` → Count	Total enrichment failures
Timeline	Date histogram + filter `compliance_controls.confidence : "High"`	High-confidence matches over time

Sample aggregation query for the top matched controls:

GET logs-security-*/_search
{
  "size": 0,
  "aggs": {
    "controls": {
      "nested": { "path": "compliance_controls" },
      "aggs": {
        "top_controls": {
          "terms": {
            "field": "compliance_controls.control_id",
            "size": 10
          },
          "aggs": {
            "frameworks": {
              "terms": { "field": "compliance_controls.framework_id" }
            }
          }
        }
      }
    }
  }
}

Configuration Reference

Fields written by Logstash enrichment

Field	Mapping type	Description
`compliance_controls`	`nested`	Array of matched controls across all configured frameworks
`compliance_controls.framework_id`	`keyword`	Framework identifier (e.g., `pci_dss_v4`)
`compliance_controls.control_id`	`keyword`	Control identifier (e.g., `8.4.2`)
`compliance_controls.family`	`keyword`	Control family name
`compliance_controls.similarity`	`float`	Similarity score (0.0–1.0)
`compliance_controls.confidence`	`keyword`	`High`, `Medium`, `Low`, or `Very Low`
`compliance_error`	`keyword`	Set when the API call failed; absent on success

Why `nested` mapping matters

Elasticsearch stores arrays of objects as flat parallel arrays by default. Without nested, the query compliance_controls.framework_id: "pci_dss_v4" AND compliance_controls.confidence: "High" would match any event where some control has that framework AND some (possibly different) control has that confidence — not necessarily the same control. The nested type preserves object boundaries so each control is queried independently.

API Quick Reference

Detail	Value
Endpoint	`POST https://compliance.secberus.ai/v1/map`
Auth header	`authorization: <api-key>`
Content-Type	`application/json`
List frameworks	`GET https://compliance.secberus.ai/v1/frameworks`
Similarity range	0.01–1.0
Confidence levels	`Very Low`, `Low`, `Medium`, `High`
Default topk	1