Cribl Integration

Map compliance controls in real time as events flow through a Cribl Stream or Cribl Edge pipeline. This guide walks through calling the POST /v1/map endpoint from within a Cribl pipeline to enrich events with matched framework controls.

Overview

A JavaScript Eval function in your pipeline calls /v1/map for each event, then attaches the matched controls as new fields. Downstream routes or outputs can then filter, tag, or forward events based on the compliance results.

---

Step 1 — Discover Available Framework IDs

Before writing any pipeline code you need to know which framework IDs to pass in the frameworks array. Retrieve the full list with a single API call:

curl -s -H "authorization: $SECBERUS_API_KEY" \
  https://compliance.secberus.ai/v1/frameworks \
  | jq '.[] | {id, name, region}'

Sample output:

{"id": "pci_dss_v4",      "name": "PCI DSS v4.0",                        "region": "Global"}
{"id": "nist_800_53_r5",  "name": "NIST SP 800-53 Rev 5",                "region": "US"}
{"id": "nist_csf_v2",     "name": "NIST Cybersecurity Framework v2.0",   "region": "US"}
{"id": "iso_27001",       "name": "ISO/IEC 27001:2022",                  "region": "Global"}
{"id": "soc2",            "name": "SOC 2 Type II",                       "region": "US"}
{"id": "aicpa_tsc",       "name": "AICPA Trust Services Criteria",       "region": "US"}

Use the id values (e.g., "pci_dss_v4") in your pipeline configuration. The framework list changes infrequently, so you only need to run this once when setting up or updating the integration.

---

Step 2 — Store the API Key as a Secret

In Cribl, navigate to Settings → Global Settings → Secrets and add a secret:

Field	Value
Name	SECBERUS_API_KEY
Value	Your Secberus API key

Reference it in pipeline functions as C.env("SECBERUS_API_KEY").

---

Step 3 — Add a JavaScript Eval Function

In your pipeline, add an Eval function and paste the following code. Update policy_text to match the field in your events that contains the text to map, and set frameworks to the IDs from Step 1.

async (event) => {
  const text = event.policy_text; // change to your source field
  if (!text) return event;

  const apiKey = C.env("SECBERUS_API_KEY");
  const frameworks = ["pci_dss_v4"]; // add framework IDs from Step 1

  let resp;
  try {
    resp = await fetch("https://compliance.secberus.ai/v1/map", {
      method: "POST",
      headers: {
        "authorization": apiKey,
        "Content-Type": "application/json"
      },
      body: JSON.stringify({
        frameworks,
        min_similarity: 0.3,
        topk: 3,
        documents: [{ id: "event", document: text }]
      })
    });
  } catch (err) {
    event.compliance_map_error = `fetch failed: ${err.message}`;
    return event;
  }

  if (!resp.ok) {
    event.compliance_map_error = `API error ${resp.status}`;
    return event;
  }

  const body = await resp.json();

  // Attach results as fields — one array per framework
  for (const fw of (body.frameworks ?? [])) {
    const matches = fw.controls.map(c => ({
      control_id:  c.control.id,
      family:      c.control.family,
      similarity:  c.similarity,
      confidence:  c.confidence
    }));
    event[`compliance_${fw.framework_id}`] = matches;
  }

  // Surface any API warnings
  if (body.messages?.length) {
    event.compliance_warnings = body.messages.map(m => m.message);
  }

  return event;
}

This adds a field like compliance_pci_dss_v4 to each event — an array of matched controls with their IDs, family names, similarity scores, and confidence levels.

---

Step 4 — Route or Filter on Results

After the Eval function, use a Filter or Route step to branch on the compliance results.

Pass through only High-confidence matches:

(event.compliance_pci_dss_v4 ?? []).some(c => c.confidence === "High")

Flatten control IDs to a string field (useful for SIEM ingest):

// In a second Eval function
event.pci_controls = (event.compliance_pci_dss_v4 ?? [])
  .map(c => c.control_id).join(", ");

---

Configuration Reference

Key request parameters

Parameter	Type	Default	Description
frameworks	array[string]	—	Framework IDs from Step 1
min_similarity	float	—	Exclude matches below this score (0.01–1.0)
min_confidence	string	—	Exclude matches below this level: High, Medium, Low, Very Low. Mutually exclusive with min_similarity.
topk	integer	1	Max controls returned per framework/document pair

Result fields added to each event

Field	Type	Description
compliance_<framework_id>	array	Matched controls for that framework
compliance_<framework_id>[].control_id	string	Control identifier (e.g., 8.3.6)
compliance_<framework_id>[].family	string	Control family name
compliance_<framework_id>[].similarity	float	Similarity score (0.0–1.0)
compliance_<framework_id>[].confidence	string	High, Medium, Low, or Very Low
compliance_warnings	array[string]	API warning messages, if any
compliance_map_error	string	Set when the API call fails; absent on success

---

Multi-Framework Mapping

Pass multiple framework IDs to map against several frameworks in a single API call. The result will include a separate compliance_<framework_id> field for each:

const frameworks = ["pci_dss_v4", "nist_800_53_r5"];

Each matched framework adds its own field to the event:

• compliance_pci_dss_v4
• compliance_nist_800_53_r5

---

Batching High-Volume Pipelines

The /v1/map documents array accepts multiple entries per request. For high-throughput pipelines, use a Reduce function to group events into batches before the Eval, call the API once per batch, then fan results back out. This significantly reduces per-event API overhead.

For repeated text (e.g., identical policy statements appearing across many events), cache results in a Cribl Lookup table keyed by a hash of the document text.

---

Error Handling

Errors are written to compliance_map_error rather than dropping the event, so failed enrichments continue flowing downstream. Add a Route after the Eval to handle these separately:

// Route condition — catch API failures
event.compliance_map_error !== undefined

The API returns standard HTTP status codes:

Status	Meaning
400	Bad request — check frameworks IDs and document format
403	Invalid or missing API key
500	Server error — retry with backoff; contact support if persistent

---

API Quick Reference

Detail	Value
Endpoint	POST https://compliance.secberus.ai/v1/map
Auth header	authorization: <api-key>
Content-Type	application/json
List frameworks	GET https://compliance.secberus.ai/v1/frameworks
Similarity range	0.01–1.0
Confidence levels	Very Low, Low, Medium, High
Default topk	1