ServiceNow integration — full reference
This is the single, consolidated reference for the SARC ServiceNow integration.
It merges the 22 source documents under docs/ci/servicenow/ into one page for
both engineers (field names, tables, scripts, endpoints) and users (what each
area is for). Each section opens with a plain-language intro, then the detail.
A Reference index at the end links every source
doc to the section that covers it.
Overview & architecture
Section titled “Overview & architecture”SARC uses ServiceNow as the system of record for change management. Every
production (and optionally pre-production) deploy gates on a ServiceNow Change
Request (CR) whose state is driven by the CI pipeline and by Fides compliance
attestations. SARC ships as a multi-cloud demo (TARGET_CLOUD is one of aws,
azure, gcp, k3d) with Helm-deployed charts and the Fides flow
karc-pipeline.
The integration runs in two directions:
- ARC writes. The pipeline (and a scheduled reconciler) write the CR, its custom fields, the CI links, the artifact/evidence graph, and — via the CMDB sync jobs and Service Graph Connectors — the whole infrastructure inventory into ServiceNow.
- ServiceNow decides. ServiceNow owns the approval and the CR state; ARC reads that back (the portal shows CR status and approvals, and a deploy gate can act on it). The maxim is “Fides advises, ServiceNow decides.”
The demo instance throughout these docs is calitiiltddemo3.service-now.com.
The pipeline authenticates as a dedicated integration service account (referred
to as sarc.ci, github_integration, or integration.user depending on the
path) over the ServiceNow Table API.
End-to-end shape
Section titled “End-to-end shape”Developer push → GitLab / GitHub / ADO CI (flow: karc-pipeline) ├─ build + scan (Trivy, Grype, Semgrep, syft, checkov, kubeconform) ├─ attest to Fides (trail keyed on commit SHA) ├─ create/advance ServiceNow change_request └─ helm upgrade / ArgoCD sync into Kubernetes (karc-<env> namespace) ServiceNow → approves (auto or CAB) → CI deploys → CI closes the CR Fides trail ← linked from the CR via a deterministic URLCR ↔ Fides linkage
Section titled “CR ↔ Fides linkage”Every CR carries a u_commit_sha. Given that SHA, the Fides trail URL is
deterministic:
https://fides.13.134.88.9.nip.io/calitti/flows/karc-pipeline/trails/<u_commit_sha>A Client Script + UI Action (installed by
scripts/servicenow/install-fides-iframe.sh) render a View Fides Trail
button on the CR form that opens that URL. A business rule also populates
u_fides_trail_url server-side so the trail is queryable in ServiceNow reports.
Multi-cloud notes
Section titled “Multi-cloud notes”- All CRs carry
u_target_cloudinaws,azure,gcp,k3d. - Namespaces follow
karc-<env>and are identical across clouds; the CR differentiates deploy targets viau_target_cloud, not via namespace. - Dashboards filter by
u_target_cloudso CAB reviewers see per-cloud posture. - The ServiceNow DevOps plugin is optional — the demo path uses only the Table API. The plugin adds richer pipeline linkage but is not a hard dependency.
Change Request workflow
Section titled “Change Request workflow”The CR is the record that says “this specific change, to this specific thing, was approved by this specific person, and here is the evidence.” A SARC pipeline run creates exactly one CR per environment and walks it through the stock ServiceNow Change Management state machine. Low-risk, low-environment changes auto-approve and never involve a human; higher-risk or higher-environment changes route to a named CAB approval group.
State machine
Section titled “State machine”| ServiceNow state | state value |
Entered by | Exit condition |
|---|---|---|---|
| New | -5 | CI (POST change_request) |
business rule or pipeline advances to Assess |
| Assess | -4 | business rule | risk evaluated; routes to Authorize or Scheduled |
| Authorize | -3 | business rule | approver acts (right group) or 1h timeout; CI poll jobs watch this state |
| Scheduled | -2 | CI or auto-approve | CI begins deploy |
| Implement | -1 | CI (PATCH) | helm upgrade completes or fails |
| Review | 0 | CI (PATCH) | post-deploy checks |
| Closed | 3 | CI (PATCH) | terminal |
| Rejected | 4 | approver or CI | terminal |
The numeric values are the stock Change Management values; SARC does not remap them. Note the closure
closeaction usesstate=6in some helper paths (the ITIL “Closed” choice on the instance) — always confirm the instance’s choice list if astatePATCH returns400 Invalid value for choice field.
Risk scoring
Section titled “Risk scoring”Risk is a 0–100 integer computed by CI at CR-creation time (in
scripts/ci/servicenow-cr.sh, with scripts/ci/fides-score.sh as a fallback
path). Both paths emit the same 0–100 scale and identical thresholds.
score = 100 - (60 if Fides trail compliant else 0) - (20 if tests passed else 0) - env_bonus # dev -> 10, qa -> 5, prod -> 0# clamped to [0, 100]| Input | Source | Effect |
|---|---|---|
| Fides trail compliant | GET /api/v2/organizations/<org>/flows/karc-pipeline/trails/<sha> → .compliant == true (or .compliance_status.compliance == "COMPLIANT") |
-60 |
| Tests passed | TESTS_PASSED=true env var |
-20 |
| Environment bonus | dev -10, qa -5, prod 0 | additive |
If FIDES_API_TOKEN is unset or the Fides call fails, compliance defaults to
false (worst case) so the CR routes to human review.
Risk level mapping (written to u_risk_level): score <= 5 is low,
<= 20 is medium, > 20 is high. Baseline scores for a nominally
compliant, tests-passing pipeline: dev 10 (medium), qa 15 (medium), prod 20
(medium).
Auto-approval thresholds
Section titled “Auto-approval thresholds”u_auto_approved is true when u_risk_score <= SN_RISK_THRESHOLD_<ENV>.
Thresholds are CI variables; CI is the authoritative writer and the ServiceNow
business rule is a belt-and-braces duplicate.
| Env | CI var | Default | Effect |
|---|---|---|---|
| dev | SN_RISK_THRESHOLD_DEV |
100 | always auto-approve |
| qa | SN_RISK_THRESHOLD_QA |
5 | auto-approve only if low |
| prod | SN_RISK_THRESHOLD_PROD |
2 | auto-approve only if near-zero risk |
At defaults a nominally compliant dev pipeline auto-approves; qa and prod queue
for CAB review. Any u_target_cloud=k3d run always auto-approves (local demo
short-circuit). Raise SN_RISK_THRESHOLD_QA to 20 during a demo week;
SN_RISK_THRESHOLD_PROD past the medium cutoff (20) requires a documented
exception.
Auto-deployment story
Section titled “Auto-deployment story”The demo headline is “80%+ of deploys ship without a human touching the CR.”
Dev push → CI build+scan → Fides attestations (trail=SHA) → compute risk → POST change_request {u_commit_sha, u_risk_score, u_target_env, u_target_cloud} → business rule sarc_auto_approve_low_risk if risk <= threshold (or cloud=k3d): state=-2 Scheduled → CI helm upgrade → deployment attestation to Fides → PATCH state -1 → 0 → 3 (Closed) else: state=-3 Authorize → approval group → approve (state -2) or 1h timeout (state 4 Rejected)The auto-path does not fire when: critical CVEs push risk over the env
threshold; Fides policy evaluation fails; a required attestation is missing
(e.g. no SBOM); env is staging/prod with risk > 2; or a change-freeze
window is active (freeze handling is future work).
Approvals
Section titled “Approvals”When a CR routes to Authorize (-3), a named group must sign off before the pipeline advances to Scheduled (-2).
u_target_env |
Approval group (default) | Notes |
|---|---|---|
| dev | n/a (auto-approved) | only a u_risk_score=10 override reaches sarc_cab_devops |
| qa | sarc_cab_devops |
DevOps leads; async ping |
| staging | sarc_cab_platform |
1 approver |
| prod | sarc_cab_primary |
2 approvers |
Optional per-cloud CAB splits override the group by u_target_cloud (e.g.
sarc_cab_primary_aws), implemented as a second assignment rule ordered after
the env rule. Default ServiceNow Change notifications suffice; the demo path
also wires an outbound Slack/Teams webhook (a business rule on
after insert/update when state == -3 using sn_ws.RESTMessageV2).
1-hour timeout policy. If a CR sits in Authorize for more than 60 minutes without approval, SARC treats it as implicit rejection for that run. Enforced on both sides:
- CI (primary): the deploy job polls state on a 30s cadence with
max_wait=3600; on timeout it fails the job and PATCHes the CR tostate=4withclose_notes='CI timeout: no approval within 1h'. - ServiceNow (safety net): a Scheduled Job
sarc_expire_stale_authorizeruns every 15 min and flips anystate==-3CR older than 1h tostate=4.
Polling is implemented by scripts/ci/servicenow-wait-approval.sh, wired via
.gitlab/ci/templates/servicenow-approval-poll.yml as servicenow:cr:wait-qa
and servicenow:cr:wait-prod.
| CI variable | Default | Meaning |
|---|---|---|
SN_APPROVAL_TIMEOUT |
3600 | seconds to wait (raise to 28800 for prod demos, drop to 300 for smoke tests) |
SN_APPROVAL_POLL_INTERVAL |
30 | seconds between polls |
Each poll reads GET change_request/<sys_id>?sysparm_fields=state,approval:
approval == "approved" opens the gate; approval == "rejected" fails;
state == -1 or state == 3 proceeds (already past approval); state == 4
fails; anything else keeps polling until timeout. The job skips clean
(exit 0) when SERVICENOW_INSTANCE_URL is empty, SN_CR_SYS_ID is empty, or
SN_AUTO_APPROVED=true. A rejected CR is terminal — a new pipeline run on
the same commit must create a new CR, not resurrect the old one.
Business rules SARC expects
Section titled “Business rules SARC expects”SARC does not yet ship a ServiceNow Update Set (tracked in #26). These
server-side rules are assumed present on the change_request table (Global or
x_sarc scope):
sarc_auto_approve_low_risk(before insert/update, order 100) — setsstate=-2whenk3dorrisk <= threshold(dev 10, qa 5, staging/prod 2), elsestate=-3.sarc_force_review_high_risk(order 90) — ifu_risk_score == 10, forcestate=-3andapproval=requestedregardless of env.sarc_work_notes_on_state_change(after update, onstate.changes()) — appends a timestamped audit note per transition.sarc_populate_fides_trail_url(before insert/update, onu_commit_sha.changes()) — derivesu_fides_trail_urlfrom the SHA.sarc_block_implement_without_approval(before update,changesTo(-1)and env not dev) — aborts the transition unlessapproval == 'approved'or the previous state was Scheduled. Last-line defence against a pipeline PATCHing straight from New to Implement.
Conflict checking
Section titled “Conflict checking”servicenow-cr.sh create writes two ITIL fields on every CR so CAB reviewers
see a conflict check was performed. Tier 1 always writes conflict_status
(No Conflict / Conflict Detected) and conflict_last_run (UTC timestamp).
Tier 2 (default on) pre-queries change_request for active CRs
(state<6) in scope within a look-back window; on overlap it flips
conflict_status, records the overlapping CR numbers in work_notes, and (if
CONFLICT_ABORT=true) exits 2 without creating the CR. Any query failure
degrades gracefully to No Conflict — a ServiceNow outage never blocks a
deploy.
| Var | Default | Effect |
|---|---|---|
CONFLICT_CHECK_ENABLED |
true |
master switch for Tier 2 |
CONFLICT_ABORT |
false |
true = exit 2 on overlap |
CONFLICT_WINDOW_HOURS |
1 |
look-back window |
CONFLICT_SCOPE |
namespace |
namespace | cluster | env |
CONFLICT_OVERLAPS_JSON |
unset | test hook replacing the HTTP GET |
The check is advisory (SARC never auto-rejects). It is a Layer-1 complement to ServiceNow’s separately-licensed Change Conflict Calculator (CCC), which remains the authoritative Layer-2 gate when enabled.
Closure lifecycle
Section titled “Closure lifecycle”Closure is on by default for qa and prod (CLOSE_CR_AFTER_DEPLOY=true). No CR
is created for dev promotions (promote.sh rejects --env dev). Closure fires
only when the flag is true, a CR was created (CR_SYS_ID non-empty), and
SERVICENOW_INSTANCE_URL is set; any missing precondition logs a skip and the
pipeline stays green.
Close codes (validated against the SN ITIL choice list): successful
(gitops-bump exit 0), unsuccessful (non-zero), cancelled (manual), risk
(manual). close_notes is a structured multi-line narrative (start/end/
duration, rollbacks, health result, GitOps run URL, Fides trail, platform,
cloud, env, services, and portal deep-links). Manual override:
bash scripts/ci/servicenow-cr.sh close \ --cr-sys-id <sys_id_from_CR_URL> \ --close-code successful \ --deployment-duration-seconds 1800 \ --rollback-count 0 \ --health-check-url https://portal.qa.aws/api/health \ --health-check-result "200 OK"Evidence attachments
Section titled “Evidence attachments”Two attest-stage jobs enrich every CR (both allow_failure: true, both
skip clean when ServiceNow is unconfigured):
servicenow:cr:compliance-reportaggregates every scanner + Fides result intocompliance-report.json/.txt(and optional.pdfwhenSN_COMPLIANCE_PDF=true) and attaches it, plus a work note linking pipeline, job, and Fides trail.servicenow:cr:sbom-attachbundles every CycloneDX SBOM (*.cdx.json, up to 9 on a full build) intosarc-sbom-<short-sha>.zipand attaches it via the Attachment API, then PATCHes awork_notesline recording the event.
Infrastructure CRs
Section titled “Infrastructure CRs”There is no Terraform CI in SARC — infra/{aws,azure,gcp} stacks are
applied out-of-band by ops. A CR job servicenow:cr:create:infra stamps
those applies: it opens a CR with u_change_type=Infrastructure on
changes: infra/** on a main push. Both CR types share the qa/prod approval
gates and the risk formula; the only difference is the CMDB bucket. If the
operator supplies a plan summary via INFRASTRUCTURE_CHANGES, it lands on
u_infrastructure_changes.
CR data model & fields
Section titled “CR data model & fields”This section is the field catalogue: what a CR carries, where each value comes
from, and the contract every CR must satisfy for an auditor. The authoritative
writer is scripts/ci/servicenow-cr.sh (GitLab), with Azure DevOps calling the
same script and GitHub Actions patching the same fields onto an sn_devops-
created CR.
Two creators, one record shape
Section titled “Two creators, one record shape”| In-repo pipeline | External-service reconciler | |
|---|---|---|
| Code | scripts/ci/servicenow-cr.sh |
scripts/ci/fides-external-cr-reconcile.mjs |
| Runs | inside the deploy job | scheduled SARC job (out of any pipeline) |
| Covers | podtato-head, karc-portal (karc-pipeline flow) |
dora-dashboard, mcp-api-gateway (Janus) and future services |
| Field source | pipeline env vars | Fides trail + change-gate verdict + per-flow FLOW_META |
Both authenticate as the ServiceNow integration user. ARC writes the CR + CI links + artifact graph; ServiceNow decides approval and state.
Custom-field provisioning & counts
Section titled “Custom-field provisioning & counts”Two scripts provision the schema (both idempotent — they GET sys_dictionary /
sys_db_object first, then POST only missing rows; both exit 0 with
“not configured, skipping” when SERVICENOW_URL is unset):
scripts/servicenow/create-custom-fields.sh— adds theu_*fields to the stockchange_requesttable.scripts/servicenow/create-custom-tables.sh— creates the three Fides custom tables, their columns, anditil-scoped READ ACLs. The ACL step requires the elevatedsecurity_adminrole (it 403s for a plain integration user); table + column creation succeed without it. Without the ACLs/api/now/table/u_fides_*returns 403 to the portal evidence widget.
Documented change_request custom-field count: 43 (24 core + u_platform
- 4
u_gitlab_*+ 6u_azdo_*from #171, + 8 deployment-clearance/evidence fields). The script provisions 51 total (43 documented + 8 pre-existingu_github_*created originally in the SN UI).
Core u_* fields (representative)
Section titled “Core u_* fields (representative)”The full 43-field table lives in docs/ci/servicenow/SCHEMA.md; the most
load-bearing fields:
| Field | Type | Written by | Read by |
|---|---|---|---|
u_target_cloud |
string | CR create | portal change list |
u_deploy_env / u_environment |
string / choice | CR create | approval policy |
u_commit_sha |
string(40) | CR create | portal, audit — the golden join key |
u_fides_trail_url |
string(255) | CR create / fides-attest | portal evidence tab |
u_pipeline_url |
string(255) | CR create | portal, approvers |
u_source |
string(64) | CR create | canonical: gitlab-ci, github-actions, azure-devops |
u_platform |
choice | CR create | prefer over u_source for new reports |
u_actor |
string(128) | CR create | audit |
u_risk_score |
integer(0–100) | risk scoring | auto-approval rule, portal |
u_risk_level |
choice low,medium,high |
risk scoring | approval routing |
u_auto_approved |
boolean | auto-approve | portal, audit |
u_branch / u_repository |
string | CR create | portal |
u_change_type |
string | CR create | Deployment | Infrastructure routing |
u_deployment_method |
string(32) | deploy-helm | helm / argocd |
u_cluster_namespace |
string(64) | deploy-helm | rollback tooling (karc-<env>) |
u_rollback_available / u_previous_version |
boolean / string | deploy-helm | rollback button |
u_services_deployed |
string(4000) | deploy-helm | portal (CSV/JSON list) |
u_security_scanners / u_security_scan_url |
string | security-scan | portal, evidence |
u_infrastructure_changes |
string(4000) | INFRASTRUCTURE_CHANGES env |
CAB |
u_deployment_duration |
integer | deploy-helm | metrics (seconds) |
Platform-identification families (empty fields dropped per detected
platform): u_gitlab_* (pipeline_id, project_id, mr_iid, runner_id);
u_github_* (repository, workflow, run_id, ref, commit_sha, actor,
branch, pr_number); u_azdo_* (organization, project, pipeline_id,
build_id, pr_id, agent_pool).
Deployment-clearance / evidence fields (post-Fides schema-drift fix) —
u_risk_assessment and u_risk_evidence_chain (both string 4000, stringified
JSON), u_application_url, u_signatures_url, u_sarif_results_url,
u_sbom_url, u_infrastructure_report_url, and u_fides_attestation. These
were historically missing from the field script; ServiceNow silently drops
unknown columns on POST, so on old instances the 5-axis clearance JSON and the
evidence links were silently lost even though the CR looked complete.
Re-run the field script to close the drift.
Reference fields & sys_id resolution
Section titled “Reference fields & sys_id resolution”Two change_request fields store a 32-char sys_id, not a display name:
assignment_group (references sys_user_group) and assigned_to (references
sys_user). scripts/ci/servicenow-resolve-refs.sh converts names/emails to
sys_ids and caches them into cr.env (GROUP_SYS_ID, USER_SYS_ID):
bash scripts/ci/servicenow-resolve-refs.sh group "Change Management" # → 32-char sys_idbash scripts/ci/servicenow-resolve-refs.sh user "alice@company.com"Exit 0 = match, 1 = no match / SN not configured, 2 = auth failure. The CR
creator never hard-fails on resolution: a group miss falls back to the display
name; a user miss omits assigned_to. Override the assignee with
SERVICENOW_DEFAULT_ASSIGNEE_EMAIL (used for bot-authored GitOps commits).
Field guide (source + format)
Section titled “Field guide (source + format)”Standard ITIL narrative fields are built by build_narrative_fields() and all
exceed ServiceNow’s minimum-length constraints by construction:
short_description ([SARC <cloud>/<env>] <commit title>, ≤160 chars),
description, justification, implementation_plan (8 steps),
backout_plan (6 steps), risk_and_impact_analysis, test_plan (6 steps).
Preview any of them with:
bash scripts/ci/servicenow-cr.sh create --dry-runNotable field sources: u_approval_required_by is SLA-based
(qa 24h, prod 48h, dev 4h; override via
SERVICENOW_APPROVAL_SLA_{QA,PROD,DEV}_HOURS); u_previous_version uses a
4-step fallback chain (PREVIOUS_COMMIT_SHA env → gitops image-tag.yaml →
last successful CR in namespace → empty); u_deployment_duration is computed as
now - DEPLOY_START_TS from cr.env.
Cross-platform field parity (#445)
Section titled “Cross-platform field parity (#445)”GitLab, GitHub Actions, and Azure DevOps all land the same canonical field
set. GitLab is authoritative (servicenow-cr.sh create POSTs the full
payload). ADO calls the same script (deriving GitLab-style CI_* vars from
Build.* + git log, passing AZDO_*). GitHub creates the CR through the
ServiceNow/servicenow-devops-change action (sn_devops) to preserve DevOps
tool linkage, then a servicenow-cr.sh patch-fields step PATCHes the remaining
canonical fields using the same helper functions so computed values match.
One deliberate parity exception: assignment_group / assigned_to are excluded
from the GitHub patch-fields step (resolving a reference sys_id can trip the
“Abort changes on group” business rule; sn_devops already sets assignment).
servicenow-cr-enrich.sh sets no u_* fields — it only uploads SBOM/SARIF
attachments and HTML work notes.
Traceability requirements (the auditor’s checklist)
Section titled “Traceability requirements (the auditor’s checklist)”SARC’s policy is advisory + verify: the pipeline always sets the links,
always reads them back, writes a work_notes summary, and warns loudly on a
gap — but a gap never blocks a deploy (a missing CI link is a CMDB data
problem, not a defect in the change).
Required CR fields: cmdb_ci (the single primary CI — the anchor SN’s
impact analysis and conflict detection read), u_commit_sha,
u_fides_trail_url, business_service, service_offering.
Required related lists: Affected CIs (task_ci, one row per touched CI),
Impacted Services (task_cmdb_ci_service, one row per impacted service),
and — as a known gap — Artifact Versions (sn_devops_artifact_version,
pending the sn_devops.integration role; the traceability work note records
artifact_versions pending sn_devops.integration role so the gap is visible).
The binary-anchor rule: key cmdb_ci on the image digest (sha256:...),
never the tag. A digest identifies one binary forever; a tag is a mutable
pointer. _resolve_primary_ci() resolves the digest against
cmdb_ci_kubernetes_workload / cmdb_ci_kubernetes_pod (u_image_digest),
then name match, then the business-service anchor.
The related-list tables
Section titled “The related-list tables”These are separate tables, not CR fields. When adding rows via REST, use the exact column names — ServiceNow silently discards unknown properties (returns 201 with a blank row):
| Related list | Table | Link columns |
|---|---|---|
| Affected CIs | task_ci |
task (= CR sys_id), ci_item (= a cmdb_ci) |
| Impacted Services | task_cmdb_ci_service |
task, cmdb_ci_service (NOT ci_item) |
| Artifact Versions | scripted sys_relationship |
walks the chained sn_devops_* records |
The Artifact Versions “related list” is a scripted relationship
(getArtifactVersions) that walks:
change_request → sn_devops_change_reference → sn_devops_evidence_repository_version → sn_devops_evd_repo_version_relationship → sn_devops_package → sn_devops_m2m_artifact_version_package → sn_devops_artifact_version → sn_devops_artifact. Alongside it a CI-authored native spine
(sn_devops_tool → sn_devops_pipeline/sn_devops_repository → sn_devops_pipeline_execution and sn_devops_commit, joined by
sn_devops_m2m_artifact_version_commit) makes the full
commit → artifact_version → pipeline_execution → change_reference → CR chain
dot-walk. There is exactly one sn_devops_change_reference per CR (the
instance’s “Prevent duplicates” rule), carrying package_ref +
pipeline_executions. The version is derived deterministically from the digest
(1.<int(first 8 hex)>) so re-runs are stable. No sn_devops.integration role
is required to write these tables directly (only the vendor orchestration path
needs it).
The two join keys
Section titled “The two join keys”Everything on a CR points back to one of:
- Commit SHA (
u_commit_sha) — the golden key across source, pipeline, CR, and reconciler idempotency. - Image digest (
sha256:...) — the binary anchor forcmdb_ci, the Affected CI, and the artifact version. Never a tag.
CMDB & CSDM model
Section titled “CMDB & CSDM model”Beyond the CR, ARC writes a complete, connected picture of the platform into ServiceNow’s CMDB. The north star: point at any binary (a container image digest deployed at a point in time) and see the full picture around it — what produced it, where it runs, what it depends on, and why it is allowed to run. The binary is the anchor; everything else is a relationship hop away. The model sticks to native ServiceNow classes (Common Service Data Model / CSDM and the certified cloud Service Graph Connector classes) so ServiceNow’s own apps — Service Mapping, Dependency Views, DevOps Change Velocity, GRC — light up with no extra work.
The four pivots from the binary
Section titled “The four pivots from the binary” ┌─ WHY ── Fides trail + controls + framework (SOC2/ISO) + approved CR │ commit → package → BINARY (cmdb_ci_docker_image, by digest) ← anchor │ │ WHAT ────────────┘ ├─ WHERE ─ workload → pod → node → EC2 → EBS / subnet / VPC │ (+ cluster, cloud, region) └─ TOUCHES ─ service dependencies (upstream & downstream)Join keys: the commit SHA (through Fides trail, CR u_commit_sha,
DeploymentRecord.commitSha) and the image digest (stamped on
sn_devops_artifact, cmdb_ci_docker_image, cmdb_ci_kubernetes_workload).
Rule: never key traceability on a tag.
Layer-to-class mapping (all verified on calitiiltddemo3)
Section titled “Layer-to-class mapping (all verified on calitiiltddemo3)”Infrastructure / cloud: cmdb_ci_kubernetes_cluster (each carries
cluster_version, e.g. 1.34), cmdb_ci_kubernetes_namespace,
cmdb_ci_kubernetes_workload, cmdb_ci_kubernetes_pod,
cmdb_ci_kubernetes_service, cmdb_ci_kubernetes_node, cmdb_ci_vm_instance
(EC2), cmdb_ci_storage_volume (EBS), cmdb_ci_network (VPC),
cmdb_ci_cloud_subnet, cmdb_ci_compute_security_group,
cmdb_ci_cloud_load_balancer.
Application: cmdb_ci_service (CSDM Application Service), a
per-environment cmdb_ci_service_auto App Service (<service> - <env>,
carrying the live version + environment), cmdb_ci_business_app (the
Business Application anchor), cmdb_ci_appl, cmdb_ci_spkg (SBOM package), and
cmdb_ci_docker_image (the deployment-side binary anchor). cmdb_sw_component_install
is Discovery/SAM-owned (write-guarded) and is NOT written by the pipeline/portal.
Build lineage (DevOps Change Velocity): sn_devops_tool (the hub),
sn_devops_pipeline, sn_devops_pipeline_execution, sn_devops_repository,
sn_devops_commit, sn_devops_artifact (the build-side anchor; carries image
name + digest + version), sn_devops_package, and
sn_devops_m2m_artifact_version_commit (the commit↔artifact-version join).
sn_devops_artifact ↔ cmdb_ci_docker_image are the same binary seen from
build vs runtime — they join on the digest.
Compliance (GRC): sn_compliance_policy (framework — SOC 2, ISO 27001, the
karc-pipeline control set), sn_compliance_control (one per pipeline control),
sn_grc_profile (the entity pointing at a CMDB CI), sn_grc_item (pass/fail
evidence per control per binary).
Relationship types (only those verified present): Runs on::Runs,
Hosted on::Hosts, Depends on::Used by, Contains::Contained by,
Members::Member of, Uses::Used by. Storage/network attachment uses
Uses::Used by (the Provides storage for / Connected to types are absent).
Custom CMDB fields (portal-written enrichment)
Section titled “Custom CMDB fields (portal-written enrichment)”The portal writes to cmdb_ci_service by default (configurable per tenant via
the cmdbCiClass setting; the previous default was cmdb_ci_application). CIs
are named after the repo/project/app from the webhook source, and
(name, version) uniquely identifies a deployed artefact. Every ARC-written CI
carries discovery_source=karc-portal so re-syncs reconcile via IRE instead of
duplicating.
The 9 cmdb_ci_service custom fields (provisioned by
scripts/servicenow/create-cmdb-fields.sh): u_container_image,
u_image_digest, u_namespace, u_helm_release, u_argocd_app, u_flux_kustomization, u_provider,
u_pipeline_url, u_commit_url, u_last_synced_at. All are best-effort — if
the Kubernetes API is unreachable the upsert still proceeds with the five core
fields.
K8s-native CI topology (#173)
Section titled “K8s-native CI topology (#173)”A second, parallel layer represents Kubernetes infrastructure directly. All classes are stock (from the CMDB CI Class Models plugin); SARC only adds extension fields and relationship types.
| Class | Key fields | Parent |
|---|---|---|
cmdb_ci_kubernetes_cluster |
name, u_cluster_uid |
(top-level) |
cmdb_ci_kubernetes_namespace |
name |
Contained by cluster |
cmdb_ci_kubernetes_workload |
name, u_workload_kind |
Contained by namespace + cluster |
cmdb_ci_kubernetes_service |
name, namespace (or k8s_uid) |
Contained by namespace |
cmdb_ci_kubernetes_ingress |
k8s_uid |
Contained by namespace |
cmdb_ci_kubernetes_pod |
(cluster, namespace, name) composite |
Contained by namespace + cluster |
Two version-field gotchas to remember:
- Cluster version uses the stock native
cluster_versionfield (not au_column). The old code wrote a non-existentu_versionon the cluster, which SN silently dropped. The portal reads it live via the KubernetesVersionApi, so an EKS/AKS/GKE upgrade shows on the next sync. - Workload version uses the custom
u_versionfield (the first container’s image tag, viaparseImageTag), a second provisioning pass oncmdb_ci_kubernetes_workload. It drives the CMDB Version column for rollback / change assessment.
The ingress class is cmdb_ci_kubernetes_ingress (identified by k8s_uid,
inheriting the cmdb_ci_kubernetes_component identify rule). The portal
originally used the generic cmdb_ci_ingress, which IRE rejected as a non-CMDB
class — the sync now passes metadata.uid as k8s_uid on every ingress upsert
or IRE fails with MISSING_MATCHING_ATTRIBUTES.
Workload extension fields: u_pod_count, u_ready_replicas,
u_desired_replicas, u_workload_kind, u_version. Pod extension fields
(#173 C3): u_workload_name, u_workload_kind, u_phase, u_image_digest,
u_node_name. Provision the layer with scripts/servicenow/create-cmdb-k8s.sh.
Pod sync churn: pods sync on a 5-minute cadence (vs 15 min for
cluster/namespace/workload), rate-limited via a token bucket
(CMDB_POD_SYNC_RATE_LIMIT_PER_MIN default 200,
CMDB_POD_SYNC_BUCKET_CAPACITY default 50). Disappeared pods are retired
(operational_status=2), not deleted, to preserve the audit trail; terminal
pods (Succeeded/Failed) are retired too.
Known gap: IRE identification rules are warn-only for the K8s classes. The rule API’s payload shapes are undocumented and version-varying, so the scripts check for rules and print manual UI instructions but do not create them. Without rules, IRE creates a new CI on every sync (duplicates) — an operator must create the rules once in the UI, then run a dedup job.
CMDB sync via ArgoCD notifications
Section titled “CMDB sync via ArgoCD notifications”The CMDB CI upsert can be driven by ArgoCD: the notifications controller POSTs a
webhook to the portal (/api/webhooks/argocd?tenant=<slug>) every time an
Application reaches Synced + Healthy. The portal verifies an HMAC-SHA256
signature (X-Argo-E2E-Signature), strips the ApplicationSet cloud+env suffix
(karc-portal-aws-dev → karc-portal), and calls triggerCmdbUpsert with
provider=argocd. u_argocd_app is set directly from the Application name.
Configure the HMAC secret in argocd-notifications-secret (per cluster,
openssl rand -hex 32, via ExternalSecret in production), add the
service.webhook.karc-portal + template.cmdb-sync-payload +
trigger.on-cmdb-sync blocks to argocd-notifications-cm, subscribe
webhook:karc-portal to on-cmdb-sync, and store the encrypted secret on the
tenant’s argocdWebhookSecretEncrypted. Verify with a synthetic payload — first
call returns status ok, a repeat returns status duplicate
(idempotency guard); an Invalid signature error means the secrets differ.
CMDB sync via Flux notifications
Section titled “CMDB sync via Flux notifications”The Flux (GitOps Toolkit) parity of the ArgoCD path — for clusters that deploy
with Flux instead of ArgoCD. The Flux notification-controller POSTs an
HMAC-signed webhook to /api/webhooks/flux?tenant=<slug> every time a
Kustomization or HelmRelease reconciles successfully. The portal verifies
X-Signature: sha256=<hex> against the tenant’s fluxWebhookSecretEncrypted,
fires only on reason=ReconciliationSucceeded, derives the service from
involvedObject.name, and calls triggerCmdbUpsert with provider=flux,
stamping u_flux_kustomization = <kind>/<namespace>/<name>.
Important: this is the real-time layer, not the only path. The portal
already observes Flux (Kustomizations/HelmReleases on the GitOps page) and the
engine-agnostic full-cluster sweep (/api/cmdb/sync-clusters, reads the raw
Kubernetes API) already writes Flux clusters’ CMDB CIs — so Flux CMDB data is
never missing, this just adds real-time updates + GitOps provenance on parity
with ArgoCD.
Configure by applying gitops/flux/cmdb-notifications.yaml on the Flux cluster
(a Provider of type generic-hmac + an Alert selecting Kustomization/
HelmRelease info events), create the karc-portal-cmdb-webhook secret holding
the HMAC token (via ExternalSecret in production), and store the same value on
the tenant’s fluxWebhookSecretEncrypted. Full runbook:
docs/ci/servicenow/CMDB-FLUX-NOTIFICATIONS.md.
Note: SARC’s own demo deploys with ArgoCD, so there are no Flux workloads to fire this today — it is a customer-readiness feature for Flux-based clusters.
Binary-traceability showcase (live on sarc-aws)
Section titled “Binary-traceability showcase (live on sarc-aws)”Real discovered data on sarc-aws proves the model end to end. Live class counts:
cmdb_ci_network 1, cmdb_ci_cloud_subnet 3, cmdb_ci_compute_security_group
1, cmdb_ci_vm_instance 4, cmdb_ci_storage_volume 9,
cmdb_ci_kubernetes_node 4, cmdb_ci_cloud_load_balancer 1,
sn_compliance_policy 1, sn_compliance_control 6.
Walkthrough on one binary — janus@sha256:6382f90a9c1a (MCP API Gateway),
namespace janus, cluster sarc-aws:
- WHERE — image ← workload
mcp-api-gateway→ Runs on nodeip-10-20-94-175→ Runs on EC2i-0cbc46e4934f1b03d→ Uses EBS/subnet/SG; node is a Member of clustersarc-aws. - WHAT — image Uses package
janus-d92fd7235fcecarrying the commit SHA and source repo. - TOUCHES — the
cmdb_ci_servicemcp-api-gatewayDepends on the workload. - WHY — framework “SARC Compliance Pipeline” (6 controls mapped to
SOC 2 / ISO 27001), a
sn_grc_profilefor the binary, and approved Change CHG0032935 whoseu_commit_shamatches the deployed commit.
Follow-ups (documented, not blockers): production automation of infra discovery
via the cmdb-infra-sync module (needs ec2:Describe* /
elasticloadbalancing:Describe* read-only on the karc-portal-eks-reader IRSA
role); native DevOps lineage (needs the sn_devops.integration role); GRC
control→policy binding; and coverage across all 3 EKS clusters plus Azure/GCP.
Fides custom tables
Section titled “Fides custom tables”create-custom-tables.sh creates three tables joined to a CR by
u_fides_*.trail = change_request.u_commit_sha. A CI job
servicenow:fides:sync writes one row per artifact / test / vulnerability
(idempotent, allow_failure: true, skips when Fides or ServiceNow is
unconfigured). The auto-approval rule can query u_fides_vulnerability by
trail + severity to escalate a CR whose trail carries unresolved criticals.
| Table | Row source | Key columns |
|---|---|---|
u_fides_artifact |
trail artifacts[] |
artifact_fingerprint, pipeline_id, artifact_name, flow, trail |
u_fides_test_result |
trail test attestations | test_suite_name, passed_tests, failed_tests, skipped_tests, trail |
u_fides_vulnerability |
SARIF + GitLab scanner reports | vulnerability_id, severity, scanner_name, package, version, fixed_in, trail |
Business-service mapping
Section titled “Business-service mapping”This is portal behaviour on top of the CMDB: how a karc-portal Service links
to its ServiceNow cmdb_ci_service CI, and how that link drives the
ServiceNow-backed surfaces (incidents, change requests, problems, SLA,
vulnerability correlation). Both data structures live in the portal’s Postgres
(Prisma), not in ServiceNow.
The two data structures
Section titled “The two data structures”Service.businessServiceSysId— a singlesys_idon eachServicerow pointing at onecmdb_ci_serviceCI. The canonical “which CMDB CI is this service?” link.nullmeans no CMDB CI yet. Each portal service maps to its own same-named Application/Technical Service CI (cmdb_ci_serviceis used rather thanservice_offeringbecause the integration user usually lacks ACL Insert onservice_offering).ServiceBusinessMapping— a Prisma model unique on(tenantId, workloadKey)whereworkloadKey = "<cloud>:<cluster>:<namespace>:<name>". It maps an individual Kubernetes workload to a business service and supplies the human-readablebusinessServiceNamefor display.
Writers (order matters)
Section titled “Writers (order matters)”| Writer | What it does |
|---|---|
prisma/seed-servicenow-push.ts |
upserts a cmdb_ci_service per non-archived Service (queried by name), writes the sys_id back to Service.businessServiceSysId. Idempotent. Run first. |
prisma/seed-business-mapping.ts |
walks every operational CmdbWorkloadCi, upserts a ServiceBusinessMapping per workloadKey. Skips a workload if the service has no businessServiceSysId. Run second. |
POST/PUT/DELETE /api/cmdb/business-mapping |
admin CRUD from Settings → Business Mapping |
The two seeds are steps 7 and 8 of the demo seed Job. Re-establish the link on a live tenant:
SEED_TENANT=<tenant> npx tsx prisma/seed-servicenow-push.tsSEED_TENANT=<tenant> npx tsx prisma/seed-business-mapping.tsDisplay
Section titled “Display”The Service Portfolio (/services) renders a Business Service label:
businessServiceSysId → businessNameBySysId[sysId] (from
ServiceBusinessMapping) → falls back to the service’s own displayName
(the CMDB CI is named after the service, so that is the correct label — it never
shows the raw sys_id hex). A dash means one thing only:
businessServiceSysId is null (no CMDB CI).
Consumers
Section titled “Consumers”The sys_id filters ServiceNow records down to a service and gates tenant
visibility: /api/servicenow/incidents (filter by business_service),
/api/servicenow/sla/[serviceSlug], /api/servicenow/crs (tenant allowlist:
CR visible if its cmdb_ci / business_service is in the owned-CI set),
/api/servicenow/problems, /api/vulnerabilities/[id]/related, and
/api/settings/tenant-scope (builds the owned-CI allowlist).
CSDM note: SARC’s demo keeps it flat — one
cmdb_ci_serviceper portal service. Do not repointService.businessServiceSysIdat a shared Business-Service roll-up, or the incident/SLA/CR filters would collapse every service onto the same records.
Portal & API surface
Section titled “Portal & API surface”The karc-portal Next.js app consumes ServiceNow through a tenant-scoped
TypeScript client plus app-router API routes. Credentials come from tenant
config (encrypted in the DB), not process.env, keeping the portal
multi-tenant — each tenant has its own instance URL and service account.
ServiceNow REST API (as SARC uses it)
Section titled “ServiceNow REST API (as SARC uses it)”SARC talks to ServiceNow via the Table API only. Auth is Basic
(Authorization: Basic base64(user:password)), credentials from CI variables
SERVICENOW_USER / SERVICENOW_PASSWORD, instance SERVICENOW_URL (must
include the scheme). Recommended CI user: a dedicated sarc.ci service account
with roles itil, rest_service, x_sarc.api.
| Purpose | Method + Path |
|---|---|
| Create CR | POST /api/now/table/change_request |
| Read CR | GET /api/now/table/change_request/<sys_id> |
| Find CR by SHA | GET /api/now/table/change_request?sysparm_query=u_commit_sha=<sha>^u_target_env=<env>^u_target_cloud=<cloud>&sysparm_limit=1 |
| Advance / close / reject | PATCH /api/now/table/change_request/<sys_id> (set state, work_notes, close_code, close_notes) |
| Install iframe Client Script | POST /api/now/table/sys_script_client |
| Install iframe UI Action | POST /api/now/table/sys_ui_action |
| Attachment upload | POST /api/now/attachment/upload?table_name=change_request&table_sys_id=<sys_id>&file_name=<name> |
Rate limits: the default REST message limit is 60 req/sec per instance (a
single pipeline run stays well under). Aggressive CAB dashboards querying
change_request can starve the deploy pipeline — prefer scheduled reports.
Common errors: 401 (rotate SERVICENOW_PASSWORD, check MFA not required
for REST), 403 (itil or scoped write ACL missing on
sys_script_client/sys_ui_action), 404 on PATCH (stale sys_id — re-find by
SHA), 400 on state (wrong choice integer), 409 (duplicate CR — another run won
the race, re-find by SHA and attach), 429 (back off 2s/5s/10s), 500 (retry up to
3× with backoff).
Portal TypeScript client & routes
Section titled “Portal TypeScript client & routes”src/lib/servicenow.ts (ServiceNowClient) wraps the Table API + the IRE:
getChangeRequests, getChangeRequest, approveCR / rejectCR,
updateChangeRequest, getCIs / getCI / getCIRelationships,
upsertCIviaIRE, testConnection. src/lib/servicenow-sarc.ts layers SARC
types (SarcTargetCloud, SarcDeployEnv, SarcChangeRequest) and query
builders (buildSarcCRQuery, parseTargetCloud, parseDeployEnv,
parseCRState, listSarcChangeRequests).
| Method | Path | Purpose |
|---|---|---|
| GET | /api/servicenow/crs |
list CRs (?query=, ?pending=true, ?cloud=, ?env=, ?state=, ?limit=, ?offset=, ?tenant=) |
| GET | /api/servicenow/crs/[sysId] |
single CR |
| POST | /api/servicenow/crs/[sysId]/approve |
combined approve/reject (RBAC: APPROVER or ADMIN) |
| POST | /api/servicenow/crs/[sysId]/reject |
standalone reject (RBAC: APPROVER or ADMIN) |
| GET | /api/servicenow/cmdb |
list CMDB CIs for the tenant’s CI class |
| GET | /api/health/servicenow |
connectivity probe |
Routes instantiate the client per-request from tenant config; return 503
ServiceNow not configured when the tenant has no credentials (the list route
soft-returns {result: []} so empty widgets do not flap), and 502 with the
upstream body on ServiceNow errors. Write routes require an APPROVER/ADMIN
session and write an auditLog row on success. Dashboard widgets
(PendingCRsWidget, CmdbHealthWidget) are server components fetching via the
client directly.
CI variables & rotation
Section titled “CI variables & rotation”The pipeline reads credentials from protected (usually masked) GitLab CI variables:
| Variable | Protected | Masked | Notes |
|---|---|---|---|
SERVICENOW_INSTANCE_URL / SERVICENOW_URL |
yes | yes | instance URL (+ alias) |
SERVICENOW_USERNAME |
yes | yes | Basic auth user |
SERVICENOW_PASSWORD |
yes | no | contains a char outside GitLab’s mask allowlist — rotate to a mask-compatible value |
SN_ORCHESTRATION_TOOL_ID / SN_ORCHESTRATION_PROJECT_ID |
yes | yes | ServiceNow DevOps extension |
SERVICENOW_APP_ID / SERVICENOW_APP_SECRET |
yes | yes | OAuth (future/parity) |
SNOW_TOKEN / SNOW_TOOLID / SNOW_URL |
yes | yes | KARC aliases (deprecated) |
FIDES_API_TOKEN |
yes | yes | all fides:attest:* jobs |
FIDES_API_ORG_ID |
yes | no | calitii, 7 chars — below the 8-char mask minimum; not sensitive |
Rotation cadence: SERVICENOW_* and FIDES_* quarterly;
SERVICENOW_APP_SECRET on-incident + quarterly; SN_ORCHESTRATION_* never (SN
side auto). Troubleshooting: “not configured, skipping” means the variable is
unset or the job runs on a non-protected branch; a 401 from ServiceNow means the
password rotated on the SN side but not here.
Network / IP allowlist
Section titled “Network / IP allowlist”For a customer whose ServiceNow instance is perimeter-controlled, this covers
the egress IPs that GitLab.com, GitHub.com, and Azure DevOps hosted runners use
to call in. Direction is one-way, CI platform → ServiceNow on TCP 443;
reverse callbacks are not required. The full CIDR lists and fetch/refresh
commands live in docs/ci/servicenow/IP-ALLOWLIST.md — this is the essentials.
The full allowlist is large and weak. Hosted runners deploy dynamically across public cloud ranges, so a UK customer still needs ~500–700 CIDRs across the three platforms, they rotate weekly, and the allowed set is essentially “chunks of GitHub/GCP/Azure public IP space.” Prefer one of the stronger alternatives first:
| Option | Allowlist size | Security | Best for |
|---|---|---|---|
| Full IP allowlist | 500–700, weekly refresh | weak | compliance tick-box baseline |
| A: self-hosted runners + static NAT | 1–2 | strong | most pragmatic for fintech |
| B: ServiceNow MID Server broker | 1 | strong | already SN-native |
| C: mTLS client certs + loose IP | zero/loose | very strong | mature sec teams |
| D: OIDC federation (JWT bearer) | zero | very strong | green-field stacks |
| E: SARC Phase-C broker relay | 1 | moderate | brokers already running |
Recommendation for a regulated customer: combine A or B (small static allowlist) with C or D (identity by cert/JWT, not IP).
If the full list is mandated, fetch live: GitLab Linux runners are on GCP
us-east1/us-central1 (~250–300 CIDRs via cloud.json), macOS on AWS
us-east-1 (~50–80 via ip-ranges.json); GitHub Actions publishes .actions[]
in api.github.com/meta (~180–200, weekly); Azure DevOps hosted-agent egress
uses AzureCloud.<region> service tags (the AzureDevOps tag is inbound only).
Apply on the SN side via System Security → IP Address Access Control
(plugin com.snc.ipauthenticator) or a per-endpoint REST API Access Policy.
Verify with a curl -sf -u user:pass .../api/now/table/incident?sysparm_limit=1
from each platform — HTTP 403 means the runner’s IP is not allowed.
Service Graph Connectors
Section titled “Service Graph Connectors”Service Graph Connectors (SGC) are ServiceNow’s MID-less integrations that pull
cloud resource inventory (clusters, VMs, VPCs, storage, IAM) into the CMDB
directly over REST. SARC wires SGC for AWS, Azure, and GCP so the
sarc-<cloud> clusters and their surrounding infrastructure appear natively.
The credential and RBAC side is Terraform-managed; the connection binding is
finished once in the ServiceNow UI.
Common shape
Section titled “Common shape”Each connector: a cloud service principal / service account with read-only
roles (created by infra/<cloud>/servicenow-sgc-*.tf), the credential stored in
the cloud secret store, a cmdb_ci_cloud_service_account (CSA) row, a
scheduled_data_import set active, and a set of connection records that flip
from Pending to Green after the first successful run.
The one manual step (all three clouds): REST-triggered execution of
scope-restricted Store-app schedules is blocked for non-system users on
calitiiltddemo3. The Table API accepts the active=true PATCH but the
scheduler ignores the rescheduled run. In the SN UI: filter-navigate to the
parent schedule, open the scheduled_data_import record, right-click header →
Execute Now; the child schedules cascade. Or wait for the natural cron tick.
SGC-AWS
Section titled “SGC-AWS”Set up in an earlier session: an IAM user + access key, the
SG-AWS-Credentials-Org credential populated, and SG-AWS-Organization active.
Same manual Execute-Now step applies.
SGC-Azure
Section titled “SGC-Azure”Discovers the sarc-azure AKS cluster and the Development subscription. The
Azure AD app sarc-servicenow-sgc has Reader + Log Analytics Contributor
scoped to the subscription, secret in Key Vault sarc-servicenow-sgc-secret.
The parent schedule SG-Azure Subscriptions cascades ~25 child schedules
(Resource Group, Virtual Machine, Network, AKS Clusters, Storage, Key Vault,
SQL, Functions, …). Within 15 min of the first run, cmdb_ci_kubernetes_cluster
gets sarc-azure, plus VM/VNet/subnet/LB/Key Vault/storage/resource-group rows.
Rotate the client secret (Terraform-issued, 2-year lifetime; policy 90 days) via
terraform taint azuread_application_password.servicenow_sgc then
scripts/servicenow/configure-sgc-azure.sh. The Hardware connection staying
Pending is expected (it needs Azure ARC-enabled servers, which SARC does not
have).
SGC-GCP
Section titled “SGC-GCP”Discovers the sarc-gcp project (sarc-493418) via the Cloud Asset Inventory
API. The service account servicenow-sgc-discovery@sarc-493418.iam.gserviceaccount.com
holds seven read-only roles (cloudasset.viewer is the primary discovery grant,
plus compute.viewer, container.viewer, iam.securityReviewer,
storage.objectViewer, monitoring.viewer, logging.viewer), JSON key in
Secret Manager sarc-servicenow-sgc-key. The parent schedule
SG-GCP Organization cascades ~20 child schedules.
The REST path does not fully automate the connection setup: the SGC-GCP
Store app runs in the sn_gcp_integ scope with cross-scope ACLs that block
integration users from writing credential/property fields, and the wizard
triggers server-side JWT provider creation that cannot be invoked externally. So
Setup is a one-shot wizard walkthrough (Credentials → Discovery scope
projects/sarc-493418 → Schedule → first run); rotations can still be
automated. Provision the SA with a targeted terraform apply; fetch the key for
the wizard with
gcloud secrets versions access latest --secret=sarc-servicenow-sgc-key --project=sarc-493418
(shred the temp file after pasting). Common wizard errors: a 403 on
cloudasset.googleapis.com means the API is not enabled
(gcloud services enable cloudasset.googleapis.com); an empty CMDB after import
usually means the Discovery Scope is missing the required projects/ prefix.
Dashboards
Section titled “Dashboards”Two audiences consume CR data from the ServiceNow side: CAB reviewers who
want a pending-approval queue segmented by env and cloud, and compliance /
audit who want evidence of closure + Fides linkage per release. Both are
served by standard ServiceNow Dashboard widgets backed by list reports on
change_request filtered with PQL (platform query language).
Representative encoded queries (paste into
<SERVICENOW_URL>/change_request_list.do?sysparm_query=<encoded>):
Open CRs in Authorize, any cloud state=-3^ORDERBYsys_created_onPending prod approvals, per cloud state=-3^u_target_env=prod^u_target_cloud=awsAuto-approved in last 24h state=3^work_notesCONTAINSAuto-approved^sys_updated_onONToday@...High-risk (score 8-10) u_risk_scoreBETWEEN8@10Evidence completeness (audit finding) u_fides_trail_url=ISEMPTY^state!=1CAB dashboard widgets: pending approval by env (pie, state=-3), pending by
cloud (bar), oldest waiting (list sorted by sys_created_on asc),
auto-approval rate over 24h (single score), Fides trail links (list with
u_commit_sha + u_fides_trail_url). Compliance dashboard: closure SLA,
per-cloud traffic over time, rejected/timeout breakdown by close_code, and
evidence completeness (any row with an empty trail is an audit finding).
PQL tips: ^ is AND, ^OR is OR; u_* fields are case-sensitive; prefer
ONToday / ONLast 7 days over brittle @javascript:gs... date macros. The
same queries work against /api/now/table/change_request for a nightly report
scheduled outside ServiceNow.
Setup & provisioning runbooks
Section titled “Setup & provisioning runbooks”The scripts below provision the ServiceNow-side schema and integrations. They
are idempotent (GET-before-POST) and CI-safe (exit 0 with a skip message when
SERVICENOW_URL is unset). Setup scripts need admin-level credentials —
higher privilege than the pipeline’s integration user. Never store admin
credentials in CI variables; run these locally with the instance credential from
.envrc (dev) or 1Password/Keychain (prod).
| Script | Provisions | Credential note |
|---|---|---|
scripts/servicenow/create-custom-fields.sh |
the 43 documented + 8 pre-existing u_* fields on change_request |
dictionary-admin |
scripts/servicenow/create-custom-tables.sh |
the 3 u_fides_* tables, columns, and READ ACLs |
ACL step needs security_admin (403s otherwise); table/column creation works without it |
scripts/servicenow/create-cmdb-fields.sh |
the 9 cmdb_ci_service fields + u_version on cmdb_ci_kubernetes_workload |
dictionary-admin |
scripts/servicenow/create-cmdb-k8s.sh |
K8s CI classes’ extension fields + relationship types (10 steps) | dictionary-admin; IRE identification rules stay manual |
scripts/servicenow/install-fides-iframe.sh |
the View Fides Trail Client Script + UI Action |
needs write ACL on sys_script_client / sys_ui_action |
scripts/servicenow/configure-sgc-azure.sh |
pushes the Azure SGC credential into SN | — |
scripts/servicenow/normalize-u-source.sh |
backfills legacy u_source values to canonical slugs |
— |
Typical field-provisioning run:
export SERVICENOW_URL=https://<instance>.service-now.comexport SERVICENOW_USER=adminexport SERVICENOW_PASSWORD=...bash scripts/servicenow/create-custom-fields.shbash scripts/servicenow/create-custom-tables.sh # ACL step needs security_adminVerify a field via sys_dictionary, a table via sys_db_object:
curl -u "$SERVICENOW_USER:$SERVICENOW_PASSWORD" \ "$SERVICENOW_URL/api/now/table/sys_dictionary?sysparm_query=name=change_request^column_name=u_risk_score&sysparm_fields=column_name,internal_type,max_length"Rotation runbook: edit the field/column spec in the script, edit the matching row in the source doc, run the script against the target instance, and verify. Schema migrations must ship together with the CI or portal change that depends on them — never ahead, never behind. Neither script deletes fields or tables; drop a field manually in the UI (System Definition → Dictionary) and remove it from the script + doc in the same commit.
Reference index (source docs)
Section titled “Reference index (source docs)”Every source file under docs/ci/servicenow/ and the section here that covers
it. Open the raw file for the exhaustive detail (full 43-field tables, complete
CIDR lists, verbatim scripts).
| Source doc | Covered in |
|---|---|
docs/ci/servicenow/README.md |
Overview & architecture |
docs/ci/servicenow/CR-WORKFLOW.md |
Change Request workflow |
docs/ci/servicenow/AUTO-DEPLOYMENT-WORKFLOW.md |
Auto-deployment story |
docs/ci/servicenow/APPROVALS.md |
Approvals |
docs/ci/servicenow/BUSINESS-RULES.md |
Business rules SARC expects |
docs/ci/servicenow/SCHEMA.md |
CR data model & fields |
docs/ci/servicenow/FIELD-GUIDE.md |
Field guide |
docs/ci/servicenow/CR-FIELD-PARITY.md |
Cross-platform field parity |
docs/ci/servicenow/CR-DATA-FLOW.md |
Two creators, related-list tables |
docs/ci/servicenow/CR-TRACEABILITY-REQUIREMENTS.md |
Traceability requirements |
docs/ci/servicenow/CMDB-CSDM-MODEL.md |
CMDB & CSDM model |
docs/ci/servicenow/CMDB-SCHEMA.md |
Custom CMDB fields, K8s-native CI topology |
docs/ci/servicenow/CMDB-ARGOCD-NOTIFICATIONS.md |
CMDB sync via ArgoCD notifications |
docs/ci/servicenow/CMDB-FLUX-NOTIFICATIONS.md |
CMDB sync via Flux notifications |
docs/ci/servicenow/CMDB-BINARY-TRACEABILITY-SHOWCASE.md |
Binary-traceability showcase |
docs/ci/servicenow/BUSINESS-SERVICE-MAPPING.md |
Business-service mapping |
docs/ci/servicenow/API.md |
ServiceNow REST API |
docs/ci/servicenow/PORTAL.md |
Portal TypeScript client & routes |
docs/ci/servicenow/CI-VARIABLES.md |
CI variables & rotation |
docs/ci/servicenow/IP-ALLOWLIST.md |
Network / IP allowlist |
docs/ci/servicenow/SGC-AZURE-CONFIGURE.md |
SGC-Azure |
docs/ci/servicenow/SGC-GCP-CONFIGURE.md |
SGC-GCP |
docs/ci/servicenow/SGC-GCP-SETUP.md |
SGC-GCP |
docs/ci/servicenow/DASHBOARDS.md |
Dashboards |