Human-Centered and Trustworthy Artificial Intelligence for Cybersecurity in Cyber-Physical Systems: A Reproducible Confidence-Aware Triage Framework

2.1 Cyber-physical security is a decision-support problem

Operational technology differs from ordinary information technology in ways that matter to artificial intelligence design. The available response time can be constrained; maintenance windows may be limited; observations may arise from legacy systems; and an apparently suitable protective response can interrupt an essential physical process. NIST Special Publication 800-82 Revision 3 identifies industrial control systems, building automation, transportation, physical-access systems, and physical-environment monitoring as examples within the operational-technology scope [1]. In this setting, a false negative can leave harmful activity undetected, whereas a false positive can consume scarce analyst time or encourage a disruptive intervention. An alert classifier is thus not a stand-alone predictor; it is one component in a safety- and mission-sensitive decision workflow.

This property also explains why generic “human in the loop” claims are insufficient. A person nominally placed at the end of an automated pipeline may see only a label and a color-coded confidence value, lack access to the evidence needed for independent assessment, and face workload conditions that reward rapid acceptance. Such an arrangement retains formal human involvement while delegating substantive judgment to the model. Conversely, a carefully designed workflow can use automation to filter volume, present information provenance, disclose uncertainty and limitations, solicit an analyst’s preliminary assessment in selected cases, and retain rationale-bearing decisions for post-incident learning. The relevant question is not whether a human appears somewhere in the process; it is whether the human has the authority, information, and time needed to make a meaningful decision.

2.2 Machine-learning attack surfaces and operational consequences

A trustworthy intrusion-detection workflow must address threats to the model as well as threats detected by the model. NIST’s adversarial-machine-learning taxonomy organizes relevant concepts across machine-learning methods, life-cycle stages, and attacker goals, objectives, capabilities, and knowledge [4]. It recognizes data poisoning, evasion, and privacy breaches among the central security concerns. For an operational system, these categories have distinct consequences. Evasion may involve values or traffic patterns manipulated to move a malicious activity across a learned decision boundary. Poisoning may occur through compromised training data, corrupted labels, or feedback imported into a retraining pipeline. Privacy threats can arise if learned models or their explanatory outputs reveal whether a sensitive event record was part of training, an attack commonly conceptualized as membership inference [5].

These risks should not be reduced to a general statement that the model is “robust.” Robustness is conditional on a threat model that identifies protected assets, adversary access, feasible manipulations, and system constraints. In a cyber-physical environment, the set of feasible manipulations is narrower than arbitrary synthetic perturbation but may be more consequential. A protocol or register value cannot be altered in evaluation if the altered combination is physically impossible or incompatible with the device role. Conversely, a slow change in traffic rate, a spoofed measurement within a plausible range, or a compromised gateway emitting plausible logs may be operationally credible. Evaluation needs to document those choices so that reviewers can assess what a robustness result means and what it does not mean.

2.3 Trust calibration and explanation limits

Many security interfaces imply trust through a probability, severity band, or visual salience. Such signals are problematic when model confidence does not correspond to the empirical likelihood of correctness. Guo and colleagues define confidence calibration as producing probability estimates representative of the true correctness likelihood and show that modern neural networks can be poorly calibrated; they identify temperature scaling as a simple post-processing approach that was effective across many of their evaluated datasets [6]. The implication for security triage is limited but important: raw classifier confidence is not adequate evidence for allowing a recommendation to bypass routine review. Calibration must be fitted on held-out data and assessed separately from discrimination metrics.

Even a calibrated probability does not fully characterize trust. A high probability may correspond to a familiar training distribution but an explanation whose salient features vary sharply under minor, domain-valid changes. That instability can indicate sensitivity to irrelevant correlations or a fragile boundary. Conversely, a stable explanation does not prove that a model has recovered a causal attack mechanism. Shapley Additive Explanations provide a framework for assigning feature-importance values to an individual prediction [7], but these values describe a model’s local use of its inputs under specified modeling and background-distribution assumptions. They should be presented as evidence for review, alongside input provenance and known limits, not as forensic proof or an assertion of attacker intent.

The four NIST principles of explainable artificial intelligence—explanation, meaningfulness, explanation accuracy, and knowledge limits—provide a practical standard for this distinction [8]. An alert needs a reason; the reason must be understandable to its intended user; the explanation should accurately describe the model behavior being represented; and the interface should indicate what the system cannot know from the available inputs. In a cyber-physical setting, “knowledge limits” include unavailable sensor modalities, delayed logs, missing asset context, and attacker behavior outside the reference data. A system that hides such limits can be more dangerous than one that offers no explanation, because it can create unwarranted epistemic confidence.

2.4 Human factors, cognitive security, and feedback risk

Security operations are also exposed to cognitive attacks. Phishing, social engineering, and disinformation seek to shape beliefs and decisions, often by exploiting time pressure, authority cues, and incomplete information. The machine-learning interface can reproduce related vulnerabilities if it presents automation as authoritative or makes dissent costly. Automation bias has been studied as a decision-support phenomenon across domains, with reviewers documenting that decision aids can improve performance yet can also induce omissions or commissions when users defer to automated advice [9]. The present paper does not assume that effect sizes from other settings transfer to cyber-physical security. Rather, it treats the risk as a design and measurement question that must be tested with target users.

A second risk concerns the status of human feedback. Analyst dispositions can contain valuable context unavailable to telemetry alone: maintenance activity, current operations, an incident commander’s assessment, or an observed false alarm. However, feedback can be incomplete, inconsistent across analysts, or compromised by a malicious insider. Directly appending clicks to a training table transforms a human-interface convenience into a poisoning pathway. A responsible design preserves feedback as a versioned, reviewable artifact, distinguishes immediate operational decisions from adjudicated labels, and applies validation before any feedback enters a new model version. This separation is central to accountable hybrid intelligence.

2.5 Research gap and research questions

Available industrial intrusion datasets make it possible to evaluate classifiers on diverse attack patterns, but a typical benchmark compares accuracy, precision, recall, or an aggregate F1 score under a single data split. That approach leaves several connected questions unanswered. Does a model’s score correspond to its likelihood of being correct? Which alerts are safe enough to be recommended, and which should be deferred? Does its explanation remain stable under harmless measurement variations? Are failures concentrated in particular operational contexts? Can analysts identify and challenge an inadequate recommendation? What information is preserved when the system changes over time?

This paper addresses that gap through three research questions:

The first two propositions can be evaluated with public testbed data once datasets, versions, and preprocessing are documented. The third requires a prospective human-subject study; no human-performance result is claimed here. This separation is methodologically important. It permits an engineering framework to be reviewed as a reproducible, falsifiable design while preventing a simulated dataset from being mistaken for evidence about human trust, workload, or safety.

3.1 Design principles

The framework is guided by five principles. First, prediction is not authorization: a predicted class remains a model output, whereas a recommendation is a governed communication to an operator. Second, deferral is informative: a review-required status must include the evidentiary condition that caused it. Third, the analyst retains decision authority: the framework supports triage and escalation but does not automatically change a physical process. Fourth, feedback is controlled evidence: operational dispositions are not automatically training labels. Fifth, trustworthiness is plural: calibration, explanation, privacy, fairness, security, and accountability cannot be replaced by a single score without concealing trade-offs.

These principles align with the AI Risk Management Framework’s lifecycle orientation [3] while adapting it to a specific operational workflow. Governance defines decision rights, unacceptable use, data ownership, and criteria for criticality. Mapping characterizes assets, users, input sources, threat models, and potential harms. Measurement evaluates model and human-system behavior under prespecified conditions. Management assigns controls, response actions, and change-approval responsibilities. The resulting architecture is intended to make those functions observable in routine operation rather than confining them to a policy document.

3.2 System architecture

The framework contains six components: ingestion and provenance, detection, trust assessment, explanation generation, analyst triage, and feedback/audit management. Ingestion receives telemetry, flow records, logs, and asset context. It time-aligns records, checks schemas, flags missingness, and stores source and transformation provenance. The detection component estimates a binary or hierarchical attack label. It is deliberately model-agnostic: an interpretable baseline, a tree ensemble, or a neural classifier may be used, provided its training and calibration process is documented.

The trust-assessment component decides whether a detection can be displayed as a recommendation. It examines the calibrated class probability, a novelty or uncertainty signal, explanation stability, asset criticality, and mandatory policy constraints. The explanation component generates a concise evidence card that may include contributing features, the reference window, source systems, model version, and explicit limitations. The triage workspace enables an analyst to confirm, reject, relabel, escalate, defer, or request more evidence. It records a structured rationale, not merely a binary click. Finally, the feedback and audit service maintains immutable records, monitors drift and disagreement, and gates any retraining activity through validation and approval.

3.3 Decision policy

For an observation $x$, let the detector yield class probabilities $p(y\mid x)$. A calibration function $g$, fitted on an independent validation partition, transforms those probabilities into $q(y\mid x)=g(p(y\mid x))$. Let $c(x)=\max_y q(y\mid x)$ denote calibrated confidence. Let $u(x)$ denote an uncertainty or novelty score, $s(x)$ an explanation-stability score, $h(x)$ a documented asset-criticality score, and $P(x)$ an indicator that a mandatory human-review policy applies.

A model output may be communicated as a decision-support recommendation only when all of the following conditions are satisfied:

Otherwise the workspace displays review required. The policy is conjunctive by design. A high class probability does not compensate for a novel input, unstable explanation, policy-restricted context, or high-impact asset. Where an asset has high criticality, the framework routes the case as a high-priority review rather than silently automating it. Thresholds are not universal constants; they are selected on validation data using a documented cost model, then approved by accountable system owners. Any change in the model, data distribution, asset inventory, or response policy triggers re-evaluation.

The policy’s purpose is not to claim a perfect trust decision. It offers a transparent mechanism for managing selective prediction. A classifier may abstain from routine recommendation when its evidence is inadequate, preserving an analyst’s time for cases in which machine judgment has the least defensible basis. Importantly, the framework logs whether the abstention occurred because of confidence, novelty, explanation instability, criticality, or policy. Those categories allow later analysis of whether the system is deferring for sensible reasons or merely transferring ambiguity to operators without useful prioritization.

3.4 Explanation card and analyst workflow

Every alert card should identify the predicted class, calibrated confidence, alert priority, data sources, observation window, model version, top contributing signals, and the reason a recommendation was issued or withheld. It should contain a concise limitation statement such as: “This assessment reflects network-flow and host-log features observed in the listed time window; it does not establish physical process impact or attacker intent.” The card should not describe the system as certain, autonomous, or sentient. This wording choice is a security control: it reduces the risk that interface rhetoric inflates confidence beyond the available evidence.

For a randomized subset of cases, the framework uses an evidence-first interaction. The analyst initially assesses the evidence and records a preliminary disposition before seeing the model recommendation. The recommendation is then revealed, and the analyst may revise or retain the decision with a recorded rationale. This design does not burden every routine case; it provides a measurement instrument for detecting anchoring and overreliance while retaining realistic workflow constraints. The sampling probability, eligibility rules, and randomization seed must be logged so that the intervention can be analyzed without post hoc selection.

The workflow also provides explicit challenge mechanisms. An analyst can mark the underlying data as incomplete, contest the explanation, request an alternative view, escalate to a second reviewer, or classify the event as insufficiently evidenced. Such options are not cosmetic. They allow disagreement to become structured information. A high rate of data-quality challenges may reveal a failing ingestion path; repeated explanation contests may reveal an interface or modeling problem; asymmetric acceptance rates across shifts or asset contexts can indicate inappropriate reliance. These signals should be reviewed in governance meetings, not only during a model incident.

3.5 Feedback, retraining, and auditability

A decision-support system should distinguish at least four labels: model prediction, analyst disposition, adjudicated incident outcome, and retraining label. The first is generated by the current model. The second reflects a time-bounded operational judgment. The third may emerge only after investigation and may be unavailable for many cases. The fourth is an approved representation used to train or validate a future model. Conflating these labels hides uncertainty and creates an attack surface.

The proposed feedback pipeline therefore uses a quarantine stage. New feedback is versioned with analyst role, timestamp, evidence references, reason codes, and model version. It is checked for duplication, anomalous concentration, conflict with available incident outcomes, and unusual behavior by source. A reviewer or an adjudication protocol then determines whether feedback can be used for learning. The full retraining candidate is subject to the same data-card, split, calibration, and security review as the initial dataset. This process does not remove poisoning risk, but it makes potential manipulation inspectable and prevents a single operational click from silently altering the next model.

Auditability also requires that an alert can be reconstructed. A reconstruction record contains the raw-data identifier or protected reference, transformations, model and calibration versions, threshold configuration, explanation background distribution, criticality policy, recommendation state, analyst interactions, and final disposition. Privacy controls may restrict who can access the underlying data, but restricted access does not eliminate the need for provenance. The aim is a complete, access-controlled trace rather than indiscriminate data retention.

4.1 Scope and evidence posture

This section defines a protocol rather than reporting completed performance results. That choice is intentional: without a documented acquisition, preprocessing, execution environment, and independently reproduced output, numerical results would be unverifiable. The protocol enables a future implementation to generate such evidence using public datasets while preventing common overclaims. Its principal outcome is not a single highest score; it is an auditable comparison of recommendation quality, coverage, error distribution, calibration, and deferral behavior.

The primary candidate dataset is X-IIoTID. Its official record describes a testbed with industrial control-loop devices, including sensors, actuators, and controllers, and combines network traffic, system logs, application logs, device-resource signals, and intrusion-detection logs [10]. The record reports 820,834 instances, 68 features, normal and attack records, and multiple label levels, providing a useful basis for binary and hierarchical classification. Its documented attack scenarios include reconnaissance, brute force, malicious-insider activity, reverse shell, man-in-the-middle, lateral movement, data exfiltration, false-data injection, ransomware, and denial-of-service events [10].

Edge-IIoTset is an important secondary benchmark. Its source documentation describes a layered Internet-of-Things and industrial Internet-of-Things testbed spanning cloud, network-function virtualization, blockchain, fog, software-defined networking, edge, and perception layers [11]. It documents fourteen attacks in denial-of-service/distributed-denial-of-service, information-gathering, man-in-the-middle, injection, and malware categories [11]. ToN_IoT provides a complementary suite of network, telemetry, and operating-system resources from an official university project page [12]. These sources provide public paths to replication, but the final empirical paper must identify the exact files, access date, license terms, and checksums; a dataset name alone is not a reproducibility artifact.

4.2 Task definition and preprocessing

The initial task is binary intrusion detection, followed by hierarchical classification only when the selected dataset’s label definitions support it. This ordering prevents a multiclass label hierarchy from obscuring the practical first question: whether a case requires cybersecurity attention. Any mapping from raw labels to binary classes must be published. Labels that are ambiguous, unavailable at decision time, or derived from post-incident knowledge must not be used as features. Similarly, record identifiers, capture-file names, timestamps that encode scenario identities, or fields containing the answer by construction must be excluded or separately audited.

Preprocessing occurs inside the training partition. Categorical encoders, imputers, scaling parameters, feature-selection routines, and resampling strategies must be fitted only on training folds and then applied to validation and test data. Deduplication must be performed before splitting when duplicate records would otherwise appear in both training and test partitions. The implementation should write a manifest listing raw file paths, SHA-256 checksums, excluded columns, transformation parameters, and transformed-table checksums. This information allows an independent investigator to distinguish a genuine model difference from a data-version difference.

Class imbalance requires careful treatment. A weighted loss or training-only resampling approach may be used, but it should be chosen before inspecting test performance and reported alongside the original test-set prevalence. Oversampling before splitting contaminates the evaluation by allowing near-identical synthetic or copied records to influence both training and testing. Conversely, undersampling can remove attack diversity. The protocol therefore reports both the original counts and all training-only balancing steps, and it evaluates with macro-average measures as well as class-specific errors.

4.3 Leakage-resistant split regimes

Three split regimes are required. A random stratified split serves only as a compatibility baseline with prior benchmark practice. A time-aware split trains on earlier observations and evaluates later observations where trustworthy timestamps are available. A source- or scenario-aware split holds out a device, simulator context, protocol source, or attack scenario when metadata permits. The final method must state exactly which grouping keys were used. If group metadata are unavailable, that limitation must be disclosed rather than obscured by a nominally realistic claim.

The rationale is simple. Neighboring rows from the same capture session or repeated traffic flow can be highly correlated. If they are divided at random, a model may learn capture-specific artifacts rather than generalizable attack behavior. A system deployed later or on another asset may then perform materially worse than its random-split score implies. The proposed protocol treats a divergence between split regimes as a substantive finding about dependence and generalization, not as an inconvenient result to be hidden. It is particularly important for datasets with scripted attack episodes or testbed-specific background traffic.

A fixed test partition is not tuned against. Hyperparameter search is conducted only with nested or training-only validation. The calibration partition is disjoint from both model-fitting and final-test records. Each full experimental configuration is repeated under prespecified random seeds, with the seed values published. If resource limits prevent repeated runs, the paper must report that limitation and avoid confidence intervals that imply independent replications.

4.4 Models and calibration

The minimum model set comprises regularized logistic regression and random forest. Logistic regression provides a transparent reference and can reveal whether a complex model offers material value beyond a linear boundary. Random forest captures nonlinear relations while remaining broadly reproducible. A gradient-boosted tree model or a neural model may be added where justified by the dataset and compute environment, but it does not change the framework’s contribution. The paper should not claim novelty because it uses a particular classifier; the methodological novelty lies in the conditional recommendation policy, the measured oversight, and the traceable evaluation.

Models are tuned with training-only cross-validation. After finalizing a model, a calibration function such as temperature scaling is fitted on the independent calibration partition. The main calibration measures are expected calibration error, Brier score, and reliability diagrams. Expected calibration error should be reported with a stated binning scheme because its magnitude varies with bin construction. Reliability diagrams should show sample density or counts to prevent sparse bins from being overinterpreted. Calibration results are reported per task and, where sample sizes permit, per justified operational subgroup.

An uncertainty or novelty component is also required. The precise method may be an ensemble-disagreement measure, distance-based score, conformal predictor, density estimate, or other method appropriate to the selected classifier. Its choice must be operationally defensible and available at inference time. An explanation-stability component computes whether the ranks or signs of top attributions remain consistent across small, domain-valid perturbations such as plausible measurement noise, benign missingness, or resampling of the explanation background. Perturbations that create impossible device-protocol combinations or violate known physical constraints must not be used merely because they are mathematically convenient.

4.5 Selective prediction and risk–coverage reporting

Let $D$ denote the independent test set and $A_\tau\subseteq D$ the set of records for which the policy issues a recommendation rather than deferring to review. Recommendation coverage is $|A_\tau|/|D|$. Selective risk is the mean loss among recommended cases. A useful loss may assign different weights to false negatives, false positives, and misclassified attack families, but the weights must be prespecified with operational justification. If no agreed cost model is available, the report should present several sensitivity scenarios rather than choose a favorable weight after observing results.

The principal result is a risk–coverage plot that compares raw-confidence gating, calibrated-confidence gating, and the full policy with confidence, novelty, stability, criticality, and mandatory-review checks. At selected coverage levels, the report provides classwise recall, precision, macro-F1, balanced accuracy, expected calibration error, Brier score, and the number and proportion of deferred cases. Crucially, it breaks deferral down by cause. A method that produces low selective risk only because it defers most meaningful attack cases is not operationally useful. A method that recommends high-confidence cases but repeatedly defers a particular device context may indicate a data-coverage or fairness concern.

The result table below is intentionally a reporting template, not an assertion of measured performance. It prevents the reader from mistaking a proposed experimental protocol for completed evidence.

4.6 Fairness, privacy, and adversarial evaluations

Fairness evaluation requires restraint. Public industrial intrusion datasets often do not include legitimate demographic attributes about people. Researchers must not invent demographic groups, infer sensitive characteristics, or label a system “fair” based solely on aggregate accuracy. Where justified non-sensitive operational groups are available—such as device type, site simulator, traffic modality, asset criticality, or protocol family—the framework reports sample size, false-positive rate, false-negative rate, recall, precision, calibration, and deferral rate by group. These diagnostics assess reliability and equitable burden across operational contexts; they are not demographic fairness certification.

A privacy review begins with data minimization. The data card identifies direct identifiers, quasi-identifiers, sensitive operational fields, access controls, retention periods, and redaction procedures. If model probabilities or explanations are exposed outside the tightly controlled operational boundary, the implementation should evaluate membership-inference exposure under a documented attacker model. Shokri and colleagues formalize membership inference as determining, given a data record and black-box access to a model, whether that record was used in training [5]. Whether that scenario is relevant depends on the deployment and data exposure, so the analysis should be threat-driven rather than perfunctory.

The adversarial evaluation documents test cases for feasible evasion, poisoning, and extraction or privacy attacks. For example, a poisoning analysis may introduce bounded label corruption or anomalous feedback patterns only in a controlled offline experiment, then measure whether quarantine and validation detect it. An evasion analysis may apply physically and protocol-valid feature changes, then compare clean and stressed risk–coverage behavior. Each result must name the adversary knowledge, access, perturbation budget, and constraints. In line with the NIST taxonomy, the objective is to manage specific attack consequences and provide a common terminology for assessment [4], not to imply universal security from one benchmark.

4.7 Reproducibility package

Reproducibility is treated as an engineering deliverable. The package should include a README, data card, source manifest, split-generation code or fixed split identifiers, environment lockfile, configuration files, model card, alert-card schema, experiment log, and evaluation notebook. The contents are summarized below.

The code repository should preserve failed runs and deviations where practical. A negative result—such as no improvement in selective risk, instability of explanations, or severe degradation under a time-aware split—is not evidence of a failed research process. It is evidence about the limits of a proposed technique, and it should be reported with the same care as a favorable metric.

5.1 From nominal oversight to measurable control

Human oversight becomes meaningful only when it changes system behavior or provides independently assessable evidence. In the proposed framework, an analyst can reject a recommendation, ask for more evidence, reclassify an event, escalate it, or defer action. These choices influence the operational case record but do not immediately retrain the model. The system documents its own limits and indicates why it chose to defer. This allocation of roles prevents a superficial arrangement in which the analyst simply ratifies an opaque system output.

The quality of oversight should be evaluated across four dimensions: authority, information, timing, and learning. Authority asks whether an analyst can meaningfully overrule the model and whether escalation pathways are available. Information asks whether the interface discloses input provenance, confidence, constraints, and relevant evidence. Timing asks whether the workflow allows an independent initial assessment in selected cases and whether alerts arrive with sufficient context. Learning asks whether disagreements, overrides, and later incident outcomes are preserved and reviewed. None of these dimensions can be inferred from a classifier’s F1 score.

A practical audit can sample completed cases and calculate trace-completion rate: the proportion for which source provenance, model version, threshold configuration, explanation information, analyst disposition, and final outcome status are available. Low trace completion is a governance defect even when the model’s aggregate prediction score is high. Similarly, the proportion of cases in which analysts use the challenge mechanisms, the rate of recommendation reversals, and the pattern of disagreement by operational context can reveal whether the interface encourages independent reasoning.

5.2 Prospective human-subject study

A future human-subject study should recruit qualified cybersecurity analysts or advanced practitioners and obtain institutional ethical review, informed consent, and data-handling approval where required. Participants receive a balanced, independently sampled set of adjudicated or simulated cases. They work under two counterbalanced conditions: evidence-first, in which they assess the case before seeing the system recommendation, and immediate-recommendation, in which the recommendation appears at once. A prespecified subset contains deliberately incorrect model advice, designed to be credible but not deceptive beyond the research purpose.

Primary outcomes are final decision accuracy against adjudicated ground truth, time to disposition, self-reported confidence, acceptance of seeded wrong advice, rate of explanation challenge, and use of escalation. Analysis should account for repeated measures by participant and case difficulty. Baseline expertise, prior exposure to similar systems, and workload should be recorded because they can influence reliance. A workload instrument may be used only when licensed and included in the approved protocol. The study should distinguish performance improvement from agreement: greater agreement with a model is not a success if the model is wrong.

The main test concerns a concrete safeguard. If immediate recommendation disclosure produces more acceptance of seeded wrong advice than evidence-first disclosure, then the interface should retain or expand the consider-then-reveal mechanism for relevant alert classes. If it does not, the added interaction cost may not be justified. This is an example of falsifiability: the framework makes a design claim testable and allows the data, not intuition, to determine whether the control is retained.

5.3 Explainability evaluation

Explanation quality has at least three distinct aspects. Fidelity asks whether the explanation corresponds to the model behavior it purports to describe. Stability asks whether the explanation changes disproportionately under small, valid changes to the input or explanation procedure. Usefulness asks whether an intended analyst can use it to make a better or more defensible decision. These aspects should not be conflated. An explanation can have technical fidelity yet be incomprehensible to an analyst, or be visually appealing while unrelated to the actual model decision.

For tabular intrusion detection, the explanation card should display only the most relevant attributions with direction, observed value, comparison context, and provenance. It should avoid a long ranked list that gives an illusion of completeness. The stability test repeats an attribution method over domain-valid perturbations and reports top-$k$ rank correlation, sign agreement, and any changes to the recommendation eligibility decision. A low stability result is itself a reason for review-required status. Analysts should be told that explanations describe the model’s use of features and do not prove a sequence of malicious actions.

Knowledge limits should be explicit. An alert generated from network-flow features cannot determine whether a physical actuator was actually commanded unless that information was observed. A model trained on simulated testbed traffic cannot establish that a live industrial environment will behave similarly. A feature attribution cannot establish causality or exonerate an omitted feature. Making these limits visible is not an admission of weakness; it is a condition of calibrated trust.

5.4 Privacy and cognitive security

Privacy in cyber-physical security can concern operator identifiers, device identity, operational schedules, location-linked data, business-sensitive process information, and the learnability of records from model outputs. The framework uses purpose limitation and minimization: collect and retain data necessary for the specified security objective, segregate access by role, and redact exports used for external analysis. Raw events should not be copied into demonstration interfaces unless needed and authorized. Audit logs must preserve enough information for reconstruction without becoming unrestricted surveillance records.

Cognitive security extends the privacy concern into interface and organizational design. An adversary may target the analyst, supply misleading incident context, compromise a feedback channel, or exploit the model interface’s perceived authority. Controls include authenticated evidence provenance, visible data-quality flags, controlled recommendation language, explicit challenge pathways, dual review for high-impact contexts, and quarantined feedback. These measures do not eliminate social engineering; they create friction and traceability at the points where manipulated information could alter a security decision.

5.5 Fairness as context-sensitive reliability

In this domain, fairness must be tied to a defensible unit of analysis. If a deployed system serves different operational sites, device categories, or traffic modalities, persistent false positives in one context may impose disproportionate workload and degrade trust. Persistent false negatives in another may create unequal security exposure. The framework therefore examines group-conditional rates and calibration where valid group definitions exist. It also reports the sample size and uncertainty around each estimate so that small groups are not overinterpreted.

A fairness finding should yield an action. If a device group has materially poorer recall, the team may collect additional representative data, adjust a review threshold, restrict the model’s intended use, or add a group-specific monitoring control. If sample sizes are inadequate, the report should state that no reliable group comparison can be made. This approach reflects NIST’s emphasis on fairness with mitigation of harmful bias [2] while avoiding an unsupported claim that operational categories are proxies for human demographic groups.