Intent & Behavior Evaluation

To ensure that policy becomes reality, there should be one or more evaluations which consist of structured inspection of either intent or behavior. These two activities do not need to happen in parallel, but they are most informative when the results are compiled.

Evaluations typically include multiple assessments with independent failure modes. The best way to ensure compliance with organizational policy is to clearly document the relationship between each assessment and an assessment requirement that is defined for a particular control.

Intent Evaluation is an opinionated interview of the actors in a sensitive activity. Examples of this include both human-to-human interviews and automated or manual examinations of configurations. In information systems, many traditional audit activities rely on manual intent evaluation such as dashboard screenshots, and modern trends are increasingly shifting toward automated configuration scanning and code analysis. Analog intent evaluations may look at a team’s procedures, training records, hardware, or facilities. A digital equivalent would be things like software composition analysis or cloud resource configuration scanning. As an example, Prowler is an open source tool for Intent Evaluation on public cloud resources.

Behavior Evaluation is an opinionated observation of simulated or real-world activities. This is any action which observes or simulates user behavior to ensure that the expected outcomes are achieved. While simulation of bad behavior is useful for identifying security gaps or non-compliance with policy, regular simulation of good behavior is also useful for ensuring that the system always operates as expected. In an analog environment this could manifest as secret shoppers, while evaluating an information system might include penetration testing. As an example, Privateer is an open source tool for orchestrating customizable Behavioral Evaluation plugins.

The opinions of an evaluator are firmly held approximations of reality formed within the constraints of its philosophy, perspective, and capabilities (or those of its creator). The process involves comparing the evaluator’s findings to the organization’s expectations — which are ideally captured in policies through controls and assessment requirements. While evaluators may be provided by vendors and industry groups alike, robust evaluation should be informed by specific policies in order to custom-tailor the assessment to the needs of the compliance program.

As seen in Figure 7.2, proper maintenance of relationships, also known as “mappings,” between each artifact allows evaluations to provide multiple opportunities to demonstrate a system’s state of compliance with various relevant artifacts.

Figure 7.1: Mapping Evaluations Logs to Definition Artifacts Figure 7.1: Mapping Evaluations Logs to Definition Artifacts
Figure 7.1: Mapping Evaluations Logs to Definition Artifacts

Continue Reading