Intent & Behavior Evaluation

As the Risk Assessment activities in Layer 3 produce Policies, the activities in Layer 5 produce opinions through evaluation of Policy Compliance. These Evaluations consist of structured inspection in the form of either Intent Evaluation or Behavior Evaluation. These two types of Evaluation do not need to happen in parallel, but they are most informative for Enforcement decisions when the results are compiled.

Evaluations typically include multiple Assessments with independent failure modes. The best way to ensure compliance with organizational Policy — and assess unmitigated Risk — is to clearly document the relationship between each Assessment and an Assessment Requirement that is defined for a particular Control.

Examples of Intent Evaluation include both human-to-human interviews and automated or manual examinations of configurations. In information systems, many traditional Audit activities rely on manual Intent Evaluation such as dashboard screenshots, and modern trends are increasingly shifting toward automated configuration scanning and code analysis.

Analog Intent Evaluations may look at a team’s procedures, training records, hardware, or facilities. A digital equivalent would include software composition analysis or cloud resource configuration scanning.

Behavior Evaluation may take the form of any action which observes or simulates user behavior to ensure that the expected outcomes are achieved. While simulation of bad behavior is useful for identifying security gaps or non-compliance with Policy, regular simulation of good behavior is also useful for ensuring that the system always operates as expected. In an analog environment this could manifest as secret shoppers, while evaluating an information system might include penetration testing.

The Opinions of an evaluator are firmly held approximations of reality formed within the constraints of its philosophy, perspective, and capabilities (or those of its creator). This is a post hoc Risk Assessment that establishes an understanding of what Risks have been introduced or proposed.

The process involves comparing the Evaluation Findings to the organization’s expectations, which are ideally captured in Policies through Controls and Assessment Requirements. While evaluators may be provided by vendors and industry groups alike, robust Evaluation should be informed by specific Policies in order to custom-tailor the Assessment to the needs of the Compliance program.

As seen in Figure 7.1, proper maintenance of relationships, also known as “mappings,” between each artifact allows Evaluations to provide multiple opportunities to demonstrate a system’s state of Compliance with various relevant artifacts.

Figure 7.1: Mapping Evaluations Logs to Definition Artifacts Figure 7.1: Mapping Evaluations Logs to Definition Artifacts
Figure 7.1: Mapping Evaluations Logs to Definition Artifacts

Continue Reading