Skip to content

Oscal linting validation

brandt keller requested to merge oscal-linting-validation into master

General MR


Adds OSCAL linting and validation through the use of Lula

OSCAL Linting

Previously the pipeline had a single component-definition schema for OSCAL version 1.0.4 with which to validate the file structure of OSCAL files. 6+ OSCAL releases later and there is now even more potential for the pipeline to use the wrong schema for validation.

Lula contains a linting function that can Lint any OSCAL model across any supported version (Currently 1.0.4 -> 1.1.1) and will continue to develop support for newer releases of the OSCAL schema. The pipeline then only has to consume updates to Lula releases in order to gain new linting functionality.

OSCAL Validation

Work in progress is adding functional validations to the OSCAL the resides in the Big Bang packages that maps a given control in a given standard to declarative configuration in a live environment. This will accelerate Big Bang accreditation processes when aggregated for many packages across a deployment of Big Bang.

Changes included in this MR establish a process for assessing a live-cluster before we tear down to ensure the package still contains relevant configuration to support meeting control objectives. Note that it is expected that there will be controls that are not satisfied without other dependencies or packages - and a control that is not-satisfied does not mean package failure. Rather as we iterate we will be establishing a threshold that will be stored in Git for evaluating regressions that reduce standards accreditation.

Relevant logs/screenshots

Updating the linting logic from the older implementation to the Lula implementation is a straight-forward swap for any place it is invoked.

As for control validation against live clusters - the current epic is focused solely on packages currently with logic for umbrella planned for future epic.

Followed documentation for testing with the creation of a package validation MR using the promtail package as a baseline.

In the configuration validation job you will see the linting passing successfully with intended output printed. OSCAL version is no longer a maintenance burden of Big Bang and can be owned by Lula.

Intent is then for the pipeline to validate the controls in the OSCAL against a live deployment of the package in a cluster.

When there is no threshold for comparison of assessments - this conducts the assessment but ultimately lets the pipeline pass as some evaluation criteria is required for pipeline quality gating.

When there is an established threshold for comparison of assessments - this performs an evaluation and - if any controls have regressed - then it will fail.

Evaluation is based on the premise that all controls that previously passed should continue to pass. This allows for iteration of control validation that allows for instances where controls are not intended to pass given some context.

Linked Issue

Closes #251 (closed) Closes #250 (closed)

Edited by brandt keller

Merge request reports