Add initial support for generating SPDX SBOM documents (COMPOSER-2274)

[thozza]

Depends on:

• Add initial support for generating SPDX SBOM documents (COMPOSER-2274) osbuild#1818

• Add initial support for generating SPDX SBOM documents (COMPOSER-2274) images#930

• Extend the worker Depsolve job to allow requesting an SBOM document for each transaction. The SBOM documents are returned in the DepsolveJobResult.

• CloudAPI:

  • Unconditionally request SBOM documents for every Depsolve job.
  • Add new /composes/{id}/sboms API endpoint for requesting SBOM documents for composes.
  • Extend the non-Koji and Koji compose unit tests to verify the newly added /composes/{id}/sboms API endpoint.

• There's no support for SBOMs added to Weldr API.

• Koji:

  • Extend the Koji target result to include SBOM documents uploaded to the Koji build.
  • Extend the KojiFinalize job to import all SBOM documents uploaded to the Koji build and listed in the Koji target result.
  • Extend the Koji target handling in the OSBuild job, to upload all SBOM documents, related to the OSBuild job, to the Koji build and list them in the Koji target result. Uploaded SBOM documents use the following filenames: <image_filename>.<pipeline_purpose>-<pipeline_name>.spdx.json.
  • Extend the koji.sh test case to verify the uploaded SBOM documents in Koji builds.
  • Update the tested scenarios for koji.sh. Specifically, test the main scenario on RHEL-9 and in addition, test building of RHEL-8 and RHEL-10 on RHEL-9.

Details

SBOM documents are attached to DepsolveJobResult. For each package set that was depsolved, there will be a corresponding SBOM document. DepsolveJob usually contains at least two package sets for any image type - for the at least one image pipeline and for the buildroot pipeline.

Koji composes may contain multiple image builds as part of a single compose request. Therefore the /composes/{id}/sboms endpoint returns multiple SBOM documents for each image build. The returned value may look as follows for a Koji compose that consists of two image builds:

{
	"href": "/api/image-builder-composer/v2/composes/356f4620-c03e-3830-b013-ddbfee6665aa/sboms",
	"id": "356f4620-c03e-3830-b013-ddbfee6665aa",
	"kind": "ComposeSBOMs",
	"items": [
		[
			{
				"pipeline_name": "build",
				"pipeline_purpose": "buildroot",
				"sbom": <sbom_doc>,
				"sbom_type": "spdx"
			},
			{
				"pipeline_name": "os",
				"pipeline_purpose": "image",
				"sbom": <sbom_doc>,
				"sbom_type": "spdx"
			}
		],
		[
			{
				"pipeline_name": "build",
				"pipeline_purpose": "buildroot",
				"sbom": <sbom_doc>,
				"sbom_type": "spdx"
			},
			{
				"pipeline_name": "os",
				"pipeline_purpose": "image",
				"sbom": <sbom_doc>,
				"sbom_type": "spdx"
			}
		]
	]
}

pipeline_purpose describes the purpose of a pipeline. image means that the content described in the SBOM document is installed in the actual image. buildroot means that the content described in the SBOM document was installed in the environment used to build the image. This terminology is consistent with how we name thing internally.

pipeline_name is the actual name of the osbuild pipeline, in which the described content got installed.

sbom is the actual SBOM document.

sbom_type describes the SBOM standard / type of the SBOM document. Currently, it will be always spdx.

[achilleas-k]

Great work. This is a lot.

Couple of notes and nitpicks. I think there's a bug in the sbomsFromOSBuildJob() function (noted below) but I'm not entirely sure if I'm reading it right so not Requesting changes.

[achilleas-k]

NITPICK: Flipping the condition with continue would decrease the indentation for the largest part of the loop body, which is nicer IMO.

[thozza]

Yeah, I don't see any problem with it.

[achilleas-k]

Should this break be inside the depJobType == worker.JobTypeDepsolve block? The way it is now, only one the first manifestJobInfo.Deps element is considered. Is that always a depsolve job and nothing else? I guess no, otherwise we'd just be using manifestJobInfo.Deps[0] right?

[thozza]

Should this break be inside the depJobType == worker.JobTypeDepsolve block?

It actually is inside block as far as I can tell 🤔

So the break is at the correct place, but this is a bit of a premature optimization based on the current implementation. Let me explain.

I implemented sbomsFromOSBuildJob() in a way, so that it uses the same approach to get SBOMs from OSBuildJob, regardless if it is from a Koji or non-Koji compose.

Just for the completeness, for Koji composes, the DepsolveJob is a dependency of the OSBuildJob, so that it can access SBOMs and upload them to Koji. For non-Koji composes, the DepsolveJob is not a direct dependency of the OSBuildJob and one must go through the ManifestJob as to get to DepsolveJob.

There's always only a single DepsolveJob in the compose job chain, thus the break. But there may be more than one item in manifestJobInfo.Deps and DepsolveJob does not have a fixed index.

[achilleas-k]

Copy-paste leftover?

[thozza]

yes.

[achilleas-k]

NITPICK: This is a bit awkwardly phrased and might be misunderstood as "ID of SBOMs". In other words, it kinda sounds like "compose-SBOM" is the object being fetched, so it wants the ID of that, but instead it's the compose ID it needs.

Maybe something like:

ID of compose for which to get SBOMs

though I'm not too happy about that one either.

[thozza]

I agree that this is not great by any means 😄 I'll happily use your suggestion.