Refactor required packages definition for packaging to correctly select dependencies

[pchila]

Elastic Agent packaging is using a list of ExpectedBinaries defined in go code.

The expected binaries list is then used when packaging in multiple places:

in checksums.go there is some prefix/suffix checks to match the necessary dependencies:

elastic-agent/dev-tools/mage/checksums.go

Lines 111 to 119 in 12c5ef7

	for _, spec := range manifest.ExpectedBinaries {
	// If the individual package doesn't match the expected prefix, then continue
	// FIXME temporarily skip fips packages until elastic-agent FIPS is in place
	if !strings.HasPrefix(pkgName, spec.BinaryName) || strings.Contains(pkgName, "-fips-") {
	if mg.Verbose() {
	log.Printf(">>>>>>> Package [%s] skipped", pkgName)
	}
	continue
	}

elastic-agent/dev-tools/mage/checksums.go

Lines 202 to 207 in 12c5ef7

	for _, spec := range manifest.ExpectedBinaries {
	// If the given component name doesn't match the external binary component, skip
	// FIXME temporarily skip fips packages until elastic-agent FIPS is in place
	if componentName != spec.ProjectName || strings.Contains(pkgName, "-fips-") {
	continue
	}

more assumptions are made for the package name based on the binary name, we can see that here

elastic-agent/dev-tools/mage/manifest/manifest.go

Lines 138 to 143 in 12c5ef7

	func (proj BinarySpec) GetPackageName(version string, platform string) string {
	if proj.PythonWheel {
	return fmt.Sprintf("%s-%s.zip", proj.BinaryName, version)
	}
	return fmt.Sprintf("%s-%s-%s", proj.BinaryName, version, PlatformPackages[platform])
	}

This kind of assumptions created issues when *-fips* package names started appearing in manifests since the code above started selecting the wrong packages since those would have a matching prefix and suffix.
This has led to a quick hack implemented in #7486 to filter out fips packages for dependencies and allow packaging of elastic-agent to succeed.

The goal of this issue is to allow for greater control and more precise definition of necessary dependencies for elastic-agent packages, replacing the hacky FIPS dependency exclusion with better package selection checks.

This is a pre-requisite for implementing Elastic Agent FIPS packaging, where a different set of packages for the same elastic agent dependencies needs to be selected.

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)