This avoids the wrong binaries from the previous build being included in
the installer binary when (cross-)compiling locally for multiple targets
in sequence.
go generate always needs to be run natively, even when cross-compiling.
Handling that in build.sh simplifies cross-compiling both manually and
in installer-artifacts.
Add verification that the vendoring of the terraform sub-modules is
in-sync with what is needed for building the sub-modules.
https://issues.redhat.com/browse/CORS-1909
Update the go version used for verifying vendoring from 1.16 to 1.17.
The CI job uses its own image and does not use the image specified
in the script. Therefore, the go version in the script must match
the go version in the CI job, particularly when there are breaking
changes between go versions (as there are between 1.16 and 1.17
when it comes to modules).
To reduce the time to build the installer binary, let's change the terraform
Makefile so that it only builds the binaries when there are changes to the
respective go.mod. For example, the aws terraform provider will only be
re-built when there is a change to the go.mod file in the aws sub-module.
Remove the script for downloading the terraform binary. Since terraform
does not officially support some of the architectures that the installer
targets, the terraform binary needs to be built locally rather than
downloaded.
To reduce the size of the installer binary, zip each terraform provider
after building it. Terraform can use the zip directly without requiring
the installer to unzip the provider first.
Unpack the providers needed for completing a stage from the embedded
data in the installer's binary. This replaces the previous method of
creating symlinks to the installer binary, where the installer binary
masqueraded as each of the terraform providers.
This will download remote terraform providers individually. The remote terraform
providers are the providers that can be used as-is from a public registry.
This will also grab the terraform binary to embed in the installer from the
same terraform image used to download the terraform providers.
Use the latest version available for all of the remote providers.
This will build local terraform providers individually. The local terraform
providers are the providers that either (1) are not published in a public
regisrty or (2) require patches.
azureprivatedns: private to the installer repo
azurestack: requires patches
ibm: arm build is not published
ignition: arm buid is not published
ironic: not published
ovirt: not published
vsphere: requires patches
vsphereprivate: private to the installer repo
Building these has been added hack/build.sh. In the future a better way of
handling the vendoring in the individual directories will be necessary.
co-authored: staebler
The hack/verify-codegen.sh script was using an image that included
go 1.16. However, the updated k8s.io/json module calls the
`(reflect.StructField) IsExported` function, which is new in go 1.17.
Consequently, the script needs to be updated to use an image that
include go 1.17 rather than 1.16.
Currently, the installer relies on a generated go file for determining
the AWS region in which the RHCOS image is published. The `go generate`
directive was inadvertently removed in https://github.com/openshift/installer/pull/4582.
Rather than resurrecting the directive, this commit removes the generated
code in favor of gathering the regions directly from the rhcos stream
data when needed.
https://issues.redhat.com/browse/CORS-1838
if the go codegen commands failed, the the hack/verify-codegen.sh script
continued and the CI check passed. see #5406 for an example. the script
was not set to fail on a non-zero return code. this pr adds the bash e
option to the set builtin to catch errors and exit on failure.
Signed-off-by: Christy Norman <christy@linux.vnet.ibm.com>
It's more logically owned by the CoreOS team and this will
allow us to have a separate `OWNERS` file.
The `OWNERS` file is copied from the current one in openshift/os.
In https://github.com/openshift/enhancements/pull/679
we landed support for a stream metadata format already used
by FCOS and now by RHCOS/OCP consumers to find the bootimages.
We kept the old metadata because the UPI CI jobs used it.
In https://github.com/openshift/release/pull/17482 I tried
to port the UPI CI jobs, but ran into huge levels of technical debt.
As best I can tell, the UPI CI jobs are not running on this repo
now and are likely broken for other reasons.
Let's remove the old data and avoid the confusing duplication.
Anyone who goes to work on the UPI CI jobs and sanitizes things
should be able to pick up the work to port to stream metadata
relatively easily.
A recent commit passed the CI test for go mod tidy but introduced a bug
in master when we run go mod tidy. By including go mod tidy, we can
catch these earlier.
All installer binaries extracted from a payload, regardless of their
runtime OS or architecture, are built on the payload architecture.
Therefore, GOHOSTARCH can be used to assume the cluster architecture for
which its payload was built. This is set through the Dockerfiles so that
manual builds of installer will continue to default to amd64.
This unblocked the build on other architectures, but the resulting
ppc64le binary had issues. The golang bug which affected static linking
has been fixed with a one-line patch, which we are carrying downstream
until the upstream backports land.
This reverts commit 24ac0a15e7.
Marshal the coreos-bootimages ConfigMap included in the installer
manifests as yaml instead of json. This allows `oc` to replace
the `0.0.1-snapshot` value of the `releaseVersion`. The quotes
around the value when marshalled as json were preventing the
value from matching the regular expression that `oc` uses.
/fixes https://github.com/openshift/installer/issues/4797
Split out from https://github.com/openshift/installer/pull/4582
This copies the bits from https://github.com/cgwalters/rhel-coreos-bootimages
which builds a ConfigMap out of the stream metadata and injects
it into the cluster.
We have an `installer` image in the release image today; this adds
the "is an operator" label, even though it's not really an
operator. We just want the CVO to inject the manifest.
Among other important semantics, this will ensure that in-place
cluster upgrades that have new pinned CoreOS stream data will
have this configmap updated.
Briefly describe the history and future of the pinned {RHEL, Fedora} CoreOS
metadata in the installer.
Co-authored-by: Matthew Staebler <staebler@redhat.com>
This implements part of the plan from:
https://github.com/openshift/os/issues/477
When we originally added the pinned RHCOS metadata `rhcos.json`
to the installer, we also changed the coreos-assembler `meta.json`
format into an arbitrary new format in the name of some cleanups.
In retrospect, this was a big mistake because we now have two
formats.
Then Fedora CoreOS appeared and added streams JSON as a public API.
We decided to unify on streams metadata; there's now a published
Go library for it: https://github.com/coreos/stream-metadata-go
Among other benefits, it is a single file that supports multiple
architectures.
UPI installs should now use stream metadata, particularly
to find public cloud images. This is exposed via a new
`openshift-install coreos print-stream-json` command.
This is an important preparatory step for exposing this via
`oc` as well as having something in the cluster update to
it.
HOWEVER as a (really hopefully temporary) hack, we *duplicate*
the metadata so that IPI installs use the new stream format,
and UPI CI jobs can still use the old format (with different RHCOS versions).
We will port the UPI docs and CI jobs after this merges.
Co-authored-by: Matthew Staebler <staebler@redhat.com>
Rather than building an image to run mockgen and having that image
always get the latest verion of mockgen, use the mockgen from the
vendor directory. This has a couple benefits.
1) We do not get changes to the client mocks that pop up in random
PRs when a new version of mockgen is released.
2) Contributors that do not have access to the CI registry can
generate the mock clients.
The example URL is missing the /art/ path prefix, which makes it unclear
how to derive the external URL from the meta.json link in the ART build
browser.
With Go 1.14 the handling of modules has improved in the sense that all the subcommands `go {test, generate}` now use the vendor when available by default. This makes it easier for us to run generate using the vendored tools like controller-tools etc. as it now uses the checked in vendor.
