get_treefile() crashes when parsing versions with fewer than 3
components (e.g., "44"). Now we pad the version parts list with
"0" to ensure we always have at least 3 elements.
As a follow-up of e6d3ed2.
Basically we need to support setting it to:
- fedora-43
- centos-9
- rhel-9.8
So we need to include the minor in one case but not hte other two.
Let's add some conditional logic here for that.
I found issues where this was conflicting at times with the built-in
function id in module builtins.
Let's just rename to decrease chance of conflicts.
Now you can source in the build-args.conf and print the treefile
to stdout.
```
[coreos-assembler]$ source src/config/build-args.conf
bash: CoreOS: command not found
[coreos-assembler]$ export ID NAME STREAM VERSION MUTATE_OS_RELEASE MANIFEST PASSWD_GROUP_DIR IMAGE_CONFIG
[coreos-assembler]$ src/config/build-rootfs --srcdir src/config/ parse-treefile
{"packages": ["/usr/bin/kdumpctl", "/usr/share/makedumpfile", "NetworkManager-cloud-setup",
```
This removes SRCDIR from being a global/constant into a --srcdir
parameter to allow for execution of build-rootfs outside of
the Containerfile where things have been copied into /src/.
The default is still set to /src/ if the --srcdir paramter isn't
provided.
Assisted-By: <google/gemini-3-pro-preview>
This adds a `build-rootfs make-rootfs` subcommand that has the
same behavior as the original intention of the script. This is
prep for adding more subcommands.
At this point it just pulls the first field off the version string
which should be good enough. Only Fedora uses $releasever in yum repos
and the major version number works there.
These bits of information are used in various places. I thought
previously that `BASE_VERSION` would suffice and we'd be able to
use that fore mutating the os-release file, but it turns out it's not
a 1:1 relationship there. For example:
For Fedora 43 the BASE_VERSION is 43 and the string we need to find
and replace for VERSION and PRETTY_NAME in os-release is 43. For CentOS
Stream 9 the string we need to find/replace is `9`, but for RHEL 9.8
it's `9.8`. But this is complicated by the fact that we want our version
for c9s and c10s to be X.0 (i.e. 9.0, 10.0).
Here we drop BASE_VERSION (we can just use `VERSION` for what we need
in versionary) and introduce MUTATE_OS_RELEASE just for the string
to find and replace with the actual version.
These are all defined in env vars so let's just pick them up from
there rather than trying to explicitly add them to the CLI each time
we add a new argument.
When using subprocess in Python, spaces in the queryformat string can be
misinterpreted due to shell argument parsing. Switch to comma-separated
format without spaces to avoid parsing issues.
Assisted-by: Claude (Opus 4)
Signed-off-by: Joel Capitao <jcapitao@redhat.com>
..and use it for variable replacement in the image.yaml files rather
than the manifest since stream is the only var we are formatting/replacing
right now.
This further removes our dependence on the manifest.
In hermetic builds there is no access to the network. Detect this by
looking for the `cachi2.repo` that is injected by konflux during the
build.
In this case we make sure to not use any of our own repo and rely on
the repo created by hermeto.
This will make it easier to support downstream rhel-coreos-config
where we support building more than one variant from the main branch
and the image.yaml files are named per variant.
When verifying that the RPMs we locked on are what is in the built
container we ran into a corner case where if a RPM has explicitly
set their EPOCH to 0 then the NEVR from `rpm -q --qf` will include
that `0:`.
`rpm-ostree` just assumes any `0` EPOCH shouldn't be included
in the string [1] so it won't get written to the lockfile.
We need to account for this here.
We stumbled upon this by accident because perl was getting
included [2] when it shouldn't and it have subpackages that
set an EPOCH of `0`.
[1] 0ad2ee53f3/src/libpriv/rpmostree-rpm-util.cxx (L76-L84)
[2] https://github.com/coreos/fedora-coreos-tracker/issues/2059
Add the link target to the hash instead.
This broke the ostree builds with
`[2025-11-11T03:36:28.572Z] FileNotFoundError: [Errno 2] No such file or directory: '/src/overrides/rootfs/etc/grub.d/15_ostree'`
Signed-off-by: Colin Walters <walters@verbum.org>
This builds on the upstream changes [1][2] to support being able to build
the rootfs without generating the initramfs, which allows us to hook
in our own post-process scripts before then generating the initramfs
ourselves in build-rootfs.
The effect of this change is that our postprocess scripts can directly
impact the generated initramfs. Without this change we either needed
to implement hacks (like inject_dracut_version() and cleanup_dracut_version(),
which are deleted in this commit) OR run dracut twice (throwing away
the initial run as part of the `rpm-ostree compose rootfs`).
[1] 481fbb0342
[2] https://gitlab.com/fedora/bootc/base-images/-/merge_requests/320
This was being done at the bootc level but got changed in
https://gitlab.com/fedora/bootc/base-images/-/merge_requests/310
so we need to do it here again.
Do it in a postprocess because that's the way we can do it in a
container build (i.e. where `remove-from-packages:` isn't supported).
We also unfortunately need a SUPER HACK rip this out of the initramfs
build when dracut runs. Unfortunately in the build-with-buildah path
today our postprocess scripts here are only run *after* dracut has
run and so they won't affect the initramfs. I'm hoping this is
very temporary because we really need a better answer for this.
Right now, there is no change detection in our builds. We just always do
a new build, which is wasteful.
Let's re-implement a concept similar to rpm-ostree's inputhash, which
just hashes all the relevant inputs. We then add a label to the final
image with that input. On the cosa side, we can compare the hash to the
previous build to know if to no-op.
One major difference from rpm-ostree's inputhash is that it could know
the hash right after doing the depsolve (to have the final list of RPMs)
and thus avoid a full build. It's possible to do this here too though
it'd require bootc-base-imagectl and rpm-ostree changes. We'd need to
also define an API for having a `podman build` actually signal "no-op";
e.g. writing a file in the build context and erroring the build...
awkward.
Much more interesting and related to this is reproducible builds;
if our builds were reproducible, we wouldn't have to worry about
this because we'd just build the exact same artifact everytime.
There's some work required to get there though, and likely we'd
have to rework how we calculate our versions, since that's a dynamic
value which affects the rootfs and OCI labels (e.g. always have
the same version for the set of inputs; see also discussions in
https://github.com/coreos/fedora-coreos-tracker/issues/2015).
In the legacy path, rpm-ostree handles `mutate-os-release` before
calling dracut. So the initrd-release in the initramfs (built from
os-release) contains the updated values.
In the buildah path, we handle version mutation after the rpm-ostree
compose done by bootc-base-imagectl and so the initrd doesn't have the
updated values.
I find that versioning information in the initrd useful because systemd
will print it out on the console (the "Welcome to ..." string), and a
lot of our bug reports are from failures in the initramfs.
So this patch restores this information for the buildah path, but does
it in a subtle way to avoid having to regenerate the initramfs just
for that: we create a temporary dracut module and then delete it in
postprocessing.
This re-implements rpm-ostree's `--lockfile-strict`, in which we want to
ensure that only locked packages were installed.
We can't do this the same way as rpm-ostree did, i.e. as excludes
pre-compose, but we can trivially check the result in post.
On prod streams, we _only_ have the lockfile repos. This made obvious
a slight logic bug here. We want to inject our repo files if there's
either a non-empty `repos` key or a non-empty `lockfile-repos` key.
I like the efficiency of being able to just mount the build context
instead of having to copy it in. For example, I often have git worktrees
in my git checkout and I don't want `COPY` to dumbly copy those too
(though I then realized we could just use a `.containerignore` for it).
However, a down side of mounting is that we render layer caching even
more useless because it can't know which files from the mount we're
actually using.
More recently, an issue around setgid leaking into
the context dir (because OpenShift) came up (see
https://github.com/coreos/coreos-assembler/pull/4277) and it's just not
acceptable to chmod the real build context directory.
Of course, `build-rootfs` could e.g. copy the overlay dirs into a
temporary location and then adjust the perms there, but at that point,
we're quite close to just doing a `COPY` upfront anyway.
So let's just switch to that, strip the bits as we want, but include a
`.containerignore` to only copy what we need.
This then limits the build context mounting to just the rechunk and
`FROM oci-archive:` hack. Which... hopefully one day we could normalize
that.
I mistakenly removed the pool repo hack in 534fa61e ("build-rootfs:
rework lockfile handling") thinking it was no longer needed, but it is.
While we're here, make lockfile-repos handling more correct by only
enabling them if we have locked packages and if we know how to add
excludes to the repo so only those are available through there.
A more generic solution to this that wouldn't require repo file fiddling
would have been to set that includepkgs in `/etc/dnf/dnf.conf` directly
since you can change settings for specific repos there, but rpm-ostree
disables that config file in its submoduled libdnf.
We weren't quite matching the cosa semantics of lockfiles, and this
was causing us trouble. Notably, doing `dnf install -y` for all locked
packages doesn't work for autolocked packages because those lockfiles
may contain packages that don't exist at all on other arches.
It also meant that e.g. even if removing a package from the manifest,
you'd still have it installed if you didn't also remove it from the
lockfiles.
Really, we need the `packages` list to be canonical and the lockfiles to
only act as filters.
There is a new `--lock` switch now in bootc-base-imagectl that we can
use for that (which underneath just uses rpm-ostree lockfiles, like we
do now). So let's use that.
This also allows us to drop the hack around modifying the pool repo by
adding `includepkgs` manually. As well as the caveat around local RPM
overrides handling.
We don't need the 'packages' intermediate key. I think I initially did
that so it follows the same schema as regular lockfiles, but anyway we
desugar them to the `{pkgname: evra}` format pretty quickly.
Similarly to the previous commit, this supports the longstanding cosa
API of having an `overrides/rootfs` dir with bits that get overlaid on
top of the final rootfs.
Except here the dir is in the build context directory. cosa can use this
API to wire its `overrides/rootfs` to keep the same API, but `podman
build` users can also use this directly.
This is almost equivalent to having a
`COPY overrides/rootfs/ /target-rootfs` instruction after the
`build-rootfs` but before rechunking.
The nice thing about folding it into build-rootfs is that it can more
cleanly be made conditional than a Containerfile directive, and we can
support the override being fed as a bind-mount (as cosa will use it)
instead of forcing it to be in the context dir.
Allow a yum repo at `overrides/rpm` to exist in the context dir, in
which case we automatically create a lockfile and repo file that we
inject out of that.
This matches the `overrides/rpm` functionality in cosa, but the
API lives here. cosa can then just use that API to retain its
`overrides/rpm` behaviour.
A lot of the code is lifted from cosa's `cmdlib.sh`, which conveniently
had it in Python already among its many liberal uses of inline Python
scripts.
So far, we've been using the default repos in the bootc base
image. But e.g. those repo files use mirrors and so are subject to
lag/inconsistencies across runs.
In the end, we want our repo files to be canonical.
Do this by nuking any default repo file and injecting our own. Also
use the new `bootc-base-imagectl build-rootfs --repo` switch to control
enablement.
Some of the postprocess scripts need to be able to affect the top-level
rootfs (e.g. make `/opt` back into a symlink). So we need to mount the
full rootfs.
