The rhsm feature was not being propagated from the CLI crate to the
lib crate, causing `bootc internals publish-rhsm-facts` to never be
compiled in even when building with CARGO_FEATURES=rhsm.
I think this was broken when I refactored the build recently.
Change things so we build the manpages before the production
binary, ensuring the production binary always ends up with
the right feature flags.
Fixes: https://issues.redhat.com/browse/RHEL-130799
Assisted-by: Claude Code (Sonnet 4.5)
Signed-off-by: Colin Walters <walters@verbum.org>
Now that we've cut over to always building + installing via
an (RPM) package in our build system, we need to always install
the dracut module.
Signed-off-by: Colin Walters <walters@verbum.org>
- The ci/Dockerfile.fcos has been unused for some time and
testing with FCOS derivatives is already done outside of CI
here
- Drop the bits which reference git in the Makefile, a new rule
I'm trying to impose here is that nothing in the build system
should rely on git (or at least not rely on being built
from a git repository)
Signed-off-by: Colin Walters <walters@verbum.org>
When commit cd8fa591 moved unit enablement to a systemd generator, it also
removed the installation of the fedora-bootc-destructive-cleanup script from
the Makefile. This was unintentional - while the symlink enablement should
be handled by the generator, the script itself still needs to be installed.
The trap is that because we weren't accurately removing old files
from the build, the old version was still there...until the new upstream
release.
Fix this in two ways:
First, continue to install the script on Fedora-like systems as before, but leave unit
enablement to the generator as intended.
Second, change how we remove prior state to clean out all files from
the RPM. (I did look at changing out the build system so we build
an RPM in this case which would be cleaner, but it's a large change)
Fixes: https://github.com/bootc-dev/bootc/issues/1748
Assisted-by: Claude Code (Sonnet 4.5)
Signed-off-by: Colin Walters <walters@verbum.org>
Right now this service fails in `bcvk run-ephemeral`, but
also likely fails in any non-bootc system that has `subscription-manager`
installed.
A problem is that dependencies of units are started even
if the dependee has a condition that disables it.
This basically the target and path depend on `/run/ostree-booted`
being present (which yes, won't work for composefs...)
Tests: Covered by extant `012-test-unit-status.nu`
Signed-off-by: Colin Walters <walters@verbum.org>
The emphasis here is on trying to have
the `Justfile` be the default entrypoint,
wrapping other tools.
- Replace mentions of podman-bootc with bcvk
since I hope the latter supercedes the former
- Unify the unit test entrypoint
- Set up /var/tmp as a tmpdir to fix the etc merge
test (otherwise, selinux failures w/tmp)
- Run the unit+container tests in integration.yml
- Have `just validate` run in a container
Signed-off-by: Colin Walters <walters@verbum.org>
First change `.dockerignore` to be an allowlist.
This avoids spurious rebuilds when touching e.g. `tmt/`, and
also crucially we don't leak `.git/` into the sources which
can easily change as one makes commits/branches.
This also requires touching the `hack/` directory which
is now fully self contained.
While we're here, make clear the roles of Justfile vs Makefile.
Remove the `make test`.
Signed-off-by: Colin Walters <walters@verbum.org>
- Remove duplicated logic between xtask and makefile
for converting markdown; it needs to be in xtask
as we handle the version substitution there and
some other tweaks
- Really just make the developer entrypoint `just update-generated`
in general
- Fix the rendering of booleans
- Remove unnecessary emoji from prints
Signed-off-by: Colin Walters <walters@verbum.org>
A key thing for me is that the `Justfile` should be a one-stop
shop for development of the project. It can't have everything but
it should answer the basic questions of "how do I build and test
this project".
This aligns the recently added tmt-on-GHA flow a *bit* more closely
with some of that. Biggest is to use the `just build-integration-test-image` as the canonical
way to build a container image with our testing stuff in it;
which uses our main Dockerfile
Other cleanups:
- Change test script to move into tests/tmt/ as a workaround for
https://github.com/teemtee/tmt/pull/3037#issuecomment-3259585271
- Change the qemu logic to use SMBIOS credentials so we don't
have to carry around both a disk image and a SSH key
- Change qemu to use `-snapshot` so we can reuse disks
- Change the scripts to accept data via argv[1] and not environment
- Drop the hardcoded testing directory and use `target/` as
a generic build artifact dir
Signed-off-by: Colin Walters <walters@verbum.org>
I was hitting a compliation error which *looks*
like it was actaully an incremental compilation bug?
Or it might have been rust-analyzer and local builds
fighting over enabled features.
Anyways, this ensures that we're gating on the composefs
backend compiling.
Signed-off-by: Colin Walters <walters@verbum.org>
See the updates to `Justfile` for how to use this.
Closes: #1428
Assisted-By: Claude Code (opus + sonnet)
Signed-off-by: Colin Walters <walters@verbum.org>
The Dockerfile right now doesn't build an RPM, so this
ensures the two build systems match. We should probably
also change the RPM to use this Makefile.
Motivated by accidentally merging a change that breaks
with the feature on.
Signed-off-by: Colin Walters <walters@verbum.org>
This adds scaffolding to install a stub binary which can
optionally be added into the initramfs;
prep for us doing real work during setup as we aim
to move to the native composefs backend.
The binary is *built* but is only installed by a
new `Makefile` target, so existing build system
users won't pick it up. Our development-only
`Dockerfile` gains a build option to use it
(and also ensures the initramfs is regenerated).
However previously we also discussed moving the fstab
logic into the initramfs:
https://github.com/bootc-dev/bootc/pull/1113
I might try doing that once this lands.
One notable thing is that even this trivial nearly-no-op
binary is still 4MB which I think is mostly due
to linking in a whole copy of prebuilt rust `std`.
In theory we could try going to `#[no_std]` but I
don't think it'll be viable once we start doing more here.
Probably most practical thing re size is `-Z build-std` + LTO.
Signed-off-by: Colin Walters <walters@verbum.org>
This is aligning with what I did in https://github.com/ostreedev/ostree/pull/3439
- What gets invoked in e.g. GHA should ideally most be `just` commands
that are easy to run locally too (with sudo in GHA, without sudo locally)
- Move the "core build" to the toplevel so that one can just `podman build`
directly too (without the Justfile) and have it do something useful
- The "always build and test in a container" helps for LLM-assisted coding
because what they can do is inherently sandboxed
Signed-off-by: Colin Walters <walters@verbum.org>
In some cases rust-analyzer seems to be building without the `bootc`
feature which triggers some unused variable warnings here.
Signed-off-by: Colin Walters <walters@verbum.org>
When set, the bootc-destructive-cleanup flag is added to /sysroot/etc
which enables the bootc-destructive-cleanup systemd service to remove
the previous installation's rpm packages and podman containers/images.
The service is only installed on fedora based systems.
Signed-off-by: ckyrouac <ckyrouac@redhat.com>
This took me some thinking and experimenting. Basically we want:
- Hard deny some warnings (this is covered by the Cargo.toml
workspace.lints.rust)
- Gate merging to main in CI on an exact set of warnings we
want to forbid, but *without* also using a blanket
-Dwarning deny policy because that could break our build
when the compiler revs.
- A corollary to the previous: allow developing locally
without killing the build just because
you have an unused import or some dead code (for example).
So we don't want to add `dead_code = deny` into the Cargo.toml.
- Be able to easily reproduce locally what CI is gating on
in an efficient way.
We already had `make validate-rust` which was intending to navigate
this, but what was missing was the "deny extended set of warnings"
so we got code committed to git main which hit `unused_imports`.
Clippy upstream docs recommend the `RUSTFLAGS = -Dwarnings`
approach in e.g.
https://doc.rust-lang.org/clippy/continuous_integration/github_actions.html
but again I think this is a problem because it can break with
updated Rust/clippy versions (unless you pin on those, but that
becomes a pain in and of itself).
The problem also with doing `RUSTFLAGS = -Dwarnings` *locally*
is it blows out the cargo cache.
So here's the solution I came to: We run `cargo clippy -A clippy:all`,
and then deny some specific clippy lints *and* the core Rust
warnings we want (`unused_imports` type things) at this stage.
The advantage is this doesn't blow out the main Cargo cache,
and I can easily reproduce locally exactly what CI would gate on.
Also while we're here, add `make fix-rust` which is a handy
way to use the existing `clippy --fix` to locally fix things
like unused imports as well as other machine-applicable
things that are in e.g. `clippy::suspicious`.
Signed-off-by: Colin Walters <walters@verbum.org>
# Background
The current usage instructions for bootc involve a long podman
invocation.
# Issue
It's hard to remember and type the long podman invocation, making the
usage of bootc difficult for users.
See https://issues.redhat.com/browse/BIFROST-610 and https://issues.redhat.com/browse/BIFROST-611
(Epic https://issues.redhat.com/browse/BIFROST-594)
# Solution
We want to make the usage of bootc easier by providing a new Fedora/RHEL
subpackage that includes a new binary `system-reinstall-bootc`. This binary
will simplify the usage of bootc by providing a simple command line
interface (configured either through CLI flags or a configuration file)
with an interactive prompt that allows users to reinstall the current
system using bootc.
The commandline will handle helping the user choose SSH keys / users,
warn the user about the destructive nature of the operation, and
eventually report issues they might run into in the various clouds (e.g.
missing cloud agent on the target image)
# Implementation
Added new system-reinstall-bootc crate that outputs the new
system-reinstall-bootc binary. This new crate depends on the existing utils crate.
Refactored the tracing initialization from the bootc binary into the
utils crate so that it can be reused by the new crate.
The new CLI can either be configured through commandline flags or
through a configuration file in a path set by the environment variable
`BOOTC_REINSTALL_CONFIG`.
The configuration file is a YAML file.
# Limitations
Only root SSH keys are supported. The multi user selection TUI is
implemented, but if you choose anything other than root you will get an
error.
# TODO
Missing docs, missing functionality. Everything is in alpha stage. User
choice / SSH keys / prompt disabling should also eventually be supported
to be configured through commandline arguments or the configuration
file.
Signed-off-by: Omer Tuchfeld <omer@tuchfeld.dev>
rpm can't handle hardlinked binaries with distinct labels today
gracefully; ostree (and bootc) always break the hardlink which
I think is generally right.
Anyways, switch to a dispatcher shell script to fix this.
To fix https://issues.redhat.com/browse/RHEL-76188
Signed-off-by: Colin Walters <walters@verbum.org>
The bootc-status-updated.target unit cannot directly be
WantedBy=multi-user.target, because we rely on StopWhenUnneeded=true
so the target will be activated each time bootc-status-updated.path
triggers it. Because multi-user.target remains active, this would
hold bootc-status-updated.target in an active state as well, and the
path unit activating would not have the desired effect of retriggering
the target unit.
The new bootc-status-updated-onboot.target is intended to serve the
same purpose as bootc-status-updated.target but will activate (and
remain active) on boot. Any interested units should be WantedBy both
targets. This updates bootc-publish-rhsm-facts.service to do just
that.
Signed-off-by: John Eckersberg <jeckersb@redhat.com>
Any services that react to status changes should be idempotent, and
should run on boot. It is likely (but not guaranteed) that during
boot we are in the process of switching from a previous deployment to
a new one, and thus what was previously "staged" is now "booted" and
what was previously "booted" is now "rollback", so anything that cares
about status is going to want to handle that.
Signed-off-by: John Eckersberg <jeckersb@redhat.com>
Added a kernel-install configuration that enables
the rpm-ostree integration.
The second configuration file sets the Journal to persistent
as on bootc systems /var is empty by default and systemd
by default enables persistent journaling on the presence of
/var/log/journal.
Signed-off-by: Joseph Marrero Corchado <jmarrero@redhat.com>
Followup to https://github.com/containers/bootc/pull/977
In chatting with John we had some agreement to statically
enable the `.path` unit by default upstream here since
it's a feature we want to be available by default
so that tooling can react dynamically to changes.
Signed-off-by: Colin Walters <walters@verbum.org>
ref https://gitlab.com/fedora/bootc/tracker/-/issues/32#note_2270189835
This will keep track of the generic bootc+dracut integration glue
here instead of replicating it in different base images.
(Perhaps bootc actually injects code into the initramfs in the
future instead of ostree, for example)
Signed-off-by: Colin Walters <walters@verbum.org>
This directory will contain expected files in the base image.
That said, I may change the container import path to auto-create
at least the sysroot dir and symlink at some point and these
can just be dropped.
And for that matter after
9a0acd7249
"libostree/deploy: enable composefs by default"
we can likely just drop the prepare-root bit too.
But for now this is needed.
Motivated by improving base image generation from
https://gitlab.com/fedora/bootc/tracker/-/issues/32
Signed-off-by: Colin Walters <walters@verbum.org>
This adds a new systemd path unit which activates on bootc status
changing, and in turn triggers a new systemd target. This allows
adding arbitrary new systemd services with `WantedBy =
bootc-status-updated.target` that will be activated each time the
bootc status is updated.
Signed-off-by: John Eckersberg <jeckersb@redhat.com>
This helps ensure our CI is covering our copy of ostree-ext, not
the one currently vendored in rpm-ostree.
Signed-off-by: Colin Walters <walters@verbum.org>
Add `dead_code = "deny"` to our default lints; we had
a compiler warning for this in main.
Fix the warning by moving the human readable test code into
`#[cfg(test)]`.
While we're here, move the other lib.rs lints into the crate;
enforcing docs for *everything* at first I thought might be heavy
handed but actually is fine as it only applies to things that
are `pub`, of which we don't actually have that much so it
mainly forced me to add some stub docs for the modules, which
is probably a good idea.
Signed-off-by: Colin Walters <walters@verbum.org>
Closes: https://github.com/containers/bootc/issues/721
- Initialize a containers-storage: instance at install time
(that defaults to empty)
- Open it at the same time we open the ostree repo/sysroot
- Change bound images to use this
We are *NOT* yet changing the base bootc image pull to use this.
That's an obvious next step (xref
https://github.com/containers/bootc/pull/215 ) but will come later.
Signed-off-by: Colin Walters <walters@verbum.org>
I plan to add more generated-from-source files, so generalize
the target which is currently specialized to manpages.
Signed-off-by: Colin Walters <walters@verbum.org>
Let's ensure these directories exist by default, because
we want people to drop things there - and having to create
the directory in derived images invites typos.
Signed-off-by: Colin Walters <walters@verbum.org>
The `make validate-rust` reproduces locally the same checks
we run in CI as gating by default; hook it up to the existing
`make validate`.
Signed-off-by: Colin Walters <walters@verbum.org>
I've been trying to keep this project in "one" programming
language by writing even tests in Rust...but specifically
for our integration tests it's pretty painful not just to
compile them but have to deal with baking them into the base image.
The tmt framework is very GHA like in that it scrapes the
git source tree and copies it into the target environment, which
works really well with scripts.
Now, if you know me you know I am not a fan of dynamic programming
languages like bash and Python. I'm one of those folks that actually
tries to use Rust for things that feel like "scripts" i.e. they're
*mostly* about forking external processes (see the xtask/
crate which uses "xshell").
Some of our testing code is in Rust too. However...there's a giant
tension here because:
- Iteration speed is very important for tests and scripts
- The artifact being an architecture-dependent binary pushes us
to inject it into container images; having the binary part
of the bootc image under test conceptually forces us to reprovision
for each test change, which is super expensive
Most other people when faced with the testing challenge would
just write shell scripts (or Python); that's definitely what tmt
expects people to do.
The podman project has a mix of a "bats" suite which is all
bash based, and a Go-based framework.
The thing is: bash is easy to mess up and has very little ability
to do static analysis. Go (and Python) are very verbose for forking external
processes.
I've been using https://www.nushell.sh/ for my interactive shell
for quite a while; I know just enough to get by day to day
(but honestly sometimes I still type "bash" and run a few things there
that I know how to express in bash but not nu)
Anyways though, nushell has a lot of desirable properties for
tests (which are basically scripts):
- Architecture independent
- Running an external process requires zero ceremony; it's the
default!
- But it *is* easy to e.g. scrape JSON from an external binary
into a rich data structure
- A decently rich standard library
The downside is, it's a new language. And in the end, I'm
not going to say it's the only way to write tests...maybe we
do end up with some more bash. It wouldn't be the end of the world.
But...after playing with this, I definitely like the result.
OK, and after some debate we decided to add Python too, so this
demos a pytest test.
Signed-off-by: Colin Walters <walters@verbum.org>
Part of https://github.com/containers/bootc/issues/543
I'm not totally happy with this, but it does demonstrate the basic
wiring flow of:
- `cargo xtask test-tmt`
That will do a container build, make a disk image from it,
and run a "hello world" tmt test.
A lot more to do here including wiring up our existing tests
into this, and deduplicating with the other integration tests.
A key aspect too will be exploring workflows that e.g. expose
a registry locally.
Signed-off-by: Colin Walters <walters@verbum.org>
This ensures that when we rerun the container build without
changing the source, we get the same thing. Which then
further ensures we don't invalidate any cached disk images,
etc.
Signed-off-by: Colin Walters <walters@verbum.org>
Previous work started moving our tests into an external binary;
this is just cleaner because it can test things how a user
would test. Also, we started using `libtest-mimic` to
have a "real" test scaffolding that e.g. allows selecting
individual tests to run, etc.
Complete the picture here by moving the remaining bits into
the tests-integration binary. We now run the `tests-integration`
binary in two ways in e.g. Github Actions:
- It's compiled directly on the Ubuntu runner, and orchestrates
things itself
- It's built in our default container image (Fedora) but as an external
`/usr/bin/bootc-integration-tests` binary
Also while we're here, drop the kola tests.
Signed-off-by: Colin Walters <walters@verbum.org>
