Previously, the check to verify install to-filesystem is being run
within a container happened fairly late in prepare_install. This moves
the check up before some of the other container specific checks. Now,
the error should occur when trying a basic install to-filesystem
invocation, e.g. `bootc install to-filesystem /var/mnt`.
This also adds a test to verify the error occurs with minimal arguments
and adds host_is_container to the global state.
Signed-off-by: Chris Kyrouac <ckyrouac@redhat.com>
Along with the test, this will install the LBIs on the machine running
the tests prior to running the tests. Also refactors the existing LBI
test into a separate test plan to make room for the install tests.
Signed-off-by: Chris Kyrouac <ckyrouac@redhat.com>
These tests validate the basic use case of switching and upgrading
an image with bound images that need to be pulled.
Signed-off-by: Chris Kyrouac <ckyrouac@redhat.com>
Because tmt right now doesn't support isolation automatically,
and many of the tests we'll want to write mutate the system,
we'll need to copy-paste the base plan.
This changes the `make test-tmt` to have its own little
tmt wrapper that dynamically scans the plans/ and sorts
them by a priority, then runs them one by one currently.
Closes: https://github.com/containers/bootc/issues/735
Signed-off-by: Colin Walters <walters@verbum.org>
1. rename old integration test to e2e tet
2. drop aws test, use libvirt test (for bootc install to-existing-root
and bootc switch test) instead
3. drop rhel test
4. replace quay.io registry by local registry
Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>
I've been trying to keep this project in "one" programming
language by writing even tests in Rust...but specifically
for our integration tests it's pretty painful not just to
compile them but have to deal with baking them into the base image.
The tmt framework is very GHA like in that it scrapes the
git source tree and copies it into the target environment, which
works really well with scripts.
Now, if you know me you know I am not a fan of dynamic programming
languages like bash and Python. I'm one of those folks that actually
tries to use Rust for things that feel like "scripts" i.e. they're
*mostly* about forking external processes (see the xtask/
crate which uses "xshell").
Some of our testing code is in Rust too. However...there's a giant
tension here because:
- Iteration speed is very important for tests and scripts
- The artifact being an architecture-dependent binary pushes us
to inject it into container images; having the binary part
of the bootc image under test conceptually forces us to reprovision
for each test change, which is super expensive
Most other people when faced with the testing challenge would
just write shell scripts (or Python); that's definitely what tmt
expects people to do.
The podman project has a mix of a "bats" suite which is all
bash based, and a Go-based framework.
The thing is: bash is easy to mess up and has very little ability
to do static analysis. Go (and Python) are very verbose for forking external
processes.
I've been using https://www.nushell.sh/ for my interactive shell
for quite a while; I know just enough to get by day to day
(but honestly sometimes I still type "bash" and run a few things there
that I know how to express in bash but not nu)
Anyways though, nushell has a lot of desirable properties for
tests (which are basically scripts):
- Architecture independent
- Running an external process requires zero ceremony; it's the
default!
- But it *is* easy to e.g. scrape JSON from an external binary
into a rich data structure
- A decently rich standard library
The downside is, it's a new language. And in the end, I'm
not going to say it's the only way to write tests...maybe we
do end up with some more bash. It wouldn't be the end of the world.
But...after playing with this, I definitely like the result.
OK, and after some debate we decided to add Python too, so this
demos a pytest test.
Signed-off-by: Colin Walters <walters@verbum.org>
Sometimes the deployed testing farm runner does not have big disk,
to avoid this issue, use smaller tmt vm disk instead
Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>
build.fmf: used to deploy testing farm runner and run
tmt integration on it
build-tmt.fmf: tmt test to work with discover.fmf which is
requited to make .git always present
Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>
Part of https://github.com/containers/bootc/issues/543
I'm not totally happy with this, but it does demonstrate the basic
wiring flow of:
- `cargo xtask test-tmt`
That will do a container build, make a disk image from it,
and run a "hello world" tmt test.
A lot more to do here including wiring up our existing tests
into this, and deduplicating with the other integration tests.
A key aspect too will be exploring workflows that e.g. expose
a registry locally.
Signed-off-by: Colin Walters <walters@verbum.org>
1. add bootc swtich test
2. add bootc install to-disk test
3. cover more distros, like rhel 9.5, fedora 40 and 41(rawhide)
Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>
To avoid galaxy server unstable issue, like 504 gateway timeout,
error when getting available versions of collection community.general,
download two required collection packages and install then locally
Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>
