Change the install data format to keep a list of containers for each
image installed, so that multiple containers for the same image can be
installed and uninstalled.
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1559935
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1214
Approved by: baude
do not reload the list of all the images every time we check if an image
is dangling. This makes "images list" extremely slow when used on the
Docker backend.
On my system (with ~100 images), this patch brings down the wall clock
from ~10 seconds to ~0.5 seconds:
$ /usr/bin/time -v ./atomic images list -a 2>&1 | grep "wall clock"
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.84
to:
$ /usr/bin/time -v ./atomic images list -a 2>&1 | grep "wall clock"
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.58
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1200
Approved by: rhatdan
use the implementation of validate_image_manifest and
adapt it to be used in place of validate_layer.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1150
Approved by: rhatdan
Fix test failures after updating papr to test with f26 atomic/cloud
images instead of f25, with the following changes:
1. Remove dependency on docker hub tester image. Instead, mimic
what ostree/rpm-ostree does and use a recursive .papr.sh script
to install the necessary packages to the base f26 image in the
fedora registry. This fixes tests on the atomic host since python3.6
is being used, and prevents future tests from testing the wrong
version. (Note this is slightly slower due to having to install
packages during the test rather than using a pre-built image).
2. Fix some pylint errors, and mask others for now
3. Fix failing integration tests due to inter-test interference
4. Remove unnecessary deepcopy in container filter
5. Add compatibility for both c-s-s and d-s-s in storage
6. Update expected sha256 values for dockertar test
Remaining issues:
1. test_storage should possibly be reworked. The current test
setup is conflicting with the new default of overlay as a driver.
For now, the test for generated d-s-s is disabled.
2. some storage commands are still using "docker-storage-setup"
instead of "container-storage-setup". There is a backward
compatible check in place that should be reworked in the future
3. some masked pylint errors should be further investigated
4. keep the dockerfile for the project atomic tester image for now
(bump to 26), since its a little easier and faster to set up with
Signed-off-by: Yu Qi Zhang <jerzhang@redhat.com>
Closes: #1076
Approved by: baude
When wanting to run a command in a already running container,
a NoneType error was being thrown when the image lookup
failed.
This was reported in BZ #1481967
Closes: #1069
Approved by: rhatdan
A bug was reported in issue #1022 where if the atomic install data failed,
the installation data was still written allowing the user to proceed
with an atomic run.
Disabling test_install.sh until i can follow up with Thomas.
Closes: #1025
Approved by: baude
As pointed out in issue #1023, when you run an interactive
container and then subsequently run it with commands, you can
get a case where the container is running. If you run another
image that will end up using the same container name but is born
of a different image, you get confusion.
In this case, we adopt the --replace function as we did with earlier
run/image mix-ups.
Closes: #1026
Approved by: baude
There were two primary cases where a secondary atomic run with a command
would trigger an exception. The first was reported in
https://github.com/projectatomic/atomic/issues/1006. Basically it can
be summarized as:
```
atomic run registry.fedoraproject.org/fedora:25 date # works fine
atomic run registry.fedoraproject.org/fedora:26 date # tries to run in the existing f25 container
```
The second case is as simple as:
```
atomic run registry.fedoraproject.org/fedora:25 date # works fine
atomic run registry.fedoraproject.org/fedora:25 date # fails
```
This fails because atomic starts the stopped f25 container and then attempts a docker exec. The
exec fails because the 'date' command is short-lived and the container exits prior to the exec
being run.
We now catch those exceptions and notify the user. We added a `--replace` option to run where
atomic will now delete the container in question and re-run it from the correct image.
Closes: #1019
Approved by: baude
The atomic tag function was not working correctly for docker images
in the dockerd. Also, when attempting to tag an image from one
backend into another, we didn't handle the error correctly. This should
not work.
Add integration tests for tagging invalid images, dockerd, and ostree.
This should fix BZ #1454656.
Closes: #1014
Approved by: baude
If a shortname is used to run an image, we need to transform the short
name into the fq-name when doing the lookup in the installed images
data.
Reported in BZ #1454292
Closes: #1010
Approved by: baude
Bugzilla #1430708 recommends that if an atomic user attempts to pull
an image that is already present, we should not exit with a '1' which
indicates a failure; rather a 0.
Closes: #997
Approved by: baude
As requested in BZ #1447848, if a ostree mount is desired, we should
not block on requiring an active dockerd. Same for unmount.
Closes: #998
Approved by: baude
For some reason when shorthand is used to add a list and str/unicode,
a type Error is not thrown where shorthand is cmd += foo versus
cmd = cmd + foo. Reverting to the long hand appears to resolve issue
https://github.com/projectatomic/atomic/issues/986
Closes: #987
Approved by: baude
We need to make sure the atomic_dbus_client code works and if something
in atomic_dbus.py changes, the client should either be updated or still
work.
Closes: #975
Approved by: rhatdan
and add the implementation for the Docker and the OSTree backends.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #958
Approved by: rhatdan
Bugzilla 1437447 has brought up an issue where if a user runs:
atomic pull rhel7
The resulting image in dockerd is docker.io/rhel7. The Atomic CLI
does tell skopeo the right information"
/usr/bin/skopeo --debug --policy=/etc/containers/policy.json --debug copy --remove-signatures docker://registry.access.redhat.com/rhel7:latest docker-daemon:rhel7:latest
But somewhere along the line, we think in skopeo, docker.io is prepending to the destination
image name. One way to resolve this is to always use the fq name for the destination and not
what the user wanted.
This is a change in the default behaviour of atomic and I am not sure I am confortable with this. But
given that we have several folks on PTO or travelling, I'm putting this PR together so that
if we decide this is the proper route for fixing, it will be done.
Closes: #959
Approved by: baude
When installing an image, we now write a small bit of json
to /var/lib/atomic/install.json. The json format is:
{
<image_name>: {
id: <image_id>,
install_date: <install_date_in_utc
}
}
This will be used in update, run, etc to ensure that any image
with an INSTALL label is first installed.
Closes: #950
Approved by: giuseppe
When running an image with atomic run, if the container already exists,
we were accidently displaying a message stating that the container's
image was old. This was due to an invalid comparison between the container's
image id and the image's id. The comparison was failing because we were
comparing two different variables and not the ids.
This was reported in bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=1434430
Closes: #945
Approved by: rhatdan
If the image is built (and updated) locally, we need to handle this
differently than if the image came from a registry. We now
can try/except our way through things and properly handle.
Closes: #938
Approved by: rhatdan
Prior to refactoring, we had a check_latest method that would check
to see if the image being used by container was the latest image
by simply comparing hash IDs.
Adding this back in and refactoring into docker.py.
Closes: #938
Approved by: rhatdan
An exception was being thrown when trying to run a command inside
an already running container. For example:
sudo atomic run foobar echo true
The problem was a lack of a setter for user_command in the container
object as well as some mishandled string <-> list conversions.
Closes: #938
Approved by: rhatdan
The following integration tests have been added:
* ScanList
* ImagePull
* ImageUpdate
* ImageUpate when at latest
* ImagePull when already at latest
* Run
* ContainersDelete
* ContainersDelete when container doesnt exist
Closes: #904
Approved by: rhatdan
The bug https://bugzilla.redhat.com/show_bug.cgi?id=1425495 demostrates
a case where atomic attempts to "add" a list and str resulting in a
TypeError exception. This should resolve that BZ.
Closes: #903
Approved by: rhatdan
The new 2.x version of the docker python API has non-backward
compatible changes. These changes are described here:
https://docker-py.readthedocs.io/en/stable/change-log.html#breaking-changes
We need to account for docker.Client and docker.APIClient as well
as changes in the way kwargs are handled. Also, it appears the
AutoVersion method is deprecated.
Closes: #894
Approved by: rhatdan
BZ #1422448 actually points out one regression and one
bug related to atomic stop. The BZ itself points out
a TypeError exception in the code when trying to glue
together a python list and str.
Then the atomic stop command was failling in the case where a
STOP label was defined and it uses the variable $NAME resulting
in a subprocess exception.
The self.name variable was not being set by set_args after
refactoring occured. Ideally, this should all be moved
into the image|container object handling but for now we
just handle it in set_args.
Also added a test in test_display to catch any future
regressions.
Closes: #899
Approved by: rhatdan
Refactor the uninstall class to use the backends, container and image
objects.
Add --display so one can run atomic uninstall --display <image>
and see what the outcome would be. This will be helpful in integration
testing and was discussed in BZ https://bugzilla.redhat.com/show_bug.cgi?id=1248038.
Add --storage option so users can dictate priority when uninstalling
an image. This also mimics the newest install class as well.
Closes: #888
Approved by: giuseppe
There was a bug in the update code where passing the
kwargs and the named variable force resulted in a TypeError
for multiple values existing.
Fixed by not passing the force keyword to _docker.py and
extracting the keyword from the kwargs.
Closes: #881
Approved by: rhatdan
There were bugs in the run behavior in several scenarios:
* running an image with a command
* running a container that is stopped with and without a command
* running a container that is running with and without a command
Closes: #880
Approved by: rhatdan
To have parity function with pull, we need to add --storage
to install and run. This allows for overriding when atomic.type
is defined. For example, if you pull an image that has
atomic.type=system but you really want to store it
on the docker backend.
Closes: #861
Approved by: baude
If an image from a registry has a label like:
atomic.type=system
the Atomic CLI will now detect that prior to pulling
the image. It will then switch the backend storage
to ostree. In the case of install, it will both
pull to ostree and install it.
If the backend is switched, a message informing the user
as such will be shown.
The detection and switch of the backend can be overriden
with the --storage option.
The following is an example of install:
$ sudo atomic install atomic-registry.usersys.redhat.com:5000/brentbaude/helloworld:latest
Note: Switching from the 'docker' backend to the 'ostree' backend based on the 'atomic.type' label in the image. You can use --storage to override this behaviour.
Extracting to /var/lib/containers/atomic/helloworld.0
systemctl daemon-reload
systemctl enable helloworld
$ sudo ./atomic images list
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE TYPE
atomic-registry.usersys.redhat.com:5000/brentbaude/helloworld latest 0056366e4ebd 2017-02-01 18:04 ostree
$ sudo ./atomic containers list -a
CONTAINER ID IMAGE COMMAND CREATED STATE BACKEND RUNTIME
helloworld atomic-registry.user /usr/bin/run.sh 2017-02-01 18:04 failed ostree runc
Closes: #861
Approved by: baude
Fix a slew of issues related to deletion of images and containers. Also
added --assume_yes to be consistent with other commands. This will benefit
dbus interaction as well given that we do not want confirmation occuring
over dbus.
Issues that were fixed:
* Proper catch and error for trying to delete all images|containers when no images|containers exist
* Catch and error when no image|container is given by user and no --all
* when --storage and --all are used, only images|containers in that backend are now deleted
* when --all is given but not --storage, all images|containers from all backends are now deleted.
* Fixed logical error when deleting containers
* Deletion of images|containers now is consistent in its confirm and abort messages.
Added unittests.
Closes: #864
Approved by: rhatdan
`docker-run` man page on Fedora 25 and CentOS 7 states `label=disable`
as one of the security options. However, we check for `label:disable`
only. This commit should help us check for both
Closes: #863
Approved by: rhatdan
Users wish to be able to pull images by digest. This can be
done by docker and therefore parity is preferred. The request
is described in https://github.com/projectatomic/atomic/issues/691.
$ atomic pull busybox@sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912
Pulling docker.io/busybox@sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912 ...
Copying blob sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190
495.23 KB / 652.49 KB [=========================================>-------------]
Copying config sha256:e02e811dd08fd49e7f6032625495118e63f597eb150403d02e3238af1df240ba
0 B / 1.43 KB [---------------------------------------------------------------]
Writing manifest to image destination
Storing signatures
1.43 KB / 1.43 KB [===========================================================]
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/busybox 29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912 e02e811dd08f 3 months ago 1.093 MB
Note how the tag is populated with the digest. Docker's default behaviour is
to populate the tag with nothing. We cannot currently replicate that exactly.
Closes: #856
Approved by: baude
Ideally, the atomic CLI should be able to operate independently
of the backends it supports. For example, if dockerd is inactive,
the ostree backend and atomic cli should still work.
This requires some tweaking to the backendutils code and the work
flow. We also need to specifically know if the user passes
--storage so that we treat that as an explicit override. The work
flow is now roughly:
* a default storage can be defined in atomic.conf (was always this way)
* if not defined, defaults to docker.
* if --storage is passed, treat explictly and fail if cannot execute
* if no --storage is specified, use default. if default is not available, move
onto the next backend.
Fix regression related to atomic run <image> in the case the image
has already been run and a container is still around with the same
name as <image>.
Closes: #843
Approved by: rhatdan
Refactor several of the atomic verbs and subverbs to take advantage
of object refactoring.
Also, do not pull images with skopeo if the local image is already
at the latest.
$ sudo python ./atomic --debug pull busybox
Namespace(_class=<class 'Atomic.pull.Pull'>, assumeyes=False, debug=True, func='pull_image', image='busybox', reg_type=None, storage='docker')
Latest version of busybox already present.
Closes: #825
Approved by: baude
