upstream Docker has no support for additional registries, if that is
the case hardcode to ['docker.io'].
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1171
Approved by: baude
if skopeo doesn't include "msg=" in its error message, atomic fails
returning the wrong message.
Easily reproducible with:
SKOPEO_PATH=/bin/false atomic run foo
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
the capsh approach doesn't work on RHEL as the version of libcap is not
updated and doesn't know all the possible capabilities available on the
system. This is the output I get with getpcaps on RHELAH 7.4.2:
Capabilities for `1': = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,35,36+ep
Fallback to the capsh method if there will be more capabilities that we
know of, and hopefully libcap does.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1130
Approved by: rhatdan
Make sure that when TOML input for registries is used,
it is parsed correctly. Also, remain able to parse
YAML if still being used.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #1137
Approved by: rhatdan
In image_by_name, if a given image has a registry name in it,
do not consider it equal to another image with the same name and
tag.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #1112
Approved by: baude
http: is now required by skopeo when working with insecure registries.
However, having the prefix causes other calls/tools problems. This
change strips off http:/https: from the image uri outside of skopeo
usage.
Signed-off-by: Steve Milner <smilner@redhat.com>
Closes: #1114
Approved by: giuseppe
New versions of Skopeo support "ostree" as a destination for copy. The
missing layers are written directly to the OSTree storage without any
additional handling from atomic.
Check if the used version of Skopeo has support for ostree and use
"skopeo copy" in this case. In future we might drop completely the
other code path and assume ostree is always supported.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1082
Approved by: baude
Fix test failures after updating papr to test with f26 atomic/cloud
images instead of f25, with the following changes:
1. Remove dependency on docker hub tester image. Instead, mimic
what ostree/rpm-ostree does and use a recursive .papr.sh script
to install the necessary packages to the base f26 image in the
fedora registry. This fixes tests on the atomic host since python3.6
is being used, and prevents future tests from testing the wrong
version. (Note this is slightly slower due to having to install
packages during the test rather than using a pre-built image).
2. Fix some pylint errors, and mask others for now
3. Fix failing integration tests due to inter-test interference
4. Remove unnecessary deepcopy in container filter
5. Add compatibility for both c-s-s and d-s-s in storage
6. Update expected sha256 values for dockertar test
Remaining issues:
1. test_storage should possibly be reworked. The current test
setup is conflicting with the new default of overlay as a driver.
For now, the test for generated d-s-s is disabled.
2. some storage commands are still using "docker-storage-setup"
instead of "container-storage-setup". There is a backward
compatible check in place that should be reworked in the future
3. some masked pylint errors should be further investigated
4. keep the dockerfile for the project atomic tester image for now
(bump to 26), since its a little easier and faster to set up with
Signed-off-by: Yu Qi Zhang <jerzhang@redhat.com>
Closes: #1076
Approved by: baude
In order to avoid warnings from skopeo, we should be using
tls-verify on the skopeo subcommand (like copy) rather than
skopeo itself.
This was reported in issue #1067.
Closes: #1072
Approved by: rhatdan
The command switch for --gnupghome was not being honored when an atomic
push was being done. We now export it to an os.environ so that skopeo
can use it.
Closes: #1071
Approved by: rhatdan
It is easier to test different versions if we can override the
program used by an env variable.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1043
Approved by: baude
If a shortname is used to run an image, we need to transform the short
name into the fq-name when doing the lookup in the installed images
data.
Reported in BZ #1454292
Closes: #1010
Approved by: baude
Ideally atomic will be able to run multiple container
runtimes. In and effort to do so, we must have a concept
of a global registries configuration file which will allow
us to pull images with atomic and skopeo. The tooling
that parses the global configuration file is:
https://github.com/projectatomic/registries
This is step one in the implementation of this tooling
and direction.
Closes: #1003
Approved by: rhatdan
http_proxy and https_proxy are currently supported in /etc/atomic.conf.
This change adds no_proxy support. no_proxy is for urls which should
not go through a proxy.
Closes: #999
Approved by: baude
Bugzilla #1430708 recommends that if an atomic user attempts to pull
an image that is already present, we should not exit with a '1' which
indicates a failure; rather a 0.
Closes: #997
Approved by: baude
The syscontainers mount procedure follows a try and fallback approach. If the
first attemp fails, it displays an error message to the user but then the
fallback attempt works. This makes the user think the attempts failed. So
not we supress the error message of the first attemp unless --debug is
provided.
Closes: #998
Approved by: baude
In cases where a short-hand name is provided to run and the image
was installed under its fq_name, we should check againt the
fq_name as well when determining if it has been installed.
This was reported as issue #995
Closes: #996
Approved by: baude
We shouldn't do anything at module import time, as that happens during builds,
where we don't want to touch the host system (and may not have privileges to do
so).
This required some refactoring of the locking code too, as simply
instantiating the decorator was creating files too.
Closes: https://github.com/projectatomic/atomic/issues/963
Closes: #966
Approved by: rhatdan
If HTTP[S]_PROXY is defined, honor it in python requests usage
as well as pass it on to skopeo.
If http[s]_proxy is defined in atomic.conf, use it; however, environment
variables will override these if defined.
Added --insecure to Atomic push so the user can override the logic
(or lack thereof) around deducing if a registry is insecure. Also
needed for integration tests.
Closes: #964
Approved by: rhatdan
Refactor the RPM generation code for system containers in a new file so
that it can be shared with the Docker backend.
For fully supporting the same /exports structure we will need to add the
support for reading the manifest.json file as well, but since the Docker
backend doesn't use --set for settings of the container, preprocessing
files won't be very useful.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #949
Approved by: baude
When installing an image, we now write a small bit of json
to /var/lib/atomic/install.json. The json format is:
{
<image_name>: {
id: <image_id>,
install_date: <install_date_in_utc
}
}
This will be used in update, run, etc to ensure that any image
with an INSTALL label is first installed.
Closes: #950
Approved by: giuseppe
Instead of calling the default_docker_lib function a bunch of times
in the arg parser, we call it one and use the result in the parser.
Also, tighten up some exit conditions where self.d.close() is being
called when dockerd is not running.
Closes: #921
Approved by: rhatdan
When Atomic unmount is called on an overlayfs based dockerd,
the default docker library is incorrectly determined resulting in
failures to unmount.
We now first ask the running dockerd what its root library directory
is and use that if dockerd is running.
This resolves https://bugzilla.redhat.com/show_bug.cgi?id=1381696
Closes: #912
Approved by: rhatdan
The new 2.x version of the docker python API has non-backward
compatible changes. These changes are described here:
https://docker-py.readthedocs.io/en/stable/change-log.html#breaking-changes
We need to account for docker.Client and docker.APIClient as well
as changes in the way kwargs are handled. Also, it appears the
AutoVersion method is deprecated.
Closes: #894
Approved by: rhatdan
In the case of cockpit, it would be preferable to be able
to lookup scan results by a container or image's id. If the
container or image has not been scanned, we throw an exception;
otherwise we return the resulting json file as a str.
One other possible exception can be thrown when attempting to read
the desired file from the filesystem. If the file cannot be read,
an exception will be thrown. Either way, it is a clear indicator
that the object needs to be scanned for fresh results.
The following is a simple *python* example:
from atomic_dbus_client import AtomicDBus
ad = AtomicDBus()
results = ad.GetScanResultsById('6858a846fb6b557331e068252fd910b5dc93f8e6341e641400bf4582dc34e10d')
Note the use of the full ID. As of now, we only look up against the full id
as opposed to the short id form which is often used.
Closes: #874
Approved by: baude
When a fq image name is used, the error messages contained the full
stderr formed by skopeo. It was preferable to only show the msg
portion of the error.
Closes: #862
Approved by: baude
Added two tests for pull by digest: one where the image is not
present and the other when it is.
Also, addressed review comments.
Closes: #856
Approved by: baude
Ideally, the atomic CLI should be able to operate independently
of the backends it supports. For example, if dockerd is inactive,
the ostree backend and atomic cli should still work.
This requires some tweaking to the backendutils code and the work
flow. We also need to specifically know if the user passes
--storage so that we treat that as an explicit override. The work
flow is now roughly:
* a default storage can be defined in atomic.conf (was always this way)
* if not defined, defaults to docker.
* if --storage is passed, treat explictly and fail if cannot execute
* if no --storage is specified, use default. if default is not available, move
onto the next backend.
This patch will allow user to specify the graphdriver on atomic reset
If /var/lib/docker or /var/lib/docker-latest is the only thing installed
it will reset the correct path. If both exists or the user as chosen
a different location, the --graph option must be specified.
Closes: #745
Approved by: rhatdan
With the exception of fstrim, the containers verb has now been
refactored. It primarily now uses the containers object in its
implementation.
Closes: #792
Approved by: rhatdan
Using our refactoring model, verify is now streamlined. We no longer
compare base images as that is not currently possible for both
V1 and V2 schemas.
Verify will now always look at the release and version labels for
comparison. Should those labels not exist, it will use the
manifest digest for ostree; and it will use the image IDs for
docker.
Closes: #785
Approved by: rhatdan
