Fixed E305: expected 2 blank lines after class or function definition, found 1
and re-enabled the test for it in tox.
Change-Id: I4e9ef63491b6f4063880b8e2db64857cd8c826ee
Related-To: https://bugzilla.redhat.com/show_bug.cgi?id=2004412
Signed-off-by: Sandro Bonazzola <sbonazzo@redhat.com>
We don't need to differentiate py2 and py3 builds anymore.
Change-Id: I38f31a8a4db9cc2bfc4e1e04c25480b95fa5c664
Signed-off-by: Marcin Sobczyk <msobczyk@redhat.com>
Starts a full backup for a specified VM.
start_backup creates an XML from the given parameters in the following
format:
<domainbackup mode='pull'>
<server transport='unix' socket='/run/vdsm/backup-id.sock'/>
</domainbackup>
The XML doesn't specify any of the VM disks (will be added in a later
patch), therefore, the backup will include all of the VM disks.
Result XML from libvirt when running backupGetXMLDesc():
<domainbackup mode='pull' id='0'>
<server transport='unix' socket='/run/vdsm/backup-id.sock'/>
<disks>
<disk name='vda' type='file'>
<driver type='qcow2'/>
<scratch file='/path/to/scratch/disk.qcow2'/>
</disk>
<disk name='sda' type='file'>
<driver type='qcow2'/>
<scratch file='/path/to/scratch/disk.qcow2'/>
</disk>
...
</disks>
</domainbackup>
Starts a backup for the specified VM by calling
libvirt backupBegin() job.
The path to a socket is passed to QEMU and it will create the socket.
Imageio will then connect to this socket during backup.
All the sokets are stored under VDSM_RUN_DIR/backup.
Example for VDSM log when starting a backup -
INFO (jsonrpc/2) [api.virt] START start_backup(config={'backup_id': '123',
'disks': [...], 'from_checkpoint_id': None, 'to_checkpoint_id': None}
) from=..., flow_id=789, vmId=345
Example for successful backup operation return value-
{
'disks':{
'111':'nbd:unix:/var/run/vdsm/backup_sockets/111.sock:exportname=sdb',
'222':'nbd:unix:/var/run/vdsm/backup_sockets/222.sock:exportname=sda'
}
}
Change-Id: Ibf83077066d5cfed4f5c7551668c8e4c441f6ed5
Signed-off-by: Eyal Shenitzky <eshenitz@redhat.com>
The transientdisk module created in order to provide an API for adding
a temporary disks on a temporary directory.
The module will be responsible to create the transient disks directory under -
/var/lib/vdsm/storage/transient_disks which creates when VDSM starts,
when all the transient disks will be removed from the directory,
the directory will be deleted.
This module is needed for creating "scratch" disks for the full VM backup
which is the first step in the incremental backup flow.
The creation and removal of the transient disk added as public API for
future usage, currently the API is used internally only.
Change-Id: If01c59b9185fa4eb7ce4be247605563db41303a2
Signed-off-by: Eyal Shenitzky <eshenitz@redhat.com>
Commit sequence numbers make little sense when used with timestamped
release numbers. The purpose of timestamped release numbers is to
make Vdsm updates on hosts easier during development, by always
producing newer versions that we can easily upgrade to. But when
prefixed with a commit sequence number, switching between development
branches or different commits doesn't work this way, or we can easily
be behind upstream repos available on the system.
Let's drop commit sequence numbers from the timestamped release
numbers, we won't miss anything.
Change-Id: I24f9fd4428b4f29af2b6a2eebebce4ef93aa727e
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Vdsm versions are retrieved automatically from git and tags. This
works well with most commits but doesn't cooperate very well with
tagging.
There are currently several problems with tagging:
- When a commit is tagged with a new version, the commit is built with
the old version (before it is tagged) in Jenkins and must be
re-built after tagging again to change its version to the new one.
- When a new stable branch is created, the very next master patch,
once it is committed, must be tagged with a new master version. And
until this is done, all the master commits are built with the old
version in Jenkins and possibly elsewhere.
- Builds from commits in different branches may have the same
VERSION-RELEASE (minus git hash) number until the master branch is
tagged.
These are not very serious problems, but the versioning and releasing
process is still somewhat messy. We can improve it, by putting an
explicit version number to the spec, making version bump commits on
each release tagging, and ensuring that new versions are used even
before the tags are created. This approach, addressed by this patch,
has the following advantages:
- Releases are clearly identified by the version bump commits and they
are visible e.g. in ChangeLog.
- Jenkins builds new releases with proper versions immediately and
they can be used as upstream sources without further steps.
- There are no double-version or old-version-only builds for tagged
commits.
- The version bump commit can be tagged immediately when branching,
without waiting for a followup commit to be merged. We branch out
from the preceding commit.
We also don't tag, unlike Engine, releases from master branch. This
is confusing, especially regarding resulting upstream and downstream
versions, which deviate. This patch doesn't address that problem, all
what is needed to start making version change commits also for
releases created from master.
Change-Id: I6433a4939d0ae790b9d77ab10fcb204a9bfedb72
Backport-To: 4.3
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
If7e157448a stopped using build-aux/pylint-py3k-whitelist, py3k-pylintrc
and py3-blacklist.txt, but kept them in the code base.
Change-Id: Ic3e978bf4b7d0c506f7a1af09fa7934b9aba40be
Signed-off-by: Dan Kenigsberg <danken@redhat.com>
it could happen that one of the .git files match the python expression
Change-Id: I1950feb8cd5952223d0fbcf95306a82c3f6e11d6
Signed-off-by: Michal Skrivanek <michal.skrivanek@redhat.com>
use POSIX compatible option for sed and bash for "echo -e"
Change-Id: Ia648d1cf4268b6385fb8b44571c5b4526e6ce37d
Signed-off-by: Michal Skrivanek <michal.skrivanek@redhat.com>
Currently, vdsm.spec is generated from vdsm.spec.in using values
produced by configure. That means vdsm.spec must be generated in the
same environment as the target build environment. This is not always
desirable, i.e. it should be possible to build a SRPMS in a different
environment or build dependencies should be available before building
the package (chicken or egg problem). These changes are especially
needed for current automated package building infrastructures.
This patch replaces the current configure variables with default
values. While this duplicates some of the values already defined in
configure.ac, the benefit of having a ready to use spec file
outweighs the maintenance cost of the duplication. The default values
can be overridden by passing corresponding --define command line
options to rpmbuild.
We cannot have a completely working autonomous vdsm.spec, because
package version and release are defined dynamically, by running
scripts from the source directory. A spec file doesn't know the
location of the source directory when generating a SRPM and thus can't
run pkg-version script from it. There are the following ways to have
a complete spec file:
- Running configure and make.
- Using build-aux/make-dist script (with or without running
configure beforehand).
- Using vdsm.spec.in directly and defining vdsm_version and
vdsm_release macros on rpmbuild command line, e.g.:
rpmbuild -bs -D 'vdsm_version 4.40.0' -D 'vdsm_release 1' vdsm.spec.in
The source tarball must be generated manually in this case.
Change-Id: I3ee44a76c8120dc6e8f17f75be62e25fdba1336d
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
This patch introduces a dist hook, that will replace all occurences of
'#!/usr/bin/python[23]?' shebangs with ones that use
'VDSM_TARGET_PY_MAJOR_VERSION' variable. The hook doesn't modify your
tree - it works in the dist directory, which is then archived.
Warning! Currently a lot of our are scripts are *not* yet
py3-compatible - this is a build-oriented change only.
Change-Id: I807e8982fbca0027f138d9c8a9732625c2cc159a
Signed-off-by: Marcin Sobczyk <msobczyk@redhat.com>
Extracted 'list-all-python-files' script out of 'pylint-py3k-whitelist'.
Additionally, the output of the listing has been sorted and ensured
to be deprived of duplicates.
Change-Id: I845a0e404f68b17ffeaec24694175c1bed922ea1
Signed-off-by: Marcin Sobczyk <msobczyk@redhat.com>
To install vdsm from source, the best way is to do:
rm -rf ~/rpmbuild/RPMS
make rpm
(cd ~/rpmbuild/RPMS && dnf upgrade */*.rpm)
This upgrades the installed packages from your build, and correctly
fetch new dependencies added by the current vdsm version.
But there is a big issue - if you fix your code and build again, upgrade
may fail since the git commit hash may be considered older than the
installed package git commit hash. A developer can work around this by
using "downgrade" or "reinstall" but it is pretty annoying.
Fix this issue by adding a timestamp before the git commit hash. Here is
an example package name:
vdsm-4.40.0-115.201903241356.git8e34445b2.fc28.x86_64.rpm
The timestamp is enabled only if vdsm is configured with
--enable-timestamp, so it does affect packages built in the CI.
To enable timestamps run:
./autogen.sh --enable-timestamp
Change-Id: I5d07103138d0cb4f93d1c5cefb80ab30c3b91d4e
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Several targets' paths had nested directories
(i.e. './usr/share/man/man8/vdsmd.8'). When doing in-tree builds
everything was fine, because the directory with source '.in' file
existed. However, when doing out-of-tree builds, the directory
of the output file doesn't exist and we have to create it by ourselves.
Change-Id: I77d4c0e649f9a2744665223f457ce93a4081716c
Bug-Url: https://bugzilla.redhat.com/1297808
Signed-off-by: Marcin Sobczyk <msobczyk@redhat.com>
A 'pylint-py3k' make target has been added along with a helper script
to provide a list of Python files. Files known to be incompatible at the
moment are placed in 'py3-blacklist.txt'. The rationale behind this change
is to avoid introducing new incompatibilities in development process.
Depends on: https://gerrit.ovirt.org/#/c/94253/
Change-Id: I719f3a6a906356d2fe54be128da58509e73ce07a
Signed-off-by: Marcin Sobczyk <msobczyk@redhat.com>
starting from fc25, python3 will be default, and
/usr/bin/python path will point to python3.
All interpreter paths in python scripts were changed
to python2.
This is a quick fix in order to ensure the scripts will
continue to run without errors.
All scripts should be py3 compatible in the future.
Signed-off-by: Irit Goihman <igoihman@redhat.com>
Change-Id: Ie8eaefe5f7e0831c362546733a2fa9301b7f9ad6
We make sure that all the references to certificates location are in
single place. As part of this fix we remove trust store path from
configuration which is not needed anymore.
Change-Id: I58dd3a5f7c1503fc38b6c6a204c036c06d09941b
Signed-off-by: pkliczewski <piotr.kliczewski@gmail.com>
Reviewed-on: https://gerrit.ovirt.org/52354
Continuous-Integration: Jenkins CI
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Tested-by: Irit Goihman <igoihman@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Reviewing the vdsm spec found some remaining of vdsm reg
deprecated in early 3.6 versions.
Change-Id: I9f83b9e18ca24df4553e5b39e5b862759b9ac1b4
Signed-off-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Reviewed-on: https://gerrit.ovirt.org/62463
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
Continuous-Integration: Jenkins CI
Over RHEL 7.2 yum contains only tox-1.4.2 which doesn't fit to vdsm
requirements.
We require to install tox with pip currently.
This patch validates tox and its version and fails the make process if tox
is older than 2.1.1
tox check was removed from configure.ac and now it's existence will be
checked only during the build process.
Change-Id: I665025dacdd5346a5e021ac98e864f7b6461917c
Signed-off-by: Irit Goihman <igoihman@redhat.com>
Reviewed-on: https://gerrit.ovirt.org/59306
Continuous-Integration: Jenkins CI
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Generated script tests/run_tests_local.sh is not executable after
`make clean; make' is run. This bug was introduced by commit 6e6f89b7,
but there is probably nothing wrong with that commit, it just introduces
removal of some files including run_tests_local.sh. The problem is
tests/run_tests*.sh files are not set as executable if they are
recreated any time later.
This patch tries to solve the problem by making the following changes:
- tests/run_tests*.sh are no longer handled in configure.ac. They are
not config files and such a use of AC_OUTPUT is obsolete anyway.
- New tests/Makefile.am rule is introduced ensuring generated
run_tests*.sh files are set as executable.
- top_srcdir substitution is added to Makefile.subs to make the inserted
relative paths in the generated files always correct.
- run_tests_local.sh is moved to *_SCRIPTS target in tests/Makefile.am.
There may be other solutions of the problem but whatever solution is
chosen, it should be checked that tests/run_tests*.sh files are
executable and the paths inside the generated files are right in all the
following situations:
- After running ./autogen.sh + make in a fresh git directory.
- After `rm tests/run_tests*.sh; make'.
- After `(cd tests; rm run_tests*.sh; make)'.
Change-Id: Ibc1e3dc8ace7f69801b765262352903020cc8aef
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-on: https://gerrit.ovirt.org/55949
Reviewed-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
Continuous-Integration: Jenkins CI
Reviewed-by: Tomas Golembiovsky <tgolembi@redhat.com>
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
For stable branches, we should never have release < 1.
E.g. we should avoid
v4.y.z-0
instead we should have
v4.y.z-1
Change-Id: Idace2ac53d6aafe674c7ad11f611d0de54fcbe5f
Signed-off-by: Francesco Romani <fromani@redhat.com>
Reviewed-on: https://gerrit.ovirt.org/59395
Continuous-Integration: Jenkins CI
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
On startup the storage pool directory (/rhev/data-center/<spUUID>)
is removed by __cleanStorageRepository preventing the automatic
reconnection (_connectStoragePool) from actually taking place.
Since vdsm has been running for quite some time (years) without the
automatic reconnection we can safely remove it.
Change-Id: Ia57afc04db02b6c15633d09349a55b3ff5ae7fda
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/21424
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
'/var/run' is symlink of '/run', so the dir '/run/vdsm' also be
created when the tempfiles service create '/var/run/vdsm'. And
we don't use '/run/vdsm' directly, so we don't need care about it.
Change-Id: I3872ed37850142d81dd324f38f7f018168f6850c
Signed-off-by: Mark Wu <wudxw@linux.vnet.ibm.com>
Reviewed-on: http://gerrit.ovirt.org/16905
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
tmpfiles.d is a service in Fedora 15 and later for managing temporary files and
runtime directories for daemons. Directories /var/run and /var/lock are symlinks
into the /run tmpfs filesystem. As such, they are created empty on every reboot.
For files intended to be placed into those directories, this should normally
not pose any problems. For directories, however, we often need to create the
directories ahead of time. This is best done using the tmpfiles.d mechanism.
Change-Id: Iea9ec4b7ba1e362c1141c4e0cb734d2b1520dbfd
Signed-off-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/15707
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Previously the upstream dirty builds were released using a pre-release
versioning (prefixing a 0). This doesn't fit the release process in
which the upstream builds are nightly.
More information on pre-releases at:
http://fedoraproject.org/wiki/Packaging:NamingGuidelines#Pre-Release_packages
In this patch:
* remove the leading 0 used for pre-releases (instead of nightly)
* remove vdsm.spec on make clean
* use any tag (not just the annotated ones) to identify the release
Change-Id: Ibbf52173ccaa4a4eb0de5aa2cf61f821ba1fa215
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/14954
Reviewed-by: Kiril Nesenko <knesenko@redhat.com>
Tested-by: Kiril Nesenko <knesenko@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
According to the man page for 'test', the correct token to use for string
equality is '=', not '=='. '==' works for the bash builtin version but we
should not depend on it (especially since the pkg-version shebang calls for
/bin/sh). This fixes the build on my Ubuntu system.
Change-Id: I26c317392a55881f96cb4859db822ecd28cc1a65
Signed-off-by: Adam Litke <agl@us.ibm.com>
Reviewed-on: http://gerrit.ovirt.org/7834
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Shipping the VERSION file allows running the autoreconf tool also from a
tarball package (no need of the entire git repository).
In this patch:
* generate and ship the VERSION file
* move, unify (and ship) version.sh and release.sh in pkg-version
* use the VERSION file when the git repository is not available
(eg: tarball)
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Change-Id: I8b72a1740803a9401e4b5a4504a4faa07c29f2b9
Reviewed-on: http://gerrit.ovirt.org/7295
Reviewed-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Reviewed-by: Alon Bar-Lev <alonbl@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Dan Kenigsberg <danken@redhat.com>
Move the creation of /data/updates out of init script,
make init script more simple.
* move the creation of /data/updates to vdsm_reg/vdsm-reg-setup.in
vdsm-reg-setup can create '/data/updates' after host register to engine.
* remove /data/updates after vdsm rpm uninstalled
Change-Id: I9f8617fdaca3110791a93bb3a5906f30ace2499d
Signed-off-by: Xu He Jie <xuhj@linux.vnet.ibm.com>
Reviewed-on: http://gerrit.ovirt.org/4025
Reviewed-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Moving all the substitution at make time we can avoid the double pass
on the .in.in files.
Change-Id: I1f6956350971a88294559039e4ff00fce2637c78
Reviewed-on: http://gerrit.ovirt.org/1034
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Federico Simoncelli <fsimonce@redhat.com>
In this patch:
* Use the PYTHON macro to compile python files
* Remove the redhat-rpm-config build dependency
* Move the vdsm sos plugin into its own directory to avoid the custom
file renaming and to use the PYTHON macro
* Update the gitignore file and the copyright in the touched files
* Add the all-local target to force the substitutions in the python
files
* Unify the macro substitution in build-aux/Makefile.subs
Change-Id: I7dce5c317605b84ec883feb04bfc83c5f64d921b
Reviewed-on: http://gerrit.ovirt.org/708
Tested-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
In this patch:
* revise the release versioning according to fedora guidelines
Change-Id: I1f7b267a3ea9f415219fec0d45d4e4dad8eadb70
Reviewed-on: http://gerrit.ovirt.org/218
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Dan Kenigsberg <danken@redhat.com>
This adds a script that will automatically generate the ChangeLog
file content from the git changelog. This is performed only when
running 'make dist', via an automake hook
* .gitignore: Don't ignore the entire of build-aux/
* Makefile.am: Add hook to generate ChangeLog
* build-aux/gitlog-to-changelog: Script for generating ChangeLog
from GIT history
Change-Id: If2a947d6ee31178d799f9e7bcbbbfe9cee0ba170
Reviewed-on: http://gerrit.usersys.redhat.com/840
Reviewed-by: Federico Simoncelli <fsimonce@redhat.com>
Tested-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-by: Saggi Mizrahi <smizrahi@redhat.com>
