We were not passing xattr_req when doing a name self heal
as well as a meta data heal. Because of this, some xdata
was missing which causes i/o errors
Change-Id: Ibfb1205a7eb0195632dc3820116ffbbb8043545f
Fixes: bz#1728770
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Problem: useradd fails with: Cannot allocate memory
useradd: cannot lock /etc/passwd; try again later.
Solution:
Lock files should get automatically removed once "usradd" or "groupadd"
command finishes. But sometimes we encounter situations (bugs) where
some of these files may not get properly unlocked after the execution of
the command. In that case, when we execute useradd next time, it may show
the error “cannot lock /etc/password” or “unable to lock group file”.
So, to avoid any such errors, check for any lock files under /etc and
remove those.
updates: bz#1193929
Change-Id: If6456a271c2bc0717f768d7101a40ce44a9af3d7
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
While running a test case the following warning messages are seen
on the display. To avoid suh warnings changing usleep() to sleep().
warning: usleep is deprecated, and will be removed in near future!
warning: use "sleep 0.25" instead...
updates: bz#1193929
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
Change-Id: I48b79ede1c70b101f654635dd4cc83e50ea55b73
When tests timeout, the timeout command sends TERM
signal to the command being executed. In the case of run-tests.sh
it invokes prove, which further invokes perl and finally the test
is run using bash. The TERM signal does not seem to be reachnig
the end bash that is actually executing the tests, and hence
when any test is terminated due to a timeout, the cleanup routine
in include.rc does not get a chance to run and preserve the
tarball.
Further, cleanup invokes tarball generation, but is invoked at
the beginning and end of every test, and at times in beteween
as well. This caused way too many tarballs in case we decide to
preserve the same whenever generated by cleanup.
This patch hence moves the tarball generation to run-tests.sh
instead, and further stores them named <test>-iteration-<n>.tar
and also prints tarball name generated and stored per iteration.
This should help relate failed runs to the tarball iteration #
and to look at relevant logs.
Further the patch also provides a -p option to run-tests.sh for
unit testing purposes, where running a test in a loop without the
option will generate as many tarballs, and using the option will
reduce this to preserving the last tarball, saving space in
smaller unit test setups.
Fixes: bz#1614062
Change-Id: I0aee76c89df0691cf4d0c1fcd4c04dffe0d7c896
Signed-off-by: ShyamsundarR <srangana@redhat.com>
This patch addresses following:
1. On volume stop, for the last brick, pmap_registry_remove () is
invoked by glusterd.
2. If a brick process is sigkilled, remove all the associated brick
instances from the portmap.
3. Bump up PROCESS_UP_TIMEOUT to 45.
4. gf_attach to kill a brick takes more time in mux (which is an
issue that needs a fix), but in the interim, give br-state-check.t
more time to complete (there are 2 kill_bricks, each taking 120
seconds, and the test usually passes in 30 odd seconds, hence bumping
this up to 350 seconds)
5. The test bug-1559004-EMLINK-handling.t is taking ~950 seconds at
times on master without mux, in mux cases, when it fails, it is almost
at the last iteration, hence bumping the timeout for this test case
to reduce regression error rates
Updates: bz#1577672
Change-Id: I1922675e112baca4c125c4c094eaa42a11e34e67
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Test case:
# while true; do uuid="`uuidgen`"; echo "some data" > "test$uuid"; mv
"test$uuid" "test" -f || break; echo "done:$uuid"; done
This script was run in parallel from multiple mountpoints
Along the course of getting the above usecase working, many issues
were found:
Issue 1:
=======
consider a case of rename (src, dst). We can encounter a situation
where,
* dst is a file present at the time of lookup
* dst is removed by the time rename fop reaches glusterfs
In this scenario, acquring inodelk on dst fails with ESTALE resulting
in failure of rename. However, as per POSIX irrespective of whether
dst is present or not, rename should be successful. Acquiring entrylk
provides synchronization even in races like this.
Algorithm:
1. Take inodelks on src and dst (if dst is present) on respective
cached subvols. These inodelks are done to preserve backward
compatibility with older clients, so that synchronization is
preserved when a volume is mounted by clients of different
versions. Once relevant older versions (3.10, 3.12, 3.13) reach
EOL, this code can be removed.
2. Ignore ENOENT/ESTALE errors of inodelk on dst.
3. protect namespace of src and dst. To protect namespace of a file,
take inodelk on parent on hashed subvol, then take entrylk on the
same subvol on parent with basename of file. inodelk on parent is
done to guard against changes to parent layout so that hashed
subvol won't change during rename.
4. <rest of rename continues>
5. unlock all locks
Issue 2:
========
linkfile creation in lookup codepath can race with a rename. Imagine
the following scenario:
* lookup finds a data-file with gfid - gfid-dst - without a
corresponding linkto file on hashed-subvol. It decides to create
linkto file with gfid - gfid-dst.
- Note that some codepaths of dht-rename deletes linkto file of
dst as first step. So, a lookup racing with an in-progress
rename can easily run into this situation.
* a rename (src-path:gfid-src, dst-path:gfid-dst) renames data-file
and hence gfid of data-file changes to gfid-src with path dst-path.
* lookup proceeds and creates linkto file - dst-path - with gfid -
dst-gfid - on hashed-subvol.
* rename tries to create a linkto file dst-path with src-gfid on
hashed-subvol, but it fails with EEXIST. But EEXIST is ignored
during linkto file creation.
Now we've ended with dst-path having different gfids - dst-gfid on
linkto file and src-gfid on data file. Future lookups on dst-path will
always fail with ESTALE, due to differing gfids.
The fix is to synchronize linkfile creation in lookup path with rename
using the same mechanism of protecting namespace explained in solution
of Issue 1. Once locks are acquired, before proceeding with linkfile
creation, we check whether conditions for linkto file creation are
still valid. If not, we skip linkto file creation.
Issue 3:
========
gfid of dst-path can change by the time locks are acquired. This
means, either another rename overwrote dst-path or dst-path was
deleted and recreated by a different client. When this happens,
cached-subvol for dst can change. If rename proceeds with old-gfid and
old-cached subvol, we'll end up in inconsistent state(s) like dst-path
with different gfids on different subvols, more than one data-file
being present etc.
Fix is to do the lookup with a new inode after protecting namespace of
dst. Post lookup, we've to compare gfids and correct local state
appropriately to be in sync with backend.
Issue 4:
========
During revalidate lookup, if following a linkto file doesn't lead to a
valid data-file, local->cached-subvol was not reset to NULL. This
means we would be operating on a stale state which can lead to
inconsistency. As a fix, reset it to NULL before proceeding with
lookup everywhere.
Issue 5:
========
Stale dentries left out in inode table on brick resulted in failures
of link fop even though the file/dentry didn't exist on backend fs. A
patch is submitted to fix this issue. Please check the dependency tree
of current patch on gerrit for details
In short, we fix the problem by not blindly trusting the
inode-table. Instead we validate whether dentry is present by doing
lookup on backend fs.
Change-Id: I832e5c47d232f90c4edb1fafc512bf19bebde165
updates: bz#1543279
BUG: 1543279
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Instead send the SIGTERM (default, 15) first, and at the end
send SIGKILL. If SIGKILL is sent directly, we miss many tests
like valgrind, lcov etc., not able to process the information
properly.
BUG: 1549000
Change-Id: I664de12ee7dbf47eb98b8141004cd51f6006b314
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).
These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.
So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
BUG: 1258561
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: https://review.gluster.org/13580
Tested-by: MOHIT AGRAWAL <moagrawa@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Problem:
Enabling optimistic changelog on EC volume was not
handling node down scenarios appropriately resulting
in volume data inaccessibility.
Solution:
Update dirty xattr appropriately on good bricks whenever
nodes are down. This would fix the metadata information
as part of heal and thus ensures data accessibility.
BUG: 1468261
Change-Id: I08b0d28df386d9b2b49c3de84b4aac1c729ac057
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-on: https://review.gluster.org/17703
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Problem-1 : Recursive healing of same file is happening
when IO is going on even after data heal completes.
Solution:
RCA: At the end of the write, when ec_update_size_version
gets called, we send it only on good bricks and not
on healing brick. Due to this, xattr on healing brick
will always remain out of sync and when the background
heal check source and sink, it finds this brick to be
healed and start healing from scratch. That involve
ftruncate and writing all of the data again.
To solve this, send xattrop on all the good bricks as
well as healing bricks.
Problem-2: The above fix exposes the data corruption
during heal. If the write on a file is going on and
heal finishes, we find that the file gets corrupted.
RCA:
The real problem happens in ec_rebuild_data(). Here we receive the
'size' argument which contains the real file size at the time of
starting self-heal and it's assigned to heal->total_size.
After that, a sequence of calls to ec_sync_heal_block() are done. Each
call ends up calling ec_manager_heal_block(), which does the actual work
of healing a block.
First a lock on the inode is taken in state EC_STATE_INIT using
ec_heal_inodelk(). When the lock is acquired, ec_heal_lock_cbk() is
called. This function calls ec_set_inode_size() to store the real size
of the inode (it uses heal->total_size).
The next step is to read the block to be healed. This is done using a
regular ec_readv(). One of the things this call does is to trim the
returned size if the file is smaller than the requested size.
In our case, when we read the last block of a file whose size was = 512
mod 1024 at the time of starting self-heal, ec_readv() will return only
the first 512 bytes, not the whole 1024 bytes.
This isn't a problem since the following ec_writev() sent from the heal
code only attempts to write the amount of data read, so it shouldn't
modify the remaining 512 bytes.
However ec_writev() also checks the file size. If we are writing the
last block of the file (determined by the size stored on the inode that
we have set to heal->total_size), any data beyond the (imposed) end of
file will be cleared with 0's. This causes the 512 bytes after the
heal->total_size to be cleared. Since the file was written after heal
started, the these bytes contained data, so the block written to the
damaged brick will be incorrect.
Solution:
Align heal->total_size to a multiple of the stripe size.
Thanks "Xavier Hernandez" <xhernandez@datalab.es>
to find out the root cause and to fix the issue.
Change-Id: I6c9f37b3ff9dd7f5dc1858ad6f9845c05b4e204e
BUG: 1428673
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: https://review.gluster.org/16985
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Reset brick currently kills of the corresponding brick process.
However, with brick multiplexing enabled, stopping the brick
process would render all bricks attached to it unavailable. To
handle this correctly, we need to make sure that the brick process
is terminated only if brick-multiplexing is disabled. Otherwise,
we should send the GLUSTERD_BRICK_TERMINATE rpc to the respective
brick process to detach the brick that is to be reset.
Change-Id: I69002d66ffe6ec36ef48af09b66c522c6d35ac58
BUG: 1446172
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17128
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
It can often be useful while debugging to know how many times
EXPECT_WITHIN had to retry a command before it got the result we were
looking for. This patch just adds a variable EW_RETRIES that can be
inspected to find this info for the last EXPECT_WITHIN.
Change-Id: I1bcb09bb7eb118c3d76c60317ef99e02df6b6ee6
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: https://review.gluster.org/16451
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
This patch adds support for multiple brick translator stacks running
in a single brick server process. This reduces our per-brick memory usage by
approximately 3x, and our appetite for TCP ports even more. It also creates
potential to avoid process/thread thrashing, and to improve QoS by scheduling
more carefully across the bricks, but realizing that potential will require
further work.
Multiplexing is controlled by the "cluster.brick-multiplex" global option. By
default it's off, and bricks are started in separate processes as before. If
multiplexing is enabled, then *compatible* bricks (mostly those with the same
transport options) will be started in the same process.
Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
BUG: 1385758
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: https://review.gluster.org/14763
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Problem: For all the tests using get_pending_heal_count, EXPECT_WITHIN
is taking full time given to it even if the heal count matches with
expected value.
Solution:
RC - In most of the tests, to check heal count, wildcards are
being used. In EXPECT_WITHIN, in if condition, when we use it in
double quotes (" "), it gives string with wildcards which does not
match with the output of get_pending_heal_count.
For example, (0 =~ ^0$).
So, "while" loop was running for full time and at the end, after
coming out of loop, in next if condition it was matching with the
expression without quotes. That is why it was passing.
Remove double quotes in "if condition" in EXPECT_WITHIN and match
as we are matching it in test_expect_footer.
Change-Id: Ia161594774d05b9b888efb2f7ed1950590d8ac1b
BUG: 1412549
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: https://review.gluster.org/16382
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Commit c916a2f added a validation to restrict add-brick operation if a
replica configuration is changed and any of the bricks belonging to the
volume is down. However we should bypass this validation with a force
option if users really want to have add-brick to go through at the sake
of the corner cases of data loss issue.
The original problem of add-brick getting failed when layout is not set
will still be a problem with a force option as the issue has to be taken
care in the DHT layer.
Change-Id: I0ed3df91ea712f77674eb8afc6fdfa577f25a7bb
BUG: 1406411
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/16358
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Problem:
In CentOS-7, the file was receving an extra removexattr(security.ima)
FOP which changed its ctime, breaking the assumption that a particular brick
had the latest ctime based on the writevs done in the .t
Fix:
1. Compare the ctime of both files in the backend and pick the one with
the latest ctime for the fav-child policy. Also unmount the volume
before comparing, to avoid any further FOPS on the file that
can possibly modify the timestamps.
2. Added floating point handling in stat function. Thanks to Pranith for
the helping debugging the regex.
Change-Id: I06041a0f39a29d2593b867af8685d65c7cd99150
BUG: 1408757
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/16288
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
PROBLEM:
Consider a volume with granular-entry-heal and sharding enabled. When
a replica is down and a shard is created as part of a write, the name
index is correctly created under indices/entry-changes/<dot-shard-gfid>.
Now when a read on the same region triggers another MKNOD, the fop
fails on the online bricks with EEXIST. By virtue of this being a
symmetric error, the failed_subvols[] array is reset to all zeroes.
Because of this, before post-op, the GF_XATTROP_ENTRY_OUT_KEY will be
set, causing the name index, which was created in the previous MKNOD
operation, to be wrongly deleted in THIS MKNOD operation.
FIX:
The ideal fix would have been for a transaction to delete the name
index ONLY if it knows it is the one that created the index in the first
place. This would involve gathering information as to whether THIS xattrop
created the index from individual bricks, aggregating their responses and
based on the various posisble combinations of responses, decide whether to
delete the index or not. This is rather complex. Simpler fix would be
for post-op to examine local->op_ret in the event of no failed_subvols
to figure out whether to delete the name index or not. This can occasionally
lead to creation of stale name indices but they won't be affecting the IO path
or mess with pending changelogs in any way and self-heal in its crawl of
"entry-changes" directory would take care to delete such indices.
Change-Id: Ic1b5257f4dc9c20cb740a866b9598cf785a1affa
BUG: 1408712
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/16286
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Problems:
1) flush-behind is on by default, so just because write completes doesn't mean
it will be on the disk, it could still be in write-behind's cache. This
leads to failure where if you write from one mount and expect it to be there
on the other mount, sometimes it won't be there.
2) Sometimes the graph switch is not completing by the time we issue read which
is leading to opens not being sent on brick leading to failures.
Fixes:
1) Disable flush-behind
2) Add new functions to check the new graph is there and connected to bricks
before 'cat' is executed.
BUG: 1379511
Change-Id: I0faed684e0dc70cfd2258ce6fdaed655ee915ae6
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/15575
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Currently, md-cache only processes IATT_UPDATE_FLAGS, UP_XATTR and
UP_XATTR_RM. We also need to process UP_RENAME_FLAGS, UP_FORGET,
UP_PARENT_DENTRY_FLAGS and UP_NLINK_FLAGS. Otherwise the files
unlinked or renamed will not be reflected on other mounts.
Change-Id: Icb8b03da51482c3fc2e2a7292d16d56e11a341d9
BUG: 1211863
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/15324
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
This is a combination of three previous low-impact changes, combined to
reduce patch-pushing burden.
((( GF_INTERACTIVE )))
To use this, just define GF_INTERACTIVE (value doesn't matter as long as
the length is non-zero) before running your test. It replaces the TEST
alias with one that will prompt you before executing that line. You can
answer:
'y' to execute the line
'q' to exit the test immediately
anything else to skip this line and continue
This is particularly useful to inspect state in another window while a
test is paused, or to do manual experimentation in the (often complex)
configuration created during a test.
((( CLEANUP.SH )))
tests: add cleanup.sh
Often, a developer might want to run a test up to some point, then bail
out and poke around manually. That leaves state that needs to be
cleaned up before the next test can run properly. This patch adds a
trivial script to invoke that cleanup machinery.
Along the way, code in include.rc to find env.rc was changed to be more
robust across arbitrarily deep (or shallow) directory hierarchies.
((( REPLACE EXISTING TAR FILES INSTEAD OF APPENDING )))
We currently use "tar rf" to collect log files from each test. This
*appends* the new data to whatever's there already, which has two bad
effects when a test is run repeatedly.
* Ever-increasing size of the tar file.
* Ever-increasing time to extract logs from the tar file, with each
copy completely overwriting any previous.
This doesn't seem to be a problem in our regression tests, because the
entire directory is nuked during package removal and reinstallation.
However, when running a test repeatedly during a debug session, the
effects can be quite severe. This is particularly evident with JBR,
because the "logs" that get archived include large journal files.
Certain other translators, such as changelog and CTR, might be prone to
similar effects.
There's no point to having multiple copies of the logs in each tar file.
As far as I know, nobody ever takes advantage of that. Therefore, use
"tar cf" to overwrite any existing archive instead of appending. This
change also handles excluding other .tar files in a portable way.
Change-Id: Iebf77d496a71976c321bbacb49776338a9da586f
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/14874
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Commit 23ccabbeb7 introduced a new key "disperse.eager-lock" which
causes a conflict with key "cluster.eager-lock" when option is used
without the qualifying namespace. group-virt.example which gets
installed as /var/lib/glusterd/ groups/virt contains options without
namespace qualifiers. This patch adds the appropriate namespace to all
options in group-virt.example.
Change-Id: I2c09dd10d44138410d889ddeb805f01c641c6780
BUG: 1314649
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/13929
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
If 'a' and 'b' are hardlinks, we need to generate a virtual
gfid for these files so that the inode number for 'a' and 'b'
are same.
Generate gfid as below:
gfid_of_a = MD5(snapname + back_end_gfid(a))
if '/dir1/a' and '/dir2/b' are hardlinks, then inode number should be
same for
all below files:
/mnt/.snaps/snap1/dir1/a
/mnt/.snaps/snap1/dir2/b
/mnt/dir1/.snaps/snap1/a
/mnt/dir2/.snaps/snap1/b
Change-Id: Ifda793455610e554f3f1e4cbb90d44c02cda4b0f
BUG: 1171703
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9255
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Requirements:
Should be able to skip tests from run-tests.sh run.
Should be granular enough to disable on subset of OSes.
Solution:
Tests can have special comment lines with some comma separated values
within them.
Key names used to determine test status are
G_TESTDEF_TEST_STATUS_CENTOS6
G_TESTDEF_TEST_STATUS_NETBSD7
Some examples:
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
You can change status of test to enabled or delete the line only if all the
bugs are closed or modified or if the patch fixes it.
Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
BUG: 1295704
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/13393
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
NSR needs logging that is different than our existing changelog in
several ways:
* Full data, not just metadata
* Pre-op, not post-op
* High performance
* Supports the concept of time-bounded "terms"
Others (for example EC) might need the same thing. This patch adds such
a translator. It also adds code to dump the resulting journals, and to replay
them using syncops, plus (very rudimentary) tests for all of the above.
Change-Id: I29680a1b4e0a9e7d5a8497fef302c46434b86636
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/12450
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
When a loopback device is configured and we forcibly unmount the
filesystem containing the backing store, further vnconfig -l will
complain "vnconfig: VNDIOCGET: Bad file descriptor" causing
failures.
We fix this by iterating on all loopback devices available in /dev,
testing for this condition and manually unconfiguring when it
happens.
BUG: 1129939
Change-Id: I17b956a8ed28a7767f2d0dda83b93c523d3238c2
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/13204
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Sending a SIGTERM to test processes and waiting
a second for them to gracefully exit before
sending a SIGKILL seems like a waste of time.
Just send SIGKILL directly.
Change-Id: Icc73b07eae47876ba41955793a8daf77a964a0e0
BUG: 1294826
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/13121
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Problem: For EC volume, If a file descriptor is open and
file has been unlinked, any further write on that fd will
fail. When a write request comes, EC internally reads some
blocks using anonymous fd. This read will fail as the file
has already been unlinked.
Solution: To solve this issue, we are using .unlink directory
to keep track of unlinked file. If a file is to be unlinked
while its fd is open, move this to .unlink directory and unlink
it from .glusterfs and real path. Once all the fd will be closed,
remove this entry form .unlink directory.
Change-Id: I8344edb0d340bdb883dc46458c16edbc336916b9
BUG: 1286029
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/12816
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
This is one more attempt to eliminate corruption
of files by our tests scripts on NetBSD.
Changes done:
1. Have every local variable with a unique name.
2. Change date format to match with gluster's.
3. Pass the parameters to G_LOG without interpretation,
hence the change from $* to $@.
Change-Id: I833a93555da93179a1b39a9e4e7086216c335c3d
BUG: 1251592
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/11993
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
problem:
when executing testcases manually, some time we may want to terminate the
testcase execution in between due to various reasons.
Existing testcase flow has no mechanism to call cleanup before they terminate
abnormally, hence we endup with volume setups and mount points uncleaned.
Solution:
This patch traps such kind of abnormal terminations and calls 'cleanup'
function soon after they are caught and then terminates the testcases
with appropriate status..
$ ./tests/basic/mount-nfs-auth.t
1..87
=========================
TEST 1 (line 8): glusterd
ok 1
RESULT 1: 0
=========================
TEST 2 (line 9): pidof glusterd
ok 2
RESULT 2: 0
=========================
TEST 3 (line 10): gluster -mode=script --wignore volume info
No volumes present
ok 3
RESULT 3: 0
^C
received external signal --INT--, calling 'cleanup' ...
$ glusterd && gluster vol status
No volumes present
Change-Id: Ia51a850c356e599b8b789cec22b9bb5e87e1548a
BUG: 1252374
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/11882
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
This is a new simple regression test suite for
geo-replication. This is written keeping in mind
the run time for regression test. The existing
regression test suite is rigorous one and could
be run nightly. Hence the existing geo-rep tests
are being removed as part of this.
Also re-enable geo-rep regression with this patch.
Thanks Aravinda for initial template and plan.
Change-Id: If544ac295eaf67ac66e0b071903cc1096e71d437
BUG: 1227624
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11058
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Appends all commands being run under the test framework into the logs
with time stamps. Its a hack but I find it very useful to see what
sections of the logs (across all logs) need to be analysed between
execution of 2 commands.
Here's a sample output:
[2015-06-26 13:25:15.866764479]:++++++++++ G_LOG:tests/basic/afr/quorum.t: TEST: 46 ! test_write ++++++++++
[2015-06-26 13:25:15.872002] I [afr-common.c:1682:afr_local_discovery_cbk] 0-patchy-replicate-0: selecting local read_child patchy-client-1
[2015-06-26 13:25:15.874559] W [fuse-bridge.c:723:fuse_truncate_cbk] 0-glusterfs-fuse: 81: TRUNCATE() /a => -1 (Read-only file system)
[2015-06-26 13:25:15.880554623]:++++++++++ G_LOG:tests/basic/afr/quorum.t: TEST: 47 abc cat /mnt/glusterfs/0/b ++++++++++
[2015-06-26 13:25:15.897767878]:++++++++++ G_LOG:tests/basic/afr/quorum.t: TEST: 48 gluster --mode=script --wignore volume set patchy cluster.quorum-reads on ++++++++++[2015-06-26 13:25:15.994410] I [glusterfsd-mgmt.c:51:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-06-26 13:25:17.098519] I [glusterfsd-mgmt.c:51:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-06-26 13:25:17.099241] I [glusterfsd-mgmt.c:51:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-06-26 13:25:17.099685] I [glusterfsd-mgmt.c:51:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-06-26 13:25:17.100055] I [glusterfsd-mgmt.c:51:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-06-26 13:25:17.105896] W [MSGID: 108003] [afr.c:94:fix_quorum_options] 0-patchy-replicate-0: quorum-type auto overriding quorum-count 2
[2015-06-26 13:25:17.105936] W [MSGID: 108001] [afr.c:189:reconfigure] 0-patchy-replicate-0: Client-quorum is not met
[2015-06-26 13:25:17.107438] I [glusterfsd-mgmt.c:1507:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-06-26 13:25:17.108724] I [glusterfsd-mgmt.c:1507:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-06-26 13:25:17.110082] I [glusterfsd-mgmt.c:1507:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-06-26 13:25:17.110599] I [glusterfsd-mgmt.c:1507:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-06-26 13:25:17.109678070]:++++++++++ G_LOG:tests/basic/afr/quorum.t: TEST: 49 1 mount_get_option_value /mnt/glusterfs/0 patchy-replicate-0 quorum-reads ++++++++++
[2015-06-26 13:25:17.117801] I [afr-common.c:1682:afr_local_discovery_cbk] 0-patchy-replicate-0: selecting local read_child patchy-client-1
Change-Id: Ib51284a0384508350579babaf1ae69cb372e0baa
BUG: 1233018
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/10667
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
1) Avoid hangs on unmounting NFS on NetBSD
NetBSD umount(8) on a NFS mount whose server is gone will wait forever
because umount(8) calls realpath(3) and tries to access the mount before
it calls unmount(2). The non-portable, NetBSD-specific umount -R flag
prevent that behavior.
We therefore introduce UMOUNT_F, defined as "umount -f" on Linux and
"umount -f -R" on NetBSD to take care of forced unmounts, especially
in the NFS case.
2) Enforce usage of force_umount wrapper with timeout
Whenever umount is used it should be wrapped in force_umount with
tiemout handling. That saves us timing issues, and it handles the
NetBSD NFS case.
3) Cleanup kernel cache flush.
We used (cd $M0 && umount $M0 ) as a portable kernel cache flush
trick, but it does not flush everything we need on Linux. Introduce
a drop_cache() shell function that reverts to previously used
echo 3 > /proc/sys/vm/drop_caches on Linux, and keeps
(cd $M0 && umount $M0 ) on other systems.
BUG: 1129939
Change-Id: Iab1f5a023405f1f7270c42b595573702ca1eb6f3
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/11114
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
There is (or was) a spurious test in glupy.t for which a separate log
file was setup. The directory where the log is saved, is not available
on NetBSD and this causes glupy.t to always fail, without a log.
Instead of hard-coding the path to the log, use "gluster --print-logdir"
to provide a LOGDIR environment variable. glupy.t now writes the log to
an existing directory.
BUG: 1163543
Change-Id: Ifa73198d06fa267856d0da9d25a4380329909124
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10801
Tested-by: NetBSD Build System
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Our failure to unmount meant that both mkdir and rmdir would fail in
cleanup(). Because one of those mkdirs was the last thing cleanup()
executed, it would fail, so the test would fail, so the entire
regression run would fail. The fix has two parts.
(1) Unmount the offending directory.
(2) Make sure cleanup() returns success even if that last mkdir
failed.
That might keep us from consistently blowing up regression runs on the
very first tests (basic/afr/data-self-heal.t) that we execute.
Change-Id: I7a9761bd28761a5ee2face3db8112e9c3f6c5dc8
BUG: 1163543
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/10536
Reviewed-by: Justin Clift <justin@gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
In performance/open-behind.t, a test for open fop reaching the brick is done by
switching off open-behind and performing a read operation. If the read operation
is performed before a graph switch, the read happens on the old graph and hence
open does not get accounted in the brick.
To overcome this EXPECT_WITHIN 10 seconds has now been added to ensure that a
graph switch has happened. The read operation happens subsequently after the
graph switch.
Cleaned up a "No volumes present" message from stderr while doing this.
Change-Id: I1e1c0d7e4bd2057520b4dd46157d18f30837b8c9
BUG: 1213066
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/10293
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Raghavendra Bhat <raghavendra@redhat.com>
