1
0
mirror of https://github.com/gluster/glusterfs.git synced 2026-02-06 09:48:44 +01:00
Commit Graph

970 Commits

Author SHA1 Message Date
XiJinyu
0f2b25ccd4 nit: the "data" was used twice. (#2858)
dict_t *dict = data;
dict = data;

Change-Id: I578ce05dd5a18b62eac6632778e94044c028951b
Updates: #2857
Signed-off-by: Xi Jinyu <xijinyu_yewu@cmss.chinamobile.com>
2021-10-11 23:14:41 +05:30
Dmitry Antipov
b48e64cebc cli: drop global RPC client contexts (#2767)
Embed glusterd and quotad RPC client context pointers
into CLI state instead of using global variables, adjust
related initialization functions and comments.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-09-22 16:04:10 +05:30
Dmitry Antipov
db36432eec cli: fix a few memory use-after-free and leak errors (#2737)
Fix a few use-after-free error detected by ASan, for example:

==1089284==ERROR: AddressSanitizer: heap-use-after-free on address ...

WRITE of size 8 at 0x61100001ffd0 thread T2
    #0 0x7fa6ab385633 in dict_unref libglusterfs/src/dict.c:798
    #1 0x41664a in cli_cmd_volume_stop_cbk cli/src/cli-cmd-volume.c:564
    #2 0x40f2d6 in cli_cmd_process cli/src/cli-cmd.c:133
    #3 0x40e772 in cli_batch cli/src/input.c:29
    #4 0x7fa6aae3e298 in start_thread (/lib64/libpthread.so.0+0x9298)
    #5 0x7fa6aaa78352 in clone (/lib64/libc.so.6+0x100352)

0x61100001ffd0 is located 80 bytes inside of 224-byte region ...
freed by thread T2 here:
    #0 0x7fa6ab732647 in free (/lib64/libasan.so.6+0xae647)
    #1 0x7fa6ab43171a in __gf_free libglusterfs/src/mem-pool.c:362
    #2 0x7fa6ab38559b in dict_destroy libglusterfs/src/dict.c:782
    #3 0x7fa6ab38565e in dict_unref libglusterfs/src/dict.c:801
    #4 0x40cd28 in cli_local_wipe cli/src/cli.c:783
    #5 0x4165c4 in cli_cmd_volume_stop_cbk cli/src/cli-cmd-volume.c:562
    #6 0x40f2d6 in cli_cmd_process cli/src/cli-cmd.c:133
    #7 0x40e772 in cli_batch cli/src/input.c:29
    #8 0x7fa6aae3e298 in start_thread (/lib64/libpthread.so.0+0x9298)

previously allocated by thread T2 here:
    #0 0x7fa6ab732af7 in calloc (/lib64/libasan.so.6+0xaeaf7)
    #1 0x7fa6ab430a91 in __gf_calloc libglusterfs/src/mem-pool.c:151
    #2 0x7fa6ab382562 in get_new_dict_full libglusterfs/src/dict.c:84
    #3 0x7fa6ab38268d in dict_new libglusterfs/src/dict.c:127
    #4 0x415fcf in cli_cmd_volume_stop_cbk cli/src/cli-cmd-volume.c:501
    #5 0x40f2d6 in cli_cmd_process cli/src/cli-cmd.c:133
    #6 0x40e772 in cli_batch cli/src/input.c:29
    #7 0x7fa6aae3e298 in start_thread (/lib64/libpthread.so.0+0x9298)

Thread T2 created by T0 here:
    #0 0x7fa6ab6da8d6 in pthread_create (/lib64/libasan.so.6+0x568d6)
    #1 0x40eb2b in cli_input_init cli/src/input.c:76
    #2 0x40d0ee in main cli/src/cli.c:863
    #3 0x7fa6aa99fb74 in __libc_start_main (/lib64/libc.so.6+0x27b74)

Also tweak CLI_LOCAL_INIT() to take an extra reference to related
'dict_t' object and ensure correct reference counting sequence for
the most of CLI commands, which should follow the scheme below:

int
cli_cmd_xxx(...)
{
    int ret = -1;
    dict_t *options = NULL;
    ...
    ret = cli_cmd_xxx_parse(..., &options);      /* refcount 1 */
    if (!ret)
        goto out;
    ...
    CLI_LOCAL_INIT(..., options);                /* refcount 2 */
    ...
    ret = proc->fn(..., options);
    ...
out:
    CLI_STACK_DESTROY(...);                      /* refcount 1 */
    if (options)
       dict_unref(options);         /* freed due to refcount 0 */
    return ret;
}

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-09-14 11:55:59 +05:30
mohit84
2936241ea9 Revert "libglusterfs: drop core infrastructure of memory pools (#2675)" (#2777)
In case of glibc_pool it is good to use glusterfs mempool so revert
the commit 9cd6735ff5.

Change-Id: I780f0a1b7dae815becfd8c072735b6fdecb936f8
Updates: #1000
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2021-09-10 07:26:04 +05:30
Tamar Shacked
76fb94f505 cli: fix coverity issue (#2766)
fixing null dereference reported by coverity

CID: 1462029
Updates: #1060

Change-Id: I2c42a677b02ea23da4d18d4c662a020f40081e03
Signed-off-by: Tamar Shacked <tshacked@redhat.com>
2021-09-09 17:35:13 +03:00
Dmitry Antipov
77b9bf1034 cli: drop global state variable (#2753)
Drop 'global_state' and pass CLI state as a parameter
or indirectly via call frame or THIS where appropriate.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-09-04 13:11:35 +05:30
Dmitry Antipov
368d5b5891 glusterd, cli: more time_t usage where appropriate (#2749)
More time_t values for time intervals and timeouts,
drop weird is_valid_int() since strtol() is capable
enough to handle errors, adjust relevant comments.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-09-04 13:08:58 +05:30
Dmitry Antipov
9cd6735ff5 libglusterfs: drop core infrastructure of memory pools (#2675)
Separate memory allocation type tracking from core infrastructure of
memory pools and drop the latter alongside with '--disable-mempool'
configuration option, provide '--disable-allocation-tracking' to control
the former instead.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-09-04 12:02:48 +05:30
Dmitry Antipov
21f1d246a9 build: use pkg-config to check whether readline is available
Use convenient PKG_CHECK_MODULES macro to check whether readline
library is present, drop specific check for rl_do_undo() since
it is available since ancient readiline 2.0 or maybe even older.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-09-02 14:08:27 +02:00
Dmitry Antipov
35039253ba libglusterfs, cli, rpc, xlators: consistently use time_t (#2710)
Tweak dict and xlator interfaces to allow explicit values of 'time_t' type
and consistently use it for timeouts, time intervals, and whatever similar.
Adjust related tests to prefer 0 over -1 for disabled timeouts.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-08-30 17:10:20 +05:30
Dmitry Antipov
bcf09403ed cli: do not allow to use help as the volume name (#2721)
Add 'help' to the list of unacceptable volume names because using
it as such silently confuses the most of CLI commands, for example:

$ gluster volume info

Volume Name: abcd
Type: Replicate
Volume ID: db862536-61da-404b-98c9-b5450eb1f80d
Status: Created
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.110:/home/pool/3
Brick2: 192.168.222.110:/home/pool/4
Brick3: 192.168.222.110:/home/pool/5
Options Reconfigured:
cluster.granular-entry-heal: on
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

Volume Name: help
Type: Replicate
Volume ID: 10442b69-18fa-4ad8-8dc9-3867c0598523
Status: Created
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 192.168.222.110:/home/pool/0
Brick2: 192.168.222.110:/home/pool/1
Brick3: 192.168.222.110:/home/pool/2
Options Reconfigured:
cluster.granular-entry-heal: on
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

$ gluster volume set abcd performance.cache-size 256MB

volume set: success

$ gluster volume set help performance.cache-size 256MB
volume set: failed

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-08-20 19:23:52 +05:30
Dmitry Antipov
329491acae xlators, cli: prefer timespec over timeval (#2458)
Prefer timespec_now_realtime() over gettimeofday() to simplify and
avoid extra multiply by 1000.

Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
2021-05-23 06:47:18 +05:30
Dmitry Antipov
9c7e6594ae libglusterfs: prefer mallinfo2() to mallinfo() if available
Since glibc 2.33, mallinfo() is marked obsolete due to the limited
range of 'int' type used in 'struct mallinfo', and new 'size_t'-based
counterpart 'struct mallinfo2' with corresponding function mallinfo2()
should be preferred instead. To avoid compatibility mess, it's also
reasonable to use 'uint64_t' values for storing and operating with
mallinfo data internally, as suggested by Xavi Hernandez.

Fixes: #2414
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
2021-05-14 11:20:51 +02:00
nik-redhat
e210d5c61a cli-line-coverage: Added test for negative case, removed stale code
Fix:
Added new .t file for increasing the line,function coverage
in cli component, and removed/commented out stale code.

Updates: #1052

Change-Id: I0a3c12a12f18bee39824157129c1d0187f4030d4
Signed-off-by: nik-redhat <nladha@redhat.com>
2021-05-13 10:47:15 +02:00
Ayush Ujjwal
b4314878f9 Coverity : Use after free
CID: 1430111

Description:
The previous condition `if(tmp)` was freeing  `tmp` always.
Hence,  replaced the condition with a new condition which checks if `rv >= 0`.

Updates: #1060

Change-Id: I0ebe18d1978b73b577ded259915b7305a58253ff
Signed-off-by: Ayush Ujjwal <aujjwal@redhat.com>
2021-05-06 11:55:51 +02:00
Karthik Subrahmanya
803d53103f cluster/afr: Code coverage improvement for afr-self-heald.c & cli-rpc-ops.c (#2366)
Problem 1:
The code to handle the volume heal split-brain was there in both
glfs-heal.c and afr-self-heald.c. The code path in the later file
was never hit since it will always be handled by self-heal.c.

Fix:
Removed the code to handle heal info split-brain request from
afr-self-heald.c and cli-rpc-ops.c files.

Problem 2:
The volume heal statistics was never called in any of the test cases.

Fix:
Calling volume heal statistics in afr-heal-info.t which will increase
the code coverage of both afr-self-heald.c an cli-rpc-ops.c.

Change-Id: I242ef088bb2da9c8d9a200986514758cb0f08b37
Signed-off-by: karthik-us <ksubrahm@redhat.com>
Updates: #1000
2021-05-03 16:34:44 +05:30
Nikhil Ladha
4336edecaf cli: Increased spacing in cli for option table (#2322)
* cli: Increased spacing in cli for option table

Issue:
Some options have name larger than length 40,
due to which the output of command `gluster vol get <volname> all`
mixes up the option, value for long option names.

Fix:
Increased the spacing in cli for `gluster vol get <volname> all`
output to 50.

Fixes: #2313

Change-Id: I841730ced074547a81171a4432d15ec9c35f39cd
Signed-off-by: nik-redhat <nladha@redhat.com>

* Added separator

Change-Id: I210877c89bc468ed6a3090cd14fde7ecee1d33b6
Signed-off-by: nik-redhat <nladha@redhat.com>

* Removed separator and added space

Change-Id: Ic0eb9c9bc39a354465aabd939f72bc65be738f6c
Signed-off-by: nik-redhat <nladha@redhat.com>
2021-04-13 17:55:12 +05:30
nishith-vihar
5fd3d4175f Removal of force option in snapshot create (#2110)
The force option does fail for snapshot create command even though
the quorum is satisfied and is redundant.

The change deprecates the force option for snapshot create command
and checks if all bricks are online instead of checking for quorum
for creating a snapshot.

Fixes: #2099
Change-Id: I45d866e67052fef982a60aebe8dec069e78015bd
Signed-off-by: Nishith Vihar Sakinala <nsakinal@redhat.com>
2021-04-06 16:10:50 +02:00
Pranith Kumar Karampuri
ec189a499d cluster/dht: use readdir for fix-layout in rebalance (#2243)
Problem:
On a cluster with 15 million files, when fix-layout was started, it was
not progressing at all. So we tried to do a os.walk() + os.stat() on the
backend filesystem directly. It took 2.5 days. We removed os.stat() and
re-ran it on another brick with similar data-set. It took 15 minutes. We
realized that readdirp is extremely costly compared to readdir if the
stat is not useful. fix-layout operation only needs to know that the
entry is a directory so that fix-layout operation can be triggered on
it. Most of the modern filesystems provide this information in readdir
operation. We don't need readdirp i.e. readdir+stat.

Fix:
Use readdir operation in fix-layout. Do readdir+stat/lookup for
filesystems that don't provide d_type in readdir operation.

fixes: #2241
Change-Id: I5fe2ecea25a399ad58e31a2e322caf69fc7f49eb
Signed-off-by: Pranith Kumar K <pranith.karampuri@phonepe.com>
2021-03-22 10:19:27 +05:30
Ravishankar N
f41f7dec05 cli: syntax check for arbiter volume creation (#2207)
commit 8e7bfd6a58 changed the syntax for
arbiter volume creation to 'replica 2 arbiter 1', while still allowing
the old syntax of 'replica 3 arbiter 1'. But while doing so, it also
removed a conditional check, thereby allowing replica count > 3. This
patch fixes it.

Fixes: #2192
Change-Id: Ie109325adb6d78e287e658fd5f59c26ad002e2d3
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
2021-03-05 11:54:46 +05:30
Ryo Furuhashi
ea86b664f3 glusterd-volgen: Add functionality to accept any custom xlator (#1974)
* glusterd-volgen: Add functionality to accept any custom xlator

Add new function which allow users to insert any custom xlators.
It makes to provide a way to add any processing into file operations.

Users can deploy the plugin(xlator shared object) and integrate it to glusterfsd.

If users want to enable a custom xlator, do the follows:

1. put xlator object(.so file) into "XLATOR_DIR/user/"
2. set the option user.xlator.<xlator> to the existing xlator-name to specify of the position in graph
3. restart gluster volume

Options for custom xlator are able to set in "user.xlator.<xlator>.<optkey>".

Fixes: #1943
Signed-off-by:Ryo Furuhashi <ryo.furuhashi.nh@hitachi.com>
Co-authored-by: Yaniv Kaul <ykaul@redhat.com>
Co-authored-by: Xavi Hernandez <xhernandez@users.noreply.github.com>
2021-02-05 09:26:03 +05:30
mohit84
90f18d12f5 tests: ./tests/bugs/replicate/bug-921231.t is continuously failing (#2006)
The test case (./tests/bugs/replicate/bug-921231.t )
is continuously failing.The test case is failing because
inodelk_max_latency is showing wrong value in profile.
The value is not correct because recently the profile
timestamp is changed from microsec to nanosec from
the patch #1833.

Fixes: #2005
Change-Id: Ieb683836938d986b56f70b2380103efe95657821
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2021-01-13 17:58:28 +05:30
Sheetal Pamecha
1d74669ab9 stripe cleanup: Remove the option from create and add-brick cmds (#1812)
* stripe cleanup: Remove the option from create and add-brick cmds

This patch aims to remove the code for stripe option instead
of keeping a default values of stripe/stripe-count variables and
setting and getting dict options and similar redundant operations.
Also removing tests for stripe volumes that have been already
marked bad.

Updates: #1000
Change-Id: Ic2b3cabd671f0c8dc0521384b164c3078f7ca7c6
Signed-off-by: Sheetal Pamecha <spamecha@redhat.com>

* Fix regression error

tests/000-flaky/basic_changelog_changelog-snapshot.t
was failing due to 0 return value

Change-Id: I8ea0443669c63768760526db5aa1f205978e1dbb
Signed-off-by: Sheetal Pamecha <spamecha@redhat.com>

* add constant stripe_count value for upgrade scenerios

Change-Id: I49f3da4f106c55f9da20d0b0a299275a19daf4ba

* Fix clang-format warning

Change-Id: I83bae85d10c8c5b3c66f56c9f8de1ec81d0bbc95
2021-01-05 20:00:32 +05:30
Mohammed Ashour
bba4e92de9 Fixing the split-brain link where going for a 2 replicated volume. (#1964)
Fixes in functions:
	- cli_cmd_volume_create_parse()
	- cli_cmd_volume_add_brick_parse()
	- cli_cmd_volume_remove_brick_parse()

Fixes: #1959
Signed-off-by: Mohammed Ali Ashour <m.aly.ashour@gmail.com>
2020-12-31 09:45:47 +05:30
Ravishankar N
aa29aaf1c9 cli/glusterd: conscious language changes for geo-rep
Replace master and slave terminology in geo-replication with primary and
secondary respectively.

All instances are replaced in cli and glusterd.

Changes to other parts of the code to follow in separate patches.
tests/00-geo-rep/* are passing thus far.

Updates: #1415
Change-Id: Ifb12b7f5ce927a4a61bda1e953c1eb0fdfc8a7c5
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
2020-12-30 15:55:22 +05:30
Tamar Shacked
ae8cfe5baa glusterd/cli: enhance rebalance-status after replace/reset-brick (#1869)
* glusterd/cli: enhance rebalance-status after replace/reset-brick

Rebalance status is being reset during replace/reset-brick operations.
This cause 'volume status' to shows rebalance as "not started".

Fix:
change rebalance-status to "reset due to (replace|reset)-brick"

Change-Id: I6e3372d67355eb76c5965984a23f073289d4ff23
Signed-off-by: Tamar Shacked <tshacked@redhat.com>

* glusterd/cli: enhance rebalance-status after replace/reset-brick

Rebalance status is being reset during replace/reset-brick operations.
This cause 'volume status' to shows rebalance as "not started".

Fix: change rebalance-status to "reset due to (replace|reset)-brick"

Fixes: #1717
Signed-off-by: Tamar Shacked <tshacked@redhat.com>

Change-Id: I1e3e373ca3b2007b5b7005b6c757fb43801fde33

* cli: changing rebal task ID to "None" in case status is being reset

Rebalance status is being reset during replace/reset-brick operations.
This cause 'volume status' to shows rebalance as "not started".

Fix:
change rebalance-status to "reset due to (replace|reset)-brick"

Fixes: #1717

Change-Id: Ia73a8bea3dcd8e51acf4faa6434c3cb0d09856d0
Signed-off-by: Tamar Shacked <tshacked@redhat.com>
2020-12-08 16:21:35 +05:30
nik-redhat
680e1bbfbc CLI: added an error message in case of timeout
Added an error message in CLI when there are volumes present in cluster
but timeout happens on fetching them.

This PR fixes #1738

Signed-off-by: nik-redhat <nladha@redhat.com>
2020-11-05 09:57:16 +01:00
schaffung
3b6be62cf3 cli-rpc: conditional init of global quota rpc (#1578)
Issue: It is seem that the initialization of rpc to
connect with quotad is done in every glusterfs cli command,
irrespective of whether the quota feature is enabled or disabled.
This seems to be an overkill.

Code change: The file /var/run/quotad/quotad.pid is present
signals that quotad is enabled. Hence we can put a conditional
check for seeing when this file exists and if it doesn't we
just skip over the initialization of the global quotad rpc.

This will go on to reduce the extra rpc calls and operations
being performed in the kernel space.

Fixes: #1577
Change-Id: Icb69d35330f76ce95626f59af75a12726eb620ff
Signed-off-by: srijan-sivakumar <ssivakumar@redhat.com>

Co-authored-by: srijan-sivakumar <ssivakumar@redhat.com>
2020-11-02 11:18:01 +05:30
Kaleb S. KEITHLEY
0f9797a931 cli: type mismatch global_quotad_rpc in cli-quotad-client.c
...
[  236s] (.text+0x0): multiple definition of `cli_default_conn_timeout'; cli.o (symbol from plugin):(.text+0x0): first defined here
[  236s] cli-quotad-client.c:13:24: warning: type of 'global_quotad_rpc' does not match original declaration [-Wlto-type-mismatch]
[  236s]    13 | extern struct rpc_clnt global_quotad_rpc;
[  236s]       |                        ^
[  236s] cli.c:80:18: note: 'global_quotad_rpc' was previously declared here
[  236s]    80 | struct rpc_clnt *global_quotad_rpc;
[  236s]       |                  ^
[  236s] cli.c:80:18: note: code may be misoptimized unless '-fno-strict-aliasing' is used
[  239s] collect2: error: ld returned 1 exit status
...

move several global extern decls to cli.h.

Found on SUSE Tumbleweed w/ gcc-10 and LTO.  This bug goes back
at least as far as release-6. Curiously this only popped up as an
error when building 6.9, not 7.4 or 8.0. And not when building
8.0 on Fedora rawhide/33.

Change-Id: I765642cea77b524a36368a7ff1a2976ef049dd0f
Fixes: #1356
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2020-07-06 16:19:08 -04:00
Dmitry Antipov
8fe08736f1 cli: fix data race when handling connection status
Found with GCC ThreadSanitizer:

WARNING: ThreadSanitizer: data race (pid=287943)
  Write of size 4 at 0x00000047dfa0 by thread T4:
    #0 cli_rpc_notify /path/to/glusterfs/cli/src/cli.c:313 (gluster+0x40a6df)
    #1 rpc_clnt_handle_disconnect /path/to/glusterfs/rpc/rpc-lib/src/rpc-clnt.c:821 (libgfrpc.so.0+0x13f04)
    #2 rpc_clnt_notify /path/to/glusterfs/rpc/rpc-lib/src/rpc-clnt.c:882 (libgfrpc.so.0+0x13f04)
    #3 rpc_transport_notify /path/to/glusterfs/rpc/rpc-lib/src/rpc-transport.c:520 (libgfrpc.so.0+0xf070)
    #4 socket_event_poll_err /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:1364 (socket.so+0x812c)
    #5 socket_event_handler /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:2958 (socket.so+0xc453)
    #6 socket_event_handler /path/to/glusterfs/rpc/rpc-transport/socket/src/socket.c:2854 (socket.so+0xc453)
    #7 event_dispatch_epoll_handler /path/to/glusterfs/libglusterfs/src/event-epoll.c:640 (libglusterfs.so.0+0xcaf23)
    #8 event_dispatch_epoll_worker /path/to/glusterfs/libglusterfs/src/event-epoll.c:751 (libglusterfs.so.0+0xcaf23)
    #9 <null> <null> (libtsan.so.0+0x2d33f)

  Previous read of size 4 at 0x00000047dfa0 by thread T3 (mutexes: write M3587):
    #0 cli_cmd_await_connected /path/to/glusterfs/cli/src/cli-cmd.c:321 (gluster+0x40ca37)
    #1 cli_cmd_process /path/to/glusterfs/cli/src/cli-cmd.c:123 (gluster+0x40cc74)
    #2 cli_batch /path/to/glusterfs/cli/src/input.c:29 (gluster+0x40c2b9)
    #3 <null> <null> (libtsan.so.0+0x2d33f)

  Location is global 'connected' of size 4 at 0x00000047dfa0 (gluster+0x00000047dfa0)

Change-Id: Ie85a8a80a2c5b82252c0c1d45e68ebe9938da2eb
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Fixes: #1311
2020-06-16 14:45:03 +03:00
Csaba Henk
8cb23ee277 Indicate timezone offsets in timestamps
Logs and other output carrying timestamps
will have now timezone offsets indicated, eg.:

[2020-03-12 07:01:05.584482 +0000] I [MSGID: 106143] [glusterd-pmap.c:388:pmap_registry_remove] 0-pmap: removing brick (null) on port 49153

To this end,

- gf_time_fmt() now inserts timezone offset via %z strftime(3) template.
- A new utility function has been added, gf_time_fmt_tv(), that
  takes a struct timeval pointer (*tv) instead of a time_t value to
  specify the time. If tv->tv_usec is negative,

  gf_time_fmt_tv(... tv ...)

  is equivalent to

  gf_time_fmt(... tv->tv_sec ...)

  Otherwise it also inserts tv->tv_usec to the formatted string.
- Building timestamps of usec precision has been converted to
  gf_time_fmt_tv, which is necessary because the method of appending
  a period and the usec value to the end of the timestamp does not work
  if the timestamp has zone offset, but it's also beneficial in terms of
  eliminating repetition.
- The buffer passed to gf_time_fmt/gf_time_fmt_tv has been unified to
  be of GF_TIMESTR_SIZE size (256). We need slightly larger buffer space
  to accommodate the zone offset and it's preferable to use a buffer
  which is undisputedly large enough.

This change does *not* do the following:

- Retaining a method of timestamp creation without timezone offset.
  As to my understanding we don't need such backward compatibility
  as the code just emits timestamps to logs and other diagnostic
  texts, and doesn't do any later processing on them that would rely
  on their format. An exception to this, ie. a case where timestamp
  is built for internal use, is graph.c:fill_uuid(). As far as I can
  see, what matters in that case is the uniqueness of the produced
  string, not the format.
- Implementing a single-token (space free) timestamp format.
  While some timestamp formats used to be single-token, now all of
  them will include a space preceding the offset indicator. Again,
  I did not see a use case where this could be significant in terms
  of representation.
- Moving the codebase to a single unified timestamp format and
  dropping the fmt argument of gf_time_fmt/gf_time_fmt_tv.
  While the gf_timefmt_FT format is almost ubiquitous, there are
  a few cases where different formats are used. I'm not convinced
  there is any reason to not use gf_timefmt_FT in those cases too,
  but I did not want to make a decision in this regard.

Change-Id: I0af73ab5d490cca7ed8d07a2ce7ac22a6df2920a
Updates: #837
Signed-off-by: Csaba Henk <csaba@redhat.com>
2020-03-12 08:20:31 +01:00
Dmitry Antipov
58d0e43ee0 cli: fix several signed integer overflows and format specifiers
Initially found with GCC UBsan:

cli/src/cli-rpc-ops.c:5347:73: runtime error: left shift of 1 by 31
                               places cannot be represented in type 'int'
cli/src/cli-rpc-ops.c:5355:74: runtime error: left shift of 1 by 31
                               places cannot be represented in type 'int'

Ditto in cli/src/cli-xml-output.c.

Change-Id: I14ed51d06dafe5039f154b0c4edf25a0997d696e
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Fixes: #1279
2020-06-02 12:28:17 +03:00
Dmitry Antipov
7e0d3eb61a cli: fix memory leak in gf_cli_gsync_status_output()
In gf_cli_gsync_status_output(), call to gf_cli_read_status_data()
overwrites 'sts_vals' pointers to areas allocated by GF_CALLOC()
with pointers to dict data, thus making the allocated areas not
accessible.

Change-Id: I00c310aec1a1413caf13ade14dc4fed37b51962c
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Fixes: #1259
2020-05-22 11:27:29 +03:00
Dmitry Antipov
aca7961dae core, cli, quota: cleanup malloc debugging and stats
1. Since mcheck()/mprobe() etc. features are no longer used,
   mcheck.h isn't required to be included.

2. Since mallinfo() is used to obtain malloc statistics,
   it should be detected instead of malloc_stats().

Change-Id: I54c7d2ee568e06ab29938efc01d1a2153c5bd5db
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Fixes: #1172
2020-04-15 15:03:03 +03:00
karthik-us
df296f8f31 cluster/afr: Removing unsupported options from code base to improve coverage
Support for gluster volume heal <volname> info healed/heal-failed
was removed by commit bb02cfb56a in
release-3.6. cli parser will display the usage message in all the
supported versions whenever these clis are run, leading to some
dead code in the latest branches. Since support for these clis
were removed long back, this should not give any backward
compatibility issues as well. Hence removing the dead code from
the code base which will lead to better code coverage by the
regression runs as well.

Updates: #1052
Change-Id: I0c2b061469caf233c06d9699b0d159ce48e240b9
Signed-off-by: karthik-us <ksubrahm@redhat.com>
2020-04-02 13:03:35 +05:30
Sanju Rakonde
abd56b0fd4 cli: display the error while probing the localhost
With bc6e206c6, we regressed in displaying the error
message when a user tries to probe localhost. With
this change, we display "probe on localhost not needed"
message to the user.

credits: Sachin Prabhu <sprabhu@redhat.com>

fixes: bz#1810042

Change-Id: Ibf82b5a658c371c08290a0b4f655e5ac5f436c06
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
2020-03-10 15:05:08 +05:30
Kotresh HR
8c6406e46b bitrot: Make number of signer threads configurable
The number of signing process threads (glfs_brpobj)
is set to 4 by default. The recommendation is to set
it to number of cores available. This patch makes it
configurable as follows

gluster vol bitrot <volname> signer-threads <count>

fixes: bz#1797869
Change-Id: Ia883b3e5e34e0bc8d095243508d320c9c9c58adc
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2020-02-03 18:10:17 +05:30
Barak Sason Rofman
a248368e74 cli - removing fetch of unnecessary items.
Somem methods dict_get...(...) values and then not use them anywhere.
Removed found occurrences.

fixes: #753

Change-Id: Iaeb8f4cec18f76078f6b2f4e4bd6f9795a3467bc
Signed-off-by: Barak Sason Rofman <bsasonro@redhat.com>
2019-12-04 18:48:23 +02:00
Barak Sason Rofman
a4c627a643 cli - fixing a coverity issue
Removed unused variable.

fixes: CID#1412106
updates: bz#789278

Change-Id: I1d4e1c1625cecf882d51e9cf4f5290383f63d405
Signed-off-by: Barak Sason Rofman <bsasonro@redhat.com>
2020-01-05 12:11:07 +02:00
Kaleb S. KEITHLEY
a68516d55d cli: duplicate defns of cli_default_conn_timeout and cli_ten_minutes_timeout
Winter is coming. So is gcc-10.

Compiling with gcc-10-20191219 snapshot reveals dupe defns of
cli_default_conn_timeout and cli_ten_minutes_timeout in
.../cli/src/cli.[ch] due to missing extern decl.

There are many changes coming in gcc-10 described in
https://gcc.gnu.org/gcc-10/changes.html

compiling cli.c with gcc-9 we see:
   ...
        .quad   .LC88
        .comm   cli_ten_minutes_timeout,4,4
        .comm   cli_default_conn_timeout,4,4
        .text
   .Letext0:
   ...

and with gcc-10:
   ...
        .quad   .LC88
        .globl  cli_ten_minutes_timeout
        .bss
        .align 4
        .type   cli_ten_minutes_timeout, @object
        .size   cli_ten_minutes_timeout, 4
   cli_ten_minutes_timeout:
        .zero   4
        .globl  cli_default_conn_timeout
        .align 4
        .type   cli_default_conn_timeout, @object
        .size   cli_default_conn_timeout, 4
   cli_default_conn_timeout:
        .zero   4
        .text
   .Letext0:
   ...

which is reflected in the .o file as (gcc-9):
...
0000000000000004 C cli_ten_minutes_timeout
0000000000000004 C cli_default_conn_timeout
...

and (gcc-10):
...
0000000000000020 B cli_ten_minutes_timeout
0000000000000024 B cli_default_conn_timeout
...

See nm(1) and ld(1) for a description C (common) and B (BSS) and how
they are treated by the linker.

Note: there is still a small chance that gcc-10 will land in Fedora-32,
despite 31 Dec. 2019 having been the deadline for that to happen.

Change-Id: I54ea485736a4910254eeb21222ad263721cdef3c
Fixes: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2020-01-02 07:46:23 -05:00
Shwetha K Acharya
ee1a43824a geo-rep: Note section is required for ignore_deletes
There exists a window of 15 sec, where the deletes are picked up
by history crawl when the ignore_deletes is set to true.
And it eventually deletes the file/s from slave which is/are not
supposed to be deleted. Though it is working as per design, a
note regarding this is needed.

Added a warning message indicating the same.
Also logged info when the worker restarts after ignore-deletes
option set.

fixes: bz#1708603
Change-Id: I103be882fac18b4cef935efa355f5037a396f7c1
Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
2019-07-31 11:34:19 +05:30
Dmitry Antipov
67f54115e2 cli, libglusterfs: rely on libpthread static initializers
cli-cmd.c: drop cli_cmd_cond_init() because static mutex and
condition variable are initialized with PTHREAD_xxx_INITIALIZERs.

syncop-utils.c: since assignment PTHREAD_xxx_INITIALIZERs are
never failed, a few overengineered bits may be simplified.

Change-Id: Ic4d250a1697047386989f73f058b0abc8b55627b
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Updates: bz#1193929
2019-12-27 13:35:33 +03:00
Dmitry Antipov
63386b5e41 glfsheal: install as architecture-dependent binary helper
Since glfsheal is a binary helper which is not intented to be
invoked directly, install it under %{libexecdir}/glusterfs rather
than %{sbindir}, adjust invocation from CLI code and RPM spec.

Credits: Ravishankar N <ravishankar@redhat.com>.
Fixes: bz#1780190

Change-Id: I4b41892d96b89c24a332470ac8c1e82f6795159a
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
2019-12-05 16:57:28 +03:00
Yaniv Kaul
97d798869a cli-rpc-ops.c: cleanups
1. Move functions and structs to static
2. Use dictionary functions with fixed key length.
3. Reduce key length when not needed.
4. Use const char* for some messages.
5. Use fixed strings for some logs which is repeated in the code.
6. Remove redundant checks. Specifically, cli_to_glusterd() does
NULL checks already, so no need to do it before calling it.
7. Aligned some messages - not sure why they were cut over several
lines, but it made grep on the code harder.
8. Move dictionary fetching of values closer to where they are
actually used.

Overall, object size is ~4 smaller, hopefully without functional changes.

There's more that can be done, but as this is a very long (>10K lines)
file, I think it's enough for one change.
Specifically, some functions fetch values from the dictionary
without using it - this is a bit of a waste.
Filed https://github.com/gluster/glusterfs/issues/753 about it.

Change-Id: I31f88d94ab25398e00aef2ea84a8c4af9383b75b
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2019-10-28 21:29:50 +02:00
Barak Sason Rofman
42e1764f59 glusterd: refactoring long method
- Refactored set_fuse_mount_options(...) in order to shorten it.
- Removed dead code and moved some method to it's apropriate
location.
- Converted loggin in set_fuse_mount_options(...) to structured logs

fixes: bz#1768896
Change-Id: If865833d4c60d517da202871978691ef21235fe4
Signed-off-by: Barak Sason Rofman <bsasonro@redhat.com>
2019-11-05 15:54:29 +02:00
kshithijiyer
e1fa3dacf8 Removing old log rotate command.
The old command for log rotate is still present removing
it completely. Also adding testcase to test the
log rotate command with both the old as well as the new command
and fixing testcase which use the old syntax to use the new
one.

Code to be removed:
1. In cli-cmd-volume.c from struct cli_cmd volume_cmds[]:
{"volume log rotate <VOLNAME> [BRICK]", cli_cmd_log_rotate_cbk,
 "rotate the log file for corresponding volume/brick"
 " NOTE: This is an old syntax, will be deprecated from next release."},

2. In cli-cmd-volume.c from cli_cmd_log_rotate_cbk():
 ||(strcmp("rotate", words[2]) == 0)))

3. In cli-cmd-parser.c from cli_cmd_log_rotate_parse()
if (strcmp("rotate", words[2]) == 0)
   volname = (char *)words[3];
else

fixes: bz#1750387
Change-Id: I56e4d295044e8d5fd1fc0d848bc87e135e9e32b4
Signed-off-by: kshithijiyer <kshithij.ki@gmail.com>
2019-11-30 15:25:11 +05:30
Yaniv Kaul
2def277c56 cli-*: make some structs static to reduce space consumed.
There's a small difference when structs are defined static.
Whenever possible, define them as such.

Specifically, before:
   text    data     bss     dec     hex filename
    678     216       0     894     37e ./cli/src/cli-cmd-misc.o
 150024    1264      16  151304   24f08 ./cli/src/cli-rpc-ops.o
  71980      64       0   72044   1196c ./cli/src/cli-cmd-parser.o
  66189       4      16   66209   102a1 ./cli/src/cli-xml-output.o

After:
   text	   data	    bss	    dec	    hex	filename
    670	    216	      0	    886	    376	./cli/src/cli-cmd-misc.o
 149848	   1392	     16	 151256	  24ed8	./cli/src/cli-rpc-ops.o
  70346	   1320	      0	  71666	  117f2	./cli/src/cli-cmd-parser.o
  66157	      4	     16	  66177	  10281	./cli/src/cli-xml-output.o

Change-Id: I206bd895290595d79fac7b26eee66f4279b50f92
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2019-10-04 23:14:08 +03:00
Sanju Rakonde
3d8c095063 cli: display detailed rebalance info
Problem: When one of the node is down in cluster,
rebalance status is not displaying detailed
information.

Cause: In glusterd_volume_rebalance_use_rsp_dict()
we are aggregating rsp from all the nodes into a
dictionary and sending it to cli for printing. While
assigning a index to keys we are considering all the
peers instead of considering only the peers which are
up. Because of which, index is not reaching till 1.
while parsing the rsp cli unable to find status-1
key in dictionary and going out without printing
any information.

Solution: The simplest fix for this without much
code change is to continue to look for other keys
when status-1 key is not found.

fixes: bz#1764119

Change-Id: I0062839933c9706119eb85416256eade97e976dc
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
2019-10-22 15:06:29 +05:30
Vishal Pandey
8e5faf6860 cli: raise error in cli when replica != 2 while ta vol creation
When creating a thin-arbiter volume, if the replica-count
provided is other than 2, the cli command
fails(which is the expected behaviour) but it fails
while transaction is being committed on localhost.
This, ideally should be handled in cli itself.

Change-Id: I52ecdaf49a7df85670505e4743cdcf3101c71c9f
Signed-off-by: Vishal Pandey <vpandey@redhat.com>
Fixes: bz#1754477
2019-09-23 20:12:21 +05:30
Sanju Rakonde
552f9f9a9e cli: fix distCount value
gluster volume info --xml id displaying wrong distCount
value. This patch addresses it.

fixes: bz#1758878

Change-Id: I64081597e06018361e6524587b433b0c4b2a0260
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
2019-10-06 19:05:28 +05:30