GCC and Clang communities are hinting that in versions 14 and 16
respectively they will deprecate or disable legacy C89 features,
e.g. K&R1 style function definitions, among others.
In parallel, Fedora is going to start enforcing C99 as the minimum,
expected to land in F40. (I.e. around Spring 2024 IIRC.)
Currently Fedora is recommending that use of -Werror=implicit-int,
-Werror=implicit-function-declaration, -Werror=int-conversion,
-Werror=strict-prototypes, and -Werror=old-style-definition a set
of options to build with in the mean time to clean up code.
This change fixes a subset of the errors found when compiling with
this set of options.
Change-Id: Ia436ef321ad7963331d5cc813c98880525d0525e
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
GCC and Clang communities are hinting that in versions 14 and 16
respectively they will deprecate or disable legacy C89 features,
e.g. K&R1 style function definitions, among others.
In parallel, Fedora is going to start enforcing C99 as the minimum,
expected to land in F40. (I.e. around Spring 2024 IIRC.)
Currently Fedora is recommending that use of -Werror=implicit-int,
-Werror=implicit-function-declaration, -Werror=int-conversion,
-Werror=strict-prototypes, and -Werror=old-style-definition a set
of options to build with in the mean time to clean up code.
This change fixes a subset of the errors found when compiling with
this set of options.
Change-Id: I2025e39566aeb1f68836226d6d18c43418ca7f95
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
GCC and Clang communities are hinting that in versions 14 and 16
respectively they will deprecate or disable legacy C89 features,
e.g. K&R1 style function definitions, among others.
In parallel, Fedora is going to start enforcing C99 as the minimum,
expected to land in F40. (I.e. around Spring 2024 IIRC.)
Currently Fedora is recommending that use of -Werror=implicit-int,
-Werror=implicit-function-declaration, -Werror=int-conversion,
-Werror=strict-prototypes, and -Werror=old-style-definition a set
of options to build with in the mean time to clean up code.
This change fixes a subset of the errors found when compiling with
this set of options.
Change-Id: I9c4b4a0b002835d6555d8fef9456fea16c31b28a
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
GCC and Clang communities are hinting that in versions 14 and 16
respectively they will deprecate or disable legacy C89 features,
e.g. K&R1 style function definitions, among others.
In parallel, Fedora is going to start enforcing C99 as the minimum,
expected to land in F40. (I.e. around Spring 2024 IIRC.)
Currently Fedora is recommending that use of -Werror=implicit-int,
-Werror=implicit-function-declaration, -Werror=int-conversion,
-Werror=strict-prototypes, and -Werror=old-style-definition a set
of options to build with in the mean time to clean up code.
This change fixes a subset of the errors found when compiling with
this set of options.
Change-Id: I4c1726915883e4a9912f41d92644c1e8916a979d
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
GCC and Clang communities are hinting that in versions 14 and 16
respectively they will deprecate or disable legacy C89 features,
e.g. K&R1 style function definitions, among others.
In parallel, Fedora is going to start enforcing C99 as the minimum,
expected to land in F40. (I.e. around Spring 2024 IIRC.)
Currently Fedora is recommending that use of -Werror=implicit-int,
-Werror=implicit-function-declaration, -Werror=int-conversion,
-Werror=strict-prototypes, and -Werror=old-style-definition a set
of options to build with in the mean time to clean up code.
This change fixes a subset of the errors found when compiling with
this set of options.
Change-Id: I8271269e4f6d504a7b812f89c7d6b4e60edf661f
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
GCC and Clang communities are hinting that in versions 14 and 16
respectively they will deprecate or disable legacy C89 features,
e.g. K&R1 style function definitions, among others.
In parallel, Fedora is going to start enforcing C99 as the minimum,
expected to land in F40. (I.e. around Spring 2024 IIRC.)
Currently Fedora is recommending that use of -Werror=implicit-int,
-Werror=implicit-function-declaration, -Werror=int-conversion,
-Werror=strict-prototypes, and -Werror=old-style-definition a set
of options to build with in the mean time to clean up code.
This change fixes a subset of the errors found when compiling with
this set of options.
Change-Id: Ibb393780e8e6cce70f0f7f9910a91f4b588f70c3
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Currently glusterd always sets send-gids to the negated value of
manage-gids. This is not correct when we just want to check user
permissions in the client side. In this case we don't need to enable
manage-gids, but we also don't need to enable send-gids.
Updates: #3781
Change-Id: Ia42825e7f6993896b1000a07d420b84d1d0422b6
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
In addition to files glfs_h_open() is capable of handling directory
opens. But there are various other components like DHT and probably
other client xlators which are tightly coupled to work with directory
opens using just opendir. One such example is the case where fsetxattr()
is called with a file descriptor opened for the directory using
glfs_h_open() resulting in EBADFD.
Therefore we make a differentiation in glfs_h_open() to correctly call
syncop_open() or syncop_opendir() for file and directory entries
respectively to avoid any possible file descriptor errors.
Credits: Xavi Hernandez <xhernandez@redhat.com>
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
Use a flexible array member and allocate the memory (pointed by freeptr/ptr), together with the iobuf itself.
Also, reoder the initialization of the structure's variables (but this could be done separately as well).
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
In addition to files glfs_open() and glfs_openat() are capable of
handling directory opens. But there are various other components
like DHT and probably other client xlators which are tightly coupled
to work with directory opens using just opendir. One such example is
the case where fsetxattr() is called with a file descriptor opened for
the directory using glfs_open() or glfs_openat() resulting in EBADFD.
Therefore we make a differentiation within these APIs to correctly
call syncop_open() or syncop_opendir() for file and directory entries
respectively to avoid any possible file descriptor errors.
Fixes: #3876
Credits: Xavi Hernandez <xhernandez@redhat.com>
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
* protocol/server: fix the server_getspec to serve the volfiles
This commit fixes an option of serving volume files through
brick processes. While this feature is not a required feature
for any deployments using `glusterd`, it would be very useful
in scenarios where glusterd is not present, eg., container usecases,
and in projects like kadalu which only deals with management layer
changes.
few changes done with this commit:
* core: Add 'EVENT_SIGHUP' event to notify framework
* make `volfile` based process also handle SIGHUP
* add port parsing along with server, so we can have process hosted in any port
* test to demonstrate all this.
Updates: #3635, #3668
Change-Id: I5c8dfdee7d06b8d5fced4cc99059dfd8bed65260
Signed-off-by: Amar Tumballi <amar@kadalu.io>
- gf_mem_update_acct_info() is not needed when not in DEBUG mode
- re-order variables in the structure according to access pattern
- Turn xlator_mem_acct_unref() into xlator_mem_acct_destroy() and call it only when refcnt euqals 0 - which is quite rare.
Updates: #3855
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* dictionary data_pair: make key a flexible array member
We always ended up calling allocation - even though we tried to avoid it using the free pair which was supposed to handle
the common case of a single data pair in a dictionary, we still called GF_MALLOC for the key.
Instead, allocate the key together with the data pair, unconditionally, in a single call.
This also simplifies the code a bit by getting rid of the logic around the free pair and removes multiple checks for NULL of pair->key.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* Multiple files: remove dict_pair_pool memory pool
It's not needed anymore, now that we allocate the pair (along with the pair->key).
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Allocate the IOV vector as part of the gio struct allocation (and same for deallocation).
Also change from GF_CALLOC() to GF_MALLOC() as we initialize all variables.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
It was later allocated anyway, via strdup(), so now it's allocated and
deallocated along with the dentry structure. Switched to using malloc()
instead of calloc() as we were initializing all members. Lastly, removed
the (now unused) pool for dentries.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* inode.c/h: use flexible array member for inode _ctx struct
We always know the _ctx array size on inode creation - so we can allocate it along with the inode struct itself
This also moves the inode creation from using a memory pool (if memory pool are enabled anyway) to regular GF_CALLOC().
Next patch will remove the inode pool.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* inode: remove unused inode_pool memory pool
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
For locality, reduce the no. of calls to CALLOC and the number of NULL checks, it's easier to just allocate
them together (since the no. of iobufs in an arena is known and fixed at the time of arena allocation)
This allows removal of some checks. I've also made the no. of iobufs a 32bit unsigned and ensured the struct is correctly aligned and without padding.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* client_t: move scratch structure variables to be directly accessible from the client struct
Easier to read and a step into allocating with the client struct itself.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* client_t: remove not needed scratch ctx count
It doesn't change - it's always GF_CLIENTCTX_INITIAL_SIZE
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* client_t: turn client scratch ctx to a flexible array member variable of client struct
Allocate it along with the whole client structure allocation.
Useful, as it's fixed size, the allocation is done under lock, and for locality.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* mem-types.h: remove unused memory type
The code is under #ifdef, commented out at the moment.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* client_t.c: move client_uid to be a flexible array member (instead of scratch ctx, which is actually constant size)
It wasn't the best choice to move scratch ctx to be a variable size - it isn't - it's very fixed and constant.
So turned client_uid into such.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* Remove additional GF_FREE() that is not needed.
* Fix review comments - remove unused memory type
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* index xlator: return a pointer instead of passing a variable to store the ctx
Similiar to other patches, we can just return the ctx object and check if it's NULL or not.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* index.c: use the return value of snprintf() to save few strlen() calls.
Instead of a strlen(), use the return value of a snprintf() call.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* index.c: make all relevant functions static
Updates: #1000
signed-off-by: Yaniv Kaul <ykaul@redhat.com>
OpenSSL 3.0.x deprecated some functions.
This patches changes one implementation to be compatible with the newer version.
Updates: #2916
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Since the number of samples is known in sample struct creation time, we can allocate them in a single allocation call.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* event-epoll: allocate table 0 as part of the event pool object allocation
Since we allocate it initially anyway as part of the event pool allocation process, let's allocate it in the same call.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* event-epoll: turn slot_used to slot_avail and make it unsigned 64bit
It's faster to just compare it to 0 (if none are available) and there's no need for the padding - use a 64bit unsigned for it.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
[inode.c] alternative hash strategy that grabs different bytes, without
mixing, and then modulo's that into the hash table size.
The old mechanism used the last two bytes only, thus allowing for a
distribution from 0x0 to 0xffff - which since 65536 is too small a hash
table to maintain sensible performance in some use cases is
insufficient. We require a much larger (minimum 4x larger but ideally
16x larger) distribution.
This isn't as predictive, it will use the sizeof(int) which can vary in
theory, but is generally 4 bytes, so we will use the first four completely
random bytes, and just straight modulo that, retaining the requirement
that the modulo must be a power of two (which further improves
performance by eliminating the multi-cycle division).
Glusterfs uses libuuid, which by default the entire 16 bytes is just
random, where time-based is used as a fallback. Should we be using a
time-based UUID values this might be a problem if a lot of files were
initially created around the same time, or files are created in clusters
around the same time. Previously if we were falling back to this two
bytes from the node portion of the UUID would have been used, which
would have been extremely bad for performance, as such I believe this
should not be a problem.
Signed-off-by: Jaco Kroon <jaco@uls.co.za>
The specified commit in 'gftest build' command was interpreted as a
branch name instead of a commit hash, which was incorrect and caused
failures when trying to use a it.
Fixes: #3847
Change-Id: I3ae3f255996cced722573fdc9ae189acfb22eb85
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* index.c: skip some strcmp() in case entry length is short enough.
In most cases we'll calculate - strlen() the length of the entry.
Use that length - only if it's longer than 42, which is 'dirty-<uuid in ascii...>',
then there's a point in checking if it matches 'dirty-...' or 'xattrop-...' string.
(Note: unsure, but I suspect we could have and should have skipped . and .. in as well)
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* bit-rot-stub-helpers.c: skip checking for stale entries based on file name length.
skip some strcmp() in case entry length is short enough.
In most cases we'll calculate - strlen() the length of the entry.
Use that length - only if it's not 41, which is 'stub-<uuid in ascii...>',
then there's a point in checking if it matches 'stub-...' string.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
The posix lock is not working correctly(1x3) if a process
has received an interrupt after wind a fop and the POSIX lock is
already held by another process.
Suppose there are two processes (P1, P2), P1 has requested an
exclusive POSIX lock on the file and it has received a successful response.
After that a process P2 is also tried to request the same lock and it
has been blocked because a lock is holding by another process.
After requesting the POSIX lock if process P2 received an interrupt then
fuse will send a clear request to the brick process to clear the blocked lock.
The issue is after receiving the error(if it is not EAGAIN) afr winds a fop
on other children. The afr first wind a fop on first active children once
it has received a status then winds a call on other children if it is not EAGAIN.
Solution: Along with EAGAIN afr needs to check EINTR error, the expectation
is after receive an interrupt the lock should be clear by
the brick process so avoid to wind a call on other children's
in case if it has received EINTR error.
Fixes: #3831
Change-Id: I35c07c68708050a7497a1da6b05131e6e5848579
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Newly added glfs_openat() failed to link inode causing the next
immediate open without a lookup to fail with ENOENT. Therefore
add the missing inode link using glfs_loc_link().
This also reverts a hunk from 8cec218856
as iatt struct is required for glfs_loc_link() in glfs_openat().
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
Simplify '__server_getspec()' by using 'gf_asprintf()', add
proper error handling.
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Updates: #1000
A new tool gftest has been created to automate and simplify the
preparation of an environment to run tests. It can also process
the full set of regression tests in parallel to significantly
reduce the total execution time.
It's written in python and requires these components:
- libvirt
- qemu-kvm
- gnupg
- python-click
All other modules should already be present in a default installation.
Using this tool is quite simple:
1. Create the VM
```
gftest create [OPTIONS] <name>
Options:
--template <name> Template to use (default: centos7)
--port <num> Port to access the VM (default: 2222)
--key <path> Private key for SSH (default: ~/.ssh/id_rsa)
--cpus <num> Number of virtual cores of the VM
--memory <num> Memory assigned to the VM in GiB
--disk <num> Disk space for the VM in GiB
```
This creates a KVM VM named <name> that can be accessed through the
specified port on the local host (i.e. `ssh -p <port> root@127.0.0.1`)
using the specified private key. All other option will be taken from
the template if not present.
This process also creates a container inside the VM with all the
required packages and configurations to build Gluster and run the
tests.
2. Build the code
```
gftest build [OPTIONS] <name>
Options:
--commit <id> SHA/name of the commit to compile
```
This copies the current git repo to the VM and compiles the specified
commit in a container image.
3. Start the workers
```
gftest spawn [OPTIONS] <name>
Options:
--workers <num> Number of workers to create
--space <num> Space allocated for running tests in GiB
```
This starts the specified number of container instances of the compiled
Gluster image.
4. Run the tests
```
gftest run <name> <output dir>
```
This runs the full set of tests in parallel in all available containers
and puts the results into the <output dir>.
It also collects statistics for each test and CPU and memory state
during the full run.
5. Access the VM or container for manual testing/debugging
```
gftest sh <name> [<idx>]
```
Without `idx` it opens a shell session to the VM. With an index it opens
a shell session to the idx-th testing container.
There are some additional commands:
- `gftest kill <name>`
Stops and destroys all testing containers.
- `gftest shutdown <name>`
Gracefully shuts down the VM.
- `gftest poweroff <name>`
Forcibly stops the VM.
- `gftest poweron <name>`
Starts the VM.
Updates: #3469
Change-Id: I169877a4c5197d001bf2822ad7116559f7d78754
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Move bash-completion script for the `gluster` command
in `/usr/share/bash-completion/completions`
instead of `/etc/bash_completion.d`.
Allow bash-completion to dynamically load the script
only when a user tries to tab complete on the `gluster` command.
Fixes: #3750
Change-Id: Ifca300bdf5b14a68726a6fdfdcc19052cf624f85
Signed-off-by: Frédéric Moulins <frederic@moulins.org>
They are optional anyway, so if the caller does not intend on using them later on, just pass NULL instead
and skip copying them.
Updates: #3717
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
There's no point in doing a dict_ref() (or creating a new dictionary) and then eventually
unref (or deleting) if version <= 0, since no work will be done anyway.
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Problem: It takes a while for /var/log/glusterfs/glustershd.log
to get created and populated
after the volume start command firing.
Hence, a grep expecting particular value inside
/var/log/glusterfs/glustershd.log always fails
when executed soon after firing the volume
start command
Solution:Verify online_brick_count to make sure volume start is
complete and the log file is created
before grepping for the values in it.
Fixes: #3836
Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
Sequence to patch #3532
Reducing variable indirection and making use of more scoped variables
when possible in the afr component.
Updates: #2986
Change-Id: Ide73d8b5a679e6edd670110f1e3ac3d39347e0e9
Signed-off-by: harshita-shree <hshree@redhat.com>
Simple-quota works on the idea that each brick knows what its capacity
at the directory level are. In EC, the brick doesn't have full data but
reduced by a factor of fragments. Thus, in this special case, while
sending the limit itself, good to handle the fragment size part.
Updates: #3638
Change-Id: Id341de5afe59a7348c5eb15c51555243363550ef
Signed-off-by: Amar Tumballi <amar@dhiway.com>
Force `LANGUAGE` to parse `git remote` output.
Stop the script if the upstream cannot be determined.
Change-Id: Id7bd53d003ebb329b866044ac69fd47f64e4f3d0
Updates: #3823
Signed-off-by: Frédéric Moulins <frederic@moulins.org>
* multiple files: fd_ctx_get() should just return the value parameter.
There's no point in both 'ret' and 'value' params.
* Multiple files: implement fd_ctx_get_ptr() macro that does the conversion to the relevant struct pointer
Simplifies nicely the code in many places. Thanks Xavi for the suggestion!
* Added __fd_ctx_get_ptr() macro, similar to fd_ctx_get_ptr() functionality
* Fix __fd_ctx_del() to return as soon as the ctx is found and not continue the loop
* Add fd_ctx_del_ptr() macro which will delete the context and return already a cast to a pointer.
* Fix __posix_fd_ctx_get() - ret was not set to 0 on success.
* Fix dht_fd_ctx_set() ret value
* Fix (unrelated to this whole series) pl_check_n_create_fdctx() logic.
it was not really using existing ctx. Luckily(?) it never/rarely(?) had an existing ctx...
* Minor changes
Make refcount 32bit and proper padding for the structure are the main changes
Updates: #1000
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Per #2253 we need to disable lookup-optimize feature.
More importantly, there are some FSYNCDIR (4 of them) that appear for some reason too soon.
Re-order the profile start - give 5 seconds for those to be done with and then start profiling.
Fixes: #3708
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
The size of the stack for synctasks created by marker xlator was
only 16 KiB. Now they use the default stack size for synctasks.
Fixes: #3710
Signed-off-by: 10255046 <wu.shiwei@zte.com.cn>
