Implement glustercli command to disable and delete the current tracing
configuration on the cluster. The changes include gd2 transaction that
first deletes the trace configuration from the store on one node and then
a subsequent step clears the in-memory trace configuration on all nodes.
closes #1368
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Implement glustercli command to update the current tracing status on the
cluster. All trace config options are passed as flags to the command. If
any option is not passed, the existing value for that option will be
retained.
closes #1368
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Implement glustercli command to get the current tracing status on the
cluster. The tracing info is read from the store and presented to the
user in table format with info like Status, Jaeger Endpoints, Sampler
type and sample fraction. For e.g.,
+------------------------+----------------------------+
| TRACE OPTION | VALUE |
+------------------------+----------------------------+
| Status | enabled |
| Jaeger Endpoint | http://192.168.122.1:14268 |
| Jaeger Agent Endpoint | http://192.168.122.1:6831 |
| Jaeger Sampler | 2 (Probabilistic) |
| Jaeger Sample Fraction | 0.99 |
+------------------------+----------------------------+
Add "trace enable" e2e test cases. The tests also exercise the
"trace status" request.
closes #1368
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Implement glustercli command to enable tracing. The rest client performs
basic checks of the tracing options prior to sending the request.
closes #1368
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
If no start-up options are specified (i.e. command line or config file),
but a valid trace config exists in the store, then read the config as
part of GD2 start-up sequence and apply the trace config on the node that
is coming up. This way, all the GD2 nodes read the trace configuration
and apply it on themselves on start-up. This scenario is applicable when,
- trace configuration was applied via glustercli (not start-up config)
and,
- GD2 node(s) restart for some reason. If there's a valid trace config
in the store, then it must be applied when a node comes up.
closes #1368
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
The new client configurable options that allows trace control are,
jaeger-sampler:
---------------
This option can take any one of the following values. Each value
maps to,
- 0 (Never): Disable tracing.
- 1 (Always): Samples all traces.
- 2 (Probabilistic): A probabilistic sampler. Sample traces based on
an additional option called "sampling fraction" described below using
which Jaeger decides whether to trace the operation or not.
jaeger-sample-fraction:
-----------------------
If "jaeger-sampler-type" is set to 2 (probabilistic), then an option to
set the sampling fraction (or frequency) can be specified. By default this
value is set to 0.1 (i.e. sample every 1 in 10 traces). Valid values for
this option ranges from >0.0 and <1.0. Higher the value, the greater the
probability that a trace is sampled. This allows a control on the volume
of traces captured in a highly scaled environment.
Other changes in this commit:
- Encapsulate Jaeger specific trace configuration information in a separate
structure that is instantiated only once and updated subsequently.
- Factor out code that validates and sets trace config information into
separate functions for other modules to re-use.
closes #1368
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Currently, to GET/DELETE any block volume we mount all the block hosting
volumes and readdir all the volumes, and loop through it, to find the
block hosting volume, the block belongs to. This approach is not scalable.
Hence, in the metadata of the block hosting volume, we keep a list of all
the block volumes present in that host volume. This way, GET/DELETE walks
through the volume metadata rather than the readdir.
Signed-off-by: Poornima G <pgurusid@redhat.com>
If a Volume is cloned from another volume, the bricks of cloned volume
will also belong to same LV thinpool as the original Volume. So before
removing the thinpool a check was made to confirm if number of Lvs in
that thinpool is zero. This check was causing hang when parallel
Volume delete commands were issued.
With this PR, number of Lvs check is removed, instead of that captured
the failure of thinpool delete and handled it gracefully.
This PR also adds support for gracefully delete the volume if lv or
thinpool already deleted by previous failed transaction or manual delete.
Signed-off-by: Aravinda VK <avishwan@redhat.com>
- moved hosts parameter from mandatory to optional field in CreateBlockVolume
method of BlockProvider interface,since hosts field may not required for
other block providers like loopback.
- a common function for updating available hosting volume size will prevent
from duplicate code
Signed-off-by: Oshank Kumar <okumar@redhat.com>
Sunrpc Read buffer was created by allocating `1MiB` each time when
client connects. Go can automatically expand the buffer when required
so now empty buffer is initialized.(`1MiB` per brick connect is `1GB`
for 1000 bricks)
Signed-off-by: Aravinda VK <avishwan@redhat.com>
`vgdisplay` command will be used during Volume create to update the
device size after each Volume Create. `lvs` command is used to check
number of LVs in a Thinpool before deleting the Thinpool. Added
readonly to both the commands to avoid lvm lock.
Signed-off-by: Aravinda VK <avishwan@redhat.com>
The implementation is an extension to brick-multiplexing. The max-bricks-per-process is being
monitored by using entries in pmap corresponding to a particular port.
So, If a brick needs to be multiplexed following are the series of steps that needs to be followed. The following are the genralised sets of steps -
- look into all started volumes with same options as the current volume(one by one)
- traverse through all local bricks of taregtVolume
- find the port of the local brick by using its path
- look for number of entries in pmap corresponding to particular port No. and count num of bricks already attached to that port
- If the number of bricks attached to a port is less that max-bricks-per-process constraint
- Then we have our target brick or repeat from step 1 until all targetVolumes are covered.
- Since current volume is not considered into started volumes list, so if we don't have any target brick yet from any of the
started volumes then
- look for any target brick in the current volume
- If even the current volume doesn't have any target brick then
- start seperate glusterfsd.
Cases Handles in this PR and approach followed -
1- If no started volume already present
- check current volume for target brick
- if taregt brick not found, start seperate glusterfsd
2- If atleast one started volume present
- if Target volume found, out of all the started volumes
* if target brick found, attach to the brick
* if taregt brick not found, look through other taregt volumes
* if still target brick not found, look into current volume
* if still taregt brick not found, start seperate glusterfsd process
- if target Volume not found
* look into current volume for any target bricks
* if taregt brick not found, start seperate glusterfsd
Signed-off-by: Vishal Pandey <vpandey@redhat.com>
While creating auto provisioned volume, support added to automatically
calculate the distribute count based on max brick size specified in
the request.
For example, below command creates 2x3(Distributed replicate) volume
```
glustercli volume create gv1 --replica 3 --size 1G \
--max-brick-size 512M
```
Fixes: #999
Signed-off-by: Aravinda VK <avishwan@redhat.com>
- Added default group profiles for each volume
types(replicate,disperse,distribute)
- Included perf xlators in volfile template and disabled in default
profiles. So that these xlators can be enabled/disabled if
required.
- Included "features/shard" xlator in client volfile(Fixes: #954)
- Enabled self heal by default for replicate and disperse volumes.
- On glusterd2 start/restart, group profiles are saved in
`$workdir/templates/profiles.json` (Ex:
`/var/lib/glusterd2/templates/profiles.json)
- To modify profile defaults, update respective default profile and
restart glusterd2.(Note: Already created Volume options will not
change, only applicable for new volumes)
- All perf xlators are disabled by default till we decide the best
defaults.
- Disabled set/reset of default option profiles.
Signed-off-by: Aravinda VK <avishwan@redhat.com>
- Group profile for transactional DB workload added
- Fixed the option names used in other group profiles
- Fixed the validation issues related to setting the options
glusterfs PR to support enable/disable of xlators
https://review.gluster.org/21813
Fixes: #1250
Signed-off-by: Aravinda VK <avishwan@redhat.com>
MountLocalBricks() and MountVolumeBricks() are called during GD2 startup.
Both these methods are responsible for mounting bricks for all the volumes in the store.
MountLocalBricks() exits as soon as there is some issue with mounting any volume's localBricks because of
which other volumes' localBricks(which are yet to be processed) are skipped for mounting.
MountVolumeBricks() exits as soon as any one of the brick of that volume is unable to be mounted because of which
other localBricks of that volume(which are yet to be processed), are skipped for mounting.
FIX-
Modify both the methods to not exit after an error but log the error and continue the mounting process for othe volumes and localBricks
Signed-off-by: Vishal Pandey <vpandey@redhat.com>
Moved lvm related functions from `$SRC/glusterd2/snapshot/lvm`
and `$SRC/plugins/device/deviceutils/` to `$SRC/pkg/lvmutils`
Also moved fs related functions from `plugins/deviceutils` to
`$SRC/pkg/fsutils`.
Fixes: #1187
Signed-off-by: Aravinda VK <avishwan@redhat.com>
This patch creates the arbiter brick for the smart volume as per the
calculation:
brick size = 4 KB * ( size in KB of largest data brick in volume or
replica set / average file size in KB)
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
Brick volfiles generation added during Snapshot activate and
volfiles delete added during snapshot deactivate.
Also fixed client volfile regeneration of snapshot volume during
snapshot restore undo.
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Snapshot bricks are provisoned by glusterd. During a glusterd
restart, we should check for brick mounts for activated
snapshots.
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Firewalld is now notified over dbus to open TCP port when a brick signs
in. Correspondingly, the firewalld is notified to close the TCP port
when a brick signs out - gracefully or otherwise.
During glusterd2 shutdown, firewalld is NOT notified to remove port.
This is to ensure that the client can talk to bricks even when
glusterd2 is down.
The package github.com/godbus/dbus has been added as a new dependency
and is available as a RPM package in centos7.
Caveats:
All firewalld configuration changes are (by choice) runtime changes i.e
these are not permanent. Reloading firewalld configuration will result
in the current runtime configuration being wiped out.
Signed-off-by: Prashanth Pai <ppai@redhat.com>
'--advanced' in the CLI and REST API during volume create doesn't intuitively
and clearly indicate that it's for the volume options. A new user may read this
as creation of an "advanced volume" which isn't true.
The flags have now been prepended with 'allow' in the fields of request structs.
The API has been renamed as 'allow-<type>-options'.
Also, created 'VolOptionFlags' struct for flags of options and embeded it in
all request structs.
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Added capacity field to volume GET AP response. This is currently
returned only for volumes provisioned dynamically and their volume
snapshots.
The CLI prints the volume capacity in a human friendly format.
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
The benign log messages logged by net/rpc package's server.Register()
is very annoying. The enhancement introduced by bdaa2c97 (PR #1237)
amplified this already excessive logging.
Logging by code in standard library is now disabled. Earlier it was
being redirected to logrus's io.Writer.
Signed-off-by: Prashanth Pai <ppai@redhat.com>
