## Major Changes and Features

Documentation about major changes and features is also included in the `doc/features/` directory of GlusterFS repository.

### AFR_CLI_enhancements

The AFR reporting via CLI has been improved. This feature provides a coherent
mechanism to present heal status,information and the logs associated.
This makes the end user more aware of healing status and provides statistics.

For more information refer [here](https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/AFR%20CLI%20enhancements.md).

### File_Snapshot

This feature provides ability to take snapshots of files in GlusterFS.
File snapshot is supported on the files of QCOW2/QED format.

This feature adds better integration with Openstack Cinder, and
in general ability to take snapshots of files (typically VM images)

For more information refer [here](https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/File%20Snapshot.md).

### gfid-access

This feature add a new translator which is designed to provide direct access
to files in glusterfs using its GFID

For more information refer [here](https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/gfid%20access.md).

### Prevent NFS restart on Volume change
Earlier any volume change (volume option, volume start, volume stop, volume
delete,brick add, etc) required restarting NFS server.

With this feature, it is no longer required to restart NFS server, thereby
providing better usability with no disrupts in NFS connections

### Features/Quota_Scalability

This feature provides support upto 65536 quota configurations per volume.

### readdir_ahead

This feature provides read-ahead support for directories to improve sequential
directory read performance.

### zerofill

zerofill feature allows creation of pre-allocated and zeroed-out files on
GlusterFS volumes by offloading the zeroing part to server and/or storage
(storage offloads use SCSI WRITESAME), thereby achieves quick creation of
 pre-allocated and zeroed-out VM disk image by using server/storage off-loads.

For more information refer [here](https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/Zerofill.md).

### Brick_Failure_Detection

This feature attempts to identify storage/file system failures and disable
the failed brick without disrupting the rest of the NODE operation.

This adds a health-checker that periodically checks the status of the
filesystem (implies checking of functional storage-hardware).

For more information refer [here](https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/Brick%20Failure%20Detection.md).

### Changelog based distributed geo-replication

New improved geo-replication which makes use of all the nodes in the master volume.
Unlike previous version of geo-replication where all changes were detected and synced
on a single node in master volume, now each node of master volume participates in the
geo-replication.

Change Detection - Now geo-rep makes use of changelog xlator to detect the set of files
which needs to be synced. Changelog xlator runs per brick and when enabled, records
each fops which modifies the files. geo-rep consumes these journal created by this
xlator and syncs the files identified as 'changed' to slave.

Distributed nature - Each of the nodes take the repsonsibility of syncing the data
which is present in that node. In case of replicated volume, one of them will be
'Active'ly syncing the data, while the other one is 'Passive'.

Syncing Method - Apart from the using rsync as the syncing method, now there tar+ssh
syncing method, which can be leveraged by the workload where there is large amount
of smallfiles.

### Improved block device translator

This feature provides a translator to use logical volumes to store VM images
and expose them as files to QEMU/KVM.

The Volume group is represented as directory and logical volumes as files.

### Remove brick CLI Change

remove-brick CLI earlier used to remove the brick forcefully ( without data migration ),
when called without any arguments. This mode of 'remove-brick' cli, without any
arguments  has been deprecated.

### Experimental Features

The following features are experimental with this release:

- RDMA-connection manager (RDMA-CM).
- support for NUFA translator.
- disk-encryption
- On-Wire Compression + Decompression [CDC]

## Minor Improvements:

- Old graphs are cleaned up by FUSE clients

- New command "volume status tasks" introduced to track asynchronous tasks like rebalance and remove-brick

- glfs_readdir(), glfs_readdirplus(), glfs_fallocate(), glfs_discard() APIs support added in libgfapi

- Per client RPC throttling added in rpc server

- Communication between cli and glusterd happens over unix domain socket

- Information on connected NFS clients is persistent across NFS restarts.

- Hardlink creation failures with SMB addressed

- Non-local clients function with nufa volumes

- Configurable option added to mount.glusterfs to use kernel-readdirp with fuse client

- AUTH support for exported nfs sub-directories added


### Known Issues:
- The following configuration changes are necessary for qemu and samba
  integration with libgfapi to work seamlessly:

             1) gluster volume set <volname> server.allow-insecure on

             2) Edit /etc/glusterfs/glusterd.vol to contain this line:
                    option rpc-auth-allow-insecure on
            Post 1), restarting the volume would be necessary.
            Post 2), restarting glusterd would be necessary.

-  RDMA connection manager needs IPoIB for connection establishment. More
   details can be found [here](https://github.com/gluster/glusterfs-specs/blob/master/done/Features/rdmacm.md).


- For Block Device translator based volumes open-behind translator at the
client side needs to be disabled.

- libgfapi clients calling glfs_fini before a successfull glfs_init will cause the client to
  hang as reported [here](http://lists.gnu.org/archive/html/gluster-devel/2014-04/msg00179.html).
  The workaround is NOT to call glfs_fini for error cases encountered before a successfull
  glfs_init.
