IBM Storage Ceph vs. Red Hat Gluster Storage

It is absolutely, hands down the best storage solution for Open Stack. I would even argue it is the only solution if a company is operating at petabyte scale and need resiliency. The storage solution allows any organization to scale their environment using commodity hardware from top to bottom. It has a battle tested track record where it is even being used as the data storage back end for the Large Hadron Collider at Cern

Incentivized

GFS is well suited for DEVOPS type environments where organizations prefer to invest in servers and DAS (direct attached storage) versus purchasing storage solutions/appliances. GFS allows organizations to scale their storage capacity at a fraction of the price using DAS HDDs versus committing to purchase licenses and hardware from a dedicated storage manufacturer (e.g. NetApp, Dell/EMC, HP, etc.).

Incentivized

• Highly resilient, almost every time we attempted to destroy the cluster it was able to recover from a failure. It struggled to when the nodes where down to about 30%(3 replicas on 10 nodes)
• The cache tiering feature of Ceph is especially nice. We attached solid state disks and assigned them as the cache tier. Our sio benchmarks beat the our Netapp when we benchmarked it years ago (no traffic, clean disks) by a very wide margin.
• Ceph effectively allows the admin to control the entire stack from top to bottom instead of being tied to any one storage vendor. The cluster can be decentralized and replicated across data centers if necessary although we didn't try that feature ourselves, it gave us some ideas for a disaster recovery solution. We really liked the idea that since we control the hardware and the software, we have infinite upgradability with off the shelf parts which is exactly what it was built for.

Incentivized

• Scales; bricks can be easily added to increase storage capacity
• Performs; I/O is spread across multiple spindles (HDDs), thereby increasing read and write performance
• Integrates well with RHEL/CentOS 7; if your organization is using RHEL 7, Gluster (GFS) integrates extremely well with that baseline, especially since it's come under the Red Hat portfolio of tools.

Incentivized

• Authorization on object level could be improved
• Helper libraries to access Red Hat Ceph Storage from various languages could be improved
• Ability to attach structured metadata to stored objects could be improved

Incentivized

• Documentation; using readthedocs demonstrates that the Gluster project isn't always kept up-to-date as far as documentation is concerned. Many of the guides are for previous versions of the product and can be cumbersome to follow at times.
• Self-healing; our use of GFS required the administrator to trigger an auto-heal operation manually whenever bricks were added/removed from the pool. This would be a great feature to incorporate using autonomous self-healing whenever a brick is added/removed from the pool.
• Performance metrics are scarce; our team received feedback that online RDBMS transactions did not perform well on distributed file systems (such as GFS), however this could not be substantiated via any online research or white papers.

Incentivized

VSAN (Virtual SAN) and Ceph are both software-defined storage solutions, but they have some key differences in terms of their architecture and capabilities.VSAN is a software-defined storage solution that is built into the VMware vSphere hypervisor. It allows organizations to create a shared storage pool using locally attached storage on multiple ESXi hosts. VSAN is designed to be highly available, and it can automatically detect and recover from hardware failures.Ceph, on the other hand, is an open-source software-defined storage solution that can run on a variety of different hardware and virtualization platforms. It provides object, block, and file storage in a single platform, and is designed to be highly scalable and highly available. Ceph is also known for its ability to handle large amounts of data, and it can be integrated with a wide variety of different applications and services.In terms of functionality, VSAN is more suited for virtualized environments, as it is built into vSphere and it is designed to work well with vSphere's other features such as vMotion and DRS. Ceph on the other hand provides more flexibility as it can run on multiple platforms and it can handle more types of storage like object, block and file storage.I

Incentivized

Gluster is a lot lower cost than the storage industry leaders. However, NetApp and Dell/EMC's product documentation is (IMHO) more mature and hardened against usage in operational scenarios and environments. Using Gluster avoids "vendor lock-in" from the perspective on now having to purchase dedicated hardware and licenses to run it. Albeit, should an organization choose to pay for support for Gluster, they would be paying licensing costs to Red Hat instead of NetApp, Dell, EMC, HP, or VMware. It could be assumed, however, that if an organization wanted to use Gluster, that they were already a Linux shop and potentially already paying Red Hat or Canonical (Debian) for product support, thereby the use of GFS would be a nominal cost adder from a maintenance/training perspective.

Incentivized

• Cost saving in terms of capex (you can reduce EMC & NetApp like dependency)
• One time setup & then easy provisioning of storage
• Requirement of competent engineers for maintenence

Incentivized

• Positive - Alignment with the open source community and being able to stay abreast of the latest trending products available.
• Positive - Reduced procurement and maintenance costs.
• Negative - Impacts user/system maintainer training in order to teach them how to utilize and troubleshoot the product.

Incentivized