I recently went looking to see what sort of open source scalable filesystem projects existed.  I wanted to see about putting together a storage solution that would scale to upwards of 100 TB using open source software and commodity hardware.  During the search I became reacquainted with the GluserFS project.

I had configured a 3 brick ‘unify’ cluster a while back with one of their 1.3.x builds, however I had not gotten an opportunity to play with it much more after that.

After looking at the various other options out there, spending a considerable amount of time on IRC and reviewing the contents of their mailing lists, I ended up settling on GlusterFS due to it’s seemingly simple design, management, configuration and future roadmap goals.

As it turns out a few days after I started my search, the gluster team released version 2.0 of their software.  At this point I have setup a 5 brick ‘distribute’ (DHT) cluster on a few of our Proxmox (OpenVZ) servers.

I now have 5 independent 4GB bricks and a 20GB mountpoint it representes to the client.  In this case I am currently exporting  CIFS (Samaba) on top of the gluster mountpoint.  I found some very useful instructions on setup, etc here.  I plan to test NFS as well at some point on some real physical hardware, due to current OpenVZ limitations on NFS servers inside of a container.

One thing I was unable to get working at this point is to have the glusterfs client and server running on the same machine.  The single client/server setup worked flawlessly on my Ubutu laptop, so I suspect that is just an OpenVZ issue that I need to work out.