return to is this even a language?

Setting up NFS Caching in Debian

13 Oct 2013

Returning the favor and documenting this.

This post describes all necessary steps to enable NFS caching on Debian Wheezy. You can find the bits and pieces elsewhere, but as far as I know there is no complete tutorial.

Setting up the cache storage

You can use your main disk. However the cache daemon only allows limits in remaining space percentage which makes it impossible to set aside, say, 1 GB for NFS caching. For that reason you need to setup a seperate mount. You can either repartition your drive or use a loopback device, like I did:

dd if=/dev/zero of=/var/cache/loopback.img bs=1M count=1000
mkfs.ext3 -m 0 /var/cache/loopback.img

mkfs will complain about the file not being a block device, but it doesn’t matter. Most likely you want to add this to your /etc/fstab to automatically mount it on boot:

/var/cache/loopback.img  /var/cache/fscache  ext3  loop,auto,defaults,user_xattr  0  0

Note the user_xattr here. This is required for the cache, which will not work otherwise but give no hint about it.

Next, mount your new fstab entry:

rm -rf /var/cache/fscache    # optional
mkdir /var/cache/fscache     # optional
mount /var/cache/fscache

Installing Cache-FS Daemon

apt-get install cachefilesd

Edit /etc/default/cachefilesd and comment in RUN=yes. If you would like to adjust the cache limits, edit /etc/cachefilesd.conf. Don’t forget to restart the service once done:

service cachefilesd restart

Instructing NFS shares to use the cache

Simply add fsc as mount option to your NFS share. This works fine in fstab, using mount or in your autofs setup files. Here’s an example from my autofs config:

backup   -fsc,rw,hard,intr,nonempty,sync  192.168.2.42:/store/backup

Obviously, you’ll need to remount or restart autofs.

Testing if it works

$ mount
/var/cache/loopback.img on /var/cache/fscache type ext3 (rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered)
…

$ ls -lh /var/cache/fscache
total 24K
drwx------ 3 root root 4,0K 2013-10-13 12:50 cache
drwx------ 2 root root 4,0K 2013-10-13 12:30 graveyard
drwx------ 2 root root  16K 2013-10-13 12:26 lost+found

$ cat /proc/fs/fscache/stats
FS-Cache statistics
Cookies: idx=9 dat=368 spc=0
Objects: alc=302 nal=0 avl=302 ded=273
ChkAux : non=0 ok=0 upd=0 obs=0
…

$ cat /proc/fs/nfsfs/servers
NV SERVER   PORT USE HOSTNAME
v4 0a135c32 801   1 192.168.2.42


$ cat /proc/fs/nfsfs/volumes
NV SERVER   PORT DEV     FSID              FSC
v4 0a135c32  801 0:35    0:0               yes

Enjoy your improved speed:

$ ls -hs spatulamadness.mov
107M spatulamadness.mov

$ time cp spatulamadness.mov /tmp
cp spatulamadness.mov /tmp  0,00s user 0,14s system 0% cpu 53,936 total

$ time cp spatulamadness.mov /tmp
cp spatulamadness.mov /tmp  0,00s user 0,12s system 5% cpu 2,105 total