preload
Dec 28

One issue with zfs send over network is that the documentation available from Oracle only shows examples with ssh. This is not the best case if one is dealing with large datasets. It is very slow over and does not applicable to 10 Gigabit networks.

I have noticed that mbuffer is very capable to deliver high speeds on zfs send. It works client based and needs mbuffer client on sending and receiving end.

Set up receiver to accept connection from senders ip-address 192.168.1.100:

# mbuffer -s 128k -m 1G -I 192.168.1.100:50001 | zfs receive tank/backup/data

mbuffer listens on port 50001 and uses 1 Gigabyte of memory to buffer incoming data. Blocksize is set to 128k which is default on ZFS.

Send the data to client on address 192.168.1.10:

# zfs send tank/nfs/data@today | mbuffer -s 128k -m 1G -O 192.168.1.10:50001

One should see significant increase of speed to zfs send. If the dataset is busy with bursty writes, mbuffer will hold it on its own buffer on both ends.

Remember that datastream is not crypted and this can be issue on some cases. So use dedicated networks for sending snapshots with mbuffer.

mbuffer can be found here