NFS Shared Storage

Quick Reference

Linux
Storage
Published

May 14, 2024

NFS (Network File System) …network-mounted file-sharing system

# install packages
sudo dnf install nfs-utils

# start the server
sudo systemctl enable --now nfs-server rpcbind

# open the firewall
sudo firewall-cmd --add-service={nfs,nfs3,mountd,rpc-bind} --permanent 
sudo firewall-cmd --reload

/etc/exports

Or /etc/exports.d/*.exports…

  • …controls which file systems are exported to remote hosts
  • …comments use hash mark # …line-wrap with \

One NFS share per line:

/exported/directory 192.168.*(rw,async) node*.example.org(ro)
  • /path …absolute path to the shared directory
  • Followed by a list of authorized hosts…
    • FQDN or IP address …network addresses …* for everybody
    • …multiple hosts separated by space
  • Options
    • ro read-only (default) …remote can not change data
    • rw read-write
    • sync waits for write requests to be finished before response
    • async asynchronous writes
    • root_squash prevents root users connected remotely from having root privileges
# modifications to exports requires a re-export to take effect
exportfs -arv

# view current loaded exports
exportfs -v

Client

Mount an NFS share:

# list available shares on a server
>>> showmount -e nfs01
Export list for nfs04:
/srv/nfs4 *.example.org

# mount a shared-directory from a server
>>> mount -t nfs4 nfs01:/srv/nfs /mnt/nfs

# remove an NFS share
>>> umount /mnt/nfs

nfsiostat

Reads from /proc/self/mountstats…

# NFS I/O statistics for a given mount point
nfsiostat /scratch/hpc 1
Statistic Description
ops/s number of operations per second
rpc bklog length of the backlog queue
kB/s number of kB written/read per second
kB/op number of kB written/read per each operation
retrans number of retransmissions
avg RTT (ms) duration …client’s kernel sends RPC until it receives a reply
avg exe (ms) duration …NFS request until completion (includes RTT)

mountstats

mountstats displays various NFS client per-mount statistics…

  • …for each NFS operation:
    • Number of times operation has been called and number of errors
    • Bytes send/received per operation
    • RTT Round-Trip Time in milliseconds
    • Total execution time in milliseconds
  • Complete list of NFS operations1 in RFC7530
# statistics for a specific mount point
mountstats /scratch/hpc | less

Example output:

#...
LOOKUP:
        256 ops (0%)    254 errors (99%)
        avg bytes sent per op: 215      avg bytes received per op: 97
        backlog wait: 1.835938  RTT: 1.429688   total execute time: 3.292969 (milliseconds)
#...
OPEN:
        126 ops (0%)    26 errors (20%)
        avg bytes sent per op: 321      avg bytes received per op: 320
        backlog wait: 0.047619  RTT: 1.468254   total execute time: 1.555556 (milliseconds)
#...

Footnotes

  1. RFC7530, NFSv4 Operations
    https://www.rfc-editor.org/rfc/rfc7530#page-214↩︎