








                   Disk Quotas in NetBSD


                      David A. Holland
        Based upon an earlier version by Robert Elz.


                          ABSTRACT

          In most computing environments, disk space is
     not infinite.  The disk quota  system  provides  a
     mechanism  to  control  usage  of disk space on an
     individual basis.

          Quotas may be set for each  individual  user,
     and  for  groups.   Quotas may be set (or not set)
     independently on each mounted file system.  Quotas
     are not supported on all file system types, but do
     work over NFS.

          The quota system will warn  users  when  they
     exceed  their allotted limit, but allow some extra
     space for current work.  Remaining over quota  for
     long periods of time will eventually cause a fatal
     over-quota condition.

          File system independent access to  quotas  is
     provided via a library, libquota(3), shared by the
     quota tools and available to third-party software.
     Dump and restore tools are provided for backing up
     quota information.


1.  Users' view of disk quotas

     To an ordinary user, having a disk quota is almost  the
same  as  having  a  smaller disk.  When the amount of space
available is consumed, further attempts  to  use  more  will
fail.

     To  make  this  less painful, the quota system actually
provides two limits for every resource: a hard limit  and  a
soft  limit.   The hard limit may not be exceeded.  The soft
limit may be exceeded temporarily: when the  soft  limit  is
first exceeded, a timer is initialized.  If usage is reduced
below the soft limit, the timer is cleared.  Otherwise, when
the  timer  expires,  the soft limit becomes the hard limit:
until usage is reduced below the soft limit, further use  of
space  is  denied.  The length of time allowed is called the
grace period and is configurable by the  system  administra-
tor.









SMM:4-2                                Disk Quotas in NetBSD


     Quotas  may  be applied to both users and groups.  When
both a user quota and a group quota apply, the most restric-
tive is used.

     The basic purpose of quotas is to restrict the usage of
disk space.  This is measured in  blocks.   (For  historical
reasons,  quotas  are  counted in terms of 512-octet blocks,
even though most file systems use larger blocks internally.)
Quotas  can  be imposed on other resource types too.  Gener-
ally there will also be a quota on the number of files  that
may be owned.  This functionality is supported by nearly all
file system types that support quotas  at  all.   Some  file
system  types  may  support  additional quotas on other file
system resources.

     The quota(1) command provides information on  the  disk
quotas applying to a user or group.  For each resource type,
and each file system, it prints the current usage, the  soft
limit  (`quota'),  the hard limit (`limit'), and the expira-
tion time of the current grace period, if any.  User  quotas
are reported by default; the -g flag allows retrieving group
quotas.

     The soft limit is the usage  level  that  the  user  is
expected  to  remain  below.   As  described above, the soft
limit can be exceeded  temporarily;  doing  so  generates  a
warning.  The hard limit cannot be exceeded.  When the usage
exceeds the applicable limit, further requests for space (or
attempts to create a file, or allocate other resources) will
fail with the EDQUOT error.  The first time this  occurs,  a
message  will  be  written to the user's terminal.  To avoid
flooding, the message will not be repeated until  after  the
usage is lowered again.

1.1.  Surviving when quota limit is reached

     In  most  cases,  the only way to recover from an over-
quota  condition  is  to  stop  whatever  activity  was   in
progress,  remove  sufficient  files to bring the limit back
below quota, and retry the failed program.

     Be careful not to exit applications whose document save
operations have or may have failed because of the over-quota
conditions.  This can often lead to losing data:  the  prior
saved version of the document may have been replaced with an
empty, partial, or invalid half-saved version.  If it is not
possible to open an additional terminal or other tools to do
the clean-up work, use job control to suspend these applica-
tions  rather than exiting.  It may also be possible to save
documents to a different file system (perhaps in  /tmp)  and
then move them back into place after cleaning up.












Disk Quotas in NetBSD                                SMM:4-3


2.  Back-end quota implementations

     In NetBSD there are three categories of quota implemen-
tations.  First are file systems with native  internal  sup-
port  for quotas.  In these, the quota data is stored inter-
nally within the file system, maintained by  the  file  sys-
tem's  own  consistency  management and fsck(8) utility, and
active whenever the file  system  volume  is  mounted.   The
mechanism  for  setting  up quotas on a particular volume is
file-system-dependent but generally  involves  arguments  to
newfs(8)  and/or  tunefs(8).   The ``quota2'' quotas for FFS
that appeared in NetBSD 6.0 are an example of this type.

     The second category is NFS.  In NFS, quotas are handled
by  a separate SunRPC protocol not directly connected to the
NFS mount.  The quota utilities  speak  this  protocol  when
invoked  on  NFS  volumes.   On an NFS server, the rpc.rquo-
tad(8) daemon must be enabled to serve this  protocol.   See
below for further details.

     The third category is instances of the historical quota
system, which  was  developed  in  the  1980s  and  remained
largely  unchanged  until being replaced during the develop-
ment of NetBSD 6.0.  In the historical quota system,  quotas
are  stored  in external (user-visible) files.  These appear
by default in the root directory of the  volume  they  apply
to,  but  may  be  placed  anywhere,  including sometimes on
another file system.   While  file-system-level  support  is
required,  and  enforcement is done in the file system, much
of the work involved in  maintaining  the  quota  system  is
external  to  the file system.  The program quotacheck(8) is
run at boot time as a form of fsck(8) for the quota informa-
tion.   Once this is done, the program quotaon(8) is used to
activate the quotas.  The program  quotaoff(8)  is  used  to
deactivate  the quotas during system shutdown.  This is nor-
mally done before unmounting; if not, unmounting will do  it
implicitly.   If  the  file  system volume is modified after
quotaoff is run, quotacheck must be run  before  quotaon  is
run again, or the quota data will become corrupted.  Setting
up the historical quota system requires several  steps;  see
below.   The  original  or  ``quota1'' quotas for FFS are an
example of this type of quota implementation.

     The libquota(3) library provides a common interface  to
all  three  implementation types, so the same quota adminis-
tration tools can be used with all of them.  Not  all  func-
tionality  is available in all cases, however.  For example,
NFS quotas are read-only.  To adjust quotas on an  NFS  vol-
ume, one must log into its file server.

3.  Setting up quotas

     To  set  up  quotas of any type requires several steps.
These are described in detail, for  each  file  system  type









SMM:4-4                                Disk Quotas in NetBSD


supporting quotas, in the following sections.

     The first step in all cases, however, is to decide what
file systems need to have quotas.  Typically, only file sys-
tems  that  house  users'  home  directories,  or other user
files, will need quotas.  Do not forget the mail spool.   It
may  also  prove  useful to also include /usr.  If possible,
/tmp should preferably be free of quotas.

     It is also necessary to decide what the quotas will be,
keeping in mind the available space and the legitimate needs
of the user population.

     Note that because the quota system tracks per-user  and
per-group  disk  usage, it can be useful in some contexts to
enable quotas purely to provide  that  information.   It  is
perfectly reasonable in this case to set quotas that are the
same size as, or larger than, the  space  available  on  the
volume.

3.1.  Setting up new (``quota2'') FFS quotas

     First,  make  sure  your  kernel  includes  FFS and FFS
quota2 support.  These are present if the lines

     file-system  FFS
     options      QUOTA2

(respectively) are present in the kernel configuration.   In
NetBSD 6.0 and higher the GENERIC kernels should all include
FFS and FFS quota2 support.  If FFS is compiled as  a  load-
able  module, the module will always include quota2 support.

     Then, when creating the volume, use the -q user  and/or
-q  group  options to newfs(8) as desired.  This will create
and initialize the on-disk data structures for  user  and/or
group  quotas respectively.  These can also be created using
tunefs(8).

     There is no need to set mount options in fstab(5).   In
particular,  do  not set the userquota or groupquota options
as these are used for controlling the historic quota  system
only.  Trying to use both quota systems at once will fail in
possibly obscure ways.

3.2.  Setting up quotas on an NFS client

     There is nothing special that needs to be done  on  NFS
clients.   Enforcement  (and configuration) of quotas occurs
on the NFS server.  The libquota(3) library  recognizes  NFS
mounts and automatically uses the rquotad protocol to handle
them.











Disk Quotas in NetBSD                                SMM:4-5


3.3.  Setting up quotas on an NFS server

     To set up quotas on an NFS server, first set up  quotas
on  the  file  systems  that  will  be served via NFS.  Then
enable  the  rquotad(8)   service   in   /etc/inetd.conf(5).
Restart (or start) inetd(8) if necessary.  Make sure

     inetd=YES

is  present in /etc/rc.conf.  This is the default and should
not need to be changed explicitly.

3.4.  Setting up historical (``quota1'') FFS quotas

     First, make sure  your  kernel  includes  FFS  and  FFS
quota1 support.  These are present if the lines

     file-system  FFS
     options      QUOTA

(respectively)  are present in the kernel configuration.  In
all NetBSD versions the GENERIC kernels should  include  FFS
and  FFS  quota1  support.  If FFS is compiled as a loadable
module, the module will always include quota1 support.

     Note that it is possible that the historic quota system
will  be removed entirely at some point in the future.  This
is not presently intended, however.

     No special options are required when creating the  file
system.    Instead,  add  the  userquota  and/or  groupquota
options to the file system's entry in  fstab(5).   The  file
system  must  be listed in fstab(5) for the historical quota
system to work.  To use quota files other than the  default,
use      the     form     userquota=/path/to/file     and/or
groupquota=/path/to/file as desired.  The default files  are
quota.user and quota.group respectively.  Create empty quota
files with touch(1).

     If the file system is  not  brand  new,  now  run  quo-
tacheck(8) on it.  (The file system must be mounted for this
step.  Be sure nothing else is writing to  the  file  system
during this time.)

     Now run quotaon(8) on the file system.

     You must also make sure that the setting

     quota=YES

is  present in /etc/rc.conf.  This is the default and should
not need to be changed explicitly.  This setting causes quo-
tacheck(8) and quotaon(8) to be run at system boot time.










SMM:4-6                                Disk Quotas in NetBSD


3.5.   Setting  up  historical  quotas  on other file system
types

     In theory, the historical quota system can also be used
on  LFS  and ext2fs file systems.  The procedure for this is
the same as for FFS.

     There is definitely at least some code present to  sup-
port  quotas  on  LFS;  however,  as  of  this writing it is
believed that quotas do not actually work with either LFS or
ext2fs.

4.  Quota administration

     After  the  quota system has been set up, and also on a
continuing basis as users come and go and their requirements
change, the actual limit values need to be configured.  This
is done with the edquota(8) command.

     In cases where large classes of users are to  be  given
the  same quota settings, the -p option of edquota(8) can be
used to streamline the process.

     There is also a default entry.  Users (or groups) whose
quotas  have  not  been  set explicitly are subjected to the
default quotas instead.

     The quota(1) command can be used to inspect an individ-
ual  user  or  group's  quotas across all file systems.  The
repquota(8) command can be used to  retrieve  quotas  across
all users or groups on a per-file-system basis.

4.1.  Administrative considerations for native quotas

     The quotacheck(8), quotaon(8), and quotaoff(8) programs
cannot be used with native quotas.  They will fail if run.

     To avoid confusion be sure not to use the userquota  or
groupquota  options in fstab(5) on a file system with native
quotas.  Due to implementation quirks of the historic  quota
system,  using  these  options  will  not  necessarily cause
errors at mount time, but may confuse  quotacheck(8)  and/or
quotaon(8) and produce bizarre results.

     Native quotas do not require quotacheck(8) and thus can
be much faster at boot time.  In particular, the native  FFS
quotas  used  in conjunction with WAPBL journaling are them-
selves journaled and require only a journal replay  after  a
crash.

     Note  however  that native FFS quotas are not backward-
compatible to older NetBSD installations.  As of this  writ-
ing  they are also not understood by FreeBSD's FFS implemen-
tation.









Disk Quotas in NetBSD                                SMM:4-7


     There  is  currently  no  way  to  temporarily  suspend
enforcement of native quotas.

4.2.  Administrative considerations for NFS quotas

     Most  quota administration should (must) be done on the
NFS server by working with the volumes  being  served.   The
rquotad  protocol is very limited and supports only the most
basic operations.  Notably, repquota(8) does not work on NFS
volumes.

4.3.  Administrative considerations for historic quotas

     The  historic  quota system does not support all of the
possible functionality.  There is no separate default entry,
and  the  grace  period  cannot  be configured individually;
instead, one grace period is configured for all users.

     In the historic quota system the default values and the
global  grace  period  are stored in the quota entry for UID
(or GID) 0.  This scheme partly shows through to  the  quota
tools.   It  is  also possible to attempt to establish quota
configurations that cannot be represented.  These will fail.
edquota(8)   attempts  to  detect  these  before  submitting
changes to the kernel in order to offer a cogent error  mes-
sage; however, it may not always succeed.

     Before  quotaon(8) is run, the quota information is not
accessible to the kernel.  The libquota(3)  library  detects
this  case  and  falls  back  to  direct access to the quota
files.  This should be fully transparent but it is  possible
that glitches may arise.

     It is possible to temporarily disable quota enforcement
by using quotaoff(8).  However,  this  also  disables  usage
tracking.  Consequently, if this is done while the system is
live, it is in general necessary  to  run  quotacheck(8)  to
correct  the  usage  information  before  running quotaon(8)
again, and the file system must be  idle  between  the  time
quotacheck  is started and the time quotaon completes.  Oth-
erwise the usage information in the quota files will go  out
of  sync  with  the  file system.  This can lead to improper
behavior later on, and in some cases may cause panics.

     The historical quota system is 32-bit and  thus  cannot
cope with quotas or usage amounts that cannot be represented
in a 32-bit value.  Use native quotas on large volumes.

5.  Backups

     While the disk usage information in the quota data  can
be  reconstructed  by scanning the file system (this is what
quotacheck does), the configured quotas themselves are  sys-
tem configuration that should in general be backed up.  With









SMM:4-8                                Disk Quotas in NetBSD


the historical quota system, the quota information is stored
in  regular  files  and  is backed up in the normal way like
other files.  However, with native quotas, the quota  infor-
mation is hidden inside the file system and additional steps
are necessary to back it up.

     The quotadump(8) and quotarestore(8) programs are  pro-
vided for this purpose.

     The  quotadump(8)  program  is  the same as repquota(8)
with the -x option.  It produces a complete tabular dump  of
the  quota  settings on the selected file system.  This dump
file can be saved on backup media.

     The quotarestore(8) reads a dump file produced by  quo-
tadump(8)  and  enters the configured quota information into
the selected file system.  It can be used  to  restore  from
backup, or to migrate quotas from one volume to another.

6.  Migrating to the new native FFS quota implementation

     The   procedure   for  migrating  from  the  historical
(``quota1'') FFS quotas to the native (``quota2'') FFS  quo-
tas is as follows.

     First,  make  sure  the  volume being migrated is fully
backed up.  This is important in case something goes  wrong.

     Now, drop to single user mode.  Then, dump the existing
quota information with quotadump(8).   Save  the  dump  file
someplace  safe, i.e. not in /tmp, in case it becomes neces-
sary to reboot.

     Unmount the volume.  Edit  /etc/fstab  and  remove  the
userquota  and/or  groupquota  options.  Leave the old quota
files in place for now; they will do no harm.

     Use tunefs(8) to add native quotas to the file  system.
If the volume is very old you might first need to update the
superblock.

     Mount the file system.  Use quotarestore(8) to load the
dump file into the new quota system.

     Confirm  using  repquota(8)  and/or  quota(1) and/or by
explicit testing that the quotas have been  loaded  and  the
new  quota  system  is  behaving  as intended.  If paranoid,
reboot and test again as a precaution.

     Once you are fully satisfied, preferably  after  a  few
days'  usage,  delete or archive the old quota files and the
dump file used for transition.











Disk Quotas in NetBSD                                SMM:4-9


     Remember to set up  backup  procedures  for  the  quota
data.

7.   Summary  of changes from the historic to new/native FFS
quota implementations

     Quotas are set up with newfs or tunefs, rather than  by
editing fstab and running quotacheck.

     The  quotacheck,  quotaon,  and  quotaoff tools are not
used.  Repair is done with fsck instead.

     The quotas are integrated with WAPBL journaling, allow-
ing fast crash recovery.

     The  userquota  and  groupquota  mount  options are not
used.

     The grace period is per-user instead of global.

     The defaults do not overlap with the id 0 quota  entry.

8.  Some implementation details for the historic quotas

     The data in the quota files is an array of dquot struc-
tures, indexed by id (UID or GID).  There is  an  entry  for
every id on the system, whether or not a quota is configured
for that id.  If the id space is sparse, then the  file  may
have holes in it.  Copying the files will fill in the holes,
so it is best to avoid this.

     The  userquota  and  groupquota  options  are  actually
ignored  by mount(8).  They are instead found at run time by
the quota tools (actually by libquota);  this  is  why  file
systems  using historic quotas must be listed in /etc/fstab.

     The kernel is informed of the existence and  identities
of  the  quota  files by the quotaon(8) utility.  Until this
point (e.g. while quotacheck is running) the kernel does not
know which volumes are supposed to have quotas and which are
not.

     When quotas are turned on, the kernel reads  the  quota
entries  for each user (or group) currently active, and then
the quota entries needed for any files open owned  by  users
(or  groups)  who are not currently active.  Each subsequent
open of a file on the file system will be accompanied  by  a
pairing  with  its  quota  information.   In most cases this
information will be retained in  core,  either  because  the
user  who  owns  the  file  is running some process, because
other files are open owned by the same user, or because some
file  (perhaps  this one) was recently accessed.  In memory,
the quota information is kept hashed by id and file  system,
and  retained  in an LRU chain so recently released data can









SMM:4-10                               Disk Quotas in NetBSD


be easily reclaimed.  Information about  those  users  whose
last  process  has  recently  terminated is also retained in
this way.

     Each time a block is accessed  or  released,  and  each
time  an  inode is allocated or freed, the quota system gets
told about it, and in the  case  of  allocations,  gets  the
opportunity to object.

9.  History and acknowledgments

     The historic quota system was loosely based upon a very
early scheme implemented at  the  University  of  New  South
Wales,  and  Sydney  University in the mid 70's. That system
implemented a single  combined  limit  for  both  files  and
blocks on all file systems.

     A  later  system  was  implemented at the University of
Melbourne by Robert Elz, but was not kept highly accurately.
For  example,  chown did not affect quotas, nor did I/O to a
file owned by a different user.

     The historic quota system was put into place in January
1982 at Melbourne.

     For  the 4.2BSD release, much work was done to clean up
and sanely incorporate the quota code  by  Sam  Leffler  and
Kirk McKusick at The University of California at Berkeley.

     The  historic  quota  system  has remained in use (with
only minor modifications) through many BSD releases and ver-
sions  from then right up until 2011, and remains available.

     In 2011, Manuel Bouyer implemented a new  quota  system
for  FFS  with  the properties described above (in-FS, jour-
naled, etc.)   and  reworked  the  kernel  interface.   That
interface  was  later  withdrawn.   In 2012 David A. Holland
implemented a simpler kernel interface and  more  comprehen-
sive  quota library and reworked the quota tools to be file-
system-independent.  This material first appeared in  NetBSD
6.0.



















