Message-Id: <199709111606.JAA09667@mailhost2.cac.washington.edu> From: "James W DeRoest" <deroest@cac.washington.edu> To: "AST Webpage" <astweb@u.washington.edu> Subject: 100MB quotas Date: Thu, 11 Sep 1997 09:05:47 -0700
-----Original Message----- From: Ken Lowe <ken@cac.washington.edu> Date: Wednesday, August 13, 1997 6:53 PM Subject: 100MB quotas
I have some code written up that is almost ready to be installed to support 100MB quotas. If nothing else, read "unresolved issues" below.
In the current scheme, each galaxy has a list of filesystems in its /sy99/dat/idf.dat that describes where home directories may be placed and what disks have quotas.
x) The nightly adjquota does:
for (user = root; user < max_uid; user++) { for (disk = first_disk; disk < max_disk; disk++) { get_quota(user,disk); } massage_quotas(); for (disk = first_disk; disk < max_disk; disk++) { put_quota(user,disk); } }
on each galaxy.
x) Hourly, while processes for a particular user are running, the watcher/helper processes do an "audit" that does:
rmf.curdisk = 0; for (disk = first_disk; disk < max_disk; disk++) { get_quota(user,disk); rmf.curdisk += quota.usage; }
Nightly this process is run for all accounts that have not been updated recently (72 hours).
x) Nightly the r2 utility adds up the fields in the cdf records for each user and puts the total allocation into the rmf "threshold" fields.
x) The fnd utility runs on each file server periodically (weekly) and builds an f_list file in the root of each native filesystem listing all the files and who owns them. This file is used by account deletion.
In my new scheme, I've taken the filesystem definitions out of the idf and put them into a new file, /sy99/dat/qdf.inf, on one "qdf server". My current test qdf server is horton where you can find a sample file. This file describes all the filesystems for all the galaxies.
There's also a binary file /sy99/dat/qdf.dat that contains a matrix of users vs disks. One row for each user, one column for each filesystem. The fnd utility is modified to update the "user has files on this disk" bit in the matrix for each of its native disks. The matrix holds a full byte for each user, but currently only one bit is designated. The bit is also set by new and newdir when a directory is created. When the user has no files on a filesystem the fnd utility will clear the bit.
Another binary file, /sy99/dat/qlf.dat, defines the disk limits for each user for each galaxy. Each user has N limits. A global limit defines how much disk space the user can use across all galaxies. The remaining limits indicate how much the user can use on any particular galaxy (each filesystem's usage is attributed to a particular galaxy as defined in the qdf.inf file). The qlf columns are set each evening by r2.
Adjquota will now get the user's row from the qdf file and only get and set quotas on the disks that the user actually has files. This should speed things up considerably (the average saul user only has files on one /uaXX disk but we adjust the quotas on all 46 of them). Similarly the audit routine will only fetch the quotas for those disks where the user listed as having files.
The global quota for each user is set to the maximum galaxy quota. Thus if the CDF says you get 100MB on homer and 20MB on saul, you'll get a 100MB total, 20MB of which you could have on saul. You could have up to 100MB on homer as long as the sum of your homer and saul usage does not exceed the 100MB limit. As a phase-in, I've set up a "Grandpa" galaxy in the qlf that will initially be set to the sum of all your CDF records across all galaxies. If you currently have 20MB on homer and 20MB on saul you'll get 40MB on "Grandpa" which will be your global limit as well. When we bump the homer limits to 100MB these grand-fathered entries can disappear.
User noticable differences:
x) Adjquota and assets -q will now show all filesystems where the user has files whether they're native and mounted or not. Assets -q does a good job of supplying sub-totals, but adjquota may produce some questions for client-services as it displays things in the order they appear in the qdf.inf file and it will be difficult to determine why/where the soft quotas are coming from (not that it ever was easy). See example for noyd below.
x) Assets -q no longer shows chkout disk space. I could add that back in if it's desired. The quota command still only shows locally mounted disks so it will show the chkout space (I could change it, too).
x) Adjquota used to set the hard limit to 1.2 times the threshold. This was a fudge factor to allow you to get a warning prior to hitting the hard limit (if you didn't have any usage on any other disk your soft limit would be your threshold). I modified this to only multiply by 1.2 if in fact your soft and hard limits would otherwise be equal (see /hk1 in the example below).
x) A mailing list loop on homer may now affect the ability to use saul and/or a runaway process on saul may prevent mail delivery on homer.
x) ???
Unresolved issues:
x) Becker and hawking don't have back doors. The aesop and guido file servers don't have front doors. An adjquota (or assets -q) command on becker won't be able to access all the servers to fetch and set your quotas. One option is to find another interface for hawkingb and use it to route the becker traffic to these servers (becker and hawking's front doors happen to be on the same subnet). I'm hoping this can be resolved while I'm on vacation. Doug?
x) ??
-Ken
-----
#melville> pwd /tulsa/obj/dev/ken/aixv4/sy99/watcher #melville> ./assets -mq noyd
Acct 2B314469 Host: melville User: noyd Steven Jones
Maximum disk allowance for uid 4001: Galaxy Megabytes ------ --------- Global 810.0 Homer 50.0 Dante 100.0 Stein 50.0 Becker 10.0 Hawking 500.0 Aagaard 100.0 Grandpa 810.0
------------------------------------------------------------
Disk usage for uid 4001: Filesystem Megabytes Files Comment ---------- --------- ----- ------- /mailer2 0.113 8 Homer/Deskmail INBOXes /rc21 31.230 562 Homer Cluster Files /rc27 0.002 1 Homer Cluster Files ---------- --------- ----- Total: 31.346 571 Maximum: 50.000
/bp01 14.352 9 Dante/Deskmail INBOXes /da03 0.107 22 Dante Cluster Files ---------- --------- ----- Total: 14.459 31 Maximum: 100.000
/sy29 0.000 1 Saul Mail INBOXes /sy36 0.004 1 Mead/Goodall Mail INBOXes /sy35 0.004 1 Hardy Mail INBOXes /ua10 0.008 2 User Files and Directories /ua21 2.570 60 User Files and Directories /sy31 0.219 2 Anonymous FTP Files, /usr/ftp /www01 2.211 100 WEB pages ---------- --------- ----- Total: 5.016 167 Maximum: 50.000
/cl01 0.003 1 Becker Mail INBOXes /bk01 7.525 11 Becker Cluster Files ---------- --------- ----- Total: 7.528 12 Maximum: 10.000
/hk1 0.021 13 Hawking Cluster Files ---------- --------- ----- Maximum: 500.000
---------- --------- ----- Grand Total: 58.369 794 Maximum: 810.000
#melville> ./adjquota -q noyd Adjusting quotas for uid 4001. Threshold = 810.00Mb. Filesystem usage quota limit grace files quota limit /mailer2 0.113 18.768 1000.000 8 10000 20000 /bp01 14.352 99.893 1000.000 9 10000 20000 /sy29 0.000 44.984 50.000 1 10000 20000 /cl01 0.003 2.475 10.000 1 10000 20000 /sy36 0.004 44.988 50.000 1 10000 20000 /sy35 0.004 44.988 50.000 1 10000 20000 /rc21 31.230 49.885 50.000 562 10000 20000 /rc27 0.002 18.656 50.000 1 10000 20000 /da03 0.107 85.648 100.000 22 10000 20000 /ua10 0.008 44.992 50.000 2 10000 20000 /ua21 2.570 47.555 50.000 60 10000 20000 /bk01 7.525 9.997 10.000 11 10000 20000 /hk1 0.021 500.000 600.000 13 10000 20000 /sy31 0.219 45.203 50.000 2 10000 20000 /www01 2.211 47.195 50.000 100 10000 20000