Technical Note - Disk Quotas

Summary

Student and guest ECS accounts have a disk quota associated with them. This document outlines what the quotas are, what happens when you go over your quota and how to keep your usage below your quota.

Details

Quota Sizes

The current default disk quotas on ECS systems are:

	1st-3rd Year	4th Year+	Guests
Disk Quota	512MB	1GB	1GB

Exceeding your Quota

When you exceed your disk quota you will receive an email telling you how much over your quota you are. You should reduce your usage to below your quota within a few days otherwise administrative action, such as temporary suspension of your login account, make be taken. If that happens, to get it reactivated you will need to see one of the school's system administration team to explain your disk usage and why you didn't reduce it.

Requesting a Larger Quota

In general requests for larger disk quota from undergraduate students will not be granted. If we feel that the current quota limits are unreasonably impacting on undergraduate use of our systems we will increase the quota for all undergraduate students.

For graduate students, if your project or thesis work requires a larger disk usage, you can ask your supervisor to request this. Similarly, guest users should ask the school staff member who arranged for them to have an ECS account. The request should be made by emailing jobs@ecs.vuw.ac.nz. The school system administrators may wish to talk to you before increasing your quota to ensure that you are using disk resources in a "sensible" way and also to make sure that there are not alternative options (ie: see the following section describing /local/scratch). But if there is a genuine need for a larger quota it should be no problem to grant it.

Using /local/scratch on NetBSD Workstations

All of the ECS NetBSD workstations have a large area (typically 30-40GB) of temporary ("scratch") disk available for use by the person using that workstation. Files stored in /local/scratch do not count towards your disk quota so storing files there may be an alternative to requesting a larger disk quota. There are some advantages to doing this, but there are also disadvantages that you should be aware of.

The main advantage is the large amount of free space that is typically available. Also the disk is local to the workstation so accessing it is much (probably several orders of magnitude) faster than a network volume such as the one your home file system is stored on.

Conversely, because the disk is local to each workstation it is only available to programs running on that workstation. That means that if you don't mostly use the same workstation /local/scratch may not be very convenient. Also, be aware that the contents of /local/scratch on our lab workstations may be removed at any time without notice. We would usually check with the owners of files in /local/scratch on workstations in staff or graduate offices before doing anything with them.

A related issue is that even if you do always use the same workstation, you may have data that needs to be processed by a program that is only available on another compuer, so /local/scratch may still not be the best option.

A final disadvantage of /local/scratch is that unlike the home file systems kept on our file servers, the contents are not backed up each night. So if a workstation disk was to suffer a hardware fault, or you accidentally removed files you wanted, you would be unable to recover your data. So you should only use /local/scratch for data that you don't mind losing, or that you can easily recreate. Some examples of appropriate /local/scratch usage include:

• Large read-only data sets, collections of documents or source code archives that can easily be downloaded from the Internet.
• Large amounts of output from a program that you can easily/quickly run again if you need to reproduce the same output.
• Uncompressed copies of files that compress well, so you can keep the much smaller compressed copies safely in your home directory but use uncompressed copies in /local/scratch for easy access/analysis.

Tips On Reducing Your Disk Usage

Note that files you have moved to your Wastebin/Trash still count towards your usage; only when you right click on the Wastebin or Trash icon on your desktop and select the "Empty" option are the files deleted. So remember to empty your Trash periodically.

Programs that can help you find where you are using most of your disk quota include kdirstat and du.

The easiest to use is kdirstat, a graphical tool in the Utilities section of the K menu. It shows you visually how much space your files and directories are using and also provides a convenient user interface that allows you to delete selected files/directories.

If you prefer to use command line tools, from a shell window you can type du -s -m <names> where <names> are the files/directories you wish to check. This will output the size (in megabytes) of each named file and the total size of each named directory and all the files/directories beneath it.

An alternative form is du -m <names> (ie: omit the -s option) in which case the command lists the total size of all directories under <names> recursively. This will produce much more output than if -s was used, but it provides information on every directory in an entire directory tree rather than just the summary of each named parameter that -s produces.

The <names> parameter to du is often specified as "*" which is a wildcard pattern that expands to all (non hidden) files/directories in the current working directory. Hidden files/directories are ones whose names start with a "." (they are normally omitted from directory listings, hence the name). If you wish to include hidden files in your wildcard pattern you can specify it as "* .??*" (consult a manual for your shell if you want to understand what this means!).

You should be cautious about removing hidden files or any files contained in a hidden directory as these often contain configuration information, without which the system will not work correctly.

When using du -s -m * or du -s -m * .??* it is useful to run it at the top level of your home directory tree first, so that you can get an overview of where the areas of largest usage are. If you add | sort -nr | head on to the end of the command its output will be sorted numerically by size (largest first) and only the first 10 lines will be displayed. This may make it easier for you to see where most of your usage is. You can then repeat the command on just the larger directories to further narrow it down.

if you omit the <names> parameter (or specify it as ".") du will produce output for just the current working directory. So du -s will produce a single total for the current directory and all the files/directories (including hidden files) beneath it. Without -s du will produce a recursive listing of the total sizes the current directory and all directories beneath it.

Tips for users of Eclipse

Eclipse is a disk hog. Here are some things to do to reduce its usage.

1. Don't use multiple workspaces - rather use multiple projects within the one workspace. There is a whole lot of metadata overhead per workspace and no particular gain.
2. Occasionally run eclipse -clean. This cleans up a bunch of old cached info at the expense of a slightly longer startup time while its doing it.
3. Remove old profiles. eclipse writes multiple profiles into
   ~/.eclipse/org.eclipse.platform_*/p2/org.eclipse.equinox.p2.engine/profileRegistry/
   and doesn't delete old ones. You only need the most recent file in that directory - delete the others (this can get lots of space back)
4. Reduce the values set in Preferences > General > Workspace > Local History. Once there, you can find three options, Days to keep files, maximum entires per file and maximum file size. With the default settings you can end up keeping many copies of every file you edit, when typically you aren't going to reference any but the most recent backups.