.. vim: syntax=rst .. _policy-policy_docs-storage: ============== Storage Policy ============== NFS Shares ---------- For all managed linux desktops, group servers, and some Macs, users home directories along with group and center directories under /org are NFS mounted using autofs. This allows for easier management of storage policies such as quotas and centralized backups as described further in this document. .. warning:: It is recommended when running programs or installing tools or applications, you install and run programs from the local hard disk file system, /workspace. We allocate enough storage for the operating system and leave as much as possible for /workspace. You will get better peformance running programs locally than over NFS and it will keep the load off the NFS servers during the work day when there is higher traffic on the server. It might also keep you from getting a notification from Sysnet if we find the load is coming from a process on your workstation. See Scratch areas information further down. Why quotas? ----------- Quotas were put in place to help control storage resources. As the Institute continues to grow it is important to keep users quotas at a reasonable size. Quotas keep users home areas in check keeping our file systems at reasonable sizes should they be checked for consistency. Quotas also keep our backups at a reasonable size keeping our backup window small. Project quotas have also been placed on centers, programs and groups. What are quotas? ---------------- Disk quotas are implemented on a per user and per group basis. Quotas have soft and hard limits. When the soft limit has been exceeded, there is a grace period of 14 days to clean up and get below the soft limit. When the grace period expires, the soft limit becomes the hard limit and disk writes will fail. The hard limit cannot be exceeded, disk writes will fail if attempting to write beyond the hard limit. Established quotas for users ---------------------------- :: Users soft hard Staff, Visitors, CAM students 5G 10G Faculty and Postdocs 10G 20G To check quotas, open a terminal window and issue 'quota -s' Example :: [stew@wichita ~]$ quota -s Disk quotas for user stew (uid 64476): Filesystem blocks quota limit grace files quota limit grace nfs:/oden/stew 3430M 10000M 20000M 39952 0 0 The output from the quota command above shows that user stew has used approximately 3420 MB of disk space. The quota or soft limit is 10 GB and the limit or hard quota is 20 GB. If the soft limit has been exceeded, the grace period will indicate a time period as to when the soft limit becomes the hard limit. Checking your directory ----------------------- To determine file and directory sizes, use the 'du' (disk usage) program from a terminal window. For example, from your home directory, issue :: $ /usr/bin/du -sk .[!.]* * | sort -n . . . 41252 bin 47576 .comsol 49692 .Spotlight-V100 52608 .googleearth 64144 .java 74364 public_html 154848 .icons 155808 .local 170848 .cache 216868 Documents 242532 .thunderbird 362884 Library 474440 proj 1219984 .mozilla Above is an example from the last few lines produced by 'du', sorting files and directories by asending sizes. Appears that the .mozilla directory is consuming roughly 1.2GB of space. Looking into this directory, I noticed that my email is being cached locally. This can be fixed by unchecking 'Message Syncronization' in the Syncronization & Storage configuration in Thunderbird. The 'du' command is a common tool that can be found on all linux distributions and Mac OS X. Another tool for linux users is baobab. It offers a graphical view of your files. .. figure:: baobab/baobab.png :scale: 80 :alt: Baobab Disk Usage Analyzer :align: left .. figure:: baobab/baobab2.png :scale: 80 :alt: Baobab Disk Usage Analyzer Both utilities may take some time to run depending on the number of files and size of your home space usage. Established project quotas for groups ------------------------------------- Users belonging to those groups and centers listed below are provided access to additional storage in the directories shown below. On our current fedora desktops these directories are automounted and simply can be accessed by changing directory to the appropriate location. If you feel you are not in the proper group, please submit a help request to be added to that group. :: Group location aeg /org/groups/aeg chg /org/groups/chg gamba /org/groups/gamba hughes /org/groups/hughes oden /org/groups/oden padas /org/groups/padas press /org/groups/press rmoser /org/groups/rmoser staff /org/groups/staff :: Center location ccgo /org/centers/ccgo ccm /org/centers/ccm ccms /org/centers/ccms cdgc /org/centers/cdgc clsb /org/centers/clsb cna /org/centers/cna csm /org/centers/csm cvc /org/centers/cvc pecos /org/centers/pecos :: Program location csem /org/programs/cam visitors /org/programs/visitors postdocs /org/programs/postdocs An unfortunate side effect for using project quotas is the quota tool does not understand XFS project quotas. You can view the project quota in the daily usage report. Usage Reports ------------- A daily usage report file is generated and located in the directory 'usage' in the top of the directory tree for all groups and centers. The top of the report shows the project quota limits and usage. Scratch areas ------------- There is an additional 2 TB's of storage known as /work. All users have access to /work. This space is treated as a scratch area and is not backed up. Sysnet currently does not enforce a clean up policy on this file system, but will in the future if necessary. On all desktops maintained by Sysnet, there exists /workspace file system. The size of workspace is dependent upon the size of the drive installed and do not have quotas enabled. Workspace is **not** backed up. **Users are responsible for backing up /workspace, it serves as a scratch area. Make use of your center, group, or program storage if you need to store important work. Scratch areas are wiped clean during a reinstallation or upgrade, Sysnet is not responsible for any data lost on scratch areas!** Backups ------- Backups are done nightly for all users home directories and everything below /org. The /work area is NOT backed up. .. _policy-policy_docs-hardware: Hardware -------- In late 2020, Sysnet installed new storage hardware for users home directories and group/centers shares. The storage is comprised of two Dell PowerEdge R740xd2 servers configured with a single processor with 8 cores and 192GB RAM. Each server has a dual ported 10BaseT NIC; link aggregated to a Cisco C9300 multigigabit switch. The switch is uplinked to the building router through a single 10Gb link. Storage on the servers is a mix of solid state drives and spinning HDD's. The home directories reside on a raid10 set containing SSD's while the org storage is on a raid10 set with HDD's. Exports are served from a single host, a secondary host is configured with block replication (DRBD) in a 2-node clustered environment for resiliency. Further, communication for the block replication is on a dedicated 25Gb network, dual ported and aggregated on a separate network backbone for redundancy and performance.