HPSS Project
Updated September 22, 1997
The High Performance Storage Subsystem (HPSS) will provide archival
storage for internal C&C users (e.g. backups) and Uniform Access
customers (previous Unitree users).
- Purpose
- Design
- Operations
- Schedule
- Status
See also
- HPSS Reference Manual
- Operations Guide
Unitree
The existing uniform access archive,
UniTree,
has been useful, serving more than 1,000 users, 1,000,000 files,
and a few hundred GB of data. Exact numbers are not available, as a
full accounting takes too many days to complete.
Unitree is not suitable
for a long-term solution for our archive needs for a couple
of reasons.
- Unitree is old and unsupported technology.
There are no other sites worldwide that also use our version
of Unitree. There are no developers of Unitree.
Unitree could not grow with our projected needs.
- Unitree supports only a basic ftp access. NFS is not available.
Kerberos authentication is not available.
- Unitree can run on multiple hosts in a limited and untested fashion.
- Unitree does not support new hardware or software releases.
HPSS
HPSS solves all of these problems.
Its code, although written by many old Unitree developers, is completely
new and standardized. It uses DCE RPCs for communication and Transarc Encina
for transactions. The code is much cleaner. HPSS is actively
worked on.
We are an HPSS development partner.
- HPSS
-
An HPSS system consists of many different servers (daemons) and
database files. Schematically the HPSS daemons look something like this
Operationally, they look like this
The combined service provides a disk cache for immediate, temporary storage
and magnetic tapes for permanent storage.
Access to the files is via FTP, NFS, or a client API.
Backup uses the latter. Most clients will use FTP.
See also the HPSS reference manual.
- UW implementation
-
We plan to run two, independent HPSS systems: one for backups and other
critical C&C applications; the other for Uniform access users.
It is likely that a single HPSS system could suffice. However,
our backup system is already in production. We do not know for sure
what would be the load (and complications) from the Uniform Access system.
By isolating the systems we protect our existing service.
| HPSS system |
Host |
Additional Movers |
Production |
| Backups |
Ashur |
Nineveh |
Yes |
| Uniform Access |
Nineveh |
|
no |
Only the backup service is in production.
Operations is responsible (actually, soon will be responsible)
for some of the day-to-day monitoring and control of the HPSS system.
See the Operations Guide for details.
For the moment this means:
- Starting the servers at system boot.
- Monitoring messages and storage class statistics.
- other duties to be added
-
September 21, 1997.
-
-
The internal HPSS system is in production on Ashur.
It moves approximately 500GB of backup data per day.
-
Movers on Nineveh are configured, but are not operational
because of too few STK drives in the silos. More will be ordered.
-
In progress
-
-
Add the nineveh movers to the internal HPSS system.
Waiting for additional tape drives.
-
Configure the external system.
-
Find or write user access tools.
-
Testing.