13-Mar-1996 23:57:29 -0800,18432;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Wed, 13 Mar 1996 23:57:29 -0800 (PST) Return-Path: Received: from mailer10.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.02/UW-NDC Revision: 2.33 ) id AA18914; Wed, 13 Mar 96 23:57:28 -0800 Received: from mx1.cac.washington.edu by mailer10.u.washington.edu (5.65+UW96.02/UW-NDC Revision: 2.33 ) id AA74956; Wed, 13 Mar 96 23:57:24 -0800 Received: from stormy.clearlake.ibm.com by mx1.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA18246; Wed, 13 Mar 96 23:57:20 -0800 Received: from vnet.ibm.com by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA13132; Thu, 14 Mar 1996 02:01:09 -0600 Message-Id: <9603140801.AA13132@stormy.clearlake.ibm.com> Received: from BETVMIC1.VNET.IBM.COM by vnet.IBM.COM (IBM VM SMTP V2R3) with BSMTP id 0844; Thu, 14 Mar 96 02:54:28 EST Date: Thu, 14 Mar 96 02:54:28 EST From: coyne@vnet.IBM.COM To: hpss-exec@clearlake.ibm.com Subject: Long list of topic and issues for our consideration/comments From: Bob Coyne IBM GOVERNMENT SYSTEMS (713) 335-4040 T/L 260-4040 FAX X4231 Subject: Long list of topic and issues for our consideration/comments Here are three recent topics for our consideration, a reminder about two others, plus whole list of current status/issues/topics. Sometimes folks think that I don't share enough about what's going on and sometimes they say my notes are too long -- this is long note; some of the topics are urgent and some are just FYI. 1. Support for Ported Client Code The issue of how to handle HPSS client software ported to other platforms (e.g. SGI, CRAY, Solaris, etc.) was raised. Issues to consider are: - Inspections / Integration Test cases - Customer support - Ongoing maintenance - Access to vendor platforms The Tech Committee thought this issue should be escalated to the Executive Committee for a position statement. COMMENTS OR IDEAS ON A POSITION STATEMENT? 2. Tape Formats Meeting A Tape Format Standards meeting is being held in Chicago on 4/1/96. HPSS has been invited to support this meeting and present information on HPSS. The development sites were asked for a volunteer to represent HPSS. No volunteer has yet been identified. It would be good to send an HPSS representative who understands our tape layouts to support this meeting, so that we are kept informed in definition of this standard tape format. ANY VOLUNTEERS? 3. Disclaimer for HPSS API source code: A CIA rep has asked to review the HPSS API with the intention of joining the effort soon if HPSS meets requirements. Danny Teaff suggested putting the code on anonymous FTP for all to download including the CIA rep. I think this is a great idea but we need a disclaimer to be placed in the code or a form that folks sign stating that they will not hold the team liable any issues resulting from the use of our free code and about X more things like this. I will ask our legal staff to draft appropriate language for your review. If you have any comments on the anonymous FTP idea or the disclaimer, then please let me know. I asked Danny to hold off FTP until we have a chance to hear from the you. 4. Reminder that EMASS has asked bout the procedure for requesting Grau library/robot support. We should reply in a timely manner. Maui HPCC has responded with some suggestions. 5. Collaboration agreement. -- URGENT Only LLNL and Fermi has asked questions and there have been no approvals or rejections. LEGAL ISSUE: There is a legal issue called "implied agency" that has been brought to my attention after we agreed that ORNL and Sandia should be the single point of HPSS contact for PSC. We have agreed for instance that various members of the collab should talk to potential new members about HPSS. While this is highly desirable and required for project longevity, the "implied agency" issues has been brought to my attention. As I understand it (without any legal expertise), IBM could be held responsible for HPSS related liabilities resulting from discussions between HPSS customers and persons discussing the acquisition/use of HPSS even though the persons may not represent IBM wrt to providing HPSS licenses or services. In the collaboration agreement there is a clause declaring that we are all independent contractors wrt to HPSS affiliation. It seems that this mitigates the "implied agency" issue. Again I am not sure of all the legalities. The recent Hughes contract (for which we've received a P.O) and the ongoing PSC discussions are examples of efforts that required mitigation of the "implied agency" issue. Please work on completing the collab agreement, especially if your team plans to engage the public wrt soliciting new HPSS members and users. To date, we've had no urgent reason to press for completion of the agreement other than our desire to use it as guideline for the team effort whether our attorneys approve it or not (I think this is what we said at SC95). Now as HPSS nears deployment we will be engaging more folks than ever. It is necessary that we resolve the "implied agency" issue. Comments? We could resolve this issue separately if the collab agreement cannot be approved in a timely manner by the folks that are active in community wrt HPSS membership drives. It is necessary that team members continue to represent HPSS to the public, so lets work to get this issue resolved as soon as possible -- thus preventing "help" from our legal staff. staff. 6. More HPSS topics/issues/status a. One CIA project is looking at HPSS for immediate acquisition. b. CERN has requested that we provide a class. They contacted their local IBM rep, we replied that CERN should contact the HPSS EC requesting a class. We also asked CERN if they would like to include DKRZ and some folks at several Europeans sites in the class. CERN has offered to pay for the class. c. IBM Germany and Fenn/Hulen are discussing HPSS with DKRZ. There are a myriad of issues wrt selling HPSS outside the US. Harry Hulen and Dennis Stoker have begun the paperwork for export permission from the State Dept. One example of a hard issue is that our current Licensing of HPSS on a yearly basis may not be LEGAL in Germany -- and the list goes on. d. The Hughes P.O for HPSS was received and the legal staffs are beginning another round of negotiations :-( about Ts and Cs. e. Transarc (through Peter Olienick) has expressed an interest in taking HPSS to market if IBM provides "appropriate incentives". This is great news, unfortunately, Peter changed jobs last week to head up their Western Sales Region and is now consumed with near term revenue rather than strategic alliances. This is the second time that a major personnel change at Transarc has put HPSS discussions on hold. Last HOLD was for more than a year due to lack of a marketing manager replacement after Sandy Esais died. f. HPSS sales folks (Harry and Joe) continue to beat us up for failing a provide a UNIX interface for legacy programs and claim HPSS is not responsive to user needs at sites like OSC. They claim the NFS V2 is inadequate and that DFS/HPSS/DAMPI are too far out to keep HPSS membership growing in 1996/1997. I don't think there's much we can do for Harry and Joe on this topic. If you have a suggestion please let us know. g. Bob Steen et al continue to discuss HPSS plans with various IBM Executives. I think these discussions will continue for several months -- at least until our fall planning effort in 3q96. Mike Shields (Fort Meade) has requested a statement from IBM SSD about the status of HPSS commercialization. DoE Lab ASCI reps would like a similar statement from IBM. I am sure that there many other with the same request. There is nothing to report other than folks are talking. HUMOR: That in itself is progress considering that HPSS commercialization progress can been measured in geological time. h. DataTape has contacted us about requesting HPSS licensing and resale agreement, if one of their IC customers is willing to underwrite an integration effort. DataTape is also exploring embedding HPSS in applications for resale. They mention that there is a suggestion from CalTech as well about support DataTape D1 tapes. They may be talking to CalTech about their plans as well. This is the most promising discussion that we've had to date with any vendor about HPSS in terms of a good business "fit". i. Cray has fallen silent other than letter to Dick Watson about has well IBM is treating wrt on-going discussions. j. The DFS planning effort has been circling on four issues: 1. An AIX platform delivery to the HPSS test team 2. IPL issues wrt the use of DMIG specs 3. A joint SOW from the Tech Committee and Transarc 4. Plans to use DEC system for ASCI DFS servers, when DEC is not part of contractual obligation to deliver DSF DMAPI. Dick would like assurances about other platforms as well, and Transarc does not control what their resellers deliver in terms of DFS features. Item 1 is resolved. Item 2 is underway between IBM and Transarc attorneys. We've asked Danny Teaff to resolve Item 3 by COB this Friday. Transarc plans to review item 4 with Dick Watson when he returns from travel. k. Near term use of AIX 4.1 as a required base remains a problem for CTC; the HPSS EC and TC remain focused on AIX 3.2 for acceptance testing. CTC expressed the concern that they are basically dead in the water without AIX 4.1. We referred Doug once again to the HPSS EC. We also encouraged Doug to look at the possibility of running HPSS movers under AIX 4.1 as an interim solution if the HPSS AIX 4.1 porting effort cannot deliver a systems before they need it. Basically they need AIX 4.1 in production within a month or two. l. HPSS R3+ and R4 requirement prioritization work continues and we should see something from the HPSS TC in the next two weeks. m. Site Planning activities should be on the next telecon agenda if there are any questions about the effort. n. Ron Christman of LANL fame (now retired) has been hired as an IBM subcontractor under ORNL's contract to complete the R3 Admin Guide. Ron was the first LANL person assigned to work out the details of the HPSS project plan. Ron continued to work on HPSS for a while after retirement -- on his own nickel. Comments and discussion would be appreciated -- feel free to add topics to the list. Dick would like to schedule another HPSS EC telecon -- some of the more important topics listed should be discussed soon. Regards, Bob 12-Mar-1996 13:32:30 -0800,6851;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Tue, 12 Mar 1996 13:32:30 -0800 (PST) Return-Path: Received: from mailer3.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.02/UW-NDC Revision: 2.33 ) id AA18748; Tue, 12 Mar 96 13:32:29 -0800 Received: from mx2.cac.washington.edu by mailer3.u.washington.edu (5.65+UW96.02/UW-NDC Revision: 2.33 ) id AA26710; Tue, 12 Mar 96 13:32:29 -0800 Received: from stormy.clearlake.ibm.com by mx2.cac.washington.edu (5.65+UW96.01/UW-NDC Revision: 2.33 ) id AA27568; Tue, 12 Mar 96 13:32:27 -0800 Received: from vnet.ibm.com by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA36547; Tue, 12 Mar 1996 15:35:26 -0600 Message-Id: <9603122135.AA36547@stormy.clearlake.ibm.com> Received: from BETVMIC1.VNET.IBM.COM by vnet.IBM.COM (IBM VM SMTP V2R3) with BSMTP id 0156; Tue, 12 Mar 96 16:28:51 EST Date: Tue, 12 Mar 96 16:28:50 EST From: coyne@vnet.IBM.COM To: hpss-exec@clearlake.ibm.com, mikeh@emass.com, dave.barnes@emass.com Subject: EMASS connection to HPSS From: Bob Coyne *** Resending note of 03/12/96 16:07 IBM GOVERNMENT SYSTEMS (713) 335-4040 T/L 260-4040 FAX X4231 Subject: EMASS connection to HPSS Before IBM responses to the EMASS request for information, I would like your input and any discussions that you would like to have on the topic. Thanx, Bob ========================================================================= Content-Type: text/plain; charset="us-ascii" Date: Tue, 12 Mar 1996 14:12:14 -0600 To: coyne@vnet.ibm.com From: Dave.Barnes@emass.com (Dave Barnes) Subject: EMASS connection to HPSS Cc: mikeh@emass.com Bob, This is a followup to our brief conversation this afternoon. BACKGROUND EMASS manufactures Automated Mixed-Media Libraries (AMLs). We are interested in having HPSS support these AMLs. We have at least one prospect who has stated that "in order to bid, you must support HPSS". QUESTIONS 1. How do we make this happen? 2. What is the process? 3. Who writes the code? EMASS? IBM? other? 4. Where is the testing performed? 5. What kind of equipment is involved? Do we supply an AML for testing? 6. How de we become "officially certified"? 7. What about substaining engineering? That is, when we or HPSS makes a modification how do we achieve synchronization? 8. How much does it cost? 9. How long does it take? 10. What is the beta selection process? ACTION DESIRED Could you please provide a response to me by 25 March 1996 as we are anxious to move forward. Thanks. Yours in large storage opportunities, Dave Dave Barnes Marketing EMASS 10949 East Peakview Ave Englewood, CO 80111 USA Phone: +1-303-705-3827 Fax: +1-303-792-2465 Home: +1-303-744-6978 Mobile: +1-303-906-2264 Dave.Barnes@emass.com 20-Mar-1996 16:37:37 -0800,117664;000000000001 Return-Path: Received: from mailer5.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA18534; Wed, 20 Mar 96 16:37:36 -0800 Received: from mx2.cac.washington.edu by mailer5.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA25232; Wed, 20 Mar 96 16:37:34 -0800 Received: from stormy.clearlake.ibm.com by mx2.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA18374; Wed, 20 Mar 96 16:37:13 -0800 Received: from popcorn.llnl.gov by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA10057; Wed, 20 Mar 1996 18:38:34 -0600 Received: from quickmail.llnl.gov by popcorn.llnl.gov (8.6.10/LLNL-2.0) id QAA15997; Wed, 20 Mar 1996 16:31:43 -0800 Message-Id: Date: 20 Mar 1996 15:57:15 -0800 From: "Dick Watson" Subject: Transarc S.O.W. dtd 3/20/96 To: "HPSS Exec Reflector" , "HPSS Reflector" , "HPSS-developer" , "maher@transarc.com" Cc: "Sandy Sydnor" , "Dick Watson" X-Mailer: Mail*Link SMTP-QM 3.0.2 Content-Type: x-uuencode-datafork X-Attachments: "Trans Dr SOW 3/20 " (type: uuencode-datafork) Subject: Time:3:51 PM OFFICE MEMO Transarc S.O.W. dtd 3/20/96 Date:3/20/96 Enclosed is the Transarc DRAFT Statement of Work for your review and comment. I want to be sure that it is technically correct and addresses our main concerns. PLease note that LLNL's procurement process may make revisions to this. Thanks. Sandy (for Dick Watson) PS - The enclosure is in MS Word 5.0. I am also including the text following this message. ******** DRAFT -- STATEMENT OF WORK Date: 3/20/96 (Contains Transarc Proprietary Information) Requisition No. xxxxx Contact: Dick Watson (510 422-9216) 1.0 INTRODUCTION (Objective and Background) This project has as its goal the integration of the High Performance Storage System (HPSS), being developed by a collaboration of DOE laboratories, other agency laboratories and supercomputer centers, universities and IBM, with the Distributed File System (DFS) developed by Transarc Corporation. It is expected that the HPSS/DFS integration will take place in phases. This SOW defines Phase 1, the basic integration and planning for the next phase. Phase 2 and beyond will provide for high performance enhancements, such as third party and parallel transfer of data directly between client machines and HPSS under DFS control, when appropriate. Phase 2 will also support fileset movement not included in Phase 1. An important objective of Phase 1 is to assure no design or implementation decisions are made that would limit or compromise the future high performance enhancements. Because a significant fraction of the work that must be done for this integration is HPSS dependent, we propose a cost sharing arrangement whereby LLNL pays for the HPSS specific work and Transarc Corporation will pay for the DFS specific work. The High Performance Storage System (HPSS) is a very high performance storage system capable of storing hundreds to thousands of terabytes of data in a distributed manner, and provide fast and universal access to it. HPSS is a high-end storage management system of value to organizations that have to process large quantities of frequently changing and archival data. It uses the Distributed Computing Environment (DCE) from the Open Software Foundation (OSF), and Encina from Transarc as building blocks to provide this functionality. The DCE's Distributed File System (DCE-DFS) is a medium scale file system contributed by Transarc to the OSF, and is available as part of the DCE offering by several vendors. It provides excellent file and data management capabilities via global naming, distributed organization-wide security and remote administration servers. It is currently available on a variety of platforms that HPSS intends to run on or support. DCE-DFS fits in very well as one of the building blocks which the HPSS system can make extensive use of. Currently, DFS lacks the capability of migrating infrequently used files between tertiary storage and local storage. The approach of this work is to use, and where necessary extend, the results of the Data Management Interfaces Group (DMIG). DMIG is a voluntary group formed by employees of various companies involved in data management. The group includes filesystem vendors like Sun, IBM, HP, SGI, Veritas and Transarc, and also includes hierarchical storage management (HSM) and back up vendors like Epoch, Legato, and Cheyenne, to name a few. The group was formed in 1994 to address problems their (common) customers faced regarding data management. The HSM vendors often were programming inside various vendor kernels, and when the operating system vendors released new OS versions, the customers would get caught in the middle without being able to move forward to the new hardware available in the market place, since the new OS would break some of the hidden interfaces that were being used by the HSM vendors. One of the first tasks of this group was to come up with a public domain defacto standard programming interface that would be implemented by the filesystem vendors and used by the HSM vendors, to provide data migration capabilities. The Data Management Application Programming Interface (DMAPI) is the result of their effort. It is an interface for use by HSM vendors that provides hooks inside the filesystem to be able to trap reads and writes and manage transparent migration of data between local DFS storage and hierarchical storage systems. The interface is to remain stable across operating system and filesystem releases. There is sufficient functionality in the interface to be able to handle various types of filesystems, although it does require some enhancements to meet HPSS integration requirements. Transarc will develop the DMAPI for DFS, with extensions needed by HPSS outlined later. The HPSS collaboration can then develop the HPSS specific Data Management Application (DMAP) that interfaces on one side to the Transarc DMAPI and on the other side to HPSS. Transarc Corporation will provide the resources to implement the DMAPI, extended as mutually agreed, into the DFS and integrate the HPSS data management application process with the DFS-DMAPI implementation according to the schedule of deliverables listed below. 2.0 SCOPE OF WORK The work specifically involves providing the DMAPI in Episode which is Transarc's local file system, and adding some additional functionality in the DFS fileservers to take advantage of the DMAPI. In addition Transarc will integrate the HPSS DMAP with the Transarc DMAPI implementation, so that the outset, a working DFS filesystem with data-migration capabilities is available and can be put to immediate use. The extensions to DMAPI and the DFS implementation issues required for effective Phase 1 HPSS/DFS integration are needed to support synchronization and access to data stored in HPSS either through DFS or directly through HPSS. This implies mechanisms for dataspace, name space, access control lists (acls) and permission synchronization not currently in the DMAPI specification. 2.1 dmapi in Episode Episode is a log-based file system with fast restart, that also implements all of the functionality of the VFS+ interface as specified in the DCE-DFS specification. Episode provides the concepts of aggregates and filesets. Aggregates are like partitions; that is to say, an aggregate covers an entire partition and provides a container-like functionality on which filesets are implemented. Filesets are the basic building blocks of DFS. Filesets are essentially entire filesystems, complete with a directory hierarchy, and space for files and other filesystem objects like links. Files and directories and access-control-lists (acls) that are created in DFS finally end up taking up disk space as objects residing in filesets. DFS mounts filesets on each to create the cell-name namespace. The bulk of the DMAPI implementation will be in Episode. The implementation will consist of the following: o A user-space library that implements all the C functions defined in the DMAPI and the extensions below needed for full HPSS integration. This library will convert the function calls along with the parameters into appropriate messages and pass them into Episode, since Episode is implemented inside the kernel. The library will also provide the asynchronism necessary to implement the sessions and events that the DMAPI specifies. o A device-driver like kernel module that receives the messages sent by the library, and which in turn may generate event messages. This kernel module will also be responsible for managing DMAPI sessions that persist across different user-space processes. The module will communicate with Episode via a separate API added to Episode. o Episode will be modified to implement the functionality required by the DMAPI interface. It will provide the ability to trap read and write events, provide backdoors to read and write data without generating the events, and the ability to punch holes in files without affecting the user-visible attributes of files like mtime, length, etc.. Clones in Episode cause interesting problems, since clones share disk blocks for data. Cloning and replication will be disallowed for HPSS in the Phase 1 release. o Episode/DFS will return ACL's. 2.2 DFS fileserver changes The DFS fileserver really consists of two entities: the protocol exporter that handles all interactions with the DFS clients, e.g., it serves files and checks permissions, and the fileset server which handles all fileset operations like creating, moving, dumping filesets. We will discuss what changes will be necessary in each of those components. 2.3 Protocol exporter The file server in DFS is the protocol-exporter. It handles all interactions with DFS clients, also referred to in various places as the DFS cache-manager. The fileserver will most likely not see very many new changes to enable the DMAPI in Episode. The fileserver uses the VFS interface, and there are no new changes proposed in that interface. However, the protocol-exporter should be able to handle new types of latencies introduced by the presence of migrated files. For example, if a file is migrated and a request to read the file comes in from a client, the protocol-exporter will issue a VOP\_READ which can be delayed for an arbitrarily large interval while the migration site loads the tape to read the file. In addition, data-migration introduces a whole spectrum of error situations. For example, the tape-read in the previous example may fail due to the migration site being down. The protocol-exporter should be able to handle such an error. New, user-generated, errors are also feasible in such an environment. For example, referring to the tape-read example again, the migration site may be taking several hours to find the correct tape, and the user may decide that they do not want to read the file anymore and cancel the read attempt. The protocol exported should be able to handle such a cancel and take appropriate actions. There might be some extensions added to the DFS client to protocol-exporter protocol to handle new errors and to look at the additional attributes of files, e.g., whether a file is migrated, if so where, how large is it, what portions of it are on disk and what are off-line, etc.. 2.4 Fileset server The fileset-server in DFS handles all fileset operations. This includes creating, deleting, listing, dumping, restoring and cloning filesets. It uses the extra operations defined in the VFS+ interface, collectively known as the AG\_OPS and the VOL\_OPS. It is expected that these operations will be extended to be able to deal with data-migration issues. The fileset-server should now be able to handle filesets that contain migrated files. For example, if the someone were to request a dump of an abbreviated (we will refer to filesets containing one or more files that are migrated as abbreviated) fileset, the fileset-server must be able to produce a dump that is restorable later. Similarly, the fileset-server must be able to restore abbreviated dumps. 2.5 Fileset cloning, movement and replication Fileset cloning, movement and replication will be supported in future releases. Fileset movement and replication require that some of the DM attributes of the migrated file not be opaque. For example, the fileset server should be able to look inside some of the attributes saved by the DM-application and be able to tell whether a particular fileset move requires a full dump or an abbreviated one, since the recipient might not have network connectivity to the data migration site of the sender. 2.6 HPSS DM-application Transarc will integrate the HPSS specific hierarchical storage management client requirements for proper dataspace, name space and security synchronization. This will involve working with the HPSS providers to provide correct hooks into the kernel and provide the appropriate data structures and storage behind these data structures with the DMAPI implementation. The HPSS team will be responsible for developing the HPSS specific client for data-migration, provide the DMAPI extension and DFS implementation to Data Management Application Process (HPSS-DMAP), and Transarc will work with the HPSS team to get requirements, and make suitable modifications to the DMAPI implementation. 2.7 Future Work There are several areas where more needs to be done, some which will incorporate support for HPSS' unique needs, while others address the data management problem more completely in the context of DCE/DFS. o 3rd party transfer o Parallel IO o Support for fileset movement o Support for fileset replication As part of this project, one of the deliverables will be a Phase 2 plan and design for enhancements to support the above needs, and provide fileset movement. 3.0 Platforms, OSF, AND USE OF DFS The result of this effort will be a fully functional DMAPI meeting HPSS requirements for the IBM AIX operating system and the Sun Microsystems Solaris operating system. Upon completion of this work Transarc will provide the resulting DFS-DMAPI to OSF for porting by other vendors to their operating systems under existing DFS support agreements between Transarc and OSF. In the milestones below items (Transarc) are Transarc deliverables, items (HPSS) are HPSS team deliverables. The milestones specify what is required for a complete Phase 1 solution. Transarc Corporation will provide the resources to implement the Data Management Application Programming Interface (DMAPI), extended as mutually agreed, into the Distribute File System (DFS) and help integrate the HPSS data management application process (HPSS-DMAP) with the DFS-DMAPI implementation according to the schedule of deliverables in Table 1. Transarc will provide DFS free of charge on the platforms needed by the HPSS team during this contract to develop the complete Phase 1 solution and provide free of charge the maintenance and support required for the DFS versions being developed under this work. Transarc personnel should be available to answer questions and fix bugs that are found during the HPSS-DMAP integration process. As mentioned earlier, Transarc shall provide software to allow keeping the HPSS and DFS/Episode data spaces, name spaces and security state in sync. This will allow access to all objects through both the HPSS and DFS interfaces. HPSS and Transarc have negotiated some additional events and interfaces that will be needed to provide the capability of keeping the needed synchronization. Items below with an * indicate those requiring HPSS extensions. Transarc has phased releases of DFS. Future releases of DMAPI will be #021##016#ward compatible with what is delivered in Phase 1. 5.0 Agreed DMAPI events 5.1 Administration Events closed The description of this event appears to be sufficient. We felt this event could be useful for optimizing HPSS open/close processing. mount The description looks fine but we had some DFS/Episode specific questions about aggregates and file sets. What does Transarc propose to do? Mounts will be aggregate based. preunmount unmount nospace debut #012#Table 1. MILESTONES AND DELIVERABLES MAY 96 Project Start (Transarc/HPSS) Transarc Delivers Episode/DMAPI interface document which includes the following: o Interface to be implemented. o Assumptions concerning aggregates and file sets. o Episode functionality which will not function under the DMAPI proposal. JUN 96 DMAPI design and code work begins at Transarc (Transarc) HPSS/DMAPI design completed (Dependent on interface doc) OCT 96 DMAPI initial IBM AIX operating system code drop (Transarc) o DMAPI library o Episode file system modifications o Kernel module that interfaces to DMAPI library o Verification suites HPSS initial DMAP code drop (HPSS) o DMAP - Initialization - Event Handling - Migration - Purge o DMAP/Server that interfaces to rest of HPSS o Client API Modifications o SSM work o Unit tests written HPSS Unit Test Start (HPSS) o DMAP verification o DMAP Server o Client API Modifications Initial Draft of Phase 2 HPSS/DFS performance enhancement design and plan. Acceptance test specification. DEC 96 Episode/DMAPI (Transarc) o Transarc's first DMAPI release. At this point Transarc will only be adding bug fixes. HPSS integration tests formally started (HPSS) APR 97 HPSS/DMAP work integration tested into R3+ version of HPSS. Final Draft Phase 2 design and plan. Post APR 97 Integration into R4 version of HPSS. System Test with all other HPSS interfaces. These all looked OK as described. We felt the debut event is not mandatory as long as dm attributes are persistent. nospace *lowspace (addition/highly desirable) It would be nice to get an event when a file system starts to run low on space, so that data can be purged before space gets critical. This addition implies a new DMAPI interface to set the threshold. 5.2 Name Space Events create To keep the name spaces in sync we need some attribute or set of attributes so that we can set up the ownership info in HPSS. postcreate This looks fine as proposed but please see (syncpostcreate). If this event can be guaranteed (even when create is aborted), then syncpostcreate may not be necessary, but guaranteed asynchronous events might still be lost between the time they are generated and the time the DMAP stores them stably. *syncpostcreate (addition /mandatory) This event should be generated just before the DFS/Episode system returns back to the caller. This event should contain the same attributes as the postcreate, but should be synchronous so that it blocks until we respond. remove postremove These appear to be fine in the spec. We did have some questions about when these events are generated. Could some body at Transarc explain what assumptions that have for remove of names and deletion of the actual data files? *syncpostremove (addition/mandatory) Same reasons as syncpostcreate. rename postrename *syncpostrename (addition/mandatory) Same reasons as syncpostcreate. link postlink These appear to be fine in the spec. *syncpostlink (addition/mandatory) Same reasons as syncpostcreate. symlink postsymlink These appear to be fine in the spec. *syncpostsymlink (addition/mandatory) Same reasons as syncpostcreate. *permchange (addition/mandatory) Whenever a call is made that changes the ownership of the file this event should be generated. Enough information should be passed to update the ownership information in the HPSS name space. *postpermchange (addition/mandatory) This should function similar to the other asynchronous post events. *syncpostpermchange (addition/mandatory) Same reasons as syncpostcreate. 5.3 Data Events read write truncate These all looked fine in the spec. 5.4 MetaData Events cancel *destroy (change to synchronous) We suggest that the interface be made synchronous and the event be generated at the end of the destroy cycle. 5.5 Interfaces The following is a list of DMAPI functions in 3 categories: mandatory, desirable, and don't care. These functions are enhanced as needed to support the event above. The mandatories are necessary for the DFS/DMAPI/HPSS implementations, the desirables have an explanation along with them, and the don't cares are interfaces that we do not believe we will need, although Transarc may choose to implement them in order to have a complete DMAPI. 5.5.1 Mandatory dm_create_session dm_create_userevent dm_destroy_session dm_find_eventmsg dm_get_allocinfo dm_get_bulkattr dm_init_attrloc dm_get_dmattr dm_get_eventlist dm_get_events dm_get_fileattr dm_get_mountinfo dm_get_region dm_getall_disp dm_getall_dmattr dm_getall_sessions dm_getall_tokens dm_handle_cmp dm_handle_hash dm_handle_to_fshandle dm_handle_to_path dm_init_service dm_handle_to_fsid dm_fd_to_handle dm_path_to_handle dm_path_to_fshandle dm_handle_free dm_pending dm_punch_hole dm_probe_hole dm_query_right dm_query_session dm_read_invis dm_write_invis dm_release_right dm_remove_dmattr dm_request_right dm_respond_event dm_respond_event should be modified to accept a list of d attributes on a response to a create call. Without this functionality it opens some holes in the synchronous name spaces. dm_set_disp dm_set_dmattr dm_set_eventlist dm_set_fileattr Being able to change the size of the file must be supported, so that if data is written through the HPSS interfaces the size of the file can be altered in Episode. Changing the file size should not cause Episode to allocate disk blocks for data since data would be already migrated to HPSS. dm_set_region dm_set_return_ondestroy dm_sync_by_handle dm_upgrade_right 5.5.2 Desirable dm_downgrade_right Would be nice if supported. *dm_get_bulkall This would be a highly desirable feature. It is not absolutely necessary, but it is a highly desirable option. We could use this feature to optimize migration and purging of files. This feature is on the border of mandatory but we could use the dm_get_bulkattr if necessary, although this may not be as efficient. dm_get_dirattrs dm_get_config dm_get_config_events dm_move_event dm_send_msg 5.5.3 Don't Care dm_create_by_handle dm_handle_to_igen dm_handle_to_ino dm_mkdir_by_handle dm_obj_ref_hold dm_obj_ref_rele dm_obj_ref_query dm_symlink_by_handle 5.6 Issues Needing More Discussion HPSS and Transarc must still determine how HPSS should view fileset operations, closed events, and ACL operations. Events to be determined: Administrative events: mount preunmount unmount begin 666 Trans Dr SOW 3/20 M_C<`(P```````"0``!D```````````$````!`0``\M,```````!2QP`````` M```)````````````````````````!@``&@``KZT`6P``L`@`````L`@````` ML`@`#@``L!8`K```OH8`````OH8`````OH8`#```OI(`3```OMX`.@``OQ@` M````OQ@`0@``OUH`>```L,(-Q```O](`(```O_(`&```P?X`*@``PB@PJP`` ML`@````,``D``,'^`````,`*`?0``,'^`````,'^`````/+3`````,'^```` M`,'^`````,'^`````,'^`````,'^`````,'^``````T"=@%35$%414U%3E0@ M3T8@5T]22PT-#5)E<75I71EF%T:6]N M2X@($ET('5S97,@=&AE($1I M2!42!S979E2!A;F0@2!O9B!M:6=R871I;F<@:6YF'1E'1E;G-I;VYS M('1O($1-05!)('-P96-I9FEC871I;VYS.R!*;VEN="!(4%-3+U1R86YS87)C M(&1E2`Y-@=(4%-3(&1E6YC)V0'!T1E8R`Y-@=(4%-3(&-O9&4@3H-5&EM:6YG(&]F($AI2`M(&%L;"!C(&9U;F-T:6]N0VE1$U!4$D@9')I M=F5R#:5%<&ES;V1E('=I=&@@;6]D:69I8V%T:6]NP``$7P``!&*```1BP``$9@``!&9```1N@``$=L``!'M```6````%CL``!9P M```6\```7O@`` M%[\``!?P```8$```&#L``!A;```8T```&/```!D%```9C@``&8\``!F0```9 MI```&:4``!FS```9M```&,``#(````R(```,D`` M`#)"```R1```,D8``#)'```R2```,E(``#)3```R^ M```9CP``&9```!FC```9I```&;,``!G-```9Y@``&A$``!H2```:*@``&D`` M`!I!```:20``&F\``!IP```:<0``&H4``!J7```:G@``&N<``!LE```;)@`` M&S$``!LR```;6P``&UP``!N?```;Y0``'"@``!Q"```<0P``'$H``!Q5```< M?```''T``!RE```P``':,``!VD```=S```'7-T96T@=&5S=',[(%1R M86YS87)C('-U<'!O6YC+B!4:&ES('=I;&P@86QL;W<@86-C97-S('1O(&%L;"!O8FIE M8W1S('1H2!D97-I6YC<&]S=&-R96%T M92`H861D:71I;VX@+R!M86YD871O6YC<&]S=&QI;FL@*&%D9&ET:6]N M+VUA;F1A=&]R>3\_/RD-#2`@("`@("`@4V%M92!R96%S;VYS(&%S('-Y;F-P M;W-T8W)E871E+@T-6YC<&]S='-Y;6QI M;FL@*&%D9&ET:6]N+VUA;F1A=&]R>3\_/RD-#2`@("`@("`@4V%M92!R96%S M;VYS(&%S('-Y;F-P;W-T8W)E871E+@T-<&5R;6-H86YG92`H861D:71I;VXO M;6%N9&%T;W)Y*0T-("`@("`@("!7:&5N979E6YC<&]S=&-R M96%T92X-#41A=&$@179E;G1S("T-+2TM+2TM+2TM+2T-#7)E860-=W)I=&4- M=')U;F-A=&4-#4UE=&%$871A($5V96YT6YC:')O;F]U0TM+2TM+2TM+2T-#61M7V-R96%T M95]S97-S:6]N#61M7V-R96%T95]U5]S97-S M:6]N#61M7V9I;F1?979E;G1M2!I="!O<&5N6YC:')O;F]UF4@;V8- M("`@("`@("!T:&4@9FEL92!C86X@8F4@86QT97)E9"!I;B!%<&ES;V1E+B!# M:&%N9VEN9R!T:&4@9FEL92!S:7IE('-H;W5L9`T@("`@("`@(&YO="!C875S M92!%<&ES;V1E('1O(&%L;&]C871E(&1I0UD;5]S>6YC7V)Y7VAA;F1L90UD;5]U<&=R861E7W)I9VAT#0T-1&5S:7)E M86)L90TM+2TM+2TM+2TM#0UD;5]D;W=N9W)A9&5?2!N:6-E(&9E871U2P@86QT M:&]U9V@@=&AI2!S964@;F\@;F5E9"!F;W(@=&AI0UD;5]S>6UL:6YK7V)Y7VAA;F1L90T- M#2HJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ M*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ*BHJ#0U(4%-3(&%N M9"!42P@;F1A=&]R:65S(&%R92!N96-EM@``'K<``![;```>W```'R```!\H```?*0``'U0``!]5```? M?0``'WX``!^,```?F```'YD``!^>```?I```'ZT``!^N```?P```']```!_1 M```?V```'_@``!_Y```@2@``(&@``"!I```@=```('\``""````@P@``(00` M`"%)```AC```(:(``"&C```AK0``(;<``"&X```AR@``(=X``"'Q```B`@`` M(A,``"(C```B,P``(D$``")2```B8```(G```"*!```BCP``(IX``"*O```B MP@``(M,``"+A```B\```(P8``",8```C*```(SH``"-*```C7```(W```"-_ M```CB@``(Y@``".F```CM0``(\8``"/4```CXP``(_0``"0%```D%@``)"<` M`"0H```D:P``)*P``"3N```D_@``)/\``"4+```E&0``)2H``"4Z```E.P`` M)8,``"7+```F%0``)EL``":'```FB```)I;[^_O[^_O[^_O[^_O[^_O[^_O[ M^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[ M^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_L````$```!(<`` M#@!C```FE@``)JX``";````FT0``)M(``";3```FW@``)ND``";J```F_0`` M)OX``"=$```G4```)U$``"=@```G80``)Z,``"?I```H+P``*'(``"BT```H MQP``*,@``"C8```HV0``*2(``"DY```I.@``*4@``"E)```ICP``*:0``"FE M```IN@``*;L``"H!```J%@``*A<``"HE```J)@``*FT``"J9```JF@``*J8` M`"JG```J[0``*S(``"M)```K2@``*U4``"M@```K80``*W4``"N'```KF``` M*ZL``"N[```KRP``*]P``"OQ```K\@``*_,``"Q$```L10``+)4``"S1```L MT@``+.D``"SJ```L\0``+/<``"T"```M"P``+0P``"T>```M'P``+2<``"TH M```M*0``+2H``#)!```R]```,T8``#-K```SI```,_0``#0M```TBP``-)H` M`#2]```T[0``-0(``#4#^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[ M^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[^_O[]OO[^_O[^_O[^_O[ M^_O[^_OPZO#PZO#P\/#P\/``````!0```AN```X8`0``!0```1N```X8`0`` M!````B'```X```0```$AP``.`%QE2!N M;W0@8F4@87,@4')O:F5C="!3=&%R="`H5')A;G-A2!W:&EC:"!W:6QL(&YO="!F=6YC=&EO;B!U M;F1E0VE17!I7!E(`D@"2!T:`T) M(`D@("TM+2TM+2TM+2T)(`T-"4]T:&5R(&ES2X@1$5#(#DV17!I&```WNP``-[P``#?!```WS```-_$` M`#P````\+```/#\``#Q1```\;P``/',``#XH```^1P``/DD``#Y*```^30`` M/FP``#Z+```^M```/M0``#\6```_&```0"T``$!,``!`30``0$X``$!/``!! M.```03H``$%F``!!9P``06@````````````````````````````````````` M`````````````````````````````````````````````````````````/D` M```````````````````````````````````````*`$```A@`````&```8@`` M-0,``#4A```U)P``-3P``#51```U80``-6T``#6:```UM```-;X``#72```U MTP``->\``#8"```V#P``-E```#9B```V8P``-P```#=7```W6```-\8``#?Q M```\+```/'H``#Q[```\R0``/0L``#T,```]$P``/10``#U=```]J0``/:H` M`#VU```]O0``/<4``#W+```]S```/A4``#Y(```^20``/DH``#Y+```^3``` M/Q8``#\7```_&```/V```#^Q```__P``0$T``$!.``!`3P``0)<``$#:``!! M(0``03@``$$Y``!!.@``068``$%G``!!:```06D``$%J``!!E@``09<``$&8 M``!!F0``0<0``$'0``!!X@``0>,``$'D``!&>```1GD``$9Z``!*O0``2KX` M`$Z&^OKZ^OKZ^OKZ^OKZ^OKU]?7Z[_KF^N#U]?7U]?7U]?7U]?7U]?7U]?7U M]?7;]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7UT\[U]?7UR0````0```@A MP``.```$```)(<``#@``!P```2'```X1!E03_>0```0```,AP``.```%```` M`````!@!```(```!!5``#A4`/!8`/!@!```%```"&X``#A@!```$```!(<`` M#@``!0```1N```X8`0!/4WES=&5M(%1E6YC<&]S=&-R96%T92`-;6%Y(&YO="!B92!N96-E6YC:')O;F]U,``$'E``!$````1.X` M`$3O``!%DP``1:```$80``!&$P``1G<``$9X``!&>0``1GH``$GW``!)^``` M2?X``$H(``!*$@``2AH``$HE``!*+@``2C4``$I```!*00``2DH``$I3``!* M70``2F@``$J?``!*O```2KT``$J_``!+*@``3*H``$VW``!-N0``3<8``$WN M``!-\@``3@(``$XA``!.+@``3C,``$Y8``!.<@``3G,``$Y^``!.A0``4"<` M`%39``!5Z```6````%D)``!9"@``68(``%F$``!9A0``6:$``%FB``!9I``` M6;8``%G6``!9[P``6F@``%\V``!?.```7_<``&`Q``!@,P``83,``&%W``!A MB```8<$``&'N``!A^0``8A@``&(X``!B50``8G4``&*:``!BN@``8L0``&+& M``!BYP``8Q\``&-"``````````````#^_@#X``````````#^```````````` M``````````#^_@``````````````````````````^``````````````````` M``````````````````````````````H`0``"&``````8``&``&)4'!E&-L=7-I=F4@8F]D>2!O9B!K;F]W;&5D9V4@;F5C97-S M87)Y('1O(&-A7-T96T@=F5N9&]R7-T96US(&%S(&5V:61E M;F-E9"!B>2!T:&5I7-T96TN(%1H97D@87)E(&%W M87)E(&]F(%1R86YS87)C)W,@<')O<')I971A2!C:&%N M9V5S(&EN=&5G2X)86-C97-S;6%N M86=E;65N='%U86YT:71I97-4'!L86YA=&EO;B!A8V-O2!L86)O7-T96T@*$1&4RD@ M9&5V96QO<&5D(&)Y(%1R86YS87)C($-O7-T M96T@8V%P86)L92!O9B!S=&]R:6YG('1O('1H;W5S86YD2!C:&%N9VEN9R!A;F0@87)C:&EV86P@9&%T84]P96X@4V]F='=A2!E>'1E;F0L('1H92!R97-U;'1S(&]F('1H92!$871A($UA;F%G96UE;G0@ M26YT97)F86-E2!E;7!L;WEE97,@;V8@=F%R:6]U7-T96US('9E;F1O7-T96T@=F5N9&]R M2!T:&4@2%--('9E;F1O7-T96T@=F5N9&]R2!T:&4@2%--('9E;F1O M7-T96T@=&\@8F4@86)L M92!T;R!T7-T96T@86YD(&9I M;&5S>7-T96T@7-T96US+@T-"51R86YS87)C('=I M;&P@9&5V96QO<"!T:&4@1$U!4$D@9F]R($1&4RP@=VET:"!E>'1E;G-I;VYS M(&YE961E9"!B>2!(4%-3(&]U=&QI;F5D(&QA=&5R+B`@5&AE($A04U,@8V]L M;&%B;W)A=&EO;F-A;G1H96X@9&5V96QO<"!T:&4@2%!34R!S<&5C:69I8R!$ M871A($UA;F%G96UE;G0@07!P;&EC871I;VX@4')O8V5S'1E;G-I;VYS(&YE M961E9"!B>2!(4%-3(&]U=&QI;F5D(&QA=&5R+B`@5&AE($A04U,@8V]L;&%B M;W)A=&EO;B!C86X@=&AE;B!D979E;&]P('1H92!(4%-3('-P96-I9FEC($1A M=&$@36%N86=E;65N="!!<'!L:6-A=&EO;B`H1$U!4"D@=&AA="!I;G1E7-T96TL(&%N9"!A9&1I;F<@7-T96T@;V)J96-T2!E;F0@=7`@=&%K:6YG('5P(&1I M6YC M:')O;FES;2!N96-E2!T;R!T'!O'!O'1E;G-I;VYS(&%D9&5D('1O('1H92!$1E,@8VQI96YT M('1O('!R;W1O8V]L+65X<&]R=&5R('!R;W1O8V]L('1O(&AA;F1L92!N97<@ M97)R;W)S(&%N9"!T;R!L;V]K(&%T('1H92!A9&1I=&EO;F%L(&%T=')I8G5T M97,@;V8@9FEL97,L(&5G+"!W:&5T:&5R(&$@9FEL92!I2!K;F]W;B!A'1E;F1E9"!T;R!B92!A8FQE('1O(&1E86P@=VET:"!D871A+6UI9W)A M=&EO;B!I2!T;R!T:&4@9&%T82!M:6=R871I;VX@2!TP`` M;(X``&S```!LSP``;-```&S>``!LX```;.$``&T:``!M&P``;3T``&UO``!M MBP``;:L``&V_``!MWP``;@,``&XC``!N10``;D8``&Y\``!NG```;K4``&[5 M``!NXP``;NX``&\.``!O$P`````````````````````````````````````` M```````````````````````````````````````````````````````````` M``````````````````````````````````````!E``!O$P``;SH``&]$``!O MB0``;Y```&^R``!OT@``;]D``&_:``!OY```<`,``'`$``!P!0``<`8``'`. M``!P%P``<%4``'!6``!P@``?H@``'ZP``!^V0``?R0``']@``!_80``@E,``()4``""7@``@E\` M`(*G``""S0``@LX``(+@``""X0``@PX``(-$``"#@0``@[L``(/U``"$"@`` MA#T``(1S``"$K@``A-D``(4"``"%%OOSZ^/[^]O[T-#0T./(X^/CP./C\^/C MM*JAP./(X_O[X^/[^_N4E)24E)24E)24E(<````````````````````````` M```,``````````\.``0!N!#@%H`:N`````````P```$AP``.#PX`!`&X$.`6 M@!JX````````"````1N```X5`"@6`"@8`0``"0```1N```X1`6@5`"@6`"@8 M`0`+```!&X``#A$"T!/^F!4`*!8`*!@!```'```%(<``#@\%``$!:```!P`` M`B'```X/!0`!`6@```H```$AP``.$0%H$_Z8#P4``0+0```'```#(<``#@\% M``$!:```!P```2'```X/!0`!`6@```<```@AP``.#P4``0%H```'```````` M``\%``$!:```!````2'```X`,2!T:&4@9&5L:79E7-T96US(%-O;&%R:7,@;W!E&ES=&EN9R!$1E,@ M``"$9@``A&<``(1L``"$;0``A&X``(1R``"$6YC<&]S=&-R96%T92`H861D:71I;VX@+RTM+2TM+2TM M+6AI9VAL>2!D97-I&EM871E;'D@,2!Y M96%R+@T-0V]S="!O9B!M86YP;W=E65A<@DR('!Y#3(N"4-H86YG97,@=&\@=&AE(&9I M;&5S970@65A<@DQ('!Y#3,N(`E);7!L96UE M;G0@:V5R;F5L(&1E=FEC92!D0TT+B`);7!L96UE;G0@=7-E65A<@DP+C65A<@DQ('!Y#37-T96T@=&5S="!W:71H(')E65A<@DQ('!Y#3@N"4UA;F%G92!S;W5R8V4L(&)U:6QD M2`-.2X)1&]C=6UE M;G1A=&EO;@DQ('!E65A"`D,38P2RD))#$U,C!+#0U4:&4@86)O=F4@8G)E86MD;W=N('-H;W=S M('1H870@=&AEUQB9B!# M;W-T(&]F(&5Q=6EP;65N=#I](%Q<#3@X+B!'AX>'AX>'@@7#T@-B!W'@@7#T@7"0Q,C!+(%QK:6QL#3$N(%P^(%=O"!<)#$T2R`]"5P^(%PD,CA+(%Q<#2!"!<)#1+("`]"5P^(%PD M-$L@7%P-("`@7#X)"0D)"5P^"5P^"0E'AX>'AX7#UX M>'AX>'AX>'AX>'AX>'AX>'AX>'AX>'AX>'AX>'AX>'A'AX>'A'AX(%QK:6QL#5QB9B!4;W1A;"!C;W-T(&]F('!R;VIE8W0Z(%Q<#5P^"4-O M65A0``BKL``(K9``"+#P``BT4``(M?``"+ M>@``BWT``(N2``"+O@``B^8``)$C``"1?@``D94``)&L``"1N@``D;L``)() M``"2"@``ER(``)^OK;7KM>EFP`)```!&X``#A$!:!4` M%!8`%!@!``@```$;@``.%0`4%@`4&`$```<```4AP``.#P4``0%H```'```& M(<``#@\%``$!:```"````2'```X%`0\%``$!:```!P```B'```X/!0`!`6@` M``<`````````#P4``0%H```'```!(<``#@\%``$!:```!P```2'```X/!0`! M%H````0```$AP``.```$```"(<``#@``!`````````````D`````````#P@` M`@&X&E0````]"2`@<&5RF``"7J@``E[(``)>V``"7O0``E\8``)?/``"7W@``F`H``)@0``"8 M%```F!4``)A:``"8:@``F'$``)B#``"8B@``F*H``)C$``"9)@``F2P``)E^ M``"9?P``F8(``)F#``"9A```F84``)F&``"9B@``FP8``)L*``";#@``FQ(` M`)L6``";&@``FQX``)LF``";5P``FW8``)O0``";U```F]P``)OK``";[@`` MF_H``)O^``"<#0``G"<``)Q&``"2!);F9OF%T:6]N(&%N9"!A8V-E M2!I;B!T:&4@1$U!4$D@'1E;G-I M;VX@86YD($1&4R!I;7!L;65N=&%T:6]N('1O($1A=&$@36%N86=E;65N="!! M<'!L:6-A=&EO;B!02!T:&4@2%!34R!T96%M(&1U2!S=&%T96)O=&@@;F5E9&5D'1E;G-I;VYS+C4N,`E!9W)E960@979E;G1S-2XQ"4%D;6EN:7-T6YC<&]S=&-R96%T92DN($EF(`8``$`N``!`+P``00T``$$. M``!!%0``018`````G),``)X=``">'P``GB$``)XC``">)0``GB<``)XI``"> M*P``GBT``)XO``">,```GC$``)XR``">-```GC8``)XX``">.@``GCP``)X^ M``">0```GD(``)YA``">8P``GGD``)YZ``">@@``GH@``)Z;``">H```GK0` M`)[U``"?````GP8``)\?``"?(```GR$``)]```"?00``GV```)^!``"?@@`` MGY<``)^8``"?G```GZ(``)^C``"?I```G\0``)_%``"?Y0``I````*0@``"D M0```I$$``*1"``"D0P``I$P``*1-``"D;0``I(T``*2M``"DK@``I,L``*3, M``"DS0``I.H``*3K``"D[```I.T``*4*``"E"P``I0P``*4-``"E*@``I2L` M`*4L``"E+0``I2X``*4O``"E,```I4T``*5.``"E40``I5(``*5=``"E7@`` MI6(``*5Q``"E<@``I7,``*62``"EE0``I98``*67``"EI@``I>D``*90``"F M5@````````````````````````````````````````#^_OT```````````#] M_0``````]P````````````````````````````````````````#]_?T`_?T` M``#]_0``````"@!```(8`````!@``8`!@@!B``">'P``GD,``)\@``"?HP`` MIE@``*:]``"FYP``IN@``*<=``"GD```IZ4``*>F``"GIP``IZT``*>N``"G MKP``I[```*>S``"GM```I[H``*>[``"GO```I[T``*>^``"GQ```I\4``*?& M``"H````JQ(``*L8``"K&0``JQH``*MV``"K>```JWD``*M_``"K@```JX$` M`*N'``"KB```JXX``*N/``"KD```KM\``*]1``"OK??OY^'9T>_$NK*MIZ>M MK;JMIZ>MK:VGIZVMK:>GK:V?K:>GK:VMIZ>MK:VMK0`````````````````` M```````````````````````````````````````````````````````````` M````````````````````````````````````````!P```2'```X/!0`!#W@` M``7S````````!0$```0````````````'``````````\%``$!:```"0```2'` M``X/"``"`6@/>``````,```!(<``#A$"'`\+``,#A!.P&E0```````<```$A MP``.#P4``0(<```'```!(<``#@\%``$"T```!0```2'```X1`M``!P```2'` M``X1`X03_I@```<```$AP``.#P4``0%H```(```"&X``#A4`%!8`%!@!`"T@ M86)O2!A="!40DJ;W-T0D)*G-T M.``"GCP``IY```*>D``"GI0``IZ8``*>G``"GJ```IZD``*>J M``"GK```IZT``*>N``"GKP``I[```*>Q``"GL@``I[,``*>T``"GM0``I[8` M`*>W``"GN0``I[H``*>[``"GO```I[T``*>^``"GOP``I\```*?!``"GPP`` MI\0``*?%``"GQ@``I^4``*?P``"G^@``I_\``*@```"K$0``JQ(``*L3``"K M%```JQ4``*L7``"K&```JQD``*L:``"K=P``JW@``*MY``"K>@``JWL``*M\ M``"K?@``JW_[``````#[``#Z^@#Z`/H`````````^OKZ``#Z```````````` M^?D`````````\P``````^@``````\P``````````\P```````/KZ`.T```#S M`````/H`````\P``"@!```(8`````!@`"@!``````````!@``8(!@`@`!``` M`````EX`#0`0``#______________P``%/________________\$`!``$``7 M_________________P<````````!``[S`/0```#V```````````````````` M``````#>``T`$```______________\``!3_________________!``0`!`` M%_________________\'`````````0`.\P#T````]@`````````````````` M````````W@`-`!```/______________```4_________________P0`$``0 M`!?_________________!P````````$`#O,`]````/8````````````````` M`````````-X-T"`"(-`-#0T`#0`0``#______________P``%/__________ M______\$`!``$``7_________________P<````````!``[S`/0```#V```` M``````````````````````#>#0P-#=`@`B#0#0T->'AX>'@-#=`@`B#0#0T- M8F5T=V5E;B!L;V-A;&AI97)A6YC M:')O;FEZ871I;VYS=7!P;W)T1&5S:7)A8FQE9&5S:7)A8FQE#0`-`!```/__ M____________```4_________________P0`$``0`!?_________________ M!P````````$`#O,`]````/8``````````````````````````-YD;5]S>6UL M:6YK7V)Y7VAA;F1L90D)#0`-`!```/______________```4____________ M_____P0`$``0`!?_________________!P````````$`#O,`]````/8````` M`````````````````````-X-``T`$```______________\``!3_________ M________!``0`!``%_________________\'`````````0`.\P#T````]@`` M````````````````````````W@```````%+10`(```$!````````"=H``!5" M```>N@``*:(``#(-```YF```/RL``$2^``!'W@``3#(``%#K``!2T1`"__\` M```!`XL``O__`````@"8$`+__P````,!#P`"__\````$`"X0`O__````!0"X M``+__P````8`)A`"__\````'``$``O__````"``!$`+__P````D`)``"__\` M```*`!00`O__````"P`!0`+__P````P````````````;````*@```%8```!7 M````6````&X```"4````E0```)8```#"````PP``!#H```0[```%,P``!30` M``=/```'4```"=D```G:```-90``#68``!"9```0F@``$:(``!&C```2JP`` M$JP``!*^```2OP``%%P``!1=```5V@``%=L``!7P```5\0``%WH``!=[```9 M$0``&1(``!E_```9@```&S$``!LR```<@```'($``!Y[```>?```'IT``!Z> M```>N0``'KH``"`9```@&@``(#```"`Q```@S@``(,\``",#```C!```)7<` M`"5X```FDP``)I0``":G```FJ```*`P``"@-```I40``*:$``"FB```IT``` M*=$``"J.```JCP``*\4``"O&```KW@``*]\``"Z-```NC@``+IX``"Z?```O M;0``+VX``"^#```OD0``+[```"_2```OTP``,'(``#!S```PE@``,)<``#(, M```R#0``,L4``#+&```T*P``-"P``#6U```UM@``-WH``#=[```W^```-_D` M`#@1```X$@``."P``#@M```X-```.#4``#B]```XO@``.,4``#C&```Y=``` M.74``#F````YB```.9```#F6```YO@``.<```#G'```YY0``.C8``#I6```Z MB@``.M8``#K=```[%@``.U```#M7```[DP``.Z0``#O)```[^P``/!(``#PU M```\/0``/%,``#QI```\>@``/(<``#RV```\T@``/-X``#ST```]$```/24` M`#TT```]4```/9L``#V[```]P@``/=L``#XT```^9```/FL``#ZG```^S0`` M/MD``#[^```_*P``/RP``#\M```_H@``/Z,``#^K```_K```/](``#_3``!` MG0``0)X``$"T``!`M0``0+P``$"]``!!/0``03X``$%)``!!2@``0G@``$)Y M``!"GP``0J```$-_``!#@```0X<``$.(``!#DP``0Y0``$1V``!$=P``1)P` M`$2=``!$O@``1+\``$3&``!$T0``1-(``$3W``!$^```11D``$4:``!%'P`` M12@``$4I``!%3P``15```$5S``!%=```194``$66``!%G@``19\``$6K``!% MK```1=(``$73``!%^0``1?H``$8;``!&'```1CT``$8^``!&_P``1P```$W``!'N```1\@``$?)``!'S@``1]0` M`$?=``!'W@``2`(``$@#``!(%P``2!@``$@?``!(0```2$$``$BP``!(L0`` M2,```$C!``!*?```2GT``$J-``!*C@``2J```$JT``!*QP``2M@``$KI``!* M^0``2PD``$L7``!+*```2S8``$M&``!+5P``2V4``$MT``!+A0``2Y@``$NI M``!+MP``2\8``$O<``!+[@``2_X``$P0``!,(```3#(``$Q&``!,50``3&`` M`$QN``!,?```3(L``$R<``!,J@``3+D``$S*``!,VP``3.P``$S]``!,_@`` M3;(``$VS``!-OP``3``!-[@``3>\``$\3``!/%```3R(``$\Z``!/ M3```3UT``$]>``!/;@``3V\``$^"``!/@P``3Y\``$^@``!/L```3[$``%#K M``!0[```4/P``%#]``!1"P``40P``%$A``!1(@``43```%$Q``!1/0``43X` M`%%/``!14```460``%%V``!1AP``49H``%&J``!1N@``4'P``KZT``@`*`!<`&``=`"L` M/0!'`%$````"``,`!``.`!``%``5`!8`%P`A!`$$`@<`!]`'T0?2!],'U`?6 M!]<'V`?A!^@'[0?T!_8'^`?^)G`F6UB;VP````A"T%V86YT($=A7!E(%-O6QE```'U`Y#96YT=7)Y($=O=&AI8P``!]8036]N;W1Y<&4@ M0V]R'0```?V!$]N>7@```?X!D]X9F]R9```!_X%4W=I M;F<``"9P"45S<'D@4V%N2!386YS($)O;&0``#G9#&57;W)L M9"!4:6=H=```.GX/2&5L=F5T:6-A($)L86-K@`$!````4L8``%+&``:``(`` M``!2Q@`%````30!3`OT"30!2`!X#`@(8`0`,%```%P4``0%H`#R!`0`*%``` M%P4``0%H``$`"Q<)`0%H`!D!`AP``0`'%P4``0%H``$`$A$``!,``!0``!<% M``$!:``\@`$`$!$``!,``!0``!<%``$!:``!``T1```3```7!0`!`6@``0`. M$0(<$P``%P8!`6@`&0`!``T7"0$!:``9`0(<`#R!`0`)%P4``0(<`#R!`0`2 M$0+0$_Z8%PH"`M`$.``9`!D``0`$/(%"@0$`#A0``!<%``$!:``\@4*!`0`2 M%```%PD!`6@`&0$"'``\@4*!`0`0%```%PD!`6@`&0$"'``\@0$`$1$#A!/^ MF!0``!<&`0%H`!D``0`8!0`4```7$0,!:!`L$D@`&0`9`!D!#W@``0!%!P`( M``D`#@`0```1```3```4```5`!06`!07!0`!`6@`&@``&P``'```'0`>```? M```@```A```B```C```D```X`@```0!*!P`(``D`#@`0```1```3```4```5 M`!06`!07"@(!:`+0`!D`&0`:```;```<```=`!X``!\``"```"$``"(``",` M`"0``#@"```!`#@'``@`"0`.`!```!0``!4`%!8`%!H``!L``!P``!T`'@`` M'P``(```(0``(@``(P``)```.`(```$`2@<`"``)``X`$```$0+0$_Z8%``` M%0`4%@`4%PH"`6@"T``9`!D`&@``&P``'```'0`>```?```@```A```B```C M```D```X`@```0!*!P`(``D`#@`0```1`FP3_N@4```5`!06`!07"@(!:`+0 M`!D`&0`:```;```<```=`!X``!\``"```"$``"(``",``"0``#@"```!`"X4 M```7!0`!`6@`HP$"`@``HP`!!0!`HP`!`@!`HP`!"`!`HP`"!0!`HP$""`!` M`0`6%```%P4``0%H`*,``0$`0*,``@\`0`$`2@<`"``)``X`$```$0%H$P`` M%```%0`4%@`4%PH"`6@"T``9`!D`&@``&P``'```'0`>```?```@```A```B M```C```D```X`@```0!*!P`(``D`#@`0```1`;@3```4```5`!06`!07"@(! M:`+0`!D`&0`:```;```<```=`!X``!\``"```"$``"(``",``"0``#@"```! M`#X'``@`"0`.`!```!$!N!,``!0``!4`%!8`%!H``!L``!P``!T`'@``'P`` M(```(0``(@``(P``)```.`(```$`$!0``!<%``$!:`"C``(/`$`!`$H'``@` M"0`.`!```!$";!/^_!0``!4`%!8`%!<*`@%H`M``&0`9`!H``!L``!P``!T` M'@``'P``(```(0``(@``(P``)```.`(```$`&1$#A!/^F!0``!<.`_S@_[`! M:``9`!D`&0`!``X1`X03_I@7!@$!:``9``$`%1$#A!/^F!0``!<*`@%H`X0` M&0`9``$`$A$#A!/^F!<*`@%H`X0`&0`9``$`#A0``!<)`0%H`!D!`AP``0`4 M$0(<$P``%```%PD!`6@`&0$#A``!``P4```7!0`!`6@`10$!`!84```7$0,! M:`(<`FP`&0`9`!D!`M```0`1$0+0$P``%```%P8!`6@`&0`!`!$1!#@3```4 M```7!@$!:``9``$`#A0``!<)`0%H`!D!`M```0`.$0+0$P``%P8!`6@`&0`! M`!(4```7#0(!:`(<`!D`&0$"T``!`!$1`6@3```4```7!@$!:``9``$`$`4` M%```%P4``0%H`'8`@`@"*VP`````````"0```!H````I````*@```%8```!H M````;0```&\```"%````BP```)(```"3````E````)8```":````P0```,(` M``##````Q````/0```#W```!,````9H```&A```!Y0```@0```(&```"'0`` M`B\```)/```"L````Q<```,V```#.0```SH```-,```#D0``!`(```0$```$ M$0``!#D```0Z```$.P``!#P```1<```$9P``!&L```1]```$C0``!,\```38 M```$^```!30```4U```%?0``!9P```6E```%L@``!@````8&```&*@``!C0` M``9$```&:P``!G````9Q```&>P``!G\```:D```&J```!N0```;^```'`0`` M!P(```<4```''```!U````=1```'CP``!YH```@-```(%@``",T```C9```( MX```".H```CU```(^```"3````DW```)5P``":L```FR```)O```"<,```G5 M```)V```"@H```H3```*JP``"M<```L>```+'P``"WD```PM```,-```#68` M``UI```-S0``#=8```W7```-VP``#E4```YU```.CP``#R8```]0```/70`` M#VX```]Z```/B@``#YX```^H```/K@``$$4``!"7```0F```$:$``!&C```1 MI```$>4``!'I```1Z@``$>L``!('```2"```$@H``!(2```2%0``$B,``!(\ M```2BP``$JH``!*K```2K```$K```!*]```2O@``$L@``!+)```39P``$VD` M`!1;```57@``%6T``!6@```5KP``%=D``!7;```5WP``%>\``!7P```60P`` M%F,``!9X```6>@``%RH``!=*```78```%V@``!>?```7H@``&-\``!D$```9 M$```&<\``!H)```:"P``&PL``!LT```;10``&WX``!NK```;M@``&]4``!OU M```<$@``'#(``!Q7```<=P``'($``!R#```3P``'F,``!Z>```>H@`` M'K@``!ZY```>N@``'KL``![[```>_```'R(``!]"```?9```'X0``!^B```? MS```'^P``"`,```@&@``(!X``"`O```@,```(#$``"`R```@,P``(#L``"!B M```@8P``((4``""E```@SP``(-```"#Q```A+@``(5(``"&.```ALP``(=,` M`"'K```B"P``(DD``")-```B<@``(HP``"*-```BD@``(K,``"+3```C!``` M(P4``",F```C1@``(U(``"-Y```CM```(](``"/R```D!P``)`P``"0-```D M*P``)#@``"1=```D?0``)*```"3````DXP``)1P``"4I```E/0``)5T``"5X M```E>0``)9,``"6S```EU@``)?8``"86```F&@``)AX``"8^```F40``)H,` M`":+```FDP``)I0``":8```FI@``)J<``":H```FJ0``)N(``";C```G!0`` M)S<``"=3```GG```GRP``)^L``"@-```H#@``*$0``"AD```H M?0``*)T``"BK```HM@``*-8``"C;```I`@``*0P``"E1```I6```*7H``"F: M```IH0``*:(``"FF```IL```*<\``"G0```IT0``*=(``"G:```IXP``*B$` M`"HB```J/P``*E\``"J/```JD```*K8``"K6```J_0``*QT``"M#```K8P`` M*X<``"NG```KQ0``*\8``"O*```KUP``*]T``"O>```KWP``*^```"P!```L M"0``+"(``"PJ```L20``+%@``"Q<```L;```+'L``"Q\```L?0``+*(``"S" M```LVP``+.@``"T(```M+```+4P``"UO```MCP``+:@``"VM```MS@``+=P` M`"X#```N#P``+A@``"XH```N+0``+C$``"XV```N0@``+F```"Y]```NC``` M+HX``"Z2```NG0``+IX``"Z?```NH```+L,``"[+```NY```+N4``"\*```O M*@``+VT``"]N```O<```+X(``"^#```OA0``+Y$``"^3```OL```+[(``"_2 M```OTP``+_,``"_\```P&P``,%(``#!A```P9```,'```#!S```P=P``,((` M`#"%```PA@``,)4``#"6```PTP``,.T``##N```Q#0``,54``#%9```QF``` M,9\``#(-```R'@``,GP``#+$```RQ0``,L8``#,8```S(P``,S```#-!```S M8```,XL``#.0```SOP``,],``#/<```SX```,_0``#/_```T'@``-"D``#0J M```UM```-;4``#6V```UMP``-C```WJ```-ZD``#>P```W ML0``-\P``#?-```WU```-_<``#?X```W^0``.`0``#@*```X$```.!$``#@2 M```X*P``."P``#@M```X,P``.#4``#@V```X7```.'L``#C&```XQP``..@` M`#D'```Y4@``.7,``#F6```YF```.:(``#FL```YL0``.;T``#F^```YOP`` M.<```#G&```YQP``.>0``#GE```Z'0``.AX``#HU```Z-@``.C<``#HY```Z M50``.E8``#I7```Z60``.HH``#J+```ZC0``.KH``#J[```ZU```.M4``#K6 M```ZV0``.MP``#K=```[%0``.Q8``#M.```[3P``.U```#M3```[5@``.U<` M`#ME```[?@``.Y(``#N3```[E```.Y8``#NC```[I```.Z4``#NG```[R0`` M.\H``#O,```[^P``._P``#O^```\$0``/!(``#P?```\)```/#0``#PU```\ M-@``/#@``#P]```\00``/$(``#Q#```\1```/%(``#Q3```\6```/%D``#Q: M```\:0``/&T``#QN```\;P``/'```#QZ```\?@``/'\``#R````\@@``/(<` M`#R(```\B@``/+4``#RV```\MP``/+D``#S2```\TP``/-4``#S>```\WP`` M/.$``#SS```\]```/1```#T1```]$P``/20``#TE```])@``/2@``#TT```] M-0``/3<``#U/```]F@``/;D``#VZ```]NP``/<$``#W"```]V@``/=L``#W< M```]W@``/?4``#X5```^,P``/C0``#YB```^8P``/F0``#YG```^:```/FH` M`#YK```^H```/J$``#ZF```^RP``/LP``#[-```^S@``/M(``#[8```^V0`` M/OT``#[^```_*0``/RH``#\K```_+```/RT``#\N```_6@``/V\``#^"```_ MH0``/Z(``#^C```_J@``/ZL``#^L```_K0``/\<``#_0```_T0``/](``#_3 M```_U```/_,``$`2``!`.P``0%L``$!]``!`G```0)T``$">``!`H@``0*T` M`$"S``!`M```0+L``$"\``!`O0``0+X``$#?``!`_P``04@``$%*``!!2P`` M06L``$&+``!!O```0=P``$(*``!"*@``0E@``$)W``!">0``0GH``$*4``!" MG0``0J```$*A``!"O```0MP``$,"``!#(@``0T4``$-E``!#A@``0X<``$.2 M``!#DP``0Y0``$.5``!#M0``0]4``$/X``!$&```1#\``$1?``!$=@``1'<` M`$1X``!$?0``1)H``$2=``!$G@``1)\``$32``!$TP``1-@``$3U``!$^``` M1/D``$4G``!%*0``12H``$5/``!%4```15$``$54``!%<0``170``$5U``!% MG0``19X``$6J``!%JP``1:P``$6M``!%T@``1=,``$74``!%V@``1?<``$7Z M``!%^P``1AP``$8=``!&/@``1C\``$9]``!&?P``1L0``$;%``!'````1P$` M`$7``!'N``` M1[L``$>\``!'QP``1\@``$?<``!'W0``1]X``$??``!(`@``2`,``$@'``!( M%@``2!<``$@?``!((```2$$``$A"``!($``%(#``!2!```4@8``%)5``!25@``4JD``%*J``!2L```4L4` M`%+&``!2QP``4L@``%+)``!2S0``4LX``%+/``!2T```4M&``@``E\8``(`" M```!!#R!``(``)?/`````@```16````"``"7W@````(```$6@`"``@``JX$` M```"```!*(``@`(```RC@`&``@``F`H``(`"```,N8`!@`(```%*@````@`` M#**``0`"```!88`!@`(``)@0``"``@```6.````"``!*O(`"``(``$J]@`.` M`@``F8(``(`"``!*OX`#@`(``)=A``"``@``2O&``X`"``!+*H`#@`(``)=D M``"``@``2YJ``X`"``"3)0``@`(``$O^@`.``@``DT0``(`"``!,%X`#@`(` M`)-;``"``@``3$F``X`"``!,JH`#@`(``)-[``"``@``33&``X`"``"8%``` M@`(``$TT@`.``@``F!4``(`"``!-1H`#@`(``$VW@`.``@``3;F``X`"``!- MQH`#``(```&*@`0``@```8N`!(`"``"9@P``@`(```<[@`6``@``!UN`!8`" M``!-[H`&@`(```=K@`6``@``3?*`!H`"```'@8`%@`(```?#@`6``@``!\R` M!0`"```'[(`%@`(``)F$``"``@``""F`!8`"``!.`H`&@`(```B@@`6``@`` M3B&`!H`"```(J8`%@`(``$GX@`6``@``"/R`!8`"``!)_H`%@`(``)A:@`:` M`@``"4&`!8`"``!.+H`&@`(```EL@`6``@``2@B`!8`"```)>(`%@`(``$XS M@`:``@``"9J`!8`"```)GH`%@`(``$Y8@`:``@``"=J`!8`"``!.`!8`" M```,)X`%@`(``)AJ@`:``@``#'V`!8`"```,AX`%@`(``)AQ@`:``@``#(Z` M!0`"``!.A8`&@`(``)=K@`:``@``3L&`!H`"``"3FH`&@`(``$^&@`:``@`` MD\:`!H`"``!/S8`&@`(``%`G@`:``@``F(.`!@`"``!0VX`&@`(``)/'@`:` M`@``4A&`!H`"``"7=(`&@`(``%)^@`:``@``EWV`!H`"``!2@H`&@`(``)B* M@`:``@``F*J`!H`"``!3`H`&@`(``)C$@`:``@``JY```(`"``"8^H`&@`(` M`*N=``"``@``F1:`!H`"``!4%8`&@`(``%0R@`:``@``F2:`!H`"``!408`& M@`(``)DL@`:``@``5-B`!@`"``!8`(`&``(``%39@`:``@``60F`!H`"``!9 M"H`&@`(``%E^@`:``@``F7Z`!H`"``!9A(`&@`(``%F%@`:``@``6:&`!H`" M``!9HH`&@`(``%FD@`:``@``F7^`!H`"``!9R(`&@`(``%G6@`:``@``6>^` M!H`"``!^B8`&``(``'YX@`8``@``?GF`!X`"``"9A@``@`(``'YZ/($``@`` M?H>`"``"``!:7@``@`(``'Z(``"``@``6F@``(`"``"3Z@``@`(``%L(```` M`@``F8H``(`"``"KJ0``@`(``)J;``"``@``J[@``(`"``":W`````(``%OZ M``"``@``FP8``(`"``!;_#R!``(``%P,@`@``@``7`T``(`"``!T,P``@`(` M`%R```"``@``D^P``(`"``!!``"``@``7RH````"``!? M.```@`(``%_W``"``@``8#$``(`"``!@,P````(``&$S``"``@``87<``(`" M``!AB```@`(``&'!``"``@``8>X``(`"``!A^0``@`(``&(8``"``@``8C@` M`(`"``!B50``@`(``&)U``"``@``8IH````"``!BN@``@`(``&+$``"``@`` M8L8``(`"``!BYP``@`(``&,?``"``@``8T(``(`"``!C8@``@`(``&.&``"` M`@``8Z8``(`"``!CS0``@`(``&/M``"``@``8_L``(`"``!D&P``@`(``&1! M``"``@``9'(``(`"``!DD@````(``&2F``"``@``FPH``(`"``!DX3R!``(` M`&3W@`@``@``9/@``(`"``!D^0``@`(``&3Z``"``@``93H``(`"``!E.P`` M@`(``&5A``"``@``98$``(`"``!EHP``@`(``&7#``"``@``9>$``(`"``!F M"P``@`(``&8K`````@``9DL``(`"``";#@``@`(``&99/($``@``9FJ`"``" M``!F:P``@`(``&9L``"``@``9FT``(`"``"6*P``@`(``&9X``"``@``9I\` M`(`"``!FH```@`(``&;"`````@``9N(``(`"``!G#```@`(``&<-``"``@`` M9RX``(`"``!G:P``@`(``&>/``"``@``9\L``(`"``!G\```@`(``&@0``"` M`@``:"@``(`"``!H2```@`(``&B&``"``@``:(H``(`"``"6,P``@`(``)9- M``"``@``:,H``(`"``!HSP``@`(``&CP`````@``:1```(`"``!I00``@`(` M`&E"``"``@``EDX``(`"``!I@P``@`(``&F/``"``@``:;8``(`"``!I\0`` M@`(``&H/``"``@``EFX``(`"``!J10``@`(``):#``"``@``EH0``(`"``!J M:@``@`(``&IW``"``@``:IP``(`"``!JO```@`(``):B``"``@``:O\``(`" M``!K(@``@`(``);"``"``@``:V@``(`"``!K?`````(``&N<``"``@``:[<` M`(`"``!KN```@`(``&O2``"``@``:_(``(`"``!L%0``@`(``&PU``"``@`` MEZ8``(`"``!L5P``@`(``&Q;``"``@``;'L``(`"``!LC@``@`(``)>J```` M`@``;,<````"``!LSP``@`(``)L2``"``@``;-`\@0`"``!LWH`)``(``&S? M``"``@``;.```(`"``!LX0``@`(``&T:``"``@``;1L``(`"``!M/0``@`(` M`&UO``"``@``;8L``(`"``!MJP``@`(``&V_``"``@``;=\``(`"``!N`P`` M``(``&XC``"``@``;D4``(`"``!N1@``@`(``&Y\``"``@``;IP``(`"``!N MM0``@`(``&[5``"``@``;N,``(`"``!N[@``@`(``&\.``"``@``;Q,``(`" M``!O.@````(``&]$``"``@``;XD``(`"``!OD```@`(``&^R`````@``;](` M```"``!OV0``@`(``)L6``"``@``;]H\@8`"``!OY#R!``(``'`#@`D``@`` M<`0``(`"``!P!0``@`(``'`&``"``@``<`X``(`"``!P%P``@`(``'!5``"` M`@``<%8``(`"``!PR``"``@``?1X``(`"``"7M@````(` M`'UC``"``@``ENX``(`"``!]X@``@`(``)Q&`````@``?D`````"```,H8`! M@`(``!=3@`&``@``2C6``8`"```7KX`!@`(``!D%@`&``@``?D$``(`"```9 M%H`!@`(``)R.``"``@``&4&``8`"``!^8```@`(``!EP@`&``@``?G0``(`" M```9>8`!@`(``$IH@`&``@``I\8``(`"``"GY0``@`(``!F-@`$``@``G),` M```"```7O(`!``(```S!@`"``@``EO\``(`"``">8P``@`(``'](``"``@`` M%]B``8`"```7\(`!@`(``)YY``"``@``&`2``8`"``">>@``@`(``)Z"``"` M`@``&`N``8`"```8$(`!@`(``)Z(``"``@``&!:``8`"```8.X`!@`(``)Z; M``"``@``&$Z``8`"```86X`!@`(``!C0@`&``@``J_@``(`"``"NM@``@`(` M`!D#@`&``@``IY```(`"``">R0````(``']?``"``@``EP```(`"``!_80`` M@`(``'^'``"``@``?X@``(`"``!_C0``@`(``'^.``"``@``?Y4``(`"``!_ ME@``@`(``'^Q``"``@``?[(``(`"``!_N0````(```S`@`$``@``IZ\``(`" M``">]0``@`(``!GF@`R``@``GP`````"```:$(`-``(``)8`/@`(``)\@``"``@`` M/(.`#X`"``"?(0````(``#S)@`^``@``GT```(`"```]'(`/@`(``)]!``"` M`@``/5V`#X`"``"?8`````(``#VH@`\``@``JW4``(`"``"G\```@`(```S# M@`"``@``I_H``(`"``!_W@````(```S-@!"``@``#->``0`"```,XX`!@`(` M`'ZH`````@``#9^`$8`"```RUH`1``(``)X=``"``@``,O6`$H`"```S+8`2 M@`(``#,N@!(``@``,T6`$H`"``!^KH`3@`(``)X?``"``@``,TZ`$@`"``!^ MKX`3@`(``'ZP@!.``@``GB$````"```S)0``@`(``#2, M@!(``@``-)F`&(`"```TFH`8@`(``)XG`````@``-)N`&(`"```TO8`8@`(` M`)XI`````@``-+Z`&(`"```T[8`8@`(``)XK``"``@``/"R`&``"```U`H`8 M@`(``#4#@!B``@``?LN`$X`"```U$(`8``(``#4@@!*``@``-2&`&(`"``"> M+0````(``#4B@!B``@``-2>`$H`"``!^T(`3@`(``#4L@!*``@``GB\``(`" M```U+8`2``(``#4[@!F``@``?M&`&H`"```U08`9@`(``)XP`````@``-4*` M&8`"```U48`9@`(``'[6@!J``@``-5:`&8`"``">,0````(``#57@!F``@`` M-6&`&8`"``!^UX`:@`(``#5F@!F``@``GC(````"```U:(`9@`(``#5M@!*` M`@``GC0``(`"```U;H`2``(``#69@!B``@``-9J`&(`"``">-@````(``#6; M@!B``@``-;2`&(`"``">.`````(``#6U@!B``@``-;Z`&(`"``">.@``@`(` M`#P_@!@``@``-=*`&``"```UTX`2@`(``#7O@!*``@``GCP``(`"```U\(`2 M``(``#8!@!B``@``-@*`&(`"``">/@````(``#8#@!B``@``-@^`&(`"``"> M0```@`(``#80@!@``@``?MB`$P`"``">0@````(```^(@!(``@``#XF`&X`" M```VX8`!``(```^0@!&``@``-N>`$0`"```V_X`2@`(``#<`@!*``@``GF$` M`(`"```W`8`2@`(``#<8@!*``@``/%&`$@`"```W5X`<@`(``#=8@!(``@`` M#Y&`$@`"```/DH`;@`(``!=-@`&``@``2D"``8`"```1)H`!``(``!$.@!&` M`@``-X:`$8`"```WNX`1@`(``#>\@!$``@``?R.`$P`"```1#X`2``(``!$0 M@!N``@``-\&``8`"```\;X`!@`(``#?&@`$``@``$1&`$8`"```WS(`1``(` M`#?P@!*``@``/`"`$@`"```1$H`2``(``!$3@!L``@``#Y:``0`"```9CX`! M@`(``*>Q``"``@``/=2`#X`"``"?@@``@`(``#X5@`^``@``/BB`#P`"```^ M1X`/``(``*>/``"``@``&D&`#P`"```^28`/``(``!I(@`^``@``I$$``(`" M```:28`/@`(``$I!@`^``@``&FV`#P`"```^2H`=``(``)`'X`"``"D30``@`(``$"7@!^``@`` MI&T``(`"``!`VH`?@`(``*2-`````@``02&`'P`"```<5(`?@`(``*2M``"` M`@``'%6`'X`"``"DK@````(``!QZ@!^``@``I,L``(`"````'X`"``"D[``` M@`(``!T8@!^``@``I.T````"```=.X`?@`(``*4*`````@``'4:`'P`"``!! M9X`?``(``!UN@!\``@``06B`'P`"``!!:8`?@`(``*4+`````@``07&`'P`" M```=>H`?@`(``*4,``"``@``'7N`'X`"``"E#0````(``!VA@!^``@``I2H` M```"```=K(`?@`(``*4K`````@``'-H`?@`(``"VA@!\``@``'GR`'X`"``"E+0````(``!ZW@!^` M`@``I2X``(`"```>Y(`?@`(``"VB@!\``@``'R"`'X`"``"E+P``@`(``!\I M@!^``@``I3`````"```?4H`?@`(``*5-`````@``'UV`'X`"``"E3@``@`(` M`*51``"``@``I5(````"```?BX`.``(``!^8@`\``@``09>`#P`"``"7)(`> M@`(``*5=`````@``0:&`#P`"```?K8`/@`(``*5>``"``@``I6(````"```? MOX`.``(``!_0@`^``@``I7$````"```?V(`/@`(``*5R``"``@``'_F`#X`" M``"E@`&` M`@``+<.``8`"```A!(`!@`(``$I3@`&``@``,@J``8`"``!*78`!@`(``#(? M@`&``@``(4F``8`"```R((`!@`(``*7I`````@``(:&`(@`"```AHH`B@`(` M`*90``"``@``IE8``(`"```AI(`C``(``"&L@"0``@``(;>``0`"```AR8`E M``(``*97``"``@``)#"`)8`"``"F6```@`(``"1S@"6``@``IG<``(`"```D MM(`E@`(``*:6``"``@``)/:`)0`"```D_8`F``(``"3^@"4``@``,D"`)8`" M```E0X`E@`(``#)"@"6``@``)8R`)8`"```R1(`E@`(``"74@"6``@``,D:` M)8`"```F'8`E@`(``#)'@"6``@``)F.`)0`"```FAH`F``(``":'@"4``@`` M)M*`)8`"``"FM@``@`(``*[,`````@``IKP````"```FW8`G@`(``";J@`$` M`@``)OR`)8`"```G!H`E``(``"=.@"6``@``IKT````"```G48`E@`(``"=I M@"6``@``@D*`*(`"```R=8`E@`(``">K@"6``@``KM4``(`"```GTX`E@`(` M`#*4@"6``@``)_&`)8`"```RE8`E@`(``"@W@"6``@``,I:`)8`"```H>H`E M@`(``#*V@"6``@``*+R`)0`"```HQH`F``(``"C'@"4``@``*3J`)0`"```I MI(`E``(``"H7@"4``@``-D^`)0`"```JFH`E``(``"M)@"6``@``IKX``(`" M```K2H`C``(``"M4@"D``@``*V"``0`"```K=(`E@`(``*\Z`````@``*_*` M)0`"```L1(`!@`(``*;$`````@``IN8````"``"FYP``@`(``"Q%@`&``@`` M-F"``0`"```LE8`!``(``"SH@"H``@``+/"`*@`"```L]H`E``(``()3```` M`@``KZP````"``"KAP``@`(``*N(``"``@``JXD````"``"KC0````(``*N. M`````@``JX\````"```!`(`K```````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M``````^``@``I$$``(`"```:28`/@`(``$I!@`^``@``&FV`#P`"```^2H`= M``(``)`'X`"``"D M30``@`(``$"7@!^``@``I&T``(`"``!`VH`?@`(``*2-`````@``02&`'P`" M```<5(`?@`(``*2M``"``@``'%6`'X`"``"DK@````(``!QZ@!^``@``I,L` M`(`"````'X`"``"D[```@`(``!T8@!^``@``I.T````"```=.X`?@`(``*4* M`````@``'4:`'P`"``!!9X`?``(``!UN@!\``@``06B`'P`"``!!:8`?@`(` M`*4+`````@``07&`'P`"```=>H`?@`(``*4,``"``@``'7N`'X`"``"E#0`` M``(``!VA@!^``@``I2H````"```=K(`?@`(``*4K`````@``'-H`?@`(``"VA@!\``@``'GR`'X`" M``"E+0````(``!ZW@!^``@``I2X``(`"```>Y(`?@`(``"VB@!\``@``'R"` M'X`"``"E+P``@`(``!\I@!^``@``I3`````"```?4H`?@`(``*5-`````@`` M'UV`'X`"``"E3@``@`(``*51``"``@``I5(````"```?BX`.``(``!^8@`\` M`@``09>`#P`"``"7)(`>@`(``*5=`````@``0:&`#P`"```?K8`/@`(``*5> M``"``@``I6(````"```?OX`.``(``!_0@`^``@``I7$````"```?V(`/@`(` M`*5R``"``@``'_F`#X`"``"E@`&``@``+<.``8`"```A!(`!@`(``$I3@`&``@``,@J` M`8`"``!*78`!@`(``#(?@`&``@``(4F``8`"```R((`!@`(``*7I`````@`` M(:&`(@`"```AHH`B@`(``*90``"``@``IE8``(`"```AI(`C``(``"&L@"0` M`@``(;>``0`"```AR8`E``(``*97``"``@``)#"`)8`"``"F6```@`(``"1S M@"6``@``IG<``(`"```DM(`E@`(``*:6``"``@``)/:`)0`"```D_8`F``(` M`"3^@"4``@``,D"`)8`"```E0X`E@`(``#)"@"6``@``)8R`)8`"```R1(`E M@`(``"74@"6``@``,D:`)8`"```F'8`E@`(``#)'@"6``@``)F.`)0`"```F MAH`F``(``":'@"4``@``)M*`)8`"``"FM@``@`(``*[,`````@``IKP````" M```FW8`G@`(``";J@`$``@``)OR`)8`"```G!H`E``(``"=.@"6``@``IKT` M```"```G48`E@`(``"=I@"6``@``@D*`*(`"```R=8`E@`(``">K@"6``@`` MKM4``(`"```GTX`E@`(``#*4@"6``@``)_&`)8`"```RE8`E@`(``"@W@"6` M`@``,I:`)8`"```H>H`E@`(``#*V@"6``@``*+R`)0`"```HQH`F``(``"C' M@"4``@``*3J`)0`"```II(`E``(``"H7@"4``@``-D^`)0`"```JFH`E``(` M`"M)@"6``@``IKX``(`"```K2H`C``(``"M4@"D``@``*V"``0`"```K=(`E M``(``*\Z`````@``*_*`)0`"```L1(`!@`(``*;$`````@``IN8````"``"F MYP``@`(``"Q%@`&``@``-F"``0`"```LE8`!``(``"SH@"H``@``+/"`*@`" M```L]H`E``(``()3``"``@``IND````"``""78`"``(``()>``"``@``@E\` M`(`"``""8```@`(``*;V``"``@``@GD````"``""I@````(``(+,`````@`` M@LT``(`"``"G`@````(``(+?@`D``@``@N```(`"``""X0``@`(``(+D``"` M`@``@N4``(`"``"#"0````(``(,-@"N``@``@PZ`*X`"``"#$(`K@`(``(,1 M@"N``@``@RZ`*X`"``"#+X`K@`(``(,P@"N``@``@S&`*X`"``"#-X`K@`(` M`(,Y@"N``@``@SZ`*X`"``"#/X`K``(``(-#@"N``@``@T2`*X`"``"#1X`K M@`(``(-(@"N``@``@V>`*X`"``"#<(`K@`(``(-Z@"L``@``@X"`*X`"``"# MA(`K@`(``)>]@"N``@``@XV`*X`"``"#H8`K@`(``(.J@"N``@``@[2`*P`" M``"#NH`K@`(``(.]@"N``@``@[Z`*X`"``"#V8`K@`(``(/:@"N``@``@^*` M*X`"``"#XX`K@`(``(/L@"N``@``@^V`*P`"``"#](`K@`(``(/V@"L``@`` MA`F`*X`"``"$#(`K@`(``(0-@"N``@``A">`*X`"``"$*(`K@`(``(0P@"N` M`@``A#&`*X`"``"$-X`K``(``(0\@"N``@``A$"`*X`"``"$08`K@`(``(1= M@"N``@``A%Z`*X`"``"$9H`K@`(``(1G@"N``@``A&R`*X`"``"$;8`K@`(` M`(1N@"L``@``A'*`*X`"``"$``P`" M``"']P``@`(``*<8@"Z``@``D<>`+@`"``"((X`O``(``)$B``"``@``D2,` M`(`"``"(@0``@`(``)$F``"``@``B*0``(`"``"1)P``@`(``(BL``"``@`` MB*\``(`"``"1*`````(``(C!@#"``@``D2V`,8`"``"(R(`P@`(``)$P@#&` M`@``B/6`,(`"``"1.(`Q``(``(D'@#"``@``D3V`,8`"``")#8`P``(``(DN M@#"``@``D3^`,8`"``")-8`P@`(``)%`@#&``@``B5^`,(`"``"12(`Q``(` M`(ER@#"``@``D4V`,8`"``")>8`P@`(``)%.@#$``@``B:N`,(`"``"15X`Q M@`(``(FR@#"``@``D5B`,8`"``")X8`P@`(``)%@@#&``@``D6&`,8`"``"1 M8H`Q``(``(GS@#"``@``D66`,8`"``")^H`P@`(``)%F@#&``@``BAZ`,(`" M``"1;H`Q@`(``)%O@#&``@``D7"`,0`"``"*+X`R@`(``(H\@#(``@``IQP` M`(`"``"*58`R@`(``)%S@#.``@``D7N`,X`"``"*;8`R``(``(IR@#(``@`` MD7V`-(`"``"G'H`N@`(``)'8@"X``@``BMB`-0`"``"1E```@`(``(K<``"` M`@``D94``(`"``"*Y```@`(``)&=`````@``BPZ`-H`"``"+$H`V@`(``)&D M@#8``@``BT2`-@`"``"1JX`V``(``(MY@#8``@``D;D``(`"``"G(H`N@`(` M`)'M@"X``@``BY&`-P`"``"1N@``@`(``(N5``"``@``D;L````"``"+O8`X M@`(``(O!@#B``@``D?F`.(`"``"G)@``@`(``)'^@#B``@``B\N`.(`"``"1 MP8`X``(``)((@#@``@``#Z:``8`"``"G-```@`(```^G@`"``@``B_,``(`" M```/L(````(```^W@`X``@``+2>``8`"``!!Q(`!@`(``$'(@`&``@``0&`(0`"```/T(`!@`(``*<\``"``@``ET:`+@`"``!!XH`Z``(``$'C M@`&``@``1`"``8`"``"G0```@`(``$0W@`&``@``1.Z``8`"``!$[X`!@`(` M`))T``"``@``IT0``(`"``"2G@``@`(``$4X@`&``@``DK```(`"``!%3(`! M@`(``$63@`&``@``1:"``8`"``!&$(`!@`(``$83@`$``@``1G>``0`"```/ MX(`%@`(``*=/``"``@``EU2`+@`"``!&>(`Z``(``$9Y@`&``@``2?>``8`" M``!&>H`!@`(``*=3``"``@``1Q2``8`"``"2OP``@`(``$=5@`&``@``1UZ` M`8`"``"G5```@`(``$>0@`&``@``IU4``(`"``!'E8`!@`(``*=6``"``@`` M1Z&``8`"``"G5P``@`(``$>I@`&``@``IU@``(`"``!($(`!@`(``)+E``"` M`@``2&:``8`"``"G90``@`(``$AQ@`&``@``DP0``(`"``!(P(`!@`(``*=R M``"``@``2.R``8`"``"3(@``@`(``$DW@`&``@``IW8``(`"``!)BX`!``(` M``_O@`$``@``$`*``0`"``"O4`````(``*N'``"``@``JX@``(`"``"KB0`` M``(``*N-`````@``JXX````"``"KCP````(```$`@#L````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` 6`````````````````````````````'.` ` end 21-Mar-1996 09:32:57 -0800,2884;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Thu, 21 Mar 1996 09:32:57 -0800 (PST) Return-Path: Received: from mailer5.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA20740; Thu, 21 Mar 96 09:32:56 -0800 Received: from mx2.cac.washington.edu by mailer5.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA50792; Thu, 21 Mar 96 09:32:51 -0800 Received: from saul2.u.washington.edu by mx2.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA04216; Thu, 21 Mar 96 09:32:50 -0800 Received: by saul2.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA29943; Thu, 21 Mar 96 09:32:48 -0800 From: Jim Fox Message-Id: <9603211732.AA29943@saul2.u.washington.edu> X-Sender: fox@saul2.u.washington.edu Subject: HPSS, DFS, and Transarc moving along To: deroest@cac.washington.edu (Jim Deroest) Date: Thu, 21 Mar 1996 09:32:44 -0800 (PST) X-Mailer: ELM [version 2.4 PL25] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1675 Looks like the DFS business is indeed moving along. Jim Fox > Richard P Ruef says: > From ruef@anduin.ocf.llnl.gov Wed Mar 20 08:13:54 1996 > Date: Wed, 20 Mar 96 08:12:41 PST > From: ruef@anduin.ocf.llnl.gov (Richard P Ruef) > Message-Id: <9603201612.AA05488@anduin.ocf.llnl.gov> > To: fox@cac.washington.edu > > Jim, > It looks like the DFS/DMAPI stuff is a go. Transarc has said they > will add the enhancements need to support HPSS and the synced name spaces. > If you or one of your co-workers would like to attend the following is the > note I put out yesterday. I later found that you were not on the list, so > I am sending it to you. The meeting is just an informal get together of > the tech guys a and a discussion of how we believe the pieces will fit > together. Hopefully we will be able to meet all the Transarc workers attached > to the project. > > If somebody from your site plans to attend, please drop me a note. > > > =============================================================================== > > There will be a meeting at Transarc to discuss and start of the DMAPI work. > The meeting has been scheduled for April 10 and 11. Rooms are > scarse on those days so here is the information on where I am staying. The > plain ride is kind of expensive also. > > Ramada Hotel > 1 Bigelow Square > > April 9-11 > > 412-281-5800 > > The meeting will start around 9 on the 10th and will probably go for > most the the day both days. The main purpose of the meeting is to get to > know who is on the team and what exactly is to be done. Transarc is > interested in knowing exactly how we plan to use the DMAPI for DFS. > > > Rich > > 26-Mar-1996 09:03:38 -0800,7230;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Tue, 26 Mar 1996 09:03:38 -0800 (PST) Return-Path: Received: from mailer1.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA19496; Tue, 26 Mar 96 09:03:38 -0800 Received: from mx2.cac.washington.edu by mailer1.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA25092; Tue, 26 Mar 96 09:03:30 -0800 Received: from [192.94.47.33] by mx2.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA16758; Tue, 26 Mar 96 09:03:27 -0800 Received: from popsicle.llnl.gov by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA15575; Tue, 26 Mar 1996 11:02:21 -0600 Received: from quickmail.llnl.gov by popsicle.llnl.gov (8.6.10/LLNL-2.0) id TAA03193; Mon, 25 Mar 1996 19:46:27 -0800 Message-Id: Date: 25 Mar 1996 09:19:19 -0800 From: "Dick Watson" Subject: FWD>RE>Transarc S.O.W. dtd To: "HPSS Exec Reflector" X-Mailer: Mail*Link SMTP-QM 3.0.2 Mail*Link(r) SMTP FWD>RE>Transarc S.O.W. dtd 3/20/96 The following message from Chris is sent for your info. We will be working the 2 questions he raises with Transarc. Dick -------------------------------------- Date: 3/22/96 12:03 PM From: Chris_Maher@transarc.com Dick, We have reviewed the statement of work that you sent and I have a few changes to your text and some typo types of fixes. First, on the issue which is probably the trickiest, I want modify the language around the provision of the results of this work to third parties for inclusion in their products. Let's replace your sentence which begins "Upon completion..." at the end of the first paragraph of section 3.0 with "Upon completion of this work Transarc will encourage availability of the results of this work on additional platforms. Transarc will choose one or more of the following mechanisms for this. It will either offer to include the work in future DCE releases as part of the ongoing DCE PST mechanism with OSF; work to create a new PST including the work; offer to license the work under reasonable commercial terms to other DFS providers, or develop and port the results of the work to other platforms." By way of explaination of the above I should point out that the current PST agreements do not go beyond DCE 1.2.2, whose content has already been determined. If they are extended it is not clear what the collection of PST particpants will want to do or that if they wanted to do it that the terms would be reasonable to TA etc. There has also been talk of a separate PST for DFS in the future, which might make more sense. Also, it is possible that a better (and faster!!) way for this to become available on other platforms is for us to directly lisence this to other vendors (this in not unusual) or for us to do it ourselves on other platforms (also not unusual as we have a DFS client product for Windows NT.) Also, in this SOW or in other parts of the contract it should be clearly stated that DFS, the changes to DFS and other code developed by TA as part of this agreement are owned by TA. Other clarifications: First paragraph of section 2.0: Change sentence beginning with "In addition Transarc will integrate..." to "In addition Transarc will work with the HPSS team to integrate the HPSS DMAP with DFS. " Change section 3 paragraph 5 beginning "Transarc shall provide software to allow keeping the HPSS and DFS/Episode ... state in sync" to "Transarc shall provide additional events and interfaces to the DMAPI, as given below, to allow/enable HPSS-DMAP software to keep the data spaces, name spaces, and access control state in sync." Section 3.0 para 4: Add "The number of clients and servers provided for this purpose to the HPSS team shall be kept to the number to reasonably develop and test the work and will not exceed 10 clients and 5 servers, unless mutually agreed to by Transarc and the HPSS team" Is ``closed'' a MANDATORY event or simply optional? It's optional in the DMAPI spec. Also our eagle eyes have noticed the following: 1.0, para 5; The DMIG work had begun by 1993 and probably earlier; there was a ``Data Management API: Architecture Overview'' document, version 2.0 of which existed in March 1993. How about replacing ``in 1994'' with ``by 1993''? 1.0, para 6: ``migration of data between local DFS storage and''; since this talks about DMAPI in general, ``DFS'' should be replaced with either ``filesystem'' or ``disk''. 1.0 para 6: Changes "The interface is to remain..." ``A goal of DMIG is for the interface is to remain stable across operating system and filesystem releases'' 1.0 para 7. Transarc will ``implement,'' not ``develop,'' the DMAPI for DFS. HPSS will interface on one side to ``the Transarc DMAPI implementation,'' not ``the Transarc DMAPI.'' 2.0 para 1. The work specifically involves ``implementing the DMAPI in Episode'' or ``providing a DMAPI implementation for Episode,'' not ``providing the DMAPI in Episode.' 2.4 para 2. I believe the word ``now'' in the first sentence should be dropped. The intent is that the ftserver should be able to handle filesets that contain migrated files--once the work is done, not ``now.'' 2.6 para 1. Transarc will ``adopt'' the HPSS specific hierarchical storage managment requirements, not ``integrate.'' Also, ``security synchronization'' should probably be ``access control synchronization.'' 3.0 para 6: ``#021##016#ward'' I think should be ``backward.'' The text after the dm_respond_event description should be deleted. Rich R. agreed that it would be unnecessary if there were a syncpostcreate event. ------------------ RFC822 Header Follows ------------------ Received: by quickmail.llnl.gov with SMTP;22 Mar 1996 11:57:17 -0800 Received: by transarc.com (5.54/3.15) id for Dick.Watson@quickmail.llnl.gov; Fri, 22 Mar 96 14:51:05 EST Received: via switchmail; Fri, 22 Mar 1996 14:51:05 -0500 (EST) Received: from unix2 via qmail ID ; Fri, 22 Mar 1996 14:50:04 -0500 (EST) Received: from unix2 via qmail ID ; Fri, 22 Mar 1996 14:49:59 -0500 (EST) Received: from VUI.Andrew.3.70.CUILIB.3.45.SNAP.NOT.LINKED.unix2.sun4.40 via MS.5.6.unix2.sun4_40; Fri, 22 Mar 1996 14:49:58 -0500 (EST) Message-Id: Date: Fri, 22 Mar 1996 14:49:58 -0500 (EST) From: Chris_Maher@transarc.com To: "Dick Watson" Subject: Re: Transarc S.O.W. dtd 3/20/96 Cc: M._C._Srivas@transarc.com, Craig_Everhart@transarc.com, Vikram_Biyani@transarc.com, Peter_Oleinick@transarc.com, Brian_Herhusky@transarc.com, Carolyn_G._Dietrich@transarc.com In-Reply-To: References: 26-Mar-1996 09:04:05 -0800,10875;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Tue, 26 Mar 1996 09:04:05 -0800 (PST) Return-Path: Received: from mailer12.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA19508; Tue, 26 Mar 96 09:04:04 -0800 Received: from mx2.cac.washington.edu by mailer12.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA61250; Tue, 26 Mar 96 09:04:03 -0800 Received: from [192.94.47.33] by mx2.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA16779; Tue, 26 Mar 96 09:04:00 -0800 Received: from popsicle.llnl.gov by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA15578; Tue, 26 Mar 1996 11:03:03 -0600 Received: from quickmail.llnl.gov by popsicle.llnl.gov (8.6.10/LLNL-2.0) id TAA03166; Mon, 25 Mar 1996 19:44:25 -0800 Message-Id: Date: 25 Mar 1996 15:00:11 -0800 From: "Dick Watson" Subject: Collaboration Agreement (M To: "HPSS Exec Reflector" X-Mailer: Mail*Link SMTP-QM 3.0.2 Subject: Time:2:55 PM OFFICE MEMO Collaboration Agreement (MOU) Date:3/25/96 Enclosed below is the revised Collaboration Agreement (our legal folks want to call it a MOU). The main change is the deletion of the old section 6.2 which was removed because they felt it was covered by other documents and licenses. This has been through LLNL and LANL legal and gotten their blessing. What we need is yah or any suggestion from your legal eagles for changes. Dick -------------- MEMORANDUM OF UNDERSTANDING (COLLABORATION AGREEMENT) This Agreement is made this ______day of _____ , 1996, between the International Business Machines Corporation ("IBM"), a New York corporation, acting through its Government Systems Division located at 3700 Bay Area Boulevard, Houston, Texas, 77058, and certain Development and Deployment Members as defined below. 1.0 Purpose The High Performance Storage System ("HPSS") was originally developed jointly by IBM and the Development Members under various Cooperative Research and Development Agreements ("CRADAs"), which either have or soon will expire. HPSS is critical to the future strategic direction of the Development Members, provides commercial value to IBM, and provides functional utility to the Deployment Members. Therefore, the parties are entering into this Collaboration Agreement to oversee the ongoing development of HPSS to assure that (i) HPSS becomes and remains a reliable and useful product; and (ii) one common sour e code exists for HPSS. The parties anticipate that the purpose of this Agreement can be achieved within roughly two years. 2.0 Definitions 2.1. "Deployment Member" means those who help expand the HPSS installation base, currently including Cornell Theory Center, and the University of New Mexico. 2.2 "Development Member" means those who helped develop HPSS, currently including IBM, the four Founding DOE Laboratories, and the National Aeronautics and Space Administration, acting through its Langley Facility ("NASA Langley"). 2.3 "Founding DOE Laboratories" means the following Laboratories of the United States Department of Energy: Lawrence Livermore National Laboratory, operated by the Regents of the University of California ("LLNL"); Los Alamos National Laboratory, operated by the Regents of the University of California ("LANL"); and Sandia National Laboratory, operated by the Sandia Corporation ("Sandia"); and Oakridge National Laboratory, operated by Lockheed Martin Energy Systems, Inc. ("Oakridge"). 2.4 "HPSS" means the High Performance Storage System software originally developed under various CRADAs between IBM and the DOE Founding Laboratories. HPSS is a scalable, distributed, high speed storage management system based on version five of the IEEE Mass Storage System Reference Model, which will be capable of handling a multiple pedabyte archive with 500 MByte/second parallel I/O throughout. For purposes of this Agreement, HPSS includes any enhancements to the System made during the course of this Agreement by IBM, Development Members or Deployment Members. 3.0 Collaboration Membership 3.1 There shall be two classes of membership: Development Members, and Deployment Members. 3.2 Current Members are defined in Section 2.0. Other Members may be admitted by unanimous vote of the Executive Committee if they agree to abide by the provisions of this Agreement. 4.0 Expenses Except as may be agreed to in bilateral agreements between Members, each Member shall bear its own expenses. 5.0 Confidential Information To the extent the parties exchange confidential information under this Agreement, they shall do in accordance with a mutually agreeable nondisclosure agreement. 6.0 Rights in Technical Data 6.1 Rights in technical data between the Members are set out in their respective CRADAs or other documents executed between individual Members. Therefore, except as provided below, this Agreement conveys no rights in such works. 6.2 Notwithstanding the foregoing, IBM and the Founding DOE Laboratories and the deployment members hereby agree that they share joint title without accounting in the modules of HPSS listed below: [INSERT HERE A LIST OF HPSS MODULES] 7.0 Limitation of Liability and Warranty Exclusion 7.1 IBM, THE DEVELOPMENT MEMBERS, AND THE DEPLOYMENT MEMBERS MAKE NO WARRANTIES EITHER EXPRESSED OR IMPLIED, AS TO THE CONDITIONS OF THE RESEARCH OR ANY INTELLECTUAL PROPERTY OR PRODUCTS MADE IN CONNECTION WITH THIS AGREEMENT, OR THE OWNERSHIP, MERCHANTIBILITY OR FITNESS FOR A PARTICULAR PURPOSE OF THE RESEARCH OR RESULTING PRODUCT. NO PARTY SHALL BE LIABLE FOR SPECIAL, CONSEQUENTIAL OR INCIDENTAL DAMAGES. IN NO EVENT, MOREOVER, SHALL ANY MEMBER BE LIABLE FOR ACTUAL DAMAGES IN EXCESS OF $10,000. 7.2 Section 7.1 applies only to this Agreement and does not alter any warranties made or liabilities assumed in any individual agreements between the parties. 8.0 Collaboration Management The Collaboration shall be managed by an Executive Committee and by a Technical Committee, which shall report to the Executive Committee, as described below: 8.1 Executive Committee 8.1.1 The Executive Committee shall consist of one representative from each Member. 8.1 a The Committee shall be jointly chaired by the IBM and LLNL representatives. 8.1.3 The Committee shall convene periodically on an as needed basis, but not less often than annually. 8.1.4 The Committee is responsible for commercialization and deployment decisions; HPSS design approval, function and content; budgetary issues; HPSS promotion and publicity; HPSS proposal support; final project decisions, and the addition of new members. 8.1.5 The Committee may delegate the management of particular projects as it deems appropriate. 8.1.5 The Committee may delegate the management of particular projects as it deems appropriate. 8.1.6 Notwithstanding the Committee's oversight function, IBM shall have sole responsibility to (i) enter into agreements for the licensing of HPSS to customers, and (ii) negotiate the terms and conditions of those agreements. A copy of the software licensing terms IBM intends to use in those agreements appears as Appendix I. 8.2 Technical Committee 8.2.1 The Technical Committee shall consist of one representative from each Development Member, who must be participant of that Member's development team. Development Members may send more than one person to Technical Committee meetings, however, only one person will function as that Member's official representative at these meetings. 8.2.2 The Committee shall convene on an as needed basis, but not less often than once a month. 8.2.3 The Committee shall address matters such as the following: resource/personnel allocation; project scheduling; requirements generation and analysis; defining design, code, test, documentation and other standards for the project; and other technical matters. 9.0 Disputes The parties shall attempt to resolve all disputes arising under this Agreement. If the parties are unable to resolve their disputes within a reasonable time, the dispute shall be submitted to the Executive Committee for resolution. That Committee shall resolve the dispute by majority vote. If any Member is not satisfied with that resolution, it may pursue its dispute in a court of competent jurisdiction. 10.0 Termination 10.1 Unless otherwise determined by the Executive Committee, this Agreement shall terminate 24 months after its effective date. 10.2 This Agreement may be terminated at any time by the agreement of all the then current Members. 10.3 Any Member may resign with 30 days advance written notice to the Executive Committee. 11.0 Survival The following provisions shall survive the termination of this Agreement: Section 5.0, Confidential Information; Section 6.0, Rights in Technical Data; and Section 7.0, Limitation of Liability and Warranty Exclusion. 12.0 Independent Contractor Each Member is an independent contractor under this Agreement. No Member has authority to bind the other Members under this Agreement. 13.0 Independent Pricing Notwithstanding any provision in this Agreement to the contrary, IBM shall have unfettered discretion to set the price at which it may license HPSS to customers. 14.0 Assignment No Member may assign its rights under this Agreement to a third party without the unanimous agreement of the Development Partners. 15.0 Use of Names No Member may use the name of another Member without the prior written consent of that Members. 16.0 Entire Agreement This Collaboration Agreement and its Appendices constitute the entire agreement of the parties and supersede any prior oral or written communications between the parties with respect to this agreement as set out in section 1.0. IN WITNESS WHEREOF, the parties have executed this Agreement as of the date first noted above: INTERNATIONAL BUSINESS MACHINES CORPORATION BY: NAME: TITLE: DATE: THE REGENTS OF THE UNIVERSITY OF CALIFORNIA (LLNL) BY: NAME: TITLE: DATE: THE REGENTS OF THE UNIVERSITY OF CALIFORNIA (LANL) BY: NAME: TITLE: DATE: SANDIA CORPORATION, SANDIA NATIONAL LABORATORY BY: NAME: TITLE: DATE: LOCKHEED MARTIN ENERGY SYSTEMS, INC. (OAK RIDGE) BY: NAME: TITLE: DATE: [INSERT NAMES OF OTHER SIGNATORIES] 4-Apr-1996 09:18:48 -0800,4951;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Thu, 4 Apr 1996 09:18:48 -0800 (PST) Return-Path: Received: from mailer13.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA13352; Thu, 4 Apr 96 09:18:48 -0800 Received: from mx1.cac.washington.edu by mailer13.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA61706; Thu, 4 Apr 96 09:18:47 -0800 Received: from [192.188.31.145] by mx1.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA22586; Thu, 4 Apr 96 09:18:41 -0800 Received: from stormy.clearlake.ibm.com by denali2.nsl.nersc.gov (AIX 3.2/UCB 5.64/4.03) id AA23744; Thu, 4 Apr 1996 09:17:32 -0800 Received: from rosebud.clearlake.ibm.com by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA23404; Thu, 4 Apr 1996 11:24:07 -0600 Received: by hpss.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA28513; Thu, 4 Apr 1996 11:25:22 -0600 Date: Thu, 4 Apr 1996 11:25:22 -0600 From: chang@hpss.clearlake.ibm.com (Paul Chang) Message-Id: <9604041725.AA28513@hpss.clearlake.ibm.com> To: hpss@nsl.nersc.gov Subject: R3 IT Status (Installation) R3 Integration Test Status - 04/05/96 Installation 1. Loose Ends With Target Date for Closure Incorporate "Add ftp users" and "Add ssm users" capabilities into Initial Configuration (mkhpss). Admin Guide draft and tape image available on 03/13 for site testing Update Admin Guide chapter 3 & 4, appendix D, E, F, G, & H | Reshuffer ./tools contents; move/rename initial config scripts to ./config 2. Open PTRS None 3. Test Plan - Test cases will be run in Houston in a separate DCE cell using SP2 node5 and node6 (w/ external 8mm tape). - Checkout existing R1 capabilities. - R3 new capabilities (install via tar & multiple HPSS config. support) testing will be completed by end of March. - Have other sites install HPSS using generated tape (image). - Testing suspended on 09/01/95; - Testing resumed on 02/01/96. 4. Test Scripts Update existing test functions to include multiple HPSS configuration support. Completion date: 9/15/95 5. Test Progress - Test environment has been set up on sp2n05 - Updating existing test functions - Execution of test functions is underway. Category Total #Passed #Failed #Pending Functional 6 6 0 0 Error 1 1 0 0 Security N/A Sizing Timing N/A System Mgmt N/A Reliability N/A Note: Need to rerun 01. HPSS Installation Tape Generation and 02. HPSS SMIT Installation. 6. Subsystem Dependency Issues Update installation pakage *.list files 7. Other Problem Areas None 8. HPSS work tasks other than R3 testing CM Maintenance - 5% 9. Schedule Write Test Cases - 08/01/95 - 09/15/95 Write Test Code - N/A Execute Test Cases - 08/01/95 - 09/01/95 Resume Testing - 02/01/96 - 03/01/96 Testing Completed - 03/01/96 | Re-test - 03/11/96 - 05/31/96 10. Test Rerun Category Total #Passed #Failed #Pending Functional 6 6 0 0 Error 1 1 0 0 Rerun all test function by 03/28/96; all successfully completed. Will rerun all test function and generate a new tape image on 04/15. 4-Apr-1996 15:44:16 -0800,33860;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Thu, 4 Apr 1996 15:44:16 -0800 (PST) Return-Path: Received: from mailer14.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA13466; Thu, 4 Apr 96 15:44:15 -0800 Received: from mx1.cac.washington.edu by mailer14.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA53293; Thu, 4 Apr 96 15:44:14 -0800 Received: from [192.188.31.145] by mx1.cac.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA04098; Thu, 4 Apr 96 15:44:11 -0800 Received: from vnet.ibm.com by denali2.nsl.nersc.gov (AIX 3.2/UCB 5.64/4.03) id AA31366; Thu, 4 Apr 1996 15:42:09 -0800 Message-Id: <9604042342.AA31366@denali2.nsl.nersc.gov> Received: from BETVMIC1 by VNET.IBM.COM (IBM VM SMTP V2R3) with BSMTP id 7762; Thu, 04 Apr 96 18:41:43 EST Date: Thu, 4 Apr 96 18:42:13 EST From: teaff@VNET.IBM.COM To: hpss-rqmt@nsl.nersc.gov, lofton@mhpcc.edu, hpss-exec@nsl.nersc.gov Subject: HPSS Requirements Telecon HPSS REQUIREMENTS ASSESSMENT - 4/4/96 Initial source line costs to implement the proposed R3+ and R4 requirements was provided and discussed in today's requirements telecon. These cost inputs are listed below. In addition, 3 new requirements were submitted and are listed at the end of this note. No prioritization of these requirements has been performed yet. The next HPSS Requirements telecon will be a week from Tuesday, 4/16 at 2:00 central. The password for the next telecon is hpss. I will begin looking at an implementation phasing schedule and determine if another round of requirements prioritization is warranted. ------------------------------------------------------------------------ The requirement candidates were gathered from the following sources: o Previous Release 4 Requirements (voted on in 11/95). o R3+ / R4 requirements donated from subsystems. o Maintainability requirements from demonstations in Houston o New requirements found in Exec Committee site deployment plans. The total set of new candidate requirements is provided below. Duplicates from the lists, requirements already implemented, non-software items, and requirements with consensus to delete were removed from this list. list. The following definitions are being applied: o R3+ - limited to address system deficiencies vs. new capabilities. o R4 - includes new capabilities which are deemed mandatory for a site to be in production. Note: Within a release, there is no implied priority of requirements. ============================================================================== RELEASE 3+ AREA AI COST DESCRIPTION ---- -- ----- ---------------------------------------------------------------- UTM CH 400 1. Support 1.8 Name Server (B3) SS DF 200 2. Atomic mounts (tape to tape) BFS DC 400 Get mounts of two or more VVs that are part of single IO operations, or part of split IO operations to mount together to avoid deadlocks. (B6) SS DF 50 3. Make changes to the segment create logic so that the tape SS can control the total number of "open" storage segments on a storage class basis. This allows the BFS to simply fire away with all the tape storage segment creates it wants without having to implement some sort of hokey throttling scheme. It places the responsibility of controlling the number of tapes that are actively written on the tape storage server where it belongs. (B8) PVR JD 200 4. The 3494 PVL shall implement a method to obtain additional information on IBM3494 return codes Currently the LAN/TTY driver gives a MTCC_IO_FAILED for every return code. A fairly easy solution would be to save the request id, then issue an ioctl to query the request upon MTCC_IO_FAILED. (B17) BFS DC 200 5. Need to have the bitfile server cleanup disk segments that MPS LC 0-del are allocated but not written to on a failed disk write. Currently this is taken care of by purge MPS. It may end up becoming mandatory when we put in migration/purge performance enhancements. (B18) Difficulty: Medium BFS DC 300 6. Keep track of failed storage server end session calls to the storage server and attempt to clean them up in the background. (B20) Difficulty: Medium BFS DC 400 7. Migration/Purge performance enhancements. This is basically MPS LC 50 running MPS off migration/purge records instead of bitfile SSM RB 50 segment metadata. (B21) Difficulty: Easy for MPS: Moderately difficult for BFS: BFS DC 250 8. Detect small files and redirect to an appropriate COS. SSM RB 50 Smarter use of COS in NFS. (B22) NFS RH 50 Difficulty: Medium FTP RR 50 SS DF 100 9. Need enchancements to better control the allocation of storage resources(particulary tape volumes). Creation of segments on VVs needs to be throttled so that the number of create resources honored at any give time does not exceed greatly the number of drives available in the HPSS system for this type of volume. (B27) Difficulty: Difficult BFS DC b)100?10. Need to enhance algorithms in the HPSS system to better BFS? 300 minimize the amount of metadata created. In particular, the following is an example of the primary problem. (B29) a). A file is written to disk in a storage class with a storage segment size of 256k. b). The file is migrated to tape with a VV blocksize of 1MB c). The IOD that goes to the storage server will consist of multiple srcsink descriptors each indicating to write 256k into a single storage segment on tape. The descriptors are sequential within the storage segment. For example, the descriptors to the SS are: offset length ssegid ssegoffset 0 256 A 0 256 256 A 256 512 256 A 512 d) The request will be funneled though the storage server and the mover. e) The first 256k will be written and it will then be noticed that a short block has been written. The write terminates with this indication. f) The storage segment (A) is marked by the storage server as SEG_LENGTH_FIXED and can no longer be written to. e) This is returned to the bitfile server. The bitfile server will create a single bitfile segment to represent this 256k piece. Basically steps c-e will be repeated until the entire file is migrated. The end result in this case is that you end up with 3 bitfile segments and 3 storage segments instead of 1 each as it should really should work. This could be fixed in potentially a number of places. It will probably not be that easy anywhere. My guess is that the hardest place is the bitfile server in that chunks that go sequential in the storage server IOD may actually represent discontiguous pieces of the bitfile and the whole algorithm in bfs uses the return information in the storage server IOR to try and figure out how to build new bitfile segments. The other potential places are the move and the storage server. Difficulty: Diffcult b. Provide better allocation of disk resources. (79) -> Add capability to group storage segments creates into a single request API KE 200 11. The ability of HPSS to handle requests is limited by the BFS DC 20 number of threads plus the size of the endpoint queue. In addition in BFS, only so many long running requests are BFS, only so many long running requests are allowed to be allowed to be active at a given time as to allow short running requests a shot. This in effect means that the number of active IO requests plus the ones waiting can never exceed the number of threads you are running in BFS. We should add code to some of the client API calls to enter some type of retry loop if the request from the bitfile server is terminated with HPSS_EBUSY. (B30) Difficutly: Medium? BFS DC 400 12. Set operational state correctly in BFS (B33) Difficulty: Easy but tedious SUD HJ ??? 13. Server auto restart. (B51) SSM RB 200 Add a new bit for AutoRestart to the Flags field of the server config file. If this flag is set, SSM would try to restart the server any time it goes down, unless SSM is the one who shut it down. The operator would set the AutoRestart bit individually for each server from the server config screen. It would also be nice to have the default metadate library routine set this to "TRUE" Cost: ds: add a new button to the server config screen and support it; about half an hour sm: observe the AutoRestart flag, keep track of whether ssm had shut down a server, attempt the autorestart as needed; about 3 days with restart options mm lib: (optional) set the default AutoRestart value Other: AIXT HJ --- 14. Upgrade to next DCE, Encina (changes to connection management, security and startup daemon) (B55) SSM DS 500 15. Convert to Sammi V4 (B55) BFS DC 50 16. Ability to create stripes up to 256 wide. (B69) (client side) MPS LC 300 17. MPS shall set the operational state correctly. (B77) ============================================================================== RELEASE 4 AREA AI COST DESCRIPTION ---- -- ----- ---------------------------------------------------------------- NS 2575 1. Enhanced scalability (A2) BFS 1020 SS 100 SSM 400 MPS 400 API 2100 INST 200 SSM 1000 2. Error message categorization / consistency (680 SLOCs) All Note: The following was also included under B below: - More meaningful messages, messages coming at proper frequency (not too often and not to seldom), message at proper severity levels. (A3) All RB 750 3. Statistics - additional real-time and historical monitoring (A4) - monitor file system space where SFS archives are stored and report % space used (with warning and alarm thresholds) - may be done - display single window showing summary of what is in the HPSS system (basically a record count of all significant metadata files) - elaborate - provide error status (e.g. # of errors) on volumes - for name server, bitfile server, storage servers... some statistic(s) that tells the system administrator when to add an additional metadata file or server - needed for scalability - elaborate - number of bytes written and read through PFTP/FTP NFS, PFS, direct client API calls - needed to determine how system is being accesses; what client interfaces need to be tuned - revisit The following might already be done: - number of bitfiles - needed for capacity planning; usage reports - number of directories, symlinks, hard links - ditto - total bytes stored on each hierarchy and each level of the hierarchy - needed for capacity planning; hierarchy management - total bytes available (may be estimated) on each hierarchy - needed for capacity planning and hierarchy management - number of tape mounts per drive - needed to determine drive service requirements - number of mounts per virtual volume - needed to determine tape volume usages; needed to determine possible volume servicing requirements - number of errors per physical drive - needed to identify drives needing service DMAP RR 11500 4. DFS support - DMAP/DMIG (A6) SSM RB 700 BFS DC 500 5. File families and storage control (appendable volumes) (A7) SS DF 400 API KE 50 SSM RB 150 MVR KE 250 6. Generic Mover: Mover device switch (A8) API KE 150 7. MPI-IO support (A9) BFS DC 200 MVR KE 1750? 8. Non-DCE support (A10) GAPI KE 2000? GWAY KE 2500? SSM RB 900? PVL MG 500 9. The PVL shall be transactional for move, allocate, and deallocate. (B13) The PVL currently does not use transactions (because we thought we would be purchasing a COTS PVL). The inability to link PVL actions with HPSS transaction has been the source of a lot of code and PTRs in the past. PVL MG 100 10. Support multiple media types per drive (B15) SSM RB 50 UTIL DC 500 11. Provide a utility to change all the files in one COS to another. (B24) Difficulty: Moderately easy - Allow more flexibility in changing Class of Service (COS) definitions. ?? Should this be moved to R3+. BFS DC 50 12. Change the bfs_Open call to always pass back a copy of the API KE 100 COS parameters for the bitfile. Have the client API always FTP KE 100 provide this to its callers in hpss_Open. This would be useful so that things such as a OPTIMUM_ACCESS_SIZE could be provided to applications using HPSS. They could then select an appropriate buffer size instead of hardcoding it. Using a bad buffer size can have very negative effects on HPSS (especially in the amount of metadata created when writing files). (B25) Difficutly: Medium MPS LC ??? 13. Enhanced multiple copies. Currently, we can only do copies BFS DC 200 for hierarchies with disk. We need to do something for tape. I have some ideas on this which were my original ideas for file copy in the long term and they build upon the copy file capability we now have in the R3 bitfile server. (B32) Difficulty: Moderate SSM RB 200 14. Update device / drive config data (server shutdown allowed) (B34b) SEC RH 200 15. Integrate the security policy manager as an HPSS server SSM RB 300 (B54) PVR KE 500 16. Provide the capability for control over assignment SSM RB 100 of removable media to devices/drives (affinity) (B65) Currently we have not control over where a cartridge is mounted within a configuration that provides several crives in which the cartridge may be loaded (e.g., in a single 3494 with 8 drives, each cartridge may be mounted in any of the 8 drives - assuming the drives are all of the same type). This may have some rather serious performance consequences in a couple of different scenarios. Scenario 1: A hierarchy is defined that includes a 4-way striped disk storage class, which is spread across 4 nodes, above a 4-way tape storage class. There are 8 tape drives, 1 on each of the nodes that make up the disk stripe and 1 each on 4 other nodes. A potentially ideal situation would be for the tapes to be mounted on the drives attached to the nodes also supporting the disks (and depending on striping factors, even in the appropriate order) - if not, there will be an impact on migration/staging performance and overall system load, due to the increased networking traffic. Scenario 2: One hierarcy consists of an HIPPI attached IPI-3 disk array above 3590 tape. There are 8 3590 drives, but only 2 are connected to an HIPPI attached mover node. If the tapes for this hierarchy are mounted in the two drives that are attached to the HIPPI node, then performances of migration/staging will be relatively good. However if the tapes are mounted in other drives, performance will likely be relatively awful, since the data will be forwarded. through a HIPPI node and over some other, potentially much slower, network. Today we would need to either rely on a cartridge's affinity to certain drives (the 3494 will load a cartridge in the closest available drive) or utilize separate PVR's for each set of drives (which has the potentially undesirable side effect of only allowing tapes to be mounted on the drives within that PVR, so that if all the drives within that PVR are busy, the mount will be queued). ?? Should R4+ #33 be moved to R4. API KE 200 17. Implement interface to a Gatekeeper Policy Module in GATE MG 100 the Client API. (B68) ACCT BP ?? 18. Provide account validation (B71) SSM RB 50? MPS LC 200 19. Provide more flexibility in the migration policy so that SSM RB 100 user could specify to start the migration at a specific time. In addition, user shall be able to specify to migrate/purge files either by file size or by age. (B73) MPS LC 100 20. Migrate/Purge the files by file size. This item is BFS DC 0 related to the BFS's migration/purge performance SSM RB 50 enhancements. ============================================================================== RELEASE 4/P AREA AI COST DESCRIPTION ---- -- ----- ---------------------------------------------------------------- NS JM 850 21. Undelete (A12) UTIL JM 2000 API KE 400 FTP RR 100 SSM RB 300 MVR KE 1000?22. Incorporate CFS Mover mods (developed outside the HPSS SS DF 30? development team) into the official Mover. Metadata conversion utilities to be developed outside the HPSS development team. (B5) PVL MG 300 23. Need to enhance system to minimize tape mounts more than we SS DF 75? now do. In particular, if a volume is mounted on a drive in a SS and a request is not received in a given time, the volume is automatically dismounted. This volume should stay mounted until some other request in the system actually needs the drive. Would probably require changing the PVL as well as the storage server. storage server. (B28) Difficulty: Difficult All RB 200 24. Better way to get detailed problem info from servers. Could mean just doing alarm/event mechanism; could mean reworking server managed objects to have more detail; could or something else. Now it is difficult to figure out what's wrong when a server just says his OpState is bad. (B49) MVR DW 1000 25. Incorporate changes to allow true tape Network Attached SSM RB 150? Peripherals (tape NAPS). (B80) MPS LC 300?26. Allow MPS to run based upon current tape drive PVL MG 100 utilization. Need a mechanism (may already exist via PVL). SSM RB 50 to determine the overall busy state of HPSS tape drives and have migration run based on idle resources. (B81) ============================================================================== POST RELEASE 4 AREA AI COST DESCRIPTION ---- -- ----- ---------------------------------------------------------------- UTIL 2950 1. Metadata consistency checking (A1) ??? 5920 2. Request processing (A5) a) Provide information from Client API about end users b) Find/influence internal request status MVR 1010 3. Tape formats (ANSI and IBM labeled) (A11) BFS 300 - Provide external tape support (i.e. media export) (FNAL) SS 150 API 40 IMPU 2000 EXPU 1000 SSM 300 SSM 950 4. Configuration sanity checks (A13) - Refresh MO screens after an AttrSet call fails, to 100 replace any modified fields with the previous values. API 300 5. Sensitivity labels (A14) FTP 150 100 BFS 100 NS 100 SSM 300 NFS 2350 6. Interfaces - NFS V3 (A15) SSM 350 PE 4500 7. Parity checking (A16) SS 1500 SSM 400 VFS 13900 8. Interfaces - VFS (AIX only) (A17) SSM 350 MVR 9. Exploit switch protocols for Mover transfers on the SPx (B1) API 10. Add logic to Client API to pick transfer protocol (ipi-3 or sockets) without requiring an env. variable (B2) SS 11. Provide some sort of simple "what are you waiting on" and "stop waiting on that" feature for the SS, particularly on the tape SS. This might be a simple command line tool to interrogate the tape SS about threads that are waiting and some sort of simple scheme to break the waiting threads loose, if that's practical. (predecessor to request handling) (B9) PVL 12. The PVL shall provide an audit capability. The PVL would return a list of which volumes it manages along with each volume's respective PVR. Options would allow clients to specify a range of cartridges for audit (B10) PVL 13. The PVL shall provide an API that returns the list of drives that it manages. (B12) PVL 14. Quiesce (B14) PVR: 15. The PVR shall provide audit capabilities. (B16) Two levels of capability shall be provided: Level 1) Return location information from PVR metadata for a provided range of cartridges Level 2) Return location information after performing robotic verification for a provided range of cartridges BFS 100 16. Allow automatic redirection of a file in one COS to another. SSM (B23) Difficulty: Moderately easy BFS 200 17. Recache COS,Hierarchy, and SClass information on reinit in BFS. (B26) Difficulty: Medium BFS 18. Need a utilty to attempt to defrag a bitfile on tape that UTIL 0 has become heavily fragmented. May also need to put logic in become heavily fragments. May also need to put logic in bfs BFS to detect when a bitfile has become heavily fragmented and have automatically scheduled for defragmentation. (B31) Difficutly: Medium SSM 19. Implement reinitialization after reinitialization of any metadata (B34a) SSM 100 20. When deleting ServerConfig, first delete Server Log Policy and Specific Server Config (or remind user to do so) (B37) SSM 200 21. Upgrade how resource deletes are done (B38) SSM 70 22. Display which job is using a drive (additional info from PVL PVL) if/when PVL supports it (B39) SSM 50 23. Add "threshold status" column to hpss_StorageClassList (B40) SSM 50 24. Be able to find the PVR for a cartridge or the SS for a PV even if the PVL is down. (B41) SSM 100 25. Figure out better way to set up default Delog specs, especially for input file path (B42) SSM 50 26. Throttle screen updates caused by PVL Queue data change notifications (B43) SSM 2wk 27. A way to to examine all metadata entries, not just ones we care about. For instance, a way to read all bfs-specific config entries, not just the ones for the bfs's we've defined in our generic file. (B45) SSM 4wk 28. Better alarm/event mechanism, possibly our own. Better filering, both for the alarm/event screen & for delogging. (B47) SSM 29. Convert SSM to Motif (B60) SSM 30 Coalesce drive/device metadata files (B61) SSM 31. Name virtual volumes (B62) UTIL 32. GUI front-end to metadata editor (B63) PVL KE 500 33. Provide the capability for control over assignment of SS KE 500 removable media to devices/drives (at mount time). (B65) SSM 34. Consistent alarms/events across all SSM login sessions Currently, each SSM user will only get to see the alarms and events which are received by SSM when his session is logged on. He will not get alarms and events received when his session is down. This poses an inconsistency problem in which a logged on SSM user may not see the alarms and events other users say. The Sammi Alarm Redundancy feature could be utilized to forware any received alarm or even to all configured users so that all SSM users will see the same alarms and events regardless of when they logged on. (B67) MPS 35. SSM operator can force a Migrate/Purge for a particular SSM disk unit (B75) MPS 36. MPS shall support the re-initialize function. (B76) UTIL 37. Need utilities to checkpoint configuration metadata and identify differences between checkpoints for later cross checks. The goal is to get a snapshot of the configuration metadata when the system was considered operational so that when problems arise later we can find out if there were any changes in the system configuration. (C1) - Flesh out backup procedures. SSM 38. Allow changes to the Mover Device and PVL Drive config PVL without having to shutdown the PVL and all Movers. (C3) MVR - Allow online configuration of device/drive Support pvl_CreateDrive, pvl_DeleteDrive (SSM) - The PVL shall allow online modification of drive configuration metadata. While the PVL currently has an API allowing online changes to metadata, this API is not currently used. Integrate with SSM. UTIL 39. DCE control parameters in a server's general config (Maximum Connections and Thread Pool Size) - Need to ensure that they do not exceed what a server can handle. (C4) ??? 40. Need to address the issue of HPSS Imports, i.e. importing one HPSS system into another. (C5) UTIL 41. Need utility to inventory disk/tape volumes by certain characteristics (i.e. half full, EOM, etc.) (C6) UTIL 42. Need a way to automatically start up all configurated to execute HPSS servers after a system restart without an operator's intervention. (C9) UTIL 43. Need a means to verify that DCE and Encina are configured properly prior to configuring HPSS. (C10) ??? 44. Provide quotas by user and group (FNAL) (D2) UTIL 45. Provide command line administrative interface (FNAL, NERSC) (D3) REPK 46. Do not require Repack to use two times as many drives as widths (X/P, B78) ============================================================================== New Requirements Submissions SSM revisit 1. Correct potential for data notifications to arrive out of All order due to multithreading. FTP LANL to 2. FTP Quote Command Enhancements do; integ a. Since "dir fname" returns the entire contents of later? the directory, and "ls -options" does not allow filenames to be specified, and enhancement is needed to allow "stat'ing" an hpss directory without being sent the entire contents of the directory. Add a stat (or query?) command to avoid entire directory contents and to provide for listing NS text field and ACL info. b. Add an "access acc filename" command to provide improved messages c. Add "chgtext txt filename" command to allow setting of NS text field. d. Add a "chgacct acct filename" command to allow changing the account of a file e. Add a command to display an account id table, or at least to convert between an account id and an account name. Also, mvoe toward a situation in which the user is never presented with an account index, but rather with an account name that the user can relate to. The same principle applies to cos values as well. f. Add commands to add, delete, and modify acl's, as appropriate. revisit 3. Automatic Propagation of Account Index Modify appropriate software so that a site-selectable mechanism is available that causes the account index of a newly created file or directory to default to the account index of the parent directory. Background: UNIX-style accounting makes it difficult to associate portions of a user's data with more than one funding source, while site-style accounting puts the burden on the user to make sure that the account associated with each file is correct, regardless of where in the file system a user is creating files. Since files grouped in the same directory and/or subtree often have a strong tendency to be related in a funding sense, defaulting the accounting index of a file to that of the parent directory provides an automatic mechanism that is virtually error-free. A "chgacct" type of command would still allow the user to explicitly set the account index of an individual file, and setting the account index immediately after a "mkdir" request would address the case in which a subtree should have its own accounting index. 16-Apr-1996 06:45:38 -0700,13412;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Tue, 16 Apr 1996 06:45:38 -0700 (PDT) Return-Path: Received: from mx4.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA17024; Tue, 16 Apr 96 06:45:37 -0700 Received: from red6.cac.washington.edu by mx4.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA01181; Tue, 16 Apr 96 06:45:37 -0700 Received: by red6.cac.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA03952; Tue, 16 Apr 96 06:45:36 -0700 Date: Tue, 16 Apr 1996 06:45:35 -0700 (PDT) From: Sandy Moy To: UCS Directors -- Jim DeRoest , Sid McHarg , Thomas Profit Subject: NCSA (fwd) Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII THought you might be interested in where NCSA is planning on going with their computing... (this is a tiny piece of their preproposal, so should probably not get forwarded around... not that there is anything mind- boggling here.) ---------- Forwarded message ---------- Date: Sat, 13 Apr 1996 07:47:45 -0700 (PDT) From: Sandy Moy To: Sandra Moy Subject: NCSA In the next five years, NCSA is committed to achieving the following milestones: Hardware DSM to software DSM First visual supercomputer Serial #1 SGI/Cray First 1024 teraflop SGI/Cray Purchase a teraflop for the same price as the 512-node CM-5 in 1992 Major effort in scalable NT, coupling to Intel desktops Large funding for national teams in Grand Challenges/Computer Science Unprecedented cost sharing National participation from all 50 states Large university affiliations (CIC, SURA) Distributed leading-edge site National-scale metacomputer Strong program of participation for minorities, disabled, and EPSCoR states Distributed object framework Major software development with decentralized authoring Induce software writing of components In years one through five, we expect to accomplish the following: Build, enhance, and upgrade a national-scale integrated metacomputer with well-defined subcomponents, fine-tuned performance, and a functional problem-solving environment. Pending the state of the industry, we expect to deliver full teraflop performance to Grand Challenge applications during this timeframe. Provide our national user community with access to all major supercomputing architectures. Enable Partners to manage and communicate within their respective groups and across groups by developing a federated collaboratories infrastructure (i.e., collaborative tools that utilize the advanced computational infrastructure being developed) Expand the national academic and industrial computational science community who can benefit from an advanced infrastructure by providing access to mid-to-small-scale machines at Regional Outreach Partner sites and helping Grand Challenge users migrate codes to large-scale machines available both at Partner sites and at the leading-edge site. Similarly, as NCSA leading-edge sites move up the path to multi-teraflop systems, the previous generation production systems will be migrated to Regional Outreach sites for training, high-end servers, and targeted applications, such as technology development, that do not require the latest in processor power. By year 10, we will know we have succeeded if industry adapts and implements our prototype infrastructure and it penetrates all segments of society. We want to keep the Nation's commercial sector strong and competitive. We want to lead the revolution in how CS&E are done in the 21st century. We want to develop new methodologies to disseminate 21st century technologies and research tools and techniques to everyone-from the pre-literate to the post-literate-through national training and outreach programs. Section 3-Building the NCSA Advanced Computational Infrastructure: The National Metacomputer <> Grand Challenges have utilized supercomputers at remote centers since the inception of the program over five years ago. To move into the 21st Century, the entire computational infrastructure must not simply be "upgraded" but recreated in several dimensions. Our Grand Challenge teams have each identified several dimensions of the infrastructure that must be recast: scalable computation, visual information analysis, and object-distributed problem solving environments. Scalable Computation: Leveraging National Hardware and Software Hierarchies Leveraged National Hierarchical Computing Facility The NCSA Alliance will provide a distributed leading edge facility at NCSA, Ohio Supercomputer Center (OSC), and Argonne National Laboratory (ANL) covering the four major supercomputer families Cray, SGI, HP, and IBM. The Cray systems at OSC will provide a migration path for vector and T3D/E users to the joint SGI/Cray DSM system, which will premier in FY98 with 512 GFLOPS processors at NCSA and will grow to 2 TFLOPS by the turn of the century. During the same timeframe, we will grow the HP/Convex SPP systems to 500 GFLOPS aimed strategically at decision support and data mining. NSF will fund the NCSA systems, while cycle-sharing agreements will provide 30% of the ANL IBM cycles and 50% of the OSC Cray cycles to the Alliance users at no additional cost to NSF. These three sites will combine with regional partner sites operating major computing, database, and visualization facilities to form a tightly coupled national computational "Power Grid" forthe computational science community. This Power Grid will in turn be augmented with widely-distributed loosely coupled metacomputers, beginning with Wisconsin's more than 1000 processor Condor Flock, and later by coupling multiple campus-wide desktop-based metacomputers across the Alliance, including the several dozen partner universities such as those in SURA and CIC. National Metacomputing Software Infrastructure To make this national Power Grid function as a coherent metacomputer, we have formed a team headed by Ken Kennedy, Director of the Center for Research in Parallel Computing (CRPC). This computer science team will work with Grand Challenge teams to address the software needed within the leading edge distributed memory architectures and the middleware needed to create the tightly coupled and loosely coupled national metacomputers. Distributed Shared Memory NCSA's success in helping users move from vector to MPP (CM-5) and clusters of SMPs (SGI PowerChallengeArray), and our initial experiences moving users from MPP to DSM (HP/Convex Exemplar), underscore the need to invest in software (compilers, algorithms, scalable I/O) to make a "peak TF" machine a "usable TF" machine, beginning with a transition path for users to move from today's systems (C90, T3E, SP-2, CM-5) to DSM systems. Because most DSM systems are constructed from workstation technologies, the memory access protocols are based on workstation memory hierarchies. In particular, access to remote memory is handled by an extension of the cache miss protocol. This means that, in a naive implementation, accesses to memories on remote processor nodes will be handled synchronously. Since such accesses can take hundreds, or even thousands, of machine cycles, the requesting CPU is blocked during the time the request is being processed. By contrast, message-passing typically occurs asynchronously, and the most scalable programs rely on the ability to overlap communication and computation. To overcome this problem, processor designs for DSMs are turning to various prefetching strategies. The HP SPP-2000 will employ a new HP chip that permits each processor to have ten outstanding memory accesses without blocking. Clearly, overlapping memory operations with computation will be a complex process that should be left to the compilers or run-time library developers. An important challenge for compilers is to effectively use prefetching to achieve scalable computation on hardware DSMs. The CRPC parallel compilers team at Rice will target the SGI and HP DSM systems at NCSA to determine the extent to which scalability can be achieved by compiler technology for programs written in a machine-independent language. CRPC teams from Tennessee (Dongarra), Rice (Dennis), and UIUC (Reed) will provide tools and libraries specifically for DSM users. Dongarra's team will provide high performance parallel numerical algorithm libraries and tools (LAPACK, ScaLAPACK, SuperLU, LAPACK++) optimized specifically for DSM. Dennis and colleagues will provide prototype software for successive linear programming (SLP) and successive quadratic programming (SQP) approaches to very large scale discretized design and control problems, for integration of response surface methodology into rigorous optimal design schemes, for automatic differentiation of Fortran and C programs, and for determining the trade-off, or Pareto, surfaces for multicriterion decision making. Reed and colleagues at UIUC and Rice will adapt the D System distributed memory programming and performance analysis tools to add analysis of DSM architecture issues such as cache usage patterns and detection of false sharing. The Scalable I/O Initiative (Rice, ANL, UIUC, CalTech et. al.) has designated the NCSA SGI and HP DSM systems as large-scale testbeds for the initiative. This will mean that development work within the SIO, such as parallel file systems, high performance application programming interfaces, and abstract device interfaces, will be incorporated into the NCSA DSM systems. In addition, the I/O behavior of the Alliance Grand Challenge applications will be profiled and examined to drive the design of the parallel file systems policies, compiler techniques, and APIs. Collectively, these tasks will create a nation-scale, multi-petabyte parallel file system that can support high-performance input/output and data sharing for metacomputing applications. Portable software is also an essential requirement as we continue to move to new processor and system architectures every 24 to 36 months to reach a usable Teraflops by the end of the decade. ANL has developed a set of portable distributed systems (PETSc tools, Nexus/Globus/MPI-2/PEXEC) which will be made available on the SGI and HP DSM systems at NCSA. Tightly Coupled Metacomputers While we move toward Teraflops DSM architectures, we are also investigating multiple approaches to building a coherent metacomputer using multiple, geographically distributed large-scale resources. The Treadmarks system (Rice) as well as Penn's wide area network DSM software and vendor-proprietary DSM software will be scaled across the nationally distributed Alliance computing environment in a testbed aimed atinvestigating the transition from in-box hardware to national-scale software DSM. Teams from Indiana (Gannon; HPC++), Virginia (Wulf, Grimshaw; Legion), Wisconsin (Vernon), and Argonne will similarly use the Alliance environment as a testbed for object-oriented technologies can be used to build very large scale scientific applications. In particular, these teams will explore the integration of desktop object oriented, WWW, and agent technology such as CORBA, Java, and KQRML to provide metacomputing services such as dynamic scheduling, resource management, security and object file management on scales appropriate for grand challenge problems. Loosely Coupled National Metacomputer The Condor project at Wisconsin has extended the "Network of Workstations" model over the past decade from a building-wide to a campus-wide scale with 1000's of processors. Wisconsin's metacomputer (the Condor Flock), will provide a real-world testbed for new resource management and scheduling mechanisms, particularly dealing with very large scaling, as it serves as an allocable source of processing cycles for the computational biology portion of the NCSA user community. Similarly, the CRPC MetaWeb and WebFlow projects (Fox- Syracuse) and Web-based optimization tools (Rice, ANL) are developing the base infrastructure that will allow the use of WWW technology to integrate data and compute intensive metasystems on a national scale, including the use of intelligent servers to accept and interpret source code objects via the web, scheduling the solution on the appropriate architecture and returning the results to the user. As the HP/Intel processor on the desktop catches up with the high-end unix workstation, there are unprecidented HPC opportunities in harnessing the power of hundreds of thousands of these high performance desktop processors on the Internet, or even within institutional intranets, which currently remain idle an estimated 70-90% of the time. The Condor project as well as web-based metacomputing exploratory projects within CRPC and NT-scaling projects in collaboration with HP and Microsoft will allow us to move in this direction. 9-May-1996 13:06:04 -0700,9473;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Thu, 9 May 1996 13:06:04 -0700 (PDT) Return-Path: Received: from mailer8.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA22576; Thu, 9 May 96 13:06:04 -0700 Received: from mx2.cac.washington.edu by mailer8.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA40034; Thu, 9 May 96 13:06:03 -0700 Received: from denali2.nsl.nersc.gov by mx2.cac.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA24230; Thu, 9 May 96 13:06:01 -0700 Received: from vnet.ibm.com by denali2.nsl.nersc.gov (AIX 3.2/UCB 5.64/4.03) id AA18668; Thu, 9 May 1996 13:06:07 -0700 Message-Id: <9605092006.AA18668@denali2.nsl.nersc.gov> Received: from BETVMIC1 by VNET.IBM.COM (IBM VM SMTP V2R3) with BSMTP id 9897; Thu, 09 May 96 16:05:14 EDT Date: Thu, 9 May 96 16:05:13 EDT From: stoker@VNET.IBM.COM To: hpss-exec@nsl.nersc.gov, hpss-developer@nsl.nersc.gov Subject: HPSS Acceptance Test The following is a draft the Technical Committee developed for the EC's acceptance test of HPSS R3 with guidance from Ken Kliewer and me. The majority of the work was done by Ken, Randy Burris and Danny Teaff, thanks guys! The TC felt that unattended running of the acceptance test should be consider. Since this is a simulated load via scripts versus manually generated inputs, they didn't see any advantage to having only attended time counted. That wasn't the direction Ken and I gave them therefore we didn't include it here. As with all of the plan, it is open to discussion. Note that the % of time HPSS can be down or degraded and the site(s) to run the test are TBD. We are currently running the acceptance test script at CTC and MHPCC since they have the resources required to generate the load. This is only a test and NOT the formal acceptance period. That period will not start until we reach agreement on this plan. Please review the plan and provide your comments and inputs by 5/17 including any thoughts on the TBD items. Lack of inputs will be taken as agreement with the plan. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% TEST TEAM: The group of HPSS-knowledgeable engineers/scientists and test site operators assembled to conduct the Acceptance Test. ACCEPTANCE TEST PERIOD: The acceptance test period shall begin on a date agreeable to the members of the Test Team and shall end when HPSS has successfully met the STANDARD OF PERFORMANCE for 200 conjoint hours of ATTENDED OPERATION, or failed, necessitating initiation of a subsequent Acceptance Test Period. This 200 hour period can be measured from any time identified by the Test Team, i.e., the test period can be rolling. The Test Team will reach agreement on the impact of failure scenarios not covered by this document if they occur. The Acceptance Test can be terminated at any time through agreement of the Test Team. SITE(S) OF TEST: TBD STANDARD OF PERFORMANCE: During the ACCEPTANCE TEST PERIOD HPSS shall: * not exceed more than xx% DEGRADED OPERATION. * not exceed more than yy% UNSCHEDULED DOWNTIME. ATTENDED OPERATION: Period of time during which an operator or administrator is monitoring the operation of HPSS. The entire Acceptance Test will be conducted under Attended Operation. FULLY OPERATIONAL: All servers operational; all configured servers that are permitted to execute (i.e., "execute bit" is set) are functioning correctly. DEGRADED OPERATION: The system will be considered "up" but "degraded" if the Name Server, Bitfile Server, PVL, and at least one Storage Server, one Mover, and one PVR are providing their basic customer services. HPSS may also be declared to be in DEGRADED OPERATION if the members of the Test Team agree that substantial degradation of HPSS performance or functionality exists. DEGRADED OPERATION will become UNSCHEDULED DOWNTIME after 30 minutes and until a FULLY OPERATIONAL state is restored. SCHEDULED DOWNTIME: The period when "normal" maintenance actions (installation of updated software, for instance) occur and shall be counted as PAUSE for purposes of HPSS acceptance testing. PAUSE: A period of time that is, in effect, not part of the Acceptance Test period. PAUSE time has the effect of extending the test period beyond 200 hours. UNSCHEDULED DOWNTIME: Any period when HPSS does not satisfy at least the definition of DEGRADED OPERATION. CLARIFICATIONS: If a file is damaged in transmission and that damage is attributable to HPSS software, HPSS will be considered in UNSCHEDULED DOWNTIME until the situation is corrected. If the damage results in any loss of data or information, the Acceptance Test is terminated immediately as a failure. If any HPSS metadata become corrupted as a consequence of HPSS errata (as opposed to erroneous human data entry), HPSS will be considered to be in UNSCHEDULED DOWNTIME from the time the metadata became corrupted until the time the error and the metadata are corrected. If the metadata cannot be correctly restored, the Acceptance Test is terminated immediately as a failure. In the event that there are problems with infrastructure products (AIX, DCE, Encina, Sammi) not caused by improper HPSS use of the facilities, the Acceptance Test shall be in PAUSE mode until they are fixed. If improper HPSS use of infrastructure components is indicated, UNSCHEDULED DOWNTIME or DEGRADED OPERATION time will be credited appropriately. Performance difficulties such as network delays which are not attributable to HPSS problems shall be considered PAUSE. Any hardware problem shall be considered PAUSE. OVERRIDING CRITERION: If it is determined that any data whatsoever have been lost, the Acceptance Test will be terminated immediately as a failure. If it is determined after the Acceptance Test has nominally completed that any data whatsoever were lost, the test will have failed. LOAD: The script for the Acceptance Test will be the load.ksh script. This script is in the it_tests/d3/system/sizing_timing directory. This load represents greater than a 50% increase in the maximum current load as stated by each site. The load.ksh script will be submitted from multiple nodes. The submissions will be used to transfer small, medium, and large files. Note: Modifications to the script inputs may be required as experience in running this test is gained. The inputs to the Acceptance Test are summarized below: Min Max Min Max Average Type Time Time Size Size COS Verify API FTP PFTP Node MB/hour** ------- ---- ---- ---- ---- --- ------ --- --- ---- ---- -------- Burst 0 8 1 1MB 1 N Y N N 1 900MB/h Small 30 60 1 1MB 1 Y Y Y Y 2 120MB/h Medium 30 180 1MB 40MB 2 Y Y Y Y 2 2109MB/h Large 1800 3600 40MB 2GB 4 Y Y N N 3 4176MB/h -------- Total 7305MB/h **Average MB/h includes data transferred to verify results (if selected) and data transferred during migration from disk to tape. Average number of files created per hour is 1015. ----------------------------------------------------------------------- Notes: 1) Classes of service(s) with disk will be set up to migrate to tape . 2) To accommodate the loading profile, a high speed network must be available for transfers (e.g. HIPPI). 3) Each request actually decomposes into both a read and a write request. For FTP, there is a put followed by a get. For Client API requests, there is a write followed by a read to verify the file contents. 4) 1 and 2-way striping to disk and tape are assumed. There should be a minimum of 3 tape drives for this test. 5) If the test sites do not have the required hardware resources, the load will have to be adjusted. Thanks, Dennis Dennis E. Stoker Mgr, High Performance Storage Development and Integration IBM Government Industry - Houston 1810 Space Park Drive Houston, Texas 77058-3544 Phone: (713) 335-4181 Fax: (713) 335-4231 Internet: stoker@vnet.ibm.com 17-May-1996 07:02:00 -0700,9386;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Fri, 17 May 1996 07:02:00 -0700 (PDT) Return-Path: Received: from mailer15.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA20766; Fri, 17 May 96 07:01:59 -0700 Received: from mx1.cac.washington.edu by mailer15.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA28634; Fri, 17 May 96 07:01:59 -0700 Received: from stormy.clearlake.ibm.com by mx1.cac.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA12341; Fri, 17 May 96 07:01:53 -0700 Received: from R33N01.TC.CORNELL.EDU by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA14079; Fri, 17 May 1996 09:02:06 -0500 Received: (from root@localhost) by r33n01.tc.cornell.edu (8.6.9/8.6.6) id JAA31484; Fri, 17 May 1996 09:58:16 -0400 Date: Fri, 17 May 1996 09:58:16 -0400 From: Jeff Deutsch Message-Id: <199605171358.JAA31484@r33n01.tc.cornell.edu> To: hpss-developer@clearlake.ibm.com, hpss-exec@clearlake.ibm.com, systems-mss@tc.cornell.edu Subject: HPSS Acceptance Test at Cornell HPSS CONFIDENTIAL INFORMATION At 7:35am this morning (5/17) the HPSS acceptance test at Cornell reached 200 hours of operational time. The results are pretty impressive -- during those 200 hours we only saw 1.9 hours when HPSS was unavailable, and no time at all where the system ran in a degraded mode. That is an availability of over 99%! During the test run we had one tape drive which gave some intermittent errors, then failed completely. This was actually a great test for HPSS. By using HPSS commands to EOM volumes, disable and enable drives, and move cartridges we were able to manage the hardware failures. All that without restarting a server or impacting the acceptance test. The only true HPSS failures during the test were intermittent Storage Server crashes. We got some good delogs and core files from the crashes and the development team feels they have pinpointed the problem. The only other failure that impacted the acceptance test was a single SFS crash. The test created over 24,000 files per day and read/wrote over 110GB of data a day. That data was also migrated from disk to tape. I know there are currently some ongoing discussions concerning exactly what constitutes the official HPSS Acceptance Test. Folks reading this note should not make the assumption that HPSS has officially passed any test or milestone. I leave that determination to the Executive and Technical Committees. All we were doing at Cornell was running the HPSS test scripts according to the latest draft version of the Acceptance Test Plan. Details about the test are included below. Jeff Deutsch ============================================================================= Storage Classes | | Space | Storage | Physical | VV | | Id | Name | Avail | Segment | Block | Block|Migration|Purge ---|---------------|-------|---------|----------|------|---------|----- 1 | 1-Way Disk | 12GB | 1MB | 256KB | 1MB | 24min | 75% 2 | 2-Way Disk | 20GB | 4MB | 256KB | 2MB | 24min | 75% 4 | 4-Way Disk | 24GB | 64MB | 256KB | 4MB | 24min | 75% 11 | 1-Way Tape | 400GB | n/a | 256KB | 4MB | n/a | n/a 12 | 1-Way Tape (2)| 400GB | n/a | 256KB | 4MB | n/a | n/a 14 | 2-Way Tape | 800GB | n/a | 256KB | 4MB | n/a | n/a Hierarchy Id | Name | Storage Classes ---|----------------|-------------------- 1 | Small Files | 1 11 2 | Medium Files | 2 12 4 | Large Files | 4 14 Class of Service Id | Name | Hierarchy ---|--------------------------|---------- 1 | Small Files (default) | 1 2 | Medium Files | 2 4 | Large Files | 4 Test Parameters Min Max Min Max Average Type Time Time Size Size COS Verify API FTP PFTP Node MB/hour** ------- ---- ---- ---- ---- --- ------ --- --- ---- ---- -------- Burst 0 8 1 1MB 1 N Y N N 1 900MB/h Small 30 60 1 1MB 1 Y Y Y Y 2 120MB/h Medium 30 180 1MB 40MB 2 Y Y Y Y 2 2109MB/h Large 1800 3600 40MB 2GB 4 Y Y N N 3 4176MB/h -------- Total 7305MB/h **Average MB/h includes data transferred to verify results (if selected) and data transferred during migration from disk to tape. Data transferred during migration is only counted once, not twice like it would be in the "Bytes Moved" field of the Health and Status Window. Average data read and written (not counting migration) is 4720MB/h. Average total bytes created per hour is 2585MB. Average number of files created per hour is 1015. Run Log Total |Total |Total | | | Hours |Hours |Hours | | | Run |Unavail|Degraded| Date | Time | Event ------|-------|--------|------|-------|-------------------------------------- 0.0 | 0.0 | 0.0 | 5/07 | 19:30 | Test started 12.0 | 0.0 | 0.0 | 5/08 | 7:30 | Scheduled system down time 12.0 | 0.0 | 0.0 | 5/08 | 15:00 | Test restarted 24.0 | 0.0 | 0.0 | 5/09 | 3:00 | Tape mount failed - no system impact | | | | | wrote PTRs pvl104 and mvr089 38.0 | 0.0 | 0.0 | 5/09 | 17:00 | Switch fault 49.5 | 0.0 | 0.0 | 5/10 | 4:30 | Disk Storage Server crashed 49.5 | 0.0 | 0.0 | 5/10 | 8:00 | Crash noted - PTR ss_0112 written 49.5 | 0.5 | 0.0 | 5/10 | 8:30 | Test restarted 56.5 | 0.5 | 0.0 | 5/10 | 15:30 | Tape mount failed - no system impact | | | | | errpt shows possible hardware | | | | | problems in the tape drive; asked | | | | | system admins to investigate 71.5 | 0.5 | 0.0 | 5/11 | 6:30 | One robot offline, PVR takes drive | | | | | offline to HPSS. Tape HP0032 is | | | | | stuck in a drive. This drive is | | | | | essential to migration of storage | | | | | class 4. Drive won't be fixed | | | | | until Monday (at earliest). Class | | | | | 4 will be full by then 97.0 | 0.5 | 0.0 | 5/12 | 8:00 | Scheduled system down time 97.0 | 0.5 | 0.0 | 5/12 | 16:30 | Test restarted - Only a few GB left | | | | | in the class that can't migrate; | | | | | I EOM'ed the volume with the stuck | | | | | tape, and forced a migrate; it | | | | | failed because no drives were avail, | | | | | but that told me which VV was next | | | | | in line; I moved one PV out of the | | | | | robot with no drives and did an | | | | | HPSS move cartridge to put it in | | | | | the other robot; then kicked off | | | | | migration 121.5 | 0.5 | 0.0 | 5/13 | 17:00 | Drive fixed - all drives available | | | | | in HPSS 133.0 | 0.5 | 0.0 | 5/14 | 4:30 | Disk Storage Server crashed 133.0 | 0.5 | 0.0 | 5/14 | 8:00 | Crash noted, core file added to PTR | | | | | ss_0112. Also noted BFS memory | | | | | growth, not enough to impact test; | | | | | wrote PTR bfs0171 133.0 | 1.0 | 0.0 | 5/14 | 8:30 | Test restarted 138.5 | 1.0 | 0.0 | 5/14 | 14:00 | SFS crashed, no diagnostics in | | | | | /tmp/hpss.out 138.5 | 1.5 | 0.0 | 5/14 | 14:30 | Test restarted 157.0 | 1.5 | 0.0 | 5/15 | 9:00 | Scheduled system down time 157.0 | 1.5 | 0.0 | 5/15 | 10:30 | Test restarted 159.0 | 1.5 | 0.0 | 5/15 | 12:30 | Disk Storage Server crashed, core | | | | | and delog added to ss_0112 159.0 | 1.7 | 0.0 | 5/15 | 12:40 | Test restarted 160.3 | 1.7 | 0.0 | 5/15 | 14:00 | Entire SP2 crashed, not HPSS related 160.3 | 1.7 | 0.0 | 5/15 | 15:45 | Test restarted 179.3 | 1.7 | 0.0 | 5/16 | 10:40 | Disk Storage Server crashed, core | | | | | and delog added to ss_0112 179.3 | 1.9 | 0.0 | 5/16 | 10:50 | Test restarted 200.0 | 1.9 | 0.0 | 5/17 | 7:35 | Test reaches 200 hours, still running | | | | | 20-Mar-1996 21:05:59 -0800,4166;000000000011 Return-Path: Received: from carbon1.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA15670; Wed, 20 Mar 96 21:05:58 -0800 Received: from localhost by carbon1.u.washington.edu (5.65+UW96.03/UW-NDC Revision: 2.33 ) id AA06913; Wed, 20 Mar 96 21:05:57 -0800 Date: Wed, 20 Mar 1996 21:05:56 -0800 (PST) From: Sandy Moy X-Sender: sandy@carbon1.u.washington.edu To: Jim DeRoest Subject: Metacenter Regional Partnerships (fwd) Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Huge datasets.... note (far) below the reference to large datasets. I think that one of the things we might want to throw into the supercomputer partnership ring is the work on HPSS... both SDSC and NCSA seem to be interested in emphasizing hug volumes of data. COuld you come up with a couple of paragraphs on the work we could do with researchers requiring lots of data? (I suspect that the large files will then be combined with the highspeed network stuff that Steve will propose.) ---------- Forwarded message ---------- Date: Wed, 20 Mar 1996 09:41:32 -0800 (PST) From: George Lake To: Ed Lazowska Cc: sandy@cac.washington.edu Subject: Metacenter Regional Partnerships I will get this together shortly. I don't know if Ed noticed, but Larry has done something of an about face in his last message. Originally, he was advocating that we spend money on equipment to support metacenter use (hardware configurations that were largely cloned from one partner to another), somebody to tend it, other infrastructure to enable research, but not money to actually do research. In our discussion at NCSA, I stressed that UW can have an impact in "EHR" (NSF-speak for education and human resources) and how computers are designed and used (giving Snyder's desire to use his parallel compilers on serial computers to hide the latency in cache hierarchies). His last message is different. Equipment is absent from his list (almost certainly a mistake) and a focus on education, outreach, programmers/postdocs to support research. E.g. all "EHR". I take this as a green light to completely decide what we should do on our own. But, getting even the $500K will involve matching funds. We can likely bargain up considerably if we "overmatch". While that sum seems like a lot of dough, I don't think it will be that hard for a couple of reasons: 1) This looks like the mechanism whereby we can avoid getting a bill for $600K/yr for our networking (as recently happened to Berkely according to Ron). So, we can afford to INVEST money here. This is a place where a double negative "we can't afford not to INVEST money here" 2) The stress on EHR couples to some of the "legislative initiatives" on distance learning and the integration of technology with education. These are clearly complementary with this proposal. I will get the prose done, it will be longer than Larry wants, but likely "stronger". I will also propose some sort of matching scheme that takes real dollars as well as ones that we are already going to spend. Can you give me "ideas" for those dollars that we are already going to spend. I might include: Equipment that enables distance learning Projects that do it that will incorporate simulations or massive data sets Very high end classrooms (for example, good screens with stereo projection) High bandwidth networking for people/depts involved. ---George > > On Tue, 19 Mar 1996, Ed Lazowska wrote: > > > Sheesh! > > > > FYI: Lake met with Smarr at NCSA for a couple of hours late last > > week. He and I talked yesterday. We're putting together a skeleton > > of a couple of pages to make us one of 6 regional alliances with NCSA, > > for at least $500K/year in cash; can be bargained up. George should > > have this rolling real soon. > > > 19-May-1996 16:59:23 -0700,10693;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Sun, 19 May 1996 16:59:23 -0700 (PDT) Return-Path: Received: from mailer10.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA22982; Sun, 19 May 96 16:59:19 -0700 Received: from mx1.cac.washington.edu by mailer10.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA71453; Sun, 19 May 96 16:59:19 -0700 Received: from denali2.nsl.nersc.gov by mx1.cac.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA29689; Sun, 19 May 96 16:59:18 -0700 Received: from vnet.ibm.com by denali2.nsl.nersc.gov (AIX 3.2/UCB 5.64/4.03) id AA19604; Sun, 19 May 1996 16:58:30 -0700 Message-Id: <9605192358.AA19604@denali2.nsl.nersc.gov> Received: from BETVMIC1 by VNET.IBM.COM (IBM VM SMTP V2R3) with BSMTP id 0113; Sun, 19 May 96 19:57:24 EDT Date: Sun, 19 May 96 19:55:57 EDT From: teaff@VNET.IBM.COM To: hpss-exec@nsl.nersc.gov, hpss-tech@nsl.nersc.gov Subject: HPSS Acceptance Test Regarding the HPSS Release 3 200 hour Acceptance Test, the criteria distributed last week stated that only attended hours be counted. In canvassing the individual deployment sites, the majority consensus was the unattended hours should be counted. The rationale for treating unattended hours the same as attended is as follows: 1) The test automatically generates the mix of transfer requests. No man in the loop intervention is required. 2) Continuous testing is a more realistic test environment. Shutting down the system at the end of supported operations each day does not expose problems such as timeouts, memory growth, etc. 3) The time window for closing out Release 3 is rapidly approaching. If attended time must be supported, then 2 or 3 work shifts will be required. In addition, a. MHPCC will upgrade to AIX 4.1 on 5/20 and will be unable to continue running the tests which are now underway. b. Cornell will upgrade to AIX 4.1 within 2 weeks and will be unable ton continue running the tests which are now underway. The 200 hour test is now being run at both Cornell and Maui. The other deployment sites lack the resources to support our loading parameters. The AT criteria stated that testing would not be official until accepted by the EC. The tests which are now ongoing should be considered valid tests if they complete successfully within the set of availability numbers endorsed by the EC. Each deployment site was canvassed for their opinions on the two items described above, i.e. o Allow unattended operations o Count ongoing test results The majority consensus (11 to 1) from the site technical representatives was to allow unattended operations and tests which are currently running. Each site representative was given the action early last week to verify concurrence from their EC representative, and all but a few have reported back with that concurrence. With this input (unless redirected by the EC), we are pressing forward with the assumptions that Acceptance Test criteria will be modified to include unattended operations, and ongoing test results will count. A suggested revision to the Acceptance Test is provided below: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% TEST TEAM: The group of HPSS-knowledgeable engineers/scientists and test site operators assembled to conduct the Acceptance Test. ACCEPTANCE TEST PERIOD: The acceptance test period shall begin on a date agreeable to the members of the Test Team and shall end when HPSS has successfully met the STANDARD OF | PERFORMANCE for 200 conjoint hours of operation (attended | or unattended), or failed, necessitating initiation of a subsequent Acceptance Test Period. This 200 hour period can be measured from any time identified by the Test Team, i.e., the test period can be rolling. The Test Team will reach agreement on the impact of failure scenarios not covered by this document if they occur. The Acceptance Test can be terminated at any time through agreement of the Test Team. | SITE(S) OF TEST: Cornell and MHPCC (depending on availability of test | resources and an AIX 3.2.5 operating environment) STANDARD OF PERFORMANCE: During the ACCEPTANCE TEST PERIOD HPSS shall: * not exceed more than xx% DEGRADED OPERATION. * not exceed more than yy% UNSCHEDULED DOWNTIME. ATTENDED OPERATION: Period of time during which an operator or administrator is monitoring the operation of HPSS. | UNATTENDED OPERATION: Period of time during which the Acceptance Test | is executing, but no operator or administrator is present. | Test scripts must provide authomatic request submission. | Any transfer or verification errors must be recorded for | subsequent inspection. FULLY OPERATIONAL: All servers operational; all configured servers that are permitted to execute (i.e., "execute bit" is set) are functioning correctly. DEGRADED OPERATION: The system will be considered "up" but "degraded" if the Name Server, Bitfile Server, PVL, and at least one Storage Server, one Mover, and one PVR are providing their basic customer services. HPSS may also be declared to be in DEGRADED OPERATION if the members of the Test Team agree that substantial degradation of HPSS performance or functionality exists. DEGRADED OPERATION will become UNSCHEDULED DOWNTIME after 30 minutes and until a FULLY OPERATIONAL state is restored. SCHEDULED DOWNTIME: The period when "normal" maintenance actions (installation of updated software, for instance) occur and shall be counted as PAUSE for purposes of HPSS acceptance testing. PAUSE: A period of time that is, in effect, not part of the Acceptance Test period. PAUSE time has the effect of extending the test period beyond 200 hours. UNSCHEDULED DOWNTIME: Any period when HPSS does not satisfy at least the definition of DEGRADED OPERATION. CLARIFICATIONS: If a file is damaged in transmission and that damage is attributable to HPSS software, HPSS will be considered in UNSCHEDULED DOWNTIME until the situation is corrected. If the damage results in any loss of data or information, the Acceptance Test is terminated immediately as a failure. If any HPSS metadata become corrupted as a consequence of HPSS errata (as opposed to erroneous human data entry), HPSS will be considered to be in UNSCHEDULED DOWNTIME from the time the metadata became corrupted until the time the error and the metadata are corrected. If the metadata cannot be correctly restored, the Acceptance Test is terminated immediately as a failure. In the event that there are problems with infrastructure products (AIX, DCE, Encina, Sammi) not caused by improper HPSS use of the facilities, the Acceptance Test shall be in PAUSE mode until they are fixed. If improper HPSS use of infrastructure components is indicated, UNSCHEDULED DOWNTIME or DEGRADED OPERATION time will be credited appropriately. Performance difficulties such as network delays which are not attributable to HPSS problems shall be considered PAUSE. Any hardware problem shall be considered PAUSE. OVERRIDING CRITERION: If it is determined that any data whatsoever have been lost, the Acceptance Test will be terminated immediately as a failure. If it is determined after the Acceptance Test has nominally completed that any data whatsoever were lost, the test will have failed. LOAD: The script for the Acceptance Test will be the load.ksh script. This script is in the it_tests/d3/system/sizing_timing directory. This load represents greater than a 50% increase in the maximum current load as stated by each site. The load.ksh script will be submitted from multiple nodes. The submissions will be used to transfer small, medium, and large files. Note: Modifications to the script inputs may be required as experience in running this test is gained. The inputs to the Acceptance Test are summarized below: Min Max Min Max Average Type Time Time Size Size COS Verify API FTP PFTP Node MB/hour** ------- ---- ---- ---- ---- --- ------ --- --- ---- ---- -------- Burst 0 8 1 1MB 1 N Y N N 1 900MB/h Small 30 60 1 1MB 1 Y Y Y Y 2 120MB/h Medium 30 180 1MB 40MB 2 Y Y Y Y 2 2109MB/h Large 1800 3600 40MB 2GB 4 Y Y N N 3 4176MB/h -------- Total 7305MB/h **Average MB/h includes data transferred to verify results (if selected) and data transferred during migration from disk to tape. Average number of files created per hour is 1015. ----------------------------------------------------------------------- Notes: 1) Classes of service(s) with disk will be set up to migrate to tape . 2) To accommodate the loading profile, a high speed network must be available for transfers (e.g. HIPPI). 3) Each request actually decomposes into both a read and a write request. For FTP, there is a put followed by a get. For Client API requests, there is a write followed by a read to verify the file contents. 4) 1 and 2-way striping to disk and tape are assumed. There should be a minimum of 3 tape drives for this test. 5) If the test sites do not have the required hardware resources, the load will have to be adjusted. Thanks. 21-May-1996 16:40:16 -0700,1681;000000000001 Return-Path: Received: via tmail-4.0(2) for deroest; Tue, 21 May 1996 16:40:16 -0700 (PDT) Return-Path: Received: from mailer6.u.washington.edu by franklin01.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA17208; Tue, 21 May 96 16:39:45 -0700 Received: from mx1.cac.washington.edu by mailer6.u.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA43201; Tue, 21 May 96 16:39:45 -0700 Received: from stormy.clearlake.ibm.com by mx1.cac.washington.edu (5.65+UW96.04/UW-NDC Revision: 2.33 ) id AA21420; Tue, 21 May 96 16:39:42 -0700 Received: from scout.sdsc.edu by stormy.clearlake.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA21137; Tue, 21 May 1996 18:40:25 -0500 Received: (from gleicher@localhost) by scout.sdsc.edu (8.7.3/8.6.12) id QAA02570; Tue, 21 May 1996 16:38:26 -0700 (PDT) Message-Id: <199605212338.QAA02570@scout.sdsc.edu> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: Michael Gleicher Date: Tue, 21 May 96 16:38:26 -0700 To: hpss-exec@clearlake.ibm.com, hpss-developer@nsl.nersc.gov Subject: HPSS Web Page move Cc: gleicher@scout.sdsc.edu Folks - We've moved the HPSS Web page from: http://www.ccs.ornl.gov/hpss to http://www.sdsc.edu/hpss We have "Redirect"ed the existing link(s) at ORNL so that the move will be transparent - it worked just fine with the browsers that we tried (Netscape 2.0 and Mosaic (vintage June 1994) for SunOS and OmniWeb 2.0 for NeXTStep). Please give it a try and let me know if you encounter any problems. Thanks - Michael