Diii – D Data Management

DISCLAIMER This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. ABSTRACT The DIII–D tokamak at the DIII–D National Fusion Facility routinely acquires ~500 Megabytes of raw data per pulse of the experiment through a centralized data management system. It is expected that in FY01, nearly one Terabyte of data will be acquired. In addition there are several diagnostics, which are not part of the centralized system, which acquire hundreds of megabytes of raw data per pulse. There is also a growing suite of codes running between pulses that produce analyzed data, which add ~10 Megabytes per pulse with total disk usage of about 100 Gigabytes. A relational database system has been introduced which further adds to the overall data load. In recent years there has been an order of magnitude increase in magnetic disk space devoted to raw data and a Hierarchical Storage Management system (HSM) was implemented to allow 7×24 unattended access to raw data. The management of all of the data is a significant and growing challenge as the quantities of both raw and analyzed data are expected to continue to increase in the future. This paper will examine the experiences of the approaches that have been taken in management of the data and plans for the continued growth of the data quantity.