ویراساینس.
سند علمی

Automatic Reclustering of Objects in Very Large Databases for High Energy Physics

Koen Holtman, Peter van der Stok, Ian Willers

چکیده

In the very large object database systems planned for some future particle physics experiments, typical physics analysis jobs will traverse millions of read-only objects, many more objects than fit in the database cache. Thus, a good clustering of objects on disk is highly critical to database performance. We present the implementation and performance measurements of a prototype reclustering mechanism which was developed to optimise I/O performance under the changing access patterns in a high energy physics database. Reclustering is done automatically and on-line. The methods used by our prototype differ greatly from those commonly found in proposed general-purpose reclustering systems. By exploiting some special characteristics of the access patterns of physics analysis jobs, the prototype manages to keep database I/O throughput close to the optimum throughput of raw sequential disk access.

اسناد مشابه

  • An Approach to Automatic Indexing of Scientific Publications in High Energy Physics for Database SPIRES HEP

    سند علمی۲۰۰۲CoRRA. V. Averin, L. A. Vassilevskaya

    We introduce an approach to automatic indexing of e-prints based on a patternmatching technique making extensive use of an Associative Patterns Dictionary (APD), developed by us. Entries in the APD consist of natural language phrases with the same semantic interpretation as a set

  • A caching mechanism to exploit object store speed in High Energy Physics analysis

    مقاله‌ی دسترسی آزاد۲۰۲۲Cluster Computing['Vincenzo Eduardo Padulano', 'Enric Tejedor Saavedra', 'Pedro Alonso', 'Javier López Gómez', 'Jakob Blomer', '']

    Abstract Data analysis workflows in High Energy Physics (HEP) read data written the ROOT columnar format. Such has traditionally been stored files that are often via network from remote storage facilities, which represents a performance penalty especially for processing I/O bound

  • Use of a data base in the on-line environment of a large-scale high-energy physics experiment

    مقاله‌ی دسترسی آزاد۱۹۸۴Computer Physics CommunicationsA.Savoy-Navarro

  • Production Experience with Multi-terabyte Object Databases in High Energy Physics

    سند علمی۲۰۰۰M. Nowak, D. Düllmann

    CERN, the European Organisation for Nuclear Research, is currently building a new large-scale particle accelerator, the Large Hadron Collider (LHC). Four major physics experiments are scheduled to start collecting data at the LHC in 2005. They will collect raw data with rates fro

  • Reclustering of High Energy Physics Data

    سند علمی۱۹۹۹SSDBMMartin Schaller

    The coming high-energy physics experiments will store Petabytes of data into object databases. Analysis jobs will frequently traverse collections containing millions of stored objects. Clustering is one of the most effective means to enhance the performance of these applications.

  • Critical Database Technologies for High Energy Physics

    سند علمی۱۹۹۷VLDBDavid M. Malon, Edward N. May

    A number of large-scale high energy physics experiments loom on the horizon, several of which will generate many petabytes of scientific data annually. A variety of exploratory projects are underway within the physics computing community to investigate approaches to managing this