Content:

• Introduction
• Detector setup
• Runtime database
• Parameter containers
• Input/Output
• Version management
• Examples (Macros)
• Description of functions provided for parameter I/O

Introduction

1. To avoid the need of many different parameter files
2. To avoid to keep the connection to Oracle open all the time during a long analysis of many files
3. To be sure to have all initialization parameters available e.g. for running a batch job overnight
4. To distribute the parameters within the collaboration when the direct access to Oracle is not possible or the newest data are not yet written to the database

Detector setup

Runtime database

• Oracle database
• ROOT file
• ascii file

Parameter containers

Each container has additional data members to support input/output:

Bool_t status

If this flag is true, the container is never reinitialized again automatically.

Bool_t changed

This flag is true after the (re)initialization or is set true by tasks creating new parameters which should be saved before a reinitialization. Then the container will be written to the output (if defined) and the flag is reset.

Int_t version[3]

version[1] is the version of the parameters taken from the first input, version[2] the one taken from the second input used for the actual initialization of the container.

There is no common structure for the parameter containers. The containers for the MDC are implemented as pointer arrays forming a tree like structure.
For example the MdcCalPar container for the calibration parameters is an array of 6 pointers (for the 6 sectors) pointing to arrays of 4 pointers ( for the 4 modules) pointing to 6 arrays (for the 6 layers) pointing to arrays of pointers with their size depending on the number of wires in the corresponding Mdc layer. These arrays finally hold pointers to the objects which contain the real parameters slope and offset for this wire.
This structure is the most compact way to store the parameters: The pointer to a missing module is 0 and the layer array is not created.
These containers can be written out to a ROOT file and read in again using the default ROOT streamer function. It's not possible to plot the data directly. However the program provides a function to fill the parameters on request in a ROOT NTuple and then use the Draw function of the NTuple.

Input/Output

The class design for I/O was developed according to the following user requests:

• Provide interfaces to Oracle, ROOT files and ascii files
• The actually used I/O is defined in a Macro
• The interface to Oracle is a shared library completely separated from the other analysis code to avoid precompiler flags. To compile Oracle-related code one needs the ODBC library or the Oracle precompiler. Licenses for these tools are needed.
• Each detector has its own classes containing the concrete read() functions

---

Class diagram of runtime database

---

For each type of I/O a set of classes is implemented, with the main class deriving from the base class HParIo and containing functions to open and close the I/O, e.g. a database connection in case of HParOraIo, a ROOT file in case of HParRootFileIo, an ascii file in case of HParAsciiFileIo. The I/O class is instantiated in a macro and a pointer to it is stored as first/second input or output in the runtime database. Internally this class has as a data member a pointer to a connection class (HOraConn, HParRootFile, fstream). It holds also a list of detector I/O classes which are instantiated automatically according to the actual setup when a connection is opened and deleted when the connection is closed. All detector classes are derived from the base class HDetParIo which itself is derived from TNamed. The detector classes have member functions to initialize and write their respective containers.

I/O base class	for Oracle DB	for ROOT file	for Ascii file
HParIo	HParOraIo	HParRootFileIo	HParAsciiFileIo
-	HOraConn	HParRootFile	fstream
HDetParIo	HMdcParOraIo ...	HMdcParRootFileIo ... all derived from base class HDetParRootFileIo	HMdcParAsciiFileIo ... all derived from base class HDetParAsciiFileIo

A container itself does not know the concrete type of I/O. It receives from the runtime database a pointer of the base type HParIo. With the name of the detector I/O class (e.g. for MDC: "HMdcParIo") it then gets via the list of I/O objects access to the init() and write() functions.
For more detailed information about the member functions see Description of functions provided for parameter I/O).

Version management

It takes time to read the initialization parameters, especially to read them from Oracle (some seconds up to minutes depending on the number and type of the containers). Therefore the most important request for the implementation is to avoid reading the same information several times and storing them several times in an output. Because a container is initialized and written always as a whole object, this has the following consequences:

• A reinitialization of a container must be avoided if the parameters in this container do not change.
• A container, which has not changed, doesn't need to be written out
• The containers should not hold parameters which are fixed, together with parameters which change very often (maybe from event file to event file).
• Parameters needed by several tasks should be stored in a separate container

To achieve this, a local version management was implemented following the one in the Oracle tables. To be able to store the complete runtime database in one ROOT file this version information must be stored also in the ROOT file. (For the ascii I/O a version management is not foreseen, however.)

The runtime database (class HRuntimeDb) has as a data element a list of objects of type HEventFile, one object for each event file one wants to analyze. HEventFile is derived from TNamed and the object name is identical with the name of the event file. The event file must not be a listmode data file, however, as each event has stored the run id in the event header.
Each object has three data elements:

The run id of the event file

In Oracle the versions are related to the run id and, via a special table, also related to a listmode data file. The Oracle read routines first look if the run id is >=0. In this case, the name of the file is irrelevant, irrespective of the file type - data summary tape or ROOT data file. If the run id is negative (default value -1), the object name is taken to be the name of the listmode data file, and the run id is read from Oracle and stored.

A list of objects of type HParVersion (derived from TNamed) one for each container

Each object has the same name as the container it belongs to and has two data elements: an array for the two possible input versions and the output version (for a ROOT file as output)

A name of a reference file (optional)

If this string is not empty all containers are initialized with the same parameters than used for this reference file

Suppose one wants to analyze several event files taking the parameters from Oracle and writing them out to a ROOT file. The following actions are taken:
When a container is initialized from Oracle, first the version of the parameters needed for the run id is determined. This version is stored in the container itself and in the version table described above. Then the parameters are read and stored in the container. The flag 'changed' is set to kTRUE. All containers which are not fixed (status flag is kTRUE) are initialized in this way in a loop.
Before the analysis of the next event file, all containers which have changed are written to the ROOT file. For this, first the version table is checked if the container with this input version has already been written once before. If the container with this input version can be found, only the corresponding output version is stored in the table, but the output itself is suppressed. Otherwise, the container is written and the new output version stored in the table. Then the 'changed' flag of the container is reset.
Then for each container the version for the next event file is determined and compared to the one in the container. If the version is the same, a reinitialization is not needed and the container is not changed. Only the input version is stored in the version table.
After all files have been analyzed, the database contains a list in which for all event files the input and output versions of all containers are stored. To use the ROOT file the next time as an input, this list is written also to the ROOT file (the input versions are not written, only the output versions) together with the setup of the spectrometer. Then the output ROOT file is a snapshot of the actual runtime database and can be used again for initialization.

In the analysis the initialization is done inside the event loop (Except for some containers which hold information needed to create the categories or other parameter containers. These containers are created and initialized only once in the init() functions of the detectors.) Each time a new event file has to be opened the runtime database function getNextEventFile() is called. This function loops over all containers and writes them to the output if necessary. Then it retrieves the next file object from the list of event files, loops over all containers, calls their init() functions and returns a pointer to the file object. At the end of the file list it writes out automatically the setup and the version table and returns a NULL-pointer. Then the event loops stops (if it has not stopped before after processing the selected number of events).

Additionally the runtime database has a member function readAll() which can be called in a macro after defining the parameter I/O, the setup, the list of event files and all containers. The complete runtime database is created and written to a ROOT file without an event loop. This can be used to create a ROOT file which contains all parameters one needs for maybe very different configurations, but connecting only once to the database. Such a ROOT file can then also be sent to people who have no direct access to Oracle.

Not all data might be available from the Oracle database, especially at the beginning of an analysis or the database might not have been updated with the newest data. Also a ROOT file written earlier might not contain the complete information one needs now. Therefore two inputs have been foreseen. This can be used, e.g. to combine the newest information contained in a ROOT file (e.g. new calibration parameters) with information retrieved from Oracle or from another ROOT file. The output Root files contains the merged information. One simply has to use the ROOT file with the newest information as first input and the other one as second input in the runtime database.
Additional containers or additional versions needed for additional event files can be added to an already existing ROOT file by using it as first input, as well as output. The file must be opened in UPDATE mode. (see Example1).

By default, the newest version of a container valid for a given event file would be taken from Oracle or ROOT file. Via the member function setInputVersion() another version can be set explicitly in a macro (see Description of functions provided for parameter I/O).

It is not possible to merge the information of more than two ROOT files in one step, nor to use many ascii files in the automatic initialization. However, in a macro the information can be filled stepwise into a ROOT file to end up with a maximum of two needed inputs. (see Example2).

---

Examples (Macros)

---

The following macros show examples of how to create and fill a runtime database using different parameter sets for different run ids of event data.
(The data read from Oracle are partly dummy numbers filled into the tables only to test the version management.)

Example1: Initialize from Oracle, save into a ROOT file

This macro reads the initialization parameters needed for the analysis of the MDC up to CAL1 level for two event files. The calibration parameters and the lookup table are different for the two event files. The runtime database is saved into a ROOT file named "ora.root". ( The code is colored. )

Create a Hades object.
   Hades myHades;

Create the detector and its setup
Here the Mdc consists only of the first module in the first sector.
   HSpectrometer* spec=gHades->getSetup();
   HMdcDetector mdc;
   int mod[4]={1,0,0,0};
   mdc.setModules(0,mod);
   spec->addDetector(&mdc);

Define the event files to be analyzed in the runtime database.
   HRuntimeDb* rtdb=gHades->getRuntimeDb();
   rtdb->addEventFile("test98.dat");
   rtdb->addEventFile("test99.dat");

Create the Oracle I/O and open the connection (here default connection as user Hades with read-only access).
Define it as first input in the runtime database.
   HParOraIo ora;
   ora.open();
   rtdb->setFirstInput(&ora);

Create the ROOT file I/O and open a file with write access
Define it as output in the runtime database
   HParRootFileIo output;
   output.open("ora.root","RECREATE");
   rtdb->setOutput(&output);

Define the first event file to be analyzed
   rtdb->setCurrentEventFile(0);

Initialize the setup:
This creates the containers "MdcRawStruct" and "MdcGeomStruct" and initializes them from Oracle.
"MdcRawStruct" contains the information which motherboards are used and the number of the TDCs on the boards (needed to create the raw category).
"MdcGeomStruct" stores the number of the wires in each layer of the MDC module(s) (needed to create the cal category and the other parameter containers).
Both containers are initialized only once and are not allowed to change during the analysis.
   if (!gHades->getSetup()->init("raw")) {
      printf("Error\n");
      return;
   }

Set the reconstruction tree.
This creates the containers "MdcCalPar" and "MdcLookupGeom".
"MdcCalPar" contains the calibration parameters and "MdcLookupGeom" the information which TDC is connected to which wire. Both containers might be initialized several times during the analysis.
   gHades->getTask()->connect(new HMdcCalibrater("mdc.cal","mdc.cal"));
   gHades->getTask()->connect(NULL,"mdc.cal");

If one wants to have an other version for calibration parameters as the newest version for a certain file one can define this here: e.g.
   rtdb->setInputVersion("test98.dat","MdcCalPar",2,1);
Then version 2 would be used instead of version 1.

Define the data source, the unpacker, the output file and make the tree.
   HldFileSource source;
   source.addUnpacker(new HMdcUnpacker());
   gHades->setDataSource(&source);
   gHades->setOutputFile("test.root","RECREATE","Test",compLevel);

Now one has two possibilities:
One can read all parameters in advance from Oracle, write the runtime database to the ROOT file, close the Oracle connection and then use the ROOT file for the initialization (and the output) in the event loop.
   if (!rtdb->readAll()) {
      printf("********** ERROR **********\n\n");
      return;
   }
   rtdb->print();
   rtdb->closeFirstInput();
   rtdb->setFirstInput(&output);
   ...

Or one omits this step and keeps the connection to Oracle open during the event loop.
Afterwards the input and output are closed.
If the parameters in a container would be changed during the analysis by a given task (creating e.g. a new calibration), then these parameters would be stored in the ROOT file, not the original ones read from Oracle.
   if (!gHades->init()) {
      printf("Error in Hades::init()\n");
      return;
   }
   gHades->makeTree();
   gHades->eventLoop(1000000);

   rtdb->print();
   rtdb->closeFirstInput();
   rtdb->closeOutput();
   ...

The function call rtdb->print() leads to the following output on the screen:

-----------------------------------------------------------
---------  actual containers in runtime database  ---------
OBJ: HMdcRawStruct      MdcRawStruct    Mdc parameters for hardware structure : 0
OBJ: HMdcGeomStruct     MdcGeomStruct   Mdc parameters for geometry structure : 0
OBJ: HMdcCalPar         MdcCalPar       calibration parameters for Mdc : 0
OBJ: HMdcLookupGeom     MdcLookupGeom   Mdc lookup table from raw to cal1 : 0
-----------------  event files, versions  -----------------
file name
     container  1st-inp-version  2nd-inp-version  output-version
test98.dat   run id -98
     MdcRawStruct        0            -1            1
     MdcGeomStruct       0            -1            1
     MdcCalPar           1            -1            1
     MdcLookupGeom       2            -1            1
test99.dat   run id -99
     MdcRawStruct        0            -1            1
     MdcGeomStruct       0            -1            1
     MdcCalPar           2            -1            2
     MdcLookupGeom       1            -1            2
---------------------  input/output  ----------------------
first Input:
Oracle-Database: Hades    Username: HADES
input/output for detectors: 
OBJ: HMdcParOraIo    HMdcParIo        : 0

second Input:
 none

Output:
HParRootFile**          ora.root
 HParRootFile*          ora.root
  KEY: HMdcRawStruct    MdcRawStruct;1  Mdc parameters for hardware structure
  KEY: HMdcGeomStruct   MdcGeomStruct;1 Mdc parameters for geometry structure
  KEY: HMdcCalPar       MdcCalPar;2     calibration parameters for Mdc
  KEY: HMdcCalPar       MdcCalPar;1     calibration parameters for Mdc
  KEY: HMdcLookupGeom   MdcLookupGeom;2 Mdc lookup table from raw to cal1
  KEY: HMdcLookupGeom   MdcLookupGeom;1 Mdc lookup table from raw to cal1
  KEY: HMdcDetector     Mdc;1           Mdc setup
  KEY: HEventFile       test98.dat;1    event file
  KEY: HEventFile       test99.dat;1    event file
input/output for detectors: 
OBJ: HMdcParRootFileIo      HMdcParIo        : 0
-----------------------------------------------------------

Example2: Initialize from several ascii files, save into a ROOT file

The following example shows what one has to do in a macro if one needs for the analysis several versions of a container, but these versions have to be read from many different ascii files (or from many ROOT files).
In a first step a ROOT output file is created with the different versions inside; the version information must be stored explicitly. Then this ROOT file can be used as input in the further analysis.

Create Hades and define the setup.
   Hades myHades;
   HSpectrometer* spec=gHades->getSetup();
   HMdcDetector mdc;
   int mod[4]={1,0,0,0};
   mdc.setModules(0,mod);
   spec->addDetector(&mdc);

Define the event files where a new initialization is needed.
   HRuntimeDb* rtdb=gHades->getRuntimeDb();
   rtdb->addEventFile("run10.dat");
   rtdb->addEventFile("run20.dat");
   rtdb->addEventFile("run30.dat");

Create an ascii I/O and open the first ascii file.
   HParAsciiFileIo input;
   input.open("ascii1.tmp","in");

Create the ROOT file output
   HParRootFileIo output;
   output.open("manyfiles.root","RECREATE");
   rtdb->setOutput(&output);

Create the containers and initialize them from the locally defined ascii input.
Write them to the output and store the output versions in the version table. Then close the input.
   Int_t version;

   HMdcGeomStruct mdcpar;
   rtdb->addContainer(&mdcpar);
   mdcpar.init(&input);
   version=mdcpar.write(&output);
   rtdb->setRootOutputVersion("run10.dat","MdcGeomStruct",version);

   HMdcCalPar calpar;
   rtdb->addContainer(&calpar);
   calpar.init(&input);
   version=calpar.write(&output);
   rtdb->setRootOutputVersion("run10.dat","MdcCalPar",version);

   input.close();

Now open the next ascii file, initialize the container for the calibration parameters from it (the MdcGeomStruct container is initialized only once and then stays constant), store the output version and close the input.
   input.open("ascii2.tmp","in");
   calpar.init(&input);
   version=calpar.write(&output);
   rtdb->setRootOutputVersion("run20.dat","MdcCalPar",version);
   input.close();

Do the same for the third file.
   input.open("ascii3.tmp","in");
   calpar.init(&input);
   version=calpar.write(&output);
   rtdb->setRootOutputVersion("run30.dat","MdcCalPar",version);
   input.close();

Now the version table and the setup information must be stored in the output file.
   rtdb->writeVersions();
   rtdb->writeSetup();

The output file contains one version for the MdcGeomStruct container and three versions for the MdcCalPar container. This file can be used now as input for the initialization.
   rtdb->setFirstInput(&output);

Other event files which use the same parameters as the one already defined before can now be inserted in the list. E.g. the event file "run31.dat" below would take the same parameters as the event file "run30.dat" and the file "run32.dat" the same as "run10.dat".
At the end the actual file list pointer must be reset.
   Int_t pos=0;
   rtdb->insertEventFile("run11.dat",++pos,"run10.dat");
   ...
   pos=rtdb->getEventFilePosition("run30.dat");
   rtdb->insertEventFile("run31.dat",++pos,"run30.dat");
   rtdb->insertEventFile("run32.dat",++pos,"run10.dat");
   ...
   rtdb->setCurrentEventFile(-1);

---

Description of functions provided for parameter I/O

---

For complete list of all classes and their functions see Class index

1. Definition of the setup (the detectors and the modules in each detector)

The spectrometer object is created in the constructor of the class Hades and is accessible with the function
HSpectrometer* Hades::getSetup()
e.g. HSpectrometer* spectrometer=gHades->getSetup();

Each detector must be created in a macro by calling the constructor of the detector class. The setup of each detector is defined with the member function
void HDetector::setModules(Int_t sec,Int_t* modules) .
This function takes the number of a sector (starting with 0) and a locally defined Int_t array with the information which modules are defined in the sector. (The size of this array must be the maximum number of modules this detector could have in a sector, e.g. 4 modules in an Mdc per sector. An active module is defined by a number not equal 0.)

Each detector must be added to the list of detectors in the spectrometer class by the function
void HSpectrometer::addDetector(HDetector* detector)

e.g. // here Mdc with 2nd module in 4th sector
HMdcDetector mdc;
Int_t modules[4]={0,1,0,0};
mdc.setModules(3,modules);
spectrometer->addDetector(&mdc);

The setup information must be written to the ROOT output file. Normally this is done automatically by the runtime database. However in some cases it has to be done explicitly in a macro, e.g. if a special output file should be created not defined as output in the runtime database. One of the following functions can be called, which take a pointer to an I/O as argument:
Bool_t HSpectrometer::write(HParIo* io)
Bool_t HDetector::write(HParIo* io)

2. Definition of the event files

The class HRuntimeDb holds a list of event files to be analyzed or needed for initialization. Is has some functions to manage this list:
void HRuntimeDb::addEventFile(Text_t* name,Text_t* refFile="") to add an event file at the end of the list or
void HRuntimeDb::insertEventFile(Text_t* name,Int_t position,Text_t* refFile="") to add it at a special position.
"name" is the name of the event file to be added. If all parameters should be the same as for another event file (which must be a part of this list), the name of this reference file "refFile" must be provided as second argument.

To remove an event file from the list one uses
void HRuntimeDb::removeEventFile(Text_t* name)
void HRuntimeDb::removeEventFile(Int_t position) .

The complete list can be cleared with
void HRuntimeDb::clearEventFileList().

The files in the list are processed in the same order as defined in the list. However the analysis does not have to start with the first event file (e.g. the first file could be a reference file only needed for the initialization). The start file must be selected with the function
void HRuntimeDb::setCurrentEventFile(Int_t pos=-1) . 'pos' is the position: ((first event file to be analyzed) - 1).

To step through the list of event files the function
HEventFile* HRuntimeDb::getNextEventFile()
is used. It calls internally the write() and then the init() functions of all containers using the I/Os defined in the runtime database.

3. Definition of input(s)/output

Each I/O must be created in a macro and the connection must be opened (with Bool_t open(...) returning kTRUE if successful) and closed (with void close() ) again later. To check if a connection is open each I/O has a function Bool_t check() and a function void print() to get some information on the I/O) :

• Oracle I/O (class HParOraIo )
  The connection is opened by
  Bool_t HParOraIo::open() (connect the user "Hades" with the database "Hades" in read-only mode)
  Bool_t >HParOraIo::open(char* user) (connects "user" with database "Hades"; asks for the password)
  Bool_t HParOraIo::open(char* dbName,char* user) (connects "user" with database "dbName"; asks for the password)
  
  
• ROOT file I/O (class HParRootFileIo )
  It is opened by Bool_t HParRootFileIo::open(Text_t* filename, Option_t* option="READ",Text_t* ftitle="",Int_t compress=0) with the options
  • "READ" (opens an existing file only for reading; this is the default)
  • "NEW" (creates a new file; if the file exists already it will not be opened)
  • "UPDATE" (opens an existing file with read and write access)
  • "RECREATE" (creates always a new file and overwrites an existing file)
  ROOT has only one global file pointer which must point to the actually accessed ROOT file. This pointer is set with the function
  void HParRootFileIo::cd() .
• Ascii file I/O (class HParAsciiFileIo )
  An ascii file is opened by
  Bool_t HParAsciiFileIo::open(Text_t* filename, Text_t* status)
  with status "in" or "out".

Each I/O can be used as input and output in the runtime database for the automatic initialization or storage via
Bool_t HRuntimeDb::setFirstInput(HParIo* io)
Bool_t HRuntimeDb::setSecondInput(HParIo* io)
Bool_t HRuntimeDb::setOutput(HParIo* io) .
In this case the corresponding close() functions in the runtime database should be used to take care of the automatic storage of the setup and version information:
void HRuntimeDb::closeFirstInput()
void HRuntimeDb::closeSecondInput()
void HRuntimeDb::closeOutput()

4. Parameter containers

The pointers of all parameter containers are stored in a list in the runtime database. If a container is created explicitly in a macro it must be added to this list with the function
Bool_t HRuntimeDb::addContainer(HParSet* container)

It can be accessed via its name with
HParSet* HRuntimeDb::getContainer(Text_t* name)
and then casted to the concrete type.

void HRuntimeDb::removeContainer(Text_t* name) removes a container from the list and deletes it.
void HRuntimeDb::clearContainerList(void) clears the list, but does not delete the containers.

Bool_t HRuntimeDb::initContainers(void) initializes all containers for the actual event file from the inputs defined in the runtime database and
Bool_t HRuntimeDb::writeContainers(void) writes them to the output. The input/output versions are stored in the version table in the runtime database.

Each container has init() and write() functions to handle the I/O individually:
Bool_t init(HParIo* input) initializes the container from 'input'
Bool_t init(HParIo* input,Int_t* modules) initializes only the modules defined in the array 'modules' from 'input'.

Int_t write(HParIo* output) writes a container to 'output' returning the output version.

The init() functions should however be used directly only for ascii input.
The input/output versions are not stored automatically in the runtime database.

Once a container is initialized and a reinitialization is not needed or should be suppressed the status flag of the container can be set (and reset) with void setStatic(Bool_t flag=kTRUE) .