1. How to Configure and Write Reduced Data Sets?
2. How to Use the Tape Robot to Back Up and Restore Reduced Data Sets?
3. Using the Diagnostic Test to Look at Data

How to Configure and Write Reduced Data Sets?
by Szabi

    The Reduced DataSet (RDS) writer is a utility to record a subset of data channels from the IFO. The system requires gigabit speed access to data and a local RAID array of disk to be able to keep up with the datastream. AIT-2 tape robot should also be installed on the RDS server to make the transport of huge data files possible.
    Presently there is one computer in LLO, which satisfies these requirements. This machine is quad processor SUN Enterprise 450, which is located at the Computer User's room and it is accessible through the General Computing (GC) network. Its name is 'decatur.ligo-la.caltech.edu' (130.39.245.73). Please contact the system administrators (Tom Evans or Shannon Roddy) with computer/account related problems.
    There are two directories on decatur's disk, which could be of interest of people using the RDS software. One is the RAID0 array (located at /export/raid0/) and the other is the Diagnostic Test Tool (DTT) software at /export/disk0/gds/startup/main .

    The directory /export/raid0/RDS_example contains a complete set of neccessary files for your RDS job:

• rds.conf : The RDS writer records the set of channels given in the configuration file rds.conf
• rds      : The script you will use to start the RDS recording. It is unlikely that you will use any other function of thisthis script. In case you need it, help is available:

> decatur {40} rds -help
Usage: rds { start [mdir] | stop | smdump | smrepair | frdump }
• RdsWrite : The actual RDS writer program.

Follow the few easy steps to write your own reduced dataset:
 

1. Create your unique directory on the RAID0 disk (or use one of the standard ones)

e.g.: mkdir /export/raid0/MyUniqueDir

2. Copy the neccessary files from the master directory

e.g.: cp /export/disk0/rds/* /export/raid0/MyUniqueDir/

3. Step into your directory and tailor the channel list

e.g.: cd /export/raid0/MyUniqueDir ; emacs rds.conf
   NOTE: If you intend to include testpoints in your list, make sure that ALL OF YOUR testpoints are set before you start RdsWrite! See step 4.

4. OPTIONAL by Ed To set testpoints: at the shell prompt, go to a CDS console in the control room. Right click with the mouse and select DIAGNOSTICS:

Select command line interpreter. You need to know the NUMBER of the test point you wish to write to the reduced data set. A list of the numbers and their assignments to named channels can be found in  T980020-A.pdf. In the Command Line Interpreter:
       diag> tp set 0 <test point number>
       diag> tp show 0
To clear a test point, type
       diag> tp clear 0 <test point number>
Repeat the tp set line for each test point you have included in rds.conf. Note that not all LSC digital inputs and outputs are test points. Only if the channel appears with a test point number in T980020-A.pdf does it need to be set.

5. Start your RDS process

e.g.: rds start

6. Look at the error messages and correct your mistakes. You might want to redirect your output into a file!

e.g.: rds start >& logy.log

7. Go get a coffee or work. Periodically chek for errors.

8. You should see directories and files created.

e.g.: /export/raid0/MyUniqueDir/01-02-02_16:09:15.0/H-665165401.F

9. When finished, retrieve your process ID and kill your job.

e.g.:
decatur {71} ps -u smarka | grep RdsWrite
20120 pts/14   0:03 RdsWrite
decatur {72} kill -KILL 20120

10.Check the integrity of your files.

11.Clean up after yourself write a tape or move your files. Others might need the place. (there are exceptions... are there?)

---

How to Use the Tape Robot to Back Up and Restore Reduced Data Sets?
by Ed
 

• I will assume you are an ordinary user running the shell tcsh.
• These instructions tell you how to do the simplest thing - make some tape archives on a SINGLE TAPE on decatur.

PREAMBLE: Reduced data sets are made on decatur, the sun ultra 450 in the computer users room. The tape drive ison top of the 450. BEFORE OPENING THE DOOR, MAKE SURE THAT THE MECHANISM IS STATIONARY AND THERE ARE NO FLASHING LIGHTS INSIDE THE CASE. ALSO, CHECK AT THE DECATUR CONSOLE (TO THE LEFT OF THE 450) THAT NOBODY ELSE IS USING THE ROBOT. IF THE SCREEN IS LOCKED, CONTACT A SYSTEM ADMINISTRATOR. Also, you might try using dtt to read some data (see below) before archiving it, to make sure it is not corrupted.

INTRODUCTION: the tape robot consists of 30 tape slots arranged in two vertical towers on a rotating stage, with two tape drives mounted one above the other below it. 15 of the tape slots are visible through the front panel at any time. For simplicity, the following instructions assume that you want to make one tape, using the upper tape drive, and that one of the slots 1-15 (initially visible through the front panel) is available to place a tape in.
 

TO BACK UP REDUCED DATA SETS TO A TAPE

a) Get a tape. Open the door, insert the tape window-side-up in a spare slot (I assume slot 3 for this example), close the door again (slot numbers are embossed in plastic on the back of the slot assembly).

b) Wait 1 minute for the robot to locate the new tape.

c) At the shell prompt, set up environment variables for cybernetics robot control software:

decatur {1} setenv CYRCS /usr/bin/cyrcs
decatur {2} setenv CYSTACKER /dev/cychs0

d) Start the tape robot control software, and use it to place the tape in the upper drive.

decatur {3} /usr/bin/cyrcs
CY-RCS> move slot 3 drive 1  (watch that the robot moves the tape through the window.)
CY-RCS> quit

e) Hopefully this will be obvious from the instructions below, but the top tape drive in the robot is mounted at /dev/rmt/0n. We need to position the tape head at the end of the previously used portion of the tape. Skip this instruction if you have a blank tape.

decatur {4} mt -f /dev/rmt/0n eom

f) The reduced data set writer makes directories of frames. We will make a tape archive of a directory of frames. I will assume that the frame directory we want to back up is /export/raid0/onearm/01-02-01_04:29:33.0 . First, cd to the parent directory.

decatur {5} cd /export/raid0/onearm

g) Next, make the tape archive. The blocksize for the archive is standard for LIGO reduced data set tape archives.

decatur {6} tar -c -v --blocking-factor=256 --file=/dev/rmt/0n -p 01-02-13_04:29:33.0
(note that the -p flag is optional, it surpresses tape rewind after making the archive so more archives can be added without relocating the head again.)

h) Repeat (g) for as many tape archives as you wish to make, ensuring that you use the -p flag with each tar command.

i) When the archive is written, you should get the shell prompt again. Now rewind the tape.

decatur {7} mt -f /dev/rmt/0n rewind

j) Restart the tape robot control software and move the tape back to an empty slot

decatur {8} cyrcs
CY-RCS> move drive 1 slot 3
CY-RCS> quit

k) Wait about 1 minute, then remove your tape after the robot is done. IMMEDIATELY write on the cardboard catalog in the tape case the names of the directories you tape archived. Also write the approx. size of each tape archive so people restoring the data know how much space they will need.

TO RETRIEVE DATA FROM A TAPE ARCHIVE ONTO THE RAID DISK ON DECATUR

l) Check that there is enough space on the raid disk left for your archive. Use df -k and look for a line with 'mounted on /export/raid0' at the end. If the capacity is close to 100%, you certainly need to worry. Block sizes are in kilobytes.

m) Get your tape into the drive using instructions (a-d) above on backing up reduced data sets to tape.

n) Move to the raid array top directory.

decatur {9}cd /export/raid0

o) Make a new directory to hold the restored data, and move into this directory.

decatur {10} mkdir my_restore; cd my_restore

p) If the tape isn't rewound, rewind it now.

decatur {11} mt -f /dev/rmt/0n rewind

q) I assume that you want to restore the 3rd tar file on the tape you are using. So first skip past the first two files.

decatur {12} mt -f /dev/rmt/0n fsf 2

r) Restore the third file.

decatur {13} tar -x -v --blocking-factor=256 --file=/dev/rmt/0n -p

s) Let's say you decide to restore the 2nd tar file on the tape next. Move to the start of this file. Note that the asf option in mt takes the number of tar files to be skipped before encountering the beginning of the one to be read.

decatur {14} mt -f /dev/rmt/0n asf 1

t) If you followd (s), then untar the 2nd tape archive by repeating step (r). Repeat (s) and (r) until you have restored all the desired data.

u) Follow (i-k) above to retrieve the tape from the drive.

If you have trouble with this procedure during the e3 run, get in touch with Ed Daw, ( edaw@ligo-la.caltech.edu  or see phone number list), or Szabi Marka ( smarka@ligo-la.caltech.edu), or, if both of the above fail, Daniel Sigg at hanford,  sigg_d@ligo.caltech.edu. Good luck! Ed.

Using the Diagnostic Test Tool to Look at Data
by Szabi

After all this work you might want to access and look at your(our) data. Follow the few easy steps to be able to analyze the recording:

1. Start DTT on decatur

e.g.: /export/disk0/gds/startup/main
You should see the Diagnostics window appear:

2. Set the data source pointer to your frames

e.g.:
From the  File menu, select Setup:

Then press the File button at the bottom of the Configuration window:

Select your directory and choose the first frame directory. (Directory names are based on the starting time of the test and a new directory is created in every hour of the run.YY-MM-DD_HH:MM:SS.SerialNumber)

Click OK, you should see the name of the first frame file appear in the Configuration window:

Click OK.

3. Start your favourite DTT tool and do your analysis. Note that excitations do not work! :-)

If you have any questions about RDS please contact any(all) of us at:

Ed Daw   (edaw@ligo.mit.edu)
Szabi Marka (smarka@ligo.caltech.edu)
Daniel Sigg  (dsigg@ligo.caltech.edu)