Finding and Gathering Data

For recent projects (since late 1992, i.e. projects archived from the HP workstation at Bure), we keep on-line a list of the DAT and CDROM contents. The lists of DAT contents looking like that:

 Archive number  1
 ------------------ 
-rw-rw-rw-   1 guillote astro     308736 Oct 14 14:51 12-jan-1993-.ipb
-rw-rw-rw-   1 guillote astro     818688 Oct 14 14:51 12-jan-1993-c018.ipb
-rw-rw-rw-   1 guillote astro     496640 Oct 14 14:51 12-jan-1993-test.ipb
-rwxr-xr-x   1 guillote astro      68331 Oct 14 11:41 12-jan-1993.log-obs
-rwxr-xr-x   1 guillote astro      14901 Oct 14 11:41 12-jan-1993.log-rdi
-rwxr-xr-x   1 guillote astro       1749 Oct 14 11:41 12-jan-1993.log-red
 
 Archive number  2
 ------------------ 
-rw-rw-rw-   1 guillote astro     866816 Oct 14 14:52 13-jan-1993-.ipb
-rw-rw-rw-   1 guillote astro    31393280 Oct 14 14:52 13-jan-1993-c018.ipb
-rw-rw-rw-   1 guillote astro      59904 Oct 14 14:52 13-jan-1993-test.ipb
-rwxr-xr-x   1 guillote astro     142489 Oct 14 11:41 13-jan-1993.log-obs
-rwxr-xr-x   1 guillote astro      83965 Oct 14 11:41 13-jan-1993.log-rdi
-rwxr-xr-x   1 guillote astro       4754 Oct 14 11:41 13-jan-1993.log-red

etc...

The file naming convention is such that the data file names is composed of the date (e.g. 13-jan-1993), a minus sign (-), and the project name (e.g. c018), with a file extension .ipb. The log files (*.log-*) are interesting to have a detailed log of what happened during the observations.

For CDRom's, the naming convention is slightly different, to be compatible with ISO standards. File names are upper cases, 8 characters, 3 character extensions (IPB, OBS or RDI). The first character is the year code (A for 1990, B for 1991, etc...), the second character the month code (in hexa: 1 January, 2 March, A October, etc...), the next 2 characters the date in the month, and the last 4 the project name. CLIC automatically translates the ``long'' naming convention into ``short'' names if needed. For example H110G028.IPB translates into 10-jan-1997-g028.ipb.

Your own project can be easily located using the command

    find_project x999

where x999 is your project number. This produces an output like:

[07-feb-1997]   07-feb-1997-g067.ipb      221225472 bytes -> DAT:    198
[10-feb-1997]   10-feb-1997-g067.ipb      120860160 bytes -> DAT:    198
[20-feb-1997]           H220G067.IPB      217222144 bytes -> CDROM: 1001
[25-mar-1997]           H325G067.IPB      138128896 bytes -> CDROM: 1018
[31-mar-1997]           H331G067.IPB      226795520 bytes -> CDROM: 1020

Project requires diskspace for 879 Mbytes of raw data
Available local diskspace is 873 MBytes

File g067.list created

The command load_project creates a procedure which will download automatically the project data files, provided you mount the appropriate DAT(s) and/or CDRom(s) when asked for. The DATs and CDRoms are in a cabinet in the computer room. The list of all observations performed is available on the Web.

Only some of the above facilities are available for old projects. For older data, there are several binders in the IRAM library to help you find out what days your project was observed and which tapes contain your data. The Summary binder contains a daily list of the interferometer configuration, projects observed, and sequence numbers of the archive tapes where the data is stored. The Scanlists binders contain daily lists of each scan observed. The Tape Contents and Cassette Contents binders contain lists of all the files archived on tapes and 8mm video cassettes, respectively. Copies of the telescope operators' observing logs and on-duty astronomers' observing reports are also kept in the library.

Once the ``raw data'' is loaded, you have to created the calibration files from it. At this stage, you also want to concatenate contiguous data in a unique header file (i.e. to avoid the arbitrary splitting occuring at midnight). To do that, start CLIC, and then use the following CLIC commands

    FILE OUT B2.HPB NEW   ! Open an output file. Use long name like
                          ! W12E24N20.HPB if you prefer (but this is tedious)
    SET PROCEDURE *       ! All type of data for completeness
    SET TYPE *            ! Objects and calibrators.
    SET SCAN * *          ! Any and all scans.
    FILE IN 01-JAN-1994-X999.IPB   ! Daily archive file.
    FIND                  ! Select the scans, and
    COPY HEADER           ! Copy the HEADER only to the output file
    FILE IN 02-JAN-1994-X999.IPB   ! Now repeat this as needed.
    FIND                  ! Select the scans, and
    COPY HEADER           ! Copy the header only.
    ...
    FILE IN B2.IPB        ! Read the file just created.
    LIST IN/OUT B2.SCANLIST	      ! List what's in the file.
    LIST IN/SHORT/OUT B2.SHORTLIST    ! Long and summary format
    EXIT

In case of problems or special cases, you may have to select your data based on other critera than the project number: a range of scan numbers, for instance. With the Window environment, you can also select the ``Raw Data File'' menu to do this (almost) automatically.

Data can then be copied to optical (now or later, when calibration is finished) Mount the optical disk:

    $ load_don ipb    ! Software load the optical disk
    $ cd don_ipb      ! Move to /users/project/x999/don_ipb directory
    $ cp ~/data/*.hpb .     ! copy the *.hpb header files on the optical disk
    $ cp ~/data/*.IPB .     ! copy the *.IPB data files too

The example above assumed you have been working in a directory named /users/project/x999/data for data reduction.