Next: Scientific Results in Press
Up: IRAM Newsletter 59 (May 2004)
Previous: Pool Poll
Most cutting edge scientific programs carried out at the IRAM
observatories require excellent weather conditions. While the service observing
mode at the Plateau de Bure interferometer allows a flexible scheduling of
demanding programs, fixed scheduled projects at the 30m telescope often
suffer from limitations due to inadequate weather conditions.
To optimize the observing efficiency given the often quickly changing
weather conditions, IRAM offers the pool observing mode at the 30m
telescope. In this mode all bolometer and some very demanding
spectroscopy programs share the allocated observing time together with
some less weather demanding spectroscopy programs.
Projects are observed according to the current weather conditions,
from the program committee and the source visibility. This way the
chances of all
demanding projects have drastically improved: the average success rate of A
ranked projects requiring good or excellent weather conditions in the
pool was 80%, compared to 20% of good weather conditions during the
allocated pool observing time (Fig. 2).
Project and weather statistics during the pool
The pool observations are conducted mostly by the astronomers who's programs
are included in the pool. They are aided by the IRAM astronomers and telescope
operators. There are typically three guest astronomers, one IRAM astronomer,
and the telescope operators at the telescope. Given the flexible use of
different instruments and the large variety of scientific programs,
pooled observations are part of IRAM's training efforts for students.
Pool observations were first tested at the IRAM 30m telescope during
2000/2001, and have now expanded to 12 weeks in the winter semester,and about
4 weeks in the summer. The winter 2003/2004 pool included almost 50 different
programs with more than 500 different targets. To manage such a pool
requires an efficient organizational structure, so that at any time the status
of a program, target priorities and weather/technical requirements can
be assessed. This motivated the observational database system
The PHP/MySQL software (http://mrt-lx3.iram.es/pooldatabase/index.php) system
was first conceived by Alexandre Beelen (IAS Paris) and Frank Bertoldi
(MPIfR Bonn) in December 2001, and subsequently developed further by
Axel Weiss, who as an IRAM astronomer serves as the pool coordinator
at the 30m telescope. All data reduction tools have been developed and
are maintained by Robert Zylka (IRAM).
The IRAM pool observation database system (ODS) is based on PHP
scripts executed from an apache web-server, which accesses a MySQL
database. The database contains detailed information on all projects
including technical and metrological requirements.
Observational information is
read from the fits data headers of each scan. This
information is easily associated with the respective projects, and
thereby allows bookkeeping, planning, data quality control, and in
connection with an external software an automated
pipeline data reduction. The system also permits data archiving and
the easy access/download of data for the users. The users
interactively enter and modify their target lists, observation
instructions, and they are able to check the state of the project and
the data quality.
The software used in the development of the ODS is part of all Linux
distributions. Its main constituents are PHP and MySQL.
PHP (recursive acronym for "PHP: Hypertext Preprocessor") is a widely-used Open
Source general-purpose C-based scripting language that is especially suited for
Web development and can be embedded into HTML (see e.g. Welling &
Thomson 2003). The PHP package contains functions which allow to
retrieve and store information in a MySQL database. MySQL is the most
popular Open Source SQL (Structured Query Language) database management system.
It provides an easy way to store and sort large amounts of information and to
give limited access to different users on the MySQL server (e.g. DuBois 2000).
MySQL in conjunction with PHP is becoming the most commonly used platform for
web-based data access and processing applications.
The database system at the IRAM 30m currently runs on PHP version 3 and
MySQL 3.23.44 on SuSE Linux distribution, but has also been installed on RedHat.
The main body of ODS is a collection of PHP scripts (the PHP engine) which
perform different kinds of operations. Information can be received and/or
stored in the MySQL database (e.g. the amount of observing time spent
on a specific project). Other PHP scripts generate files associated with the
observations such as the observing log-files, the 30m source catalogs or Xephem
files for astronomical visualization. PHP also allows to call
This way the PHP engine can parse a list of scans to an external data reduction
software, e.g. the IRAM continuum data processing software MOPSIC, and store
data information such as the achieved rms in the MySQL database.
All PHP scripts are executed by an Apache web-server. Thereby all information
can be accessed through the Internet (see Fig. 3).
Online database system flow diagram
Access is restricted by PHP
and by MySQL security applications as well as the Apache web-server configuration.
This allows to setup different user accounts in ODS with limited access to
specific information. For more information see
We have installed a new tool which allows the observer to see online the
progress of observations. The data is displayed on mrt-lx10:0.1 at the
observing desk. A MOPSIC script reduces automatically incomming data every 15
to 60 sec. The reduced observations are shown in the top left corner of the
screen. Other plots show:
These additional plots allow to judge the quality of the data and the weather
conditions during the observations, and therefore to optimize the observing strategy.
The monitor warns if the observational parameters are not valid, e.g. if the map
extent in scanning direction is too small, the pointing bolometer is not
the central one for mapping, or if an OnOff data set has an odd number of subscans,
a.s.o.. In such cases it changes the default reduction mode (if possible) or does not
reduce the data.
- the total power and the line-of-sight opacity calculated from it
(bottom left plot)
- the rms of the data (bottom centre)
- the time series of a few bolometers before (in red) and after
(in blue) the sky noise filtering (bottom right plot)
- the top right plot shows - depending on the observing mode -
either the double beam map (for pointing, focus and map), the time
series of the on-source
bolometer (for OnOffs) or of the total-power bolometers (for skydips)
Two examples of the graphical output of the monitor are shown in Fig. 4
and Fig. 5.
MOPSIC monitor (Pointing)
MOPSIC monitor (Mapping)
For a quick look at the observations ODS provides the PIs and the observers
with a pipeline reduction for all kinds of bolometer observations.
is based on MOPSIC scripts which are developed and maintained by Robert Zylka.
The pipeline reduction offers data reduction with and without skynoise
subtraction. For maps also a shift & add reduction is available. The reduction
scripts are executed by pressing the link associated with each source name as
soon as the observations for this source have been carried out. The pipeline
reduction window allows to select the scans which should be included in the
reduction. For mapping projects the rms in each map as well as the combined rms
are computed and entered in the MySQL database. The users can view the combined
and individual maps. For OnOff projects the cumulative rms of all observations
is determined and saved in the database. Detailed plots of the signal and the
rms for individual scans and the sum of all scans are provided. This allows the
PIs to continuously optimize the observing strategy during the pool.
The online reduction also allows the observers to check the data quality very
quickly and to estimate the achieved S/N. Contrary to the monitor which only
emits a warning, the pipeline
refuses to reduce data with invalid observational parameters,
e.g. odd number of subscans for OnOffs, map extent in scanning
direction too small (for no shift & add mode).
All pipeline scripts together with necessary calibration files can be obtained
via the ODS. For further informations about the pipeline
see ``About the IRAM Database'' at the ODS web page
(http://www.iram.es/IRAMES/observing/flexible/flexible.html) and the
README file available with the data reduction scripts.
DuBois, P., 2000, MySQL, New Riders Publishing
Welling, L. & Thomson, L., 2003, PHP and MySQL Web Development, Sams
Axel WEISS and Robert ZYLKA
Next: Scientific Results in Press
Up: IRAM Newsletter 59 (May 2004)
Previous: Pool Poll