Principle

The CLIC data format was derived from the CLASS data format, and while there are some specificities unique to CLASS or CLIC , the data format itself is generic enough that in 2013, the low-level routines from both were factorized in a new CLASSIC library. The readers interested in more details are invited to refer to the CLASSIC documentation. In essence, a file is composed of a file header and one or more extension(s). The file header provides information about the file system used to encode the file, so both little endian and big endian are supported. CLASSIC files are fortran direct access, unformated binary files that are accessed record by record. The record length has an impact on the I/O depending on the file system and is stored in the file header. This should be considered for any discussion on performances.

The extensions contain an extension index, and observations. The extension index contains all the observation titles of the observations in the extension. The observation title is composed of 32 4-bytes word (see table ), allowing a quick search for observations matching any combination of these words without having to read all of the file. Observations are independant and can be read individually, which allows only the desired subset of data to be read. With large datasets of hundreds of gigabytes, it is also important to use memory as efficiently as possible so CLIC uses its own cache. The size of the memory that CLIC has access to is controled by the SPACE_CLIC logical name. It can be set with the command SIC LOGICAL SPACE_CLIC value, with value in MB, unless the unit is specified, e.g. 64GB or a fraction of the RAM size is specified, e.g. 50%. The ideal situation is when the value accessible to CLIC is larger than the size of the raw data file (which is another reason to have as compact as possible the files). This is typically the case on the data reduction machines available at IRAM. This reduces the I/O to reading the file once and writing a few calibrations, before producing large uv tables.