User Tools

Site Tools


**This is an old revision of the document!**

How to read the UiO FITS files catalog in Python
from pathlib import Path
import pandas as pd
# SPICE data tree path, to be changed to your SPICE data mirror
data_path = "/archive/SOLAR-ORBITER/SPICE"      # example for IAS computing servers
def read_uio_cat():
    Read UiO text table SPICE FITS files catalog
    Example queries that can be done on the result:
    * `df[(df.LEVEL == "L2") & (df["DATE-BEG"] >= "2020-11-17") & (df["DATE-BEG"] < "2020-11-18") & (df.XPOSURE > 60.)]`
    * `df[(df.LEVEL == "L2") & (df.STUDYDES == "Standard dark for cruise phase")]`
    cat_file = Path(data_path) / "fits" / "spice_catalog.txt"
    columns = list(pd.read_csv(cat_file, nrows=0).keys())
    date_columns = ['DATE-BEG','DATE', 'TIMAQUTC']
    df = pd.read_table(cat_file, skiprows=1, names=columns, na_values="MISSING",
                    parse_dates=date_columns, warn_bad_lines=True)
    df.LEVEL = df.LEVEL.apply(lambda string: string.strip())
    df.STUDYTYP = df.STUDYTYP.apply(lambda string: string.strip())
    return df

na_values=“MISSING” replaces the string “MISSING” by NaNs, it can be removed.

Then we can read the catalog and filter it:
cat = read_uio_cat()
filtered_cat = cat[(cat['DATE-BEG'] > '2021-11-05') & (cat.LEVEL == 'L2')]

cat then contains the full catalogue (as a pandas dataframe) and filtered_cat contains a catalogue in which rows have been filtered (in this particular case) by observation date and file level.

The catalogue or filtered catalogue can be exported, e.g. to a CSV table by cat.to_csv().

data/data_analysis_manual/read_catalog_python.1636712085.txt.gz · Last modified: 2021/11/12 11:14 by eric buchlin