User Tools

Site Tools


data:data_analysis_manual:read_catalog_python

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
data:data_analysis_manual:read_catalog_python [2022/08/27 10:01]
eric buchlin Add function to read the CSV catalog
data:data_analysis_manual:read_catalog_python [2022/09/21 13:57] (current)
gabriel pelouze fix function reading uio csv catalog (it was reading the txt catalog before)
Line 3: Line 3:
 ===== CSV catalog (new, recommended) ===== ===== CSV catalog (new, recommended) =====
  
-<file python ​read_uio_cat.py>+<file python ​read_uio_cat_csv.py>
 from pathlib import Path from pathlib import Path
 import pandas as pd import pandas as pd
Line 28: Line 28:
         Table         Table
     """​     """​
-    cat_file = Path(data_path) / "​fits"​ / "​spice_catalog.txt"+    cat_file = Path(data_path) / "​fits"​ / "​spice_catalog.csv"
     if not cat_file.exists():​     if not cat_file.exists():​
         print(f'​Error:​ Catalog file not available at {cat_file.as_posix()}'​)         print(f'​Error:​ Catalog file not available at {cat_file.as_posix()}'​)
         sys.exit(1)         sys.exit(1)
     date_columns = ['​DATE-BEG','​DATE',​ '​TIMAQUTC'​]     date_columns = ['​DATE-BEG','​DATE',​ '​TIMAQUTC'​]
-    df = pd.read_table(cat_file, parse_dates=date_columns,​ date_parser=date_parser)+    df = pd.csv(cat_file, parse_dates=date_columns,​ date_parser=date_parser)
     return df     return df
 +</​file>​
 +
 +The same applies for the catalog included in the data releases (here: release 2.0), which can simply be read by:
 +
 +<file python read_release_cat.py>​
 +import pandas as pd
 +
 +def date_parser(string):​
 +    try:
 +        return pd.Timestamp(string)
 +    except ValueError:
 +         ​return pd.NaT
 +
 +date_columns = ['​DATE-BEG','​DATE',​ '​TIMAQUTC'​]
 +cat = pd.read_csv(
 +    '​https://​spice.osups.universite-paris-saclay.fr/​spice-data/​release-2.0/​catalog.csv',​
 +    date_parser=date_parser,​
 +    parse_dates=date_columns
 +)
 +# TODO interpret the JSON included in columns `proc_steps` and `windows`.
 </​file>​ </​file>​
  
Line 40: Line 60:
 ===== Text catalog ===== ===== Text catalog =====
  
-<file python ​read_uio_cat.py>+<file python ​read_uio_cat_txt.py>
 from pathlib import Path from pathlib import Path
 import pandas as pd import pandas as pd
data/data_analysis_manual/read_catalog_python.1661587288.txt.gz · Last modified: 2022/08/27 10:01 by eric buchlin