data:data_analysis_manual:read_catalog_python
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
data:data_analysis_manual:read_catalog_python [2022/08/27 10:01] eric buchlin Add function to read the CSV catalog |
data:data_analysis_manual:read_catalog_python [2024/03/29 14:11] (current) eric buchlin Page is deprecated |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== How to read and use the UiO FITS files catalog in Python ====== | ====== How to read and use the UiO FITS files catalog in Python ====== | ||
| + | |||
| + | <note warning>These scripts are deprecated in favor of the current version, available in the [[https://sospice.readthedocs.io/en/stable/|sospice Python module]].</note> | ||
| ===== CSV catalog (new, recommended) ===== | ===== CSV catalog (new, recommended) ===== | ||
| - | <file python read_uio_cat.py> | + | <file python read_uio_cat_csv.py> |
| from pathlib import Path | from pathlib import Path | ||
| import pandas as pd | import pandas as pd | ||
| Line 28: | Line 30: | ||
| Table | Table | ||
| """ | """ | ||
| - | cat_file = Path(data_path) / "fits" / "spice_catalog.txt" | + | cat_file = Path(data_path) / "fits" / "spice_catalog.csv" |
| if not cat_file.exists(): | if not cat_file.exists(): | ||
| print(f'Error: Catalog file not available at {cat_file.as_posix()}') | print(f'Error: Catalog file not available at {cat_file.as_posix()}') | ||
| sys.exit(1) | sys.exit(1) | ||
| date_columns = ['DATE-BEG','DATE', 'TIMAQUTC'] | date_columns = ['DATE-BEG','DATE', 'TIMAQUTC'] | ||
| - | df = pd.read_table(cat_file, parse_dates=date_columns, date_parser=date_parser) | + | df = pd.read_csv(cat_file, parse_dates=date_columns, date_parser=date_parser) |
| return df | return df | ||
| + | </file> | ||
| + | |||
| + | The same applies for the catalog included in the data releases (here: release 2.0), which can simply be read by: | ||
| + | |||
| + | <file python read_release_cat.py> | ||
| + | import pandas as pd | ||
| + | |||
| + | def date_parser(string): | ||
| + | try: | ||
| + | return pd.Timestamp(string) | ||
| + | except ValueError: | ||
| + | return pd.NaT | ||
| + | |||
| + | date_columns = ['DATE-BEG','DATE', 'TIMAQUTC'] | ||
| + | cat = pd.read_csv( | ||
| + | 'https://spice.osups.universite-paris-saclay.fr/spice-data/release-2.0/catalog.csv', | ||
| + | date_parser=date_parser, | ||
| + | parse_dates=date_columns | ||
| + | ) | ||
| + | # TODO interpret the JSON included in columns `proc_steps` and `windows`. | ||
| </file> | </file> | ||
| Line 40: | Line 62: | ||
| ===== Text catalog ===== | ===== Text catalog ===== | ||
| - | <file python read_uio_cat.py> | + | <file python read_uio_cat_txt.py> |
| from pathlib import Path | from pathlib import Path | ||
| import pandas as pd | import pandas as pd | ||
data/data_analysis_manual/read_catalog_python.1661587288.txt.gz · Last modified: 2022/08/27 10:01 by eric buchlin
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
