data:data_analysis_manual
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
data:data_analysis_manual [2024/01/23 19:27] eric buchlin [Burn-in Effects] corrected files published in data release 4.0 |
data:data_analysis_manual [2025/01/09 11:25] (current) eric buchlin Moved Python line fitting |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| <WRAP center round important 70%> | <WRAP center round important 70%> | ||
| - | This manual is very preliminary, please be patient or help adding what is missing. | + | This manual is preliminary, please be patient or help adding what is missing. |
| </WRAP> | </WRAP> | ||
| Line 22: | Line 22: | ||
| The data levels, FITS files, and headers are described in | The data levels, FITS files, and headers are described in | ||
| - | * The [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|Data Products Description Document]] (DPDD) | + | * The [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-2.1-data_product_description_document_spicefits_.pdf|Data Products Description Document]] (DPDD) |
| * The LLDPDD for the Low-Latency (LL) files | * The LLDPDD for the Low-Latency (LL) files | ||
| Line 58: | Line 58: | ||
| - Run SPICE\_CAT | - Run SPICE\_CAT | ||
| * The catalogue of all files in a specific data release is included with the data release (see below). This version of the catalogue also include additional columns for the list of windows, the file size, and a reference to the dark used for on-ground dark subtraction. | * The catalogue of all files in a specific data release is included with the data release (see below). This version of the catalogue also include additional columns for the list of windows, the file size, and a reference to the dark used for on-ground dark subtraction. | ||
| - | * How to use it in Python: a [[data:data_analysis_manual:read_catalog_python|simple script]] can be used to read it as a Python pandas DataFrame object. The same can be done using the ''Catalog'' object from the [[https://sospice.readthedocs.io/en/stable/|''sospice'' Python module]] (see examples in [[https://git.ias.u-psud.fr/spice/data-analysis-club/-/blob/main/20230627-sospice/sospice-demo.ipynb|this tutorial]]). | + | * How to use it in Python: the catalogue can be read as a ''Catalog'' object from the [[https://sospice.readthedocs.io/en/stable/|''sospice'' Python module]], and then used to find observations with specific characteristics (see examples in [[https://git.ias.u-psud.fr/spice/data-analysis-club/-/blob/main/20230627-sospice/sospice-demo.ipynb|this tutorial]]). |
| * How to use it in IDL: [[data:data_analysis_manual#the_spice_catalog|see below]] | * How to use it in IDL: [[data:data_analysis_manual#the_spice_catalog|see below]] | ||
| Line 65: | Line 65: | ||
| * All L1 and L2 files with their latest versions (at a given date) are collected in a data release | * All L1 and L2 files with their latest versions (at a given date) are collected in a data release | ||
| - | * The releases are provided under https://spice.osups.universite-paris-saclay.fr/spice-data/ | + | * The releases are provided under https://spice.osups.universite-paris-saclay.fr/spice-data/, in subdirectories named ''release-{release_tag}/'' |
| + | * The latest release tag can be obtained from the content of [[https://spice.osups.universite-paris-saclay.fr/spice-data/metadata/latest-release.txt|latest-release.txt]] | ||
| * Released files are also sent to the SOAR (see below) | * Released files are also sent to the SOAR (see below) | ||
| * The latest release is updated with new data as they arrive (and after a superficial validation step). | * The latest release is updated with new data as they arrive (and after a superficial validation step). | ||
| Line 72: | Line 73: | ||
| SOAR: | SOAR: | ||
| - | * [[https://soar.esac.esa.int/|SOAR]] ([[https://issues.cosmos.esa.int/solarorbiterwiki/display/SOSP/SOAR+(Solar+Orbiter+Archive)+-+How+to+Use|how to use it]], [[https://www.cosmos.esa.int/web/soar/home|SOAR help pages]]) | + | * [[https://soar.esac.esa.int/|SOAR]] ([[https://issues.cosmos.esa.int/solarorbiterwiki/display/SOSP/SOAR+(Solar+Orbiter+Archive)+-+How+to+Use|how to use it]], [[https://www.cosmos.esa.int/web/soar/home|SOAR help pages]], [[https://www.cosmos.esa.int/web/soar/inventory-plots|SOAR data coverage]]) |
| * SOAR has a TAP interface that can be used for automated queries ([[https://www.cosmos.esa.int/web/soar/tables-views-and-columns|documentation of available tables]]) | * SOAR has a TAP interface that can be used for automated queries ([[https://www.cosmos.esa.int/web/soar/tables-views-and-columns|documentation of available tables]]) | ||
| * The ''sunpy-soar'' [[https://github.com/sunpy/sunpy-soar/|Python module]] uses this TAP interface for queries through SunPy/Fido | * The ''sunpy-soar'' [[https://github.com/sunpy/sunpy-soar/|Python module]] uses this TAP interface for queries through SunPy/Fido | ||
| Line 80: | Line 81: | ||
| * Quicklook archive (to be developed) | * Quicklook archive (to be developed) | ||
| * [[https://github.com/ITA-Solar/solo-spice-ql/wiki/spice_xfiles|Quicklooks procedure in SSW]] | * [[https://github.com/ITA-Solar/solo-spice-ql/wiki/spice_xfiles|Quicklooks procedure in SSW]] | ||
| - | * A quicklook app (in Python) is in development at IAS. | + | * A [[https://git.ias.u-psud.fr/spice/data_quicklook|quicklook app]] (in Python) is in development at IAS. |
| * [[https://git.ias.u-psud.fr/spice/pre-analysis/|"Pre-analysis"]]: already processed quicklook images (per STP) | * [[https://git.ias.u-psud.fr/spice/pre-analysis/|"Pre-analysis"]]: already processed quicklook images (per STP) | ||
| Line 86: | Line 87: | ||
| * Released SPICE files are ingested from SOAR to the VSO ([[https://sdac.virtualsolar.org/cgi/search|search form]]) | * Released SPICE files are ingested from SOAR to the VSO ([[https://sdac.virtualsolar.org/cgi/search|search form]]) | ||
| * Accessible through IDL/SSW and Python/Fido | * Accessible through IDL/SSW and Python/Fido | ||
| + | * FIXME As of 2024-03-29, the SPICE files on VSO are old files from release 3.0 only | ||
| ===== Tips and Advice on Analyzing SPICE Data ===== | ===== Tips and Advice on Analyzing SPICE Data ===== | ||
| Line 189: | Line 191: | ||
| | d->get\_number\_windows() | No. of spectral windows | | | d->get\_number\_windows() | No. of spectral windows | | ||
| | d->get\_window\_data(i,/load) | Extract a data window | | | d->get\_window\_data(i,/load) | Extract a data window | | ||
| - | | d->get\_lambda\_vector(i) | Get wavelength vector for the window | | + | | d->get\_lambda\_vector(i) | Get wavelength vector for the window | |
| + | | d->get\_time\_vector(i) | Get the mid-time for each exposure | | ||
| | d->get\_header(i) | Extract a data window header | | | d->get\_header(i) | Extract a data window header | | ||
| | d->get\_window\_id(i) | Get the data window ID | | | d->get\_window\_id(i) | Get the data window ID | | ||
| Line 260: | Line 263: | ||
| ==== Line fitting ==== | ==== Line fitting ==== | ||
| - | === Using EIS fitting software for SPICE === | + | For IDL Gaussian fitting software, please check: **[[data:line_fitting_idl|IDL software for emission line fitting]]**. |
| - | The routine spice\_getwindata returns a "windata" structure in a form compatible with the eis\_auto\_fit suite of routines for Hinode/EIS. Further information on these routines is given in [[https://doi.org/10.5281/zenodo.6339584 | + | Fitting in Python can be done with any fitting library, including [[https://docs.astropy.org/en/stable/modeling/|astropy.modelling]]. From astropy 7.0, there is a user-friendly way of doing the fitting in parallel ([[https://github.com/aperiosoftware/parallel-modeling-examples/blob/main/spice/Final_Tutorial.ipynb|tutorial]] by Stuart Mumford). |
| - | |EIS Software Note 15]]. The mask spectra fitting routines (Case 2 in the document) do not currently work with SPICE, however. | + | |
| - | The EIS software enable a number of operations to be performed on windata structures, as described in [[https://sohoftp.nascom.nasa.gov/solarsoft/hinode/eis/doc/eis_notes/21_WINDATA/eis_swnote_21.pdf|EIS Software Note 21]]. For example, eis\_bin\_windata allows spatial binning of the rasters. | ||
| - | If you have problems running the EIS software on SPICE data, please contact [[https://science.gsfc.nasa.gov/sed/bio/peter.r.young|Dr. Peter Young]]. | ||
| === Line window width and spectral tilt === | === Line window width and spectral tilt === | ||
data/data_analysis_manual.1706034441.txt.gz · Last modified: 2024/01/23 19:27 by eric buchlin
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
