data:data_analysis_manual
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
data:data_analysis_manual [2023/11/09 15:48] eric buchlin [Burn-in Effects] add O VI ion name to corrected lines |
data:data_analysis_manual [2024/03/29 14:21] (current) eric buchlin [Find data] format warning |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| <WRAP center round important 70%> | <WRAP center round important 70%> | ||
| - | This manual is very preliminary, please be patient or help adding what is missing. | + | This manual is preliminary, please be patient or help adding what is missing. |
| </WRAP> | </WRAP> | ||
| Line 22: | Line 22: | ||
| The data levels, FITS files, and headers are described in | The data levels, FITS files, and headers are described in | ||
| - | * The [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/documents_public:spice-uio-dpdd-0002-1.7-data_product_description_document_spicefits_.pdf|Data Products Description Document]] (DPDD) | + | * The [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|Data Products Description Document]] (DPDD) |
| * The LLDPDD for the Low-Latency (LL) files | * The LLDPDD for the Low-Latency (LL) files | ||
| The full SPICE data set has a DOI: [[https://doi.org/10.5270/esa-lbmdy7c|doi:10.5270/esa-lbmdy7c]]. DOIs are also attributed to [[https://spice.osups.universite-paris-saclay.fr/spice-data/|each data release]]. | The full SPICE data set has a DOI: [[https://doi.org/10.5270/esa-lbmdy7c|doi:10.5270/esa-lbmdy7c]]. DOIs are also attributed to [[https://spice.osups.universite-paris-saclay.fr/spice-data/|each data release]]. | ||
| - | A typical Level 2 SPICE filename has the form ''solo\_L2\_spice-w-exp\_20230604T073743\_V02\_184550205-019.fits'', where "w-exp" is the code, "184550205" is the unique observation ID (''SPIOBSID''), and "019" is the raster number (''RASTERNUM'') within the observation. (More information is given in the [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/documents_public:spice-uio-dpdd-0002-1.7-data_product_description_document_spicefits_.pdf|DPDD]].) When browsing the SPICE catalog and archive you will see the following codes ("file descriptors" in Solar Orbiter terminology): | + | A typical Level 2 SPICE filename has the form ''solo\_L2\_spice-w-exp\_20230604T073743\_V02\_184550205-019.fits'', where "w-exp" is the code, "184550205" is the unique observation ID (''SPIOBSID''), and "019" is the raster number (''RASTERNUM'') within the observation. (More information is given in the [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|DPDD]].) When browsing the SPICE catalog and archive you will see the following codes ("file descriptors" in Solar Orbiter terminology): |
| * **n-ras**: a raster scan with one of the narrow slits (2", 4" or 6")\\ | * **n-ras**: a raster scan with one of the narrow slits (2", 4" or 6")\\ | ||
| * **w-ras**: a raster scan with the wide slit (30")\\ | * **w-ras**: a raster scan with the wide slit (30")\\ | ||
| Line 58: | Line 58: | ||
| - Run SPICE\_CAT | - Run SPICE\_CAT | ||
| * The catalogue of all files in a specific data release is included with the data release (see below). This version of the catalogue also include additional columns for the list of windows, the file size, and a reference to the dark used for on-ground dark subtraction. | * The catalogue of all files in a specific data release is included with the data release (see below). This version of the catalogue also include additional columns for the list of windows, the file size, and a reference to the dark used for on-ground dark subtraction. | ||
| - | * How to use it in Python: a [[data:data_analysis_manual:read_catalog_python|simple script]] can be used to read it as a Python pandas DataFrame object. The same can be done using the ''Catalog'' object from the [[https://sospice.readthedocs.io/en/stable/|''sospice'' Python module]] (see examples in [[https://git.ias.u-psud.fr/spice/data-analysis-club/-/blob/main/20230627-sospice/sospice-demo.ipynb|this tutorial]]). | + | * How to use it in Python: the catalogue can be read as a ''Catalog'' object from the [[https://sospice.readthedocs.io/en/stable/|''sospice'' Python module]], and then used to find observations with specific characteristics (see examples in [[https://git.ias.u-psud.fr/spice/data-analysis-club/-/blob/main/20230627-sospice/sospice-demo.ipynb|this tutorial]]). |
| * How to use it in IDL: [[data:data_analysis_manual#the_spice_catalog|see below]] | * How to use it in IDL: [[data:data_analysis_manual#the_spice_catalog|see below]] | ||
| Line 65: | Line 65: | ||
| * All L1 and L2 files with their latest versions (at a given date) are collected in a data release | * All L1 and L2 files with their latest versions (at a given date) are collected in a data release | ||
| - | * The releases are provided under https://spice.osups.universite-paris-saclay.fr/spice-data/ | + | * The releases are provided under https://spice.osups.universite-paris-saclay.fr/spice-data/, in subdirectories named ''release-{release_tag}/'' |
| + | * The latest release tag can be obtained from the content of [[https://spice.osups.universite-paris-saclay.fr/spice-data/metadata/latest-release.txt|latest-release.txt]] | ||
| * Released files are also sent to the SOAR (see below) | * Released files are also sent to the SOAR (see below) | ||
| * The latest release is updated with new data as they arrive (and after a superficial validation step). | * The latest release is updated with new data as they arrive (and after a superficial validation step). | ||
| Line 72: | Line 73: | ||
| SOAR: | SOAR: | ||
| - | * [[https://soar.esac.esa.int/|SOAR]] ([[https://issues.cosmos.esa.int/solarorbiterwiki/display/SOSP/SOAR+(Solar+Orbiter+Archive)+-+How+to+Use|how to use it]], [[https://www.cosmos.esa.int/web/soar/home|SOAR help pages]]) | + | * [[https://soar.esac.esa.int/|SOAR]] ([[https://issues.cosmos.esa.int/solarorbiterwiki/display/SOSP/SOAR+(Solar+Orbiter+Archive)+-+How+to+Use|how to use it]], [[https://www.cosmos.esa.int/web/soar/home|SOAR help pages]], [[https://www.cosmos.esa.int/web/soar/inventory-plots|SOAR data coverage]]) |
| * SOAR has a TAP interface that can be used for automated queries ([[https://www.cosmos.esa.int/web/soar/tables-views-and-columns|documentation of available tables]]) | * SOAR has a TAP interface that can be used for automated queries ([[https://www.cosmos.esa.int/web/soar/tables-views-and-columns|documentation of available tables]]) | ||
| * The ''sunpy-soar'' [[https://github.com/sunpy/sunpy-soar/|Python module]] uses this TAP interface for queries through SunPy/Fido | * The ''sunpy-soar'' [[https://github.com/sunpy/sunpy-soar/|Python module]] uses this TAP interface for queries through SunPy/Fido | ||
| Line 80: | Line 81: | ||
| * Quicklook archive (to be developed) | * Quicklook archive (to be developed) | ||
| * [[https://github.com/ITA-Solar/solo-spice-ql/wiki/spice_xfiles|Quicklooks procedure in SSW]] | * [[https://github.com/ITA-Solar/solo-spice-ql/wiki/spice_xfiles|Quicklooks procedure in SSW]] | ||
| - | * A quicklook app (in Python) is in development at IAS. | + | * A [[https://git.ias.u-psud.fr/spice/data_quicklook|quicklook app]] (in Python) is in development at IAS. |
| * [[https://git.ias.u-psud.fr/spice/pre-analysis/|"Pre-analysis"]]: already processed quicklook images (per STP) | * [[https://git.ias.u-psud.fr/spice/pre-analysis/|"Pre-analysis"]]: already processed quicklook images (per STP) | ||
| Line 86: | Line 87: | ||
| * Released SPICE files are ingested from SOAR to the VSO ([[https://sdac.virtualsolar.org/cgi/search|search form]]) | * Released SPICE files are ingested from SOAR to the VSO ([[https://sdac.virtualsolar.org/cgi/search|search form]]) | ||
| * Accessible through IDL/SSW and Python/Fido | * Accessible through IDL/SSW and Python/Fido | ||
| + | * FIXME As of 2024-03-29, the SPICE files on VSO are old files from release 3.0 only | ||
| ===== Tips and Advice on Analyzing SPICE Data ===== | ===== Tips and Advice on Analyzing SPICE Data ===== | ||
| Line 95: | Line 97: | ||
| ==== Burn-in Effects ==== | ==== Burn-in Effects ==== | ||
| - | The SPICE detectors have microchannel plates (MCPs). Over time, the sensitivity of MCPs decreases at the locations of strong emission lines. This effect is referred to as "burn-in" and results in a flat-top or "self-reversal" to the line profile. This has already been noticed for the C III 977 line. As of November 2023, internal level-2 files are corrected for burn-in for six lines (files planned to be part of release 4.0). The effects of burn-in continue to be monitored, particularly for weaker lines for which a correction is not currently available. | + | The SPICE detectors have microchannel plates (MCPs). Over time, the sensitivity of MCPs decreases at the locations of strong emission lines. This effect is referred to as "burn-in" and results in a flat-top or "self-reversal" to the line profile. This has already been noticed for the C III 977 line. From data release 4.0, level-2 files are corrected for burn-in for six lines. The effects of burn-in continue to be monitored, particularly for weaker lines for which a correction is not currently available. |
| The lines that are being corrected (as of 2023-11) are: N IV 76.5 nm, Ne VIII 77.0 nm, Ne VIII 78.0 nm, C III 97.7 nm, Lyman-beta 102.6 nm, and O VI 103.2 nm. | The lines that are being corrected (as of 2023-11) are: N IV 76.5 nm, Ne VIII 77.0 nm, Ne VIII 78.0 nm, C III 97.7 nm, Lyman-beta 102.6 nm, and O VI 103.2 nm. | ||
| Line 225: | Line 227: | ||
| * ''EAR\_TDEL'': light travel time difference between Sun centre to Spacecraft and Sun centre to Earth (seconds) | * ''EAR\_TDEL'': light travel time difference between Sun centre to Spacecraft and Sun centre to Earth (seconds) | ||
| - | The full list of keywords is given in the [[https://spice.ias.u-psud.fr/spice-data/documents/SPICE-UIO-DPDD-0002-1.4-Data_Product_Description_Document.pdf|DPDD]] - search for "Solar ephemeris keywords". | + | The full list of keywords is given in the [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|DPDD]] - search for "Solar ephemeris keywords". |
| Users should make sure to correct the SPICE observing times for EAR\_TDEL in order to compare with observations from Earth-orbiting spacecraft. This is especially important for highly-dynamic structures such as flares or CMEs. | Users should make sure to correct the SPICE observing times for EAR\_TDEL in order to compare with observations from Earth-orbiting spacecraft. This is especially important for highly-dynamic structures such as flares or CMEs. | ||
| Line 251: | Line 253: | ||
| ==== Units and uncertainties ==== | ==== Units and uncertainties ==== | ||
| - | The data cube (returned by the SSW object method ''get\_window\_data'' or by the Python ''astropy'' or ''sunraster'' libraries) of the SPICE level-2 files contains spectral radiances in units of W/m²/s/sr/nm. The wavelength vector has units of nm. To convert a radiance in W/m²/s/sr to erg/cm²/s/sr, it is necessary to multiply by 1000. | + | The data cube (returned by the SSW object method ''get\_window\_data'' or by the Python ''astropy'' or ''sunraster'' libraries) of the SPICE level-2 files contains spectral radiances in units of W/m²/sr/nm. The wavelength vector has units of nm. To convert a radiance in W/m²/sr to erg/cm²/s/sr, it is necessary to multiply by 1000. |
| Uncertainties on the spectral radiances can be obtained with: | Uncertainties on the spectral radiances can be obtained with: | ||
data/data_analysis_manual.1699541334.txt.gz · Last modified: 2023/11/09 15:48 by eric buchlin
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
