data:data_analysis_manual
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
data:data_analysis_manual [2023/11/09 15:04] Peter Young [Burn-in Effects] |
data:data_analysis_manual [2024/03/29 14:21] (current) eric buchlin [Find data] format warning |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| <WRAP center round important 70%> | <WRAP center round important 70%> | ||
| - | This manual is very preliminary, please be patient or help adding what is missing. | + | This manual is preliminary, please be patient or help adding what is missing. |
| </WRAP> | </WRAP> | ||
| Line 22: | Line 22: | ||
| The data levels, FITS files, and headers are described in | The data levels, FITS files, and headers are described in | ||
| - | * The [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/documents_public:spice-uio-dpdd-0002-1.7-data_product_description_document_spicefits_.pdf|Data Products Description Document]] (DPDD) | + | * The [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|Data Products Description Document]] (DPDD) |
| * The LLDPDD for the Low-Latency (LL) files | * The LLDPDD for the Low-Latency (LL) files | ||
| The full SPICE data set has a DOI: [[https://doi.org/10.5270/esa-lbmdy7c|doi:10.5270/esa-lbmdy7c]]. DOIs are also attributed to [[https://spice.osups.universite-paris-saclay.fr/spice-data/|each data release]]. | The full SPICE data set has a DOI: [[https://doi.org/10.5270/esa-lbmdy7c|doi:10.5270/esa-lbmdy7c]]. DOIs are also attributed to [[https://spice.osups.universite-paris-saclay.fr/spice-data/|each data release]]. | ||
| - | A typical Level 2 SPICE filename has the form ''solo\_L2\_spice-w-exp\_20230604T073743\_V02\_184550205-019.fits'', where "w-exp" is the code, "184550205" is the unique observation ID (''SPIOBSID''), and "019" is the raster number (''RASTERNUM'') within the observation. (More information is given in the [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/documents_public:spice-uio-dpdd-0002-1.7-data_product_description_document_spicefits_.pdf|DPDD]].) When browsing the SPICE catalog and archive you will see the following codes ("file descriptors" in Solar Orbiter terminology): | + | A typical Level 2 SPICE filename has the form ''solo\_L2\_spice-w-exp\_20230604T073743\_V02\_184550205-019.fits'', where "w-exp" is the code, "184550205" is the unique observation ID (''SPIOBSID''), and "019" is the raster number (''RASTERNUM'') within the observation. (More information is given in the [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|DPDD]].) When browsing the SPICE catalog and archive you will see the following codes ("file descriptors" in Solar Orbiter terminology): |
| * **n-ras**: a raster scan with one of the narrow slits (2", 4" or 6")\\ | * **n-ras**: a raster scan with one of the narrow slits (2", 4" or 6")\\ | ||
| * **w-ras**: a raster scan with the wide slit (30")\\ | * **w-ras**: a raster scan with the wide slit (30")\\ | ||
| Line 58: | Line 58: | ||
| - Run SPICE\_CAT | - Run SPICE\_CAT | ||
| * The catalogue of all files in a specific data release is included with the data release (see below). This version of the catalogue also include additional columns for the list of windows, the file size, and a reference to the dark used for on-ground dark subtraction. | * The catalogue of all files in a specific data release is included with the data release (see below). This version of the catalogue also include additional columns for the list of windows, the file size, and a reference to the dark used for on-ground dark subtraction. | ||
| - | * How to use it in Python: a [[data:data_analysis_manual:read_catalog_python|simple script]] can be used to read it as a Python pandas DataFrame object. The same can be done using the ''Catalog'' object from the [[https://sospice.readthedocs.io/en/stable/|''sospice'' Python module]] (see examples in [[https://git.ias.u-psud.fr/spice/data-analysis-club/-/blob/main/20230627-sospice/sospice-demo.ipynb|this tutorial]]). | + | * How to use it in Python: the catalogue can be read as a ''Catalog'' object from the [[https://sospice.readthedocs.io/en/stable/|''sospice'' Python module]], and then used to find observations with specific characteristics (see examples in [[https://git.ias.u-psud.fr/spice/data-analysis-club/-/blob/main/20230627-sospice/sospice-demo.ipynb|this tutorial]]). |
| * How to use it in IDL: [[data:data_analysis_manual#the_spice_catalog|see below]] | * How to use it in IDL: [[data:data_analysis_manual#the_spice_catalog|see below]] | ||
| Line 65: | Line 65: | ||
| * All L1 and L2 files with their latest versions (at a given date) are collected in a data release | * All L1 and L2 files with their latest versions (at a given date) are collected in a data release | ||
| - | * The releases are provided under https://spice.osups.universite-paris-saclay.fr/spice-data/ | + | * The releases are provided under https://spice.osups.universite-paris-saclay.fr/spice-data/, in subdirectories named ''release-{release_tag}/'' |
| + | * The latest release tag can be obtained from the content of [[https://spice.osups.universite-paris-saclay.fr/spice-data/metadata/latest-release.txt|latest-release.txt]] | ||
| * Released files are also sent to the SOAR (see below) | * Released files are also sent to the SOAR (see below) | ||
| * The latest release is updated with new data as they arrive (and after a superficial validation step). | * The latest release is updated with new data as they arrive (and after a superficial validation step). | ||
| Line 72: | Line 73: | ||
| SOAR: | SOAR: | ||
| - | * [[https://soar.esac.esa.int/|SOAR]] ([[https://issues.cosmos.esa.int/solarorbiterwiki/display/SOSP/SOAR+(Solar+Orbiter+Archive)+-+How+to+Use|how to use it]], [[https://www.cosmos.esa.int/web/soar/home|SOAR help pages]]) | + | * [[https://soar.esac.esa.int/|SOAR]] ([[https://issues.cosmos.esa.int/solarorbiterwiki/display/SOSP/SOAR+(Solar+Orbiter+Archive)+-+How+to+Use|how to use it]], [[https://www.cosmos.esa.int/web/soar/home|SOAR help pages]], [[https://www.cosmos.esa.int/web/soar/inventory-plots|SOAR data coverage]]) |
| * SOAR has a TAP interface that can be used for automated queries ([[https://www.cosmos.esa.int/web/soar/tables-views-and-columns|documentation of available tables]]) | * SOAR has a TAP interface that can be used for automated queries ([[https://www.cosmos.esa.int/web/soar/tables-views-and-columns|documentation of available tables]]) | ||
| * The ''sunpy-soar'' [[https://github.com/sunpy/sunpy-soar/|Python module]] uses this TAP interface for queries through SunPy/Fido | * The ''sunpy-soar'' [[https://github.com/sunpy/sunpy-soar/|Python module]] uses this TAP interface for queries through SunPy/Fido | ||
| Line 80: | Line 81: | ||
| * Quicklook archive (to be developed) | * Quicklook archive (to be developed) | ||
| * [[https://github.com/ITA-Solar/solo-spice-ql/wiki/spice_xfiles|Quicklooks procedure in SSW]] | * [[https://github.com/ITA-Solar/solo-spice-ql/wiki/spice_xfiles|Quicklooks procedure in SSW]] | ||
| - | * A quicklook app (in Python) is in development at IAS. | + | * A [[https://git.ias.u-psud.fr/spice/data_quicklook|quicklook app]] (in Python) is in development at IAS. |
| * [[https://git.ias.u-psud.fr/spice/pre-analysis/|"Pre-analysis"]]: already processed quicklook images (per STP) | * [[https://git.ias.u-psud.fr/spice/pre-analysis/|"Pre-analysis"]]: already processed quicklook images (per STP) | ||
| Line 86: | Line 87: | ||
| * Released SPICE files are ingested from SOAR to the VSO ([[https://sdac.virtualsolar.org/cgi/search|search form]]) | * Released SPICE files are ingested from SOAR to the VSO ([[https://sdac.virtualsolar.org/cgi/search|search form]]) | ||
| * Accessible through IDL/SSW and Python/Fido | * Accessible through IDL/SSW and Python/Fido | ||
| + | * FIXME As of 2024-03-29, the SPICE files on VSO are old files from release 3.0 only | ||
| ===== Tips and Advice on Analyzing SPICE Data ===== | ===== Tips and Advice on Analyzing SPICE Data ===== | ||
| Line 95: | Line 97: | ||
| ==== Burn-in Effects ==== | ==== Burn-in Effects ==== | ||
| - | The SPICE detectors have microchannel plates (MCPs). Over time, the sensitivity of MCPs decreases at the locations of strong emission lines. This effect is referred to as "burn-in" and results in a flat-top or "self-reversal" to the line profile. This has already been noticed for the C III 977 line. As of November 2023, internal level-2 files are corrected for burn-in for six lines (files planned to be part of release 4.0). The effects of burn-in continue to be monitored, particularly for weaker lines for which a correction is not currently available. | + | The SPICE detectors have microchannel plates (MCPs). Over time, the sensitivity of MCPs decreases at the locations of strong emission lines. This effect is referred to as "burn-in" and results in a flat-top or "self-reversal" to the line profile. This has already been noticed for the C III 977 line. From data release 4.0, level-2 files are corrected for burn-in for six lines. The effects of burn-in continue to be monitored, particularly for weaker lines for which a correction is not currently available. |
| - | The lines that are being corrected (as of Nov 2023) are: N IV 76.5 nm, Ne VIII 77.0 nm, Ne VIII 78.0 nm, C III 97.7 nm, Lyman-beta 102.6 nm and 103.2 nm. | + | The lines that are being corrected (as of 2023-11) are: N IV 76.5 nm, Ne VIII 77.0 nm, Ne VIII 78.0 nm, C III 97.7 nm, Lyman-beta 102.6 nm, and O VI 103.2 nm. |
| ==== Flare lines ==== | ==== Flare lines ==== | ||
| Line 225: | Line 227: | ||
| * ''EAR\_TDEL'': light travel time difference between Sun centre to Spacecraft and Sun centre to Earth (seconds) | * ''EAR\_TDEL'': light travel time difference between Sun centre to Spacecraft and Sun centre to Earth (seconds) | ||
| - | The full list of keywords is given in the [[https://spice.ias.u-psud.fr/spice-data/documents/SPICE-UIO-DPDD-0002-1.4-Data_Product_Description_Document.pdf|DPDD]] - search for "Solar ephemeris keywords". | + | The full list of keywords is given in the [[https://spice-wiki.ias.u-psud.fr/lib/exe/fetch.php/public:spice-uio-dpdd-0002-1.8-data_product_description_document_spicefits_.pdf|DPDD]] - search for "Solar ephemeris keywords". |
| Users should make sure to correct the SPICE observing times for EAR\_TDEL in order to compare with observations from Earth-orbiting spacecraft. This is especially important for highly-dynamic structures such as flares or CMEs. | Users should make sure to correct the SPICE observing times for EAR\_TDEL in order to compare with observations from Earth-orbiting spacecraft. This is especially important for highly-dynamic structures such as flares or CMEs. | ||
| Line 247: | Line 249: | ||
| ==== Change of radiometric calibration with time ==== | ==== Change of radiometric calibration with time ==== | ||
| - | The SPICE calibration pipeline applies the same radiometric calibration to all of the SPICE data, irrespective of the time it was taken. However, the SPICE team has found that the instrument sensitivity decreased by a factor 2-3 in the first two years of the mission and has remained constant since then. The SPICE team is characterizing these changes and they will be incorporated into the SPICE pipeline. Until then, users should be careful about comparing emission line radiances at different times during the mission. | + | The SPICE instrument sensitivity has decreased by a factor 2-3 in the first two years of the mission and has remained approximately constant since then. The SPICE team have characterized these changes and, as of 8-Nov-2023, the level-2 SPICE files incorporate the sensitivity degradation. |
| ==== Units and uncertainties ==== | ==== Units and uncertainties ==== | ||
| - | The data cube (returned by the SSW object method ''get\_window\_data'' or by the Python ''astropy'' or ''sunraster'' libraries) of the SPICE level-2 files contains spectral radiances in units of W/m²/s/sr/nm. The wavelength vector has units of nm. To convert a radiance in W/m²/s/sr to erg/cm²/s/sr, it is necessary to multiply by 1000. | + | The data cube (returned by the SSW object method ''get\_window\_data'' or by the Python ''astropy'' or ''sunraster'' libraries) of the SPICE level-2 files contains spectral radiances in units of W/m²/sr/nm. The wavelength vector has units of nm. To convert a radiance in W/m²/sr to erg/cm²/s/sr, it is necessary to multiply by 1000. |
| Uncertainties on the spectral radiances can be obtained with: | Uncertainties on the spectral radiances can be obtained with: | ||
data/data_analysis_manual.1699538667.txt.gz · Last modified: 2023/11/09 15:04 by Peter Young
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
