Differences

This shows you the differences between two versions of the page.

--- spice-internal:ias-ral-operations-2018-09-2526 [2018/10/15 17:38]
claude mercier [Summary of actions]
+++ spice-internal:ias-ral-operations-2018-09-2526 [2018/10/16 10:29] (current)
eric buchlin Page to be moved to spice_internal
@@ Line 1: / Line 1: @@
-====== RAL-IAS operations meeting, Orsay, 2018-09-25P2D ======
+Page moved to [[:spice_internal:ias-ral-operations-2018-09-2526]]
-===== Participants =====
-    * IAS: Susanna, Stéphane, Claude, Miho, Éric
-    * RAL: Sunil, Steve, Alessandra, Tim
-    * UiO: Terje (Tuesday afternoon call)
-    * ESA: Dave (Wednesday afternoon call)
-===== Management =====
-==== SPICE wiki at IAS ====
-Individual accounts have been created for Tim, Sunil, Steve, and Alessandra, in addition to the existing account for Andrzej.
-These accounts have read/write access.
-==== Document management =====
-Sharing of complete documents: need a unique place to find them.
-This can be the wiki, now that everyone requiring write access has it.
-Agreement on the need to have non-final versions of documents with track change. Final versions can be in PDF.
-==== Change requests ====
-RAL presents ideas for change/configuration control.
-Usefulness of change control: informative; allow discussion on important issues; resolution of conflicts. How: create a change request (can be a Redmine issue), for things that matter (interface...). For future changes, when things will already have been frozen once.
-Configuration control: list of items, with current versions used for operations.
-On Herschel: there was a monthtly configuration control board (Steve was chair of this board; maintained list of issues that need to be discussed, with links to them). Could maybe start in the monthly telecon, and may need to move it to a specific meeting (not same people). Use Priority field in the issues raised in Redmine
-Maybe add a new field in Redmine (or just some text in description) saying that an issue might be contentious and needs to be talked about.
-Who proposes issues to be discussed: anyone. Issues for change requests are managed on wiki page with write access.
-Who will be in the configuration control board: depends on issue, goal is to reach consensus.
-This will be proposed (by who?) at next monthly telecon.
-===== Development of planning tools =====
-==== Studies and windows ====
-Studies status: Approved or not ("Created") is implemented. There will be used in timeline; we would also like "submitted" (to RAL). Rejected: like cancellation of approval, but cannot be edited anymore because has already been used.
-It is possible to see what studies were onboard at a specific date.
-Export of studies: one slot reserved for calibration, other slots can be decided by study generator, as order doesn't matter. Export should be about all studies, including the calibration studies. This should be specified in interface. The calibration studies require specific knowledge of the instrument, but restricting edition to the Planner role (like other studies) should be enough.
-Who builds the set of windows? If it is built from the set of studies only once this is completed, we have no way to check the constraints on windows while building the set of studies. Checks on windows should be done while building the set of studies (for every new study in the set).
-**Action 1: RAL to tell how to manage the spectral windows for calibration** (that should not be changed and should be always on board).
-Tim: full detector windows are special, parameters like width are ignored.
-Window sorting in study: 3 levels, by detector, width, compression type (SHC, then others). Because of bugs in FPGA, that's why complex studies with very different window types have to be run on the flight spare. Easier if planning tool exports studies with windows already sorted.
-Intensity window are like regular windows, but "collapsed" to 1 column. Its background window cannot be in the other detector. Both windows (main and background) can be of different width before collapsing. Same background window could in principle be used for several intensity windows.
-==== BOPs ====
-How a study is inserted in a BOP: use STUDY_RUN TCS, choose study (only approved ones) and specify number of repeats. This is not linked yet to the list of on-board studies; this will be checked only when inserting BOP in timeline.
-Suggestion (Sunil): avoid confusion by using different ranges for unique study ID and on-board ID.
-BOP names: is free, we could have a naming convention. Then we could choose to enforce the naming convention in the tool.
-BOP states: we discuss Claude's schema. Updated version will be included in planning tools URDs.
-BOP used at restart point: starting from assumption of power-off would be too expensive and too long. Rather assume standby mode? **Action 2: RAL to decide with MOC assumptions for restart points**.
-Now IORs are cut when there is a special BOP for "restart"; this BOP is empty for now, but in the future it could contain the TCSs for a restart point.
-Everything depends on the MIB version: if MIB version changes, need to redo all BOPs, or at least all those including TCS that have changed (if we have that information: could be a simple list of TCS), other could be put in "rejected" state. This is not implemented yet. There will be a new MIB in December/January, before next delivery of IORs. **Action 3: RAL to discuss with MOC whether we could have a list of changed TCSs for each new MIB version**
-In BOPs: now TCSs are inserted with some specified delay from the previous one; no need anymore to shift all the next ones.
-In timeline: BOPs still need to be inserted at specific absolute times.
-==== Questions from IAS ====
-**Slit selection**: if the right slit for a study is not selected at the start of study, the slit is changed by the study, this takes time and this is EMC noisy (RUN_STUDY is potentially EMC noisy!). This means that the right slit should be selected before the study. **Action 4: IAS planning tools to ensure that the right slit is selected before a study**.
-The instrument does not know about the **mosaics**, but they are in E-FECS, then can be taken into account when filling the timeline.
-BOPs: one per position, or one for whole mosaic? Depends on how we implement constraints, and whether time delays between positions change. Following principle that BOPs do not know about timelines, there should be one BOP inserted in timeline per mosaic position.
-Default **readout mode**: default is destructive readout mode (except for double exposure).
-**Dark maps** are valid for a given (exact) exposure time, for some detector temperature (which might not be maintained out of RSWs), and for some duration (but we don't know yet how long).
-A specific study allows to take a dark map and store it to some slot number (16 slots on-board, expect to use about 10).
-The relevant dark map has to be selected by a TCS in the BOP, before STUDY_RUN: the software should not allow dark map subtraction unless there is such a relevant dark map. But this is difficult to know, as the relevant dark map can be taken between the time of planning and the time when the study runs.
-The timeline tool could deduce the onboard dark maps list from history.
-There could be a check (warning) that the dark map is not too old.
-**Scanned time series** are like scans, but designed for being repeated (at STUDY_RUN level), with different constraints on parameters (like more flexibility on number of steps). Then repeats (as in timeseries) and steps (as in scan) cannot be adjusted independently. **Action 5: RAL to update constraints document for the case of scanned time series**.
-**Macros** are lists of TCs, pre-programmed on-board. This is like a BOP, but on-board. Running a macro takes 1 TC (MACRO_RUN). The planning tools have to know which macros are on-board, and they should be able to insert them in a BOP. Document in [[https://idoc-projets.ias.u-psud.fr/redmine/issues/1426|issue 1426]] states p.7 when macros are required, e.g. scan of full spectra; or multiple study runs while another parameter is adjusted (such as focus).
-Tim shows table with studies summary for NECP, and macros (now in [[https://idoc-projets.ias.u-psud.fr/redmine/issues/2094|issue 2094]].
-At least 24 macros out of the 32 are already used (switching on/off the instrument, CMS, engineering, ...) so that only about 8 are available.
-Some specific combinations of study parameters (e.g. full spectrum with scan) imply that the study is actually run by using a macro, which is created with the study set export. There is a unique relation between on-board study IDs of studies requiring a macro, and the corresponding macro IDs.
-The planning tools need to keep track of sets of macros, and when they are on-board.
-**CMS TCs** count toward the daily TC allowance. We could ask for an extra allowance because of CMS. But CMS TCs won't be used much when SPICE is busy.
-CMS TCs will have to be included in timeline produced by timeline tool.
-**SPICE TCs**: ZICxxxxxx. Some other TCs have to be included in planning (power on/off... ZCS, ZCL...). **Action 6: RAL to send IAS list of other TCs to be taken into account for planning tool** (or assess with Claude whether  set of TCs "used by SPICE" can be determined by program).
-**Calibration data** for the parameter translator could be managed at RAL (Tim, 2018-04-19 meeting), versions of calibration data should be controlled. How they will be transferred (in a machine-readable format) is to be defined.
-====  User levels and permissions ====
-As discussed at 2018-04-19 operations science meeting:
-    * For studies:
-        * Observer: read-only
-        * Planner: create/edit/delete studies
-    * For BOPs:
-        * Observer: can create BOPs, edit his own BOP
-        * Planner: can also approve/cancel approval. No need to forbid self-approval?
-==== Feedback from RAL on the demo planning tools ====
-Access to the [[http://129.175.65.241|demo planning tools]] (now [[https://spice-stable.ias.u-psud.fr/|spice-stable]] and [[https://spice-sandbox.ias.u-psud.fr/|spice-sandbox]]).
-[[https://idoc-projets.ias.u-psud.fr/redmine/issues/1937|Feedback: issue 1937]]
-==== Feedback on last IORs delivery  ====
-IAS got feedback from Dave, like letters missing in file names, easy to fix in the software (no new delivery).
-Nothing for RAL in Dave's feedback.
-===== Study parameters =====
-==== List of parameters ====
-    * Changes in Tim's document ([[https://idoc-projets.ias.u-psud.fr/redmine/issues/1426|1426]]) describing study parameters: additional/total exposure time, delay time, expert parameters for calibration studies and on-board macros.
-    * Total exposure time: both exposures start at 0, the first ends at "exposure time", the second ends at "total exposure time".
-==== Interface between planning tools and (instrument simulator, export to on-board studies) ====
-Interface definition: [[https://idoc-projets.ias.u-psud.fr/redmine/issues/1874|issue 1874]].
-Format: XML OK for everyone.
-Latest XML schema and example file: sent by Steve on 2018-07-02.
-Final version has to be agreed for both the instrument simulator and export to on-board studies, and frozen.
-Questions on schema:
-    * Total exposure time instead of additional exposure time: this is because usual constraints on exposure time apply to the total exposure time, not to the additional exposure time (that is the difference between both).
-    * Study types: scanned timeseries has to be added.
-    * Dark map: boolean, does not tell which one was used.
-Claude: engineering or raw units can be used, but must be defined clearly and frozen.
-    * Windows: defined in terms of pixels, and the interface displays the wavelengths. Maximum shift (temperature-dependent, then Sun-distance-dependent) of 10-15px, at coldest point in orbit (could affect definition of windows for out-of-RSW observations: has to be taken into account when designing windows/studies and translating pixels to wavelength).
-    * Other parameters: Steve's choice.
-In the planning tool, basically all the study parameters are shown.
-**Action 7 2018-09-28: Claude/Dimitri to send a list of the study parameters currently available in study generator** (note after meeting: this is now in [[https://idoc-projets.ias.u-psud.fr/redmine/issues/2091|issue 2091]])
-Some parameters can still be missing, e.g. some information needed by users of the resulting FITS file, but not known to the instrument, like solar target. These parameters should then also be included in the schema (they are normally already included in the planning tools URDs).
-**Action 8 2018-10-12: scientists to review and approve/complete the list**
-**Action 9 2018-10-26: Steve to issue a first official version of the XML export interface, under configuration control**
-Study numbers (for on-board table) can be included in XML for set of studies (but it seems that one cannot put several studies in a XML file following current schema).
-Terje working on DPDD. LLDPDD: Terje has sent new version today (2018-09-25).
-**Action 10: Susanna to put studies and BOPs in planning tool, with meaningful data**, in order to make good demo at the November consortium meeting.
-===== Constraints and resources =====
-==== E-FECS constraints on TCs ====
-[[https://idoc-projets.ias.u-psud.fr/redmine/issues/1519|Issue 1519]]. Table as a function of TCs or TCSs? Some decision-making is too complicated at TC level.
-    * Sunil uploaded a new file, with constraints on TCS.
-    * Constraints are given along with expected instrument modes, but we have to ignore them. "Standby +" means Standby and upper.
-    * Red lines: for emergency, not in regular timeline, then can be ignored
-    * <Allowed Range>: then depends on parameters, RAL will provide additional information.
-    * Now for EM; flight procedures will be sent to MOC by end 2018-10, for new MIB in 2019-01 (quite early compared to launch date...).
-==== Resources ====
-BOP generator will get resources information for RUN_STUDY from results of the instrument simulator run (stored with study), and for other TCS directly from RAL; could be a function of the parameters. Might also depend on starting conditions (like slit: then there is a maximum duration). **Action: RAL to define interface for resources for non-study TCS**
-    * For studies (RUN_STUDY TCS), calculated by instrument simulator and stored with study.
-    * For other TCS: to be added to E-FECS constraints table; with Python-like code in cells where functions of the parameters are needed
-    * The HK TM rate will be calculated as a function of the instrument mode.
-**Action 11 2018-10-25: Tim /Sunil to give example of formula for computing parameter-dependent
-* resources,
-* constraints wrt E-FECS**,
-then agree on format.
-==== Constraints on study parameters ====
-[[https://idoc-projets.ias.u-psud.fr/redmine/issues/1954|Issue 1954]]. New version of this document sent by Tim by e-mail, 2018-09-26T10:13 (CEST).
-    * Y-FOV should be multiple of 32, minimum 32.
-    * Full/half/quarter FOV height might need to be adjusted in flight, depending on tests. There is an offset between the two detectors and the Y FOV should take this into account.
-    *  The maximum number of Y-FOV (heights, start) pairs in a set of on-board studies is 5, this is because the Y-FOV is set by macros (there might be some flexibility to use 1-2 more macros for this, depending on availability): there is not much space for custom Y-FOV. But some adjustments can be made, e.g. build a set of studies where (half + quarter Y-FOV) is replaced by (one third Y-FOV at 2 Y positions).
-    * New rules for number of scan positions for Scanned Time Series, and Spatial Scan
-    * Maximum number of windows: was written as 2 × intensity windows + number of other windows. Actually, more accurate to count all windows as 1, including intensity and background windows.
-    * Dumbbells: Y position will depend on detector; currently it is not possible to mix (in a set of studies) studies with a given Y-FOV but with dumbbells on both detectors.
-    * Individual LL studies could be more than 0.1MB, the real constraint is the total LL volume per day < 1MB.
-    * Number of TCs for studies (User Manual 2.6.4.6) (for 150TC/d limit): section of the User Manual to be ignored, we can simply count the number of TCs in TCSs.
-    * Constraints on SPICE and heatshield doors: doors not under our control, but information in E-FECS.
-    * UM 4.10.4: what is possible during VSTP:
-        * X position for time series studies, as these don't control the mirror
-        * Y-FOV (as stated in UM): this is not possible, as implies running a different study
-    * For intensity window, mistake of document: the width is maximum 16, cannot be 32, because window collapse is obtained by binning and maximum 16 bin factor.
-**Action 12: Tim to upload ([[https://idoc-projets.ias.u-psud.fr/redmine/issues/1954|issue 1954]]) the updated version of the constraint files, completed with macros parameters and their constraints**
-Questions for Dave:
-    * VSTP: currently only one possible use case identified.
-         * Question: would a resource-negative VSTP TC be possible? Not sure...
-         * **Action 13: Susanna to send current ideas about what we want to do at VSTP**, then to be discussed at consortium meeting in 2018-11.
-         * **Action 14, 2018-11-13 (consortium meeting): David to provide further information on VSTP constraints**
-    * 150TC/d limit:
-        * This is a total, including possible additions for VSTP. Maximum number of VSTP command is given by IORorSlotNumberInSTP. The amount of TCs to be used in the VSTP has to be decided.
-        * What is the definition of a day? VSTP periods have to be aligned with STP boundaries.
-        * Do CMS TCs count? Yes. Try to make it fit. If this becomes an issue, see with ESA for solution.
-    * IOR ICD:
-        * SOC considering adding observation ID
-        * could we have also
-           * optional description of TCSs, in addition to names (mnemonic), and of formal parameters. It seems that adding description="..." to <sequence> does not validate schema.
-               * ** Action 15b: Claude to make an optional verbose description of sequences and formal parameters in IORs, **
-               * ** Action 15a: Dave to propose adding verbose description of sequence and formal parameters in IORs ICD, if refused, Claude will do 15a, make a human readable version**
-           * MIB version
-    * How is switch off scheduled (probably a question for MOC)
-    * Resources
-SOC asked ITs to sign ICDs.
-===== Summary of actions =====
-. RAL to tell how to manage the spectral windows for calibration (that should not be changed and should be always on board).
-. RAL to decide with MOC assumptions for restart points.
-. RAL to discuss with MOC on how to manage TCS changes in successive MIB versions (to invalidate only some BOPs).
-. IAS planning tools to ensure that the right slit is selected before a study.
-. RAL to update constraints document for the case of scanned time series
-Action Tim to send Excel file with macros.(?)
-. RAL to send IAS list of other TCs (not SPICE commands) to be taken into account for planning tool (or assess with Claude whether set of TCs “used by SPICE” can be determined by program)
-. (2018-09-28) Claude/Dimitri send RAL list of study parameters now available in study generator ([[https://idoc-projets.ias.u-psud.fr/redmine/issues/2091|issue 2091]])
-. (2018-10-12) Scientists to review and complete/approve the list of study parameters
-. (2018-10-26) Steve to issue a first official version of the XML export interface, under configuration control
-. Susanna to put studies and BOPs in planning tool, with meaningful data
-. 2018-10-25 Tim/Sunil to give example of formula for computing parameter-dependent
-    * resources
-    * constraints wrt E-FECS
-. Tim to upload ([[https://idoc-projets.ias.u-psud.fr/redmine/issues/1954|issue 1954]]) the updated version of the constraint files, completed with macros parameters and their constraints
-. Susanna to send current ideas about what we want to do at VSTP
-. David to provide further information on VSTP constraints.
-a. Claude to make a verbose description of sequences and formal parameters in IORs, for human reading, if not 15b.
-b. Dave to propose adding optionnal verbose description of sequence and formal parameters to IOR ICD. If refused 15a will be done.
-. IAS to update consortium meeting agenda (TVAC has been delayed for after consortium meeting, so more people than expected can come, including Martin, Andrzej, Steve, Alessandra):
-    * Calibration: Martin Caldwell for RAL.
-    * Specific discussion about VSTP
-    * ICDs (IORs...)
-    * Calibration data products
-    * Update on calibration plan

User Tools

Site Tools

Differences

Page Tools