Skip to content

Verify updated python and library version + environments produce identical data #434

Closed
1 of 1 issue completed

Description

@samaloney
  • Double check E2E tests
  • Process all TM and do some manual checks of current to new data
  • Update the E2E environment

Activity

  1. added this to the v1.6.0 milestone on Sep 30, 2025
  2. samaloney commented on Oct 21, 2025

    @samaloney
    CollaboratorAuthor

    Moved to #436

  3. samaloney commented on Oct 21, 2025

    @samaloney
    CollaboratorAuthor

    Yea with the warning configured (need to donwgrade to earlier numpy e.g. 2.1

    /Users/sm/Projects/STIXCore/stixcore/products/level0/scienceL0.py:379: UserWarning: result dtype changed due to the removal of value-based promotion from NumPy. Changed from int64 to uint8.
    
  4. samaloney commented on Nov 10, 2025

    @samaloney
    CollaboratorAuthor

    Some issues found during reprocessing:

    2025-11-08 05:29:07,975 Error processing file /data/stix/out/test/p311_comp_test/L0/1/1/solo_L0_stix-tc-verify-1_0792806400_V02.fits
    Traceback (most recent call last):
      File "/home/nicky/STIXCore/stixcore/processing/L0toL1.py", line 111, in process_type
        all_files.extend(processor.write_fits(l1))
                         ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/stixcore/io/product_processors/fits/processors.py", line 865, in write_fits
        prod = prod + existing
               ~~~~~^~~~~~~~~~
      File "/home/nicky/STIXCore/stixcore/products/product.py", line 678, in __add__
        row["control_index"] = nid
        ~~~^^^^^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/venv/lib/python3.11/site-packages/astropy/table/row.py", line 71, in __setitem__
        self._table.columns[item][self._index] = val
        ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/venv/lib/python3.11/site-packages/astropy/table/column.py", line 1373, in __setitem__
        self.data[index] = value
        ~~~~~~~~~^^^^^^^
    OverflowError: Python integer 256 out of bounds for uint8
    
    • 3. Single error in processing QL background
    2025-11-08 00:51:34,688 Error processing file /data/stix/out/test/p311_comp_test/LB/21/6/31/solo_LB_stix-21-6-31_0798422400_V02.fits for 21, 6, 31
    Traceback (most recent call last):
      File "/home/nicky/STIXCore/stixcore/processing/LBtoL0.py", line 100, in process_tm_type
        level0 = tmp.from_levelb(levelb, parent=file.name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/stixcore/products/level0/quicklookL0.py", line 259, in from_levelb
        time, duration, scet_timerange = control._get_time()
                                         ^^^^^^^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/stixcore/products/product.py", line 366, in _get_time
        bases = SCETime(base_coarse, base_fine)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/stixcore/time/datetime.py", line 373, in __init__
        raise ValueError("Coarse and fine times must be integers")
    ValueError: Coarse and fine times must be integers
    2025-11-08 00:51:34,711 Coarse and fine times must be integers
    
    • 4. Multiple error processing QL spectra (this are epexted need to check the run settings)
    2025-11-08 00:56:46,373 Error processing file /data/stix/out/test/p311_comp_test/LB/21/6/32/solo_LB_stix-21-6-32_0751334400_V02.fits for 21, 6, 32
    Traceback (most recent call last):
      File "/home/nicky/STIXCore/stixcore/products/level0/quicklookL0.py", line 384, in from_levelb
        control_indices = control_indices.reshape(-1, 32)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ValueError: cannot reshape array of size 2572 into shape (32)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/nicky/STIXCore/stixcore/processing/LBtoL0.py", line 100, in process_tm_type
        level0 = tmp.from_levelb(levelb, parent=file.name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/nicky/STIXCore/stixcore/products/level0/quicklookL0.py", line 387, in from_levelb
        raise QLSpectraReshapeError(str(e))
    stixcore.products.level0.quicklookL0.QLSpectraReshapeError: cannot reshape array of size 2572 into shape (32)
    2025-11-08 00:56:46,374 cannot reshape array of size 2572 into shape (32)
    
  5. nicHoch commented on Nov 24, 2025

    @nicHoch
    Collaborator

    @samaloney can this be closed are all findings closed or addressed by a new issue?

  6. modified the milestones: v1.6.0, v1.7.0 on Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions