Skip to content

Conversation

@tieneupin
Copy link
Contributor

@tieneupin tieneupin commented Oct 31, 2025

Adds logic to register incoming CLEM data to both ISPyB and its mirror tables on Murfey.

  • Adds new columns to the CLEMImageSeries table to keep track of image metadata needed for the ISPyB insert
  • Adds logic to create a data collection group entry for CLEM data in ISPyB
  • Adds logic to register grid overview datasets as atlases in ISPyB
  • Adds logic to register positional datasets as grid squares in ISPyB
  • Migrates hard-coded image scaling logic in some of the ISPyB insertion functions into the workflows that scale them as such, allowing the insertion function to be more universally used

* Linked 'CLEMImageSeries' to 'DataCollectionGroup' and 'GridSquare'
* Tried modern type hinting notation in 'CLEMImageSeries' table
* Added new columns to 'CLEMImageSeries' to keep track of image search string (for globbing images with) and type of dataset (i.e., atlas or grid square)
* Removed 'composite_image' column from 'CLEMImageSeries'
@codecov
Copy link

codecov bot commented Oct 31, 2025

Codecov Report

❌ Patch coverage is 71.20419% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 38.24%. Comparing base (f91762e) to head (3abe4f6).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #689      +/-   ##
==========================================
+ Coverage   36.94%   38.24%   +1.29%     
==========================================
  Files          97       97              
  Lines       11459    11593     +134     
  Branches     1510     1528      +18     
==========================================
+ Hits         4234     4434     +200     
+ Misses       7063     6964      -99     
- Partials      162      195      +33     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…of image shape, pixel size, and spatial location on grid
… scaled down information about the grid squares being registered
…reprocess' workflow instead, so that the grid square registration and updating functions just register the values as-is
@tieneupin tieneupin changed the title Register clem data collection Register CLEM data collection Nov 4, 2025
@tieneupin tieneupin changed the title Register CLEM data collection Register CLEM data in ISPyB Nov 5, 2025
@tieneupin tieneupin self-assigned this Nov 5, 2025
@tieneupin tieneupin added enhancement New feature or request server Relates to the server component cryo-clem Part of the cryo-CLEM pipeline extension labels Nov 5, 2025
@tieneupin tieneupin marked this pull request as ready for review November 5, 2025 11:33
@tieneupin tieneupin force-pushed the register-clem-data-collection branch 2 times, most recently from 0b55f57 to 2c1ac4f Compare November 5, 2025 17:02
@tieneupin tieneupin force-pushed the register-clem-data-collection branch 3 times, most recently from c77516e to d206593 Compare November 5, 2025 18:07
@tieneupin tieneupin force-pushed the register-clem-data-collection branch 6 times, most recently from d12a674 to 6bb3e69 Compare November 5, 2025 19:26
Copy link
Contributor

@stephen-riggs stephen-riggs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All seems fine, as long as you're happy it works

pixels_x: Optional[int] = Field(default=None)
pixels_y: Optional[int] = Field(default=None)
pixel_size: Optional[float] = Field(default=None)
units: Optional[str] = Field(default=None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need the units in the database? I can't see they are used anywhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a valid point. I included it as a just-in-case, so I'll revisit this in the future if no new workflows end up using this column.

@tieneupin tieneupin force-pushed the register-clem-data-collection branch from 6bb3e69 to a62941c Compare November 6, 2025 09:38
@tieneupin tieneupin force-pushed the register-clem-data-collection branch 8 times, most recently from 7c7cce2 to 31b26f4 Compare November 6, 2025 14:20
@tieneupin tieneupin force-pushed the register-clem-data-collection branch from 7917b40 to 72b2b1f Compare November 6, 2025 14:36
@tieneupin tieneupin force-pushed the register-clem-data-collection branch from 5c988b1 to df1f732 Compare November 6, 2025 14:52
@tieneupin tieneupin force-pushed the register-clem-data-collection branch from ba2d1d2 to 3abe4f6 Compare November 6, 2025 15:50
@tieneupin tieneupin merged commit 2e38cb2 into main Nov 6, 2025
17 checks passed
@tieneupin tieneupin deleted the register-clem-data-collection branch November 6, 2025 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cryo-clem Part of the cryo-CLEM pipeline extension enhancement New feature or request server Relates to the server component

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants