Skip to content

Conversation

@troyraen
Copy link
Contributor

The public website hasn't been updated since Oct. 2 because GHA is running out of memory. Hopefully this PR will fix it by excluding wise-allwise-catalog-demo.md from execution. Also includes a couple other things I noticed while tracking this down.

  • Remove wise-allwise-catalog-demo from GHA execution. It uses about 10G of memory which doesn't leave enough for other notebooks.
  • Remove obsolete .gitkeep.
  • Comment out table row count in cosmoDC2_TAP_access.md to help cosmodc2 TAP: occationally takes a very long time to run, revise if it may be scaled back somehow #144. I'm hopeful this will resolve the issue. If it doesn't, the other cell that takes a long time is the following. Perhaps the query could be restricted by adding a constraint on a quality flag or magnitudes or by reducing the redshift range.
service = vo.dal.TAPService("https://irsa.ipac.caltech.edu/TAP")
adql = f"SELECT Mag_true_r_sdss_z0, Mag_true_g_sdss_z0, redshift FROM {tablename} WHERE redshift > 0.5 and redshift < 0.54"
results = service.run_async(adql)

@troyraen troyraen requested a review from bsipocz October 25, 2025 01:33
@troyraen troyraen added maintainance infrastructure Infrastructure related issues/PRs. html rendering / skip testing Rendering related issues/PRs. Skips tests in PRs. GHA buildhtml Enable extra buildhtml job on GHA content: parquet Content related issues/PRs for notebooks with parquet/HATS relevance labels Oct 25, 2025
@troyraen troyraen marked this pull request as ready for review October 25, 2025 01:34
@troyraen
Copy link
Contributor Author

The first GHA "Buildhtml testing" job ran for about 2h 45m but it was stuck and the logs just kept repeating:

Connection lost, reconnecting in 0 seconds.
Starting WebSocket: ws://localhost:8888/api/kernels/b87f71e4-b77f-418c-b345-10a4a4cbff9f

All notebooks except for sia_cosmos.md had been "Built". Not sure what happened there.

I canceled and restarted the job and it finished successfully in about 12m. So this is ready to go as far as I can tell.

@bsipocz bsipocz merged commit c6549e3 into main Oct 25, 2025
5 of 6 checks passed
@bsipocz bsipocz deleted the raen/reduce-gha-memory-usage branch October 25, 2025 05:42
@bsipocz
Copy link
Member

bsipocz commented Oct 25, 2025

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

content: parquet Content related issues/PRs for notebooks with parquet/HATS relevance GHA buildhtml Enable extra buildhtml job on GHA html rendering / skip testing Rendering related issues/PRs. Skips tests in PRs. infrastructure Infrastructure related issues/PRs. maintainance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants