Skip to content

Sj small fixes #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Sj small fixes #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
863bff6
Add files via upload
stephjuneau Aug 6, 2025
8ec1dc1
fixed GIF display
stephjuneau Aug 6, 2025
975a337
Update README.md
stephjuneau Aug 12, 2025
1cfdac2
small fixes to tutorial
stephjuneau Oct 5, 2025
3a57375
updated RENDERED nb and date
stephjuneau Oct 6, 2025
2268207
fixes from Robert review
stephjuneau Oct 7, 2025
458ec0a
adjusted GIF for rendered to display in GitHub
stephjuneau Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified DL_SPARCL_ADASSx_Aug2025_slides.pdf
View file
Open in desktop
Binary file not shown.
58 changes: 32 additions & 26 deletions LargeScaleStructureSdssLsDesi.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"outputs": [],
"source": [
"__author__ = 'Stephanie Juneau <[email protected]>, David Herrera <[email protected]>, and the Astro Data Lab Team <[email protected]>'\n",
"__version__ = '20250804' # yyyymmdd\n",
"__version__ = '20251006' # yyyymmdd\n",
"__datasets__ = ['desi_dr1', 'ls_dr10', 'sdss_dr17']\n",
"__keywords__ = ['extragalactic', 'galaxies', 'joint query', 'sparcl', 'spectra', 'spectroscopic redshift', '3d plot']"
]
Expand Down Expand Up @@ -137,6 +137,7 @@
"import matplotlib.pyplot as plt\n",
"import matplotlib.patches as patches\n",
"from scipy.stats import binned_statistic_2d\n",
"from IPython.display import Image\n",
"from astropy.table import Table\n",
"from astropy.cosmology import Planck18 as cosmo\n",
"from astropy.convolution import convolve, Gaussian1DKernel\n",
Expand Down Expand Up @@ -187,7 +188,7 @@
"\n",
"The DESI DR1 database is called **desi_dr1** and also includes several tables. We will use the **zpix** table, which has spectroscopic information.\n",
"\n",
"**Note:** The column names and descriptions can be found directly in a notebook with `qc.schema()` or from the [Data Lab Query Interface](https://datalab.noirlab.edu/data-explorer) or using the Table Access Protocol (TAP) service with a compatible tool such as TOPCAT."
"**Note:** The column names and descriptions can be found directly in a notebook with `qc.schema()` or from the [Data Lab's Data Explorer](https://datalab.noirlab.edu/data-explorer) or using the Table Access Protocol (TAP) service with a compatible tool such as TOPCAT."
]
},
{
Expand Down Expand Up @@ -216,15 +217,14 @@
"metadata": {},
"outputs": [],
"source": [
"# sdss_dr17.specobj # SDSS DR17 specobj table\n",
"\n",
"# Write query statement (sql)\n",
"query = (f\"\"\"\n",
" SELECT S.specobjid as specobjid_dr17, S.z, S.ra as plug_ra, S.dec as plug_dec, S.class \n",
" FROM sdss_dr17.specobj AS S \n",
" WHERE S.z>0.001 AND S.ra BETWEEN {ra_min} AND {ra_max} AND S.dec BETWEEN {dec_min} AND {dec_max} \n",
" LIMIT 100000\"\"\")\n",
"\n",
"# sdss_dr17.specobj = SDSS DR17 specobj table\n",
"# S.specobjid = unique spectra identifier for SDSS (encodes plate, mjd, fiberid, run2d)\n",
"# S.z = redshift (z) from SDSS specObj table\n",
"# S.plug_ra, dec = RA, Dec of SDSS fiber from specObj table\n",
Expand Down Expand Up @@ -272,9 +272,6 @@
"metadata": {},
"outputs": [],
"source": [
"# desi_dr1.zpix # DESI DR1 zpix table (zall-pix file)\n",
"# desi_dr1.photometry # DESI DR1 photometry table\n",
"\n",
"# Write query statement (sql)\n",
"query_desi = (f\"\"\"\n",
" SELECT Z.targetid, Z.mean_fiber_ra AS ra, Z.mean_fiber_dec AS dec, Z.spectype, Z.z, \n",
Expand All @@ -285,6 +282,8 @@
" WHERE z>0.001 AND mean_fiber_ra BETWEEN {ra_min} AND {ra_max} AND mean_fiber_dec BETWEEN {dec_min} AND {dec_max} \n",
" LIMIT 100000\"\"\")\n",
"\n",
"# desi_dr1.zpix = DESI DR1 zpix table (zall-pix file)\n",
"# desi_dr1.photometry = DESI DR1 photometry table\n",
"# Z.targetid = unique spectra identifier for SDSS (encodes plate, mjd, fiberid, run2d)\n",
"# Z.mean_fiber_ra, dec = RA, Dec of DESI fiber from zpix table\n",
"# Z.z = redshift (z) from DESI zpix table\n",
Expand Down Expand Up @@ -480,7 +479,7 @@
"metadata": {},
"source": [
"## Zooming in around a Galaxy Group\n",
"Here we \"zoom in\" on the region of sky in the rectangle from the previous section's resulting plots. Within the galaxy group, we know of 4 Satellite galaxies from DESI and 1 Central galaxy from SDSS. First we confirm that they exist."
"Here we \"zoom in\" on the region of sky in the rectangle from the previous section's resulting plots. Within the galaxy group, we know of four Satellite galaxies from DESI and one Central galaxy from SDSS. First we identify these objects in their respective tables."
]
},
{
Expand All @@ -504,10 +503,10 @@
"metadata": {},
"outputs": [],
"source": [
"# 1 Central from SDSS\n",
"# Central galaxy from SDSS\n",
"ra_central = 185.4596\n",
"dec_central = 4.9611\n",
"is_central = (abs(result_sdss['plug_ra']-ra_central)<1./3600)&(abs(result_sdss['plug_dec']-dec_central)<1./3600)\n",
"is_central = (abs(result_sdss['plug_ra']-ra_central)<1./3600) & (abs(result_sdss['plug_dec']-dec_central)<1./3600)\n",
"\n",
"result_sdss[is_central]"
]
Expand Down Expand Up @@ -590,12 +589,17 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"All panels show the small region enclosed in the rectangle that we chose above. The top left for SDSS only, the top right for DESI with the selected redshift slice including both DESI and SDSS. The bottom panels are color images of the same region of the sky from the [LS Sky Viewer](https://legacysurvey.org/viewer?ra=185.480&dec=5.01&zoom=13&layer=ls-dr9). The galaxies encircled correspond to the points in black circles and blue crosses, which may belong to groups and filaments. Cyan circles mark galaxies with only a SDSS spectrum. Yellow circles have at least DESI spectra (some of them both SDSS and DESI). The bottom right image is a screenshot of the same region from the Rubin LSST First-Look image from the [LSST sky viewer](https://skyviewer.app/explorer?target=185.45957+4.96076&fov=0.84).\n",
"\n",
"#### Double-click this cell and uncomment the following to display an animated version of the image comparison\n",
"#<figure> \n",
"#<img src=\"images/LS_vs_LSST.gif\", align=\"left\", width=900> \n",
"#</figure>"
"All panels show the small region enclosed in the rectangle that we chose above. The top left for SDSS only, the top right for DESI with the selected redshift slice including both DESI and SDSS. The bottom panels are color images of the same region of the sky from the [LS Sky Viewer](https://legacysurvey.org/viewer?ra=185.480&dec=5.01&zoom=13&layer=ls-dr9). The galaxies encircled correspond to the points in black circles and blue crosses, which may belong to groups and filaments. Cyan circles mark galaxies with only a SDSS spectrum. Yellow circles have at least DESI spectra (some of them both SDSS and DESI). The bottom right image is a screenshot of the same region from the Rubin LSST First-Look image from the [LSST sky viewer](https://skyviewer.app/explorer?target=185.45957+4.96076&fov=0.84)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Animated version of the Legacy Surveys vs. LSST image comparison\n",
"Image('images/LS_vs_LSST.gif')"
]
},
{
Expand Down Expand Up @@ -709,7 +713,7 @@
"outputs": [],
"source": [
"is_desi = df_spec['data_release']=='DESI-DR1'\n",
"is_sdss = (df_spec['data_release']=='SDSS-DR16')|(df_spec['data_release']=='BOSS-DR16')\n",
"is_sdss = (df_spec['data_release']=='SDSS-DR16') | (df_spec['data_release']=='BOSS-DR16')\n",
"\n",
"print(f'N(SDSS) = {len(df_spec[is_sdss])}')\n",
"print(f'N(DESI) = {len(df_spec[is_desi])}')"
Expand All @@ -729,7 +733,7 @@
"outputs": [],
"source": [
"# Define central and group criteria for the new df_spec dataframe\n",
"is_central_spec = (abs(df_spec['ra']-ra_central)<1./3600)&(abs(df_spec['dec']-dec_central)<1./3600)\n",
"is_central_spec = (abs(df_spec['ra']-ra_central)<1./3600) & (abs(df_spec['dec']-dec_central)<1./3600)\n",
"is_group_spec = np.isin(df_spec['specid'], tids_group)\n",
"\n",
"# SDSS spectrum for central in rest-frame\n",
Expand Down Expand Up @@ -1396,18 +1400,19 @@
"sersic = result_desi['sersic']\n",
"\n",
"# plot all points in red (all redshifts)\n",
"ax1.scatter(ra, dec, s=1, color='r', marker='o', alpha=0.25)\n",
"ax1.scatter(ra, dec, s=1, color='r', marker='o', alpha=0.25, label='All redshifts')\n",
"\n",
"# overplot in black objects in narrow redshift slice\n",
"ax1.scatter(ra[rz_plot], dec[rz_plot], s=10, color='black')\n",
"ax1.scatter(ra[rz_plot], dec[rz_plot], s=10, color='black', label='DESI (z slice)')\n",
"\n",
"# Add SDSS locations in the same redshift slice\n",
"ax1.scatter(result['plug_ra'][rz], result['plug_dec'][rz], s=20, color='b', marker='x', alpha=0.6, label='SDSS')\n",
"ax1.scatter(result['plug_ra'][rz], result['plug_dec'][rz], s=20, color='b', marker='x', alpha=0.6, label='SDSS (z slice)')\n",
"\n",
"ax1.axis([xmin, xmax, ymin, ymax])\n",
"ax1.set_xlim(reversed(ax1.set_xlim())) # flip the x-axis\n",
"ax1.set_xlabel(\"RA (degrees)\", fontsize=16)\n",
"ax1.set_ylabel(\"Dec (degrees)\", fontsize=16)\n",
"ax1.legend()\n",
"\n",
"\n",
"# Second panel: plot all points in gray (all redshifts)\n",
Expand All @@ -1428,7 +1433,7 @@
"ax2.scatter(ra[rexp], dec[rexp], s=15, color='b') # blue = EXP or late-type Sersic\n",
"ax2.scatter(ra[rint], dec[rint], s=15, color='orange') # orange = Intermediate\n",
"ax2.scatter(ra[rdev], dec[rdev], s=15, color='r') # red = DEV or early-type Sersic\n",
"ax2.scatter(ra[rpsf], dec[rpsf], s=10, color='k') # red = DEV or early-type Sersic\n",
"ax2.scatter(ra[rpsf], dec[rpsf], s=10, color='k') # black = PSF (point-spread-function like)\n",
"\n",
"# Add SDSS\n",
"ra = result['plug_ra']\n",
Expand All @@ -1447,14 +1452,15 @@
"rint = np.where(((morphtype=='SER')&(sersic>=2)&(sersic<3)) & rz_plot)\n",
"\n",
"# overplot in objects in narrow redshift slice color-coded blue (EXP or Late type) or red (DEV or Early type)\n",
"ax2.scatter(ra[rexp], dec[rexp], s=15, color='b') # blue = EXP or late-type Sersic\n",
"ax2.scatter(ra[rint], dec[rint], s=15, color='orange') # orange = Intermediate\n",
"ax2.scatter(ra[rdev], dec[rdev], s=15, color='r') # red = DEV or early-type Sersic\n",
"ax2.scatter(ra[rexp], dec[rexp], s=15, color='b', label='Late Type') # blue = EXP or late-type Sersic\n",
"ax2.scatter(ra[rint], dec[rint], s=15, color='orange', label='Intermediate') # orange = Intermediate\n",
"ax2.scatter(ra[rdev], dec[rdev], s=15, color='r', label='Early Type') # red = DEV or early-type Sersic\n",
"\n",
"plt.axis([xmin, xmax, ymin, ymax])\n",
"plt.xlim(reversed(plt.xlim())) # flip the x-axis\n",
"plt.xlabel(\"RA (degrees)\", fontsize=16)\n",
"plt.ylabel(\"Dec (degrees)\", fontsize=16)\n",
"plt.legend()\n",
"plt.show()"
]
},
Expand Down Expand Up @@ -1502,7 +1508,7 @@
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python 3 (DL,Py3.10.13)",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand Down
5 changes: 3 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@
```git clone https://github.com/astro-datalab/Tutorial-ADASSx.git```

## Jupyter notebook in this repo
- LargeScaleStructureSdssLsDesi.ipynb
- LargeScaleStructureSdssLsDesi.ipynb (notebook with cleared outputs to run along)
- RENDERED_LargeScaleStructureSdssLsDesi.ipynb (pre-rendered version of the same notebook)

## Slides
- [Shared slides (will become live before event)](https://docs.google.com/presentation/d/1bLTB77aJAzdfplON880x99tf0EMJbdrcQmtINkWLyXs/edit?usp=sharing)
- [Shared slides](https://docs.google.com/presentation/d/1bLTB77aJAzdfplON880x99tf0EMJbdrcQmtINkWLyXs/edit?usp=sharing)

### Other links
- Astro Data Lab homepage: https://datalab.noirlab.edu/
Expand Down
Loading