FAIL SUBSETDATA* tests NLCOMP

[slevis-lmwg]

Brief summary of bug

Originally posted in #3422 and briefly mentioned in #3453. The tests fail only in NLCOMP.

FAIL SUBSETDATAPOINT_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default NLCOMP
FAIL SUBSETDATAREGION_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default NLCOMP

From the TestStatus.log of SUBSETDATAPOINT_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default

2025-08-26 16:59:37: NLCOMP
Comparison failed between '/glade/derecho/scratch/slevis/tests_0826-152952de/SUBSETDATAPOINT_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default.GC.0826-152952de_int/CaseDocs/lnd_in' with '/glade/campaign/cgd/tss/ctsm_baselines/ctsm5.3.072/SUBSETDATAPOINT_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default/CaseDocs/lnd_in'
  BASE: fsurdat = surfdata_-92.798085_45.402252_hist_2000_78pfts_c250825.nc'
  COMP: fsurdat = surfdata_-92.798085_45.402252_hist_2000_78pfts_c250826.nc'
  BASE: flanduse_timeseries = landuse.timeseries_-92.798085_45.402252_SSP2-4.5_1850-2100_78pfts_c250825.nc'
  COMP: flanduse_timeseries = landuse.timeseries_-92.798085_45.402252_SSP2-4.5_1850-2100_78pfts_c250826.nc'

From the TestStatus.log of SUBSETDATAREGION_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default

2025-08-26 17:02:41: NLCOMP
Comparison failed between '/glade/derecho/scratch/slevis/tests_0826-152952de/SUBSETDATAREGION_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default.GC.0826-152952de_int/CaseDocs/lnd_in' with '/glade/campaign/cgd/tss/ctsm_baselines/ctsm5.3.072/SUBSETDATAREGION_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default/CaseDocs/lnd_in'
  BASE: fsurdat = surfdata_291.0-293.0_-9.0--7.0_hist_2000_78pfts_c250825.nc'
  COMP: fsurdat = surfdata_291.0-293.0_-9.0--7.0_hist_2000_78pfts_c250826.nc'
  BASE: flanduse_timeseries = landuse.timeseries_291.0-293.0_-9.0--7.0_SSP2-4.5_1850-2100_78pfts_c250825.nc'
  COMP: flanduse_timeseries = landuse.timeseries_291.0-293.0_-9.0--7.0_SSP2-4.5_1850-2100_78pfts_c250826.nc'
Comparison failed between '/glade/derecho/scratch/slevis/tests_0826-152952de/SUBSETDATAREGION_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default.GC.0826-152952de_int/CaseDocs/datm_in' with '/glade/campaign/cgd/tss/ctsm_baselines/ctsm5.3.072/SUBSETDATAREGION_Ld5_D_Mmpi-serial.CLM_USRDAT.I2000Clm60BgcCropCrujra.derecho_intel.clm-default/CaseDocs/datm_in'
Differences in namelist 'datm_nml':
  BASE: model_maskfile = domain.lnd.fv0.9x1.25_gx1v7_291.0-293.0_-9.0--7.0_c250825_ESMF_UNSTRUCTURED_MESH.nc'
  COMP: model_maskfile = domain.lnd.fv0.9x1.25_gx1v7_291.0-293.0_-9.0--7.0_c250826_ESMF_UNSTRUCTURED_MESH.nc'
  BASE: model_meshfile = domain.lnd.fv0.9x1.25_gx1v7_291.0-293.0_-9.0--7.0_c250825_ESMF_UNSTRUCTURED_MESH.nc'
  COMP: model_meshfile = domain.lnd.fv0.9x1.25_gx1v7_291.0-293.0_-9.0--7.0_c250826_ESMF_UNSTRUCTURED_MESH.nc'

General bug information

CTSM version you are using: The problem first appears in ctsm5.3.069

Does this bug cause significantly incorrect results in the model's science? No

[samsrabin]

This will require a CIME update to fix. Not marking as external because it will require some work on our side once we bring in the CIME fix.

[samsrabin]

PR at ESMCI/cime#4846

[ekluzek]

In order to remove files with creation dates in them, we can use the subsetdata option "--out-surface".

[ekluzek]

As an aside it looks like there should be a similar option to subset for landuse timeseries datasets in order to name them explicitly.