Skip to content

Adds DST disambiguation support to the DateTime library and fixes an existing production bug with timezone conversion near DST transitions. #5275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jul 29, 2025

Conversation

taylornz
Copy link
Contributor

@taylornz taylornz commented Jul 25, 2025

Type

  • Feature
  • Bug Fix

Bug Summary

Along with the Feature Below this also squashes an existing production bug in Effect.ts DateTime which is failing to produce accurate local to UTC timezone conversations near DST change over moments.

The Effect DateTime library has an offset calculation bug in makeZonedFromAdjusted that causes incorrect UTC conversion when adjustForTimeZone: true is used. The bug affects both normal times near DST transition periods and DST transition periods.

Example: Europe/Athens DST transition day on 2025-03-30 ( transition at 3am ) (01:00 AM Athens EET = UTC+2, so UTC time is 23:00 previous day)

Node.js v24.2.0 on macOS (Darwin 24.5.0) ARM64

Temporal

  • 2025-03-30T01:00:00 Europe/Athens -> 2025-03-29T23:00:00.000Z

Effect.ts DateTime

  • 2025-03-30T01:00:00 Europe/Athens -> 2025-03-29T22:00:00.000Z
process.env.TZ = 'UTC';
import { DateTime, Effect, Option } from 'effect';

Effect.gen(function* () {
  const athensZone = DateTime.zoneUnsafeMakeNamed('Europe/Athens');
  const athensTime = DateTime.makeZoned('2025-03-30T01:00:00.000', {
    timeZone: athensZone,
    adjustForTimeZone: true,
  });

  if (Option.isNone(athensTime)) {
    console.log('Failed to create Athens time');
    return;
  }
  const utcTime = DateTime.toUtc(athensTime.value);
  console.log('Athens time (UTC):', DateTime.formatIso(utcTime));
}).pipe(Effect.runSync);

BUG: Effect DateTime is 60 minutes off

This bug is repaired as part of this feature.

Feature Summary

Adds comprehensive DST (Daylight Saving Time) disambiguation support to the DateTime API, enabling proper handling of ambiguous and gap times during timezone transitions. This implementation follows the Temporal specification and provides and maintains the Temporal default mode of 'compatible'

Problem

DST transitions create two critical scenarios when converting local times to UTC:

  1. Fall-back transitions (Repeated Times): When clocks move backward (e.g., 02:00 -> 01:00), the same local time occurs twice

  2. Spring-forward transitions (Gap Times): When clocks move forward (e.g., 02:00 -> 03:00), some local times don't exist

Without disambiguation, identical local timestamps during DST transitions cannot be correctly mapped to their intended unique UTC timestamps, breaking applications that parse time-series data across DST boundaries.

CSV Parsing Use Case

  // Parse CSV with duplicate local America/New_York timestamps during DST transition
  const csvData = [
    "2025-11-02 01:30:00,First event",
    "2025-11-02 01:30:00,Second event - one hour later!"
  ]
  // We need disambiguation to assist DateTime to map to different UTC times: 05:30 UTC and 06:30 UTC

Why Not Just "Add 1 Hour"?

A naive approach might be to simply add/subtract 1 hour for DST disambiguation, but this fails for several critical reasons:

  1. Variable DST Offsets

Not all DST transitions are exactly 1 hour:

  • Lord Howe Island, Australia: 30-minute DST shift

  • Standard: UTC+10:30, DST: UTC+11:00 (only 30 minutes!)

  • Chatham Islands, New Zealand: 45-minute DST shift

  • Standard: UTC+12:45, DST: UTC+13:45

  • Historical examples: Some regions had 20-minute, 40-minute, or 2-hour shifts

  1. Directional Ambiguity

Even with 1-hour offsets, you need to know which direction:

  • Fall-back: 01:30 happens twice - which occurrence do you want?

  • First: 01:30 EDT = 05:30 UTC

  • Second: 01:30 EST = 06:30 UTC (+1 hour in UTC, but which one?)

  • Spring-forward: 02:30 doesn't exist - synthesize before or after gap?

  • Before gap: 01:30 EST vs After gap: 03:30 EDT (2-hour local time difference!)

  1. Timezone Rule Evolution

Russia changed DST rules multiple times:

  • 2011: Permanent summer time (stopped DST)
  • 2014: Permanent winter time
  • Offset changes weren't consistently 1 hour
  1. Real-World Failure Example

Chatham Islands gap time: 02:30 doesn't exist during spring forward

  • Wrong: 02:30 + 1 hour = 03:30
  • Correct with 45-minute DST shift:
  • "earlier": 02:30 -> 01:45 (before gap)
  • "later": 02:30 -> 03:15 (after gap)

solution: Query the actual timezone database to calculate precise offset differences rather than assuming fixed increments.

Solution

Effect.ts needs disambiguation options just as the Temporal proposal library has added.

This change adds disambiguation parameter to timezone-related functions with four strategies:

export type Disambiguation = "compatible" | "earlier" | "later" | "reject"

  • "compatible": (default) Matches the new Temporal API behavior (earlier for repeated, later for gaps)
  • "earlier": Always choose earlier occurrence
  • "later": Always choose later occurrence
  • "reject": Throw error for ambiguous times, handle gaps normally

Usage Examples

Handling Repeated Times (Fall-back)

// November 2, 2025: 01:30 AM happens twice
  const timeZone = DateTime.zoneUnsafeMakeNamed("America/New_York")
  const firstOccurrence = DateTime.unsafeMakeZoned("2025-11-02 01:30:00", {
    timeZone,
    adjustForTimeZone: true,
    disambiguation: "earlier" // -> 2025-11-02T05:30:00.000Z (EDT)
  })

  const secondOccurrence = DateTime.unsafeMakeZoned("2025-11-02 01:30:00", {
    timeZone,
    adjustForTimeZone: true,
    disambiguation: "later"   // -> 2025-11-02T06:30:00.000Z (EST)
  })

Handling Gap Times (Spring-forward)

  // March 9, 2025: 02:30 AM doesn't exist
  const gapTime = DateTime.unsafeMakeZoned("2025-03-09 02:30:00", {
    timeZone,
    adjustForTimeZone: true,
    disambiguation: "later"    // -> 2025-03-09T07:30:00.000Z (after gap)
  })

Implementation Details

Core Algorithm

The implementation uses a Temporal-compatible algorithm:

  1. Query timezone database to find all possible UTC interpretations
  2. Apply disambiguation based on interpretation count:
    - 1 interpretation: Normal time (no ambiguity)
    - 2 interpretations: Ambiguous time (apply strategy)
    - 0 interpretations: Gap time (synthesize solution)

Variable DST Offset Support

Unlike naive "+1 hour" approaches, this correctly handles:

  • 30-minute DST shifts (Lord Howe Island, Australia)
  • 45-minute DST shifts (Chatham Islands, New Zealand)
  • Historical timezone changes (Russia's DST rule modifications)
  • Non-standard offset differences

API Changes

Updated Function Signatures

All timezone-related functions now accept optional disambiguation parameter:

  // makeZoned, unsafeMakeZoned
  DateTime.makeZoned(input, {
    timeZone?,
    adjustForTimeZone?,
    disambiguation?: "compatible" | "earlier" | "later" | "reject"
  })

  // setZone, setZoneNamed, setZoneOffset, etc.
  DateTime.setZone(dateTime, zone, {
    adjustForTimeZone?,
    disambiguation?: "compatible" | "earlier" | "later" | "reject"
  })
  • New explicit disambiguation available for applications requiring precise DST handling

Breaking Changes: The disambiguation default has moved to 'compatible', Relatively easy to alter the disambiguation option used.

Related

…ixes

- Add DST disambiguation parameter with 4 strategies: "compatible", "earlier", "later", "reject"
- Fix critical timezone offset calculation bug in makeZonedFromAdjusted
- Update all timezone functions to support disambiguation option
- Add comprehensive test coverage for DST scenarios
- Maintain backward compatibility with "earlier" as default strategy

Fixes timezone conversion issues where local time inputs were incorrectly
converted to UTC due to improper offset calculation.

Example fix:
Input: 01:00 Athens time on March 30, 2025
Before: 2025-03-29T22:00:00.000Z (incorrect)
After: 2025-03-29T23:00:00.000Z (correct)
@taylornz taylornz requested a review from mikearnaldi as a code owner July 25, 2025 04:55
@github-project-automation github-project-automation bot moved this to Discussion Ongoing in PR Backlog Jul 25, 2025
Copy link

changeset-bot bot commented Jul 25, 2025

🦋 Changeset detected

Latest commit: 585ec2e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 35 packages
Name Type
effect Patch
@effect/cli Patch
@effect/cluster Patch
@effect/experimental Patch
@effect/opentelemetry Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/platform Patch
@effect/printer-ansi Patch
@effect/printer Patch
@effect/rpc Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-drizzle Patch
@effect/sql-kysely Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/sql Patch
@effect/typeclass Patch
@effect/vitest Patch
@effect/workflow Patch
@effect/ai Patch
@effect/ai-amazon-bedrock Patch
@effect/ai-anthropic Patch
@effect/ai-google Patch
@effect/ai-openai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-bot effect-bot changed the base branch from main to next-minor July 25, 2025 04:55
@taylornz taylornz changed the title Adds DST disambiguation support to the DateTime library and fix a timezone offset calculation bug. Adds DST disambiguation support to the DateTime library and fixes a timezone offset calculation bug. Jul 25, 2025
@taylornz taylornz changed the title Adds DST disambiguation support to the DateTime library and fixes a timezone offset calculation bug. Adds DST disambiguation support to the DateTime library and fixes an existing production bug with timezone conversion near DST transitions. Jul 25, 2025
@mikearnaldi mikearnaldi requested review from tim-smart and fubhy July 26, 2025 16:05
Taylor H added 2 commits July 27, 2025 08:04
…est expectations. Updated test to match standard cron once mode behavior with detailed comments.
@github-project-automation github-project-automation bot moved this from Discussion Ongoing to Waiting on Author in PR Backlog Jul 27, 2025
Taylor H added 2 commits July 27, 2025 18:56
- Restore proper formatting to dateTime.ts after DST disambiguation changes
- Add comprehensive DST testing suite in packages/effect/dstProof/ (can be removed after reviewers are satisfied)
@taylornz taylornz requested review from fubhy and tim-smart July 27, 2025 14:34
fubhy
fubhy previously requested changes Jul 27, 2025
- Changed disambiguation default to compatible bringing it inline with temporal and others
- Refactored tests to use Effect.ts patterns
- Used IllegalArgumentException for a bad argument.
- Used RangeError when a disambiguation rejection occurs.
- Removed various legacy comments.
@taylornz taylornz requested a review from fubhy July 27, 2025 17:57
@fubhy
Copy link
Member

fubhy commented Jul 28, 2025

@taylornz I removed the dst proof test case generator (that's a bit overkill for inclusion here). I also took a stab at streamlining the test cases a bit more.

In doing so, I realised that it's likely possible to just create a huge test matrix for most of the test scenarios (they are all pretty much identical) instead of all the repetition we have in there atm. With a few exceptions of course.

Lmk what you think.

@fubhy
Copy link
Member

fubhy commented Jul 28, 2025

@taylornz Tim is looking into this now and will follow up. Thanks for the PR!

@taylornz
Copy link
Contributor Author

taylornz commented Jul 28, 2025

@taylornz I removed the dst proof test case generator (that's a bit overkill for inclusion here). I also took a stab at streamlining the test cases a bit more.

In doing so, I realised that it's likely possible to just create a huge test matrix for most of the test scenarios (they are all pretty much identical) instead of all the repetition we have in there atm. With a few exceptions of course.

Lmk what you think.

Good call, reorganised the DST test suite grouping up similar scenarios into one test.

@tim-smart tim-smart force-pushed the feature/datetime-dst-disambiguation branch from 3287c92 to 638718b Compare July 28, 2025 22:03
@tim-smart tim-smart force-pushed the feature/datetime-dst-disambiguation branch from 1d6c465 to 361fa5c Compare July 28, 2025 22:24
@tim-smart
Copy link
Contributor

Thanks for the PR and heads up. I have simplified the changes and optimized for the happy path to try reduce the impact on performance.

I'm not sure if the test cases cover all edge cases, so if you wanted to double check against the csv and report any issues that would be appreciated!

@taylornz
Copy link
Contributor Author

Thanks for the PR and heads up. I have simplified the changes and optimized for the happy path to try reduce the impact on performance.

I'm not sure if the test cases cover all edge cases, so if you wanted to double check against the csv and report any issues that would be appreciated!

Thanks Tim, looks good to me. Final Results: 338992 successes and 0 failures

@tim-smart tim-smart changed the base branch from next-minor to main July 29, 2025 00:10
@tim-smart tim-smart merged commit 3504555 into Effect-TS:main Jul 29, 2025
10 of 11 checks passed
@github-project-automation github-project-automation bot moved this from Waiting on Author to Done in PR Backlog Jul 29, 2025
@github-actions github-actions bot mentioned this pull request Jul 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

DateTime Named to Utc conversion bug around DST time along with an important feature that is lacking.
4 participants