Skip to content

Decline structurally invalid ISO-8601 dates instead of normalizing them - #613

Open
rahul05ranjan wants to merge 3 commits into
kurtmckee:mainfrom
rahul05ranjan:fix/iso8601-structural-validation
Open

rahul05ranjan wants to merge 3 commits into
kurtmckee:mainfrom
rahul05ranjan:fix/iso8601-structural-validation

Conversation

@rahul05ranjan

@rahul05ranjan rahul05ranjan commented Nov 8, 2026 •

Copy link
Copy Markdown

Fixes #612

The bug

_parse_date() silently normalizes structurally invalid dates through the ISO-8601 handler, instead of rejecting them the way the W3DTF handler does:

>>> _parse_date("0000-00-00T00:00:00Z")
time.struct_time(tm_year=-1, tm_mon=11, tm_mday=30, ...)   # 1899-11-30!
>>> _parse_date("2026-13-01T00:00:00Z")
time.struct_time(tm_year=2027, tm_mon=1, tm_mday=1, ...)    # month 13 rolled into 2027
>>> _parse_date("9999-12-31T23:59:59-99:99")
time.struct_time(tm_year=10000, ...)   # cannot be converted to a datetime

0000-00-00 is the classic MySQL zero date and shows up in Atom <updated>/<published> fields of database-backed feeds — it becomes the oldest possible timestamp, silently reordering consumers that sort by date. tm_year=10000 breaks datetime(*parsed[:6]) round-trips. The W3DTF handler rejects these same inputs, and tests/test_date_parsers.py documents that expectation for 9999-12-31T23:59:59-99:99; _parse_date just reaches the ISO-8601 handler first, whose templates accept month 00/13-19, day 00/32-39, year 0000, and offsets like -99:99, then let time.mktime() normalize the garbage.

The fix

After the template match, decline structurally invalid components before mktime() runs: a 4-digit year of 0, month outside 1..12, day outside 1..31, and timezone offsets with hours > 23 or minutes > 59. The handler returns None, _parse_date tries its other handlers as usual, and structurally valid dates keep their previous behavior byte for byte (including the 2-digit-year century completion and the ordinal-date forms).

Verification

  • New entries in tests/test_date_parsers.py: 7 handler-level cases (zero date, month 00/13, day 00/39, out-of-range offset) plus 4 dispatcher-level cases pinning None — all fail on current main (they get normalized dates) and pass with this change.

Follow-up (second commit), same class: the ISO-8601 handler also normalized calendar-invalid days and invalid times, which the W3DTF handler equally rejects:

  • 2026-11-31 → Dec 1, 2026-02-30 → Mar 2, 2023-02-29 → Mar 1 — a typo'd or overflow date became a different valid date;
  • 2026-11-08T25:61:61Z → Nov 9, 02:02:01 — an invalid time rolled into the next day.

Both are now declined too. Deliberately kept valid: ISO 8601 end-of-day 24:00:00Z (still next-day midnight), leap seconds (23:59:60Z), and leap days (2024-02-29, 2000-02-29) — each pinned by a test. 8 more handler-level pins and 3 dispatcher-level pins were added; 7 of them fail on the first commit, all pass now.

  • tests/test_date_parsers.py → 96 passed.
  • Full suite minus test_ill_formed.py/test_well_formed.py (they require the responses HTTP-mocking package, unavailable on this machine): 861 passed, 48 failed — all 48 are test_encoding.py::test_http_encoding_file, which also needs responses real HTTP mocking and fails identically on unmodified main under the same stub (verified). No other failures; system Python 3.13.5, run from a checkout without installing.

rahul05ranjan and others added 3 commits November 8, 2026 22:05
…ng them

The ISO-8601 templates match month 00 and 13-19, day 00 and 32-39,
year 0000, and offsets like -99:99, and mktime() then silently
normalizes the garbage into a neighboring valid date: the MySQL zero
date 0000-00-00 parsed as 1899-11-30, month 13 rolled into the next
January, and 9999-12-31T23:59:59-99:99 produced tm_year=10000, which
cannot round-trip through datetime at all. The W3DTF handler rejects
these same inputs and its tests document that expectation; the ISO-8601
handler now declines them too, so _parse_date returns None and
consumers can treat the entry as having no usable date.

Fixes kurtmckee#612

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
The same normalization issue has two more faces: a day beyond its
month's length (2026-11-31, 2026-02-30, 2023-02-29) silently became the
first days of the next month, and an invalid time (25:61:61) rolled the
timestamp into the next day. Decline both like the W3DTF handler
already does; ISO 8601 end-of-day 24:00:00 timestamps and leap seconds
keep their existing valid handling, and leap days (2024-02-29,
2000-02-29) are preserved.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant