Repository navigation
Decline structurally invalid ISO-8601 dates instead of normalizing them - #613
Open
rahul05ranjan wants to merge 3 commits into
Open
rahul05ranjan wants to merge 3 commits into
rahul05ranjan wants to merge 3 commits into
Conversation
…ng them The ISO-8601 templates match month 00 and 13-19, day 00 and 32-39, year 0000, and offsets like -99:99, and mktime() then silently normalizes the garbage into a neighboring valid date: the MySQL zero date 0000-00-00 parsed as 1899-11-30, month 13 rolled into the next January, and 9999-12-31T23:59:59-99:99 produced tm_year=10000, which cannot round-trip through datetime at all. The W3DTF handler rejects these same inputs and its tests document that expectation; the ISO-8601 handler now declines them too, so _parse_date returns None and consumers can treat the entry as having no usable date. Fixes kurtmckee#612 Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
The same normalization issue has two more faces: a day beyond its month's length (2026-11-31, 2026-02-30, 2023-02-29) silently became the first days of the next month, and an invalid time (25:61:61) rolled the timestamp into the next day. Decline both like the W3DTF handler already does; ISO 8601 end-of-day 24:00:00 timestamps and leap seconds keep their existing valid handling, and leap days (2024-02-29, 2000-02-29) are preserved. Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #612
The bug
_parse_date()silently normalizes structurally invalid dates through the ISO-8601 handler, instead of rejecting them the way the W3DTF handler does:0000-00-00is the classic MySQL zero date and shows up in Atom<updated>/<published>fields of database-backed feeds — it becomes the oldest possible timestamp, silently reordering consumers that sort by date.tm_year=10000breaksdatetime(*parsed[:6])round-trips. The W3DTF handler rejects these same inputs, andtests/test_date_parsers.pydocuments that expectation for9999-12-31T23:59:59-99:99;_parse_datejust reaches the ISO-8601 handler first, whose templates accept month00/13-19, day00/32-39, year0000, and offsets like-99:99, then lettime.mktime()normalize the garbage.The fix
After the template match, decline structurally invalid components before
mktime()runs: a 4-digit year of0, month outside1..12, day outside1..31, and timezone offsets with hours > 23 or minutes > 59. The handler returnsNone,_parse_datetries its other handlers as usual, and structurally valid dates keep their previous behavior byte for byte (including the 2-digit-year century completion and the ordinal-date forms).Verification
tests/test_date_parsers.py: 7 handler-level cases (zero date, month 00/13, day 00/39, out-of-range offset) plus 4 dispatcher-level cases pinningNone— all fail on current main (they get normalized dates) and pass with this change.Follow-up (second commit), same class: the ISO-8601 handler also normalized calendar-invalid days and invalid times, which the W3DTF handler equally rejects:
2026-11-31→ Dec 1,2026-02-30→ Mar 2,2023-02-29→ Mar 1 — a typo'd or overflow date became a different valid date;2026-11-08T25:61:61Z→ Nov 9, 02:02:01 — an invalid time rolled into the next day.Both are now declined too. Deliberately kept valid: ISO 8601 end-of-day
24:00:00Z(still next-day midnight), leap seconds (23:59:60Z), and leap days (2024-02-29,2000-02-29) — each pinned by a test. 8 more handler-level pins and 3 dispatcher-level pins were added; 7 of them fail on the first commit, all pass now.tests/test_date_parsers.py→ 96 passed.test_ill_formed.py/test_well_formed.py(they require theresponsesHTTP-mocking package, unavailable on this machine): 861 passed, 48 failed — all 48 aretest_encoding.py::test_http_encoding_file, which also needsresponsesreal HTTP mocking and fails identically on unmodified main under the same stub (verified). No other failures; system Python 3.13.5, run from a checkout without installing.