Demonstrate JSON Schema as source of truth for web-features data #2990

ddbeck · 2025-05-21T16:18:47Z

This PR demonstrates switching from generating JSON Schema to authoring a JSON Schema. This is a prerequisite for #91 and fixes #2722. I'll self-review to point out some interesting things.

This NOT ready to merge. Don't merge it! If we want to go down this route, then there are a few tasks remaining to get this into a mergeable state:

Remove old schema
Remove old generation
Add a test that the checked-in quicktype generated types file matches what we'd expect to be generated from JSON Schema (much like we do for the types → JSON Schema process today)

ddbeck · 2025-05-21T16:21:24Z

new.quicktype.ts

@@ -0,0 +1,208 @@
+/**


This file is generated by quicktype from the new, hand-authored JSON Schema. The generation is OK for now but if we add something like redirects as described in #91, it has a few inconveniences. We're going to insulate ourselves from quicktype with the next TS module.

ddbeck · 2025-05-21T16:29:41Z

newtypes.ts

+export interface WebFeaturesData
+  extends Pick<QuicktypeWebFeaturesData, "browsers" | "groups" | "snapshots"> {
+  features: { [key: string]: FeatureData };
+}
+
+export type FeatureData = Required<
+  Pick<
+    QuicktypeMonolithicFeatureData,
+    "description_html" | "description" | "name" | "spec" | "status"
+  >
+> &
+  Partial<
+    Pick<
+      QuicktypeMonolithicFeatureData,
+      "caniuse" | "compat_features" | "discouraged"
+    >
+  >;


This is a preview of what's coming when we add redirects. TypeScript and JSON Schema's type systems aren't strictly equivalent, so quicktype will generate correct but overbroad types. In this module, I'll pluck out some nuances (and badly generated names) that quicktype doesn't handle very well and clean them up by extending the generated types. While it's a bit weird looking, the benefit is that we can't completely diverge from the underlying JSON Schema.

Do the TypeScript error messages if you get types wrong end up looking completely bizarre with this layering?

They're not quite as nice as plain, from-scratch types but they're probably above-average in terms of informativeness in type error messages. I've seen some { … } & { … } & { … } & … monstrosities and this is nicer than that.

ddbeck · 2025-05-21T16:31:10Z

newtypes.ts

+// eslint-disable-next-line @typescript-eslint/no-unused-vars
+const t1: FeatureData = {
+  name: "Test",
+  description: "Hi",
+  description_html: "Hi",
+  spec: "",
+  status: {
+    baseline: false,
+    support: {},
+  },
+};


If we like, we can even "test" some properties of our types—like I do here demonstrating a FeatureData object without any of the optional fields—by instantiating some objects and letting the typechecker complain if we get it wrong.

That seems like a useful smoke test. I guess that when we load the JSON and interpret it as a TypeScript type, there's no checking going on that we could rely on? That would be helpful, but I suspect it's not a thing.

We could do this, but I'd prefer to avoid it on the initial implementation. Quicktype can generate zod runtime type check code, but it'd be a new dependency that I don't feel especially ready to embrace just yet.

ddbeck · 2025-05-21T16:32:15Z

schemas/new.schema.json

@@ -0,0 +1,302 @@
+{


And here is the JSON Schema, intended for a human (me) to work with, instead of the generated JSON Schema. Mostly, things are ordered in a more readable way.

I like it! By now I can almost read JSON Schema, and I find this quite readable.

foolip · 2025-05-22T15:17:41Z

I think we should do it! I'd rather have ugly TypeScript definitions than ugly JSON Schema.

jcscottiii · 2025-06-09T19:54:06Z

schemas/data.schema.json

+          "$ref": "#/definitions/URL"
+        }
+      }
+    },


FYI: name and spec were previously marked as required. Now they are optional. Not sure if this was intentional.

Fixed in b2b6762.

jcscottiii · 2025-06-09T20:13:59Z

schemas/data.schema.json

-          ],
-          "type": "object"
+          "description": "Whether a feature is considered a \"Baseline\" web platform feature and when it achieved that status",
+          "$ref": "#/definitions/StatusHeadline"


Similar comment here. baseline and support were previously required. But I think they should stay required. (Probably in StatusHeadline)

Fixed in b2b6762.

jcscottiii · 2025-06-09T20:17:59Z

schemas/data.schema.json

-        "status"
-      ],
-      "type": "object"
+      "required": ["name", "description", "description_html", "spec"]


Is status no longer required?

Suggested change

"required": ["name", "description", "description_html", "spec"]

"required": ["name", "description", "description_html", "spec", "status"]

Fixed in b2b6762.

Co-authored-by: James C Scott III <[email protected]>

ddbeck · 2025-06-11T09:01:28Z

Thank you for the review, @jcscottiii. Thanks to your review I was able to find some other constraints that weren't in the new schema file. This also prompted me to discover some oddities in the rolled-up TypeScript types in npm package — I'm still working on fixing that part.

jcscottiii · 2025-06-16T20:48:04Z

I tried out the latest updates and it looks really good now! Thanks

ddbeck · 2025-06-30T17:39:44Z

packages/web-features/package.json

+    "types.d.ts",
+    "types.quicktype.d.ts",
    "index.js",
    "data.json",
    "data.schema.json"
  ],
  "scripts": {
-    "prepare": "tsc && rm types.js && tsup ./index.ts --dts-only --format=esm --out-dir=."
+    "prepare": "tsc && rm types.js && rm types.quicktype.js"
  },
  "devDependencies": {
    "@types/node": "^24.0.7",
-    "tsup": "^8.5.0",
    "typescript": "^5.8.3"


One thing I discovered in following up @foolip's #2990 (comment) question about type errors for consumers:

tsup adds another layer of name-mangling, on top of the layer given by converting JSON Schema to TypeScript (via quicktype). For example, as originally configured, FeatureData is shown as extending FeatureData$1. Ugly!

To fix this, I just let plain TypeScript generate the type declaration files. It's slightly less tidy — we have to include the types.d.ts and types.quicktype.d.ts files — but it's a very tiny hit for consumers (two additional small files and two development-time file reads for TypeScript consumers) and a nice boon for us (one fewer dependency and one less layer of indirection).

This also makes it a lot easier for us to do some other things in the future (e.g., adding util functions or making this a "normal" monorepo where we're doing less magic with the web-features package).

foolip · 2025-07-07T13:32:54Z

Commit 6e383ff LGTM, less code and still works!

Demonstrate JSON Schema as source of truth

9846b90

github-actions bot added the tools and infrastructure Project internal tooling, such as linters, GitHub Actions, or repo settings label May 21, 2025

ddbeck commented May 21, 2025

View reviewed changes

ddbeck requested a review from foolip May 21, 2025 16:34

ddbeck force-pushed the 91-jsonschema-source-of-truth branch from 7c9c780 to 9846b90 Compare May 22, 2025 07:09

ddbeck added 2 commits May 26, 2025 19:18

Replace types with newly generated types

1012068

Add new types to web-features package build

77bf8f9

ddbeck added major version required This PR requires a minor version semver release (vX+1.0.0) package:web-features labels May 26, 2025

ddbeck added this to the web-features v3.0 milestone May 26, 2025

Format types.ts

7742c64

ddbeck mentioned this pull request May 27, 2025

Add move and split redirects to the schema #3000

Draft

jcscottiii reviewed Jun 9, 2025

View reviewed changes

jcscottiii mentioned this pull request Jun 9, 2025

WIP: try updated web features data.schema.json GoogleChrome/webstatus.dev#1552

Draft

ddbeck and others added 3 commits June 10, 2025 15:35

Merge branch 'main' into 91-jsonschema-source-of-truth

95bf898

Add unintentionally missing required fields

b2b6762

Co-authored-by: James C Scott III <[email protected]>

Add more property constraints to reduce any in TS

dd23730

ddbeck added 2 commits June 30, 2025 18:55

Merge branch 'main' into 91-jsonschema-source-of-truth

a8b6ba1

Drop tsup and export our types directly

6e383ff

ddbeck commented Jun 30, 2025

View reviewed changes

	"required": ["name", "description", "description_html", "spec"]
	"required": ["name", "description", "description_html", "spec", "status"]

Demonstrate JSON Schema as source of truth for web-features data #2990

Are you sure you want to change the base?

Demonstrate JSON Schema as source of truth for web-features data #2990

Uh oh!

Conversation

ddbeck commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foolip commented May 22, 2025

Uh oh!

jcscottiii Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ddbeck commented Jun 11, 2025

Uh oh!

jcscottiii commented Jun 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foolip commented Jul 7, 2025

Uh oh!

Uh oh!

ddbeck commented May 21, 2025 •

edited

Loading

jcscottiii Jun 9, 2025 •

edited

Loading