3 Dolt JSON Bugs for large JSON payloads / parallel execution of JSON payloads.

[tianhuil]

The following bugs are reproduced on CI tests in a Github repo here and a 2nd Github repo.

It affects at least Dolt version 2.1.7.

See @zachmu and @nicktobey

Bug 1: JSON column silently stores NULL for large values

When inserting JSON values > ~4004 bytes containing certain control characters into
a JSON column, Dolt silently stores NULL instead of the value. No error is
raised. The same value stored in a LONGTEXT column round-trips correctly
byte-for-byte.

Trigger conditions

The bug requires both conditions:

1. JSON input text > ~4004 bytes
2. Value contains control characters that JSON.stringify serializes as \uXXXX (6-byte escape sequences)

These control characters trigger the bug (no short escape → \uXXXX):

• U+0000–U+0007 (NUL through BEL)
• U+000B (VT, vertical tab)
• U+000E–U+001F

These do not trigger (short escape or literal):

• U+0008 (BS) → \b (2 bytes)
• U+0009 (TAB) → \t (2 bytes)
• U+000A (LF) → \n (2 bytes)
• U+000C (FF) → \f (2 bytes)
• U+000D (CR) → \r (2 bytes)
• U+007F (DEL) → literal (1 byte)
• Normal ASCII / UTF-8 → literal (1 byte each)

Minimum reproduction

{"d":"\u000b\u000b\u000b...666 times...\u000b"}

Total JSON input = 4004 bytes. A single \u000b in a 50KB string also triggers.
But 100KB of plain x with no escapes works fine.

Hypothesis

Dolt's JSON serializer allocates an output buffer proportional to the input
string length. Control characters needing \uXXXX escaping expand from 1 byte
(internal) to 6 bytes (JSON output). The serializer writes past the end of the
allocated buffer, corrupting internal state and causing a silent NULL. Characters
with shorter escapes or no expansion fit within the original buffer.

---

Bug 2: JSON_VALID returns 0 for LONGTEXT values > 2040 bytes

JSON_VALID() returns 0 for valid JSON stored in LONGTEXT columns when the
value exceeds 2040 bytes and the table has no PRIMARY KEY. The same value
passes JSON_VALID() as a query parameter literal, and parses correctly in
JavaScript.

Trigger conditions

Condition	JSON_VALID
Value ≤ 2039 bytes, no PK	1 (correct)
Value ≥ 2040 bytes, no PK	0 (bug)
Value ≥ 2040 bytes, with PK	1 (correct)
Any size, as query param	1 (correct)

The bug is purely size-based — content does not matter.

Hypothesis

Dolt's JSON_VALID implementation uses a fixed 2048-byte internal buffer.
The 8-byte overhead of JSON wrapping ({"d":"..."}) reduces usable payload to
2040 bytes. The presence of a PRIMARY KEY changes Dolt's internal storage format
(row format) such that the buffer limit is not hit.

Bug 3: JSON column returns garbled bytes under concurrent read load

Under concurrent read load, CAST(data_json AS CHAR) returns either NULL or garbled, unparseable bytes for every read of a JSON column — including small values that round-trip correctly when read single-threaded. The same reads against a LONGTEXT column succeed with zero failures under identical load. No error is raised; data is silently lost.

Trigger conditions

The bug requires all conditions:

1. Multiple concurrent SQL connections to the same Dolt server (8 connections observed)
2. Concurrent reads of a JSON column containing values that trigger Bug 1 (i.e., > ~4004 bytes with \uXXXX escapes)
3. Small JSON values on the same connection also get corrupted once any large value has been read

LONGTEXT columns under the same load show zero corruption across all sizes.

Minimum reproduction

CREATE TABLE repro_test (
  id INT AUTO_INCREMENT PRIMARY KEY,
  label TEXT NOT NULL,
  data_json JSON,
  data_text LONGTEXT
);

-- Insert small (1.3KB), medium (224KB), large (21MB) JSON
-- containing \u000b (VT) characters into both data_json and data_text

// 8 connections, 20 read iterations via Promise.all
const pool = await Promise.all(
  Array.from({ length: 8 }, () => mysql.createConnection({ ... })),
)
await Promise.all(Array.from({ length: 20 }, async (_, i) => {
  const conn = pool[i % 8]
  await conn.query("SELECT label, CAST(data_json AS CHAR) AS raw_json FROM repro_test")
  await conn.query("SELECT label, CAST(data_text AS CHAR) AS raw_text FROM repro_test")
}))

Results

Column	Reads attempted	Unparseable	NULL	Wrong format
LONGTEXT	60	0	0	0
JSON	60	60	40	20

Single-threaded, JSON reads of the small value succeed; only medium/large return NULL (Bug 1). Under concurrent load, all 60 JSON reads fail — 40 return NULL, 20 return unparseable garbage. Repro: repro.test.ts "Concurrent reads".

Hypothesis

Bug 1's buffer overflow appears to corrupt per-connection or per-server state. Once any connection reads a large JSON value, subsequent reads on that connection return garbage even for small values that single-threaded reads handle correctly. The LONGTEXT path uses a different serializer that does not suffer this issue, suggesting the corruption lives in Dolt's JSON column serialization code path.

[timsehn]

I hit what appears to be this same JSON corruption bug from an opencode-on-Dolt workload. Summary of findings from a full Dolt SQL debug log plus the live repo state:

Schema involved:

CREATE TABLE `part` (
  `id` VARCHAR(255) NOT NULL,
  `message_id` VARCHAR(255) NOT NULL,
  `session_id` VARCHAR(255) NOT NULL,
  `time_created` BIGINT NOT NULL,
  `time_updated` BIGINT NOT NULL,
  `data` JSON NOT NULL,
  PRIMARY KEY (`id`),
  INDEX `part_message_id_id_idx` (`message_id`, `id`),
  INDEX `part_session_idx` (`session_id`)
);

Observed behavior:

• 122 out of 455 part rows in one session branch fail JSON reads.
• Querying JSON_TYPE(data) for those rows fails with invalid character 'm' after array element.
• Normal app reads of data fail with JSON parse errors such as Expected ']'.
• All 122 corrupt rows have last logged writes containing control / ANSI escape sequences such as \u001b.
• 121 of 122 corrupt rows have last logged JSON literals over 4004 bytes.
• The logged SQL literals decode as valid JSON objects after MySQL string-literal unescaping, so this does not look like the app writing malformed JSON.

Example detection query that fails:

SELECT id, JSON_TYPE(data) AS jt
FROM part
WHERE id IN ('prt_ecdaac1bd001QWEk2wuDlyGEcD');
-- error: invalid character 'm' after array element

A representative final write from the SQL log for that row. This logged write decodes to a valid JSON object with keys callID, metadata, state, tool, type, but the row later cannot be read as JSON:

insert into `part` (`id`, `message_id`, `session_id`, `time_created`, `time_updated`, `data`)
values (
  'prt_ecdaac1bd001QWEk2wuDlyGEcD',
  'msg_ecdaaa9fe001jrDPg2Szoq1MTj',
  'ce0da46e617d7164641849fbec8970af4bce859c_session',
  1781566980609,
  1781566980610,
  '{\"callID\":\"call_xYv5k6ErZDHyBxYbSKe2q7py\",\"metadata\":{\"openai\":{\"itemId\":\"fc_061701002cd8e8c5016a308e047dac8198aa6ab45ba8ecd524\"}},\"state\":{\"status\":\"completed\",\"input\":{\"include\":\"*\",\"path\":\"/Users/timsehn/.local/share/opencode/tool-output/tool_ecdaaa92a001bfcCmmhCx5LlP7\",\"pattern\":\"(?i)(fail|error|mismatch|expected|got|wrong|panic|segfault|assert)\"},\"output\":\"Found 1 matches\\n/Users/timsehn/.local/share/opencode/tool-output/tool_ecdaaa92a001bfcCmmhCx5LlP7:\\n Line 401: sqllogictest\\tClone sqllogictest (official SQLite Fossil repo)\\t2026-06-15T18:37:15.4786119Z \\u001b[36;1m# per-assertion divergence list) stay stable across runs.\\u001b[0m\\n\",\"metadata\":{\"matches\":1,\"truncated\":false},\"title\":\"(?i)(fail|error|mismatch|expected|got|wrong|panic|segfault|assert)\",\"time\":{\"start\":1781566980562,\"end\":1781566980609}},\"tool\":\"grep\",\"type\":\"tool\"}'
)
on duplicate key update
  `time_updated` = 1781566980610,
  `data` = '{\"callID\":\"call_xYv5k6ErZDHyBxYbSKe2q7py\",\"metadata\":{\"openai\":{\"itemId\":\"fc_061701002cd8e8c5016a308e047dac8198aa6ab45ba8ecd524\"}},\"state\":{\"status\":\"completed\",\"input\":{\"include\":\"*\",\"path\":\"/Users/timsehn/.local/share/opencode/tool-output/tool_ecdaaa92a001bfcCmmhCx5LlP7\",\"pattern\":\"(?i)(fail|error|mismatch|expected|got|wrong|panic|segfault|assert)\"},\"output\":\"Found 1 matches\\n/Users/timsehn/.local/share/opencode/tool-output/tool_ecdaaa92a001bfcCmmhCx5LlP7:\\n Line 401: sqllogictest\\tClone sqllogictest (official SQLite Fossil repo)\\t2026-06-15T18:37:15.4786119Z \\u001b[36;1m# per-assertion divergence list) stay stable across runs.\\u001b[0m\\n\",\"metadata\":{\"matches\":1,\"truncated\":false},\"title\":\"(?i)(fail|error|mismatch|expected|got|wrong|panic|segfault|assert)\",\"time\":{\"start\":1781566980562,\"end\":1781566980609}},\"tool\":\"grep\",\"type\":\"tool\"}';

Another example from the same session:

• Row: prt_ecdb21e48001osfQgq4HCLbh5e
• Final logged JSON literal size: 7165 bytes
• Contains control / ANSI escapes (\u001b)
• Decodes as a valid JSON object after MySQL string unescaping
• Later fails:

SELECT id, JSON_TYPE(data) AS jt
FROM part
WHERE id IN ('prt_ecdb21e48001osfQgq4HCLbh5e');
-- error: invalid character 'm' after array element

The final write for that row is a large read tool result containing file contents. The payload begins like:

{"callID":"call_DIaRgwiEeBbQMiWzLHvKBm0k","metadata":{"openai":{"itemId":"fc_0f9c03e9bf80da29016a308fe6fd68819a91c793a296f52444"}},"state":{"status":"completed","input":{"filePath":"/var/folders/54/v92yp9tj6ll0j10w7rjg_3jc0000gn/T/opencode/sqllogictest-81496417640.log","limit":50,"offset":2160},"output":"<path>/var/folders/...

and ends like:

... "time":{"start":1781567464254,"end":1781567464271}},"tool":"read","type":"tool"}

This lines up with the issue description: valid large JSON values containing control characters serialized as \uXXXX are inserted successfully, but later reads of the JSON column return unparseable / corrupted JSON.

[timsehn]

#!/usr/bin/env bash

set -u

# Minimal Dolt JSON repro with no dependencies beyond bash and the dolt CLI.
# Creates a fresh Dolt repo and a table with the same value in JSON and LONGTEXT.

BASE_DIR="${BASE_DIR:-/Users/timsehn/Downloads}"
REPRO_DIR="${REPRO_DIR:-$BASE_DIR/dolt-json-repro-$(date +%s)-$$}"
TABLE="json_repro_shell"
KEEP="${KEEP:-1}"

if ! command -v dolt >/dev/null 2>&1; then
  echo "error: dolt is not on PATH" >&2
  exit 1
fi

if [[ -e "$REPRO_DIR" ]]; then
  echo "error: REPRO_DIR already exists: $REPRO_DIR" >&2
  echo "choose a new REPRO_DIR or remove it first" >&2
  exit 1
fi

mkdir -p "$REPRO_DIR" || exit 1
cd "$REPRO_DIR" || exit 1
dolt init --name "dolt-json-repro" --email "dolt-json-repro@example.com" >/dev/null || exit 1

run_sql() {
  local label="$1"
  local sql="$2"
  echo
  echo "== $label =="
  if output=$(dolt sql -q "$sql" 2>&1); then
    printf '%s\n' "$output"
  else
    echo "FAILED"
    printf '%s\n' "$output"
  fi
}

echo "Fresh Dolt repo: $REPRO_DIR"
echo "Dolt version:     $(dolt version 2>/dev/null || true)"
echo
echo "This creates valid JSON text with escaped control characters:"
echo '  {"d":"Line \u001b[36;1m \u000b\n ..."}'
echo
echo "Expected: the LONGTEXT copy has bytes, but the JSON column cannot be read."

run_sql "create repro table" "
CREATE TABLE \`$TABLE\` (
  id VARCHAR(64) NOT NULL PRIMARY KEY,
  data_json JSON,
  data_text LONGTEXT
)"

run_sql "insert small valid JSON" "
INSERT INTO \`$TABLE\` (id, data_json, data_text)
VALUES ('small_plain', '{\"d\":\"small\"}', '{\"d\":\"small\"}')"

run_sql "insert large valid JSON with escaped control characters" "
INSERT INTO \`$TABLE\` (id, data_json, data_text)
SELECT
  'large_ansi_control',
  CONCAT(
    '{\"d\":\"',
    REPEAT(CONCAT('Line ', CHAR(92), 'u001b[36;1mcolored', CHAR(92), 'u000b', CHAR(92), 'n'), 200),
    '\"}'
  ),
  CONCAT(
    '{\"d\":\"',
    REPEAT(CONCAT('Line ', CHAR(92), 'u001b[36;1mcolored', CHAR(92), 'u000b', CHAR(92), 'n'), 200),
    '\"}'
  )"

run_sql "LONGTEXT copy is present" "
SELECT id, data_text IS NULL AS text_is_null, LENGTH(data_text) AS text_len
FROM \`$TABLE\`
ORDER BY id"

run_sql "JSON column cast shows corruption / NULL" "
SELECT id, CAST(data_json AS CHAR) IS NULL AS json_cast_is_null, LENGTH(CAST(data_json AS CHAR)) AS json_cast_len
FROM \`$TABLE\`
ORDER BY id"

run_sql "JSON_TYPE on all rows fails" "
SELECT id, JSON_TYPE(data_json) AS json_type
FROM \`$TABLE\`
ORDER BY id"

run_sql "JSON_TYPE on only the large row fails" "
SELECT id, JSON_TYPE(data_json) AS json_type
FROM \`$TABLE\`
WHERE id = 'large_ansi_control'"

if [[ "$KEEP" != "1" ]]; then
  cd / >/dev/null || true
  rm -rf "$REPRO_DIR"
else
  echo
  echo "Keeping repro repo: $REPRO_DIR"
  echo "Remove it with: rm -rf '$REPRO_DIR'"
fi