-
Truck T07 lost its compressor at 14:00 and its cargo warmed to 14 C. The decoy truck with the open door is called normal handling.
-
10,000 API requests in 71 ms: /checkout in eu-west slowed down, and the database pool wait explains why.
-
Card payments went down at all 4 north stores while cash kept working. Found in 73 ms.
Inspiration
Every team has a dashboard that does something weird at 2 a.m. Today you either dig through the numbers yourself or paste them into a cloud chatbot, which costs money, needs the internet and hands your data to someone else.
We had been studying how much a small language model really needs to be useful. The surprising answer: squeeze one into 8-bit integers, teach it one kind of data, and it fits on a $150 FPGA board while still giving solid answers. So we asked a simple question. What if the expert lived in a box on your desk instead of in someone else's data center?
What it does
PYNQ Process is a Mac app plus a PYNQ-Z2 board. You drop in a CSV, type "What's wrong with my system?", and get a plain answer back, with the exact readings it used.
- The app does the math, the model does the talking. Every number in an answer (peaks, durations, how far from normal, which column moved with it) is computed by code on your Mac. The language model only explains measured facts. That is how a 77M-parameter model stays honest.
- The board runs the model. A quantized FLAN-T5-small runs on the PYNQ-Z2. All 1,021 matrix projections per question run on the FPGA fabric, and the ARM core drives generation, while the board keeps scoring live telemetry.
- Swap experts, not hardware. Models are weight files, not bitstreams. We ship three quantized checkpoints (Diagnosis, Queue history v2 and Telemetry v1) you can switch between in seconds, and you can make a named edit to a model and roll it back with one click.
- Built for real files. It handles messy CSVs and JSON lines, up to 50,000 rows, and gives Short, Summary, Transcript and Table answers, plus follow-ups, notes, history, export and an MCP connector.
The demo
We planted one problem in each of three 10,000-row files: an API, a fleet of refrigerated trucks and a chain of stores. Then we asked "What's wrong with my system?"
- API: "Latency went from 143 ms to 1,928 ms on /checkout in eu-west between 14:12 and 14:31. Likely cause: DB pool wait, which went from 6.4 ms to 1,474 ms at the same moment."
- Trucks: "Compressor went from 6.57 A to 0.45 A at Nov 19 14:00, then temperature rose." It also calls the decoy truck with the open door normal handling.
- Stores: "Revenue went from 280 per hour to 20.4 per hour on card in north (all 4 stores) between 12:00 and 17:00. Revenue stayed normal for cash in north."
Each answer took 71 to 89 ms on a laptop. A cloud model reading the same files took 37 to 57 seconds and about 370,000 tokens per file. A separate model graded both blind against the planted answers: 140 out of 180 for us, 154 for the cloud model. Nearly the same diagnosis, over 400 times faster, and the data never left the desk.
How we built it
- FPGA fabric. We wrote an INT8 projection engine in SystemVerilog for the Zynq-7000, verified it against a software reference (17,920 of 17,920 scores match), and then grew it into a DMA-fed tiled engine that serves every dense layer of the model, vocabulary head included.
- The model. We fine-tuned FLAN-T5-small with rank-16 LoRA adapters on synthetic incident data, then quantized it to signed INT8 with one scale per output row. Checkpoints ship as small deltas over the public base, with SHA-256 checks at every step, so anyone can rebuild them byte for byte.
- Speed. Our first full answer on the board took about 13 minutes. Aligned projections, resident weights and a decoder that verifies up to 16 proposed tokens per pass got that down to first text in about 9 seconds and the full diagnosis in 14 to 18 seconds, with byte-identical output.
- The analyst. Robust statistics find each incident against a rolling median, then explain it: which other column moved with it, which slice (endpoint, region, store) it came from, what stayed normal, and what to check next.
- The Mac app. A native Swift shell around a local web interface and a standard-library Python backend. Sign-in uses Auth0 Universal Login with PKCE. A board twin runs the exact board service and the same INT8 checkpoints on the Mac, so we can rehearse and test without the hardware.
Challenges we ran into
- Fitting a language model on a $150 board. 512 MB of shared memory, a 50 MHz fabric and a dual-core ARM. Every layout and cache decision mattered.
- Small models make up numbers. Our fix was structural: compute every fact in code, and let the model only phrase what was measured.
- Proving it is real. Every timing on our results page links to a recorded run, and every checkpoint is hash-pinned.
Accomplishments that we're proud of
- A real language model answering questions on an FPGA, 1,021 of 1,021 projections on the fabric.
- About 55 times faster than our first working version, with the exact same answers.
- On a held-out set of synthetic incidents, the diagnostic checkpoint answered 12 of 12 correctly, and Queue history v2 classified 15 of 15 histories.
What we learned
The weights are the product. Once the math lives in hardware, a new skill is just a new file. And a small model you can check beats a big model you have to trust.
What's next for PYNQ Process
More experts for more kinds of data (sensors, sales, logs), all on the same board, and a shared library so teams can trade the models they train.