Skip to content
coreystevensdevPublic

About

Agentic NFL betting research service: sharp-line vig removal, LangGraph HITL approval, CLV tracking

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Fairline

CI 382 tests 18-case eval

Finds sports bets that are priced better than the professional market thinks they should be, and makes a human approve every one before it goes anywhere. NFL, NBA, MLB, NHL.

The three read pages are open at fairline.coreystevens.dev, no signup. They are running on seeded sample data, which the pages say on every screen, because a graded betting record that is not real is a claim about performance.

Every bookmaker builds a margin into its odds, called vig. Strip the vig out of a sharp book's line and you get that market's honest estimate of what will happen. Fairline does that to Pinnacle's lines via The Odds API, then has Claude surface the retail prices that beat it. A LangGraph interrupt() pauses the pipeline for approval before any bet slip is prepared, so nothing is ever placed on its own.

Each pick is signed by the agent that produced it, and those agents are graded on closing line value: did the price beat where the market finished, rather than did the bet win. That is the harder standard, and the one that separates edge from luck.

Problem

Retail sports bettors lose because they bet off public lines that already carry bookmaker margin. Closing Line Value (CLV) is the market-validated signal that separates long-run winners from losers: if you consistently beat the closing line, you have genuine edge. No public tool automates this research pipeline end-to-end with HITL approval built in.

Solution

The pipeline runs as a LangGraph StateGraph: fetch Pinnacle odds, strip vig to no-vig fair probabilities for each side, filter picks by a minimum edge threshold, then call Claude with a forced submit_picks tool to generate structured pick candidates. An interrupt() checkpoint pauses the graph for user approval before any bet slip is finalized. State persists via PostgresSaver so approval sessions survive server restarts. CLV is recorded post-settlement for every approved pick, building a backtestable track record.


How it works

flowchart TD
    A[POST /api/runs] --> B[odds_agent: fetch Pinnacle odds, strip vig]
    B -->|error| Z[END]
    B -->|success, fan out via Send| C[weather_agent]
    B -->|success, fan out via Send| D[injury_agent]
    B -->|success, fan out via Send| S[stats_agent]
    B -->|success, fan out via Send| G1[signal_agent]
    C --> E[trends_agent]
    D --> E
    S --> E
    G1 --> E
    E --> F[sim_agent]
    F --> G[pick_agent: Claude forced submit_picks tool call]
    G --> H{hitl_review: LangGraph interrupt}
    H --> I[User reviews candidates via GET /api/runs/id]
    I --> J[POST /api/runs/id/approve]
    J --> K[validate_agent: record CLV baseline]
    K --> L[Bet slips prepared]
Loading

weather_agent, injury_agent, stats_agent, and signal_agent run in parallel via LangGraph's Send() API and share no data dependency; all four converge on trends_agent before the sim runs.

Data source routing:

Source Purpose Access
Pinnacle (via The Odds API) Sharp-line reference; no-vig fair probability ODDS_API_KEY
FanDuel / DraftKings / BetMGM Retail price comparison; line shopping Same key
BALLDONTLIE Team season stats (all 4 sports); player season stats (NFL/MLB only) BALLDONTLIE_API_KEY
pybaseball Per-game MLB batter stats for the matchup splits engine; scrapes Baseball Reference, Baseball Savant, and FanGraphs directly None (unofficial, unauthenticated)
NHL official API Per-game NHL skater stats (schedule + boxscore) for the matchup splits engine None (official, unauthenticated)
nba_api (stats.nba.com) Per-game NBA player stats (bulk LeagueGameLog) for the matchup splits engine None (unofficial, unauthenticated); stats.nba.com blocks most cloud/datacenter IPs, see below
Anthropic Claude Pick generation with forced submit_picks tool call ANTHROPIC_API_KEY
Stripe Subscription billing (Pro tier) STRIPE_SECRET_KEY

Tech stack

Layer Technology Why
Agent orchestration LangGraph 0.3+ Stateful graph with first-class interrupt() for HITL; PostgresSaver for durable checkpoints across restarts
LLM Anthropic Claude (forced tool call) Forced tool use (submit_picks) guarantees structured output; no output parsing
Odds data The Odds API v4 Single endpoint returns Pinnacle + 40 retail books in one call; 500 free req/month is enough for a daily picks run
Sharp-line math american_to_prob + remove_vig Converting American odds to implied probability then normalizing removes the bookmaker overround in O(n)
API FastAPI Async lifespan manages shared httpx client and graph instance
Auth Bearer API keys, SHA-256 at rest Identity derives from the key server-side, so ownership checks compare against a real principal instead of a caller-supplied user_id; no external auth dependency
Database PostgreSQL + SQLAlchemy Pick history and CLV tracking; PostgresSaver for LangGraph checkpoints
Payments Stripe webhooks Subscription lifecycle via customer.subscription.created/deleted events
Spend guard rolling-cost-cap The model call is the only metered cost here. The cap gets asked what a call could cost before it is made, so a scheduled run that loses its exit condition stops at the budget rather than past it
Testing pytest + respx respx mocks at the httpx transport layer; no network calls in CI
Observability LangSmith Traces each graph run at node level; HITL pause and resume appear as two linked traces (picks/... then approve/...), making the two-phase architecture visible without reading code

Closing Line Value (CLV)

CLV compares the price you took against the sharp book's closing price:

CLV = no-vig closing probability - implied probability of the price you bet

A positive CLV means you beat the close. Sportsbooks use CLV to identify sharp bettors and limit their accounts. The comparison is against the price actually taken, not the model's estimate: if Pinnacle closes at a no-vig 52.2% and you bet at -108 (implied 51.9%), you have +0.3% CLV whatever the model believed.

The settlement job captures closing lines:

python -m fairline settle --window-minutes 30 --sport all

It finds picks with no closing_price whose game starts within the window, pulls the current Pinnacle market, devigs it, and writes closing_price, closing_probability, and clv. Run it near kickoff (cron a few minutes before the day's first game).

After games finish, grade results:

python -m fairline grade --sport all

Grading pulls final scores (up to 3 days back, the API maximum), settles each pick against the number it was bet at, and writes result (win/loss/push) and profit_units for a 1-unit flat stake. Spreads grade against the margin plus the taken point, totals against the combined score, and exact landings push. The full performance query:

SELECT COUNT(*), AVG(clv), SUM(profit_units) FROM picks WHERE result IS NOT NULL;

Positive average CLV with a losing profit_units over a small sample means variance; negative CLV with a winning record means luck that will not hold.

Trends

ATS and over/under records come from Fairline's own data, no stats provider: the watcher's last pre-kickoff snapshot is each game's closing line, and grading already fetches final scores. fairline grade joins the two into game_results, and a trends_agent node attaches each team's recent records to the pick prompt, so the pick agent cites real form instead of guessing at it.

python -m fairline trends --team "Kansas City Chiefs" --last 10
Kansas City Chiefs: 7-3-0 SU, 6-3-1 ATS, 4-5-1 O/U (last 10)

Also served at GET /api/trends?team=...&last_n=10. Coverage grows with collection: games the watcher never saw count toward straight-up records only, since their closing lines were never captured.

Research context

Two more inputs reach the pick agent the way a professional would check them, as data, not vibes:

Schedule spots. Rest days, back-to-back flags, and games-in-last-7 are computed from stored results (no new data source) and ride the same "Recent form" prompt line as trends. Rest is the most mispriced situational factor in the NBA, and it costs nothing to know.

Injuries. An injury_agent node reads ESPN's public injury feed per league, and code (never the LLM) converts absences to bounded margin adjustments: an out NFL quarterback is worth about five and a half points, questionable counts half, everything is capped at eight per team, and every counted absence is listed in the pick prompt so the reviewer sees which name moved the number. The design doc planned LLM extraction here; the feed turned out to be structured JSON, so the extraction step needs no model at all.

Weather. For outdoor NFL games inside the 16-day forecast horizon, a weather_agent node pulls the kickoff-hour wind, temperature, and precipitation from Open-Meteo (free, keyless). The reading lands in the pick prompt, and wind feeds a bounded totals adjustment in the sim: about a third of a point off the expected total per mph over 10, capped at 7 points. Domes are skipped; the adjustment is code, not narrative.

The standing rule ties this together: Claude reads what the pipeline hands it and never invents a fact. A rationale citing 20 mph wind traces to a forecast the system actually fetched.

Agent records

Every pick is stamped with the agent that produced it (source), and the leaderboard grades each agent over settled picks:

python -m fairline agents

What the seeded fixtures actually grade to, so you can run the two commands above and compare:

matchup: 3-1-0 avg_clv=+0.0235 units=+1.84 n=4
steam:   3-2-0 avg_clv=+0.0167 units=+0.77 n=5
model:   1-1-1 avg_clv=+0.0061 units=-0.29 n=3

Note the ordering: model is the only agent with negative units and still has positive CLV, which is the case the whole metric exists for. Twelve picks is far too few to tell you anything about the agents, and is only here so the page has something to render.

Win rate is noisy and gameable; average CLV is the grade that matters, and it is the same standard for every agent. All four leagues flow through the same settlement and grading, so records are comparable across sports (--sport all on settle and grade covers every league in one cron entry, one API request per league).

Player props

The same top-down method applies per player: devig the sharp book's Over/Under pair, then flag retail books posting the identical line at a lagging price. Only exact point matches compare; pricing the gap between a 275.5 and a 280.5 needs a projection model (see docs/matchup-agent-design.md for where that goes).

python -m fairline props --sport americanfootball_nfl --min-edge 0.03 --max-events 5

Props cost one API request per event per scan, unlike game odds where one request covers the whole slate; --max-events caps the spend.

The matchup agent

The prop-filter workflow, automated (design: docs/matchup-agent-design.md). Seed player game logs, then scan:

python -m fairline backfill-players --seasons 2023 2024 2025
python -m fairline matchup --sport americanfootball_nfl --max-events 5

For each prop, the engine computes a pre-registered set of splits (last 5, last 10, season, vs this opponent) against the line, shrinks each hit rate toward the market's fair probability with a beta-binomial prior (8-of-10 reads as roughly 65%, not 80%), and combines them into a probability clamped within 6 points of the sharp fair number. Where that history-adjusted number beats a retail price, a candidate lands in the same review queue as steam picks, tagged source=matchup, with the splits spelled out in the rationale:

fair 0.500 -> matchup 0.552; Over angles: last_5 3-2 over 250.5; last_10 7-3 over 250.5; season 9-5 over 250.5; vs_opponent 1-3 over 250.5

Approved matchup picks grade automatically against box scores (fairline grade matches player, date, and stat; exact landings push), get their closing lines captured by fairline settle (which fetches props only for events holding an unsettled prop pick, one request each), and earn their own row on the agent leaderboard. Prop CLV is computed only when the closing point equals the taken point: prop lines drift too much for price-only comparison to mean anything across different numbers, so a moved line records its drift in closing_point and leaves clv honestly NULL. Every matchup pick also records which angles fed it, and python -m fairline angles grades the angles themselves over settled picks: record, units, and average CLV per split. An angle that cannot show value over a real sample loses its place in the pre-registered set. No manual filter user audits their filters; this is the audit. The splits are fixed in code, never searched per prop: letting anything hunt for the best-looking slice is the multiple-comparisons trap that makes every prop "8 of the last 10" at something.

The same NFL engine also carries home/away, primetime, weather, and surface splits, pulled from the same nflverse games CSV fairline backfill-nfl already downloads for scores and closing lines, so nothing new gets fetched or installed. Home/away and primetime read directly off the game row (weekday plus kickoff hour: Monday or Thursday, or Sunday at 19:00 or later); weather flags a game as bad when wind hits 15 mph or temperature drops to 32F or below, and is dropped from the split entirely on games with no recorded temp or wind, typically domes; surface only shows up in the rationale once the upcoming opponent's home surface is known, comparing history on that same surface against the line.

NFL also gets a position_matchup split, the same defense-vs-position idea as the NBA engine above but built on nflverse's own row-level position field (QB, RB, WR, TE) instead of a separate roster call. That field was already being read to filter which rows to keep during ingestion; it's now also stored on PlayerGame so the split can query it later. Where vs_opponent is one player's own history against the upcoming opponent, position_matchup pools every player at that position who has faced them, so a receiver with only one or two career games against a given defense still gets a matchup read built from every receiver who has played that defense, not just his own thin sample.

MLB batter props

MLB gets its own splits engine rather than reusing the NFL last-5/last-10/season pattern as-is. Baseball's schedule and scoring support splits football doesn't have, day/night games, a real home/away split, and park factors that move run totals by double digits across parks. Seed per-game batter stats via pybaseball, then scan:

python -m fairline backfill-mlb-players --start 2025-04-01 --end 2025-09-30
python -m fairline matchup --sport baseball_mlb --markets batter_hits,batter_home_runs,batter_rbis,batter_total_bases,batter_strikeouts

For each batter prop, compute_mlb_prop_splits computes last-5, last-10, season, day, night, home, away, and park-factor hit rates against the line, the same shrinkage-toward-fair math as the NFL matchup agent, combined into a probability that flags a candidate when it beats the retail price. The vs-specific-pitcher split is computed too, but only surfaces when a batter has faced that pitcher at least 10 times (MIN_VS_PITCHER_SAMPLE); below that floor the split is left out of the result entirely rather than shown on a handful of at-bats. Candidates land in the same review queue as NFL matchup and steam picks, tagged source=mlb_matchup.

vs_pitcher is now live-wired for upcoming games, not just backtesting. create_mlb_matchup_candidates resolves the probable starter for a batter's team from MLB's own free official Stats API (statsapi.mlb.com), no scraping, no key required. The same graceful-degradation guard used on the NBA and NFL defense-vs-position splits applies here: an unresolved batter team, or a starter the schedule API hasn't announced yet, omits the split rather than guessing at an opponent. NHL's vs_goalie stays unwired by contrast; no equivalent free official pregame source for confirmed starting goalies turned up when api-web.nhle.com's schedule, landing, and right-rail endpoints were checked live during planning, and the realistic sources are third-party sites that require scraping, which this codebase doesn't do.

NHL skater props

NHL gets the same shrinkage-and-market-bound treatment as MLB and NFL, on splits that fit hockey's own schedule and roster shape rather than reusing either sport's set as-is. MoneyPuck was the first data source considered and got ruled out during research: it publishes only per-player season CSVs, not a bulk per-game skater file, and its terms post an explicit anti-scraping stance. The NHL's own API is free, needs no key, and returns per-game boxscores directly, so that's the source. Seed per-game skater stats, then scan:

python -m fairline backfill-nhl-players --team EDM --season 20252026
python -m fairline matchup --sport icehockey_nhl --markets player_goals,player_assists,player_points,player_shots_on_goal

compute_nhl_prop_splits computes last-5, last-10, season, home, away, and back-to-back-rest hit rates against the line, the same shrinkage math as the other two sports' matchup agents. Rest days aren't a field the NHL API returns; they're derived from the gap between a team's consecutive scheduled game dates during ingestion. The vs-specific-opposing-goalie split is computed too, but only surfaces at 10 or more career meetings against that goalie (MIN_VS_GOALIE_SAMPLE); below that floor it's left out of the result rather than shown on a couple of games. Candidates land in the same review queue as NFL, MLB, and steam picks, tagged source=nhl_matchup.

NBA player props

NBA gets the same shrinkage-and-market-bound treatment as MLB and NHL, on splits that fit basketball's own schedule shape: last-5, last-10, season, home, away, and back-to-back rest. Data comes from nba_api's bulk LeagueGameLog endpoint, which returns every player's box score for a season in one call rather than one request per player. Seed per-game player stats, then scan:

python -m fairline backfill-nba-players --season 2024-25
python -m fairline matchup --sport basketball_nba --markets player_points,player_rebounds,player_assists,player_threes

stats.nba.com blocks most cloud and datacenter IPs, and that breaks the backfill command above on a typical hosted deploy. A GitHub issue on nba_api documents Heroku specifically being blocked, and the community consensus is that free proxy lists are already blacklisted too, so there's no free, reliable workaround. The client retries with exponential backoff and an explicit timeout (nba_api has neither by default), but backoff doesn't help against an IP-level block. backfill-nba-players takes a --proxy host:port flag that passes straight through to nba_api's native proxy support; a residential-IP relay or a non-hyperscaler VPS is the realistic way to run this from a cloud host, not a code fix.

compute_nba_prop_splits computes the same shrinkage math as the other three sports' matchup agents. Unlike MLB's vs-pitcher and NHL's vs-goalie splits, there is no vs-specific-defender split for NBA at all: no verified per-game defender-matchup data source exists, the one endpoint that identifies who guarded whom (LeagueSeasonMatchups) is season-aggregate only, confirmed during planning. That is a genuine data gap, not a deferred feature. Candidates land in the same review queue as NFL, MLB, NHL, and steam picks, tagged source=nba_matchup.

A broader defense-vs-position split fills part of that gap without needing per-game defender data: position_matchup looks at every player sharing the same broad position bucket (Guard, Forward, or Center) who has faced the upcoming opponent, not just the one player's own history against that team. Position comes from nba_api's CommonTeamRoster endpoint, one call per team, 30 calls for the whole league, fetched once during backfill-nba-players and stored in the position column rather than looked up per pick. CommonTeamRoster's real POSITION field turned out to use letter codes ("G", "F", "C", and combos like "G-F") rather than the full words assumed during planning; the roster client maps both formats to the three canonical buckets, taking the first listed position in a combo as the primary one. A player whose position never resolved (a name mismatch against the roster feed, for instance) gets no position_matchup split rather than a guessed one.

The simulation model

The sim_agent node computes probabilities per sport before picks are generated, consuming everything upstream context provides: results-fit ratings, current-season scoring rates, kickoff-hour wind, and rest spots (NBA back-to-backs cost about two points of expected margin, NFL short weeks one, bounded at three). NFL and NBA use a Normal margin family (Elo-style points ratings, sigma 13.5 and 11.5); NHL and MLB are low-scoring count sports, so they use a double-Poisson scoring model built from team rates, with regulation ties split by relative strength since neither sport can end tied, and NHL back-to-backs shaving a quarter goal off the tired side. For NFL: Elo-style points-scale team ratings fit from stored game results (home-field advantage 2.0, ratings regress a third toward zero between seasons), a Normal margin model (sigma 13.5) for win and cover probabilities, and a totals model built from current-season team scoring rates (Normal, sigma 10) for over/under probabilities. All arithmetic is code over data; Claude is never asked to estimate a probability. Seed the ratings with history in one command:

python -m fairline backfill-nfl --seasons 2023 2024 2025

The nflverse games file carries closing spreads and totals alongside scores, so the same backfill deepens the trends table. Design and phased roadmap: docs/sim-design.md. Known modeling error, accepted for v1: the Normal approximation ignores key numbers, so push probabilities near 3 and 7 are misestimated.

If you run your own simulation model instead, pass its probabilities into a run and they take precedence over the built-in model for matching games:

curl -s -X POST http://localhost:8000/api/runs \
  -H "Authorization: Bearer $FAIRLINE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"sport": "americanfootball_nfl", "sims": [
        {"home_team": "Kansas City Chiefs", "away_team": "Las Vegas Raiders",
         "market": "spreads", "selection": "Kansas City Chiefs -3.5", "probability": 0.58}
      ]}' | jq

The blend is sharp-dominant: blended = 0.75 * sharp + 0.25 * sim by default (FAIRLINE_SIM_WEIGHT overrides). Edge and EV are computed from the blended probability, so a sim that disagrees with the market changes which picks surface. Picks without a matching sim use the sharp line alone.

The weight is not a matter of taste; the sim has to earn it:

python -m fairline sim-report

This splits settled picks into sim-agreed and sim-disagreed against the sharp line and compares average CLV. The disagreed bucket is the only place a sim can prove it carries information the market lacks. If that bucket's CLV is not positive over a real sample, the sim is adding confidence, not information, and its weight should go down, not up.

Steam detection

Steam is a sharp, fast move at Pinnacle: informed money hitting the market. The watcher records line history and flags it:

python -m fairline watch --interval-seconds 120 --window-hours 3

Each cycle stores Pinnacle and retail prices for games kicking off within the window into line_snapshots, compares the newest sharp-book cycle against the recent baseline, and alerts on decisive moves: a no-vig probability jump of 2+ points inside ~10 minutes, or an NFL spread crossing a key number (3 or 7, the most common football margins) toward the favorite. Key numbers are football facts, so other leagues run on the probability threshold alone. Slow drift does not fire; steam is velocity. Alerts print to stdout and, when FAIRLINE_WEBHOOK_URL is set, POST as JSON to that URL (Slack- and Discord-compatible payload shape):

STEAM Kansas City Chiefs (spreads) -110 -> -125 point -2.5 -> -3.0 KEY prob +0.021 in 6m via pinnacle

Use --once under cron. Every poll costs one Odds API request per league; the quota math lives in Running a season.

Steam events also become pick candidates: the watcher scans the same cycle's retail prices for books still at least 2 points of implied probability behind the new sharp number and queues them as pending candidates with the edge math attached. The queue is shared with the matchup agent (candidates carry a source); review and approve:

GET  /api/steam                      pending candidates, newest first
POST /api/steam/{id}/approve         becomes a Pick with source=steam
POST /api/steam/{id}/reject

Approval is the same HITL principle as picks runs, just without the graph: no Claude call is needed when the rationale is "Pinnacle moved and DraftKings has not," so candidates go straight to a review queue, and an approved candidate enters the same settlement, grading, and leaderboard as every other pick.


Getting started

cp .env.example .env
# fill in ANTHROPIC_API_KEY, ODDS_API_KEY, STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET
docker compose up

API is available at http://localhost:8000. The /health endpoint confirms the service is running.

Without DATABASE_URL the service still runs picks end to end but skips persistence, logging a warning at boot. Set FAIRLINE_ENV=production to turn that fallback into a boot failure; a deployment that silently drops pick history has no CLV record.

Create a user and API key:

python -m fairline create-user --email you@example.com

The key is printed once; only its SHA-256 hash is stored. Identity comes from the key on every request, so there is no user_id field anywhere in the API. Without DATABASE_URL (local demo), requests run as a fixed demo principal and no key is needed.

Start a picks run:

curl -s -X POST http://localhost:8000/api/runs \
  -H "Authorization: Bearer $FAIRLINE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"sport": "americanfootball_nfl"}' | jq

Approve candidates:

curl -s -X POST http://localhost:8000/api/runs/{run_id}/approve \
  -H "Authorization: Bearer $FAIRLINE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"approved_pick_ids": ["pick-uuid-1", "pick-uuid-2"]}' | jq

Tracing

Add your LangSmith key to .env to enable run tracing:

LANGCHAIN_API_KEY=lsv2_pt_...
LANGCHAIN_TRACING_V2=true
LANGCHAIN_PROJECT=fairline

Each picks run produces two traces in the LangSmith UI:

  • picks/americanfootball_nfl/2026-01-15 -- covers odds fetch, fair-line derivation, Claude pick generation, and the HITL pause
  • approve/<run_id[:8]> -- covers the resume and pick persistence

Tracing is optional. The service starts and runs normally without LANGCHAIN_API_KEY set.

Run tests (no API keys needed):

pip install -e ".[dev]"
pytest -q

The dashboard

Three server-rendered pages, styled as a ledger: dark ink canvas, every figure in tabular monospace, no gradients, no framework (Jinja2 plus htmx over the existing FastAPI).

  • / is the agent leaderboard: each agent's record, average CLV, and units. The front page is the audit, not a pitch.
  • /queue is the review desk: pending steam and matchup candidates with their edge math and rationale, approve or pass inline (approvals send your API key from the header field).
  • /record is every settled pick with result, CLV, and units, plus the running totals. Nothing on it can be edited or backfilled.

Read pages are public by design: a graded record behind a login is marketing, in the open it is evidence. Mutations still require a bearer key.

Without an Odds API key there are no picks to show, so all three pages render empty. python -m fairline seed-demo writes a fixed set of settled picks and pending candidates for that case. The fixtures carry only the price taken and the price at close; CLV and units come from the same functions settlement and grading use, so the arithmetic on the record page is the real implementation rather than a table of chosen results. The seeded set settles 7-4-1, and three of the four losses beat the close, which is the reason this grades on CLV at all.

The pages raise a sample-data banner for as long as those rows are present, with no configuration needed. FAIRLINE_DEMO_DATA=true keeps it up afterwards, for a deployment that seeded once and now takes real picks. The flag can only add the banner, never remove it: a seeded record served without a label is a performance claim that is not true, and that is what an unset variable would otherwise default to.

Running a season

The track record only accumulates while the jobs run. Everything is a CLI subcommand designed for cron: watch exits when no games are near, settle and grade are idempotent, and backfill-nfl merges by game id. One-time setup, then the crontab below:

python -m fairline backfill-nfl --seasons 2023 2024 2025
# All times UTC. DATABASE_URL, ODDS_API_KEY, FAIRLINE_WEBHOOK_URL in the cron environment.

# Line snapshots + steam alerts, every 10 min through US game hours (noon to midnight ET)
*/10 16-23,0-4 * * *  python -m fairline watch --once --sport all

# Closing-line capture, every 30 min in the same window
0,30 16-23,0-4 * * *  python -m fairline settle --sport all

# Grade results and record game outcomes, twice daily (scores feed reaches 3 days back)
0 10,15 * * *         python -m fairline grade --sport all

# Refresh NFL results and closing lines from nflverse, Tuesdays
0 12 * * 2            python -m fairline backfill-nfl --seasons 2025

Quota math at this cadence, since every poll costs one request per league: watch runs 78 cycles a day at 4 requests each (about 9,400/month), settle adds about 3,100/month, grade about 250/month. Roughly 12,700 requests/month total, which fits The Odds API's 20K paid tier (about $30/month) with headroom and does not fit the free tier. A free-tier plan exists but is NFL-only and manual: grade daily and settle around Sunday, Monday, and Thursday kickoffs, no watch, about 250 requests/month.

Picks runs stay human-triggered by design: cron can start a run (POST /api/runs), but candidates wait at the HITL checkpoint until someone approves them, so scheduling run creation only makes sense if the reviewer shows up.


Eval harness

eval/dataset.jsonl contains 18 golden test cases that verify the deterministic math layer independently of the LLM. Run without API keys:

pip install -e ".[dev]"
python -m eval

Output:

vig_removal          5/5  [#####]
ev_calculation       4/4  [####]
clv_calculation      3/3  [###]
edge_filter          3/3  [###]
structural           3/3  [###]

Total: 18/18 passed  pass rate: 100.0%

To write a JSON report:

python -m eval --out eval/report.json

What is tested: vig removal accuracy (symmetric and asymmetric markets), EV formula correctness for favorites and underdogs, CLV sign convention (positive = beat the closing line), edge filter threshold compliance, and pick structural validity (required fields, confidence enum, non-negative edge). The harness imports directly from fairline.state so any change to the production math functions fails the eval immediately.


Known limitations

  1. Rate limiting is per-instance. There is no shared Redis counter across multiple app replicas. Fine for the current demo scale; documented trade-off.

  2. Off-season returns empty. The Odds API returns no NFL games May through July. The /api/runs endpoint returns an empty candidates list rather than an error, which is correct but may confuse first-time callers.

  3. Closing line is a near-kickoff snapshot, not the true close. The free Odds API tier has no historical endpoint, so python -m fairline settle records whatever Pinnacle shows when it runs. If the job does not run inside its window before kickoff, clv stays null for those picks; there is no backfill. Line movement in the final seconds before kickoff is also invisible to a snapshot taken minutes earlier.

  4. The clv number ignores point drift. A spread bet at -3.5 that closes at -4.0 is compared by price only. The closing line is stored in closing_point, so the drift is queryable (SELECT selection, closing_point FROM picks), but it is not folded into the single clv float; converting a half point of NFL spread movement to probability requires a push-chart model that is out of scope.

  5. Grading has a 3-day window. The scores endpoint reaches back at most 3 days. A pick whose game finished more than 3 days before fairline grade runs stays ungraded permanently; run it at least twice a week during the season.

  6. API keys are minimal. One key per user, no scopes, no expiry, rotation only via create-user re-run. Lookups compare SHA-256 hashes through a unique index; there is no per-key rate limiting, so a leaked key is fully capable until rotated.

  7. MemorySaver in tests. Run status and ownership now live in the runs table and survive restarts, but graph state (candidates awaiting approval) still uses MemorySaver in local dev. Production requires PostgresSaver for the HITL pause itself to survive a restart; the switchover is a one-line change in graph.py.

  8. The sim weight is static. FAIRLINE_SIM_WEIGHT applies uniformly to every pick and only changes when an operator reads sim-report and decides to change it. A calibrated system would fit the weight from the disagreement-bucket CLV; here that judgment is deliberately left to the human.

  9. BALLDONTLIE's free tier caps at 5 requests/minute. Stats fetch failures, including rate-limit responses, degrade to skipping that team's stats for the run rather than blocking picks.

  10. Player-level stats are fetched for NFL and MLB only. NBA's and NHL's player endpoints require a player-ID roster lookup fairline doesn't have yet.

  11. Player-level stats land in state but nothing consumes them for prop-market picks yet. fairline's automated pipeline only bets moneyline/spread/total today.

  12. MLB batter props only. Pitcher-strikeout props need a separate pitcher-grouped aggregation not built yet.

  13. The vs-specific-pitcher split is now wired into live picking via MLB's official Stats API, but probable starters are frequently unannounced more than a day or two out, and can change after being announced (a scratched start). vs_pitcher reflects whatever the schedule API has listed at the moment a candidate is generated, which may differ from the pitcher who actually takes the mound.

  14. Park factors are a static table from FanGraphs' 2025 season data, refreshed by hand once a year, not a live feed.

  15. pybaseball has no documented API contract. It scrapes Baseball Reference, Baseball Savant, and FanGraphs directly and can break if those sites change their HTML structure.

  16. Teammate-out correlated splits are not implemented. They need a lineups/roster source this codebase doesn't have.

  17. The weather split is historical only (how has this player performed in past bad-weather games), not a forecast for the specific upcoming game; wiring a live forecast into the CLI matchup path is future work.

  18. Primetime detection uses a fixed Sunday-kickoff-hour heuristic (19:00 or later) plus Monday/Thursday. A schedule change outside this heuristic, a Saturday game, won't get flagged as primetime.

  19. "Bad weather" is a fixed wind/temperature heuristic (wind >= 15 mph or temp <= 32F), not a graded scale. Games with no recorded temp or wind, typically domes, are excluded from the weather split rather than assumed good weather.

  20. The vs-specific-goalie split is computed and tested but not wired into live picking. It needs the opposing starting goalie for an upcoming game, which isn't confirmed until pre-game, the same limitation MLB's vs-pitcher split had before its Stats API wiring.

  21. No day/night split exists for NHL. Every game is played indoors, so the dimension carries no information.

  22. Rest days are derived from consecutive schedule dates, not a field the NHL's API provides directly. A team's first backfilled game in a date range has no rest-days value.

  23. The NHL's official API has no published OpenAPI spec or documented stability guarantee. Every field this integration reads was verified against a real live response during development, not assumed from third-party documentation.

  24. stats.nba.com (nba_api's data source) is documented to block requests from cloud/datacenter IPs. The backfill command may not run reliably from a typical cloud host. No free, reliable mitigation exists; a residential-IP relay or a non-hyperscaler VPS is the realistic option, passed via the --proxy flag.

  25. No vs-specific-defender split exists for NBA, unlike MLB's vs-pitcher and NHL's vs-goalie. No verified per-game defender-matchup data source was found; the one endpoint that identifies who guarded whom is season-aggregate only.

  26. No day/night or surface split exists for NBA; every game is played indoors on a standardized court.

  27. Defense-vs-position uses broad position buckets (Guard/Forward/Center), not the granular PG/SG/SF/PF/C breakdown; nba_api's own position data doesn't expose that level of detail.

  28. Position data comes from a season's current roster snapshot (CommonTeamRoster), not a per-game record. A player who changed teams or position designation mid-season may have a stale bucket until the next backfill.

  29. A player-name mismatch between the box-score feed and the roster feed leaves that player's position unresolved (None). The position_matchup split is silently omitted for their picks rather than guessed.

  30. Rest days are derived from consecutive game dates, not a field nba_api provides directly.

  31. The spend cap's ledger is in-process, so N replicas can spend N times the budget. generation_cap is a module-level CostCap holding its history in memory. One container is one budget; two containers are two. Nothing persists it across restarts either, so a crash loop resets the month's spend to zero. The absolute per-call ceiling is what actually bounds the damage in those cases.

  32. The pre-call estimate counts four characters per token. Asking the API to count tokens properly costs a request of its own, on the path whose whole job is to avoid unnecessary requests. The estimate also assumes the model fills max_tokens when a typical answer uses about a quarter of that, so it reads high by design. That gap is why the rolling-median multiplier is 8 and not the 3 used elsewhere: measured on a real slate, a worst-case estimate judged against a median of real completions looks like a 4.5x anomaly every time, and at 3 the agent refuses from its sixth run on with nothing in the logs to explain why.

  33. NFL's position_matchup split is bucketed by the player's own most-recently-recorded PlayerGame.position, not by which prop market is being bet. A traditional RB used as a receiver on a given slate is still pooled with running backs, not wide receivers, since nflverse's own position label is what gets stored.

About

Agentic NFL betting research service: sharp-line vig removal, LangGraph HITL approval, CLV tracking

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages