Documentation
¶
Overview ¶
Package leadscore is the public surface of the leadscore outbound engine: the plug-in interfaces, shared types, errors, the adapter registry, Run and Main.
Built-in adapters register in their init functions. A custom build imports its own adapter package (which registers itself) and calls Main.
Every name here is defined in internal/api and re-exported as a type alias or a thin wrapper, so the engine can use the same types without importing this package. The doc comments below repeat the public API's contract. This surface is frozen at v0.1.0: interfaces never gain methods; a new capability is a separate optional interface found by type assertion.
Index ¶
- Constants
- Variables
- func Main()
- func RegisterBackend(typ string, f func(Config) (Backend, EventLog, error))
- func RegisterDetector(kind string, f func(params Config) (Detector, error))
- func RegisterEnricher(typ string, f func(Config) (Enricher, error))
- func RegisterLookup(typ string, f func(Config) (Lookup, error))
- func RegisterPoller(typ string, f func(Config) (Poller, error))
- func RegisterSink(typ string, f func(Config) (Sink, error))
- func RegisterSource(typ string, f func(Config) (Source, error))
- type Backend
- type CompanyFacts
- type Config
- type Cursor
- type Detector
- type Enricher
- type Event
- type EventID
- type EventLog
- type InputRow
- type LeadID
- type LeadRef
- type LeaseInspector
- type LedgerRef
- type Lookup
- type Poller
- type RawEvent
- type Row
- type RunLease
- type RunOptions
- type RunResult
- type Sink
- type Source
- type StepKey
- type StepRequest
- type Subject
- type TableWrite
- type Verdict
- type WriteOp
Constants ¶
const ( OpReplace = api.OpReplace // rewrite the whole table OpAppend = api.OpAppend // add rows; on a keyed store table, a key the table already holds fails the commit OpUpsert = api.OpUpsert // insert or update by Key columns OpDelete = api.OpDelete // delete rows matching Key columns OpTrim = api.OpTrim // delete rows whose Column is before Before )
The WriteOp values.
Variables ¶
var ( // ErrRateLimited: the step stays pending, no attempt is counted, and this // sink stops for the run. An Enricher also returns it on a rate limit. ErrRateLimited = api.ErrRateLimited // ErrTransient: the step stays pending and no attempt is counted. ErrTransient = api.ErrTransient // ErrRefused: the vendor said no for a reason retrying cannot change. ErrRefused = api.ErrRefused )
Sink errors. Classify with errors.Is. Any other error counts one attempt toward `failed`. None of these says the vendor did nothing; the ledger records whether the call went out.
var ( // ErrTooLarge: Commit would exceed the store's limit; the store does not split. ErrTooLarge = api.ErrTooLarge // ErrCommittedWithProblems: Commit saved every write, but a people-owned // table needs a person to look at it; treat the commit as done, never resend. ErrCommittedWithProblems = api.ErrCommittedWithProblems // ErrLeaseHeld: another owner holds a live run lease. ErrLeaseHeld = api.ErrLeaseHeld // ErrLeaseLost: the lease expired or another owner took it. ErrLeaseLost = api.ErrLeaseLost // ErrEventsShrank: the event log shrank below a saved cursor; the engine // scores but does not push. ErrEventsShrank = api.ErrEventsShrank )
Store errors.
Functions ¶
func RegisterBackend ¶
RegisterBackend registers a store factory under a `store.type`. It panics on an empty type, a nil factory, or a type already registered.
func RegisterDetector ¶
RegisterDetector registers a Detector factory under a rubric detector kind; it receives the detector's `params:`. The kind is stored lowercased, since rubric kinds compare lowercased. It panics on an empty kind, a nil factory, or a kind already registered (two kinds differing only in case collide).
func RegisterEnricher ¶
RegisterEnricher registers an Enricher factory under a config `type:`. It panics on an empty type, a nil factory, or a type already registered.
func RegisterLookup ¶
RegisterLookup registers a Lookup factory under a sink type. It panics on an empty type, a nil factory, or a type already registered.
func RegisterPoller ¶
RegisterPoller registers a Poller factory under a sink type. It panics on an empty type, a nil factory, or a type already registered.
func RegisterSink ¶
RegisterSink registers a Sink factory under a sink type. It panics on an empty type, a nil factory, or a type already registered.
Types ¶
type Backend ¶
Backend is table-level storage.
ReadTable returns every row of a table; a missing table returns no rows and no error. Lease takes the run lease or returns ErrLeaseHeld; an expired lease is taken over; it must be a real compare-and-swap, so two callers can never both hold it. Commit applies every write all-or-nothing (one Sheets batchUpdate, one SQL transaction); it rejects an OpUpsert or OpDelete with no Key before applying anything; an OpAppend to a keyed store table of a key the table already holds (or that the same commit already wrote) fails the whole commit; it creates a missing table, and appends a missing column, the first time a write names it; it returns ErrTooLarge rather than splitting.
type CompanyFacts ¶
type CompanyFacts = api.CompanyFacts
CompanyFacts is what an enricher learned about a company domain. Region is the vendor's country, trimmed, with no bucketing. FundingStage is one of the known funding stages, or empty. Employees is nil when unknown. Extra holds other facts; the Apollo enricher writes latest_funding_at. An empty Extra value means the vendor sent a value that cannot be used, and clears that fact. NotFound means the vendor had no record (retried after max age).
type Config ¶
Config is an adapter's block from leadscore.yml: a source gets its sources[] entry, the enricher gets enrich, the store gets store, a sink of type T gets sinks.T.
type Detector ¶
Detector decides whether a signal fired. Evaluate gets the subject's window events: a lead's own, or for a company every event whose domain is the company, lead events included.
type Enricher ¶
Enricher returns company facts. Enrich calls domains in the order given and stops on a rate limit, when it returns the facts so far and ErrRateLimited. A failure on one domain is skipped; a failure every later domain would likely share (the key refused, ctx done, three failures in a row) also stops it, returning the facts so far and that error. The caller counts calls made as the index of the last domain tried, plus one. The engine calls it one domain at a time.
type Event ¶
Event is something that happened to a person or a company. ID is empty from sources (the engine sets it). Kind is a known event kind, or empty with Attrs["reject"] set for a rejected source row. Email and LinkedInURL are the person keys, both empty for a company-only event. Domain is the person's employer domain, or the company for a company-only event. At and ReceivedAt are UTC. Origin is "receiver", "polling", "hubspot", "apollo_lookup", or a source id. Attrs is kind-specific: stage, label, message_id, page, deal_id, contact_id, full_name, title, company, reject.
type EventLog ¶
EventLog is the store's append-only log of raw receiver requests.
AppendEvents stores a batch all-or-nothing, in order, and returns only once it is durable; it retries the store's "slow down" answers until ctx is done. ReadEvents returns events after cursor and the next cursor: an event not yet returned is returned by a later read from the saved cursor, and sequence numbers are never reused, even after deletion; it returns ErrEventsShrank when a partition named in the cursor is missing or holds fewer rows than the cursor says were read. DeleteProcessed removes events at or below committed that are older than olderThan and returns committed with any deleted partition dropped; the engine saves the returned cursor.
type InputRow ¶
InputRow is one row from a source: its source id, its headers in file order as written, and every column as raw text keyed by header as written. The engine, not the source, applies aliases and computes the per-row id.
type LeadRef ¶
LeadRef is what engines and adapters see of a lead. Emails holds every email for the lead and every lead merged into it, primary first. Domain is the company domain, empty when the lead has none. Status is the folded status. Fields holds merged fields by resolved name. SourcesSeen counts distinct channels. ConflictFields are fields whose sources disagreed. CompanyDealID is the stored open or won deal at the lead's company, if any. Verdict is nil before scoring. Done is set for Lookup calls only: the lookup's sink's done steps for the lead and every lead merged into it.
type LeaseInspector ¶
type LeaseInspector = api.LeaseInspector
LeaseInspector is optional on a Backend; doctor uses it to show the lease without taking it.
type LedgerRef ¶
LedgerRef is one push-ledger step: key, destination, vendor id and state (pending, done, failed, cancelled).
type Lookup ¶
Lookup checks leads just before pushing, at run step 8, on every email in LeadRef.Emails. failed names leads whose lookup failed; the engine blocks those leads. err means the whole lookup failed.
type Poller ¶
Poller reads outcomes on a schedule (Apollo reply polling), at run step 3. since is the start of the window to read; the poller uses it as given.
type RawEvent ¶
RawEvent is one stored receiver request. Seq is assigned by the store on append (empty when appending) and is a complete resume cursor: reading from it returns exactly the events after this one, across every partition the store keeps. Body has the secret already removed.
type RunLease ¶
RunLease is a held lease. Check returns ErrLeaseLost if the lease expired or another owner took it. Release gives the lease up only if this owner still holds it.
type RunOptions ¶
type RunOptions = api.RunOptions
RunOptions configures Run. ConfigPath is leadscore.yml or a hosted bundle (with a bundle, RubricPath is ignored); empty means the default locations. Closing Stop stops new vendor calls and the run saves within its budget (the graceful stop); cancelling ctx is the hard stop.
type RunResult ¶
RunResult is a run's outcome. Healthy is a run that finished, or skipped because another run held the lease (Skipped). Problems are the open problems written to Health. Pushed is the number of pushes made this run.
func Run ¶
func Run(ctx context.Context, opts RunOptions) (RunResult, error)
Run executes one run, as `leadscore run` does, with the production hooks. It prints the run's summary line (and, on a dry run, the report) to stdout. The error is non-nil when the run failed; a run that finished unhealthy returns a nil error with Healthy false.
type Sink ¶
Sink pushes leads to a vendor. Steps returns the ordered steps for one destination. Do must be find-or-create by req.Key: calling it twice with the same key leaves one vendor-side object. Classify its errors with ErrRateLimited, ErrTransient and ErrRefused.
type Source ¶
Source returns input rows and events. Fetch returns rows and events after cursor; snapshot sources (CSV, Sheet tabs) ignore the cursor and return everything every run.
type StepKey ¶
StepKey names one push step: the lead, the lane's stable id from the rubric, and one of the sink's Steps(dest).
type StepRequest ¶
type StepRequest = api.StepRequest
StepRequest is one call to Sink.Do. Dest is the part after "<sink>:" in the lane's push target; Prior holds vendor ids from this push's earlier steps, by step name; Related holds this sink's done steps for other leads at the same company, any lane, including steps finished earlier in this batch (only deals at an open stage).
type TableWrite ¶
type TableWrite = api.TableWrite
TableWrite is one write in a Backend.Commit. Table is the store table's name exactly; Key names the columns for OpUpsert and OpDelete (required for both); Column and Before are for OpTrim only.
Directories
¶
| Path | Synopsis |
|---|---|
|
adapters
|
|
|
apollo
Package apollo is the Apollo adapter.
|
Package apollo is the Apollo adapter. |
|
csv
Package csv is the CSV file source.
|
Package csv is the CSV file source. |
|
hubspot
Package hubspot is the HubSpot adapter: the `hubspot` sink, which finds or creates contacts and one deal per company; the HubSpot Lookup, which reads opt-outs and each company's deals before pushing; `leadscore setup hubspot`; and the `hubspot` check.
|
Package hubspot is the HubSpot adapter: the `hubspot` sink, which finds or creates contacts and one deal per company; the HubSpot Lookup, which reads opt-outs and each company's deals before pushing; `leadscore setup hubspot`; and the `hubspot` check. |
|
sheetsource
Package sheetsource is the Google Sheet tab source (leadscore.yml's `sources[]`): it reads people-owned tabs of the team's spreadsheet as they appear on screen (FORMATTED_VALUE), headers in order as written, one InputRow per non-blank row.
|
Package sheetsource is the Google Sheet tab source (leadscore.yml's `sources[]`): it reads people-owned tabs of the team's spreadsheet as they appear on screen (FORMATTED_VALUE), headers in order as written, one InputRow per non-blank row. |
|
cmd
|
|
|
leadscore
command
Command leadscore is the release binary: the built-in adapters plus the CLI.
|
Command leadscore is the release binary: the built-in adapters plus the CLI. |
|
internal
|
|
|
api
Package api defines the public surface of leadscore: its types, interfaces, errors and adapter registry, plus the built-in header alias table.
|
Package api defines the public surface of leadscore: its types, interfaces, errors and adapter registry, plus the built-in header alias table. |
|
check
Package check is the framework for doctor checks.
|
Package check is the framework for doctor checks. |
|
cli
Package cli is the leadscore command line.
|
Package cli is the leadscore command line. |
|
config
Package config loads leadscore.yml: every engine key with its default, the adapter blocks passed through as api.Config, the default file locations, the hosted bundle, and the rubric path.
|
Package config loads leadscore.yml: every engine key with its default, the adapter blocks passed through as api.Config, the default file locations, the hosted bundle, and the rubric path. |
|
csvsafe
Package csvsafe makes text safe to write into a CSV a person opens in a spreadsheet: a cell a spreadsheet would read as a formula is prefixed with a quote, so stored text from a lead sheet can never run as one.
|
Package csvsafe makes text safe to write into a CSV a person opens in a spreadsheet: a cell a spreadsheet would read as a formula is prefixed with a quote, so stored text from a lead sheet can never run as one. |
|
detect
Package detect evaluates the rubric's detectors: the built-in kinds count_in_window, first_seen and change, and kinds a build registers, over each lead's and each company's Window events.
|
Package detect evaluates the rubric's detectors: the built-in kinds count_in_window, first_seen and change, and kinds a build registers, over each lead's and each company's Window events. |
|
duration
Package duration reads the durations leadscore.yml and the rubric use.
|
Package duration reads the durations leadscore.yml and the rubric use. |
|
engine
Package engine is the run: one execution of the loop, from reading leadscore.yml to writing Ranked.
|
Package engine is the run: one execution of the loop, from reading leadscore.yml to writing Ranked. |
|
events
Package events holds the event rules: the de-duplication key of every event, parsing stored receiver requests, and applying an event's effect to a lead's Outcomes and People.
|
Package events holds the event rules: the de-duplication key of every event, parsing stored receiver requests, and applying an event's effect to a lead's Outcomes and People. |
|
fakes/apollo
Package apollo is a fake of the Apollo API calls leadscore makes, served from the fixtures in testdata/vendors/apollo (provisional until real captured calls replace them).
|
Package apollo is a fake of the Apollo API calls leadscore makes, served from the fixtures in testdata/vendors/apollo (provisional until real captured calls replace them). |
|
fakes/gcp
Package gcp is an in-memory fake of the Google Cloud calls internal/hosting makes, served under one base URL with each API under its own prefix (/secretmanager, /run, /cloudscheduler, /artifactregistry), as hosting.Connect expects with `base_url`.
|
Package gcp is an in-memory fake of the Google Cloud calls internal/hosting makes, served under one base URL with each API under its own prefix (/secretmanager, /run, /cloudscheduler, /artifactregistry), as hosting.Connect expects with `base_url`. |
|
fakes/gcs
Package gcs is an in-memory fake of the Cloud Storage JSON API calls the Sheets store makes for its lease file: bucket get, object get (metadata or media), multipart and media upload, delete, and testIamPermissions on a bucket (the doctor's lease check).
|
Package gcs is an in-memory fake of the Cloud Storage JSON API calls the Sheets store makes for its lease file: bucket get, object get (metadata or media), multipart and media upload, delete, and testIamPermissions on a bucket (the doctor's lease check). |
|
fakes/hubspot
Package hubspot is a fake HubSpot portal for tests: the CRM calls the HubSpot adapter makes (contacts, deals, companies, v4 associations, pipelines, properties, the token-info call), backed by in-memory records.
|
Package hubspot is a fake HubSpot portal for tests: the CRM calls the HubSpot adapter makes (contacts, deals, companies, v4 associations, pipelines, properties, the token-info call), backed by in-memory records. |
|
fakes/sheets
Package sheets is an in-memory fake of the parts of the Google Sheets v4 and Drive v3 APIs that leadscore uses: spreadsheet create and get, batchUpdate with the requests the Sheets store sends, values get and batchGet, and Drive permissions.
|
Package sheets is an in-memory fake of the parts of the Google Sheets v4 and Drive v3 APIs that leadscore uses: spreadsheet create and get, batchUpdate with the requests the Sheets store sends, values get and batchGet, and Drive permissions. |
|
fakes/sink
Package sink is an in-memory fake vendor for tests: a find-or-create Sink and a Lookup over one shared state, so the push loop, the pre-push lookups and the sinktest suite run with no vendor account.
|
Package sink is an in-memory fake vendor for tests: a find-or-create Sink and a Lookup over one shared state, so the push loop, the pre-push lookups and the sinktest suite run with no vendor account. |
|
hosting
Package hosting is the Google Cloud side of leadscore: the fixed resource names, the schedule's cron form, Secret Manager reads and writes (`config push` and the keys a local command reads on a hosted install), and the reads of the Cloud Run service and job, the Cloud Scheduler job and the Artifact Registry proxy that the `hosting` check makes.
|
Package hosting is the Google Cloud side of leadscore: the fixed resource names, the schedule's cron form, Secret Manager reads and writes (`config push` and the keys a local command reads on a hosted install), and the reads of the Cloud Run service and job, the Cloud Scheduler job and the Artifact Registry proxy that the `hosting` check makes. |
|
logredact
Package logredact masks emails, secrets and home-directory paths in log text, and summarises vendor error bodies without echoing the personal data a vendor sends back.
|
Package logredact masks emails, secrets and home-directory paths in log text, and summarises vendor error bodies without echoing the personal data a vendor sends back. |
|
merge
Package merge turns input rows from many sources into one person per human (merge owns persons, aliases and row ids).
|
Package merge turns input rows from many sources into one person per human (merge owns persons, aliases and row ids). |
|
model
Package model holds the in-memory model of the store's tables: one Go struct per store table, keyed tables in maps by primary key, keyless tables (Overrides, Log) as ordered slices.
|
Package model holds the in-memory model of the store's tables: one Go struct per store table, keyed tables in maps by primary key, keyless tables (Overrides, Log) as ordered slices. |
|
receiver
Package receiver is `leadscore serve`: the HTTP receiver for Apollo workflow requests, /healthz, and on Docker the run timer.
|
Package receiver is `leadscore serve`: the HTTP receiver for Apollo workflow requests, /healthz, and on Docker the run timer. |
|
receiver/auth
Package auth verifies that an inbound webhook came from the holder of the receiver's secret.
|
Package auth verifies that an inbound webhook came from the holder of the receiver's secret. |
|
rules
Package rules is the rubric compiler and evaluator.
|
Package rules is the rubric compiler and evaluator. |
|
store/codec
Package codec maps the in-memory model to store tables: Encode turns recorded changes into table writes, Load reads a store into a model and checks its schema version.
|
Package codec maps the in-memory model to store tables: Encode turns recorded changes into table writes, Load reads a store into a model and checks its schema version. |
|
store/sheets
Package sheets is the built-in Google Sheets store: a Backend and EventLog over one spreadsheet, one tab per table, and a run lease held as a file in a Cloud Storage bucket.
|
Package sheets is the built-in Google Sheets store: a Backend and EventLog over one spreadsheet, one tab per table, and a run lease held as a file in a Cloud Storage bucket. |
|
store/sqlite
Package sqlite is the built-in SQLite store: a Backend and EventLog over one database file in WAL mode with a busy timeout, so `leadscore serve` can append events during a run.
|
Package sqlite is the built-in SQLite store: a Backend and EventLog over one database file in WAL mode with a busy timeout, so `leadscore serve` can append events during a run. |
|
vendorhttp
Package vendorhttp holds the HTTP rules the vendor clients share, so Apollo's and HubSpot's cannot drift apart: the test keys, no redirects, a per-call timeout, a size cap on the reply, transport errors without the URL, and the error classes.
|
Package vendorhttp holds the HTTP rules the vendor clients share, so Apollo's and HubSpot's cannot drift apart: the test keys, no redirects, a per-call timeout, a size cap on the reply, transport errors without the URL, and the error classes. |
|
Package sinktest is the conformance suite a plug-in sink runs against itself: every sink is find-or-create by its step key, so a step replayed after a crash, or called twice, leaves one vendor-side object, and each kind of vendor failure maps to its error.
|
Package sinktest is the conformance suite a plug-in sink runs against itself: every sink is find-or-create by its step key, so a step replayed after a crash, or called twice, leaves one vendor-side object, and each kind of vendor failure maps to its error. |
|
Package storetest is the conformance suite a plug-in store runs against itself.
|
Package storetest is the conformance suite a plug-in store runs against itself. |