Help us shape the future of Go! Take the 2026 Go Developer Survey

jetkvm

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 22, 2026 License: MIT Imports: 36 Imported by: 0

Documentation

Overview

Package jetkvm is a browser-free client and session core for a JetKVM device: local (non-cloud) authentication, WebRTC signaling, video capture, JSON-RPC status/wheel calls and (opt-in, gated) HID control.

It deliberately does not implement or depend on anything from the jetkvm/kvm cloud relay, OIDC/Google login, virtual media, ATX/power control, firmware update, or any other device-administration RPC method - only what's needed for read-only inspection and (when explicitly enabled) keyboard/mouse input.

Index

Constants

View Source 
const (
	// EvidenceCommit is the jetkvm/kvm commit this client's protocol
	// assumptions were read from.
	EvidenceCommit = "b3c29a4"

	// EvidenceBranch is the branch EvidenceCommit was read from.
	EvidenceBranch = "dev"
)

Protocol/firmware compatibility evidence.

This client was built against github.com/jetkvm/kvm, dev branch, commit b3c29a4 ("Set video codec before native stream start (#1509)"), cloned and read on 2026-08-02. Nothing here is a published, versioned API: jetkvm/kvm ships no protocol spec, only a Go implementation of both server and reference (browser) client. These constants exist so a version drift on the device produces an actionable error instead of a silent hang or confusing low-level failure.

View Source 
const (
	BoundaryNone          = "none: a frame was captured"
	BoundaryTransport     = "transport: peer connection never reached connected"
	BoundaryAnswer        = "signaling: no SDP answer was applied"
	BoundaryNegotiation   = "negotiation: no video track (OnTrack never fired)"
	BoundaryTrackRejected = "negotiation: video track rejected by this client"
	BoundaryNoRTP         = "media: video track negotiated but zero RTP packets arrived"
	BoundaryRTPStalled    = "media: RTP arrived then stopped"
	BoundaryDepacketize   = "depacketization: RTP arrived but no access unit was assembled"
	BoundaryFragmentation = "depacketization: fragmented/aggregated NAL units were not reassembled"
	BoundaryReassembly    = "depacketization: the device sent IDR packets but no IDR access unit was reassembled"
	BoundaryNoKeyframe    = "encoder: access units arrived but the device never sent an IDR keyframe"
	BoundaryNoParamSets   = "encoder: IDR arrived but SPS/PPS parameter sets never did"
	BoundaryDecode        = "decoder: a frame was assembled but decoding failed"
	BoundaryUndetermined  = "undetermined"
)

Failure boundaries. These are the distinct stages a screenshot can die at; the whole point of this file is that the next live run reports exactly one of them instead of a single undifferentiated timeout.

View Source 
const (
	ModifierLeftControl = 1 << iota
	ModifierLeftShift
	ModifierLeftAlt
	ModifierLeftMeta
	ModifierRightControl
	ModifierRightShift
	ModifierRightAlt
	ModifierRightMeta
)

USB boot-keyboard modifier bits used by the named combo registry.

View Source 
const (
	KeyUsageC      = 0x06
	KeyUsageE      = 0x08
	KeyUsageM      = 0x10
	KeyUsageR      = 0x15
	KeyUsageT      = 0x17
	KeyUsageV      = 0x19
	KeyUsageZ      = 0x1d
	KeyUsageEnter  = 0x28
	KeyUsageEscape = 0x29
	KeyUsageTab    = 0x2b
	KeyUsageSpace  = 0x2c
	KeyUsageDelete = 0x4c
)

USB HID keyboard usage codes used by the named combo registry.

View Source 
const (
	MouseButtonLeft byte = 1 << iota
	MouseButtonRight
	MouseButtonMiddle
)

USB HID mouse-button bits used by the named mouse-button resolver.

View Source 
const (
	// MaxTypeStringRunes bounds one text-entry operation so a caller cannot
	// queue an unbounded sequence of live keyboard input.
	MaxTypeStringRunes = 4096

	// DefaultTypeDelayMS and MaxTypeDelayMS define the shared MCP/CLI
	// inter-key delay contract.
	DefaultTypeDelayMS = 0
	MaxTypeDelayMS     = 500
)
View Source 
const (
	// DefaultWaitForTextInterval is the minimum gap between the starts of two
	// successive screenshot/OCR polls when Interval is omitted.
	DefaultWaitForTextInterval = 500 * time.Millisecond

	// DefaultWaitForTextTimeout bounds a complete wait when Timeout is omitted.
	DefaultWaitForTextTimeout = 10 * time.Second

	MinWaitForTextInterval = 100 * time.Millisecond
	MaxWaitForTextInterval = 10 * time.Second
	MinWaitForTextTimeout  = 100 * time.Millisecond
	MaxWaitForTextTimeout  = 5 * time.Minute

	// MaxWaitForTextTextRunes bounds both literal needles and RE2 patterns.
	MaxWaitForTextTextRunes = 4096
)
View Source 
const (
	// DefaultWaitStableThreshold is the maximum fraction of pixels that may
	// change between two consecutive frames for that comparison to count as
	// stable.
	DefaultWaitStableThreshold = 0.01

	// DefaultWaitStableFrames is the number of consecutive stable comparisons
	// required before WaitStable reports that the screen has settled.
	DefaultWaitStableFrames = 2

	// MaxWaitStableFrames keeps all public adapters within the signed 32-bit
	// integer range, avoiding architecture-dependent typed decode overflow.
	MaxWaitStableFrames = 1<<31 - 1

	// DefaultWaitStablePollInterval is the minimum gap between the starts of
	// two successive fresh-frame polls.
	DefaultWaitStablePollInterval = 250 * time.Millisecond
)
View Source 
const DefaultControlLeaseTimeout = 30 * time.Second

DefaultControlLeaseTimeout bounds how long a caller may hold the control lease without releasing it. A caller that simply stops calling us (crash, hang, forgotten Release) therefore triggers a bounded neutralization attempt; failure remains explicit because no client can guarantee attached-host state.

View Source 
const MaxAbsoluteCoordinate = hidproto.MaxAbsoluteCoordinate

MaxAbsoluteCoordinate is the inclusive coordinate bound supported by the JetKVM absolute-pointer HID report.

View Source 
const MaxDragSteps = 256

MaxDragSteps bounds the number of optional intermediate pointer reports in one drag gesture. The start, destination, and release reports are additional to this count.

View Source 
const MaxHoldMS = 5000

MaxHoldMS bounds how long one hold-key operation may keep a keyboard chord pressed. It stays below both the default 10-second operation deadline and the 30-second control-lease watchdog, leaving time for connection setup and terminal neutralization.

View Source 
const MaxKeyComboNameRunes = 64

MaxKeyComboNameRunes bounds caller-controlled combo names before normalization allocates a lower-cased, tokenized copy. It applies to each entry of a key sequence as well as to a standalone combo.

View Source 
const MaxKeySequenceLength = 64

MaxKeySequenceLength bounds one ordered key-sequence operation so a caller cannot queue an unbounded series of live keyboard chords.

View Source 
const MaxPointerButtonMask = hidproto.MaxAbsoluteButtonMask

MaxPointerButtonMask is the inclusive five-bit button-mask bound supported by the JetKVM absolute-pointer HID report. Bits 5 through 7 are constant padding in the firmware descriptor, not additional buttons.

View Source 
const (

	// MaxScreenshotEncodedBytes bounds both FFmpeg's PNG stdout and every
	// in-process screenshot re-encode. Four bytes per pixel plus two MiB of
	// container/compression overhead comfortably covers incompressible RGBA
	// PNG data at the pixel cap while leaving normal 1080p and 4K captures
	// unchanged.
	MaxScreenshotEncodedBytes = maxScreenshotPixels*4 + 2*1024*1024
)

Screenshot decode and encode limits. They are deliberately independent of the compressed H.264 access-unit bound: a tiny compressed frame can announce dimensions that require a very large decoded allocation.

View Source 
const MaxScrollDelta = hidproto.MaxRelativeMouseDelta

MaxScrollDelta is the inclusive magnitude accepted for either wheel axis. The JetKVM absolute- and relative-mouse HID descriptors both encode Wheel and AC Pan as signed 8-bit relative values with logical bounds -127..127. Keep -128 out even though it fits in int8: it is outside that HID contract.

Variables

View Source 
var (
	// ErrHIDNotReady means the device never confirmed the HID-RPC readiness
	// handshake, so this client refuses to pretend input was delivered.
	ErrHIDNotReady = errors.New("jetkvm: HID control channel is not ready (readiness handshake not confirmed by the device)")

	// ErrHIDClosed means the HID control channel is gone (disconnect,
	// shutdown, or a failed handshake). Classified closed-channel errors
	// retain this sentinel for errors.Is without exposing an unredacted cause.
	ErrHIDClosed = errors.New("jetkvm: HID control channel is closed")

	// ErrStaleControlToken means a send was validated against a control
	// lease that has since been released, expired, or been replaced. The
	// frame was dropped before reaching the device - it is not a "maybe".
	ErrStaleControlToken = errors.New("jetkvm: control lease token is stale; the queued input was dropped and never sent")

	// ErrNeutralizeUnverified means release-all could not be confirmed on
	// the wire. Callers must treat held input as possibly still held.
	ErrNeutralizeUnverified = errors.New("jetkvm: neutral HID state is not confirmed on the wire; assume input may still be held")

	// ErrHIDBufferFull means a frame was rejected before Send because
	// accepting it would exceed the bounded Pion/SCTP outbound buffer. The
	// caller may retry under a fresh lease after the peer starts draining.
	ErrHIDBufferFull = errors.New("jetkvm: outbound HID buffer is full; frame was not sent")
)

Control-plane errors. These are the truthful, non-sensitive failure states callers (CLI, MCP tools) are expected to surface verbatim.

View Source 
var ErrControlDisabled = errors.New("jetkvm: control is not enabled for this connection")

ErrControlDisabled is returned when control was never enabled for this connection, so no lease can exist.

View Source 
var ErrControlHeld = errors.New("jetkvm: control lease is already held")

ErrControlHeld is returned by TryAcquire when another caller already holds the lease slot.

View Source 
var ErrControlLifecycleBusy = errors.New("jetkvm: control lifecycle transition is in progress")

ErrControlLifecycleBusy is returned by TryAcquire when a release or close neutralization transaction owns the HID lifecycle gate. Unlike Acquire, TryAcquire never waits for that transaction to finish.

View Source 
var ErrOCRUnavailable = errors.New("jetkvm: OCR engine unavailable")

ErrOCRUnavailable is the stable sentinel for a host that has no usable OCR executable. Callers should use errors.Is rather than matching the human-readable installation guidance returned by OCRUnavailableError.

View Source 
var ErrRPCBufferFull = errors.New("jetkvm: outbound RPC buffer is full; request was not sent")

Functions

func CanonicalBaseURL added in v0.2.0 

func CanonicalBaseURL(raw string) (string, error)

CanonicalBaseURL validates and canonicalizes the direct-device URL shape. Userinfo is rejected so credentials can never be smuggled through a CLI argument; queries/fragments/paths are rejected because JetKVM's API lives at the origin root and accepting aliases would undermine session coordination. Errors deliberately never echo raw input.

func RedactError 

func RedactError(err error) string

RedactError renders an error as a string safe to hand to a caller, a log, or an MCP response. It scrubs URLs (userinfo, query strings), key/value credential pairs, and long opaque tokens.

Everything user-facing in this module goes through here rather than calling err.Error() directly, so a wrapped error from a dependency cannot smuggle credential material into output.

func ResolveKeyCombo added in v0.5.0 

func ResolveKeyCombo(name string) (modifier byte, keys []byte, err error)

ResolveKeyCombo resolves a case-insensitive named keyboard chord to one validated HID keyboard report. Plus signs, hyphens, and whitespace are interchangeable separators. The returned key slice is owned by the caller.

func ResolveMouseButton added in v0.5.0 

func ResolveMouseButton(button, action string) (mask byte, pressed bool, err error)

ResolveMouseButton resolves the strict named-button/action contract shared by the MCP and CLI adapters. Names are deliberately exact and case-sensitive so callers cannot accidentally broaden their input contract. Errors never include either caller-controlled value.

func TypeCharacterContext added in v0.5.0 

func TypeCharacterContext(position int, r rune) string

TypeCharacterContext returns a caller-safe description for a character in text being typed. It deliberately contains only a one-based position and a Unicode general category: never the rune, its quoted form, its code point, or a byte offset.

func ValidateHoldMS added in v0.5.0 

func ValidateHoldMS(holdMS int) error

ValidateHoldMS validates a hold-key duration before an adapter converts it to time.Duration. A zero-duration hold would be indistinguishable from the existing one-shot key-combo operation. Rejecting values above MaxHoldMS also prevents duration overflow and keeps the operation inside its lease timeout.

func ValidateKeyCombo added in v0.5.0 

func ValidateKeyCombo(modifier int, keys []int) error

ValidateKeyCombo validates a resolved keyboard chord before its integer representation is narrowed to the HID wire bytes. A keyboard report can carry at most six non-modifier keys. Modifier-only chords are valid, but a report with neither a modifier nor a key is not a chord.

func ValidateKeySequenceLength added in v0.5.0 

func ValidateKeySequenceLength(length int) error

ValidateKeySequenceLength validates the number of named chords in one ordered key sequence. An empty sequence is rejected because it would report success without sending any input.

func ValidateKeypress added in v0.2.0 

func ValidateKeypress(key, modifier int) error

ValidateKeypress validates integer adapter input before any narrowing to a wire byte. CLI and MCP both call this exact function, so neither surface can wrap a negative or oversized modifier/key into an apparently valid report.

func ValidateMouseButton added in v0.5.0 

func ValidateMouseButton(button byte) error

ValidateMouseButton accepts exactly one supported named-button bit. Combined masks and zero are invalid because one mouse-button action targets one button. The error is static and never reflects the caller-controlled byte.

func ValidatePointer added in v0.2.0 

func ValidatePointer(x, y, buttons int) error

ValidatePointer validates integer adapter input before narrowing to int32 and byte for the HID wire format.

func ValidatePointerDragReports added in v0.5.0 

func ValidatePointerDragReports(reports []PointerDragReport) error

ValidatePointerDragReports validates an already-built drag sequence at an operation boundary. Release reports may use a zero button mask, but the sequence itself must include at least one nonzero button state so an empty or movement-only sequence cannot be reported as a successful drag.

func ValidatePointerGesture added in v0.5.0 

func ValidatePointerGesture(x, y, buttons int) error

ValidatePointerGesture validates click-like pointer input before narrowing to the HID wire format. Unlike ValidatePointer, it requires a nonzero button mask so click, double-click, and drag cannot succeed without pressing a button. Zero remains valid for movement and release reports.

func ValidateScreenshotDimensions added in v0.5.0 

func ValidateScreenshotDimensions(width, height int) error

ValidateScreenshotDimensions rejects dimensions that could make image decode, crop, scale, or encode allocate outside the screenshot budget. Division is used instead of width*height so hostile integers cannot overflow the check.

func ValidateScroll added in v0.5.0 

func ValidateScroll(dx, dy int) error

ValidateScroll validates integer adapter input before either wheel delta is narrowed to the firmware's int8 parameters. Like ValidatePointer, it rejects values outside the device contract rather than silently changing the caller's requested input. A zero/zero report is also rejected because both the reference UI and firmware treat it as a no-op.

func ValidateTypeDelay added in v0.5.0 

func ValidateTypeDelay(delayMS int) error

ValidateTypeDelay validates the shared inter-key delay before an adapter converts it to time.Duration.

func ValidateWaitForTextOptions added in v0.5.0 

func ValidateWaitForTextOptions(opts WaitForTextOptions) error

ValidateWaitForTextOptions resolves omitted fields and validates all caller input without performing OCR or capturing a screenshot.

func ValidateWaitStableOptions added in v0.5.0 

func ValidateWaitStableOptions(opts WaitStableOptions) error

ValidateWaitStableOptions resolves omitted fields to their defaults and validates every supplied value. It performs no I/O.

Types

type APIError 

type APIError struct {
	Path       string
	StatusCode int
	Body       string
}

APIError represents a non-2xx HTTP response from the device. The body is sanitized before storage (see sanitizeErrorBody) so an error surfaced to logs, CLI output, or an MCP response can never carry a reflected cookie/token/password - including one the device itself echoed back.

func (*APIError) Error 

func (e *APIError) Error() string

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the single session-owning core for one connected device. Status, screenshot, stable-screen waits, and legacy RPC control calls use its command lock; binary HID control uses the exclusive lease in owner.go and its release-all guarantees.

func Connect 

func Connect(ctx context.Context, opts Options) (*Client, error)

Connect performs the full browser-free handshake described in README.md: unauthenticated status probe, login (if credentials were given), signaling websocket dial with a firmware-compatibility check, then WebRTC offer/answer/ICE and data channel setup. On any failure the underlying connection is torn down before returning.

func (*Client) CaptureScreenshot 

func (c *Client) CaptureScreenshot(ctx context.Context) (Screenshot, error)

CaptureScreenshot records the current frame generation after the call has started, waits for a strictly newer decodable video frame, decodes it via the configured Decoder, and returns it as PNG bytes. Nothing is written to disk. A successful result is therefore always request-fresh: it can never be a cached frame from before this call (or a preceding control action).

The frame wait and decode subprocess observe ctx directly. The synchronous image steps check it before and immediately after their bounded work, so an expired request cannot return a successful screenshot.

func (*Client) Close 

func (c *Client) Close(ctx context.Context) error

Close neutralizes any held control input and tears down the session and signaling connection, in that order - neutralization has to happen while the transport is still up.

It is safe to call more than once and safe to call with an already-canceled context: neutralization always runs on a short, fresh context of its own. A non-nil error means neutralization could not be confirmed (ErrNeutralizeUnverified); the teardown still completes.

func (*Client) Control 

func (c *Client) Control() (*controlLease, error)

Control returns the control lease used by keyboard, pointer, and scroll commands, or an error if this Client was connected with AllowControl: false. Scroll acquires it internally because current firmware requires a separate legacy RPC wire path for wheel frames.

func (*Client) DeviceID 

func (c *Client) DeviceID() string

DeviceID returns the device ID captured from GET /device at connect time.

func (*Client) FirmwareVersion 

func (c *Client) FirmwareVersion() string

FirmwareVersion returns the deviceVersion string captured from the signaling handshake's device-metadata message.

func (*Client) SaveScreenshot 

func (c *Client) SaveScreenshot(ctx context.Context, outputPath string) (ScreenshotResult, error)

SaveScreenshot captures one screenshot and writes it to outputPath as a PNG. It exists for the CLI, where the path comes from the local user's own command line. The MCP adapter deliberately does not use it - see CaptureScreenshot.

The write is atomic: the PNG lands in a temporary file in the same directory and is renamed into place, so an interrupted capture cannot leave a truncated image at outputPath.

func (*Client) Scroll added in v0.5.0 

func (c *Client) Scroll(ctx context.Context, dx, dy int8) (err error)

Scroll sends one stateless wheel event. Positive dy scrolls up and positive dx scrolls right, matching the JetKVM HID descriptor and reference UI.

Firmware caveat: TypeWheelReport exists on the binary hidrpc channel, but the pinned firmware's hidRPC input switch has no wheel case and drops it. The only working path is the legacy wheelReport JSON-RPC method, so this operation cannot carry the control lease's generation token. It still takes the same process-local lease non-blockingly and holds it through the RPC, so a wheel event cannot bypass a competing keyboard/pointer holder. The AllowControl check remains defense in depth, and success requires both the device's RPC acknowledgement and lease neutralization. It is intentionally not retried after send by the MCP adapter because delivery would be ambiguous.

func (*Client) Status 

func (c *Client) Status(ctx context.Context) (StatusResult, error)

Status verifies the RPC data channel is alive by calling the device's "ping" method (jsonrpc.go's rpcPing), and reports what was learned about the device at connect time.

func (*Client) VideoDiagnostics 

func (c *Client) VideoDiagnostics() VideoDiagnostics

VideoDiagnostics returns a privacy-safe snapshot of the video pipeline: transport and negotiation state, RTP and NAL-unit counts, keyframe request counts, and bounded failure categories. It contains no credentials, addresses, SDP, ICE candidates, paths, or payload bytes, and is safe to log or hand to an agent.

It is most useful after a screenshot fails: FailureBoundary names the single stage the pipeline stopped at.

func (*Client) WaitStable added in v0.5.0 

func (c *Client) WaitStable(ctx context.Context, opts WaitStableOptions) (WaitStableResult, error)

WaitStable polls request-fresh decoded video frames until the fraction of changed pixels stays at or below Threshold for StableFrames consecutive comparisons. It holds the Client command lock for the complete operation, so Status, CaptureScreenshot, and legacy RPC control calls cannot interleave with the sampled sequence.

Every poll records the current completed frame generation immediately before waiting for a strictly newer one. Frames cached during a poll gap are therefore skipped rather than compared as though they were fresh.

type CompatibilityError 

type CompatibilityError struct {
	// Stage names the handshake step that failed, e.g.
	// "signaling-metadata" or "data-channel-open".
	Stage string
	// Detail explains the specific mismatch.
	Detail string
}

CompatibilityError is returned when the device's handshake behavior doesn't match what this client was built to understand. It always includes what we expected and what we saw so a human (or agent) can decide whether to update this client or investigate the device.

func (*CompatibilityError) Error 

func (e *CompatibilityError) Error() string

type Credentials 

type Credentials struct {
	Password  Secret
	AuthToken Secret
}

Credentials configures how the client authenticates to the device's local web API. At most one of Password or AuthToken should be set:

  • Password performs the real login flow (POST /auth/login-local) and receives a fresh authToken cookie, exactly like a browser logging in.
  • AuthToken supplies an already-valid session cookie value directly, skipping login. This exists for automation contexts where a session was already established out of band; it is still handled as a Secret and never logged.

Neither field is required if the device is in "noPassword" mode.

type Decoder 

type Decoder interface {
	DecodeFrame(ctx context.Context, annexB []byte) (image.Image, error)
}

Decoder turns one self-contained Annex-B H.264 frame into a decoded image. It is a narrow interface so the H.264 decode step - the most likely source of firmware/codec drift (profile changes, a future H.265 default, etc.) - can be swapped without touching session or screenshot orchestration code.

type DeviceError added in v0.3.0 

type DeviceError struct {
	Kind      ErrorKind
	Operation string
	Detail    string
	// contains filtered or unexported fields
}

DeviceError is a privacy-safe, classified device failure. Detail must already be redacted; constructors inside this package enforce that rule, and the MCP boundary applies RedactError once more before returning it.

func (*DeviceError) Error added in v0.3.0 

func (e *DeviceError) Error() string

func (*DeviceError) Unwrap added in v0.5.0 

func (e *DeviceError) Unwrap() error

Unwrap is reserved for package-controlled, privacy-safe sentinels. Raw dependency errors are flattened by newDeviceError so callers cannot recover credential-bearing transport objects through errors.As.

type DeviceMetadata 

type DeviceMetadata struct {
	DeviceVersion string `json:"deviceVersion"`
}

DeviceMetadata is the first message the signaling websocket sends, carrying the firmware's self-reported version string. Evidence: web.go's handleLocalWebRTCSignal, which writes {"type":"device-metadata","data":{"deviceVersion":builtAppVersion}} immediately after accepting the websocket, before any offer is sent.

type DeviceStatus 

type DeviceStatus struct {
	IsSetup bool `json:"isSetup"`
}

DeviceStatus mirrors web.go's DeviceStatus response from GET /device/status (public, unauthenticated).

type ErrorKind added in v0.3.0 

type ErrorKind string

ErrorKind is the stable failure taxonomy exposed to CLI and MCP callers. Callers can distinguish these values without matching dependency-specific transport strings. 

const (
	ErrorKindAuthFailed  ErrorKind = "auth-failed"
	ErrorKindUnreachable ErrorKind = "unreachable"
	ErrorKindTimeout     ErrorKind = "timeout"
	ErrorKindBadFrame    ErrorKind = "bad-frame"
)

func ErrorKindOf added in v0.3.0 

func ErrorKindOf(err error) ErrorKind

ErrorKindOf returns a classified kind, including through ordinary wrapping. A raw context deadline/cancellation is normalized to timeout at this public boundary so MCP callers never receive an undifferentiated context error.

type FFmpegDecoder 

type FFmpegDecoder struct {
	// BinaryPath overrides the ffmpeg binary to invoke. Defaults to
	// "ffmpeg" (resolved via PATH) when empty.
	BinaryPath string
	// Timeout bounds how long a single decode may take. Defaults to 10s
	// when zero.
	Timeout time.Duration
}

FFmpegDecoder shells out to the system ffmpeg binary to decode a single H.264 frame to an image. FFmpeg is treated as an external, replaceable dependency (not vendored, not assumed to be a specific version) exactly because the decode step is the piece most likely to need to change independently of the WebRTC/signaling code.

func (*FFmpegDecoder) CheckAvailable 

func (d *FFmpegDecoder) CheckAvailable(ctx context.Context) error

CheckAvailable runs `ffmpeg -version` to fail fast with an actionable error if ffmpeg isn't installed/on PATH, rather than surfacing a raw "executable file not found" from deep inside a screen-reading call.

func (*FFmpegDecoder) DecodeFrame 

func (d *FFmpegDecoder) DecodeFrame(ctx context.Context, annexB []byte) (image.Image, error)

DecodeFrame pipes annexB to ffmpeg on stdin and reads one PNG-encoded frame back on stdout, entirely in memory (no temp files, nothing written to disk that could linger with frame content).

The subprocess runs with a minimal allowlisted environment, is bounded by both ctx and the decoder's own timeout, and is always reaped.

type Held 

type Held struct {
	// contains filtered or unexported fields
}

Held is a live handle on an acquired control lease. Every method is bounded by the caller's context, and every method after the lease ends returns an error rather than silently no-op'ing.

func (*Held) Done added in v0.5.0 

func (h *Held) Done() <-chan struct{}

Done is closed after this holder has released (explicitly or through its context/watchdog). Session adapters that retain a holder across requests use it to discard stale held-input bookkeeping promptly.

func (*Held) Release 

func (h *Held) Release() error

Release ends the lease: it revokes the lease generation, writes the neutralization frames, and frees the exclusivity slot for recovery. Safe to call more than once and concurrently with the watchdog firing; the first call's result is returned to all callers.

A non-nil error means neutralization could not be confirmed (see ErrNeutralizeUnverified). The slot is freed either way so teardown can retry cleanup, but a fresh generation remains prohibited until neutralization is independently confirmed.

func (*Held) ReleaseKeyboard added in v0.5.0 

func (h *Held) ReleaseKeyboard(ctx context.Context) error

ReleaseKeyboard clears every key and modifier without changing the current mouse-button state. A nil result means the zero keyboard report, and every report ordered before it, drained from Pion's outbound SCTP buffer. On error, callers must assume keyboard state may remain held. This is used when a persistent mouse-button holder must remain live after a one-shot operation.

func (*Held) SendKeyboardReport 

func (h *Held) SendKeyboardReport(ctx context.Context, modifier byte, keys []byte) error

SendKeyboardReport sends a full keyboard state report through the held lease, bounded by ctx. See internal/hidproto for wire format details.

func (*Held) SendMouseReport 

func (h *Held) SendMouseReport(ctx context.Context, dx, dy int8, buttons byte) error

SendMouseReport sends a relative-mouse report through the held lease, bounded by ctx.

func (*Held) SendPointerReport 

func (h *Held) SendPointerReport(ctx context.Context, x, y int32, buttons byte) error

SendPointerReport sends an absolute-mouse report through the held lease, bounded by ctx.

func (*Held) Token 

func (h *Held) Token() uint64

Token returns the lease generation this holder sends under. Frames are validated against it at the last moment before they are written.

type LocalDevice 

type LocalDevice struct {
	AuthMode     *string `json:"authMode"`
	DeviceID     string  `json:"deviceId"`
	LoopbackOnly bool    `json:"loopbackOnly"`
}

LocalDevice mirrors web.go's LocalDevice response from GET /device (requires auth unless the device is in noPassword mode).

type OCREngine added in v0.5.0 

type OCREngine interface {
	CheckAvailable(context.Context) error
	ReadText(context.Context, []byte) (string, error)
}

OCREngine extracts UTF-8 plain text from an in-memory encoded image. CheckAvailable lets callers fail before capturing a screen when the optional local engine is absent.

type OCRUnavailableError added in v0.5.0 

type OCRUnavailableError struct{}

OCRUnavailableError reports that the optional local OCR dependency could not be resolved. It intentionally carries no binary path or underlying lookup error: either may contain operator-controlled or sensitive text.

func (*OCRUnavailableError) Error added in v0.5.0 

func (*OCRUnavailableError) Error() string

func (*OCRUnavailableError) Unwrap added in v0.5.0 

func (*OCRUnavailableError) Unwrap() error

type Options 

type Options struct {
	// BaseURL is the device's HTTP base URL, e.g. "http://jetkvm.local"
	// or "http://<device-ip>".
	BaseURL string

	// Credentials authenticates against the device's local web API. Zero
	// value is fine for a device in noPassword mode.
	Credentials Credentials

	// AllowControl gates every keyboard/mouse operation. It controls whether
	// this client negotiates the "hidrpc" data channel and constructs a
	// control lease; Scroll also checks it explicitly before using the legacy
	// JSON-RPC wheel path required by current firmware. This matches the
	// CLI/MCP --allow-control gate at the core client boundary.
	AllowControl bool

	// Decoder overrides the video decode backend. Defaults to
	// &FFmpegDecoder{} when nil.
	Decoder Decoder

	// HTTPTimeout bounds individual HTTP requests. Defaults to 10s.
	HTTPTimeout time.Duration
}

Options configures a Connect call.

type PointerDragReport added in v0.5.0 

type PointerDragReport struct {
	X       int
	Y       int
	Buttons int
}

PointerDragReport is one full-width absolute-pointer state in a drag gesture. Adapters keep these values as ints until validation is complete, then narrow them only at the HID send boundary.

func BuildPointerDragReports added in v0.5.0 

func BuildPointerDragReports(x1, y1, x2, y2, button, steps int) ([]PointerDragReport, error)

BuildPointerDragReports validates and constructs a complete drag gesture: press at the start, optionally move through interpolated positions, move to the destination, then release there. The operation-local button mask must be nonzero, while the generated terminal release remains a valid zero-button pointer report. Every generated coordinate is checked before any adapter can narrow it to HID wire types.

type RPCError 

type RPCError struct {
	Method string
	Raw    json.RawMessage
}

RPCError represents a JSON-RPC error object returned by the device.

func (*RPCError) Error 

func (e *RPCError) Error() string

type ResolvedKeyCombo added in v0.5.0 

type ResolvedKeyCombo struct {
	Modifier byte
	Keys     []byte
}

ResolvedKeyCombo is one named chord after it has crossed the shared resolver and integer validation boundary. Keys is owned by the caller.

func ResolveKeySequence added in v0.5.0 

func ResolveKeySequence(names []string) ([]ResolvedKeyCombo, error)

ResolveKeySequence resolves and validates a complete ordered sequence before returning any reports. Errors identify the failing array index without reflecting the caller-controlled chord name.

type Screenshot 

type Screenshot struct {
	ScreenshotResult
	PNG []byte
}

Screenshot is one captured frame as PNG bytes plus its metadata. Keeping the image in memory is what lets the MCP adapter return an image to the caller without ever accepting - or writing to - a caller-chosen path.

type ScreenshotCaptureFunc added in v0.5.0 

type ScreenshotCaptureFunc func(context.Context) (Screenshot, error)

ScreenshotCaptureFunc captures one request-fresh screenshot for an OCR poll.

type ScreenshotResult 

type ScreenshotResult struct {
	Path       string
	Width      int
	Height     int
	CapturedAt time.Time
	Fresh      bool
}

ScreenshotResult describes one captured screenshot, always including enough metadata for a caller to judge whether to trust it.

Path is set only by SaveScreenshot; CaptureScreenshot leaves it empty because it never touches the filesystem.

type Secret 

type Secret struct {
	// contains filtered or unexported fields
}

Secret wraps a credential (password, session cookie, token) so that accidental logging, error wrapping, or struct dumping (%v, %+v, JSON marshalling) can never leak the underlying value. Call Expose() at the one call site that actually needs the raw bytes (building an HTTP request); never store the result of Expose() in another field or error.

func NewSecret 

func NewSecret(value string) Secret

NewSecret wraps a raw credential value.

func (Secret) ContainedIn added in v0.2.0 

func (s Secret) ContainedIn(public string) bool

ContainedIn reports whether public metadata contains this exact credential. The raw value stays encapsulated; callers use the boolean only to replace compromised/reflected metadata with the redaction placeholder.

func (Secret) Empty 

func (s Secret) Empty() bool

Empty reports whether no credential was supplied.

func (Secret) Expose 

func (s Secret) Expose() string

Expose returns the raw credential. Use only to construct the single outbound request that needs it.

func (Secret) GoString 

func (s Secret) GoString() string

GoString implements fmt.GoStringer so %#v also redacts.

func (Secret) MarshalJSON 

func (s Secret) MarshalJSON() ([]byte, error)

MarshalJSON ensures Secret never round-trips its value through JSON (config dumps, diagnostics, error payloads).

func (Secret) String 

func (s Secret) String() string

String implements fmt.Stringer, deliberately never returning the value.

type StatusResult 

type StatusResult struct {
	DeviceID        string
	FirmwareVersion string
	RPCReachable    bool
}

StatusResult is the result of a Status call.

type TesseractOCREngine added in v0.5.0 

type TesseractOCREngine struct {
	// BinaryPath overrides the tesseract executable to resolve. It is intended
	// for embedding and tests, not for caller-controlled MCP arguments.
	BinaryPath string
	// Timeout bounds one OCR subprocess. Values at or below zero use 10s.
	Timeout time.Duration
}

TesseractOCREngine invokes the optional system tesseract executable. Image bytes travel through stdin and recognized text through stdout; no shell, image path, or temporary output file is involved.

func (*TesseractOCREngine) CheckAvailable added in v0.5.0 

func (e *TesseractOCREngine) CheckAvailable(ctx context.Context) error

CheckAvailable performs runtime detection without starting tesseract. The real ReadText call resolves the executable again so this preflight is never relied on as a stale availability guarantee.

func (*TesseractOCREngine) ReadText added in v0.5.0 

func (e *TesseractOCREngine) ReadText(ctx context.Context, imageData []byte) (string, error)

ReadText runs one bounded Tesseract recognition pass. Successful stdout is returned byte-for-byte as a Go string after UTF-8 validation: leading and trailing whitespace are meaningful, and an empty result is a valid outcome for a screen containing no recognizable text.

type TypeKeypress added in v0.5.0 

type TypeKeypress struct {
	Modifier     int
	HIDUsageCode int
}

TypeKeypress is one USB HID keyboard report produced by the US-layout text mapper. The fields remain integers until adapters have passed them through ValidateKeypress, immediately before narrowing to wire bytes.

func MapTypeString added in v0.5.0 

func MapTypeString(text string) ([]TypeKeypress, error)

MapTypeString expands a complete UTF-8 string into US-layout keypresses. It returns no partial sequence when any rune is unsupported.

func MapUSKeyboardRune added in v0.5.0 

func MapUSKeyboardRune(r rune) (TypeKeypress, error)

MapUSKeyboardRune maps one supported rune to its USB HID modifier and usage code on a US keyboard layout. Supported input is printable ASCII, newline (Enter), and tab. The function is pure and never skips input.

type VideoDiagnostics 

type VideoDiagnostics struct {
	// Transport state at snapshot time.
	PeerConnectionState string `json:"peerConnectionState"`
	ICEConnectionState  string `json:"iceConnectionState"`
	SignalingState      string `json:"signalingState"`

	// Signaling exchange.
	AnswerApplied             bool   `json:"answerApplied"`
	AnswerRejected            bool   `json:"answerRejected"`
	LocalICECandidates        int    `json:"localIceCandidates"`
	RemoteICECandidates       int    `json:"remoteIceCandidates"`
	RemoteICECandidatesBad    int    `json:"remoteIceCandidatesRejected"`
	RemoteICECandidatesQueued int    `json:"remoteIceCandidatesQueuedBeforeAnswer"`
	UnhandledSignalingMsgs    int    `json:"unhandledSignalingMessages"`
	MalformedSignalingMsgs    int    `json:"malformedSignalingMessages"`
	SignalingPumpStopped      string `json:"signalingPumpStopped,omitempty"`
	SignalingPumpStillActive  bool   `json:"signalingPumpStillActive"`

	// Negotiated video track.
	TrackObserved     bool   `json:"trackObserved"`
	TrackAfterMillis  int64  `json:"trackAfterMillis"`
	TrackMimeType     string `json:"trackMimeType,omitempty"`
	TrackPayloadType  uint8  `json:"trackPayloadType,omitempty"`
	TrackProfileLevel string `json:"trackProfileLevelId,omitempty"`
	TrackPacketMode   string `json:"trackPacketizationMode,omitempty"`
	TrackRejection    string `json:"trackRejection,omitempty"`

	// What this client offered, for comparison against what came back.
	OfferedPayloadType  uint8  `json:"offeredPayloadType"`
	OfferedProfileLevel string `json:"offeredProfileLevelId"`
	OfferedPacketMode   string `json:"offeredPacketizationMode"`

	// RTP.
	RTPPackets           uint64 `json:"rtpPackets"`
	RTPPayloadBytes      uint64 `json:"rtpPayloadBytes"`
	FirstRTPAfterMillis  int64  `json:"firstRtpAfterMillis"`
	LastRTPAfterMillis   int64  `json:"lastRtpAfterMillis"`
	TrackReadFailure     string `json:"trackReadFailure,omitempty"`
	KeyframeRequestsSent uint64 `json:"keyframeRequestsSent"`
	KeyframeRequestsFail uint64 `json:"keyframeRequestsFailed"`

	// What the RTP stream looked like on the wire, before reassembly. This
	// is the independent witness: WireNALUnitsByType["IDR"] counts keyframe
	// packets the device actually sent, whether or not any of them survived
	// into an access unit.
	RTPPacketsByClass     map[string]uint64 `json:"rtpPacketsByClass,omitempty"`
	WireNALUnitsByType    map[string]uint64 `json:"wireNalUnitsByType,omitempty"`
	MalformedRTPPayloads  uint64            `json:"malformedRtpPayloads"`
	FUStarts              uint64            `json:"fuStarts"`
	FUEnds                uint64            `json:"fuEnds"`
	RTPPacketsLost        uint64            `json:"rtpPacketsLost"`
	RTPPacketsReordered   uint64            `json:"rtpPacketsReordered"`
	RTPPacketsDuplicated  uint64            `json:"rtpPacketsDuplicated"`
	MaxSequenceJump       uint16            `json:"maxSequenceJump"`
	MarkedPackets         uint64            `json:"markedPackets"`
	TimestampChanges      uint64            `json:"timestampChanges"`
	UnmarkedFrames        uint64            `json:"unmarkedFrames"`
	MaxPacketsPerFrame    uint64            `json:"maxPacketsPerFrame"`
	MaxPacketsPerKeyframe uint64            `json:"maxPacketsPerKeyframe"`
	ReassemblyWindow      uint64            `json:"reassemblyWindowPackets"`
	MediaSourceChanges    uint64            `json:"mediaSourceChanges"`

	// Whether this client's keyframe requests named the synchronization
	// source it was actually receiving. Note that the device's firmware
	// drains inbound RTCP without parsing it, so a matching target still
	// does not mean the request was honored - see requestKeyframes.
	KeyframeRequestOnTarget bool `json:"keyframeRequestOnTarget"`

	// Depacketization.
	AccessUnits        uint64            `json:"accessUnits"`
	NALUnitsByType     map[string]uint64 `json:"nalUnitsByType,omitempty"`
	SawSPS             bool              `json:"sawSps"`
	SawPPS             bool              `json:"sawPps"`
	SawIDR             bool              `json:"sawIdr"`
	FramesAssembled    uint64            `json:"framesAssembled"`
	BuilderDropped     uint64            `json:"reassemblyDroppedPackets"`
	ParameterSetSource string            `json:"parameterSetSource,omitempty"`

	// Decode.
	DecodeAttempts  int    `json:"decodeAttempts"`
	DecodeBytesIn   uint64 `json:"decodeBytesIn"`
	DecodeFailure   string `json:"decodeFailure,omitempty"`
	ElapsedMillis   int64  `json:"elapsedMillis"`
	FailureBoundary string `json:"failureBoundary"`
}

VideoDiagnostics is a point-in-time, privacy-safe summary of the video pipeline. Every field is a count, a duration in milliseconds, a bounded enum, or a codec parameter. It is safe to serialize and hand to an operator or an agent.

func (VideoDiagnostics) Boundary 

func (v VideoDiagnostics) Boundary() string

Boundary localizes the failure to a single stage. It is a pure function of the snapshot, so it is exhaustively unit-testable without any device.

Order matters: the earliest stage that did not complete is the one worth reporting, because every later stage is downstream of it.

func (VideoDiagnostics) Summary 

func (v VideoDiagnostics) Summary() string

Summary renders a single safe line suitable for embedding in an error message. It carries the boundary plus the few counts that make the boundary actionable, and nothing else.

type WaitForTextOptions added in v0.5.0 

type WaitForTextOptions struct {
	Text     string
	Regex    bool
	Interval *time.Duration
	Timeout  *time.Duration
}

WaitForTextOptions configures a WaitForText call. Nil duration fields use the defaults above. Text is matched byte-for-byte unless Regex is true, in which case it is compiled with Go's RE2-syntax regexp package.

type WaitForTextResult added in v0.5.0 

type WaitForTextResult struct {
	Matched    bool
	Match      string
	TimedOut   bool
	Elapsed    time.Duration
	FrameCount int
}

WaitForTextResult reports the observations made while polling. A timeout is a normal structured outcome: TimedOut is true and the returned error is nil.

func WaitForText added in v0.5.0 

func WaitForText(
	ctx context.Context,
	opts WaitForTextOptions,
	capture ScreenshotCaptureFunc,
	engine OCREngine,
) (WaitForTextResult, error)

WaitForText captures and OCRs request-fresh screenshots until Text matches or Timeout expires. The first poll starts immediately; later polls start no sooner than Interval after the preceding poll began. OCR availability is checked before the first screenshot is captured.

Context deadlines, including Timeout, return a structured TimedOut result with a nil error. Explicit cancellation and dependency failures are errors.

type WaitStableOptions added in v0.5.0 

type WaitStableOptions struct {
	Threshold    *float64
	StableFrames *int
	PollInterval *time.Duration
}

WaitStableOptions configures a WaitStable call. Nil fields use the defaults above. Pointers preserve the distinction between an omitted field and the valid explicit zero values for Threshold and PollInterval.

type WaitStableResult added in v0.5.0 

type WaitStableResult struct {
	Settled             bool
	FramesSampled       int
	FinalChangeFraction float64
	Elapsed             time.Duration
}

WaitStableResult reports the observations made while waiting for a screen to settle. On a timeout, WaitStable returns the partial result together with a classified timeout error.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.