resource

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 14, 2026 License: GPL-3.0 Imports: 20 Imported by: 0

Documentation

Overview

Package resource is everything the application downloads at runtime, in one place that can list it, price it and remove it.

Six things fetch from the network as the application runs - terrain, basemaps, building footprints, boundaries, firmware and, once they exist, emulators and the SoftDevice - and until now each had its own cache, its own conventions, and its own idea of whether the operator was allowed to know. On the machine this was written for, the terrain cache had reached 7.1 GB and there was no way to see it, let alone delete it, without opening a terminal.

Firmware deliberately keeps its own library. It knows about roles, boards and which nodes are pinned to what, and a generic row would lose all three; the two are siblings in the same menu rather than one list serving neither.

The Nordic SoftDevice, fetched from Nordic at runtime.

It cannot be bundled and does not need to be. Whether emulating it for firmware testing is permitted at all was asked and answered - DevZone case 362437, recorded in docs/licence.md: it is, provided the end product runs on real Nordic hardware and the binary is neither reverse-engineered nor modified. How a copy reaches this machine is the same shape as everything else the application downloads: fetched from the source and cached, so MeshBench never becomes the one distributing it.

Until now this was a manual step. tools/renode/rak4631-softdevice.resc expects /tmp/s140.bin and the instructions above it tell a person to find the .hex themselves, convert it, and put it there - which is why no nRF52 board has ever been probed by anybody who had not already done that by hand.

The emulator toolchain, fetched from our own forks at runtime.

Three things stand between a source checkout and an emulated board: the SX1262 chip model, which every emulated node needs whatever its MCU; QEMU for the ESP32 family; and Renode for the nRF52s. A bundled build carries all three beside the binary and its users never meet this, but a compact one carries only the chip, and a development checkout had no path to any of them - the tools directory the lookup already searches was described as "where the installer puts anything it downloads after the fact", and for a source build no installer ever does.

So they arrive the way everything else here arrives: fetched from where they are published, verified, and cached. The destination is the tools directory itself, which is already the third step of the lookup, so a fetched tool is found with nothing else configured.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SIBytes

func SIBytes(b int64) string

SIBytes is a size in the units somebody would say it in.

Powers of a thousand rather than of 1024, because these numbers are read beside what a release page and a download manager say, and those count in megabytes of a million.

func ToolsFor

func ToolsFor(mcu string) []string

ToolsFor names the tools a board with this MCU cannot boot without.

Exported because the caller that knows how many nodes are waiting is the one that holds the scenario, and it should not have to know that every emulated node needs radioserver while only an nRF52 needs Renode. Matched on the MCU prefix, which is the board catalogue's own field: "nRF52840" and "ESP32-S3" are what a profile actually says.

Types

type DirCache

type DirCache struct {
	K Kind
	// Label names the row, and Dir is the directory it measures.
	Label string
	Dir   string
	// Purpose is what the bytes are for, said in the row.
	Purpose string
	// Fill is how this cache comes to have anything in it, in the words the
	// row shows where the Fetch button cannot be pressed. Three of these fill
	// themselves as the map is used and one does not, and a page that said the
	// first sentence about all four left buildings looking like a download
	// that had failed.
	Fill string
	// FillPanel is the page a cache nothing fills on its own is fetched from,
	// so the row can offer the way there rather than only describing it.
	FillPanel string
	// OnRequest marks the cache nothing fills on its own, so the row says
	// "only when asked" rather than claiming the application will see to it.
	OnRequest bool
	// Terms is the attribution the data arrived under, shown on demand. The
	// data is somebody else's; saying so is not optional.
	Terms string
}

DirCache is a provider for the caches that are simply a directory of files.

Terrain, basemaps, building footprints and map tiles all work the same way: the application fills them as the operator uses the map, nobody chose to download any of it, and until now nobody could see it either. The package comment names the case this exists for - a terrain cache that had reached 7.1 GB with no way to look at it, let alone delete it, without a terminal.

One row per cache rather than per file. A row per terrain tile would be tens of thousands of rows about nothing: what an operator wants to know is how much of the disk this has taken and whether it can go.

func (*DirCache) Kind

func (d *DirCache) Kind() Kind

func (*DirCache) Licence

func (d *DirCache) Licence(_, _ string) string

Licence is the attribution for this cache. name and version are ignored: a directory of tiles carries one set of terms, not one per file.

func (*DirCache) List

func (d *DirCache) List(_ context.Context) ([]Row, error)

List measures the directory. It walks rather than trusting a stored figure: the number is the point, and a stale one is worse than none.

func (*DirCache) Remove

func (d *DirCache) Remove(_ context.Context, row Row) error

Remove empties the cache, leaving the directory itself so the application can refill it without having to create anything.

type Fetcher

type Fetcher interface {
	Fetch(ctx context.Context, name, version string, progress func(done, total int64)) error
}

type HTTPDoer

type HTTPDoer interface {
	Do(*http.Request) (*http.Response, error)
}

HTTPDoer is the one method a download needs, so a test can answer without a network.

type Kind

type Kind string

Kind groups rows that came from the same place and behave the same way.

const (
	Terrain   Kind = "terrain"
	Basemap   Kind = "basemap"
	Buildings Kind = "buildings"
	Boundary  Kind = "boundary"
)
const SoftDeviceKind Kind = "softdevice"

SoftDeviceKind is its row group.

const ToolchainKind Kind = "toolchain"

ToolchainKind is its row group.

type Licensor

type Licensor interface {
	Licence(name, version string) string
}

Fetcher is a provider that can also download on demand.

Separate from Provider because most cannot: terrain and basemaps fill themselves as the map is used and have nothing to ask for out of context, so requiring them to implement a Fetch would mean writing one that refuses. Licensor is a provider that can produce the terms its bytes arrived under.

type Provider

type Provider interface {
	// Kind is what this provider produces, and names its rows in the panel.
	Kind() Kind
	// List reads what is on disk. It never touches the network: opening the
	// panel must not start a download.
	List(ctx context.Context) ([]Row, error)
	// Remove deletes one row's bytes. The caller has already confirmed.
	Remove(ctx context.Context, row Row) error
}

Provider is one source of rows.

type Row

type Row struct {
	Kind Kind
	// Name identifies the row to a person; Version distinguishes two of the
	// same thing where that idea applies, and is empty where it does not.
	Name    string
	Version string
	// Path is where it lives on disk, empty when nothing is cached. A remove
	// acts on this rather than on the name, because the name is a label.
	Path string
	// Bytes is measured when Cached and estimated otherwise, and Estimated
	// says which - a guess presented as a survey is the thing this package
	// exists to stop.
	Bytes     int64
	Estimated bool
	State     State
	// Why carries the reason for anything that is not a plain on-disk row:
	// what it is needed for, or why it cannot be had.
	Why string
	// Auto is whether the application may fetch this without asking. Not the
	// same as enabled - presence is the state above - but whether bandwidth
	// may be spent on the operator's behalf.
	Auto bool
	// Fetchable is whether this row can be downloaded on demand from the
	// panel. A SoftDevice can: it is one file at a known address. A terrain
	// cache cannot, because what to fetch depends on where the operator is
	// looking - it fills as the map is used, and the panel says so rather
	// than offering a button that has nothing to ask for.
	Fetchable bool
	// HowTo is how this is obtained when it cannot be obtained from here, and
	// the row is not finished without it.
	//
	// A row that names a missing thing and stops is a dead end: building
	// footprints sat at nothing with Fetch disabled and the words "fills itself
	// as the map is used" beside it, which was true of terrain and false of
	// them - they are pulled from Configuration > Environ, and nothing on the
	// page said so.
	HowTo string
	// HowToPanel is the page that does have it, where there is one. The row
	// carries the name and the interface decides what opening it means, which
	// is the only part of this a provider has no business knowing.
	HowToPanel string
	// Licensed is whether there are terms to show. Somebody else's data comes
	// with somebody else's conditions, and the panel is where they belong.
	Licensed bool
}

Row is one resource, as the panel and the verbs see it.

type SoftDevice

type SoftDevice struct {
	// CacheDir is the resource cache root; releases land under it by name and
	// version, so two can coexist.
	CacheDir string
	// HTTP is the client, so a test answers without a network.
	HTTP interface {
		Do(*http.Request) (*http.Response, error)
	}
	// Needed is how many nodes in this scenario cannot boot without one, so a
	// missing row can say that rather than sit there looking optional.
	Needed int
}

SoftDevice provides the Nordic SoftDevice rows.

func (*SoftDevice) Fetch

func (s *SoftDevice) Fetch(ctx context.Context, name, version string,
	progress func(done, total int64)) error

Fetch downloads one release, verifies it, and unpacks the two files worth keeping. progress is called with bytes received and the total where the server declares one.

func (*SoftDevice) HexPath

func (s *SoftDevice) HexPath(name, version string) string

HexPath is the converted-from location for a release, or "" when it is not cached. Exported because the emulator bring-up needs the file, not the row.

func (*SoftDevice) Kind

func (s *SoftDevice) Kind() Kind

func (*SoftDevice) Licence

func (s *SoftDevice) Licence(name, version string) string

Licence is Nordic's own text for a fetched release, for the interface to show. Empty when nothing is cached: there is no licence to display for a file this machine does not have.

func (*SoftDevice) List

func (s *SoftDevice) List(_ context.Context) ([]Row, error)

func (*SoftDevice) Remove

func (s *SoftDevice) Remove(_ context.Context, row Row) error

type State

type State string

State is what a row can be, and the set is deliberately small.

Present and InUse are different answers: a building set that is downloaded but not being priced into links is kept, not deleted, and an operator switching between two of them is running an experiment rather than discarding one.

const (
	// OnDisk means cached and usable.
	OnDisk State = "on disk"
	// InUse means cached and currently feeding the simulation - only
	// meaningful where the engine can hold one at a time.
	InUse State = "in use"
	// Available means it can be fetched but has not been.
	Available State = "available"
	// Needed means this scenario cannot run correctly without it, which is
	// the only state that is urgent.
	Needed State = "needed"
	// Unavailable means nothing can be fetched here - no build for this
	// platform, no published artefact - and the row says why rather than
	// offering a button that would fail.
	Unavailable State = "unavailable"
)

type Toolchain

type Toolchain struct {
	// Dir is the tools directory: where the emulator lookup already searches,
	// and therefore the only place a fetch is worth putting anything.
	Dir string
	// HTTP is the client, so a test answers without a network.
	HTTP HTTPDoer
	// Needed counts the nodes in this scenario that cannot boot without each
	// tool, keyed by the tool's name, so a missing row reads as blocking
	// rather than optional.
	Needed map[string]int
	// Find is the lookup a booting node does - the environment, then beside
	// the binary, then the tools directory, then PATH. Injected so this
	// package needs no emulator to be tested, and so the row and the node
	// cannot answer differently about the same machine.
	Find func(name string) (string, error)
}

Toolchain provides the emulator toolchain rows.

func (*Toolchain) Fetch

func (t *Toolchain) Fetch(ctx context.Context, name, _ string,
	progress func(done, total int64)) error

Fetch downloads one tool, verifies it, unpacks it and leaves it under the name the emulator lookup asks for.

Streamed to a temporary file rather than held in memory: Renode's package is 61 MB packed and several times that unpacked, and holding it would put a spike on a machine that is about to run an emulator on it.

func (*Toolchain) Kind

func (t *Toolchain) Kind() Kind

func (*Toolchain) Licence

func (t *Toolchain) Licence(name, _ string) string

Licence is the terms this tool arrives under, answerable before it is fetched. The full texts are in the application's own Licences window; this is what has to be read before pressing Fetch.

func (*Toolchain) List

func (t *Toolchain) List(_ context.Context) ([]Row, error)

func (*Toolchain) Remove

func (t *Toolchain) Remove(_ context.Context, row Row) error

type Verified

type Verified struct {
	// URL is where it is, and SHA256 is what it has to hash to. An empty
	// digest is refused rather than skipped: this type exists to check.
	URL    string
	SHA256 string
	// Bytes is what the publisher says the size is, used for progress when
	// the server declares none - a redirect often arrives with no
	// Content-Length, and a job reading "0 of 0" on a 61 MB download reads as
	// a stall.
	Bytes int64
	// Name is what to call this in an error, in the words the operator saw it
	// offered under.
	Name string
	// Dir is where the part file is written. The caller moves it into place,
	// because where "in place" is differs per caller and only they know
	// whether what landed is what they wanted.
	Dir  string
	HTTP HTTPDoer
}

Verified is a file to fetch and the digest it has to have.

func (Verified) To

func (v Verified) To(ctx context.Context, progress func(done, total int64)) (string, error)

To streams the file into Dir, hashing as it goes, and hands back the temporary file's path.

Streamed rather than held in memory: these are tens to hundreds of megabytes, and the machine is usually about to spend its memory on something else. Nothing is left behind on a failure, including a digest mismatch: a truncated or substituted file that stays on the disk is one a later run can mistake for a finished download.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL