data

func DatabaseSummaryTable ¶

func DatabaseSummaryTable() string

DatabaseSummaryTable returns a formatted table of all databases for use in help text. Each row: ID, domain, auth, formats, status.

func LocalFetchFromS3 ¶

func LocalFetchFromS3(cmd *cobra.Command, src, dstDir string, parallel int) error

LocalFetchFromS3 downloads an S3 object (or prefix) to the user's local machine by shelling out to s5cmd. This is for `abc data download --local`, not for staging to the workbench.

Why s5cmd rather than the existing tus upload flow? s5cmd supports resumable multi-part downloads (--if-size-differ, checksum-aware), handles large files efficiently, and is already a first-class tool in the abc-cluster toolchain.

s5cmd is located by checking (in order):

1. $PATH
2. /opt/abc-seedling/nf-work/bin/s5cmd (known install path on platform node, also useful when abc is invoked via SSH)

Returns an error with a helpful message if s5cmd is not found.

func LocalPushBytesToS3 ¶

func LocalPushBytesToS3(cmd *cobra.Command, data []byte, s3dest string) error

LocalPushBytesToS3 writes data to a temp file and uploads it to an S3 destination via LocalPushToS3. Used to push the run-manifest.json lineage record to the per-run prefix without the caller managing a temp file.

func LocalPushToS3 ¶

func LocalPushToS3(cmd *cobra.Command, src, s3dest string, parallel int) error

LocalPushToS3 uploads a local file (or directory) from the user's machine to an S3 destination by shelling out to s5cmd. It is the symmetric counterpart of LocalFetchFromS3 and is used by the `abc job run` data-staging seam to upload local-only inputs to the per-run inputs prefix before the job is submitted.

Endpoint + credentials are resolved from the active context the same way as LocalFetchFromS3 (resolveWorkbenchMinioCredsFromCtx); s5cmd is located via findLocalS5cmd ($PATH, then /opt/abc-seedling/nf-work/bin/s5cmd).

Returns an error with a helpful message if s5cmd is not found or the transfer fails. The caller (job-staging) treats any error as fatal — a job whose stage-in would 404 must not be submitted.

func NewCmd ¶

func NewCmd(serverURL, accessToken, workspace *string, dataFactory ...ClientFactory) *cobra.Command

NewCmd returns the "data" subcommand group.

func ResolveS3Creds ¶

func ResolveS3Creds(ctx abccfg.Context) (endpoint, accessKey, secretKey string)

ResolveS3Creds reads S3 credentials and endpoint from the active context using a universal resolution that works for all cluster types (abc-nodes, abc-seedling, abc-cloud). It is the one cred resolver for the CLI; the job data-staging seam (cmd/job) calls it to populate the stage tasks' env.

Phase 1.5b: when the context has cred_source set to a broker tier (seedling/v1 etc.), credentials are obtained from the credential broker rather than from the on-disk fields — those are deliberately empty for opaque-shape configs. The broker's returned MinIO bundle wins; if the broker fails we fall through to the legacy local-fields path.

Resolution order within each service (when not broker-routed):

admin.services.minio.{endpoint,access_key,secret_key}  (preferred)
admin.services.rustfs.{endpoint,access_key,secret_key} (fallback)
admin.abc_nodes.{s3_endpoint,s3_access_key,s3_secret_key} (legacy)

func StageS3ToWorkbench ¶

func StageS3ToWorkbench(cmd *cobra.Command, src string) error

StageS3ToWorkbench is the exported entry point for "download --workbench": copies an S3 object (or prefix) to the user's workbench ~/data/ directory. src must be an S3 URI (s3://...). If src ends with "/" it is treated as a prefix (multi-object copy).

func SupportedDatabaseIDs ¶

func SupportedDatabaseIDs() []string

SupportedDatabaseIDs returns a flat list of all database IDs and aliases, for shell completion.