IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.lib

Types to interface with ML pipelines such as text/token generation.

Submodules​

max.pipelines.lib.arch_lookupArchitecture registration and lookup tables.
max.pipelines.lib.auto_processorTurns chat conversations with inline images into AutoModel inputs.
max.pipelines.lib.decoder_scratchWorkspace buffers a registered decoder's kernels write each step.
max.pipelines.lib.generateFixed-batch generation, the lowest-level entry point to the pipeline API.
max.pipelines.lib.generate_imagesImage generation from text prompts and input images, in-process.
max.pipelines.lib.graph_specsCompiles a pipeline's graphs by role.
max.pipelines.lib.interfacesInterfaces for MAX pipelines.
max.pipelines.lib.log_probabilitiesBuilds computation graphs for log probabilities over batched input sequences.
max.pipelines.lib.module_registrationServes architectures that register their modules by role.
max.pipelines.lib.pipeline_parallelPipeline parallelism: a decoder cut into stages of contiguous layers.
max.pipelines.lib.registration_draftBuilds registered draft components for the shared speculative pipeline.
max.pipelines.lib.registration_encoderServes embedding architectures that register an ENCODER.
max.pipelines.lib.registration_graph_apiServes architectures that register a graph-API module by role.
max.pipelines.lib.registration_runtimeCompletes a registered module's config with its deployment.
max.pipelines.lib.registryModel registry, for tracking various model variants.
max.pipelines.lib.request_statePer-request state a registered decoder carries from one step to the next.
max.pipelines.lib.role_contractsThe inputs and outputs of each role a registered module fills.

Configuration​

DenoisingCacheConfigResolved denoising-cache config.
DenoisingCacheSettingsUser denoising-cache settings.
KVConnectorConfigKV cache connector configuration: the connector type and its settings.
MAXConfigAbstract base class for all MAX configs.
MAXModelConfigBaseAbstract base class for MAX model configuration.
PipelineArgsUser-settable input arguments for a pipeline.
PipelineRuntimeConfigModel-agnostic runtime settings for pipeline execution.

Pipelines​

EmbeddingsPipelineTypeBase type of EmbeddingsPipeline, a Pipeline mapping embeddings inputs to embeddings outputs.
OverlapTextGenerationPipelineOverlap text generation pipeline.

Model interface​

AlwaysSignalBuffersMixinMixin for models that always require signal buffers.
GraphPipelineModelWithKVCacheGraph-API pipeline model with shared compile-and-load template.
ModuleV3MultiGraphPipelineModelWithKVCacheModuleV3 VLM with separate vision and language compiled callables.
ModuleV3OutputsThe outputs a ModuleV3 text model's forward returns.
ModuleV3PipelineModelWithKVCacheThe base class for a ModuleV3 model architecture that uses a KV cache.
PipelineModelWithKVCacheA pipeline model that supports KV cache.
SupportsSSMStateWarmupProtocol for pipeline models holding state slots of their own.
UnifiedEagleOutputsOutputs from a unified EAGLE graph execution.
UnifiedSpecDecodeInputsShared spec-decode fields, packed in the order the graph takes them.

Tokenizers​

AudioGenerationTokenizerTurns an audio generation request into an AudioContext.
PixelGenerationTokenizerEncapsulates creation of PixelContext and specific token encode/decode logic.

Weight transform​

BitcastReinterprets the bytes of matching weights as another dtype.
CheckpointRoleA checkpoint, by its role in the pipeline.
IgnoreDrops matching weights, for a stated reason.
WeightEntryNames one checkpoint's weights in a graph.
WeightNamingThe entries that name each graph's weights, keyed by graph role.
WeightRenamingRenames weights by regular expression substitution.
WeightRuleOne step of a WeightTransform.
WeightTransformMaps a checkpoint's weights onto the names and dtypes a graph loads.

Utilities​

CompilationTimerTimer for logging graph build, compile, and init phases.
HuggingFaceRepoHandle for interacting with a Hugging Face repository (remote or local).
MemoryPlanThe memory plan computed when a pipeline is loaded.
ModelManifestRegistry mapping semantic role strings to MAXModelConfig instances.
RetrievedPipelineEverything PipelineRegistry.retrieve_factory() resolves.
VisionPreprocessCacheLRU cache of preprocessed media payloads, bounded by total bytes.
WeightPathParserParses and validates weight paths for model configuration.

Functions​

apply_weight_entriesReturns the weights a graph's entries name.
build_eos_tracker_for_requestBuilds an EOSTracker from request sampling params.
convert_max_config_valueConverts a config value to the appropriate type.
deep_merge_max_configsDeep merge two MAXConfig configuration dictionaries.
generate_local_model_pathGenerates the local filesystem path where a Hugging Face model repo is cached.
get_default_max_config_file_section_nameGets the default section name for a MAXConfig class.
max_tokens_to_generateReturns the max number of new tokens to generate.
parse_quant_configParses scaled quantization config from HuggingFace config and state dict.
process_ragged_kv_outputsMaps standard ragged+KV logits buffers to ModelOutputs.
ragged_kv_symbolic_inputsReturns symbolic graph inputs for a standard ragged KV text model.
resolve_eos_token_idsReturns every token id that ends generation for the model.
resolve_max_config_inheritanceResolves configuration inheritance by loading base config and merging.
try_to_load_from_cacheWrapper around huggingface_hub.try_to_load_from_cache; validates repo exists.
validate_hf_repo_accessValidates repository access and raises clear, user-friendly errors.

Expert load balancing​

EplbStatsAccumulatorOwns per-GPU persistent int64 histogram buffers.
EplbStatsMetadataStatic metadata describing the shape and semantics of a snapshot.
EplbStatsSnapshotAn immutable snapshot of per-layer logical-expert routing counts.
balanced_packingPack n weighted objects into num_packs packs evenly.
rebalance_expertsEntry point for expert-parallelism load balancing.
rebalance_experts_hierarchicalHierarchical rearrangement: groups -> nodes -> GPUs.
replicate_expertsReplicate num_log logical experts to num_phy physical slots.

Output parsers​

get_parser_clsLook up a registered reasoning parser class by name.
get_parser_clsLook up a registered parser class by name without instantiating it.

Constants​

DEFAULT_MAX_BATCH_INPUT_TOKENSDefault max batch input tokens for chunked prefill and memory estimation.