For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.lib
Types to interface with ML pipelines such as text/token generation.
Submodules
max.pipelines.lib.arch_lookup | Architecture registration and lookup tables. |
|---|---|
max.pipelines.lib.auto_processor | Turns chat conversations with inline images into AutoModel inputs. |
max.pipelines.lib.decoder_scratch | Workspace buffers a registered decoder's kernels write each step. |
max.pipelines.lib.generate | Fixed-batch generation, the lowest-level entry point to the pipeline API. |
max.pipelines.lib.generate_images | Image generation from text prompts and input images, in-process. |
max.pipelines.lib.graph_specs | Compiles a pipeline's graphs by role. |
max.pipelines.lib.interfaces | Interfaces for MAX pipelines. |
max.pipelines.lib.log_probabilities | Builds computation graphs for log probabilities over batched input sequences. |
max.pipelines.lib.module_registration | Serves architectures that register their modules by role. |
max.pipelines.lib.pipeline_parallel | Pipeline parallelism: a decoder cut into stages of contiguous layers. |
max.pipelines.lib.registration_draft | Builds registered draft components for the shared speculative pipeline. |
max.pipelines.lib.registration_encoder | Serves embedding architectures that register an ENCODER. |
max.pipelines.lib.registration_graph_api | Serves architectures that register a graph-API module by role. |
max.pipelines.lib.registration_runtime | Completes a registered module's config with its deployment. |
max.pipelines.lib.registry | Model registry, for tracking various model variants. |
max.pipelines.lib.request_state | Per-request state a registered decoder carries from one step to the next. |
max.pipelines.lib.role_contracts | The inputs and outputs of each role a registered module fills. |
Configuration
DenoisingCacheConfig | Resolved denoising-cache config. |
|---|---|
DenoisingCacheSettings | User denoising-cache settings. |
KVConnectorConfig | KV cache connector configuration: the connector type and its settings. |
MAXConfig | Abstract base class for all MAX configs. |
MAXModelConfigBase | Abstract base class for MAX model configuration. |
PipelineArgs | User-settable input arguments for a pipeline. |
PipelineRuntimeConfig | Model-agnostic runtime settings for pipeline execution. |
Pipelines
EmbeddingsPipelineType | Base type of EmbeddingsPipeline, a Pipeline mapping embeddings inputs to embeddings outputs. |
|---|---|
OverlapTextGenerationPipeline | Overlap text generation pipeline. |
Model interface
AlwaysSignalBuffersMixin | Mixin for models that always require signal buffers. |
|---|---|
GraphPipelineModelWithKVCache | Graph-API pipeline model with shared compile-and-load template. |
ModuleV3MultiGraphPipelineModelWithKVCache | ModuleV3 VLM with separate vision and language compiled callables. |
ModuleV3Outputs | The outputs a ModuleV3 text model's forward returns. |
ModuleV3PipelineModelWithKVCache | The base class for a ModuleV3 model architecture that uses a KV cache. |
PipelineModelWithKVCache | A pipeline model that supports KV cache. |
SupportsSSMStateWarmup | Protocol for pipeline models holding state slots of their own. |
UnifiedEagleOutputs | Outputs from a unified EAGLE graph execution. |
UnifiedSpecDecodeInputs | Shared spec-decode fields, packed in the order the graph takes them. |
Tokenizers
AudioGenerationTokenizer | Turns an audio generation request into an AudioContext. |
|---|---|
PixelGenerationTokenizer | Encapsulates creation of PixelContext and specific token encode/decode logic. |
Weight transform
Bitcast | Reinterprets the bytes of matching weights as another dtype. |
|---|---|
CheckpointRole | A checkpoint, by its role in the pipeline. |
Ignore | Drops matching weights, for a stated reason. |
WeightEntry | Names one checkpoint's weights in a graph. |
WeightNaming | The entries that name each graph's weights, keyed by graph role. |
WeightRenaming | Renames weights by regular expression substitution. |
WeightRule | One step of a WeightTransform. |
WeightTransform | Maps a checkpoint's weights onto the names and dtypes a graph loads. |
Utilities
CompilationTimer | Timer for logging graph build, compile, and init phases. |
|---|---|
HuggingFaceRepo | Handle for interacting with a Hugging Face repository (remote or local). |
MemoryPlan | The memory plan computed when a pipeline is loaded. |
ModelManifest | Registry mapping semantic role strings to MAXModelConfig instances. |
RetrievedPipeline | Everything PipelineRegistry.retrieve_factory() resolves. |
VisionPreprocessCache | LRU cache of preprocessed media payloads, bounded by total bytes. |
WeightPathParser | Parses and validates weight paths for model configuration. |
Functions
apply_weight_entries | Returns the weights a graph's entries name. |
|---|---|
build_eos_tracker_for_request | Builds an EOSTracker from request sampling params. |
convert_max_config_value | Converts a config value to the appropriate type. |
deep_merge_max_configs | Deep merge two MAXConfig configuration dictionaries. |
generate_local_model_path | Generates the local filesystem path where a Hugging Face model repo is cached. |
get_default_max_config_file_section_name | Gets the default section name for a MAXConfig class. |
max_tokens_to_generate | Returns the max number of new tokens to generate. |
parse_quant_config | Parses scaled quantization config from HuggingFace config and state dict. |
process_ragged_kv_outputs | Maps standard ragged+KV logits buffers to ModelOutputs. |
ragged_kv_symbolic_inputs | Returns symbolic graph inputs for a standard ragged KV text model. |
resolve_eos_token_ids | Returns every token id that ends generation for the model. |
resolve_max_config_inheritance | Resolves configuration inheritance by loading base config and merging. |
try_to_load_from_cache | Wrapper around huggingface_hub.try_to_load_from_cache; validates repo exists. |
validate_hf_repo_access | Validates repository access and raises clear, user-friendly errors. |
Expert load balancing
EplbStatsAccumulator | Owns per-GPU persistent int64 histogram buffers. |
|---|---|
EplbStatsMetadata | Static metadata describing the shape and semantics of a snapshot. |
EplbStatsSnapshot | An immutable snapshot of per-layer logical-expert routing counts. |
balanced_packing | Pack n weighted objects into num_packs packs evenly. |
|---|---|
rebalance_experts | Entry point for expert-parallelism load balancing. |
rebalance_experts_hierarchical | Hierarchical rearrangement: groups -> nodes -> GPUs. |
replicate_experts | Replicate num_log logical experts to num_phy physical slots. |
Output parsers
get_parser_cls | Look up a registered reasoning parser class by name. |
|---|---|
get_parser_cls | Look up a registered parser class by name without instantiating it. |
Constants
DEFAULT_MAX_BATCH_INPUT_TOKENS | Default max batch input tokens for chunked prefill and memory estimation. |
|---|