MongoPaginateByCountPartitioner

最新推荐文章于 2024-08-10 07:05:52 发布

Bol5261

最新推荐文章于 2024-08-10 07:05:52 发布

阅读量367

点赞数

CC 4.0 BY-SA版权

分类专栏： MongoDB(Collenction-oriented) 文章标签： mongodb

原文链接：https://www.javadoc.io/doc/

MongoDB(Collenction-oriented) 专栏收录该内容

192 篇文章

订阅专栏

本文详细介绍了MongoDB Spark RDD中的MongoPaginateByCountPartitioner类，它通过计数进行集合分页，将集合分割成指定数量的最大分区。文章讨论了配置属性，如partitionKey和numberOfPartitions，并说明了其在MongoDB输入操作中的使用。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Class/Object

com.mongodb.spark.rdd.partitioner
MongoPaginateByCountPartitioner
Related Docs: object MongoPaginateByCountPartitioner | package partitioner
Permalink
class MongoPaginateByCountPartitioner extends Logging with MongoPartitioner with MongoPaginationPartitioner

The pagination by count partitioner.

Paginates the collection into a maximum number of partitions.
Configuration Properties

The prefix when using sparkConf is: spark.mongodb.input.partitionerOptions followed by the property name:

partitionKey, the field to partition the collection by. The field should be indexed and contain unique values. Defaults to _id.
numberOfPartitions, the maximum number of partitions to create. Defaults to 64.

Note: This can be a expensive operation as it creates 1 cursor for every partition.

Since

1.0

Linear Supertypes
Known Subclasses
Ordering

Alphabetic
By Inheritance

Inherited

MongoPaginateByCountPartitioner
MongoPaginationPartitioner
MongoPartitioner
Serializable
Serializable
Logging
LoggingTrait
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Instance Constructors

new MongoPaginateByCountPartitioner()

Value Members

val numberOfPartitionsProperty: String

The number of partitions property
val partitionKeyProperty: String

The partition key property
def partitions(connector: MongoConnector, readConfig: ReadConfig, pipeline: Array[BsonDocument]): Array[MongoPartition]

Calculate the Partitions