Here's a quick look at what's landed since the preview opened and how you can start evaluating Neki for your workload today.

What's shipped

We've recently added both read-only replicas and smaller Metal instances to Neki starting at just $50 per month.

Read-only replicas were a hotly-requested feature for Postgres, particularly for customers with a large, global userbase. Neki now also supports read-only replicas for shards in your cluster.

We also now support smaller Metal node sizes, going all the way down to database nodes with 1/8th vCPU, 1GB RAM, and 10GB of locally-attached NVMe SSD.

You can try both of these capabilities on Neki in PlanetScale today. These are just a few amongst many others, including:

• Manage Neki extensions with Terraform and the CLI (Sept 18). Enable extensions by name with extensions in a planetscale_neki_configuration_profile resource or with pscale branch config-profile update --extensions, and change their settings in the same update.
• Branch metrics in the PlanetScale MCP server (Oct 9). planetscale_get_branch_metrics pulls CPU, memory, disk, replication lag, and connections into your AI agent, or wherever you connect to your infrastructure via MCP. It works for Neki, Postgres, and Vitess.
• PlanetScale on Azure (Oct 13). You can now create Neki and Postgres databases in Azure regions, alongside AWS and Google Cloud.
• Neki router parameters in Terraform (Oct 14). Terraform provider v1.13.0 adds parameter management to planetscale_neki_router, and the planetscale_databases data source can now filter by kind (mysql, postgresql, or neki).
• CLI shows which parameters need an update (Oct 14). CLI v0.346.0 adds a "requires update" column for Neki configuration profiles, admins, and sidecars, plus a requires_update field in JSON output.
• Huge pages for Neki and Postgres (Oct 15). Set huge_pages to on in a configuration profile's Parameters tab on supported cluster sizes. The change requires a restart.

Under the hood: query layer

Much of the engineering effort since preview has gone into the query layer: making more of your existing SQL run on a sharded database without changes, and making the standard Postgres tools you already use work against Neki.

The team has merged more than 650 changes since the preview opened. A few more highlights:

• More of your SQL just works. INSERT ... SELECT ... ON CONFLICT, cross-shard UPDATE ... FROM, DELETE ... USING and MERGE, correlated EXISTS almost anywhere in a query, ARRAY(SELECT ...), SQL cursors, LISTEN/NOTIFY, materialized views, and PL/pgSQL exception handlers are all now supported.
• Export with standard tools. COPY ... TO STDOUT now works on sharded tables in text, CSV, and binary, so pg_dump and other export tools can read data out of a sharded Neki database.
• Faster bulk writes. Distributed INSERT ... SELECT is about 76x faster than at launch, and COPY ingest is roughly 5x faster.
• See how your query runs on every shard. EXPLAIN (NEKI_PG_PLAN) shows Neki's distributed plan together with the Postgres plan each shard ran. The output follows Postgres's own EXPLAIN format, so pgAdmin, DBeaver, explain.dalibo.com, and explain.depesz.com can read it.

Start evaluating now

If you're running Postgres and expect to need sharding or any other Neki features like online schema changes or no downtime version upgrades, now is a good time to start the conversation. Email us at neki@planetscale.com or fill out the form, and we'll set up a call with our solutions team.

If you want to learn more about Neki, we recommend starting with When to shard, Neki best practices, and Data topology.

For the past eight months, that work has been the Neki query planner and its runtime engine. In this blog, I will discuss how Neki, PlanetScale's sharded Postgres, implements sequences in our sharded system to match native Postgres. I will focus on two aspects that need special attention: volatile expressions and cross-shard synchronous evaluation. For some additional background on Neki, see these blogs: Neki's architecture and What is a data topology.

Implementing Postgres sequences in Neki

In PostgreSQL, sequences are ubiquitous. A sequence is abstractly a unique number generator, which can be used to implement a counter, an identity, or a surrogate key.

A key property of sequences is their guarantee to never generate the same value more than one time. (That guarantee has a caveat of its own: a sequence can be reset or cycled, which clearly would repeat values, but these aspects are out of scope for this post.) Interestingly, a sequence does not guarantee that its output numbers are always consecutive; when transactions roll back, for example, consumed sequence numbers are not reclaimed.

Can Postgres handle sequences for us?

Each Neki shard is its own complete database system with its own query planner and an extensive SQL language support, so the Neki query planner can offload a lot of work to it. In fact, we view it as a virtue to do so: if we can relay the entire query to the shard, that is the best possible case. But generally, that is not possible to do.

The most obvious reason is that no shard has all the data, they only have their slice. Detecting these best cases requires a careful and complicated analysis of the SQL statement and the Neki data topology, with a strong understanding of the semantics of the SQL language and its PostgreSQL dialect with its various unique features.

That's where we come in.

But this post is not about these best cases and how we detect them. Instead, our focus is on some of the pathological cases where we cannot relay all the work to one of the shards. As I will explain, sequences fall into this category where we must assume centralized control. But before we go into sequences, we must understand some details of Neki query planning.

The general case of Neki query planning

In the most general case, each shard only needs to provide raw data. Everything else has to be possible in the router itself. For a simple query like Query 1 below, each shard hosting the (sharded) table animals returns its rows with the requested columns, and the router concatenates those rowsets into the full result and returns it to the client.

Query 1: SELECT...FROM

If we extend this query with some simple expressions over the columns, seen in Query 2, we now have a choice: evaluate the expressions in the router, or on the shard?

Query 2: add simple expressions

Both choices will give the correct result, so the question is one of optimization, which as it turns out is nontrivial:

• Is the router or the shard more resource-constrained?
• Which of the two is actually fastest to evaluate?
• Is it more efficient to network-transfer the evaluated expression, or the raw data from the column?
• What about nested expressions; shall we split them and evaluate one part here and one part there?
• and so on

For the purpose of simplicity, we will here assume that Neki is using a simple heuristic whereby we always push the evaluation to the shard if we can.

Router-only expressions

But some expressions don’t give us that choice. Neki defines this class of expressions as router-only: they must be evaluated on the router and we must never push them down to the shard.

One example of such an expression is the now() function. It returns the wall clock time recorded at the start of the transaction.

If we let each shard evaluate it, the natural clock drift and the different arrival times of the request would produce different values on different shards, which violates the semantics of the expression. So we need to evaluate it in the router.

Consider Query 3, which adds now() to the mix.

Query 3: adding now()

A valid approach here is to run Query 2 on the shards, concatenate the results, and then evaluate now() in the router and append the value to each row. We don’t know how many rows the shards will return, so we have to make sure evaluating now() repeatedly in the router always gives the same value. There are a couple of ways to guarantee that.

We could evaluate it once into a single-row, single-column table and join that with the rows from the shards. Or we could cache the value per statement and read from the cache on repeated evaluations. Either way, every row gets the same wall clock time.

And this summarizes the Neki query planner's general model:

• Detect what parts of the query cannot run on the shards
• Compose a plan wherein those parts instead run in the router
• Send most other things into the shards

How Postgres classifies expressions

So far, so good, but what about sequences? In order to fully understand how to deal with sequences, we need to understand PostgreSQL's expression classification of immutable, stable, and volatile. During semantic analysis, one of the phases of query planning, every expression in the statement gets one of these classifications based on its own definition and its inputs.

I've already shown you examples of the first two classes.

The immutable class

In Query 2, all the expressions are immutable. This means that they will produce the same output each time they operate on the same inputs, regardless of how many times they are evaluated and even between statements. Such expressions are the majority, and they are simple enough to not warrant additional discussion here.

The stable class

A stable expression produces the same output within a statement. The now() function from Query 3 is one example. to_char() is another, and a useful one since it takes inputs.

The output of now() depends on the TimeZone GUC, as shown below:

Table 1

to_char() takes a datetime value and a format string. Even with identical inputs, its output can differ because it depends on the lc_time GUC for display formatting. Table 2 shows this for a few lc_time values.

Table 2

For Neki, this means we cannot blindly push stable expressions to the shards. We must also make sure their dependencies are the same on every shard. Otherwise we might get 'Wednesday' back on one row and 'Onsdag' on another.

This can happen when a user includes a SET lc_time = ...; in their session. In this case, we have to push that statement to any shard that needs it for evaluation. The set_config() function is treated the same way. now() as I explained above is router-only for clock-synchronization reasons, but to_char() is not.

Enter volatility

A stable expression returns the same value every time it's evaluated within a statement. A volatile expression makes no such promise. It may produce a different output on every single invocation. It doesn't always, but it can.

As an example, consider the nextval() function. It takes the name of a sequence and returns a new, unique value each time it's evaluated. This is very useful, and can be used to implement a counter in a query or a table.

A sequence is created either using CREATE SEQUENCE in a dedicated DDL statement, or declared on a column with GENERATED AS IDENTITY (or serial in older versions), or DEFAULT nextval('seq') syntax in a CREATE TABLE statement.

In Neki, these sequence tables exist on all shards, but they all start with an empty sequence. Since it is possible to reference any sequence in any query (assuming no ACL rules), evaluations on sequences require centralized governance to avoid duplicate values across shards.

Thus, sequence operations are all router-only. The authoritative shard is the physical host for all sequences, which the router uses to get new sequence values to distribute. This is how Neki achieves cross-shard synchronization for sequence allocation.

So, all's good then? Well, there is one more complication: cardinality. This is a fancy word for "number of rows", and is relevant in the context of volatility. In order to evaluate volatile expressions the correct number of times, we need to evaluate them with the correct cardinality (again, number of rows). So how do we know what the cardinality is? We need to understand the definition of a table expression.

Table expressions

The driving force to evaluate expressions in the query execution model is the row cardinality. That is, each expression must be evaluated once per row to obtain the output needed for that row. That seems simple enough, but the complexity lies in "once per row" — as it turns out, determining that is also non-trivial!

A SQL statement is made up of clauses that form an abstract data model. We will not go into all details of that here, but I will define what a table expression is.

Consider Query 4, which joins two tables animals and feedings, forming an intermediate result table, with an additional predicate in the WHERE clause.

Query 4: Building a table expression from a JOIN

The composition of FROM ... JOIN ... WHERE ... (and GROUP BY, HAVING, and more) form a logical unit known as a table expression. The table expression forms the basis for the evaluation of all expressions in the SELECT clause's output list. It is the rows of the table expression that determine the "once per row" condition of the volatile expressions. To understand the nuances of this, assume that the tables animals and feedings exist as shown in Table 3 and Table 4, and that they are both sharded.

Table 3, animals

Table 4, feedings

Joining the tables on their id columns gives us every pair of rows where the id values match. That’s an inner join, which is the default when JOIN has no qualifier in front of it. In our case, that matches both rows of animals with the first three rows of feedings, so the join table has three rows, shown in Table 5.

Table 5, animals JOIN feedings ON animals.id = feedings.id

But we also have the predicate in the WHERE clause, which only retains rows where feedings.time is 'morning'. That leaves us with just one row in our final table expression, shown in Table 6.

Table 6, animals JOIN feedings ON animals.id = feedings.id WHERE feedings.time = 'morning'

It is this table expression that forms the basis of evaluation for our SELECT clause expressions. So, we would compute 8 + 1, lower('BISMUTH'), nextval('tag_seq'), and then we are good to return results! We know that we will only evaluate the volatile nextval('tag_seq') expression one time, and thus we will only consume one sequence value.

Now, assume that we drop the WHERE predicate, and we use instead Query 5.

Query 5: drop the WHERE

Now, our table expression would be equal to Table 5, and we now have three rows to evaluate. The keen observer will note, as could a sophisticated optimizer, that the values of age and name are equal over two rows, since these are join-multiplied copies from the same source row from animals.

So it would be tempting to not evaluate these expressions more than is necessary. For the data scale we are using in this example, it would not make much difference, but for millions of rows it could. But here we must take into account the volatility aspect: we cannot make this optimization for the nextval('tag_seq') expression, since it will give a different value each time and must be evaluated afresh on each row.

The Neki case: a distributed query planner

Now, I must also mention that it is entirely valid to swap the order of evaluation between the JOIN and the WHERE. That is, it would be fine for us to first filter down feedings to its two 'morning' rows, and then join that subset table with animals. Which order to do it is a general optimization problem, but it is often beneficial to filter first and join afterwards, since evaluating the predicate is often cheap.

Optimizing the operation of computing the table expression is a core focus of any query planner's optimizer, known generally as join ordering optimization (e.g. consider a longer list of JOINs and which one to do first). The PostgreSQL planner has its own optimization algorithms for this, and it is not guaranteed that Neki and Postgres agree on the order of evaluation. If volatile expressions are part of any of the predicates (in WHERE or in JOIN ... ON), then results are not always deterministic, but they are correct. But let's return to the case for Neki, where we have a distributed query planner.

In Neki, a very large cost is the network-transport time between router and shard. In the case of a cross-shard join, which we are focusing on here, we have to go to some shards to fetch data for animals and some (same or different) shards to fetch data for feedings. Only when we have both concatenated sets of data are we able to perform our join. We are unable to do it on any given shard, since it does not have all of the data either. Thus, we have a great interest in minimizing the traffic going between router and shard.

Neki is able to determine when all rows of the table expression can be assembled independently on each shard, which typically requires that the join predicate uses the table's shard index. That is, if animals and feedings were both sharded on their id column, then we know that we can do a shard-local join, and concatenate the results from that instead of computing the join on the router. This is correct because rows from other shards would all be unequal on the join columns. In this case, each shard contributes a subset of the table expression.

Putting it all together

All right, bringing everything together, assume again that we have sharded tables animals and feedings on their id columns, respectively, and that we are processing Query 6.

Query 6: add now() back

We note that the table expression can be evaluated shard-locally, and thus we can push down to shard-local evaluation on all the immutable expressions (first two columns). But we still have to evaluate both now() and nextval('tag_seq') on the router to avoid the uncoordinated shards from generating conflicting values.

now() is stable, so we evaluate it once and use the same value on all the outgoing rows. nextval('tag_seq') is volatile, so we must generate a new value for each row. And since we don't know how many rows each shard will return, we cannot generate the sequence values in advance and then draw from a cache. We must interpose the evaluations on top of the dynamic stream.

We will close by considering a more complex case, shown in Query 7, and by applying our concepts from above.

Query 7: an UPDATE with nextval()

This is a DML (Data Modifying Language) query. It updates each row of the animals table by setting the tag_id column to the next sequence value from the sequence 'tag_seq'. In an UPDATE query, the table expression is expressed in a different syntactic form. You can build a complex one with WHERE and FROM clauses, but here we'll stick with the trivial case: the table expression is just the relation named after UPDATE, which is animals.

This is a challenge for Neki.

The cardinality of animals is unknown, and its rows are spread out across the shards in an unknown distribution. We cannot send nextval('tag_seq') to the shard, as we discussed above, and we don't know how many values to produce. So what query do we send to the shards?

The answer is not obvious, and there is more than one solution. What Neki does today is first establish the table expression in the router using Query 8, interpose evaluation of nextval('tag_seq') on top, and run the actual update, feeding it with parameterized sequence values.

Query 8: what Neki sends first

While in theory the trivial table expression includes all columns of animals, Neki knows that for Query 7, only the primary key columns and sharding index columns are necessary. In this case, id serves both purposes. The additional FOR UPDATE locks the rows until the Query 9 requests come in to complete the modification.

For each incoming row to the router, Neki evaluates nextval('tag_seq') and sends the shard-global unique sequence value down to be set on the correct shard's row. Neki routes to the shard using the id value as the shard index, and filters down to the correct row with a WHERE clause using the id value as primary key.

Query 9: what Neki sends per row

An illustration of the data flow:

I leave for the reader's exercise the question of how a stable expression such as now() would be handled for a variation of Query 7. It is much simpler than the volatile discussion since it is not dependent on the cardinality.

I hope this deep dive into Neki query planning brought you insights into our complex system, and how the classical problems of query planning map to the distributed system that is Neki.

Happy hacking!

Mats

Why Azure?

Many teams run their apps on Azure, and we've received consistent requests, especially over the past year, to bring PlanetScale to where they're building. Now those teams can use PlanetScale alongside all their other Azure compute and infrastructure.

What's included?

Databases on Azure get the same PlanetScale features you use on our other clouds: high availability with a primary and replicas spread across 3 availability zones, Query Insights, schema recommendations, automated backups, and more. That includes Metal clusters with locally attached NVMe storage, the fastest way to run databases in the cloud.

Supported Azure regions

Azure support is currently available in the following regions:

• centralus (Iowa, USA)
• eastus2 (Virginia, USA)
• westeurope (Netherlands)

If you'd like to see PlanetScale in other Azure regions, let us know. See the regions documentation for the full list across all cloud providers.

What it took

Bringing PlanetScale to a new cloud took significant time, particularly in testing and ensuring it behaves just as seamlessly on Azure as it does on AWS and GCP. Every cloud has its quirks, and Azure is no exception.

Kubernetes pod networking is a good example. PlanetScale runs databases on Kubernetes, and every database pod needs its own IP address on the network. We wanted those addresses to come from the Azure virtual network directly, with native routing and no overlay for maximum performance and low latency.

The simple approach is to request one IP from Azure per pod, which causes two problems. Azure caps a virtual network at 65,536 private IP addresses, which would limit the whole network to about 65,000 pods. It also puts the Kubernetes API server in the path of every pod start, which slows things down under load.

We solved both by handing out addresses in blocks. Each node gets blocks of 16 IPs (/28 prefixes) on its network interface, and Azure counts each block as a single address toward the cap. That increases our overall IP capacity by 16x, to over a million pod IPs per network. A background service on each node tracks its blocks, and a lightweight CNI plugin assigns pod IPs from them locally, so the API server stays out of the way. When a node gets down to 8 free addresses, it grabs another block, up to 256 blocks (4,096 pod IPs) per node.

This sounds like a simple solution, but it took a surprising amount of custom work to get just right.

Azure had a few additional networking challenges along the way. /28 is the only block size it offers, so the design had to work around that. Older versions of the network interface API don't return prefixes at all, and the block's range comes back in a field normally used for a single address. There's also a chicken-and-egg problem: system pods need IPs before the Kubernetes operator that manages the blocks is running. To handle that, the operator has a bootstrap mode that runs on the host network.

In scale testing, one cluster ran over 6,700 pods across more than 100 node pools, with the busiest nodes holding 15 blocks (240 IPs) each.

This is just one of many pieces we had to carefully construct to make PlanetScale on Azure a seamless experience for our customers. Storage, load balancing, identity, and backups all needed the same level of care. All of that work culminates in today's launch.

Getting started

To create a database on Azure, choose Azure as the cloud provider when you create a new database, then select a region. Existing databases can be migrated to Azure by contacting support.

If you have questions or want help planning a move, get in touch.

That one extra byte is enough to trigger TOAST, the part of Postgres that compresses values once their row gets too big, and moves them out of the row when compressing isn't enough.

Postgres 19, now in beta, changes how TOAST compresses, switching the default from pglz to LZ4 in builds with LZ4 support. PlanetScale already made this switch on Postgres 17 and 18.

Here, we look at which values get TOASTed, what TOAST does to them, and when LZ4 is the right choice. Let's dig in!

What a row costs in bytes

Compression is all about saving bytes, so we'll start with where the bytes in a row go.

Postgres stores table data in fixed 8 KB pages. A row can't be split across pages, so every row has to fit inside one.

Each row starts with a 24-byte header that Postgres uses to track which transactions can see it, followed by the values of its columns. The space those values take depends on their type, and every type falls into one of two groups:

• Fixed-width types like bigint, timestamptz, and boolean always take the same number of bytes.
• Variable-length types like text, jsonb, and bytea take as many bytes as the value needs, plus a length header: 1 byte for values up to 126 bytes, and 4 bytes for anything longer.

pg_column_size shows both groups in a single row:

SELECT pg_column_size(ROW(1::bigint, now(), true, 'Ada'::text));
 pg_column_size
----------------
             45
(1 row)

That's the 24-byte header, 17 bytes for the bigint, timestamptz, and boolean (8, 8, and 1), and 4 bytes for 'Ada': three characters plus a 1-byte length header. Change 'Ada' to 'Ada Lovelace', and the row grows to 54 bytes while the other columns stay at 17.

Variable-length values are what make a row grow, and the only ones Postgres can compress.

Small rows don't trigger compression

TOAST stands for The Oversized-Attribute Storage Technique. As the name suggests, TOAST only steps in when a row gets too big.

Postgres treats a row as oversized once it's larger than 2,032 bytes, about a quarter of an 8 KB page. This threshold was set so that at least four rows fit on every page, after room for the page header and a pointer to each row.

Here's what happens when we store the most compressible value possible, a single repeated letter, at lengths just below and just above the threshold. pg_column_compression returns the algorithm that compressed each value, or null if the value isn't compressed, which we display as none. The COMPRESSION lz4 clause picks the algorithm.

CREATE TABLE t (id int PRIMARY KEY, body text COMPRESSION lz4);

INSERT INTO t VALUES
  (1500, repeat('x', 1500)),
  (2000, repeat('x', 2000)),
  (2001, repeat('x', 2001));

SELECT id AS chars,
       pg_column_size(body) AS stored_bytes,
       coalesce(pg_column_compression(body), 'none') AS compression
FROM t ORDER BY id;
 chars | stored_bytes | compression
-------+--------------+-------------
  1500 |         1504 | none
  2000 |         2004 | none
  2001 |           26 | lz4
(3 rows)

The first two values aren't compressed. Each takes 4 bytes more than its length, for the length header. The third value, however, is just one character longer, and it's compressed to 26 bytes.

The 2,000-character row is exactly 2,032 bytes: 24 for the row header, 4 for the int, and 2,004 for the text. That's right on the threshold, so TOAST leaves it alone. The 2,001-character row is one byte over.

TOAST does two different jobs

The threshold applies to the whole row, but TOAST works on individual variable-length values. Once a row is over the threshold, TOAST has two ways to shrink it:

• Compress a value, and keep the smaller version in the row, like the 2,001-character row.
• Move a value out of the row, into a separate table.

That separate table is called a TOAST table. Tables with columns that can hold large values get one automatically in the pg_toast schema.

A value moved to the TOAST table isn't stored in one piece, though. TOAST splits it into chunks of up to 1,996 bytes and stores each chunk as its own row, tagged with the value's ID (chunk_id) and its position in the value (chunk_seq). The rest of the 2,032 bytes holds the chunk row's header and IDs, so four chunk rows fit on a page. The original row keeps an 18-byte pointer, so Postgres can find the chunks and reassemble them.

TOAST works through a row's values from largest to smallest. As soon as the row is back under 2,032 bytes, the table's toast_tuple_target, it stops.

To see every outcome, we store four very different values in one table: a short string, 10 KB of repeated text, 200 KB of random characters, and 1.1 MB of a repeated sentence. stored_bytes is the value's size wherever it ended up, including the TOAST table for a moved-out value. octet_length returns a value's uncompressed size, and pg_column_toast_chunk_id (Postgres 17+) returns its ID in the TOAST table, or null if the value is still in the row.

CREATE TABLE docs (id int PRIMARY KEY, body text COMPRESSION lz4);

INSERT INTO docs VALUES
  (1, repeat('x', 200)),
  (2, repeat('abc', 3400)),
  (3, (SELECT string_agg(md5(random()::text), '') FROM generate_series(1, 6250))),
  (4, repeat('The quick brown fox jumps over the lazy dog. ', 25000));

SELECT id,
       octet_length(body) AS raw_bytes,
       pg_column_size(body) AS stored_bytes,
       coalesce(pg_column_compression(body), 'none') AS compression,
       pg_column_toast_chunk_id(body) IS NOT NULL AS moved_out
FROM docs ORDER BY id;
 id | raw_bytes | stored_bytes | compression | moved_out
----+-----------+--------------+-------------+-----------
  1 |       200 |          204 | none        | f
  2 |     10200 |           60 | lz4         | f
  3 |    200000 |       200000 | none        | t
  4 |   1125000 |         4471 | lz4         | t
(4 rows)

Each value lands somewhere different:

• Row 1 is under the threshold, so TOAST never touches it.
• Row 2 compresses from 10 KB to 60 bytes, small enough to stay in the row.
• Row 3 is random, so compressing it doesn't save space. TOAST moves the value out uncompressed.
• Row 4 compresses to 4,471 bytes, which is still too big for the row, so TOAST compresses it and moves it out.

Each column's storage strategy decides which of those jobs TOAST can do:

• PLAIN: neither. Used by fixed-width types like int.
• EXTENDED: compress, then move out if needed. The default for text, jsonb, and bytea.
• EXTERNAL: move out, but never compress.
• MAIN: compress, but keep the value in the row even above the usual TOAST target if the row still fits on a page. Move it out only as a last resort. The default for numeric.

You can change a column's strategy with ALTER TABLE docs ALTER COLUMN body SET STORAGE EXTERNAL. It doesn't, however, change values already stored.

TOAST isn't a large object

Postgres has a second way to store big values, called large objects, built for whole files like videos or PDFs.

In Can you put a video in Postgres?, we stored a 4 GB video as a large object. lo_import loads the file into the pg_largeobject system table, split into about 2 million 2 KB chunks, and returns an OID that our video table keeps as a reference. To stream the video to a browser, lo_get reads any byte range straight from those chunks.

TOAST and large objects both store big values in chunks, but they differ in three ways:

• TOAST is automatic. It applies to any variable-length column without you asking. A large object only exists if you create one.
• TOAST values top out at 1 GB. Large objects can hold up to 4 TB.
• TOAST values are deleted with their row. A large object stays until you remove it with lo_unlink, even after the row that references it is gone.

For files over 1 GB, you need a large object. Anything smaller fits in a regular column, where TOAST can also compress it.

How pglz and LZ4 work

Postgres can compress a TOAST value with one of two algorithms, pglz or LZ4. A column uses the one its COMPRESSION clause names, or the default_toast_compression setting if it doesn't name one. Changing either only affects values compressed from then on. Values already compressed keep their algorithm, even when you copy them with INSERT ... SELECT or rewrite the table with VACUUM FULL.

Both algorithms work the same basic way. They scan a value from start to finish, looking for bytes that already appeared earlier. When they find a repeat, they replace it with a short note saying where the earlier copy is and how long it is.

In {"status":"active"},{"status":"active"}, the second object is stored as a few-byte note pointing back to the first.

There's a limit, though, to how far back that note can point, called the window. If the earlier copy is farther back than the window, the algorithm can't see it and stores the repeat in full.

pglz is Postgres's own compressor. Its window is 4 KB, and at each position it checks several earlier matches and keeps the longest. Postgres throws a pglz result away unless it saves more than 25%, and pglz gives up early on data where it finds no repeats at all.

LZ4 is a general-purpose compressor used well beyond Postgres. Postgres has supported it since version 14. Its window is 64 KB, but instead of searching, it checks a single earlier candidate from a small lookup table, which makes it much faster. That table keeps overwriting older entries with newer ones, so the farther back a repeat is, the less likely LZ4 is to find it. Postgres keeps an LZ4 result as long as it saves more than 2 bytes.

To test the window, we build a long string out of 30,000 random six-character words. With a space after each one, every word takes 7 bytes, so the string is about 210 KB.

We build it twice, once picking from 500 different words and once from 5,000. With 500, a word shows up again about 500 words later on average, or 3.5 KB back. With 5,000, it's about 35 KB back.

We store each string in both a pglz column and an LZ4 column. setseed makes the random words the same on every run, so the sizes below repeat on Postgres 18:

CREATE TABLE words (vocab int, pglz text COMPRESSION pglz, lz4 text COMPRESSION lz4);

SELECT setseed(0.42);

INSERT INTO words
SELECT vocab, body, body
FROM (
  SELECT vocab, string_agg(substr(md5(floor(random() * vocab)::text), 1, 6), ' ') AS body
  FROM (VALUES (500), (5000)) AS v(vocab), generate_series(1, 30000)
  GROUP BY vocab
) s;

SELECT vocab,
       octet_length(pglz) AS raw_bytes,
       pg_column_size(pglz) AS pglz_bytes,
       pg_column_size(lz4) AS lz4_bytes
FROM words ORDER BY vocab;
 vocab | raw_bytes | pglz_bytes | lz4_bytes
-------+-----------+------------+-----------
   500 |    209999 |     105457 |     96022
  5000 |    209999 |     209999 |    177590
(2 rows)

With 500 words, the average gap is inside pglz's 4 KB window, but the gaps vary, and about 30% of repeats land farther back. LZ4 finds most of those, so its value comes out about 9% smaller than pglz's.

With 5,000 words, most repeats are out of pglz's reach. It can't save more than 25%, so Postgres stores the value uncompressed. LZ4 still reduces it by about 15%, from the repeats that happen to land close.

pglz wins on these JSON documents

In a JSON array, the same keys come back in every object, close enough for both algorithms to find them.

We loaded 10,000 order documents into three tables on PlanetScale Postgres 18: one using EXTERNAL so nothing gets compressed, one using pglz, and one using LZ4. Each document has 25 to 100 line items, and every line item repeats the same five keys: sku, name, qty, price_cents, and tags.

The uncompressed table is 103 MB. The pglz table is 19 MB and the LZ4 table is 24 MB.

Since both find the keys, the size comes down to pglz's extra searching, which finds longer matches. LZ4 makes up for it in speed: in two runs each, it loaded the same documents in 0.48 and 1.09 s, against 1.59 and 1.73 s for pglz.

pglz is faster for previews

pglz also wins when you only read the start of a large value, like a 200-character preview with left(body, 200). pglz never needs more than 9 bits to store a byte of the original, plus 2 bytes overall, so Postgres can put a ceiling on how many compressed bytes a prefix needs. A character takes 1 to 4 bytes, so 200 characters are at most 800 bytes, and 800 bytes of output need at most 902 compressed bytes. That's less than the 1,996 bytes in a chunk, so Postgres fetches only the first one. Postgres has no equivalent ceiling for LZ4, so it fetches the whole value first.

We stored 100 text values of about 1 MB each, which both algorithms compress to about half their size. For the first 200 characters of each, pglz needed 301 page lookups, all from memory, in about 1 ms, while LZ4 needed 6,234 in about 16 ms. The uncompressed EXTERNAL version also needed 301, but takes about twice the disk.

When to skip compression

Compressed values take fewer pages, which means less disk, more of a table in memory, and less WAL to send to replicas and back up. The cost is CPU, on every write that crosses the threshold and every read of a compressed value. Compression also doesn't reduce egress, since Postgres decompresses values before sending them.

Two kinds of data are better off uncompressed, with EXTERNAL:

• Already-compressed data, like the JPEG frames in our video conferencing demo. LZ4 spends CPU on every frame and saves little or nothing.
• Large bytea values read from the middle, like seeking in a file.

With EXTERNAL, every chunk but the last holds exactly 1,996 bytes of the value, so Postgres can work out which chunk any byte is in. The first 250 chunks cover 499,000 bytes, so the 200 bytes starting at 500,000 are all in chunk 250, and Postgres fetches only that one. Compressed, a chunk still holds 1,996 bytes, but there's no telling how much of the value they decompress to. So with LZ4, Postgres fetches every chunk and decompresses from the start. pglz has its ceiling from left(), but for byte 500,000 that ceiling is larger than the whole stored value, so it reads every chunk too.

Doing that read on the same 100 values, stored as bytea, took 0.6 ms uncompressed with EXTERNAL, against 43 ms with LZ4 and 116 ms with pglz.

Wrapup

TOAST only steps in once a row passes 2,032 bytes, so plenty of values, like the 2,000 xs from the start, never get compressed at all. For the ones that do, LZ4 is the right default for most tables: its speed matters more than the size difference, which can go either way. For a few kinds of data, it's worth choosing pglz or EXTERNAL by hand:

Some customers asked for stemming support. Another customer asked for heap attribute tiebreakers in top-k ORDER BY clauses. One competitor optimized their full-text index and identified one read-only use case where their index was briefly faster than TIN. So we got to work. The result is TIN v1.0.4, released last week, and TIN v1.0.6, which is rolling out to all customers over the next week.

Spoiler alert: TIN is back to being the fastest.

Stemming

In full-text search, stemming means replacing each word with its stem form when indexing and querying. The stem form of a word is often a base word, but not always. For example, the stem of database is databas. The idea is to map a group of similar words to a single stem. In a search index, for example, if the text we want to index contains the word running, we would store an entry for the word run in the index. Then, when a user searches for the word runs, we would also search for run. As a result, a query for any of run, runs, and running would match any document containing any of those three.

Because plural words, noun forms, and verb conjugations vary from one language to another, stemming is language specific. TIN uses the rust_stemmers crate, which means that we now support the same 18 languages that crate supports, using stemming algorithms from the Snowball project. Those languages are, in alphabetical order: Arabic, Armenian, Danish, Dutch, English, French, German, Greek, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, and Turkish.

Because stemming changes what content goes into an index, you'll have to rebuild TIN indexes if you want to take advantage of the feature. First, you have to update your TIN to at least v1.0.4. If you do not already have TIN v1.0.4, update your cluster to get it. Then rebuild the index. For example:

ALTER EXTENSION tin UPDATE;
CREATE INDEX CONCURRENTLY new_idx_name ON some_table
  USING tin(some_column) WITH (stemmer = 'en');
DROP INDEX CONCURRENTLY old_idx_name;

Tiebreakers

Normally, a ranked, top-k TIN search looks like this:

SELECT fields
  FROM table
  WHERE column ==> 'query'
  ORDER BY tin.score(ctid) DESC
  LIMIT 10;

For some cases, though, multiple rows have equal scores, and we want a deterministic tiebreaker, like insert time, or just a primary key ID:

SELECT fields
  FROM table
  WHERE column ==> 'query'
  ORDER BY tin.score(ctid) DESC, created_at DESC, id
  LIMIT 10;

In the GA version of TIN, that first query used a block-max algorithm, but the tiebreaker query had to identify and score all documents matching the specified keywords. TIN v1.0.6 uses the same block-max plan for both sorts of queries.

Benchmarks, again

Competition is good. It's our goal for TIN to be the fastest Postgres full-text index for all supported use cases. ParadeDB optimized one use case until their extension was faster: BM25-ranked, top-10 queries of read-only data. TIN remained faster at counting queries and dramatically faster at any workload that involved concurrent writes, because ctid-based document identifiers make both counting and segment merging much more efficient than they would be with ordinal document identifiers. We believe TIN's on-disk index format and opportunities for SIMD optimizations meant we could be the fastest at top-k queries, too. So we got to work optimizing top-k queries. Results are below; TIN is the fastest again.

Corpus and test environment

After our optimizations, we ran another round of benchmarks with the same benchmark methodology as before, which ParadeDB also used for their tests. It's still the same corpus: 85 GB of questions and answers from Stack Exchange, totaling 150 million documents. It's the same search trace: 1,254 synthetic queries, each interpreted as a disjunction query (match any word), a conjunction query (match all words), and a phrase query (match all words in consecutive order). It's the same EC2 instance: an i7i.8xlarge. It's the same parameters as before: Postgres defaults from our fork of the ParadeDB Benchmarker, except for three: max_parallel_workers is 8, shared_buffers is 24 GB, and maintenance_work_mem is 24 GB, to best match the resources of the container. As before, we ran the Benchmarker on the same EC2 instance as the target Postgres server, to ensure that network latency did not impact the measurements. We have increased the resources of the containers on that host in which we run each target Postgres instance; they now have 8 vCPUs and 64 GB of RAM, so that anyone who wants to reproduce the results can use a PlanetScale M-640 i7i database to do it. Finally, we now use CPU pinning in the target container to minimize the effects of CPU migration and contention on measurements.

ParadeDB ran their benchmarks on version 0.26.0-rc.2 of their extension. Our benchmarks here are based on the more recent, final release version: 0.26.0. Following ParadeDB's recommendations, we used the |||, &&&, and ### operators specific to disjunction, conjunction, and phrase queries. Our benchmarks for TIN use v1.0.6, which is rolling out to all customers over the course of the next week.

Benchmark results

We ran the four scenarios from ParadeDB's blog post: conjunction queries, disjunction queries, phrase queries, and a mix of all three types. For all scenarios, the query requests the top ten results by decreasing BM25 score. All four benchmarks have an unchanging table, an index built in one shot, and a clean VACUUM. Aside from the fact that the index is about twice the size of the shared buffers (24 GB), this is basically a best-case scenario for any text-search index.

Each table and graph shows performance for both an Intel CPU (i7i EC2 instances use 5th generation Xeon processors) and an ARM64 CPU (i8g EC2 instances use Graviton4 processors). ARM CPUs are a little bit cheaper, but x86-64 CPUs are significantly faster for text-search workloads, most likely because AVX-512 has wider vector registers and larger SIMD issue width than Neoverse V2. There's enough of a difference from one CPU type to another that we strongly encourage anyone performing benchmarks to disclose the specific instance types (either the EC2 node type, or the CPU model number), for the sake of both consistency and reproducibility.

Throughout the results, "TIN" refers to TIN in its default configuration, which omits certain common terms from BM25 scoring. For "TIN_FULL" runs, we executed the same queries with BM25 scores based on all query terms. More on that trade-off below.

Conjunction queries, top-10 ranked

┌──────────────────────────────────────────────────────────────────┐
│                  QPS    p99    MB/query   QPS    p99    MB/query │
│                │    x86-64 (i7i)        │    ARM64 (i8g)         │
├────────────────┼─────┬───────┬──────────┼─────┬───────┬──────────┤
│ TIN            │ 842 │  57ms │       21 │ 594 │  94ms │       21 │
├────────────────┼─────┼───────┼──────────┼─────┼───────┼──────────┤
│ TIN_FULL       │ 501 │ 117ms │       34 │ 336 │ 195ms │       34 │
├────────────────┼─────┼───────┼──────────┼─────┼───────┼──────────┤
│ ParadeDB       │ 169 │ 253ms │       44 │ 142 │ 278ms │       43 │
└────────────────┴─────┴───────┴──────────┴─────┴───────┴──────────┘

Disjunction queries, top-10 ranked

┌───────────────────────────────────────────────────────────────────┐
│                  QPS     p99    MB/query   QPS    p99    MB/query │
│                │    x86-64 (i7i)         │    ARM64 (i8g)         │
├────────────────┼─────┬────────┬──────────┼─────┬───────┬──────────┤
│ TIN            │ 236 │  132ms │       31 │ 143 │ 243ms │       31 │
├────────────────┼─────┼────────┼──────────┼─────┼───────┼──────────┤
│ TIN_FULL       │  85 │  331ms │      106 │  58 │ 548ms │      106 │
├────────────────┼─────┼────────┼──────────┼─────┼───────┼──────────┤
│ ParadeDB       │  65 │  532ms │      128 │  45 │ 684ms │      126 │
└────────────────┴─────┴────────┴──────────┴─────┴───────┴──────────┘

Phrase queries, top-10 ranked

┌────────────────────────────────────────────────────────────────────┐
│                  QPS     p99    MB/query   QPS     p99    MB/query │
│                │    x86-64 (i7i)         │    ARM64 (i8g)          │
├────────────────┼─────┬────────┬──────────┼─────┬────────┬──────────┤
│ TIN            │ 514 │  120ms │       28 │ 381 │  202ms │       28 │
├────────────────┼─────┼────────┼──────────┼─────┼────────┼──────────┤
│ TIN_FULL       │ 386 │  157ms │       37 │ 284 │  246ms │       37 │
├────────────────┼─────┼────────┼──────────┼─────┼────────┼──────────┤
│ ParadeDB       │ 166 │  327ms │       73 │ 122 │  415ms │       72 │
└────────────────┴─────┴────────┴──────────┴─────┴────────┴──────────┘

Conjunction, disjunction, and phrase queries, top-10 ranked

┌───────────────────────────────────────────────────────────────────┐
│                         QPS     p99    MB/q   QPS    p99     MB/q │
│                       │    x86-64 (i7i)     │    ARM64 (i8g)      │
├───────────────────────┼─────┬────────┬──────┼─────┬────────┬──────┤
│ TIN                   │ 413 │  119ms │   27 │ 265 │  218ms │   27 │
├───────────────────────┼─────┼────────┼──────┼─────┼────────┼──────┤
│ TIN_FULL              │ 180 │  275ms │   59 │ 125 │  444ms │   59 │
├───────────────────────┼─────┼────────┼──────┼─────┼────────┼──────┤
│ TIN v1.0.2            │ 110 │  643ms │   64 │ 135 │  435ms │   63 │
├───────────────────────┼─────┼────────┼──────┼─────┼────────┼──────┤
│ TIN_FULL v1.0.2       │  36 │ 2150ms │  128 │  44 │ 1743ms │  130 │
├───────────────────────┼─────┼────────┼──────┼─────┼────────┼──────┤
│ ParadeDB              │  98 │  442ms │   81 │  79 │  548ms │   80 │
└───────────────────────┴─────┴────────┴──────┴─────┴────────┴──────┘

Scoring common terms

The discussion in ParadeDB's benchmark post takes issue with an important TIN optimization: omitting terms that appear in more than 10% of documents from BM25 scores. We believe that's a good trade-off, which is why we made it the default. But if you need BM25 scores to include all the words in your query, that's tunable at query time: order by tin.full_score(ctid). If you want to omit scores for some terms, but at a different threshold than the 10% default, make your queries order by tin.score(ctid, dense_ratio => F), where F is the fraction (between 0 and 1) of documents below which a term will be included in scoring.

Terms that appear in more than 10% of documents do not affect BM25 scores very much, since each term's score increases with the inverse document frequency, or rarity, of the term. But it takes a lot of CPU time to include those common terms in the scores. TIN still ensures that conjunctions, disjunctions, and phrases match as usual; the scoring optimization only affects the scores.

Does TIN's version of scoring still qualify as BM25? That's a bit of a semantic question. Most text indexes omit a fixed list of common words, called stop words, from the index, which removes the ability to search for those terms or use them for scoring. TIN doesn't. Instead, it includes all words in the index and in search logic, and it omits a configurable, corpus-specific set from scoring. The words TIN scores use the standard BM25 formula.

For completeness, we included both scoring approaches in today's benchmarks. TIN is faster than TIN_FULL, as expected; both are once again faster than ParadeDB.

Free lunch, or lack thereof

One of the optimizations in ParadeDB 0.26.0 increases the index size from 52.1 GB to 67.3 GB for the 85 GB Stack Exchange corpus, an increase of 29%. Benefiting from that optimization requires customers to rebuild all their text indexes. TIN v1.0.6 is a drop-in replacement; the index format did not change or grow, and customers do not need to take any action.

Summary

TIN is once again the fastest Postgres full-text index. When used with its default scoring methodology, TIN has 3.1-5.0x higher throughput and 2.7-4.4x lower p99 latency than ParadeDB on x86-64; on ARM64, it has 3.1-4.2x higher throughput and 2.1-3.0x lower p99 latency. For queries that use tin.full_score, TIN has 1.3-3.0x higher throughput and 1.6-2.2x lower p99 latency on x86-64, and 1.3-2.4x higher throughput and 1.2-1.7x lower p99 latency on ARM64.

When creating a new database, consider an x86-64 instance with AVX-512 support if text search is a large part of your workload.

Get started with TIN here.

Sometimes, the answer is obvious. Is the user logged in? Does their role have the required permissions?

Sometimes it's less obvious. Are there sufficient resources available? Is this work a higher priority than everything else?

There are many names for the mechanisms that block or allow actions in software. Most commonly they are referred to as admission control. Perhaps you'll be familiar with a leaky bucket implementation.

A brief history of leaky buckets

Engineering applications for leaky buckets predate computers. From around 1600 BC the Babylonians, Egyptians, and Persians all used vessels with a small hole in the bottom as primitive clocks, measuring elapsed time by the volume of the container as it drained at a constant rate.

By around 200 BC the Greeks had advanced leaky buckets into accurate clocks. A floating regulator kept water flowing steadily from one bucket into a second, where a rising float moved a pointer to show the hour. A siphon emptied the second bucket once it was full. Most notably, Ctesibius added gears to regularly advance each "tick" of the clock, turning the display which indicated the current season.

A mechanical movement driven by a leaky bucket was born, and it became the gold standard of timekeeping for centuries.

Water-based clocks were used as a form of admission control. In Athenian courts, the length of time a speaker had to plead their case was measured by the time water took to drain from a vessel. The amount of water poured into the vessel was determined by the severity of the case.

Finally, in the computer age, Jonathan Turner is said to have used the term "leaky bucket" first in 1986, writing "New directions in communications" for IEEE Comms Magazine. The piece describes how a system can police the pace at which packets are sent over broadband connections. (If you haven't read it, spoiler: it's not much slower than your connection at home today.)

Leaky buckets in software

There are two kinds of leaky buckets in software, to implement a meter or a queue.

In both, each unit of "work" is assigned a value which is checked against the current volume of a "bucket." If the bucket lacks capacity for the size of new work, that work is rejected. Otherwise the work is allowed and its value adds to the bucket's current volume. The bucket drains (or leaks, if you will), so over time, new work can't be admitted faster than the bucket drains. The drain rate is based on time elapsed, not on the completion of work.

When used as a meter, the bucket defines if and how much work is permitted to immediately proceed. As a queue, the bucket defines the volume of work that can wait to proceed, but work only begins as it drains from the bucket.

In this post we'll focus on leaky buckets as a meter.

Typically, leaky buckets are implemented in software to keep systems healthy. A sort of resource management to ensure the system is not overwhelmed.

It's not hard to imagine this being useful in a database. Imagine being able to block or allow queries, before they execute, based on available resources or concurrency limits. One can dream!

The leaky bucket can be tweaked by modifying its volume and the rate at which it drains.

At its core a leaky bucket prevents work, but it also forces us to reason about the total volume of work that results in a block, how quickly more capacity becomes available, and how to handle the expected failure state when work is rejected.

Leaky buckets in API design

Shopify's GraphQL Admin API implements rate limiting with a documented leaky bucket. Each app on each store gets a bucket with a fixed size and restore rate.

Each request costs a different amount based on what the query asks for, not how many requests are made. Before a query runs, the bucket must have room for its estimated cost. Once it completes, the bucket is refunded or charged any difference between the estimated and actual cost. Apps can burst faster than the restore rate until the bucket has no room left, at which point requests are throttled.

Responses include the current state of the bucket, so apps can slow down before they're throttled and treat throttling as an expected outcome, not a surprise.

Classification as admission control

Now let's consider the AI age. Jev, a "System One" model (perhaps better named a "Decision Model"), recently launched, making classification through inference cheap and fast. It could classify "should this work proceed or not" based on a predefined set of options and return a response in hundreds of milliseconds.

Whether our decision gate is mechanical (leaky bucket) or requires inference (Jev), the requirement is the same: answering "should this proceed?"

Low cost throttling

One of the great benefits of leaky buckets in software is how cheap they are. What we've described as a slowly draining bucket isn't a long-running process that is using CPU to "drain".

Simply put, a bucket is a stateful object that holds two values:

1. how full it was the last time anyone looked
2. when that was

From those values, you can work out how full the bucket is now:

level_now = max(0, level_then - drain_rate × (now - then))

Any work rejected or admitted by the bucket updates its current state. In this way, buckets are lazily drained (as in, only when checked) and empty buckets hold no values.

Admission controls and databases

Just like hardware, operating systems, applications, and ancient Greeks, databases benefit from leaky buckets.

However, open-source databases like Postgres and MySQL don't have this functionality baked in.

One of the most common failure modes for databases in production is the inability to deliberately reject work they cannot complete. Blast a billion queries at your typical relational database, and it'll try its best to execute them. Success isn't likely. Some hardware, configuration, or connection limitation will unwittingly play the role of an admission control system and start blocking work in some unhandled, undesirable way.

Some proprietary databases have a form of resource management so that workloads can be throttled (but not rejected). For example, in 1999 the Oracle Database Resource Manager added mapping rules to sessions for users, services, clients, and more, which could then be used to apply limits on things like resource use and concurrency.

In 2008, SQL Server added the SQL Server Resource Governor, which added classifier functions to sessions that can then apply rules to queries in that session.

Postgres has some configuration options to control expensive queries that have already consumed resources, such as statement_timeout, but nothing to prevent expensive work before it happens. And so in 2026, PlanetScale brought decision gates to Postgres with Database Traffic Control®.

Database Traffic Control

The health of your Postgres database is laid bare at the mercy of your applications. An unexpected connection storm or failure of the query planner can ask more of your database than it can reasonably be expected to handle.

To protect your Postgres database from your application (and at times, from itself), we built Traffic Control, admission control for Postgres. In it, resource budgets determine how many resources or how much concurrency one or many queries can consume.

Each budget admits or blocks queries that match a specified set of tags. Those tags can be specified as comments in the query, or they can refer to the user or application that sent it. If the resource budget determines the query should not proceed, it will log a warning or block it, depending on your resource budget configuration.

Creating a resource budget opts you into an expected failure state, so it forces you to reason about the behavior your application should exhibit when queries are blocked. For this reason, we recommend creating a new budget in warning mode first, monitoring how much traffic is affected, preparing your application for blocked queries, and then updating the mode.

Admission controls but for traffic

You configure three major settings on a resource budget.

Server share and burst limit, the token* bucket. Server share determines how fast the bucket replenishes with tokens, expressed as a percentage of server resources. The burst limit sets the maximum capacity of the bucket, expressed in whole-server seconds.

Because a database doesn't know the cost of a query before execution, Traffic Control estimates its cost based on the query plan and historical behavior of queries with that shape. If the budget has sufficient capacity the query proceeds. After execution, the cost of the query is calculated and any difference is either refunded or charged to the token bucket.

Why is the refill rate measured in server capacity instead of seconds? Because not all time spent executing a query is equal. A query that took 8 cores a single second is different from a query that took one core a single second. Consider different server sizes as well: a 25% server share means a quarter of your server's capacity drains back every second, whether that server has 2 cores or 64.

Per-query limit sets an upper limit on the maximum capacity a single query can consume. This makes an admission-control decision directly on a single query instead of measuring it against available capacity in a bucket.

Maximum concurrent workers similarly controls the resources a single query can consume, but measures it by available workers instead of execution time.

A query may be targeted by multiple resource budgets and must satisfy all of them to proceed.

For specific scenarios where resource budgets help, along with how to tag queries in your codebase, see Patterns for Postgres Traffic Control.

Summary

Admission control forces you to decide what should happen when work can't proceed, and leaky buckets have been a cheap way to make that decision for a very long time. Traffic Control makes that decision for Postgres before an expensive query runs, not after.

Get started with Database Traffic Control

Why dedicated read replicas?

Dedicated read replicas are additional Postgres instances that replicate asynchronously from the primary cluster.

A common use case for this is reducing read latency for a global set of clients. Your primary database cluster exists in one region (say, us-east-1), while dedicated read replicas are stood up in European data centers (eu-west-1 and eu-central). Read traffic for clients and customers in Europe can be directed to one of these nodes, speeding up the end-user experience.

In your PlanetScale dashboard, this looks like:

You may not require a globally distributed database, but dedicated read replicas are useful even within a single cloud region.

You can create one or more dedicated read replicas within the same region and create connection credentials targeting them individually, allowing you to control precisely which work each is doing. For example, you can have your primary set up to handle all writes and general OLTP traffic, a dedicated read replica for vector search with pgvector, and an additional replica to handle analytics and longer-running queries that you don't want impacting production.

Creation and management

You can create and manage dedicated read replicas from the "Clusters" page in the PlanetScale dashboard, the API, or the CLI.

Each replica may have different requirements for RAM, latency, and performance. You can configure the instance size and disk characteristics of every node, letting you tune them for each use case.

For example, above we can see the eu-read-node instance is running on an M-640 with 8 vCPUs, 64GB of RAM, and a fast local NVMe drive. The dublin-double is configured as two PS-320 instances, with storage options like disk size, IOPS, and throughput being configurable separately from the primary configuration.

Get started

Learn more about using dedicated read replicas and their limitations in the docs and give them a try on your database today.

Neki adds a layer between a PostgreSQL client and PostgreSQL. We need that layer to route queries to shards and to make many PostgreSQL servers look like one. The downside is that every query now has another network hop and has to pass through the router.

We can't make that hop disappear, so a lot of the performance work in Neki comes down to what we can avoid doing while the query passes through it. If the router doesn't need to understand a result, why decode it? If it only needs one column, why decode the other ten columns in the same row?

The key part here is that we can design this to be lazy and composable. Walking the PostgreSQL messages in a response does not mean we also need to construct rows. Finding the rows does not mean we need to decode their columns either. Even if the router needs one value, the rest of the row can stay as protocol bytes.

Why not decode everything up front?

The usual way to build a layer like this is to parse whatever comes over the wire into your own types as soon as it arrives. Every row becomes a list of values, and the rest of the code only ever deals with those. It's a sensible default because it gives the rest of the system one consistent thing to work with.

Vitess, the MySQL sharding system we maintain, works this way, so it's a useful comparison. Vitess sends query results between its components using protobuf. Leaving out the fields that aren't important here, QueryResult and Row look like this:

message QueryResult {
  repeated Field fields = 1;
  uint64 rows_affected = 2;
  uint64 insert_id = 3;
  repeated Row rows = 4;
  string info = 6;
  string session_state_changes = 7;
  bool insert_id_changed = 8;
}

message Row {
  // -1 means NULL.
  repeated sint64 lengths = 1;

  // All non-NULL values concatenated together.
  bytes values = 2;
}

This is a pretty compact protobuf representation. lengths tells us where the values are in the shared values byte string, so the protobuf doesn't need a separate allocation for every value. The query engine can't use the protobuf rows directly though. At the RPC boundary, proto3ToRows converts all of them:

func proto3ToRows(fields []*querypb.Field, rows []*querypb.Row) [][]Value {
    result := make([][]Value, len(rows))
    for i, r := range rows {
        result[i] = MakeRowTrusted(fields, r)
    }
    return result
}

And MakeRowTrusted walks every column in every row:

func MakeRowTrusted(fields []*querypb.Field, row *querypb.Row) []Value {
    sqlRow := make([]Value, len(row.Lengths))
    var offset int64
    for i, length := range row.Lengths {
        if length < 0 {
            continue
        }
        sqlRow[i] = MakeTrusted(
            fields[i].Type,
            row.Values[offset:offset+length],
        )
        offset += length
    }
    return sqlRow
}

MakeTrusted doesn't turn every integer into a Go integer. A Value can still point at the bytes in the protobuf row. What we do get is a slice for every row and a Value for every non-NULL column before the query engine can do anything with the result. Going back across an RPC boundary does the reverse. RowToProto3Inplace first walks all the values to build the lengths and then walks them again to concatenate their bytes.

The protobuf result sits between separate Vitess components, while the query engine works with [][]Value. Converting at the boundary gives the engine one consistent type to work with. The downside is that every result gets converted, even when the router never looks at a value.

We wanted Neki to perform better, and therefore do less work in such cases.

Keeping the PostgreSQL response

Neki also uses protobuf between the router and the sidecar that sits in front of each PostgreSQL instance, but the result protobuf looks very different. Our ExecuteResponse is mostly a byte string:

message ExecuteResponse {
  bytes raw = 1;
  int64 conn_id = 2;
  ...
}

raw contains PostgreSQL response messages. A DataRow in there is already a valid PostgreSQL DataRow that we can send to the client. Of course we still have to decode the ExecuteResponse protobuf to get the raw byte slice. That still leaves us with one byte slice. What we avoid is a protobuf object for every row, another slice of lengths for that row and a value for every column.

That doesn't mean the router ignores the response. NewQueryResponse walks the five-byte message headers, validates the lengths, handles CommandComplete and finds the region containing the DataRow and NoticeResponse messages. This is stored as one byte slice in DataRows. We only open it further if the query needs us to.

For the examples here, we'll use an orders table sharded by account_id:

CREATE TABLE orders (
    id uuid NOT NULL,
    account_id bigint NOT NULL,
    status text NOT NULL,
    created_at timestamptz NOT NULL,
    total numeric NOT NULL,
    PRIMARY KEY (account_id, id)
);

A result we don't need to decode

The simplest example is a point lookup:

SELECT id, status, total
FROM orders
WHERE account_id = $1
  AND id = $2;

Because account_id is the sharding key, Neki needs the value in $1 to pick the shard. It doesn't need to understand id, status or total when they come back. Those values only need to make it from PostgreSQL to the client. We can use the same approach on the way into the router too.

Bind parameters are rows going the other way

With PostgreSQL's extended query protocol, $1 and $2 arrive in a Bind message. Each parameter is length-prefixed as a column in a DataRow:

NULL:     Int32(-1)
non-NULL: Int32(length), followed by value bytes

The difference is where the format comes from. Bind includes format codes for its parameters, while the formats for DataRow columns come from RowDescription. The PostgreSQL protocol message documentation has all the details if you want to know all the fun little bits about the protocol.

Neki keeps the original Bind message and a cache for the parameters it has decoded. For our point lookup, the router scans the parameter frames, decodes the bigint in $1 and caches it. We don't need $2 for routing. When the query is sent to the shard, Neki writes the original parameter frames directly as long as the client encoding hasn't changed.

Keeping the original frames also helps when we rewrite a query. A rewritten query might use only some parameters, put them in a different order or use the same one multiple times. A mapping like [1, 0, 1] copies the parameter frames into a new Bind. We can use the same approach when adding parameters to the existing list.

Nested loop joins have a similar case. A column from a row on the left becomes a parameter for a query on the right. Neki copies the complete length-prefixed column frame from the DataRow into the new Bind. We only turn the join key into a typed value if we also need it for routing or evaluation.

Text stays text and binary stays binary

There is one more wrinkle: PostgreSQL values can be sent as text or binary. The client uses Bind to tell PostgreSQL which formats it used for the parameters and wants for the result columns. Instead of up-front conversions like Vitess would require, Neki keeps values in the original formats.

When a query selects id and account_id but orders by created_at without selecting it, id and account_id use the formats requested by the client. The extra created_at column uses binary because only Neki needs it. The client will never see those column values.

If Neki needs to decode a value, RowDescription tells it whether to use the text or binary decoder. If it doesn't, the bytes stay untouched. When Neki produces a new value, it uses the format requested by the client.

Conversion is still needed if client_encoding changes between Bind and Execute. Affected text parameters are recoded before they are sent to PostgreSQL. The other parameters stay as they are.

Now let's go back to the result from the point lookup. DataRows can hold one raw byte slice, a slice of rows that point into those bytes or a slice of pointers used to select and reorder those rows. It only creates the next representation when a query needs it.

In this case, Neki writes the raw bytes straight to the client.

No DataRow structs and no columns! Just the PostgreSQL messages that we already had in the correct Postgres wire format. This also works for cross-shard queries. In the query below, status isn't the sharding key so it will have to get passed along to every shard to collect results:

SELECT id, account_id, status
FROM orders
WHERE status = 'pending';

There is no global order and no sums / averages to calculate across the results. This means we can optimize. After sending the scatter queries, Neki streams response chunks from each shard, combines the separate CommandComplete messages with Collapse, and DataRows can be passed along unmodified. It's a "complex" distributed query, but there is nothing in the rows we need to decode.

Adding LIMIT

Now let's add a limit:

SELECT id, account_id, status
FROM orders
WHERE status = 'pending'
LIMIT 100;

Now we need to know when we've sent 100 rows, which means the response cannot be an opaque byte slice. Limit calls DataRows.Len(), and on the first call we walk the raw response and count the DataRow messages so the row slice can be allocated once. In a second pass, we create the DataRow wrappers which all point into the original response buffer.

Once we have the row boundaries, Limit builds a pointer slice for the requested window:

ptrs := make([]*runtime.DataRow, n)
for i := range n {
    ptrs[i] = response.Rows.Row(int(startIdx) + int(i))
}

response.Rows = runtime.NewDataRowsFromPtrsWithNotices(
    ptrs,
    notices,
)

ptrs is the third representation in DataRows. It lets us take a subset of the rows or put them in a different order without moving row messages.

For this query Neki knows where every row starts and ends. What it doesn't know is the value of a single id, account_id or status column. Counting rows doesn't require that... but what if it did?

Sorting means decoding the sort key

For a pending orders page, we'd probably want a stable sorting:

SELECT id, account_id, created_at
FROM orders
WHERE status = 'pending'
ORDER BY created_at DESC
LIMIT 100;

Each shard sorts its subset of results (within Postgres), but the router has to merge those distinct result sets into a global order. Now we finally have to decode a column. First we need the rows, the same as for LIMIT, and then we need to find created_at inside each row.

At the protocol level, a DataRow keeps the complete PostgreSQL message and a view of the bytes containing its columns. The column bytes start after the five-byte message header and the two-byte column count. Each column has a four-byte length followed by the value bytes. A length of -1 means NULL.

To get to the third column, we have to read the lengths of the first two columns and skip their payloads. Reading the lengths of these columns doesn't decode the values. We now know where id and account_id are, but the bytes still have no meaning to the router. With the offsets, we can jump to the correct bytes to grab created_at for each row.

These timestamps are cached because a row will normally be compared more than once while merging results. At this point all rows have been exposed and we've looked at their column framing. The actual values look like this though:

id          frame located, payload not decoded
account_id  frame located, payload not decoded
created_at  frame located, payload decoded and cached

The merge produces another DataRows with pointers to the original rows, now in the right order. Sorting rows doesn't require copying the rows.

Dropping a column without decoding

A small change can have a big impact on what work we can and cannot skip. How about we don't select created_at:

SELECT id, account_id
FROM orders
WHERE status = 'pending'
ORDER BY created_at DESC
LIMIT 100;

Ordering by a column without returning it is a very normal query, but it does give the router one extra problem. The shards have to return created_at because Neki needs it for the merge. Before we send the rows to the client, that extra column has to be removed.

Recall from the last section that by the end we reached the point where created_at had been decoded, but id and account_id were still only protocol frames. For this one, there's more to do.

The naive implementation would read id and account_id as values, build a new row and serialize them again. That's a lot of work to remove a few bytes from the end of a message.

Instead, SimpleProjection stores the projection as runs of adjacent input columns. Each run contains the first and last column, so {0,2} is one run covering columns 0, 1 and 2. This lets ProjectInto copy all three column frames as one byte range. Skipping, reordering or duplicating columns creates more runs, stored in output order:

stored runs          byte ranges copied
[{0,2}]              columns 0–2 in one copy
[{0,2}, {4,6}]       columns 0–2, then 4–6 in two copies
[{3,3}, {0,2}]       column 3, then 0–2 in two copies
[{0,0}, {0,0}]       column 0 twice in two copies

Our query needs only one run, {0,1}. At execution time we find the byte range containing both length-prefixed columns, write a new DataRow header with a column count of two and append both columns at once.

The byte-backed projection does one append per run. Values is the row's view of the column bytes, and offsets contains the frame boundaries found while scanning it:

for _, r := range runs {
    dst = append(
        dst,
        dr.Values[offsets[r.Start]:offsets[r.End+1]]...,
    )
}

This isn't zero-copy because we are creating a different PostgreSQL message. The new header and the columns have to go somewhere. We do copy the exact id and account_id frames though, still saving a good amount of work through careful optimization.

Normally we're projecting a complete response batch, not one random row. Neki appends all the new rows to one shared wire buffer. The finished rows are then bounded views into that buffer, so there isn't a separate backing buffer for every output row.

The slightly funny part is that we decoded created_at, which the client didn't ask for, and didn't decode id or account_id, which it did ask for. That's exactly the work this query required. In Neki, we take every opportunity to skip unnecessary work.

When we really need values

Of course some queries do require the router to understand and produce values. A cross-shard aggregate is a good example:

SELECT status, count(*)
FROM orders
GROUP BY status;

The same status can exist on many shards, so we can't rely on PostgreSQL to do the full GROUP BY. The Neki router must handle it.

We need the group key and the partial counts as values. There isn't a useful byte-level trick for grouping equal values and adding the counts. The final count is also a new value produced by the router, so there are no original PostgreSQL bytes to copy.

For a row like this, we cannot get away with just passing along bytes from Postgres to clients.

When a results row from this query is sent to the client, Neki serializes those values into a new PostgreSQL DataRow in the format the client requested. This query MUST go through every layer:

response messages
    -> result batches
        -> rows
            -> column frames
                -> typed values
                    -> newly computed values

Bytes and values at the same time

Neki doesn't have one internal row representation. A row received from PostgreSQL starts wire-backed, while a row produced by an aggregate can be completely value-backed. The sorted rows from a few sections ago are a mix of the two: they keep the original message and have a cached created_at value.

Original: complete PostgreSQL DataRow

values[0]: no decoded value
values[1]: no decoded value
values[2]: decoded created_at

The original message is still available for forwarding or projection, while the cache avoids decoding created_at again every time the row is compared.

Cached values survive projection too. If a projection keeps or moves a decoded column, Neki copies the original frame and remaps the cached value to its new position. We took the performance hit in decoding the timestamp because the query needed it, but we want to make sure we don't decode or serialize it again.

For PostgreSQL records this can also be necessary for correctness. A record can have a transient typmod describing it, which isn't always recoverable from the wire bytes. Raw projection won't allocate a cache in the case where none of the projected columns have decoded values.

Using this trick elsewhere

This way of working with protocol frames isn't limited to sorting and projection. For joins, we decode the keys needed to match rows. The output can still be assembled from contiguous runs of columns from the left and right rows. We avoid doing the extra work of producing values whenever we can!

When we need to spill rows to disk, we also keep things in the self-framing PostgreSQL DataRow format. We can write and read from disk with minimal additional decoding overhead.

Keeping bind parameters in their original length-prefixed representation is not just about performance either. Parsing and serializing a value isn't guaranteed to give us identical bytes. A floating point NaN payload, for example, can survive PostgreSQL's receive function but change when it is sent again.

PostgreSQL extension types are another useful case. The router might have no idea how to evaluate a value from an extension, but it can still pass the protocol bytes through.

Keeping the layers composable

The nice part of this model is that each representation builds on the previous one. DataRows can start with raw response bytes, expose those bytes as a row slice and then add a pointer slice for a subset or a different order. An individual row can keep its original PostgreSQL message while also caching the values that have been decoded. Going one layer deeper for one part of a query doesn't force all columns / values to do so.

Of course that doesn't make the complexity disappear. Byte-level projection still has to calculate the right offsets, write a new message header and copy the right column frames in the right order. The key thing for Neki's performance though, is that it's all contained in the row projection and batch builder.

It also gives us a small surface we can test directly. The projection tests construct PostgreSQL messages byte by byte and cover reordering, NULL, empty strings, empty bytea, duplicate columns and the protocol-backed path.

There are a few invariants we have to maintain for that containment to work. Shared response buffers have to remain alive as long as a row still points into them. We use three-index slices because an append through one row must not overwrite the next PostgreSQL message in the same buffer. Two consumers replaying the same buffered row need their own row wrapper and value cache, while the immutable protocol bytes can still be shared.

Memory accounting also has to know that a row can grow. It starts out retaining the wire bytes, but every string, numeric or record added to the cache uses more memory. And then it all has to work with NULL, text and binary formats, client encodings, arrays, domains, records and user-defined types. If moving the bytes around changes the result, we've built a very fast bug 😉.

Nothing is still the fastest

Neki does a lot of real work. It routes queries and sometimes has to merge, compare, group or compute values across shards, but it doesn't have to do all of that for every column in every query. A point lookup can keep the result as PostgreSQL messages. LIMIT needs row boundaries. Sorting needs the sort keys. Projection can move existing column frames around. An aggregate generates newly computed values because there is no earlier layer where it can stop.

For every query the router tries to do the least possible work. The best way to make software fast is to avoid work at all costs.

Why can't Lakebase provide an exact count? For that matter, why should it take over 3.6 seconds to rank what Lakebase claims is just 3,400 documents?

I knew TIN could do a better job than that, so I forked Rishi's app and started building. We've already shown that TIN is really fast, but sometimes being fast gives you the space to build more interesting features, too. Let me show you what I built.

Stop debouncing

First thing to do is stop debouncing keystrokes. Rishi's original app, on the left, waits 250ms after each keystroke before it even begins the search. The TIN version, on the right, searches immediately after every keystroke. Postgres with TIN can comfortably handle immediately sending off the query at every change, since it is so much faster.

Autocomplete

TIN supports wildcard searches. So in my app, I made any incomplete final word a wildcard: typing planetscale datab in the textbox actually searches for planetscale datab*, which naturally matches planetscale database. (Also: datablindnes, Databall, and DATAbEEF, but not in a conjunction query with planetscale.)

Autocomplete is nice if you don't know how to spell a word or just want to avoid typing the whole thing.

Fuzzy matching

TIN supports fuzzy matching, specifically Levenshtein edit distance. In my version of the app, if the search as you've typed it returns too few results overall, the app performs a COUNT(*) search for each term individually, then allows an edit distance of two for the least popular term. If that still doesn't find enough results, it repeats the process until all terms are fuzzed, if necessary. So here, I've typed planetscale databse, and because databse isn't a real word (it's found ten times across the whole history of Hacker News), we end up searching for planetscale databse~2. The results show you which word(s) got fuzzed.

Stop words

Most index implementations, including Lakebase, encourage developers to configure their index to ignore the hundred-or-so most common English words, called stop words. That's a trade-off: smaller, faster indexes, but no way to search for common words. TIN is fast enough that it can afford to just index everything. Good luck searching for "to be or not to be" or "The Who" if your index contains none of those words.

Blink and you might miss it: as I was typing, TIN searched for to b, counted 1,054,718 results, and ranked the top 30 of them in 11ms.

OK, about that tilde

TIN is especially well optimized for COUNT(*) queries. For even very large numbers of results, even on complicated multi-term queries, TIN can provide an exact count in just a few milliseconds. Here's a search for show hn, the example from Rishi's original demo. Lakebase takes 3.6 seconds and estimates there are 3,400 results. Actual count: exactly 213,447.

How much would you expect to pay?

Rishi's original app provisions a Neon instance with 32 CUs, which requires the Scale plan. It also configures the instance to never sleep, so the cost continues all month long: 32 × 730 × $0.222 ≈ $5,186.

The TIN version uses an HA cluster of three M-160 instances, specifically M-160s instances with x86-64 CPUs and 118 GB of NVMe storage each.

TIN is faster and can implement many more useful features, for less than 1/8 the price.

If you like better, cheaper, faster, we have the technology. Get started with TIN search for Postgres.

Most database indexes are b-trees or b+trees, which sort a column's values in a total order and can quickly find matching values by exact value, a range of values, or a prefix. So, for example, a b-tree can quickly find all users with the name Rick, all products with a price below $5, or all repositories with a creation date in November of a given year. But a b-tree is useless for matching content in the middle of a string. That's where full-text search indexes come in.

Do not do this, which checks every name in your users table:

SELECT * FROM users
  WHERE name LIKE '% Royal';

And especially do not do this, which checks every description in your vendors table three times:

SELECT * FROM vendors
  WHERE description LIKE '%postgres%'
  AND description LIKE '%reliable%'
  AND description LIKE '%fast%';

The right tool for that job is an inverted index, which is how essentially all full-text search engines locate documents quickly by the words they contain.

Inverted index overview

An inverted index is a data structure, usually on disk, that maps terms to locations. It works like the index at the back of a book. It even works a lot like a b-tree, except that instead of being keyed by a text column's entire value, an inverted index is keyed by each individual word within a text field.

Quick aside: then why is it "inverted?" It's inverted relative to the text itself, not relative to other indexes. The text is a series of implicit locations, each with a word. An inverted index is a list of words, each with one or more locations where it can be found.

Back to the structure of the thing. At a minimum, an inverted index has a term dictionary and postings lists. Depending on its feature set, it may also have positional data and frequency data.

Try some searches here, and see how your queries compute either the union or the intersection of the document IDs in postings lists in the index.

Term dictionary

The term dictionary maps all the terms found across all your documents to postings lists. On disk, this is often a b-tree or some other lexicographically sorted structure. When a search query arrives, it looks up all the query's terms in the term dictionary.

Queries with wildcards and fuzzy matches may scan part or all of the term dictionary looking for appropriate exact terms. For example, g* would look up both gonna and give, while u~2 would look up all terms within two character edits of u: the terms you and up. Try it in the figure above!

Postings lists

Each term in the term dictionary points to a list of locations called a postings list. These locations are document identifiers of some kind, enough for the database or search engine to find the document in a table or file storage. In most inverted indexes, these are sequential numbers; an index with ten documents uses the numbers zero through nine (or one through ten). However, any unambiguous identifier will do. Since a database needs to map values back to a row rather than to the nth document added to an inverted index, the inverted index either needs to store an additional map of document IDs to rows, or it needs to store the row identifiers directly in the postings list.

Postings lists are almost always sorted, then compressed in some way. If all the document IDs are under 256, they would be stored with at most eight bits each. If they are dense (i.e., many documents contain a given term), then the postings list may store only the differences between the numbers: id1, id2-id1, id3-id2, and so forth. These differences are always smaller than the IDs themselves, so they can be stored with fewer bits. Consider, for example, an index with 300,000 documents, of which 100,000 contain the word "who." Storing each ID literally would take lg(n) = 19 bits per posting. But the average gap between any two successive IDs (remember, the postings list is sorted) is just three, which can be stored in two bits.

If the postings are very dense, the postings list may store a bitmap. If document n contains a word, then the nth bit in the bitmap is one; otherwise, it's zero. Such an encoding takes exactly n bits for n documents and is optimal once around half of all documents contain a given term.

Taking the union or intersection of two postings lists that are sorted is fast, because it can be done in a single, O(n) pass. Taking the union or intersection of two postings lists stored as bitmaps is extremely fast, because recent CPUs with vector instructions can OR or AND 128, 256, or even 512 bits in a single instruction.

Positional data

Some search engines support span queries and phrase queries. A span query is a query that requires terms to be in a specific part of the document or requires them to be within a specific maximum distance of each other. For example, rules IN FIRST 5% or gotta NEAR/3 understand. A phrase query is a special case of a span query that requires words to appear in exact sequence.

To support span and phrase queries, an inverted index can store positional data. The postings list identifies which documents each term appears in; positional data identifies where in the documents the term appears.

Because most queries aren't span or positional queries, positional data is usually stored separately from the postings lists so it can be loaded only when needed.

Frequency data

Some search engines support scoring and ranking documents. One common scoring method is BM25 (more on that below), which needs to know some statistics about the indexed documents: the length of each document, the length of the average document, how often each term t appears in each document d, and how many total documents contain the term t. The index precomputes all of this data so it can be fetched quickly when scoring results for each query.

Index size

The set of all indexed documents is called the corpus. The size of the index is usually some factor of the size of the corpus, with that factor depending on what features the index supports. An index with no positional or score data might be roughly 20% of the size of the corpus. A code-search index with overlapping tokens (to support exact-match and regular-expression search) and positional data could be as much as 300% of the size of the corpus. English-language text indexes with positional data and frequency statistics often are 30-50% the size of the corpus. Those numbers can vary depending on factors like document length and vocabulary size, but they give you an idea of the relative cost of implementing different features.

Tokenizers

Each indexed document is a long string of text. Breaking that text into terms is called tokenizing it. Different use cases, especially different languages, require different tokenizers. Is you're one token or two? Are there any tokens at all in {[] => []}?

Unicode defines a good default set of rules to identify word boundaries.

After tokenization, a search engine may modify or omit terms before adding them to the inverted index. Eliminating linguistic suffixes is called stemming. For example, mapping strangers to stranger or mapping thinking to think allows a user to search for a word but match documents containing any form of that word.

Words so common they're not (usually) useful for searches are called stop words, and many search engines do not index them at all. However, an index that considers new a stop word can't distinguish between documents containing York or New York. An index that drops both the and who can't search for The Who at all. Stop words are a trade-off: the index is smaller and faster, but it's less precise in cases where those words matter.

Scoring: BM25 and top-k

Some search use cases require retrieving a few "best" results, rather than all matching results. This is in contrast to SQL, where SELECT <something> LIMIT 10 is allowed to return any ten rows available. One of many ways to determine the best results is to score them with BM25. BM25 is a formula that scores how good a given document is as an answer for all the terms in a given query. It multiplies a function capturing term frequency (how many times a term appears in the given document, relative to that document's length) with a function capturing the inverse document frequency (what fraction of documents contain the term at least once), then sums up that subscore across all the terms in the query. BM25 captures the intuition that rarer terms are more significant, and a document that mentions a given term lots of times is a better result for that term.

Scoring is almost always used in conjunction with a desired number of results, like SQL's LIMIT 10. That allows an important optimization.
Postings lists can be broken up into blocks, with frequency statistics for each block. When an index looks for the k documents with the highest score, known as a top-k query, it may be able to skip whole blocks of postings if the statistics for those blocks indicate that none of the documents in the block would produce a higher score than the best k documents the index has already found. This significantly speeds up top-k queries, especially for small values of k.

Segments, merging, and mutability

The simplest way to build an inverted index is all at once, in one shot. A small enough index can be built entirely in memory; after all, an inverted index is little more than a Map<String, Vector<ID>>. Indexes larger than memory are written piece by piece. The index creation process runs until memory is full, then dumps a self-contained inverted index to disk representing the first n documents. Then it repeats until memory is full again, dumps that self-contained inverted index to disk for the next n documents, and so on. Each self-contained inverted index is called a segment. When processing a query, the overall index must check for matches in all segments and then merge the results.

Of course, many data sets change over time. A corpus can grow as more documents are added. New documents can be batched in memory until there are enough to form a segment, or each can be added immediately to a mutable data structure on disk. That mutable storage has a less efficient layout than an immutable segment, but it has the advantage of being, well, mutable: documents can be added efficiently without knowing all of them in advance. Then the documents in mutable storage are searched alongside all the immutable segments in each query.

TIN uses a mutable segment containing postings lists, like a less efficient version of the immutable segments. Because the mutable segment is slow, it must eventually be sealed and converted to an immutable segment.

Try some inserts, updates, and deletes in the example segments below. For simplicity, the example shows at most two immutable segments. When the mutable segment reaches its size limit, the example immediately merges it into whichever immutable segment is smaller. In practice, a mutable segment that gets sealed would exist for a while as a small, standalone immutable segment.

Eventually, there will be too many segments. An index that searches hundreds or thousands of small segments will spend some amount of CPU and memory just tracking all the segments and merging the query results. So, inverted indexes normally need to merge segments. In a merge, two or more segments become a single, larger one. If the document IDs are sequential numbers local to each segment, they all must be reassigned; document i from one segment must not be confused with document i from another.

Merging is algorithmically straightforward; it's just a linear pass through the term dictionary and each postings list. But a merge requires a lot of disk space and a lot of I/O. Merging takes two or more immutable segments as input and produces a new output segment equal to the size of all the input segments, minus the postings for any documents that have been deleted. Segments can easily be many gigabytes each, so a merge process might read and write tens of gigabytes and consume, temporarily, that much extra disk space. Segment merging is always a trade-off between the I/O cost of merging and the efficiency penalty of keeping a larger number of segments around.

But that's just how we insert documents. What about deleting them? It's impractical to delete document IDs from the middle of a postings list, which would require recompressing part or all of the list. So deletion just creates a tombstone. A tombstone is an entry in a table indicating that a given document ID is no longer valid. The inverted index will still produce that document ID as a query result, but it checks each result against the tombstones and will remove that ID before returning it to the caller.

That creates another trade-off. Every deleted document still takes up disk space in the index and wastes CPU time retrieving it from a postings list and filtering it out of the list of results. But the only way to get rid of tombstones is to merge segments and filter the document IDs written to the output segment against the tombstone list. Sometimes, when a large fraction (nearing half) of the documents in a segment have been deleted, it may be worth rewriting that segment by itself, just to get rid of the deleted documents.

Updating a document is nothing more than deleting (tombstoning) its old version and inserting its new one. Immutable segments as described here can't do in-place updates.

How it works in Postgres

Finally, we come to how inverted indexes work to provide full-text search in Postgres. How can we make this work?

CREATE INDEX ON songs USING tin(lyrics);
SELECT title, performer
  FROM songs
  WHERE lyrics ==> 'make you cry'
  ORDER BY tin.score(ctid) DESC
  LIMIT 10;

Each document is a row's value for a single text column. So if an index is on the column lyrics in the table songs, then each song's lyrics would be a single document. The inverted index has to provide a row identifier, the ctid, to tell the Postgres executor which ten rows to fetch title and performer from.

The whole inverted index needs to be stored somewhere, preferably as a WAL-logged index relation so Postgres replication and backups include it. When segments get deleted, their old storage is freed, but space in the middle of a relation can't be returned to the OS. Instead, the index must maintain a list of freed pages so it can reuse them later.

Merges are often triggered when a segment crosses some size threshold or its tombstone list crosses a threshold fraction of the total documents. But we'd really prefer not to perform a merge (remember: tens of gigabytes read and written, possibly several minutes to execute) inline in an INSERT, UPDATE, or DELETE. So we need a job queue and background maintenance workers.

Because it's Postgres, VACUUM needs to work. VACUUM removes dead row versions from the heap and asks each index to remove entries that point to them. VACUUM FULL rewrites all the rows in a table, so the ctids change. If the inverted index stores ctids, it needs to update them.

Document insertions and deletions must be associated with a transaction number so they aren't visible to other transactions until committed, and so that they can be rolled back. The index must return all the results that are visible and none that aren't. Even in the face of that requirement, a top-k query must actually return k rows. To make COUNT(*) and top-k queries work efficiently, the inverted index needs access to row-level visibility and page-level visibility maps.

Queries often combine inverted full-text constraints like lyrics ==> 'make you cry' with traditional SQL constraints like year = 1987. The query planner needs to know when to use a full-text index, when to use a b-tree, and when to use both and combine the results with a bitmap intersection. A query with full-text constraints on multiple columns, like title and lyrics, should use a CustomScan to filter both entirely within the index implementation, because that's much faster than returning a result set for each column and letting Postgres take the intersection.

Solving this optimally for Postgres

For details on how we solved all these challenges and how well it worked, go read the deep dive on TIN, PlanetScale's new full-text search extension for Postgres.

Real Postgres

Neki is built as a sharding and scaling solution for real Postgres. It's not a fork, nor a wire-compatible reimplementation, nor a MySQL sharding idea wearing a Postgres label. Neki uses ordinary PostgreSQL instances that store rows in Postgres data pages using MVCC, carry out transactions, and work as you would expect with psql and other Postgres drivers. Neki builds around those instances to let you shard them, scale them, and manage them as one database.

Let's take a look:

PostgresManager

Using vanilla Postgres means Neki needs a way to run and manage each instance. That includes starting and stopping Postgres, owning its data directory, and configuring replication so a new instance can join a shard. PostgresManager handles this coordination, running as the first process in the Postgres container and managing the postgres process directly.

Sidecar

Postgres uses a separate backend process for each connection and limits how many can be open at once. Neki’s Sidecar sits in front of each instance and pools connections, letting many client connections share fewer Postgres backends.

The Router, which is the component that accepts external client connections, communicates with the Postgres nodes via these Sidecars.

It also reports each Postgres instance's health and whether it is a primary or replica, so the rest of the cluster knows whether it can receive write queries.

The pool doesn't treat every connection the same way. The length of time a connection is checked out for use varies depending on what it's being used for. A multi-statement transaction holds on to its connection until commit or rollback. A session-scoped advisory lock needs a connection of its own, because the lock has to outlive whatever transaction is open at the time and can't share that connection. Everything else checks a connection out and hands it back the moment the statement finishes.

The Sidecar knows which of the three to use because the Router sends the necessary information with the query: autocommit, an open transaction, or a session that has to stay on one backend.

Shards

Each Postgres instance gets its own Sidecar and PostgresManager pair. Real deployments need more than one instance: a primary and its replicas. Neki calls that group a shard, the unit it splits data across. It's always advised to run a shard with a primary and 2+ replicas for high availability, as well as for additional read query capacity.

A shard is considered one Postgres cluster. Its replicas are physical copies of the primary, so they share a catalog and the same object identifiers.

Object Identifiers (OIDs) are how Postgres tracks objects internally, rather than by name. A client reads a column’s type OID off the wire to interpret its bytes and may cache that OID for later re-use. A custom type therefore needs to carry the same OID no matter which shard answers the query. Independent shards can assign that type different OIDs, so Neki designates one shard in the entire Neki cluster as the authoritative shard. This shard is the source of truth for translating custom type OIDs in responses from other shards to match. It ensures OIDs are consistent across the many shards of the Neki cluster.

The authoritative shard's Sidecar also watches for schema changes and reports them to the Routers. This keeps the Routers' view of the schema current when a table is renamed or a column is dropped.

Admin

In a distributed system, instances can fail independently while the rest of the system lives on. Neki is no different. A primary or replica can go down at any moment while its fellow instances on the shard are healthy. The Admin's job is to detect failures, promote a replica, and maintain each shard’s durability policy.

It health-checks every Sidecar, tracks replication lag for each replica, and decides when a shard needs a new primary. When a primary goes down, it coordinates an emergency failover, promoting a replica to take its place. It can also coordinate a planned switchover, which are needed for intentional node resizes and version upgrades. In both situations, Admin uses pg_rewind to bring diverged instances onto the new primary’s timeline, copying only the data that changed since the timelines diverged.

Each shard has a durability policy that determines when a commit is acknowledged:

• Async: The primary acknowledges the commit without waiting for a replica.
• Sync: The primary waits for a replica to confirm the commit, protecting against the loss of a single node.
• Cross-zone sync: The primary waits for confirmation from a replica in another availability zone, protecting against the loss of the primary’s zone.

Postgres enforces whichever one is configured, using its own synchronous replication machinery. The Admin keeps that configuration correct as replicas join or leave shards, or a failover moves the primary to a different zone.

Much of Admin’s work, however, doesn’t involve changing the primary. It repoints replicas to the correct replication source and corrects roles when Postgres and the topology disagree.

Operator

Neki’s components need to be deployed, updated, and replaced when their machines fail. Neki is built Kubernetes-first, and the Operator manages this full lifecycle.

The Operator models a cluster as a hierarchy. A cluster owns routers and shards, and each shard owns the pods running its Postgres instances and Sidecars. When the Neki cluster configuration changes, the Operator works out which pods need to be created, updated, or removed.

How it replaces an instance depends on whether that instance is still running. For a live instance, the Operator builds a replacement and confirms it has caught up before deleting the old one. If a node fails and loses its ephemeral storage, the Operator rebuilds the lost instance from scratch once its safety checks pass.

Admin and the Router handle the database side of those disruptions. Admin coordinates a switchover for planned primary replacements or a failover when a primary goes down. The Router can buffer queries that are safe to retry while a healthy primary becomes available.

Router

We've talked a lot about how the Neki cluster operates and handles failure internally. What we've yet to dive into is how applications use the thing!

The Router is the entry point for clients connecting to a Neki cluster, presenting a single Postgres wire-protocol endpoint to connect to a (potentially) massive sharded database. Applications use Postgres drivers to send SQL and open transactions without managing connections to individual shards.

Authentication and role checks are done as if it were the Postgres instance itself, and the protocol's own extended-query flow and prepared-statement lifecycle are all built into the Router.

Once a query arrives, the Router runs a Postgres-compatible parser against the authoritative shard's catalog, plans it against the current sharding layout, and sends it to whichever Sidecar needs to run it over gRPC.

Not every query can run on a single shard. A join may need data from several shards or an aggregate may need to read from all of them. The Router coordinates that work as a distributed query.

Whenever possible, it leaves the work to the Postgres instances. If both sides of a join are on the same shard, the Router sends the join to that shard. When a join needs to run across shards, the Router executes it itself, choosing between nested-loop, hash, and merge joins based on cost estimations.

Earlier, we covered how Admin promotes a new primary during a switchover or failover. If that happens, the Router can buffer queries, giving the Admin time to complete the handover. For queries that can safely be retried after failing against a primary, the Router buffers the query and waits, for a fixed time, for a healthy primary. Once a healthy primary is available, the Router releases queued queries gradually.

Data Topology

Router, Sidecars, and Admin all need a consistent picture of which shards exist, what key ranges they own, and which tables are sharded at all. If the Router's copy is wrong, a query can land on the wrong shard. This is all specified with a Data Topology, and etcd holds the single, authoritative copy of it. When the Data Topology changes, the Router, Sidecars, and Admin pick up the updated configuration without a restart or manual synchronization.

The Data Topology defines shard groups, named sets of physical shards, each owning a range of routing keys. Each table belongs to a shard group. Shard indexes specify the columns or expressions and the strategy used to turn row values into routing keys. Those keys determine which shard receives each row.

Replicator

As a database grows, its layout may need to change. Tables need to be imported, shards need to be split, and schemas need to change all while applications keep using the database.

Neki's Replicator handles the data movement behind all such operations. It runs as a separate process colocated with a shard's Sidecar and Postgres. It is responsible for copying existing rows to new destinations, and also keeping the data current by decoding changes from a Postgres logical replication stream and applying them as SQL.

Three workflows use the Replicator:

• MoveTables relocates a set of tables, including imports from an external Postgres instance
• Reshard redistributes data across shard key ranges, allowing a shard to be split when it outgrows its capacity
• OnlineDDL changes a table's schema by building a shadow table alongside the original and keeping it current through the same change-data-capture pipeline MoveTables and Reshard use to relocate rows. A final rename swaps the new table into place. This supports changes such as repartitioning a table, alongside changes that would otherwise require a blocking operation.

Once the data has been copied and the destination is caught up, the workflow switches from the original tables or shards to their replacements. This is the cutover. The Router uses the same buffering mechanism that handles primary changes for this step. It buffers queries during that switch and releases them afterward.

Together, these components let Neki scale Postgres horizontally while presenting a single database to applications.

Get started

Neki is in Platform Preview right now.

Start a Neki cluster today: build on it from scratch, or import an existing Postgres database.

One big difference: it's slow.

What Lead is

Lead has the same features as TIN.

Search results are correct, even for complicated SQL and TINQL queries.

Tokenizing is the same. What's a word in TIN is a word in Lead. As we add fancier tokenizers to TIN, they'll come to Lead, too.

Writes work: CREATE INDEX, INSERT, UPDATE, and DELETE.

It's even ACID, with correct snapshot isolation. Really. If you insert rows in a transaction, TIN Lead searches in that transaction will see them, and concurrent transactions won't. If you ROLLBACK, the rows disappear.

It's durable across restarts. If your tests need to restart Postgres, that's OK.

BM25 scoring works, including tin.score and tin.max_score.

It has hit highlighting, including both HTML and ANSI.

What Lead isn't

Lead will never be fast. Every search is a full table scan. The 8.0 GB Wikipedia corpus that TIN can search at 10,260 QPS with a p99 of 2ms takes four minutes per query with Lead. We tried it so you don't have to. Keep your tables in CI under a few MB, and Lead will be fine. Unofficial benchmarking has it searching War and Peace -- 3.4 MB, around half a million words -- in 170ms.

We've left out all the parts that make TIN fast, but kept the rest.

Building on Lead

You can download Lead today. (And be sure to star the repo!) Build your production search apps on TIN and build your tests, CI, dev, and staging on Lead. If you find ways you need Lead to be more like TIN, open a support ticket or find us on Discord.

TIN is available immediately as a GA release for all Postgres and Neki databases. Check it out:

CREATE INDEX an_index_name ON table_name USING tin(text_column_name);
SELECT * FROM table_name
  WHERE text_column_name ==> 'some words';

We built TIN because we believe a good text index should support:

• Boolean expressions, phrase queries, and span queries
• Fuzzy, wildcard, and regular-expression matching for terms
• Case and accent folding
• COUNT(*) queries and BM25-scored top-k queries

A good text index in Postgres must support all of those things while also handling joins, complicated WHERE clauses across full-text and other column types, continuous updates, replication, backups, and correct transaction visibility.

Although there are at least three existing text-search indexes for Postgres already, none of them met all of those requirements. TIN does. TIN is also really, mind-blowingly fast.

What TIN is for

Application developers use text indexes to build a variety of search features. An e-commerce platform might need to search for the top ten products containing all keywords in the search:

SELECT * FROM products
  WHERE description ==> 'stretch denim jeans'
  ORDER BY tin.score(ctid) DESC
  LIMIT 10

A legal discovery platform might be required to return every document containing one or more of a set of keywords, but not care at all about ranking:

SELECT * FROM emails
  WHERE body ==> '[insider trading conspiracy]'

A photo tagging platform might show an exact count of photographs with a particular tag:

SELECT COUNT(*) FROM photos
  WHERE tags ==> '"san francisco"';

Most applications also need to insert, update, and delete documents, even while continuing to query the index. Search queries must return matches based on new or changed rows as soon as they've been committed.

TIN performance and benchmarking

We ran benchmarks to assess performance for all the above use cases and more. We tried workloads:

• With conjunction (must contain all words), disjunction (must contain any word), and phrase (must contain all words in sequence) queries and a mix of all three.
• That count documents or that ask for the top k by BM25 score.
• With and without clients writing new data to the index concurrently with the benchmark query workload.

Workloads and corpus

We have measured TIN against a variety of text corpora: all of Wikipedia, a collection of Reddit comments totaling 2.3 TB, and a mixed workload we call simply "pile" with 797 GB of open-access research papers, legal documents, public domain books, and Enron emails. The benchmark results we share in this article are from an export of questions and answers from Stack Exchange: an 85 GB corpus with 150 million documents. Because the corpus has no standard query trace, we generated a synthetic one by sampling substrings ranging from 2 to 15 terms. We interpreted each substring three ways: as a conjunction, as a disjunction, and as a phrase query, for a total of 3,762 queries.

Test environment

We ran our benchmarks on an AWS i7i.8xlarge EC2 instance with local NVMe storage and a modern, AVX-512-capable CPU. For each text-search extension, we set up Postgres 18.6 in an isolated container limited to 8 vCPUs and 32 GB of RAM. That's small enough to show how each index system performs when the index doesn't just fit in Postgres buffers. The benchmark phases ran sequentially, so the engines did not compete for resources. We chose a standalone EC2 instance to minimize the impact of operational overhead and replication and to ensure that anyone who wants to reproduce our benchmarks of competing text-search indexes can do so using the same instance type and container limits.

To drive the search traffic against the Postgres containers, we used the ParadeDB Benchmarker. We have a forked version that pre-warms before beginning measurement and adds metrics for bytes read and WAL bytes written. We left all Postgres parameters at the defaults that the Benchmarker supplies, except for three: we set max_parallel_workers to 8 (from 40), shared_buffers to 24 GB (from 128 MB), and maintenance_work_mem to 24 GB (from 64 MB), to best match the resources of the container. We ran the Benchmarker on the same EC2 instance as the target Postgres server, to ensure that network latency did not impact the measurements.

For each scenario, we measured the performance of TIN v1.0.2 against all the other Postgres text-search indexes that were capable of running the workload at all: ParadeDB v0.25.2, pg_textsearch v1.4.0, and the GIN index built into Postgres v18.6. Aside from TIN, only ParadeDB was able to complete all of the benchmarks.

Index build time and size

Indexes range from 33% to 61% of the size of the corpus, and they took from 8 to 129 minutes to prepare, build, and finalize. The three engines other than TIN failed with the container's configured 32 GB limit, so for index builds only, we increased the available RAM as shown in the table. Before running queries, we set the container back to 32 GB of RAM for everyone.

Mixed queries, top-10 ranked

Our first benchmark compares TIN against ParadeDB, for a workload with mixed (conjunction, disjunction, and phrase) queries, top-10 results by BM25 score, with no concurrent writes to the index. TIN handles 25× as many queries per second as ParadeDB does, with p99 latencies 26× lower. GIN can't complete this benchmark, because it runs out of memory performing the disjunction searches. pg_textsearch can't complete the benchmark because it handles only disjunction searches.

Conjunction and phrase queries, top-10 ranked

Our next benchmark compares TIN against ParadeDB and Postgres GIN, for top-10 conjunction and phrase queries, with no concurrent writes. TIN and ParadeDB rank using BM25, while GIN ranks using ts_rank_cd. TIN handles 10× as many queries as ParadeDB and 541× as many as GIN, with p99 latencies 6× and 1,356× lower, respectively. pg_textsearch is again absent because it handles only disjunction queries.

Disjunction queries with concurrent writes

Our third result compares TIN against both ParadeDB and pg_textsearch, for a workload with disjunction queries, top-10 results by BM25 score, and a concurrent client targeting 1,000 UPDATE queries per second. TIN handles 36× as many queries as pg_textsearch and 57× as many queries as ParadeDB, with p99 latencies 24× and 36× lower, respectively. Over the course of a ten-minute run, TIN completes 270,279 updates, while ParadeDB completes 185,584, and pg_textsearch completes only 735.

ParadeDB's approach to accepting writes sacrifices read throughput and latency. pg_textsearch maintains the same 3.5 QPS for readers both with and without writes because continuous read traffic prevents write traffic from ever getting the locks it needs, so writes stall after just a few seconds. GIN is again absent because it runs out of memory on disjunction queries.

When the index fits in memory

In the intro, we claimed that TIN is mind-blowingly fast.

Our final graph shows what TIN, ParadeDB, and Postgres GIN can do when the index fully fits in shared buffers. This workload counts (but does not rank) the documents that match a disjunction query against Wikipedia, an 8.0 GB corpus. pg_textsearch is absent here because it can only perform top-k queries, not counting queries.

Full results

That is perhaps enough graphs, but it doesn't cover all of our use cases. Here are those same scenarios, plus several more, in table form. The "MB/query" column shows how much data each index read from the disk or block cache for each query. TIN's lower numbers for MB/query are part of why it's faster, and they also reduce the impact of TIN queries on the block cache and I/O capacity, meaning that other queries on the same server stay fast, too.

Conjunction, disjunction, and phrase queries; top-10
┌────────────────────────────────────────────────────────────────────┐
│                            QPS        p99    MB/query     Updates  │
├─────────────────────────┬───────┬──────────┬───────────┬───────────┤
│ TIN - read-only         │  199  │   256ms  │       65  │           │
│     - with updates      │  172  │   284ms  │       88  │  271,398  │
├─────────────────────────┼───────┼──────────┼───────────┼───────────┤
│ ParadeDB - read-only    │  7.9  │ 6,765ms  │      582  │           │
│          - with updates │  6.0  │ 7,990ms  │      591  │  193,487  │
└─────────────────────────┴───────┴──────────┴───────────┴───────────┘

Conjunction and phrase queries; top-10 (read-only)
┌───────────────────────────────────────────────┐
│                  QPS         p99    MB/query  │
├───────────────┬───────┬───────────┬───────────┤
│ TIN           │  242  │     212ms │        73 │
├───────────────┼───────┼───────────┼───────────┤
│ ParadeDB      │   24  │   1,279ms │       668 │
├───────────────┼───────┼───────────┼───────────┤
│ Postgres GIN  │  0.4  │ 288,066ms │       595 │
└───────────────┴───────┴───────────┴───────────┘

Disjunction queries; top-10
┌────────────────────────────────────────────────────────────────────────┐
│                                 QPS         p99    MB/query   Updates  │
├──────────────────────────────┬───────┬───────────┬─────────┬───────────┤
│ TIN - read-only              │  148  │    324ms  │     48  │           │
│     - with updates           │  125  │    354ms  │     77  │  270,279  │
├──────────────────────────────┼───────┼───────────┼─────────┼───────────┤
│ ParadeDB - read-only         │   17  │  2,385ms  │    303  │           │
│          - with updates      │  2.2  │ 12,634ms  │    394  │  185,584  │
├──────────────────────────────┼───────┼───────────┼─────────┼───────────┤
│ pg_textsearch - read-only    │  3.5  │  8,646ms  │ 11,639  │           │
│               - with updates │  3.5  │  8,409ms  │ 11,656  │      735  │
└──────────────────────────────┴───────┴───────────┴─────────┴───────────┘

Conjunction, disjunction, and phrase queries; COUNT(*) (read-only)
┌─────────────────────────────────────────┐
│              QPS      p99     MB/query  │
├───────────┬───────┬──────────┬──────────┤
│ TIN       │  179  │   438ms  │      97  │
├───────────┼───────┼──────────┼──────────┤
│ ParadeDB  │   10  │ 2,704ms  │     544  │
└───────────┴───────┴──────────┴──────────┘

Disjunction queries; COUNT(*); Wikipedia corpus (read-only)
┌───────────────────────────────────────────────────┐
│                     QPS         p99     MB/query  │
├───────────────┬──────────┬─────────────┬──────────┤
│ TIN           │  10,260  │        2ms  │     1.7  │
├───────────────┼──────────┼─────────────┼──────────┤
│ ParadeDB      │     291  │       95ms  │      22  │
├───────────────┼──────────┼─────────────┼──────────┤
│ Postgres GIN  │     1.4  │   30,292ms  │     2.5  │
└───────────────┴──────────┴─────────────┴──────────┘

As you can see, in a wide variety of scenarios, TIN has throughput at least 8× higher than the alternatives, reads far less data from the disk, and experiences only a small performance drop even while the index is updating hundreds of rows per second.

Why TIN is fast

TIN's performance in benchmarks may be hard to believe. In the hopes of making it more believable, or at least satisfying the reader's curiosity, we'll explain a bit about architectural choices that make TIN so fast. In short: all document postings are Postgres ctids rather than contiguous document identifiers, and this lends itself to highly vectorized intersection and union operations on modern CPUs.

Document identification

A text index needs an identifier for each version of each document it indexes. It groups those identifiers into highly compressed postings lists; each postings list tracks all the documents that contain one given word. In a large corpus, a postings list for a common word like "the" may contain billions of postings, while the postings list for a term like "xyz-9876" would contain only a few.

Most text search systems organize their indexes into segments. The n documents whose postings exist in a segment are usually assigned document identifiers 1 to n. Sequential document identifiers allow postings lists to be highly compressed using various techniques such as delta-encoding and bit-packing. But it also means document identifiers in different segments are assigned independently; document ID 42 in segment 4 is a completely different document than ID 42 in segment 7.

TIN also organizes its index into segments, but not for purposes of document numbering. Instead, TIN directly uses Postgres' ctid value as a document identifier.

Every version of every row (tuple) stored in a Postgres table has an associated ctid value. ctid is short for "current tuple identifier." Any row inserted or updated gets a new ctid. It is a 48-bit number that directly identifies a tuple's physical location in the Postgres heap. Represented textually as (<block number>, <offset number>), the upper 32 bits indicate the block number and the lower 16 indicate the offset within that block. From now on, we will refer to the <block number> part as the "page number" or "page."

Given the ctid of (190, 17) we know that the tuple it represents is the one at the 17th slot on page 190. Instant O(1) lookup! You can even query and retrieve rows from the heap directly using ctids:

-- retrieve the first 10 rows from "books" in physical heap order
SELECT ctid, id, title FROM books ORDER BY ctid LIMIT 10;

-- no scan required!  instant O(1) lookup of the row
SELECT * FROM books WHERE ctid = '(190, 17)';

TIN directly uses ctids because Postgres internally uses ctids. Postgres extensions that implement a new index type must return ctids. Postgres bitmap scans are backed by potentially lossy bitmaps of ctids. Postgres' internal index types (b-tree, GIN, GiST, and hash) use ctids as their postings. ctids are everywhere within Postgres.

To operate within Postgres, a text search system that assigns sequential identifiers must, at some point, convert those identifiers back into a ctid in order for Postgres to work with it. Both ParadeDB and pg_textsearch maintain a separate data structure just to perform this mapping. If a text search matches 10 million rows, ParadeDB and pg_textsearch have to look up 10 million identifiers in their ctid mappings. TIN avoids that work completely.

48-bit identifiers are crazy

Normal postings-list compression techniques don't work well with discontiguous 48-bit numbers. Delta encoding breaks at each page boundary, and bitmaps are too sparse to be efficient. Fortunately, some interesting properties of Postgres pages make two-level bitmap encoding practical. An 8KB page can never contain more than 291 tuples (8192 bytes, minus 24 for the page header, divided by at least 28 per non-empty tuple), and for table schemas with TEXT and other columns, pages often contain 32 or fewer tuples. So the list of page numbers is dense enough to use a bitmap, and within each page, the list of offset numbers is dense enough (and small enough) to use tiny bitmaps per page.

Savings relative to naively storing 48-bit ctid values can be quite significant. Over an entire corpus, high-frequency terms approach 1 bit per posting, medium-frequency terms settle around 7 bits per posting, and rare-frequency terms can approach 25 bits per posting. Terms that appear only once are not stored as bitmaps at all.

Work elision and vectorization

TIN's page-level bitmaps (which pages contain a given term) have 256 bits, which fits nicely into vector registers on any x86 CPU with AVX2 or higher. That allows several optimizations.

Consider the query the AND rareword. TIN ANDs the page-level bitmaps, 256 bits (pages) at a time. Any bit that's absent from the intersection is a page whose offset-level bitmaps TIN doesn't need to decode at all.

For COUNT(*) disjunction queries such as the OR rareword, TIN often skips reading postings lists entirely. TIN's index metadata stores each term's exact posting counts. If the page-level bitmaps for two words have no bits in common, the count of their disjunction is just the sum of those exact posting counts.

Every page-level bitmap fits into a single AVX2 register, and every offset-level bitmap fits into either one AVX-512 register or two AVX2 registers. Conjunction and disjunction queries are just AND and OR instructions on those vector registers, respectively. Queries that count the number of matches can use CPU-native POPCNT instructions to count the bits in the resulting bitmap. Expensive loops and branch instructions are largely avoidable.

A query that wants rows rather than counts computes the ctid from the bit position rather than looking it up on disk. The position of a set bit is the ctid.

The document ctids that TIN returns to Postgres from a given segment naturally identify pages, and tuples within a page, in heap order. This means that when Postgres needs to read matched tuples from the heap, it happens in heap order. Even with modern NVMe disks, sequential access is far faster than random access; TIN gets this optimization for free.

Solving MVCC

TIN returns results that are MVCC-correct, meaning a statement executed at any point in time sees or operates only on tuples that are currently visible to it. This means every heap-backed query result needs to be checked for visibility relative to the current snapshot.

Heap checks

There are a few different approaches to this. Some queries are inherently heap checked:

SELECT a, b, c FROM lyrics WHERE content ==> 'give you up'

Because the query returns actual heap data (the a, b, c columns), TIN must fetch from the heap all matching ctids returned by ==> 'give you up' anyway. When TIN asks Postgres for the physical tuple data behind each ctid, Postgres tells TIN whether that tuple is visible to the current snapshot. If it is, TIN returns it; otherwise, TIN moves to the next matching ctid, until all visible matches have been returned.

Visibility map

Other query shapes can be executed similarly to Postgres' "Index Only Scan" where the answer is returned directly from the index without touching the heap (or at least hopefully not all of the heap). Consider a count-only query like this:

SELECT COUNT(*) FROM lyrics WHERE content ==> 'give you up'

If every heap page is marked all-visible, TIN can return that count without touching a single heap page.

Not all data is static, of course, and in the case of mutated heaps, TIN does additional optimizations to ensure it's only counting visible rows by performing direct intersections with Postgres' visibility map. TIN's page-level bitmaps are exactly the right mechanism to intersect efficiently against Postgres visibility maps, which are also page-level bitmaps. Only ctids on not-all-visible pages need to be checked against the heap. Normally, a Postgres index returns all ctids that match regardless of visibility, and the Postgres executor checks visibility for each one. TIN plans custom scans that move visibility checks into TIN itself, where they can take advantage of vector instructions on page-level bitmaps.

VACUUM and TIN's liveness bitmap

Text indexes that support deleting documents typically keep some kind of "tombstone" list that's appropriate to their engine. TIN is no different. TIN keeps a per-segment liveness bitmap, one bit per ctid, organized the same way the page-level and offset bitmaps work. When VACUUM runs and determines a ctid has been deleted from the heap (as the result of an UPDATE or DELETE), TIN clears that ctid's liveness bit. Groups of pages with at least one cleared bit are marked, and when a query touches a marked page group, TIN also ANDs the offset bitmaps from the postings list against the liveness bitmap, so it never returns or counts a tuple that has truly been deleted.

Segments and merging

When it first creates a new index for a table, TIN creates n immutable segments, each containing postings for 1/n of the pages associated with that table in the heap. As data is changed, TIN creates mutable segments, which are less efficient for searches but allow easy insertion of new documents. Eventually, a background worker promotes each mutable segment to an immutable segment: unchanging, but much more efficient to search.

After a while, TIN will begin to merge immutable segments into larger immutable segments. This also happens in the background.

Text indexing systems that use sequential document identifiers are required to renumber all documents when they create a new, merged segment. As mentioned above, document ID 42 in segment 4 is not the same as ID 42 in segment 7. So when segments 4 and 7 are merged, a new numbering must be applied to the combined set of documents and the entirety of each segment's data gets repacked, recompressed, and rewritten. While it's not quite 2× the storage to merge two segments, it can be close.

TIN does not suffer the renumbering problem nor its downstream write-amplification effects.

Because TIN uses Postgres' ctid values as its document identifiers, there is nothing to renumber. A posting like (190, 17) means the same thing in every segment. Page-level and offset-level bitmaps mean the same thing in every segment. When TIN merges segments, many bitmaps from each old segment can be reused intact in the new segment. They don't have to be recompressed or even copied; TIN can simply transfer ownership of bitmaps stored on disk from the old segments to the new one. This reduces write amplification and saves most of the CPU and I/O costs normally associated with merging segments.

Summary

So that's why TIN is at least 8× faster in every benchmark: the downstream effects of choosing ctid as the native format for each posting in the index.

If you want to see how fast TIN is on your text data, read more about the features or jump straight to the getting started guide. We look forward to seeing what you build with it.

Errata

September 20, 2026: We originally reported using 1,719 queries in the StackExchange benchmark. We actually used 3,762.

This next run ended with 512 shards running 118 million queries per second with 1.22 PiB of data.

Linear scalability

The benchmark was very simple. A single-shard point select, one row fetched per-query by primary key. No writes, joins, or cross-shard queries. The workload that each shard receives is isolated, in that there are no single queries that span multiple shards.

Our target was to sustain 200k QPS on each shard, and then grow the cluster to increase throughput. Five shards, then fifty, then 512.

Ten times the shards, ten times the throughput. Then ten times again. From 5 shards to 50 the per-shard rate held within 0.8%. At 512 the shards still had headroom, so we let the load generator use it and each shard settled at 231k QPS instead of 200k.

118.5 million QPS

We sustained 118,538,803 QPS for 16 minutes across 512 shards and 1.22 PiB of data. Our largest recording was 118,747,267.

• 512 shards, each with one Postgres primary each on an r8g.16xlarge
• 480 Neki routers, each on its own 8xlarge instance
• p99 latency of 6.06ms at the router and 13.95ms at the client
• 67 errors per second, about one query in 1.8 million
• 15.8M read IOPS across the fleet
• over 2 Tb per second on the network

Worth being clear about this run: the shards were primary-only with no replicas, the workload is read-only across queries ranging in complexity, and we did not fail over during the measured window.

We will go into details on the engineering effort and interesting challenges we faced along the way to reaching 100 million QPS in a future article.

Neki is built from lessons we’ve learned over eight years of running some of the largest sharded MySQL clusters in the world. Thousands of production workloads with millions of queries per second for companies where even a few seconds of downtime is a very public event. We know what it means to power the world’s biggest tier 0 workloads.

When we released PlanetScale Postgres a year and a half ago, we knew we needed to do more. In that time we’ve onboarded several thousands of customers on PlanetScale, some of them rivaling the size of our largest MySQL customers. Time and time again, we watched teams approach the ceiling of a single machine with Postgres. Metal bought them time, but with customers hitting the upper limit of what a single machine is capable of, we found there was no good option to hand them. Enter Neki.

What is Neki?

Neki is sharded Postgres from PlanetScale. It lets you scale a Postgres database across many machines while keeping real Postgres on every shard.

Your application connects to a Neki router over the standard Postgres wire protocol, so your existing drivers, ORMs, and connection string keep working. Each shard is a full Postgres cluster with one primary and at least two replicas across 3 availability zones. There is no custom storage engine, so extensions, SQL support, and performance behave the way Postgres does.

You choose the shard key and control how tables are grouped and distributed through a JSON data topology. Schema changes, version upgrades, failovers, imports, and resharding all run as built-in fully online workflows. You also get the PlanetScale features you already rely on, including Insights, schema recommendations, branching, and MCP.

You don't have to shard on day one. Run Neki as a single primary with replicas, and when you outgrow one machine, resharding is a workflow you run against the cluster you already have.

Why Neki?

You already know the problems that come with fast-growing Postgres databases: tables too large to vacuum or index without affecting traffic, backups taking hours, connection limits, maintenance windows for schema changes, transaction wraparound and so much more.

You can move to a bigger instance, but eventually you run out of big enough machines, and the problems don’t scale linearly as you add more cores and IOPS.

The existing answers each ask you to give something up. Application-level sharding pushes routing into your code. Postgres-”compatible” distributed databases hide the shard key from you, take away your extensions, and add complexity and latency which becomes difficult to handle and debug.

So we built Neki with a few principles, the biggest one being: stick to Postgres, don’t work around it, fake it, or turn away from it.

How does Neki work?

We architected Neki from first principles for Postgres, with real Postgres on every shard. There are four moving parts.

Neki routers

Your application first connects to a Neki router. The router speaks the Postgres wire protocol so your existing drivers and ORMs keep working with a single connection string. A router has a full Postgres query parser, a distributed query planner, query buffering and more. It parses your query, builds a plan that decides which shards should run it, sends the work out, and combines the results back into one stream. Routers can scale vertically and horizontally, so no single router becomes the bottleneck.

Sharding and shard groups

Every shard in Neki is real Postgres with 1 primary and at least 2 replicas, spread across availability zones. There is no modified storage engine. Extensions, SQL support, and performance behave the way Postgres behaves, because it is Postgres.

Shards are organized into shard groups, so different tables or workloads can live on different sets of shards. Each shard uses a configuration profile that defines its instance size, replica count, storage, Postgres parameters, and extensions, so you can size each group for its own traffic.

Connection pooling

Sidecars run alongside every Postgres instance. This is the piece that makes Neki's connection handling meaningfully better than just sticking PgBouncer in front of a database. Because Neki controls both ends of the connection, the router side and the Postgres side, it can size pools to what each instance can actually serve instead of estimating from outside the process.

Control plane

The control plane tracks the health of every node, runs planned switchovers and unplanned failovers, and coordinates the workflows that reshard data, apply schema changes, and perform version upgrades.

Data topology

Tying it together is the data topology, a JSON configuration that maps your logical tables onto physical shards. You define shard indexes, which specify the column Neki routes on and how that value gets hashed, and shard groups, which control how many shards a set of tables spreads across and which shards those are. Routers cache the topology and consult it on every plan.

What you get beyond sharding

Everything you would normally schedule a maintenance window for runs as a built-in workflow in Neki. Workflows provision new target nodes, catch them up with replication, switch traffic with a __neki metafunction, and retire the old nodes. All through the same psql connection your application uses.

This online operations model covers schema changes, version upgrades, planned and unplanned failovers, imports, and resharding.

Neki also includes all of the features you’ve come to rely on with PlanetScale: Insights, schema recommendations, branching, MCP, and more.

You can also run Neki unsharded, as a single primary with replicas. You get the improved connection pooling, online DDL, zero downtime upgrades, and health monitoring before you need to shard. When you do, resharding is a workflow you run against the cluster you already have.

What is a platform preview?

We wanted to get Neki into your hands as soon as possible. You should not run production workloads on Neki during the platform preview. The product is still changing, and some of those changes will be breaking.

If you have any feedback, questions, or face any issues during the platform preview, please let us know. Fill out a support ticket or join our Discord

Try Neki today

Sign in to PlanetScale, opt in to the platform preview, and create a Neki cluster. Read the Neki docs for more information about Neki's architecture, how to shard, and more.

If you have a large Postgres cluster and are curious whether Neki is a good fit, get in touch. We would love to do a private demo for your team, dig into your schema and query patterns, and give you real suggestions on how to shard.

You've just been paged. Every query is failing to execute. Find out why.

Years ago at GitHub, this was a question Sam Lambert would ask in interviews.

The solution involved a downed database that started with a MySQL schema change needing an exclusive lock on a table, but it couldn't get one. Another session had already touched that table and never committed. That open transaction blocked the schema change.

Once the schema change was waiting, new queries on the same table queued up behind it. Nothing was deadlocked. Nothing would time out on its own. The whole pile-up sat there until someone found the connection at the top of the chain and killed it.

(By the way, you'd avoid this MySQL lock on PlanetScale Vitess since ALTER would have been an online schema change, copying in the background and only taking a brief lock at cutover.)

This isn't rare, and it isn't exclusive to MySQL. You can reproduce the same pile-up on Postgres just as easily.

Want to see a stuck database?

You won't need elevated permissions. A normal application connection is enough.

All it takes is an unhandled exception and a poorly timed migration.

Say you have a Postgres database with a hot table named orders.

On one connection, a transaction begins a SELECT query, but the app throws an exception.

-- Connection A: Ordinary application user
BEGIN;

SELECT * FROM orders WHERE id = 123;

-- ...then the app throws an exception 💀

The SELECT query finished, but Postgres keeps holding the connection, waiting for the transaction to end with COMMIT or ROLLBACK, which it never does because of the app's thrown exception.

While Postgres waits on that connection, another connection attempts to run a migration.

-- Connection B: Migration role waiting on lock
ALTER TABLE orders ADD COLUMN foo integer;

The SELECT query only needed a read lock (ACCESS SHARE). The migration needs a lock that blocks all other access (ACCESS EXCLUSIVE), so it waits.

Postgres will not let later queries jump the queue. Every new query on orders waits behind the already-stuck migration, which is itself stuck behind the initial SELECT connection that never closed.

-- Connections C, D, E, etc: Waiting on the lock
SELECT *
FROM orders
WHERE customer_id = ?;

If the app doesn't handle errors gracefully and depends on quick responses from orders, you've got downtime.

Not because of a hack, but because one unhandled exception prevented a COMMIT, kept a connection open, blocked a migration, and that migration blocked everything else.

By the way, this is preventable. It happens because the default for idle_in_transaction_session_timeout is disabled. Give it a value and Postgres times out transactions that haven't closed.

Freeing a stuck database

Debugging the state of connections in MySQL and Postgres can be clunky.

As covered in See what your database is doing right now with Connections it is possible to run a query on a loop in your terminal to see active connections. But it's not an ideal interface.

It also depends on your database having spare connection slots. The worst-case scenario is your database is overwhelmed with connections that haven't been closed, meaning even you can't connect to debug and kill them.

Because this experience is so terrible, we've added the ability to view and kill connections in Postgres and MySQL databases on PlanetScale.

Connections in the dashboard

PlanetScale's connections tooling uses a reserved administrative connection so it can still show you a list of active connections and processes even when connections are exhausted and your application can't connect.

In the PlanetScale dashboard, you can now click the Connections tab to view a live list of connections. You'll see a bunch of useful columns like Process ID, State, and Duration, but most importantly, you will see a column for Blocked Queries.

From here, it's easy to see which connections are currently blocking others from proceeding. You can click into any connection to see more details, such as the query currently being performed by that connection.

Kill process gives you three options, from least to most disruptive:

• Cancel query stops the running statement and leaves the connection open, so the app can send another query
• Terminate transaction rolls back the open transaction and closes the connection, but only if that same transaction is still running
• Terminate connection drops the backend entirely, whether it is idle or mid-query

Thinking back to the downtime example, the idle SELECT had already finished, so canceling the query will not release the lock. In that scenario you would need to terminate the transaction or the connection.

Over on Vitess, each process has two options: cancel the query or terminate the connection. In a sharded database, you can view the process list per keyspace and shard.

Connections via CLI

If you'd rather work with a terminal UI, launch:

pscale branch connections top <database> <branch>

This opens an interactive live view that refreshes about once every second. The most important connections are sorted toward the top. You can navigate through the view using keyboard shortcuts and open any connection to see more details.

If your agents prefer the current state of connections in JSON, use this one-shot command instead of the live view:

pscale branch connections show <database> <branch> --format json

That output includes the IDs needed to cancel a query or terminate a connection, which you can still require confirmation for. See the CLI reference for the kill commands that go with it.

Migrate today, stress less later

The correct answer to land your dream job today is "Use PlanetScale."

Next time you're paged because everything is erroring, I hope you've already migrated over. The interview answer hasn't changed, but connection management on PlanetScale has made it easier than ever before.

Product success leads to lots of data, lots of data leads to big tables, but big tables lead to predictable problems.

A table can be large (many rows), wide (many columns), or "fat" (oversized values). Any of these can cause you problems.

How a large table can cause an outage

First, let me tell you a story.

A customer had two tables, A and B. Table B is large and unpartitioned. It has a foreign key to A with cascade delete turned on. A single delete on A has to delete every child row in B. That can be 100s or 1000s of rows in B for every one in A.

In this customer's case, deletions from A ran in a background job and attempted to remove around 100,000 rows from A. The deletes took so long that they timed out. The job retried. Each attempt generated a pile of WAL (the write-ahead log Postgres uses to record every change), which saturated the network and CPU between the primary and its replicas. This caused the replicas to lag.

The application checked replica LSN (a replica's position in that WAL stream) before sending a read to a secondary. The secondaries were not caught up, so every query went to the primary.

The primary was already busy writing WAL. It also had to respond to all of the reads. And so a simple cascade delete on a large table quickly became an outage.

Possible fixes for a large table

The problems covered in this post are common to all Postgres databases but are made worse by those containing large tables. You might reach for one of a few solutions, which we'll examine throughout.

Partitioning splits a large table based on a key value. For example, if your large table stores rows with a specific date, such as when an event occurred, cutting that table into partitions of months or years creates more, smaller, and more manageable tables.

This can be useful for some large-table problems.

Vertical scaling refers to adding more resources to make more space or brute-forcing your way out of performance bottlenecks such as increasing CPU, RAM, or buying larger and faster disks.

It's almost always a band-aid solution that only hides the real problem.

Sharding (horizontal scaling) splits the large table, or the entire database, into separate database clusters connected by a router to act as one. Just as Vitess is sharded MySQL, Neki is sharded Postgres.

With more, smaller, and isolated database clusters, almost all of your expected large table problems are resolved. More on that later in the post.

Slow, late vacuum

You may already know that deleted rows in Postgres aren't automatically removed; they are marked for deletion. These marked rows hold space until the vacuum process reclaims it. Read Every UPDATE leaves a ghost to learn more.

What you might not realize is that vacuum runs per table.

Vacuuming a single large table can tie up a worker for a long time. Vacuum uses a ring buffer so it does not evict the rest of shared_buffers (Postgres's shared page cache), but on a table larger than RAM it still competes with user queries for the same disks.

Large tables take longer to vacuum, and they also wait longer to start.

On Postgres 17, the default trigger is 20% of the table. A table with ~500 million rows needs ~100 million dead tuples before autovacuum begins. You can modify this value cluster-wide or per table by changing autovacuum_vacuum_scale_factor.

Postgres 18 added autovacuum_vacuum_max_threshold with a default value of 100,000,000. So a billion-row table triggered autovacuum at 100 million (still a high number) instead of 200 million. Below 500 million rows, the old scale factor was still in play.

Postgres 19 adds parallel autovacuum, off by default. Extra workers can finish one large table faster, but that still doesn't change when vacuum starts.

Autovacuum also runs ANALYZE, which has the same default-doesn't-scale problem. It samples a fixed number of rows no matter how large the table is. So the planner's estimates potentially get worse as the table grows. Raise the target with ALTER TABLE ... SET STATISTICS on the columns the planner actually needs to get right.

Vacuum on large tables is late to start, slow to finish, with more dead rows while it runs. Each cycle is worse than the last, even when vacuum reports success.

Partitioning can be a good first move to split your large table into more reasonable-size chunks, if it has a suitable key.

Partitioning also helps avoid autovacuum's late start. Each partition is its own smaller heap, so the 20% trigger fires after far fewer dead tuples. Workers vacuum partitions in parallel and ANALYZE samples a smaller relation.

Disks and shared_buffers are still shared with the rest of the cluster, so it's not a perfect solution.

Vertical scaling can help more workers finish sooner with faster disks. That does not change the trigger on one huge heap, and the table may still not fit in RAM.

Sharding (horizontal scaling) gives you the best of both worlds. Smaller tables on smaller databases, each with their own isolated resources.

Sharding also solves foundational parts of Postgres that can't be resolved by more resources.

Every write transaction gets a 32-bit ID, and vacuum uses these IDs (xmin) to decide which row versions are still visible. These IDs are cluster-wide, not per-table. Vacuum freezes old tuples so XIDs can be reused; wraparound is the age of the oldest unfrozen XID (relfrozenxid in pg_class), not how many XIDs the database has ever issued. If that age reaches around 2.1 billion, Postgres goes into read-only mode. Postgres backups under the hood covers transaction wraparound in more detail.

Since each shard is its own cluster, a large table separated across shards is less likely to put a cluster into read-only mode.

Additionally with isolated I/O, any work vacuum does on one shard does not contend with queries on another. ANALYZE becomes a per-shard operation.

Wasted time on incomplete repacking

Vacuum can be prevented from reclaiming space even if it finishes successfully.

Vacuum can't remove a dead tuple while any transaction in the cluster still needs it.

But a slow vacuum on a large table may have wasted its time to completion. Many of the rows it found to remove may still be required by transactions that started before vacuum finished and haven't completed yet. Vacuum will need to walk the entire table again later.

This can be an issue on tables of any size, but is more annoying on large tables as they take longer to complete.

Free space that was successfully reclaimed can be reused for new rows, but the table file usually does not shrink. That leftover file size is bloat.

Should your table get bloated, PlanetScale Insights scans once a day and opens a recommendation when estimated bloat is over 25% and 100MB.

To compact a bloated table while reads and writes continue, enable pg_squeeze on the database's Clusters page (that requires a restart), then run a one-time squeeze or register the table for regular cleanup.

Be aware that pg_squeeze has no throttle. Squeezing a large table can saturate I/O and CPU and cause an outage. Only run it when the cluster has spare capacity.

Partitioning creates more, smaller heaps. So a wasted pass is cheaper and can be retried sooner. Squeeze is per partition, so the spare disk you need to run squeeze is the size of a partition, not the whole table.

Splitting the table does not split the snapshot horizon. xmin is still cluster-wide, so a long query anywhere on the cluster still pins every partition.

You can drop old or bloated partitions instead of compacting them.

Vertical scaling adds more spare disk space, which may be the only reason squeeze can run at all and/or finish sooner. That lets you compact the same large file faster.

Sharding creates distinct Postgres database clusters, so a long query or dump on one shard doesn't pin vacuum on any other. Squeeze runs only on the copy in its shard, not the entire large table. Each cluster tracks its own xmin.

In a sharded database, wasted vacuum passes are cheaper and less likely to happen.

Slow queries hold connections longer

It is impossible to write a Postgres article about performance and not mention that Postgres has a connection-per-process architecture. Connection hygiene matters even more when your database has a large table.

Sequential scans (reading large segments of the table from start to finish), big sorts, and heavy joins against a large table take longer, so they hold connections longer.

Raising max_connections feels like a logical solution to avoid "too many clients already," but this only lets more concurrent connections perform these slow queries. At best, this starves other workloads of new connections. At worst, a traffic spike on your large table is more likely to trigger an out-of-memory (OOM) event.

A large table will not stay in shared_buffers. More processes scanning it means more cache eviction, more disk usage, slower queries, and longer-held connections.

Partitioning splits your large table, which can help queries finish faster (and release connections sooner) so long as they access only one or a few partitions. But cross-partition queries can get worse.

Buy all the RAM and CPU you like with vertical scaling, but you're still not putting a hundreds-of-GB table into cache. You cannot spend your way out of inefficient connection handling.

Sharding spreads your backends across multiple clusters, where each shard-distinct query is shorter and more likely cached.

A connection held by a large table is one the rest of the product cannot use. With workloads spread across shards there may be less overlap.

A scatter-gather query with no shard key can hold more connections, not fewer. Selecting the right key to shard on is critical to seeing performance benefits from sharding.

Slower backups and recovery

Postgres gives you three ways to take a backup. A large table complicates each one in a unique way.

A logical dump with pg_dump writes live rows, so dead tuples stay out of the output. The dump still walks the bloated heap, and it holds a transaction snapshot the whole time. On a small table, that pin is brief. On a large table, the backup is a long-running transaction against the cluster.

A file system backup copies the data directory, bloat and all. Faster than dumping rows. A consistent copy usually means shutting the database down, or relying on atomic snapshots from the file system. You might be okay with a maintenance window for a database of small tables, but a single large table could add hours just to copy one relation.

Continuous archiving copies the data files while Postgres is still running, then keeps the WAL so you can replay through the modified bits and land on a consistent copy. No dump pin, no shutdown. You still copy the large table, bloat, and its indexes. What grows with the table is restore time. How long an incident waits on this heap plus the WAL written while the copy ran.

That third option is how PlanetScale backs up Postgres. We copy the data files and continuously archive WAL on a throwaway node rather than on your primary, so the backup doesn't compete with production queries for disk and doesn't pin the primary.

We can't skip the size of those files. A large table that doesn't fit in memory, plus its indexes, plus its bloat, is what has to land in object storage.

Partitioning rearranges files but doesn't shrink the copy. A pin held during dump is still cluster-wide. A physical backup includes the same bloat even if it is split across smaller tables.

Vertical scaling means faster copies through more resources. But the files aren't any smaller, so restore is only quicker because the disks are faster.

But neither partitioning nor vertical scaling changes the size of the data.

Sharding breaks down backups and restores into smaller, distinct units of work, allowing them to complete much faster. Recovery time depends on the slowest shard instead of the largest table.

Too many indexes

A large table isn't viable to scan from start to finish and is likely queried in many different ways.

To keep queries fast, you keep adding indexes to the table. While indexes make queries fast, they aren't free. They take disk, get vacuumed, get backed up, and make every write touch more files.

There are no good options here. Prune indexes and your queries get slower. Accumulate indexes and pay for it with an enlarged heap, while also vacuuming, packing, and copying many access paths.

The indexes that make a large table usable are part of what makes it too large to keep indexing.

Indexes on a partitioned table are created on the logical table but stored and updated on each partition. Those smaller indexes are faster to build and cheaper to REINDEX.

Vertical scaling buys you more space to store more indexes ... but that's like fixing traffic congestion by adding another lane to the freeway.

Sharding creates more, smaller databases, so having more indexes is less of a problem. You can afford to create many more indexes than if your large table was on a single cluster since a write only updates indexes on the shard that owns that row. Sharding is the good option for balancing a lot of data and many indexes.

Wide tables split across pages and files

A table is not only large because it has many rows; it might be that the rows are wide with many columns, or each row contains a large amount of data.

If a row exceeds ~2KB, oversized values go to TOAST, a separate table and indexes next to the heap you are already struggling to vacuum.

Each toasted value gets a 32-bit OID, about 4 billion per table. Updates of toasted values take a new OID. High-churn wide tables fill that space, and inserts slow down as Postgres hunts for a free one before the hard stop.

Whatever is left still has to fit into a single 8KB page, as Postgres will not split a row across pages. If a tuple cannot fit in the free space on any existing page, it gets a new page.

For a read-heavy table, you want pages packed tight (a high fillfactor), so scans walk as few pages as possible. For an update-heavy table, you want the opposite. Lower the fillfactor so an update can often stay on the same page instead of allocating a new one.

On a small table, the wrong fillfactor is a few wasted pages. On a large, wide table it could be terabytes of sparse pages, or a constant stream of new ones that vacuum, WAL, and backups all have to follow.

A large table is a heap, plus indexes, plus TOAST, and some operations still walk those files one at a time. pg_database_size() is one example. It stats every file in the database serially. On a large, wide table, that can pin a core.

Partitioning is a genuinely good solution for this class of large-table problems. Fillfactor can differ per-partition. Each partition gets its own TOAST table, so OID space is per slice.

However, with more files in one cluster, pg_database_size() can get worse.

Vertical scaling can hide a bad fillfactor for a while with extra disk space. Faster disks make TOAST vacuum and file stat()s cheaper. But you cannot buy greater limits than 8KB pages, the 2KB TOAST threshold, or the 4 billion OIDs.

Sharding creates individual Postgres instances, so TOAST OID space resets per cluster. A bad fillfactor hurts less because the heap is smaller. File walks are per shard.

Sharding does not, by itself, let hot and cold rows use different fillfactors. Partitioning still wins there.

Sharding is the solution to large tables

Just as partitioning breaks up a single table into many small pieces, sharding distributes the workload of your reads and writes across many database clusters.

Let's revisit the cascade delete story.

When replicas lagged, every read went to the primary. On a sharded database, each shard is its own cluster, with its own WAL and replicas. If the affected rows of A and B lived on one shard, that shard could still lag, and its reads pile onto its primary, but that could not fail the LSN check for the rest of the product. If those 100,000 deletes were spread across shards, the WAL burst would be split. No replica stream would have to replay the whole thing.

Your single-cluster large table likely processes a high throughput of traffic. Vertical scaling can help, but sharding does it better. With more clusters receiving and processing writes, a single table doesn't strain the entire cluster.

Additionally, cluster limits that cannot be solved by partitioning or vertical scaling are resolved. Postgres' hard-coded ceilings of 32-bit XIDs, TOAST OID and more become per-shard limits instead.

Sharded database backups are individually smaller and can be massively parallelized, making database restoration faster as well. Operations finish at the time of the slowest shard, not all at once waiting on large, bloated tables.

Cache eviction is a problem with large tables and their working sets. While more resources won't fit a large table into RAM, you often can per shard. Shard-local queries become shorter and faster.

Neki, sharded Postgres, lets your application write queries as if it were a single database, while splitting a large table into smaller, automatically distributed parts.

It contains the logic for where to send reads and writes. You still have to determine how that data is spread. That fan-out is covered in our post on data topologies.

Sharding shares the large table load across multiple clusters. If that's a problem you need solved, sign up for a PlanetScale account to use Neki today.

Most people have no idea what this means, but it could take down your entire database if you're not careful managing your connection pooler. An engineer's worst nightmare is waking up to a seemingly read-only database with no clear issue in sight.

Unfortunately, this is exactly what one of our discord friends ran into on a Tuesday evening.

At PlanetScale, we've helped many customers debug this issue across a variety of programming languages, ORMs, and app platforms. All of them boil down to complex interactions with connection pooling.

What is a connection pool

Connection pools are what let Postgres scale to thousands of simultaneous connections without overwhelming it with too many query-processing backends.

PgBouncer is the most common connection pooler for Postgres, and is what powers PlanetScale Postgres clusters. PgBouncer maintains multiple connections to the database, and multiplexes client connections over them. This is how 1,000 clients can connect to a Postgres instance while only using 20-50 direct connections under the hood.

What are poisoned connection pools

When PgBouncer runs in transaction mode, the most practical mode for the majority of apps and the default for PlanetScale clusters, each underlying connection can be reused across multiple clients. This is what allows them to use a lower number of direct connections through PgBouncer.

Each new client has an ability to set attributes throughout the lifecycle of its queries to the database. When a previous client leaves the pool in a undesired state, it can impact future client queries from working properly, leaving that pool "poisoned" with stale state.

Often, this leaked state will cause major disruptions for future queries. If a query set default_transaction_read_only = on or applied session characteristics such as SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLY, the underlying connection will be stuck with that state.

For one customer I was helping, the latter case caused their PgBouncer connections to get stuck as read-only sessions. In one route in their API, they had set SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLY for a read only transaction, unknowingly forcing the session into read only-mode permanently, even outside the transaction.

Every time another API request was run after that route was hit, PgBouncer reused the previous underlying connection that was still set to read-only, causing a flood of errors.

If you have ever seen Postgres error code 25006 or seen writes fail with the below error, you have probably poisoned your pool.

ERROR: cannot execute INSERT in a read-only transaction

pg_readonly: invalid statement because cluster is read-only

This error not only shows up from poisoned pools, but can also be returned from attempts to send write queries to a replica. The first thing to check if you think you have poisoned pools is to ensure the errors are not from a replica.

With PlanetScale Insights under the Errors tab, you can see a breakdown of every SQL error from the primary or replica.

The antidote to poisoned pools

The quick fix to restore a poisoned pool is to execute DISCARD ALL. DISCARD ALL resets the connection state to its defaults, removing default_transaction_read_only = on and any other session settings.

This fix needs to be applied to every connection the pooler holds, because it's hard to tell which individual connection(s) are at fault. This can be accomplished by creating a script that calls both in parallel using as many connections as possible, or using the pscale cli.

pscale branch connections top [database] [branch] lets you see every session currently running, allowing you to kill specific sessions you suspect may be poisoned.

The next step is fixing the application code that is causing the leaks in the first place.

Quick fix with PlanetScale's MCP

PlanetScale's MCP server can see query insights, errors, and even mint temporary connection strings to check for stuck session variables. Combining PlanetScale's MCP with the context of your codebase, an agent can quickly find where your codebase is accidentally modifying session state.

If you have a poisoned pool, set up the PlanetScale MCP server and instruct your agent to find and fix the suspected culprit with the following prompt:

Use the PlanetScale MCP server and this repository to find and fix connection pools stuck in read-only mode (25006 / cannot execute INSERT in a read-only transaction).
Search for session-level SET default_transaction_read_only, abandoned BEGIN transactions, and timeout paths that skip ROLLBACK.
Prefer transaction-scoped read-only with strict timeouts, or replica connections.
Reset with ROLLBACK or DISCARD ALL.

You can also point an agent at the PlanetScale documentation on read-only issues for more context.

Preventing read-only poisoned pools

To prevent read-only poisoned connection pools, the easiest solution is to send read traffic to a replica instead of enforcing read-only on primary transactions. When it's not possible to target a replica, ensure all transactions have strict timeouts and shouldn't set session variables such as default_transaction_read_only when connecting to the database through PgBouncer.

Refactoring a codebase can be a daunting task, but ORMs like Drizzle can make this easy with replica routing. Combined with the PlanetScale MCP, keeping your application safe from poisoned pools can be an afternoon of work instead of a month long refactor.

If you want to keep your connection pool healthy and your application online, install our MCP server and see if you're in danger of poisoned queries today.