Store the data of a tenant-aware entity in a separate search index per tenant by fractal3000 · Pull Request #5769 · jmix-framework/jmix

fractal3000 · 2026-10-05T08:41:18Z

An entity with a @TenantId attribute was mapped to one shared index. Documents of different tenants lived together, and nothing in the search path separated them: the add-on constrains database queries, not the search engine. Such an entity is now mapped to one index per tenant, search_index_<entityName>_<tenantId>. Isolation became a property of storage instead of a property of the query.

Everything else follows from that. An index name is computed rather than stored. A deletion carries the tenant, because after the fact there is nowhere to read it from. An administrative operation returns one row per affected index.

The commercial jmix-dynmodel module consumes the changed API and is updated separately, in jmix-framework/jmix-premium#323.

Upgrading

Index names change for an application where Search and Multitenancy already worked together. The former index stays on disk but is neither read nor written, so search over such entities returns nothing until a full reindex.

jmix.search.split-indexes-by-tenants=false keeps the former layout: the same index names, no splitting, and none of the new definition checks apply. The property defaults to true.

Breaking changes

Seven public types changed without a compatibility bridge. Fifteen methods differ only in return type, and Java will not hold two such signatures in one interface, so a default bridge is impossible here rather than merely undesirable. Collapsing the list back into a scalar would lie: a split entity has many indexes, and one flag over fifty of them either reports a false success or reports a failure without naming the index.

IndexManager — the four single-index methods are gone; the rest return List<IndexOperationResult<…>> instead of Map<IndexConfiguration, …> and take a tenant.
IndexingQueueManager — the four session operations return a list instead of boolean/void. A call that ignored the result still compiles.
IndexConfiguration — getIndexName() removed; ask IndexLayout.indexName(cfg, tenantId).
EntityIndexer — deleteCollectionByTargets added without a default.
IndexSearchRequestScope — the record gained an indexName component.
SearchProperties, IndexConfigurationManager — constructor parameters appended.

Behaviour changes: records of a tenant-aware entity with an empty tenant are no longer indexed; three new checks reject an index definition instead of accepting it (the application still starts, only that entity drops out); JMX operations print a Tenant column.

One break is silent. The indexing queue used to call EntityIndexer.deleteCollectionByEntityIds and now calls deleteCollectionByTargets. An application that subclassed an engine indexer and overrode the former still compiles, and its override no longer runs for queued deletions. The old method is deprecated so that the compiler warns on that @Override, which is the only signal that reaches such an application on its own.

Five deprecations, all since = "3.1", forRemoval = true: SearchProperties.searchIndexNamePrefix and its getter, EntityIndexer.deleteCollectionByEntityIds, IndexConfigurationManager.getIndexConfigurationByIndexName and its Opt form. All of them keep working.

Limitations

Records of a split entity that carry no tenant are not indexed and not searchable. They are what an administrator without a tenant creates. A separate index for them would either be visible to every tenant or to nobody.
Deleting a tenant removes neither its indexes nor its data nor its queue items: tenant deletion is reversible, index deletion is not. Removing them is an explicit operation.
Restoring a deleted tenant is not announced to the module, so its index stays unmarked until schemas are synchronized.
An entity whose @TenantId sits outside JPA is not split. The add-on fills the attribute from JpaDataStore alone, so such an entity has no tenants to be split by.
Availability markers live in one JVM and do not travel across a cluster. This predates the task; splitting made it more visible, because there are now many more indexes.

Checked

:search, :search-elasticsearch, :search-opensearch, :search-starter. A test application covers both engines end to end, including tenant isolation in a live index, the no-add-on path, dynamic attributes, a contributed definition, and the compatibility mode: 31 of 31 on OpenSearch, 30 of 31 on Elasticsearch (the one failure is a race inside the scenario itself, which passes when run alone).

(cherry picked from commit 78953206ec282c0a380f7725722ed2089a5c11a5)

…s the cluster (cherry picked from commit 9f36bb2e25544d2a2252ed85088bfd6c4198868c)

(cherry picked from commit c071f5465b7ed5f3afe47c223fbadfafa91a0ab5)

(cherry picked from commit 60900db99687eec2df4fe3e26bc121dd4fbe6c83)

…dex is missing (cherry picked from commit 7de6e4c711a9606ee9d4deaa840c7068e36a98c0)

…n use (cherry picked from commit 031d8ad2feb99807dddea733f31cc78e192aa6f3)

…ng queue (cherry picked from commit 5dd6fe1596139c2626d2acc52df7bdf86b628f7f)

(cherry picked from commit d654102941b18baeb23628b36b03b8793f4cf183)

… worded (cherry picked from commit 5ddb3d407fd3d4e4af78b98682b43f6d69d917b9)

…efinitions at startup (cherry picked from commit 56cb88bd1547284bd3bdd651d69727910a6faacc)

(cherry picked from commit 1526cbfc10bad01f3563ededa0b14060bf0d3aad)

(cherry picked from commit b908c92eabab2d46935c3822229c355982e85509)

(cherry picked from commit 77464c313e349e751299d08a983cc6ce1de9f37d)

(cherry picked from commit 0c0789fd42426c4c5c57b0cef187fb375d5936ae)

(cherry picked from commit a7d66e2a6b8fdda4c9f1b1ad2e04e3495410cd3c)

(cherry picked from commit 70bc3a773e74dec70efdc2f2f5b23fff76b3e2a1)

…formatting (cherry picked from commit 58c6e5e0362b939d263363531f9617c182414acb)

(cherry picked from commit 975cbdbc1b1faaf0912f5d48407f228da6b3dc66)

Gavrilov-Ivan

General notes & questions:

What will super-user find if he searches tenant-aware entity in multitenancy env?
Why can't we index data of super-user into separate index (for tenant-aware entity)? Any limitations?
Why exactly do we need tenant id in queue item? what will be if tenant id is not specified? or specified incorrectly?
We need to do something with lefover sessions of async enqueuing

Gavrilov-Ivan · 2026-10-29T08:59:34Z

+ */
+@Internal
+@Component("mten_TenantLoader")
+public class TenantLoader {


Need to retest.
Tenant condition is applied to Query as additional criteria and constraint\unconstraint DM doesn't matter.
Run it asSystem instead. And related tests are useless because you actually mocks the outcome.

Do we need so restricted fetch plans here instead of just local?

Do we need this class at all?

Gavrilov-Ivan · 2026-10-29T09:04:53Z

+ * tenant, and these indexes become available independently of each other.
 */
 @Component("search_IndexStateRegistry")
 public class IndexStateRegistry {


API methods changed their parameter from entityName to indexName.
But signatures are still the same.
Silent breaking change.

Looks like there should be new methods with new names with the indexName parameter.
Deprecate old methods if there is still possible to "redirect" call from entityName to indexName. Remove otherwise.

Gavrilov-Ivan · 2026-10-29T11:04:04Z

+ * Provides functionality for enqueuing entity instances and processing the queue.
 */
 @NullMarked
 public interface IndexingQueueManager {


BIg overgrowth of API due to massive overriding.
Looks like we should introduce some kind of scope-like object to pass it as an argument.
New methods have default behavior. Old methods redirect to the scoped ones.

Gavrilov-Ivan · 2026-10-29T11:08:33Z

+     * @param tenantId tenant the record belongs to, or {@code null} if it is unknown
+     * @return amount of enqueued instances
+     */
+    int enqueueIndexByEntityId(Id<?> entityId, @Nullable String tenantId);


Bad design.
What if provided tenantId doesn't match the actual tenant of the instance with the provided Id.
Confusing at least. Error-prone at most.

Gavrilov-Ivan · 2026-10-29T12:04:18Z

+     * @return tenant of the record, or null
+     */
+    @Nullable
+    String getTenant(Object entity);


Gavrilov-Ivan · 2026-11-02T13:30:42Z

+     * the record was saved outside this data store, its tenant attribute was not loaded, or it has none
+     */
+    @Nullable
+    protected String rememberedTenant(Id<?> entityId) {


Gavrilov-Ivan · 2026-11-02T13:30:53Z

+     * @return tenant of the record, or {@code null} if the entity has no tenant attribute or the value is not set
+     */
+    @Nullable
+    protected String tenantOfDeletedRecord(MetaClass metaClass, AttributeChanges changes) {


Gavrilov-Ivan · 2026-11-02T13:34:00Z

+                .collect(Collectors.toSet());
+    }
+
+    protected void assertReady() {


Gavrilov-Ivan · 2026-11-02T13:36:48Z

     * for search using the provided {@link VirtualSubfieldsProvider}.
     *
-     * @param entities list of entity names to evaluate for search scope. If empty, all indexed entities are considered.
+     * @param entities                 list of entity names to evaluate for search scope. If empty, all indexed entities are considered.


Gavrilov-Ivan · 2026-11-02T13:39:02Z

    </changeSet>
-</databaseChangeLog>
+
+    <changeSet author="search" id="4">


Separate file

Gavrilov-Ivan

General notes & questions:

What will super-user find if he searches tenant-aware entity in multitenancy env?
Why can't we index data of super-user into separate index (for tenant-aware entity)? Any limitations?
Why exactly do we need tenant id in queue item? what will be if tenant id is not specified? or specified incorrectly?
We need to do something with lefover sessions of async enqueuing

Pavel Aleksandrov added 20 commits October 5, 2026 11:54

Find the tenant attribute of an entity through MetadataTools

a83daad

(cherry picked from commit 78953206ec282c0a380f7725722ed2089a5c11a5)

Expose a facade for reading tenants and announce tenant changes acros…

2df6461

…s the cluster (cherry picked from commit 9f36bb2e25544d2a2252ed85088bfd6c4198868c)

Store the data of a tenant-aware entity in a separate index per tenant

3a26977

(cherry picked from commit c071f5465b7ed5f3afe47c223fbadfafa91a0ab5)

Cover tenant-separated indexes

d3c2c6b

(cherry picked from commit 60900db99687eec2df4fe3e26bc121dd4fbe6c83)

Keep the indexed set consistent when definitions are rebuilt or an in…

aec8cf8

…dex is missing (cherry picked from commit 7de6e4c711a9606ee9d4deaa840c7068e36a98c0)

Publish a freshly built index registry instead of refilling the one i…

23ad4e7

…n use (cherry picked from commit 031d8ad2feb99807dddea733f31cc78e192aa6f3)

Name the tenant when a change of dynamic attributes enters the indexi…

868f97f

…ng queue (cherry picked from commit 5dd6fe1596139c2626d2acc52df7bdf86b628f7f)

Shorten the javadoc of the bulk enqueueing lock

0e78c22

(cherry picked from commit d654102941b18baeb23628b36b03b8793f4cf183)

Word the scoped enqueue result as the operations of the same name are…

e126a3a

… worded (cherry picked from commit 5ddb3d407fd3d4e4af78b98682b43f6d69d917b9)

Let the indexing locks fill themselves instead of reading the index d…

760ec37

…efinitions at startup (cherry picked from commit 56cb88bd1547284bd3bdd651d69727910a6faacc)

Let the rejected definition name itself once in the message

719bac5

(cherry picked from commit 1526cbfc10bad01f3563ededa0b14060bf0d3aad)

Let the queue wait out the items of a tenant that is gone

dffc8ee

Answer in one place whether an entity is split by tenants

91bbe5d

(cherry picked from commit b908c92eabab2d46935c3822229c355982e85509)

Let the session of an entity that left the indexed set be removed

8c4dbb9

Deprecate deleting by ids alone, which cannot name the tenant

b45049d

(cherry picked from commit 77464c313e349e751299d08a983cc6ce1de9f37d)

Stop threading a name generator the index managers never read

6c49dc3

(cherry picked from commit 0c0789fd42426c4c5c57b0cef187fb375d5936ae)

Deprecate the lookup by index name and ask the tenants once

b3fe657

(cherry picked from commit a7d66e2a6b8fdda4c9f1b1ad2e04e3495410cd3c)

Let the disabled queue answer an operation that names no entity

00401b2

(cherry picked from commit 70bc3a773e74dec70efdc2f2f5b23fff76b3e2a1)

Clean up leftovers of the branch: unused imports, dead suppressions, …

4f69812

…formatting (cherry picked from commit 58c6e5e0362b939d263363531f9617c182414acb)

Keep the queued work of an index that was not deleted

82b1ef2

(cherry picked from commit 975cbdbc1b1faaf0912f5d48407f228da6b3dc66)

$@fractal3000$

fractal3000 requested a review from Gavrilov-Ivan October 5, 2026 08:44

Gavrilov-Ivan mentioned this pull request Oct 29, 2026

Indexing queue processing fairness improvement #5924

Open

Gavrilov-Ivan requested changes Nov 2, 2026

View reviewed changes

Navigation Menu

Repository navigation

Store the data of a tenant-aware entity in a separate search index per tenant - #5769

Store the data of a tenant-aware entity in a separate search index per tenant#5769
fractal3000 wants to merge 20 commits into
masterfrom
feature/4191-tenant-separated-search-indexes-squashed

$@fractal3000$ fractal3000 commented Oct 5, 2026 •

edited

Loading

Uh oh!

Gavrilov-Ivan left a comment

Uh oh!

Gavrilov-Ivan Oct 29, 2026

Uh oh!

Gavrilov-Ivan Oct 29, 2026

Uh oh!

Gavrilov-Ivan Oct 29, 2026

Uh oh!

Gavrilov-Ivan Oct 29, 2026

Uh oh!

Gavrilov-Ivan Oct 29, 2026

Uh oh!

Gavrilov-Ivan Nov 2, 2026

Uh oh!

Gavrilov-Ivan Nov 2, 2026

Uh oh!

Gavrilov-Ivan Nov 2, 2026

Uh oh!

Gavrilov-Ivan Nov 2, 2026

Uh oh!

Gavrilov-Ivan Nov 2, 2026

Uh oh!

Gavrilov-Ivan left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

fractal3000 commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gavrilov-Ivan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gavrilov-Ivan left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

$@fractal3000$ fractal3000 commented Oct 5, 2026 •

edited

Loading