[Obs AI Assistant] Remove semantic_text migration #220886

sorenlouv · 2025-05-18T19:05:10Z

Background

The semantic text migration was added in 8.17 along with the move to semantic_text field (#186499).

Suggestion

We should remove the semantic_text migration starting in 9.1. Reasons:

The migration has recently caused severe disruption on a number of clusters ([Obs AI Assistant] Use update-by-query for semantic_text migration #220255).
The migration may no longer be needed. Users are required to upgrade Elasticsearch to 8.18 in order to update to 9.0, or 8.19 to upgrade to 9.1. This could mean that users are guaranteed to have run the migration previously, when upgrading Kibana to 9.1 (needs to be double checked with Kibana folks)

…-text-migration

elasticmachine · 2025-05-22T14:23:41Z

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

github-actions · 2025-05-22T14:23:47Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

x-pack/platform/plugins/shared/observability_ai_assistant/server/service/inference_endpoint.ts

…-text-migration

sorenlouv · 2025-05-27T06:43:14Z

...ic/apis/observability/ai_assistant/distributed_lock_manager/distributed_lock_manager.spec.ts

      const LOCK_ID = 'my_lock_with_lock';

      describe('Successful execution and concurrent calls', () => {
-        let executions: number;


Not related but this test was flaky (failed 1 out of 100 runs). The changes here should improve it - still waiting for flaky test runner to complete 🤞

kibanamachine · 2025-05-27T09:37:54Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#8298

[✅] x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.ai_assistant.serverless.config.ts: 100/100 tests passed.
[❌] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 74/100 tests passed.

see run history

...stic/apis/observability/ai_assistant/knowledge_base/knowledge_base_8.16_upgrade_test.spec.ts

x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts

…er/service/client/index.ts Co-authored-by: Viduni Wickramarachchi <[email protected]>

…-fix'

…-text-migration

elasticmachine · 2025-05-28T05:44:37Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 369cbf3
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-220886-369cbf329a6b

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #28 / Actions and Triggers app Stack alerts page Loads the page with actions but not alerting privilege Loads the page but shows missing permission prompt

Metrics [docs]

✅ unchanged

History

💚 Build #302878 succeeded 78e10f0
💚 Build #302857 succeeded b33b260
💛 Build #302794 was flaky ff5f770
💛 Build #302526 was flaky b04b5e1
💚 Build #302510 succeeded 11610fb

kibanamachine · 2025-05-28T13:43:46Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#8305

[✅] x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.ai_assistant.serverless.config.ts: 100/100 tests passed.
[✅] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.ai_assistant.stateful.config.ts: 100/100 tests passed.

see run history

## Background The semantic text migration was added in 8.17 along with the move to semantic_text field (elastic#186499). ## Suggestion We should remove the semantic_text migration starting in 9.1. Reasons: - The migration has recently caused severe disruption on a number of clusters (elastic#220255). - The migration may no longer be needed. Users are required to upgrade Elasticsearch to 8.18 in order to update to 9.0, or 8.19 to upgrade to 9.1. This could mean that users are guaranteed to have run the migration previously, when upgrading Kibana to 9.1 (needs to be double checked with Kibana folks) --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> Co-authored-by: kibanamachine <[email protected]>

kibanamachine · 2025-05-30T06:48:48Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 220886 locally
cc: @sorenlouv

## Background The semantic text migration was added in 8.17 along with the move to semantic_text field (elastic#186499). ## Suggestion We should remove the semantic_text migration starting in 9.1. Reasons: - The migration has recently caused severe disruption on a number of clusters (elastic#220255). - The migration may no longer be needed. Users are required to upgrade Elasticsearch to 8.18 in order to update to 9.0, or 8.19 to upgrade to 9.1. This could mean that users are guaranteed to have run the migration previously, when upgrading Kibana to 9.1 (needs to be double checked with Kibana folks) --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> Co-authored-by: kibanamachine <[email protected]>

[Obs AI Assistant] Remove semantic_text migration

b0c5338

sorenlouv added backport:version Backport to applied version labels v9.1.0 release_note:enhancement labels May 18, 2025

Merge branch 'main' of github.com:elastic/kibana into remove-semantic…

71138ae

…-text-migration

sorenlouv marked this pull request as ready for review May 22, 2025 14:17

sorenlouv requested a review from a team as a code owner May 22, 2025 14:17

Fix typo

d1ed1fa

botelastic bot added ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant labels May 22, 2025

Remove unused test from index file

1bc6ea0

viduni94 reviewed May 22, 2025

View reviewed changes

x-pack/platform/plugins/shared/observability_ai_assistant/server/service/inference_endpoint.ts Outdated Show resolved Hide resolved

Fix jest test

84b5252

sorenlouv requested a review from a team as a code owner May 22, 2025 21:20

sorenlouv added 3 commits May 23, 2025 11:30

Fix logging

0687715

Fix lint issues

d54f4ad

Fix KB tests

cdbc78c

sorenlouv enabled auto-merge (squash) May 23, 2025 21:39

sorenlouv disabled auto-merge May 23, 2025 21:39

sorenlouv added 7 commits May 26, 2025 14:17

Merge branch 'main' of github.com:elastic/kibana into remove-semantic…

715b988

…-text-migration

Add inference_id arg

3d945c8

Improve test utils

401f7d4

Undo change to util

a383ac3

Merge branch 'main' into remove-semantic-text-migration

a31612b

Re-add sleep

11610fb

Improve LockManager test

b04b5e1

sorenlouv commented May 27, 2025

View reviewed changes

neptunian reviewed May 27, 2025

View reviewed changes

...stic/apis/observability/ai_assistant/knowledge_base/knowledge_base_8.16_upgrade_test.spec.ts Show resolved Hide resolved

neptunian approved these changes May 27, 2025

View reviewed changes

viduni94 approved these changes May 27, 2025

View reviewed changes

x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts Outdated Show resolved Hide resolved

x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts Outdated Show resolved Hide resolved

sphilipse approved these changes May 27, 2025

View reviewed changes

sorenlouv and others added 4 commits May 27, 2025 18:20

Update x-pack/platform/plugins/shared/observability_ai_assistant/serv…

a19a4db

…er/service/client/index.ts Co-authored-by: Viduni Wickramarachchi <[email protected]>

Update x-pack/platform/plugins/shared/observability_ai_assistant/serv…

c9ec67e

…er/service/client/index.ts Co-authored-by: Viduni Wickramarachchi <[email protected]>

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

ff5f770

…-fix'

Merge branch 'main' of github.com:elastic/kibana into remove-semantic…

b33b260

…-text-migration

sorenlouv enabled auto-merge (squash) May 27, 2025 22:04

sorenlouv added 2 commits May 28, 2025 00:11

Fix comments

78e10f0

Merge branch 'main' into remove-semantic-text-migration

369cbf3

sorenlouv merged commit b4767bb into elastic:main May 28, 2025
11 checks passed

sorenlouv deleted the remove-semantic-text-migration branch May 28, 2025 11:27

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 30, 2025

sorenlouv added the backport:skip This PR does not require backporting label May 30, 2025

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 30, 2025

sorenlouv added backport missing Added to PRs automatically when the are determined to be missing a backport. and removed backport:version Backport to applied version labels labels May 30, 2025

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Obs AI Assistant] Remove semantic_text migration #220886

[Obs AI Assistant] Remove semantic_text migration #220886

Uh oh!

sorenlouv commented May 18, 2025 •

edited

Loading

Uh oh!

elasticmachine commented May 22, 2025

Uh oh!

github-actions bot commented May 22, 2025

Uh oh!

Uh oh!

sorenlouv May 27, 2025 •

edited

Loading

Uh oh!

kibanamachine commented May 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented May 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

kibanamachine commented May 28, 2025

Uh oh!

kibanamachine commented May 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[Obs AI Assistant] Remove semantic_text migration #220886

[Obs AI Assistant] Remove semantic_text migration #220886

Uh oh!

Conversation

sorenlouv commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Suggestion

Uh oh!

elasticmachine commented May 22, 2025

Uh oh!

github-actions bot commented May 22, 2025

🤖 GitHub comments

Uh oh!

Uh oh!

sorenlouv May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented May 27, 2025

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#8298

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

Uh oh!

Uh oh!

kibanamachine commented May 28, 2025

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#8305

Uh oh!

kibanamachine commented May 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sorenlouv commented May 18, 2025 •

edited

Loading

sorenlouv May 27, 2025 •

edited

Loading

elasticmachine commented May 28, 2025 •

edited

Loading