Add manual test for bulk import functionality

[eleonoramicozzi]

Summary

Closes #224911

Add test for bulk import functionality as a follow up to #223501

This test should only be run locally, not on CI.

For this reason, it has been added to a separate config for stateful and serverless, that is disabled in the ftr config files.

In order to run this test locally, you can run:

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/<stateful/serverless>/oblt.ai_assistant_local.<stateful/serverless>.config.ts

and then

node scripts/functional_test_runner --config=x-pack/test/api_integration/deployment_agnostic/configs/<stateful/serverless>/oblt.ai_assistant_local.<stateful/serverless>.config.ts

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Unit or functional tests were updated or added to match the most common scenarios
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

[elasticmachine]

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

[sorenlouv]

I can't recall if this.tags(['skipServerless', 'skipStateful']); skips in all CI envs but I like the idea of running this locally. We might decide to skip it locally as well if it's too slow.

[eleonoramicozzi]

I can't recall if this.tags(['skipServerless', 'skipStateful']); skips in all CI envs but I like the idea of running this locally. We might decide to skip it locally as well if it's too slow.

Looking at https://docs.elastic.dev/appex-qa/ftr/knowledge-base/ftr-users/skip-tests#:~:text=skipStateful,local%20and%20MKI).
I believe that skipServerless and skipStateful will skip all CI and local. So I think it has the same effect as describe.skip basically.

We could use skipCloud instead which I think will still run it on local, but I'm not sure about what would happen with the PR CI.

@sorenlouv do you know who I could ask to make sure?

[sorenlouv]

I can't recall if this.tags(['skipServerless', 'skipStateful']); skips in all CI envs but I like the idea of running this locally. We might decide to skip it locally as well if it's too slow.

Looking at https://docs.elastic.dev/appex-qa/ftr/knowledge-base/ftr-users/skip-tests#:~:text=skipStateful,local%20and%20MKI). I believe that skipServerless and skipStateful will skip all CI and local. So I think it has the same effect as describe.skip basically.

We could use skipCloud instead which I think will still run it on local, but I'm not sure about what would happen with the PR CI.

@sorenlouv do you know who I could ask to make sure?

Is it correct, that what you want is to skip it everywhere (CI, MKI, ECH) but run locally? I think @pheyos will know if that's possible (he'll probably ask why we want this uncommon behaviour :D)

[pheyos]

For tests that are not meant to run in any of our regular CI runs (local or cloud), we usually create a separate config file and list that under the disabled section in the config manifests (here for oblt stateful, here for oblt serverless - actually similar to the Cypress entries that in there already). Leaving corresponding comments in the config and/or test file will help to avoid misunderstandings. That way, you could specifically trigger these particular tests without grep or something and the structure will be very clear.

Technically, you could also use a combination of skip tags to achieve the same, but I would only recommend this approach if you think the complete CI skip is a temporary thing and you plan to change these labels soon.

[pheyos]

why we want this uncommon behaviour

I would indeed be curious to learn about your use case here. However, it sounds like you actually might run these tests locally and only skip cloud? FWIW, we have a number of test configs that are triggered only from separate pipeline or in certain workflows, so it's not a super rare thing to do.

[eleonoramicozzi]

@pheyos I've updated the PR following your recommendations, thank you!

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 2df88fe

Failed CI Steps

• FTR Configs #9

Test Failures

• [job] [logs] FTR Configs #9 / ESQL execution logic API @ess @serverless ES|QL rule type shard failures should handle shard failures and include warning in logs for query that is not aggregating

Metrics [docs]

✅ unchanged

History

• 💔 Build #313980 failed edf2566
• 💛 Build #313614 was flaky 60981d0
• 💔 Build #313466 failed be90023
• 💚 Build #313169 succeeded 7171185

cc @eleonoramicozzi

[pheyos]

Setup for local-only / non-CI tests LGTM