[Obs AI Assistant][Anonymization] Get model ID from anonymization rules #224280
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
viduni94
commented
Jun 17, 2025
Comment on lines
128
to
121
| defaultMessage: `List of anonymization rules. | ||
| <ul> | ||
| <li><strong>id:</strong> unique string identifier</li> | ||
| <li><strong>type:</strong> "ner" or "regex"</li> | ||
| <li><strong>entityClass:</strong> PER, ORG, LOC</li> | ||
| <li><strong>pattern:</strong> (regex rules only) the regular-expression string to match</li> | ||
| <li><strong>enabled:</strong> boolean flag to turn the rule on or off</li> | ||
| <li><strong>description:</strong> optional human-readable description</li> | ||
| <li><strong>modelId:</strong> ("ner" rules only) id of the trained model to use</li> | ||
| </ul>`, | ||
| values: { | ||
| ul: (chunks) => `<ul>${chunks}</ul>`, | ||
| li: (chunks) => `<li>${chunks}</li>`, | ||
| strong: (chunks) => `<strong>${chunks}</strong>`, | ||
| }, |
Contributor
Author
There was a problem hiding this comment.
Updated this to have better formatting.
Previously it looked like:
After the above update:
95a5693 to
ad67ccd
Compare
…on/types.ts Co-authored-by: Arturo Lidueña <[email protected]>
neptunian
reviewed
Jun 18, 2025
x-pack/platform/plugins/private/observability_ai_assistant_management/common/ui_settings.ts
Outdated
Show resolved
Hide resolved
neptunian
reviewed
Jun 18, 2025
| const entities = await this.detectNamedEntitiesForModel({ modelId, chunks }); | ||
|
|
||
| for (const entity of entities) { | ||
| // if an earlier rule already produced this entity class, skip |
Contributor
There was a problem hiding this comment.
we shouldn't skip if the entity class already exists. another model may have found another entity of the same class that the prior one didn't. instead of using a map, we can modify the content in place with the hash so the next model never sees the already detected entity, but can still find more entities of the same class.
neptunian
reviewed
Jun 18, 2025
x-pack/platform/plugins/private/observability_ai_assistant_management/common/ui_settings.ts
Outdated
Show resolved
Hide resolved
…ting already identified entities
Contributor
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
Contributor
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
Contributor
💚 Build Succeeded
Metrics [docs]Page load bundle
Historycc @viduni94 |
neptunian
approved these changes
Jun 18, 2025
Contributor
|
Starting backport for target branches: 8.19 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jun 19, 2025
…es (elastic#224280) Closes elastic/observability-dev#4564 ## Summary This PR includes the follows: - Updates to the description of the Anonymization advance setting - Allow specifying `modelId` per NER rule - Use `modelId` from NER rule if available, if not, fallback to default model ID - If multiple NER rules are defined, allow entity recognition via all rules. If the first rule detects an entity, it's hashed, so that the next rule won't have to detect it again. Extra entities detected by subsequent rules are combined. ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Sandra Gonzales <[email protected]> Co-authored-by: Sandra G <[email protected]> Co-authored-by: Arturo Lidueña <[email protected]> (cherry picked from commit bd9fbc0)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jun 19, 2025
…ion rules (#224280) (#224612) # Backport This will backport the following commits from `main` to `8.19`: - [[Obs AI Assistant][Anonymization] Get model ID from anonymization rules (#224280)](#224280) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Viduni Wickramarachchi","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-06-19T17:10:49Z","message":"[Obs AI Assistant][Anonymization] Get model ID from anonymization rules (#224280)\n\nCloses https://github.com/elastic/observability-dev/issues/4564\n\n## Summary\n\nThis PR includes the follows:\n- Updates to the description of the Anonymization advance setting\n- Allow specifying `modelId` per NER rule\n- Use `modelId` from NER rule if available, if not, fallback to default\nmodel ID\n- If multiple NER rules are defined, allow entity recognition via all\nrules. If the first rule detects an entity, it's hashed, so that the\nnext rule won't have to detect it again. Extra entities detected by\nsubsequent rules are combined.\n\n\n### Checklist\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: Sandra Gonzales <[email protected]>\nCo-authored-by: Sandra G <[email protected]>\nCo-authored-by: Arturo Lidueña <[email protected]>","sha":"bd9fbc0b94f1fbca3e224964eb34d190f703e0ab","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","Team:Obs AI Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Obs AI Assistant][Anonymization] Get model ID from anonymization rules","number":224280,"url":"https://github.com/elastic/kibana/pull/224280","mergeCommit":{"message":"[Obs AI Assistant][Anonymization] Get model ID from anonymization rules (#224280)\n\nCloses https://github.com/elastic/observability-dev/issues/4564\n\n## Summary\n\nThis PR includes the follows:\n- Updates to the description of the Anonymization advance setting\n- Allow specifying `modelId` per NER rule\n- Use `modelId` from NER rule if available, if not, fallback to default\nmodel ID\n- If multiple NER rules are defined, allow entity recognition via all\nrules. If the first rule detects an entity, it's hashed, so that the\nnext rule won't have to detect it again. Extra entities detected by\nsubsequent rules are combined.\n\n\n### Checklist\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: Sandra Gonzales <[email protected]>\nCo-authored-by: Sandra G <[email protected]>\nCo-authored-by: Arturo Lidueña <[email protected]>","sha":"bd9fbc0b94f1fbca3e224964eb34d190f703e0ab"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224280","number":224280,"mergeCommit":{"message":"[Obs AI Assistant][Anonymization] Get model ID from anonymization rules (#224280)\n\nCloses https://github.com/elastic/observability-dev/issues/4564\n\n## Summary\n\nThis PR includes the follows:\n- Updates to the description of the Anonymization advance setting\n- Allow specifying `modelId` per NER rule\n- Use `modelId` from NER rule if available, if not, fallback to default\nmodel ID\n- If multiple NER rules are defined, allow entity recognition via all\nrules. If the first rule detects an entity, it's hashed, so that the\nnext rule won't have to detect it again. Extra entities detected by\nsubsequent rules are combined.\n\n\n### Checklist\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: Sandra Gonzales <[email protected]>\nCo-authored-by: Sandra G <[email protected]>\nCo-authored-by: Arturo Lidueña <[email protected]>","sha":"bd9fbc0b94f1fbca3e224964eb34d190f703e0ab"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Viduni Wickramarachchi <[email protected]> Co-authored-by: Sandra Gonzales <[email protected]> Co-authored-by: Sandra G <[email protected]> Co-authored-by: Arturo Lidueña <[email protected]>
akowalska622
pushed a commit
to akowalska622/kibana
that referenced
this pull request
Jun 25, 2025
…es (elastic#224280) Closes elastic/observability-dev#4564 ## Summary This PR includes the follows: - Updates to the description of the Anonymization advance setting - Allow specifying `modelId` per NER rule - Use `modelId` from NER rule if available, if not, fallback to default model ID - If multiple NER rules are defined, allow entity recognition via all rules. If the first rule detects an entity, it's hashed, so that the next rule won't have to detect it again. Extra entities detected by subsequent rules are combined. ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Sandra Gonzales <[email protected]> Co-authored-by: Sandra G <[email protected]> Co-authored-by: Arturo Lidueña <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes https://github.com/elastic/observability-dev/issues/4564
Summary
This PR includes the follows:
modelIdper NER rulemodelIdfrom NER rule if available, if not, fallback to default model IDChecklist
release_note:*label is applied per the guidelines