Skip to content

Feat/opensearch migration#2927

Open
Dreamstick9 wants to merge 3 commits into
cartography-cncf:masterfrom
Dreamstick9:feat/opensearch-migration
Open

Feat/opensearch migration#2927
Dreamstick9 wants to merge 3 commits into
cartography-cncf:masterfrom
Dreamstick9:feat/opensearch-migration

Conversation

@Dreamstick9

Copy link
Copy Markdown

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring (no functional changes)
  • Documentation update
  • Other (please describe):

Summary

Switches the AWS OpenSearch/Elasticsearch intel module from the deprecated es boto3 client to the opensearch client.

What changed:

  • create_boto3_client now uses "opensearch" instead of "es"
  • client.describe_elasticsearch_domains() replaced with client.describe_domains() (equivalent OpenSearch API)
  • _transform_es_domains now reads EngineVersion and ClusterConfig (OpenSearch API field names) with fallbacks to ElasticsearchVersion and ElasticsearchClusterConfig for backward compatibility with any data returned by the legacy es client
  • Log messages and docstrings updated to reflect the OpenSearch branding

Why: AWS renamed Elasticsearch Service to OpenSearch Service and the es boto3 client is deprecated in favor of the opensearch client. No changes to the graph schema — the ESDomain node and all its properties and relationships are unchanged.

Related issues or links

  • Fixes #

How was this tested?

  • Updated integration tests in tests/integration/cartography/intel/aws/test_elasticsearch.py
  • Test fixture data in tests/data/aws/elasticsearch.py updated to use the real opensearch client field names (EngineVersion, ClusterConfig) so the tests validate the actual code path rather than the legacy field names

Checklist

General

  • I have read the contributing guidelines.
  • The linter passes locally (make test_lint).
  • I have added/updated tests that prove my fix is effective or my feature works.

Proof of functionality

  • New or updated unit/integration tests.

If you are adding or modifying a synced entity

  • Included Cartography sync logs from a real environment demonstrating successful synchronization of the new/modified entity.

If you are changing a node or relationship

N/A — no node or relationship schema changes.

If you are implementing a new intel module

N/A

Notes for reviewers

The _transform_es_domains function retains fallback reads for the legacy ElasticsearchVersion and ElasticsearchClusterConfig field names. This is intentional: it keeps the transform function safe if it ever receives data from the old es client path (e.g. in tests or future callers), and documents the API difference inline. All other field names (EBSOptions, EncryptionAtRestOptions, LogPublishingOptions, VPCOptions, DomainId, ARN, etc.) are identical between the two clients and required no changes.

    - Switch boto3 client from 'es' to 'opensearch'
    - Use describe_domains() instead of deprecated describe_elasticsearch_domains()
    - Update test fixtures with OpenSearch versions and instance types

Signed-off-by: Kushagar Garg <dreamstick909@gmail.com>
    - Use EngineVersion || ElasticsearchVersion for version field
    - Use ClusterConfig || ElasticsearchClusterConfig for cluster config
    - Update test fixtures to use real opensearch client field names
    - Maintains backwards compatibility with es client responses

Signed-off-by: Kushagar Garg <dreamstick909@gmail.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant