Add windows event id filtering to CWAgent #1737

Paamicky · 2025-06-18T21:16:17Z

Description of the issue

The CloudWatch Agent currently filters Windows Event Logs based on security levels (i.e Error, Critical, Information...). This does not allow customers to only collect specific and relevant logs to CloudWatch

Description of changes

Add Event ID filtering to the CWAgent Windows logs collection
Add rule for eventID validation
Add new files and moved some codes to new files to testing purposes

sample configuration of event ID filtering

  "logs": {
    "logs_collected": {
      "windows_events": {
        "collect_list": [
          {
            "event_name": "System",
            "event_ids": [4624, 4625, 4634],
            "log_group_name": "WindowsEventIds-Test",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }
}

Caution

This PR has a testing PR linked to here: aws/amazon-cloudwatch-agent-test#541, merge this first.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Added unit tests for event ID validation and processing
Manually tested on Windows to verify events are properly filtered by ID
Verified backward compatibility with existing configurations

Requirements

Before commiting your code, please do the following steps.

Run make fmt and make fmt-sh
Run make lint

Integration Tests

To run integration tests against this PR, add the ready for testing label.

plugins/inputs/windows_event_log/windows_event_log.go

plugins/inputs/windows_event_log/wineventlog/wineventlog.go

plugins/inputs/windows_event_log/wineventlog/wineventlog_test.go

translator/tocwconfig/sampleConfig/windows_eventlog_only_config.conf

translator/tocwconfig/tocwconfig_test.go

translator/translate/logs/logs_collected/windows_events/collect_list/collectlist.go

translator/translate/logs/logs_collected/windows_events/collect_list/collectlist_test.go

cmd/config-translator/translator_test.go

plugins/inputs/windows_event_log/windows_event_log.go

plugins/inputs/windows_event_log/wineventlog/utils.go

okankoAMZ · 2025-06-23T20:22:16Z

plugins/inputs/windows_event_log/wineventlog/utils.go

-	return utf16ToUTF8Bytes(renderBuf, bufferUsed)
-}
-
-func CreateBookmark(channel string, recordID uint64) (h EvtHandle, err error) {


why are we deleting this function can you explain in your PR description?

Moved to a different file due to the windows build tags.

plugins/inputs/windows_event_log/wineventlog/utils.go

okankoAMZ · 2025-06-23T20:23:01Z

plugins/inputs/windows_event_log/wineventlog/utils_windows.go

+
+var NumberOfBytesPerCharacter = UnknownBytesPerCharacter
+
+func RenderEventXML(eventHandle EvtHandle, renderBuf []byte) ([]byte, error) {


Okay so you are moving into a different file. Please document that in your description

Alright sure

translator/tocwconfig/sampleConfig/windows_eventlog_only_config.conf

okankoAMZ · 2025-06-23T20:25:33Z

translator/tocwconfig/totomlconfig/toTomlConfig.go

 	if err != nil {
 		log.Panicf("Encode to a valid TOML config fails because of %v", err)
 	}
 	return buf.String()
 }
+
+// ensure integers in arrays are preserved
+func processValue(val interface{}) interface{} {


shouldn't we have some kind of validation and returning error here. With the current implementation it seems like this function could never fail, is this true?

yeah used a different approach to integer verification, reverted to original main code.

translator/translate/logs/logs_collected/windows_events/collect_list/collectlist.go

translator/translate/logs/logs_collected/windows_events/windows_event_test.go

dricross · 2025-06-23T20:39:32Z

translator/tocwconfig/totomlconfig/toTomlConfig.go

+	// Process value to ensure integers in arrays are preserved
+	processedVal := processValue(val)


Interesting. There are some other integers that aren't mangled like log retention. I wonder why those work. Perhaps because they aren't in an array? We should understand why those integers don't require additional processing.

translator/tocwconfig/sampleConfig/windows_eventlog_only_config.conf

translator/translate/logs/logs_collected/windows_events/collect_list/collectlist.go

translator/isValid.go

translator/defaultKeyCase.go

translator/tocwconfig/sampleConfig/windows_eventlog_only_config.conf

translator/translate/logs/logs_collected/windows_events/collect_list/collectlist.go

translator/translate/logs/logs_collected/windows_events/collect_list/ruleEventIDs.go

translator/translate/logs/logs_collected/windows_events/collect_list/collectlist.go

translator/translate/logs/logs_collected/windows_events/collect_list/ruleEventIDs.go

lisguo · 2025-07-10T23:05:39Z

plugins/inputs/windows_event_log/wineventlog/utils.go

@@ -1,29 +1,18 @@
 // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 // SPDX-License-Identifier: MIT

-//go:build windows


why was this removed?

I removed for unit testing purposes. I do major testing on the windows platform but decided to create to test out my query.

plugins/inputs/windows_event_log/wineventlog/utils.go

lisguo · 2025-07-10T23:07:39Z

plugins/inputs/windows_event_log/wineventlog/utils.go

-	ind, err := strconv.Atoi(rawMessage[prevIndex+1:])
-	if searchingIndex && err == nil && ind <= len(evtDataValues) && ind > 0 {
-		sb.WriteString(evtDataValues[ind-1].Value)
+	//Ignore events older than 2 weeks


whats the reasoning for having a cut-off of 2 weeks?

This is an already existing agent behavior. I assume they wouldn't want the agent to collect and process older logs.

lisguo · 2025-07-10T23:08:53Z

plugins/inputs/windows_event_log/wineventlog/utils.go

-	if searchingIndex && err == nil && ind <= len(evtDataValues) && ind > 0 {
-		sb.WriteString(evtDataValues[ind-1].Value)
+	//Ignore events older than 2 weeks
+	cutOffPeriod := (time.Hour * 24 * 14).Nanoseconds()


can we make this a const?

Paamicky and others added 7 commits June 6, 2025 13:23

Add event ID filtering for windows logs to CWAgent

cd878d0

mend

f6d5cdf

mend

119c7d7

mend

de9a111

mend

037fe7f

update just for integ test

5dd88d6

Merge branch 'main' into feature/windows-eventId-filtering

144fac4

Paamicky force-pushed the feature/windows-eventId-filtering branch 3 times, most recently from b758b9e to 516df4e Compare June 20, 2025 16:09

fix: resolve workflow lint issues

2276ab2

Paamicky force-pushed the feature/windows-eventId-filtering branch from 516df4e to 2276ab2 Compare June 20, 2025 17:42

Paamicky marked this pull request as ready for review June 23, 2025 01:09

Paamicky requested a review from a team as a code owner June 23, 2025 01:09

Paamicky requested review from jefchien and okankoAMZ June 23, 2025 01:11

dricross reviewed Jun 23, 2025

View reviewed changes

Paamicky force-pushed the feature/windows-eventId-filtering branch from 9d99660 to 2667175 Compare June 23, 2025 19:27

okankoAMZ reviewed Jun 23, 2025

View reviewed changes

dricross reviewed Jun 23, 2025

View reviewed changes

Paamicky force-pushed the feature/windows-eventId-filtering branch 5 times, most recently from ffba86e to a4bbe23 Compare June 24, 2025 20:47

dricross reviewed Jun 24, 2025

View reviewed changes

translator/isValid.go Outdated Show resolved Hide resolved

dricross reviewed Jun 24, 2025

View reviewed changes

Paamicky force-pushed the feature/windows-eventId-filtering branch 2 times, most recently from 097c665 to dc6df12 Compare June 25, 2025 17:32

dricross reviewed Jun 25, 2025

View reviewed changes

translator/translate/logs/logs_collected/windows_events/collect_list/ruleEventIDs.go Show resolved Hide resolved

Merge branch 'main' into feature/windows-eventId-filtering

1ab8a1a

Paamicky force-pushed the feature/windows-eventId-filtering branch from dc6df12 to 1ab8a1a Compare June 25, 2025 21:36

Paamicky added 2 commits July 3, 2025 10:24

Merge branch 'main' into feature/windows-eventId-filtering

5a26a15

Merge branch 'main' into feature/windows-eventId-filtering

218f290

Paamicky changed the title ~~Feature/windows event id filtering~~ Add Windows event id filtering to CWAgent Jul 9, 2025

Paamicky changed the title ~~Add Windows event id filtering to CWAgent~~ Add windows event id filtering to CWAgent Jul 9, 2025

lisguo reviewed Jul 10, 2025

View reviewed changes

Paamicky added 2 commits July 11, 2025 10:25

changed cutoffPeriod to const

90f466e

Add eventid check to the Agent TOML

10f9f0a

Paamicky force-pushed the feature/windows-eventId-filtering branch from 5c8f615 to 10f9f0a Compare July 11, 2025 18:28


		var NumberOfBytesPerCharacter = UnknownBytesPerCharacter

		func RenderEventXML(eventHandle EvtHandle, renderBuf []byte) ([]byte, error) {

		// Process value to ensure integers in arrays are preserved
		processedVal := processValue(val)

Add windows event id filtering to CWAgent #1737

Are you sure you want to change the base?

Add windows event id filtering to CWAgent #1737

Uh oh!

Conversation

Paamicky commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of the issue

Description of changes

License

Tests

Requirements

Integration Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Paamicky Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Paamicky Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Paamicky commented Jun 18, 2025 •

edited

Loading

Paamicky Jun 24, 2025 •

edited

Loading

Paamicky Jul 11, 2025 •

edited

Loading