Skip to content

[Exchange Online Message Trace] Add Additional Look-back Time & Fix Cursor Value #8717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ShourieG merged 17 commits into from
Dec 21, 2023
Merged

[Exchange Online Message Trace] Add Additional Look-back Time & Fix Cursor Value #8717

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
f804e26
refactor order of httpjson file config
BenB196 Dec 13, 2023
d0fdb5f
fix cursor
BenB196 Dec 13, 2023
f912540
Update changelog.yml
BenB196 Dec 13, 2023
e5ee442
Revert "refactor order of httpjson file config"
BenB196 Dec 13, 2023
aff473a
add additional look-back time
BenB196 Dec 17, 2023
817519c
fix doc wording
BenB196 Dec 17, 2023
bff9eea
Apply suggestions from code review
BenB196 Dec 18, 2023
76fb689
Run build to update docs
BenB196 Dec 18, 2023
6f14743
make existing system test, test skiptoken
BenB196 Dec 19, 2023
efdd3ef
more system tests
BenB196 Dec 19, 2023
7e5b3dc
revert unneeded change
BenB196 Dec 19, 2023
0b5d0ad
update cursor
BenB196 Dec 19, 2023
6ff67b8
improve drop processor and add pipeline test
BenB196 Dec 19, 2023
7da5896
fix cursor and add tests
BenB196 Dec 20, 2023
b7b7501
further refine drop processor condition
BenB196 Dec 20, 2023
e6cd256
add comments about $filter param modification
BenB196 Dec 20, 2023
86e6475
Update packages/microsoft_exchange_online_message_trace/data_stream/l…
BenB196 Dec 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/microsoft_exchange_online_message_trace/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.17.3"
changes:
- description: Fix issue with cursor potentially causing data gaps.
type: bugfix
link: https://github.com/elastic/integrations/pull/8717
- version: "1.17.2"
changes:
- description: Fix exclude_files pattern.
Expand Down
37 changes: 19 additions & 18 deletions ...ges/microsoft_exchange_online_message_trace/data_stream/log/agent/stream/httpjson.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,6 @@ request.transforms:
- set:
target: url.params.$skiptoken
value: 0
fields_under_root: true
fields:
_conf:
local_domains:
{{#each local_domains as |local_domain i|}}
- {{local_domain}}
{{/each}}
tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#each tags as |tag i|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
publisher_pipeline.disable_host: true
{{/contains}}
response.split:
target: body.value
response.pagination:
Expand All @@ -70,7 +53,25 @@ response.pagination:
fail_on_template_error: true
cursor:
last_execution_datetime:
value: '[[(formatDate (now) "RFC3339")]]'
value: '[[.last_event.EndDate]]'

fields_under_root: true
fields:
_conf:
local_domains:
{{#each local_domains as |local_domain i|}}
- {{local_domain}}
{{/each}}
tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#each tags as |tag i|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
publisher_pipeline.disable_host: true
{{/contains}}

{{#if processors}}
processors:
Expand Down
2 changes: 1 addition & 1 deletion packages/microsoft_exchange_online_message_trace/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: "3.0.0"
name: microsoft_exchange_online_message_trace
title: "Microsoft Exchange Online Message Trace"
version: "1.17.2"
version: "1.17.3"
description: "Microsoft Exchange Online Message Trace Integration"
type: integration
categories:
Expand Down