add additional look-back time

packages/microsoft_exchange_online_message_trace/_dev/build/docs/README.md

@@ -53,6 +53,11 @@ available in your organization. They are usually under the sections [Accepted Do
 - The default `Polling Interval` and `Initial Interval` values are configured to `1h`, you can however change these to your required values. The look-back 
   value of `Initial Interval` should not exceed `200 hours` as this might cause unexpected errors with the API.
 
+- The default `Additional Look-back Time` value is configured for to `1h`. 
+  This is intended to allow for catching events that were not initially present because of eventual consistency.
+  This value does not need to exceed [`24h`](https://learn.microsoft.com/en-us/previous-versions/office/developer/o365-enterprise-developers/jj984335(v=office.15)#data-granularity-persistence-and-availability).
+    - Note: The larger this value is, the less likely events will be missed, however, this will take the integration longer to pull all events, making newer events take longer to become present.
+
 - The default value of `Batch Size` is set to 1000. This means for every request Httpjson will paginate with a value of 1000 results per page. The 
    maximum page size supported by the Message Trace API is `2000`. The API will return an empty `value` array when there are no more logs to pull and the
    pagination will terminate with an error that can be ignored.

packages/microsoft_exchange_online_message_trace/changelog.yml

@@ -1,6 +1,9 @@
 # newer versions go on top
-- version: "1.17.3"
+- version: "1.18.0"
   changes:
+    - description: Add ability to configure addition look-back time.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/8717
     - description: Fix issue with cursor potentially causing data gaps.
       type: bugfix
       link: https://github.com/elastic/integrations/pull/8717

packages/microsoft_exchange_online_message_trace/data_stream/log/agent/stream/httpjson.yml.hbs

@@ -28,7 +28,7 @@ request.ssl: {{ssl}}
 request.transforms:
   - set:
       target: url.params.$filter
-      value: StartDate eq datetime'[[.cursor.last_execution_datetime]]' and EndDate eq datetime'[[(formatDate (now) "RFC3339")]]'
+      value: StartDate eq datetime'[[(formatDate (.cursor.last_execution_datetime (parseDuration "-{{additional_look_back}}")) "RFC3339")]]' and EndDate eq datetime'[[(formatDate (now) "RFC3339")]]'
       default: StartDate eq datetime'[[(formatDate (now (parseDuration "-{{initial_interval}}")) "RFC3339")]]' and EndDate eq datetime'[[(formatDate (now) "RFC3339")]]'
   - set:
       target: url.params.$top

packages/microsoft_exchange_online_message_trace/docs/README.md

@@ -53,6 +53,11 @@ available in your organization. They are usually under the sections [Accepted Do
 - The default `Polling Interval` and `Initial Interval` values are configured to `1h`, you can however change these to your required values. The look-back 
   value of `Initial Interval` should not exceed `200 hours` as this might cause unexpected errors with the API.
 
+- The default `Additional Look-back Time` value is configured for to `1h`. 
+  This is intended to allow for catching events that were not initially present because of eventual consistency.
+  This value does not need to exceed [`24h`](https://learn.microsoft.com/en-us/previous-versions/office/developer/o365-enterprise-developers/jj984335(v=office.15)#data-granularity-persistence-and-availability).
+    - Note: The larger this value is, the less likely events will be missed, however, this will take the integration longer to pull all events, making newer events take longer to become present.
+
 - The default value of `Batch Size` is set to 1000. This means for every request Httpjson will paginate with a value of 1000 results per page. The 
    maximum page size supported by the Message Trace API is `2000`. The API will return an empty `value` array when there are no more logs to pull and the
    pagination will terminate with an error that can be ignored.

packages/microsoft_exchange_online_message_trace/manifest.yml

@@ -1,7 +1,7 @@
 format_version: "3.0.0"
 name: microsoft_exchange_online_message_trace
 title: "Microsoft Exchange Online Message Trace"
-version: "1.17.3"
+version: "1.18.0"
 description: "Microsoft Exchange Online Message Trace Integration"
 type: integration
 categories:
@@ -125,6 +125,14 @@ policy_templates:
             multi: false
             required: true
             show_user: true
+          - name: additional_look_back
+            type: text
+            title: Additional Look-back Time
+            description: Adds time to the interval to prevent missed events. Supported units for this parameter are h/m/s.
+            default: 1h
+            multi: false
+            required: true
+            show_user: true
           - name: request_timeout
             type: text
             title: Request Timeout