Skip to content

osquery generate ecs.yml #1495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leehinman merged 2 commits into from
Aug 15, 2021
Merged

osquery generate ecs.yml #1495

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9049c14
convert to ECS generated fields
leehinman Aug 8, 2021
709555c
update changelog
leehinman Aug 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/osquery/_dev/build/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dependencies:
ecs:
reference: [email protected]
5 changes: 5 additions & 0 deletions packages/osquery/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "0.5.2"
changes:
- description: Convert to generated ECS fields
type: enhancement
link: https://github.com/elastic/integrations/pull/1495
- version: '0.5.1'
changes:
- description: update to ECS 1.11.0
Expand Down
6 changes: 6 additions & 0 deletions packages/osquery/data_stream/result/fields/agent.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,3 +196,9 @@
description: >
OS codename, if any.

- name: input.type
type: keyword
description: Input type
- name: log.offset
type: long
description: Log offset
228 changes: 46 additions & 182 deletions packages/osquery/data_stream/result/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,182 +1,46 @@
- name: ecs
title: ECS
group: 2
type: group
fields:
- name: version
level: core
required: true
type: keyword
ignore_above: 1024
description: "ECS version this event conforms to."
- name: event
title: Event
group: 2
type: group
fields:
- name: ingested
level: core
type: date
description: "Timestamp when an event arrived in the central data store."
default_field: false
- name: log
title: Log
group: 2
type: group
fields:
- name: file.path
level: extended
type: keyword
ignore_above: 1024
description: "Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate."
default_field: false
- name: level
level: core
type: keyword
ignore_above: 1024
description: "Original log level of the log event."
- name: offset
type: long
description: Log offset
- name: original
level: core
type: keyword
ignore_above: 1024
description: "This is the original log message and contains the full log message before splitting it up in multiple parts."
index: false
- name: file
title: File
group: 2
description: "A file is defined as a set of information that has been created on, or has existed on a filesystem.\nFile objects can be associated with host events, network events, and/or file events (e.g., those produced by File Integrity Monitoring [FIM] products or services). File fields provide details about the affected file associated with the event or metric."
type: group
fields:
- name: accessed
level: extended
type: date
description: "Last time the file was accessed."
- name: created
level: extended
type: date
description: "File creation time."
- name: directory
level: extended
type: keyword
description: Directory where the file is located. It should include the drive letter, when appropriate.
- name: gid
level: extended
type: keyword
ignore_above: 1024
description: Primary group ID (GID) of the file.
- name: inode
level: extended
type: keyword
ignore_above: 1024
description: Inode representing the file in the filesystem.
- name: mode
level: extended
type: keyword
ignore_above: 1024
description: Mode of the file in octal representation.
example: "0640"
- name: mtime
level: extended
type: date
description: Last time the file content was modified.
- name: name
level: extended
type: keyword
ignore_above: 1024
description: Name of the file including the extension, without the directory.
- name: path
level: extended
type: keyword
multi_fields:
- name: text
type: text
norms: false
default_field: false
description: Full path to the file, including the file name. It should include the drive letter, when appropriate.
- name: size
level: extended
type: long
description: "File size in bytes."
- name: type
level: extended
type: keyword
ignore_above: 1024
description: File type (file, dir, or symlink).
- name: uid
level: extended
type: keyword
ignore_above: 1024
description: The user ID (UID) or security identifier (SID) of the file owner.
- name: input.type
type: keyword
description: Input type
- name: process
title: Process
group: 2
description: "These fields contain information about a process.\nThese fields can help you correlate metrics information with a process id/name from a log message. The `process.pid` often stays in the metric itself and is copied to the global field for correlation."
type: group
fields:
- name: name
level: extended
type: keyword
multi_fields:
- name: text
type: text
norms: false
default_field: false
description: "Process name."
- name: related
title: Related
type: group
fields:
- name: hosts
level: extended
type: keyword
ignore_above: 1024
default_field: false
- name: user
level: extended
type: keyword
ignore_above: 1024
default_field: false
- name: rule
title: Rule
group: 2
type: group
fields:
- name: name
level: extended
type: keyword
ignore_above: 1024
description: The name of the rule or signature generating the event.
default_field: false
- name: url
title: URL
group: 2
type: group
fields:
- name: full
level: extended
type: keyword
multi_fields:
- name: text
type: text
norms: false
default_field: false
- name: user
title: User
group: 2
type: group
fields:
- name: name
level: core
type: keyword
multi_fields:
- name: text
type: text
norms: false
default_field: false
description: Short name or login of the user.
- external: ecs
name: ecs.version
- external: ecs
name: event.ingested
- external: ecs
name: file.accessed
- external: ecs
name: file.created
- external: ecs
name: file.directory
- external: ecs
name: file.gid
- external: ecs
name: file.inode
- external: ecs
name: file.mode
- external: ecs
name: file.mtime
- external: ecs
name: file.name
- external: ecs
name: file.path
- external: ecs
name: file.size
- external: ecs
name: file.type
- external: ecs
name: file.uid
- external: ecs
name: log.file.path
- external: ecs
name: log.level
- external: ecs
name: log.original
- external: ecs
name: process.name
- external: ecs
name: related.hosts
- external: ecs
name: related.user
- external: ecs
name: rule.name
- external: ecs
name: url.full
- external: ecs
name: user.name
24 changes: 12 additions & 12 deletions packages/osquery/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,20 +153,20 @@ An example event for `result` looks as following:
| data_stream.dataset | Data stream dataset. | constant_keyword |
| data_stream.namespace | Data stream namespace. | constant_keyword |
| data_stream.type | Data stream type. | constant_keyword |
| ecs.version | ECS version this event conforms to. | keyword |
| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
| event.dataset | Event dataset | constant_keyword |
| event.ingested | Timestamp when an event arrived in the central data store. | date |
| event.ingested | Timestamp when an event arrived in the central data store. This is different from `@timestamp`, which is when the event originally occurred. It's also different from `event.created`, which is meant to capture the first time an agent saw the event. In normal conditions, assuming no tampering, the timestamps should chronologically look like this: `@timestamp` \< `event.created` \< `event.ingested`. | date |
| event.module | Event module | constant_keyword |
| file.accessed | Last time the file was accessed. | date |
| file.created | File creation time. | date |
| file.accessed | Last time the file was accessed. Note that not all filesystems keep track of access time. | date |
| file.created | File creation time. Note that not all filesystems store the creation time. | date |
| file.directory | Directory where the file is located. It should include the drive letter, when appropriate. | keyword |
| file.gid | Primary group ID (GID) of the file. | keyword |
| file.inode | Inode representing the file in the filesystem. | keyword |
| file.mode | Mode of the file in octal representation. | keyword |
| file.mtime | Last time the file content was modified. | date |
| file.name | Name of the file including the extension, without the directory. | keyword |
| file.path | Full path to the file, including the file name. It should include the drive letter, when appropriate. | keyword |
| file.size | File size in bytes. | long |
| file.size | File size in bytes. Only relevant when `file.type` is "file". | long |
| file.type | File type (file, dir, or symlink). | keyword |
| file.uid | The user ID (UID) or security identifier (SID) of the file owner. | keyword |
| host.architecture | Operating system architecture. | keyword |
Expand All @@ -186,10 +186,10 @@ An example event for `result` looks as following:
| host.os.version | Operating system version as a raw string. | keyword |
| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
| input.type | Input type | keyword |
| log.file.path | Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate. | keyword |
| log.level | Original log level of the log event. | keyword |
| log.file.path | Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate. If the event wasn't read from a log file, do not populate this field. | keyword |
| log.level | Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). Some examples are `warn`, `err`, `i`, `informational`. | keyword |
| log.offset | Log offset | long |
| log.original | This is the original log message and contains the full log message before splitting it up in multiple parts. | keyword |
| log.original | Deprecated for removal in next major version release. This field is superseded by `event.original`. This is the original log message and contains the full log message before splitting it up in multiple parts. In contrast to the `message` field which can contain an extracted part of the log message, this field contains the original, full log message. It can have already some modifications applied like encoding or new lines removed to clean up the log message. This field is not indexed and doc_values are disabled so it can't be queried but the value can be retrieved from `_source`. | keyword |
| osquery.result.action | | keyword |
| osquery.result.calendar_time | String representation of the collection time, as formatted by osquery. | keyword |
| osquery.result.columns.active | | keyword |
Expand Down Expand Up @@ -363,11 +363,11 @@ An example event for `result` looks as following:
| osquery.result.host_identifier | The identifier for the host on which the osquery agent is running. Normally the hostname. | keyword |
| osquery.result.name | The name of the query that generated this event. | keyword |
| osquery.result.unix_time | Unix timestamp of the event, in seconds since the epoch. Used for computing the `@timestamp` column. | keyword |
| process.name | Process name. | keyword |
| related.hosts | | keyword |
| related.user | | keyword |
| process.name | Process name. Sometimes called program name or similar. | keyword |
| related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
| related.user | All the user names or other user identifiers seen on the event. | keyword |
| rule.name | The name of the rule or signature generating the event. | keyword |
| tags | List of keywords used to tag each event. | keyword |
| url.full | | keyword |
| url.full | If full URLs are important to your use case, they should be stored in `url.full`, whether this field is reconstructed or present in the event source. | keyword |
| user.name | Short name or login of the user. | keyword |

2 changes: 1 addition & 1 deletion packages/osquery/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: osquery
title: Osquery Log Collection
version: 0.5.1
version: 0.5.2
release: experimental
description: This Elastic integration collects logs from Osquery instances
type: integration
Expand Down