CEE Logging Standard: Today and Tomorrow

CEE Logging Standard: Today and Tomorrow Dr Anton Chuvakin Chief Logging Evangelist LogLogic, Inc

Outline World of logs today Where is chaos? Everywhere! Why? Why order is needed? Past attempts to bring order to log chaos! Why ALL failed? CEE Approach Brief history 4 pillars of CEE and their today’s status Future possibilities

Log Chaos I - Login? <122> Mar 4 09:23:15 localhost sshd[27577]: Accepted password for kyle from ::ffff:192.168.138.35 port 2895 ssh2 <13> Fri Mar 17 14:29:38 2006 680 Security SYSTEM User Failure Audit ENTERPRISE Account Logon Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon account : POWERUSER Source Workstation: ENTERPRISE Error Code: 0xC000006A 4574 <57> Dec 25 00:04:32:%SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user:yellowdog] [Source:10.4.2.11] [localport:23] at 20:55:40 UTC Fri Feb 28 2006 <18> Dec 17 15:45:57 10.14.93.7 ns5xp: NetScreen device_id=ns5xp system-warning-00515: Admin User netscreen has logged on via Telnet from 10.14.98.55:39073 (2002-12-17 15:50:53)

Log Chaos II - Accept? messages:Dec 16 17:28:49 10.14.93.7 ns5xp: NetScreen device_id=ns5xp system-notification-00257(traffic): start_time="2002-12-16 17:33:36" duration=5 policy_id=0 service=telnet proto=6 src zone=Trust dst zone=Untrust action=Permit sent=1170 rcvd=1500 src=10.14.94.221 dst=10.14.98.107 src_port=1384 dst_port=23 translated ip=10.14.93.7 port=1206 Mar 6 06:06:02 winonasu-pix %PIX-6-302013: Built outbound TCP connection 315210 596 for outside:172.196.9.206/1214 (172.196.9.206/1214) to inside:199.17.151.103/1438 (199.17.151.103/1438) Apr 6 06:06:02 Checkpoint NGX SRC=Any,DEST=ANY,Accept=nosubstitute,Do Not Log,Installspyware,lieonyourtaxes,orbetteryet,dontpaythem

Definitions Log = message generated by an IT system to record whatever event happening Log format = layout of log messages in the form of fields, separators, delimiters, tags, etc Log syntax = fields and values that are present in logs Log taxonomy = a taxonomy of log messages that categorizes log messages and codifies their meaning Log transport = a method of moving logs from one system to another; typically a network protocol

Log Chaos No standard format No standard schema, no level of details No standard meaning No taxonomy No standard transport No shared knowledge on what to log and how No logging guidance for developers No standard API / libraries for log production

Chaos2order: Why Logging Standards? Common language so that people and systems understand what is in the logs Easier to report on logs and explain the reports Deeper insight into future problems as indicated by the log data Easier system interoperability leading to reduced cost and complexity Common logging practices simplify audits and compliance Easier to explain what is in the logs to management and non-IT people

Various Logging Standards by Type Log format Example: Syslog, a non-standard standard  Example: IDMEF, a failed standard Log contents No standard to speak of: logs = trash can because application developers dump what they want there (and how they want!) Log transport Example: Syslog (TCP/UDP port 514) Logging practices Example: NIST 800-92 (for security only)

Old, Dead and Vendor Log Standards Old, mostly dead standards : CIDF – DARPA (became IDMEF) IDMEF – IETF (never adopted by anybody ) CIEL – MITRE (cancelled early) Vendor “standard” efforts: CBE - IBM WELF - Webtrends CEF - ArcSight OLF – eIQnetworks SDEE – Cisco+ (also mostly dead as a standard)

What Killed’em ALL?  Lack of adoption – BIG one! “ Solution in search of a problem” IETF “overthinkers”  Overly complex standards Vendors and their tactical focus (or “marketing standards”) Narrow approach (e.g. just security)

What Worked? NIST 800-92 Guide to LM “ This publication seeks to assist organizations in understanding the need for sound computer security log management. It provides practical, real-world guidance on developing, implementing, and maintaining effective log management practices throughout an enterprise. “

What is a Common Event Expression – CEE? CEE = Syntax + Vocabulary + Transport + Log Recommendations Common Event Expression Impacts Log management capabilities Log correlation (SIEM) capabilities Device intercommunication enabling autonomic computing Enterprise-level situational awareness Infosec attacker modeling and other security analysis capability Common Event Expression Taxonomy To specify the event in a common representation Common Log Syntax For parsing out relevant data from received log messages Common Log Transport For exchanging log messages Log Recommendations For guiding events and details needed to be logged by devices (OS, IDS, FWs, etc)

CEE Introduction Brief history Used to be called: Common Event eXpression  Now called: Common Event Expression  First conceived : discussions over 2004 about CIEL Started unofficially: email conversations 8/2005 Started officially : meeting 01/2007

CEE Current Status “ Alive and well!” CEE board creative and active Public Working Group created and active (join via [email_address] ) Positioning note released Website : http:// cee.mitre.org Longer white paper under review (TBA) Project FAQ released (update TBA) Public WG list archives (soon TBA)

Key Area of CEE Common Event Expression (CEE) by MITRE Key standard areas : “ Create an event expression taxonomy for uniform and precise log definitions that lead to a common event representation. Create log syntaxes utilizing a single data dictionary to provide consistent event specific details. Standardize flexible event transport mechanisms to support multiple environments. Propose log recommendations for the events and attributes devices generate.”

CEE Components Log transport Plan : “bless” existing transports ( ongoing !) Log format/syntax Plan: analyze existing and create ( ongoing !) Log taxonomy Plan: create one ( ongoing !) Logging recommendations Plan: collect, organize and unify ( ongoing !)

1. CEE on Log Transport How logs move around? Syslog via UDP/TCP 514 XML over SOAP over HTTP (and same over SSL) XDAS event transport service SNMP traps Windows logs over WMI or RPC CEE will adopt or “bless” those that are common and/or “clean” Questions : encryption? Filtering?

2. CEE on Log Format/Syntax Need a generic syntax to map into these below: Key=value pair format Need a canonical list of keys! CSV/delimited or database format Need a list of column names XML format Needs a list of XML tags Binary format Need field names

3. CEE on Log Taxonomy The main idea – every log MUST have: OBJECT ACTION STATUS Example : User jsmith login successful OBJECT ACTION STATUS

4. CEE on Logging Recommendations Based on existing guidance (e.g. regulatory, etc) What to log? What details? What fields? What format? What scenario, industry, regulation, etc?

Adoption, ADOPTION, A-D-O-P-T-I-O-N! Why CEE WILL be adopted? MITRE can drive government procurement MITRE sponsor organizations will use it immediately Open process with everybody welcome (ALL stakeholders’ needs incorporated) If you have comments, ideas, share them at [email_address] list!

Why CEE WILL Succeed!! Time has come – logs are more important today than ever MITRE as a host – many successful standards Approach – attack key “pain” areas of “log chaos” Vendors on board – can adopt and use

Conclusions: Future of CEE Develop 4 pillars of CEE standard Publish and collect feedback from the community Drive adoption from the software vendor side Drive adoption from the government side Drive adoption from the log analysis vendor side

CEE + XDAS Together (Suggested!) XDAS audit transport service = to be “blessed” as one of the CEE log transport mechanisms XDAS event fields = to be included in CEE syntax XDAS events = to follow CEE taxonomy (i.e. have OBJECT, ACTION, STATUS or fields mapped to them) Suggestion on what XDAS audit options to enable = to include in CEE logging recommendations Common registry for changes

Thanks for Attending the Presentation Dr Anton Chuvakin, GCIH, GCFA www.chuvakin.org Chief Logging Evangelist LogLogic, Inc Coauthor of “Security Warrior” (O’Reilly, 2004) and “PCI Compliance” (Syngress, 2007) books See http://www.info-secure.org for my papers, books, reviews and other security resources related to logs. Also see http://chuvakin.blogspot.com

Backup and Example Slides from MITRE

CEE Example Sep 26 12:00:00 myhost-- root[808]: ROOT LOGIN ON tty1 Apr 10 12:30:34 hostname sshd[16682]: error: PAM: Authentication failure for user1 from host.domain.com Sep 19 08:26:10 zuric CEF:0|security|threatmanager|1.0|100|worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2 spt=1232 Example Log Messages Transport - successful log transmission Syslog (each log message is transmitted in a single UDP packet usually over port 514/UDP) Syntax - details specific to event being logged Format (3) CEF message CEF:Version|Device Vendor|DeviceProduct|DeviceVersion|Signature ID|Name|Severity|Extension Extension is a meta item – it imposes a sytax extension to specify addition details - a data dictionary can enumerate these details Syntax - details specific to event being logged Format (1 and 2) month day time host program[pid]: message In CEE, each of these would be a possible syntax element, whose value and definition would be well defined by a Data Dictionary Data Dictionary needed to enumerate details associated with the event It needs to provide flexible syntax options by defining the elements and formats Ex: dst in the CEF event – defined by a dotted quad IPv4 address

CEE Example (Cont.) Scenario: An attacker has breached our network 1. Determine if any successful logins What do we search for – ‘log in’, ‘log in’, ‘logged on’ etc. Taxonomy – a reduced language set for consistent log messages Right now they are semantically similar – ok for humans – but not for computers

CEE Logging Standard: Today and Tomorrow

More Related Content

Similar to CEE Logging Standard: Today and Tomorrow (20)

More from Anton Chuvakin (20)

Recently uploaded (20)

CEE Logging Standard: Today and Tomorrow

Editor's Notes