So I think this is the last time I’ll ask you to move with me….  I hope it is anyway….

As of last week I’ve started a new venture.  My company is named “Visible Risk”.  Visible Risk other than being a great name for a company, is my effort to help push information security forward over the next few years.  I’ll be working with certain organizations on integrating intelligence and security operations, and a huge area of focus for me will be providing “live” use-case based content for security products (like SIEM).

Additionally, I’m starting a new podcast and video/webcast under the Visible Risk brand over the next few weeks so please be on the look out for that as I’d love to involve you in it!

Visible Risk Blog RSS Feed:  http://www.visiblerisk.com/blog/rss.xml

Thank you again to everyone who has helped me over the years to better understand my strengths and weaknesses and for always pushing me forward!

If you’re not already following my new blog here are links to some of my recent postings:

1.  A primer on starting a new company:  http://www.visiblerisk.com/blog/2010/3/10/so-you-want-to-work-for-yourself.html  or   http://bit.ly/aX7WWB

2.  RSA Recap - Round 1: http://www.visiblerisk.com/blog/2010/3/10/rsa-conference-2010-recap-round-1.html  or http://bit.ly/aPA63z

Thank You, Rocky

From my NEW BLOG: securityoperations.blogspot.com

  A “point in time” snapshot of how I think 2010-2012 looks in the SIEM Market. A much more detailed analysis will be available soon (on request).

Some highlights of the preview:

1. Many companies are focused on rationalizing recent acquisitions or focusing on making their current product scalable and/or bullet-proof. I think that this is absolutely crucial for these organizations but it does create an opportunity for ArcSight to further separate from the pack in 2010.

2. Formally “niche” players are taking the lead in 2010. Q1, Tenable, Nitro all have a legitimate change to overtake their peers in terms of functionality and more importantly marketplace. Each has their own approach, all are led by very capable teams - I’m interested to watch and see what the market does with these three.

3. I don’t expect all of these SIEM players to survive to the 2012 Winter Olympics. In fact, I’d guess at least three of them will be consumed or fail completely. Many have other products that have helped them sustain, but not necessarily grow when compared to SIEM competition.

4. Most of the larger organizations have had serious setbacks with their acquisitions in this space. Based on functionality limitations and these organizations losing significant market share I expect some of these organizations to take a serious look at replacing those products (or portions of the products) with more competitive options in the market today.

5. SIEM will certainly grow into interesting areas in the next 24 months as vendors look toward cloud based solutions, supporting virtualized systems and networks, and as more mature users push these products to solve problems other than the basic Security Operations and Compliance based Use-Cases.

6. I do expect the larger picture to come in focus around SIEM soon. RSA’s acquisition of Archer is indicative of things to come. The larger companies are focused on presenting Enterprise Risk to the business and not just speeds and feeds anymore. Certainly better reporting, integration with enterprise apps and usage of other technologies will continue to evolve but I believe it will finally be centered on the user’s functional purposes and not just marketing hype.

7. SIEM also needs to evolve downward as well. Yes positioning relevant information upward in the business is the ultimate goal, but we can’t forget the analyst. The SIEM must continue to support the analytical needs of its core user base. Deeper integration with other analytical tools and resources (Content Inspection, CMDB, Custom DB’s, etc) and facility that interaction intuitively.

Personally,  I’ll count 2009 as the year of lessons learned.  I’m happy to start 2010 and begin anew.  Many of you have reached out to me in twitter (@rockyd) or email, FB, etc and asked about my status, personally and professionally - for which I’m very thankful.  It is awesome to see some many people and organizations genuinely care about me - I’m humbled.  We did make some changes late in 2009 that for all intents and purposes brought an end to Decurity as it was known.  The full plan never quite panned out the way we all hoped it would.  I joined EMC/RSA for a while and worked alongside some fantastic people over there, but in the end it just wasn’t the right place for me.  I resigned my position at RSA and took some time off to focus on my family, my health and to renew myself so that I could focus fully in 2010 and beyond. 

Personally: I had let myself get way out of shape (mentally, spiritually and physically) and let my blood sugar reach levels that truly frightened everyone.  I thought I was just more sweet, but when doctors start wondering why you’re not in a coma it’s time to pay attention.  I joke about it a lot but I’ve learned to pay much closer attention now.  Eventually, I hope to make it to P90X type workouts but for now I’m happy to be able to walk a few miles, a few times a week.  It sucks when there is no one else to blame but yourself, but then again I know I can change my habits easier than trying to make many orgs think clearly about how to handle security risks.

Professionally:  I’m currently in the midst of considering some fantastic opportunities from various organizations that have reached out to me. I can’t tell you how lucky I feel to have so many believe in me.  I’m delaying making a final decision until I’m a little healthier (should only be a few days).  I want to ensure that whichever route I take it makes sense for me, the company, their user-base and the segment of the security industry I can influence.  I’ll let everyone know where I wind up once things settle down.

Another Note:  I’m moving my personal blogging efforts over to securityoperations.blogspot.com.  I’ll probably dual post for a while as Decurity’s blog has much more critical mass, but I’d imagine I’ll keep up with securityoperations.blogspot.com more often from now on.

I was asked to provide a guest post for the FUDSEC Blog.  After reading so many of the other guest posts I felt a little overwhelmed to put my ramblings alongside those gems.  I’m thrilled Craig allowed me the opportunity and look forward to hearing your input.  Please enjoy ripping my thoughts into pieces, chewing on them and then letting me know how you really feel!  FUDSEC: Liberate Yourself: Change The Game To Suit Your Needs  Comments are encouraged directly on FUDSEC or you can reach me on Twitter (@rockyd) or reach me on the Decurity Blog any way you chose to reach out I’d love to hear your input.

When Richard asked me to participate as a moderator for the MSSP/SOC Panel I was of course flattered and thrilled to participate!  I’ll be moderating a panel discussion on MSSP and Corporate SOC capabilities.  I’m looking to expose “what works” from each perspective and hopefully we’ll gleam some valuable insight from both perspectives.  Let’s face it most larger organization flip-flop between internal/external capabilities every few years… let’s find out why and what value they gain from each perspective. 

If you’d like your questions, comments addressed in this forum please email or tweet (DM) me and I’ll do my best to incorporate your thoughts!  If you are in the DC area you really should attend this summit - The speakers are some of the best practitioners out there all looking to share techniques for successful detection in their enterprise. 

Event Description:
Following the success of the 2008 and 2009 editions of the SANS WhatWorks in Forensics and Incident Response Summits, SANS is teaming with Richard Bejtlich to create a practioner-focused event dedicated to incident detection operations. The SANS Incident Detection Summit will share tools, tactics, and techniques practiced by more than 40 of the world’s greatest incident detectors in two full days of content consisting of keynotes, expert briefings, and dynamic panels.

http://www.sans.org/incident-detection-summit-2009/

This morning as my wife was leaving for work she noticed a extended cab pickup truck parked out in front of our neighbor’s house.  As she began to pull out of the driveway she noted that the driver got out and was beginning to go through the neighbors trash.  My wife parked at the end of the street and then called me.  I dismissed it at first but as I observed for a few moments I was amazed at how thoroughly this gentleman was going through each bag.  His urgency and purpose was like he was looking for a lost wedding ring.

Needing something to do today I walked up to him and inquired about what he was doing.  Obviously and physically taken aback by me confronting him, he produced a toy out of his pocket and told me he “wasn’t doing anything” that he was just looking for toys and gave me a sheepish grin.  My kids do a much better job of acting.

Given the nature of the truck (at 50K+ it was probably either his employer’s or stolen), the fact their was no car seat was in it, his “look-out”,  and his overall demeanor I pressed a little harder.  I asked him about the pile of papers he was so carefully gathering.  Of course all of the sudden his knowledge of the english language ceased to exist and he was in a hurry to leave.  In spanish he yelled to his wife to get ready to go and that he didn’t like the situation.  So I switched to spanish and surprised him even further.  I was able to retrieve the papers from him before he ran into his truck that his wife was starting to drive away in.  Damn,  I was just starting to have fun with him, well at least the cops should be able to retrieve the stolen truck pretty quickly.

I’m fully cognizant of the fact that financially times are hard right now and people need to do what they can to survive.  I’m not against him looking for toys or taking broken household equipment to repair or any number of other things that people quickly retrieve from others discarded household items.  I’m very leery of how organized and thorough this team was.  1. The vehicle fit into it surroundings, except we live on a cul-de-sac with very low traffic.  2. He had an obvious “look-out” who was intently watching my wife 3. He dissected every bag, quickly and efficiently 4. Timing - he hit the garbage in the two hour window it sits outside.

Yes I’m probably paranoid but I can almost guarantee that this team worked as part of a larger organization paid by the pound of paper they collect or otherwise compensated for what they found.  It’s highly lucrative, insulates upper layers and incredibly simple to execute.  It could have been a precursor to a physical intrusion, but honestly that’s not going to nearly as lucrative as the identify theft angle. 

Recommendations: 

1. Be aware of your surroundings.  If it seems out of place - find out why.  At a minimum observe and report to your local police department.

2. Shred everything, no matter how “insignificant” it is.  If I’m honest with myself ‘ve been horrible about this at home.  I have a shredder two feet from me that is going to be fed well today!

3. Carefully screen who you let in your home.  Technicians, Cleaners, Painters.  There are so many ways to extend this type of collection activity it isn’t even funny.

4. Talk with your neighbors.  It’s much easier if everyone is fully aware of what is going on and can help observe and act as necessary.  You can also get trusted recommendations for service help.  Plus the holidays are coming - just go out and be nice.

5. Check your credit report from all three major players every month.  The odds are that your identity or at a minimum your credit/bank account will be compromised at least once.  The quicker you can identify it the easier the mess is to clean up.

There are a lot of major changes going on at Decurity and soon enough we’ll be in a position to announce them to the world!  In the mean time this is just a quick note to say that Rocky DeStefano will be participating in a couple of fun information security events in the near future:

1.  NetWitness User Conference Nov 4-5 2009 in DC, Gabe Martinez and I are teaming up again and presenting some real-world examples of SIEM and NetWitness integrations in a technical training session on Nov 4th.  This integration is probably one of the most powerful enhancements you can make to a security operations center, come see it for yourself.  If you haven’t already registered, please do so - http://www.netwitness.com/userconference.html

2. SANS What Works in Incident Detection Summit Dec 9-10 2009 in DC.  I’ll be moderating a panel discussion on internal CIRT’s and MSSP’s.  This panel should just be serious fun all around.  For more information see http://www.sans.org/incident-detection-summit-2009/agenda.php and http://taosecurity.blogspot.com/2009/08/sans-incident-detection-summit-in-dc-in.html

When the rest of my dance card fills up I’ll update this post. 

ArcSight Protect ‘09 was a whirlwind of activity for Decurity.  I would love to thank everyone that came up to the booth and gave us feedback on the blog, to all of our customers that stopped by and helped introduce us to their friends and of course to all my friends at ArcSight that made the week so enjoyable. 

Technology advances announced as part of Protect ‘09:

1. ArcSight Logger 4.0 While still technically in Beta, this product goes a long way to resolving any perceived flaws in the technology.  Unstructured search, incredible insert rates, better and much fast reporting, direct integration with ESM Console.  We got to spend some significant time with 4.0 and we were really impressed with the ability to just take data no matter how ugly it was into the system and deal with it very effectively.  A live demo conducted during the 2nd day keynote confirmed that the speed was incredible.  The fixes under the covers to how the system handles I/O means that not even RAID 5 slows down Logger.  The implications are huge!  Insert rates are just ridiculous, they prove 100K EPS on very basic hardware.  There is some pretty cool pixie dust in those appliances.  Now if we could just get them into VM’s or AMI’s…..

2. ArcSight FraudView:  This type of application integration purpose built solutions helps extend ArcSight ESM as a platform to look beyond Security in the enterprise.  Moving out of pure security thought processes and into solving core business problems is exactly what Use-Cases are all about.  This use-case took information from SAP and other applications/systems and applies various fraud detection techniques and facilitates workflow for the organization.  While not rocket science it is pretty cool to start finding real ways to leverage the power of SIEM tools in areas outside of perimeter security.

That’s all for now as I reflect back on this conference I’ll may update this post with more information. 

Rocky

Part 2 of Decurity’s “Back to School” Series:  SIEM 201: SIEM Use Case Definition 

For the full article click here

Course Prerequisites: A while back I published a diagram and associated text illustrating the benefits of a combined SIEM and Log Management architecture. This diagram/post did a good job of explaining the features and functionality of Log Management and SIEM at a very high level. If you haven’t seen that post or if you haven’t read Decurity’s SIEM 101 previously I would encourage you to go back and take a look. Basic concepts from those resources will help in understanding of Use-Cases and how they apply to SIEM .

Introduction:
In my experience I’ve noticed that SIEM customers use something like 30% of less of the functionality of the tool they bought. That number is actually probably pretty high when you consider the fact that a very high percentage of customers are only using the default content that came pre-installed or was “customized” during a professional services engagement. There are some very advanced users out there, no doubt and this post will help them as well, but it is really focused on providing a framework to advance the majority of SIEM users so they can gain better appreciation for how to maximize the value of their SIEM investment.

The process (and diagram) that follows, outlines how Decurity looks at use-cases related to SIEM. We are providing this information in the hopes that you’ll internalize it as part of your SIEM operations.  Decurity will also be announcing in the very near future an online solution using this methodology so that you can track/update/share your use-cases/solutions - contact us if you’re interested in learning more about that solution.

Use-Case Requirement:
The most simplistic advice I can give is that you should try to focus on the output first.  What is the point of the work effort?  What is the problem we are trying to solve? What is the intended action/output? Who benefits from this and more importantly why do they benefit from this solution?  Then you can move into questions like - what information is required to solve the problem? 

The information provided in this article will help to guide you through the process.  Implementing solutions in your SIEM in an ad-hoc manner will result in failure or at best very temporary and minimalistic gains.  If you don’t believe me you can ask any of the hundreds of organizations who tried it before you.

Decurity’s Use Case Diagram and Model Explanation.

or download as .pdf here: Decurity_siem_usecase.pdf

General:

This is the most basic logistical information related to the use-case and related solution.  It provides a documentation framework.

• Author: Who was involved in the creation/authoring of the solution?
• ID, Version and Date: What is the current version and ID and last date of update.
• Objects, Artifacts: Link to objects (externalized or within solution) used within the solution for example, the configuration objects like report, rules, dashboards, etc. 
• Solution Description: Quick reference to the solution, using categorization that makes sense for your organization. 
• References: Corporate or External documents that act as reference material for your use-case and/or solution. 

Business Justification: This is the problem being addressed from a corporate perspective.  One or more Business problems may apply, but each should be documented in some fashion.

• Business Problem Description: What are the specific problems that need to be addressed?
• Business Owner(s): Who owns the actions for output of the system? Who owns relevant Systems, Applications and Data? Who is requesting Assistance? 
• Business Perspective: Security, Compliance, Risk, Audit, Fraud, Legal, HR, Other?
• Current Solution: Today how is this problem addressed?  How can it be improved?
• Expectations: What is it that the business owners expect from the solution?
• Priority: What is the value of solving this issue, or conversely what is the cost of not solving this issue?

Technical Requirements:

• Need: Active Statements – “The system shall”, “We have to” *(DO SOMETHING)* Define that something.
• Action: Action(s) and/or Output(s) required from the system.
• Actor: Relative to a *(PERSON/TEAM)*
• Event: Specific scenario(s) to be evaluated. 
• Context: Relevant environmental conditions.  How does our knowledge of this environment affect how we can refine the analysis and output?  Some examples of context that should be considered are:  Organizational Structure, Business Units, Application and/or Data Categorizations, Network Segmentation, System Configurations, Users, “Hot Lists”, Vulnerability Data, Data/System/User Criticality, other environment specific information.
• Timing: Within, before, at, during, after.
• Logic: Boolean Logic Statements (T/F) using AND, OR, IF, THEN, NOT as conditions. 

Collection:

• Data Source(s):  What data sources would provide the best context?  What information do we have already available? 
• Data Accessibility: Are there physical, logical, business, technical or political barriers to having the relevant data?
• Data Format: is the data readily comprehended by our solution, is customization of the data necessary or possible?  Do we need to update logging standards?
• Data Relevance: 
o Content: What elements of the data provide us the necessary context? Which exact fields are relevant? 
o Timing: Do we receive it often enough to be relevant to our proposed solution?
• Data Location: Does the data reside in a centralized, easily accessed location?  Is it already aggregated, normalized or filtered in a way that would adversely affect our proposed solution?
Note:  You can and should use these questions and related answers as justification for your enterprise visibility project.  Logging Standards, Data Access and reliable access to the information are very often the proverbial “long pole”. 

Proposed Solution:

• Technology/Process:  Does SIEM make sense to solve this problem, given the data we have, our environment and the proposed solution?  Can we solve this using other technology or processes in a more efficient/effective manner?  SIEM is great, but not always the answer.
• Configuration:  What SIEM configuration(s) provide us with the most efficient and effective solution.  Is it simply a report or do we need to leverage advanced meta-correlation?  Does Statistical evaluation help?  Describe possibilities and known variables/obstacles.  Know the capabilities of your product will help you to understand how to configure it. Advanced Use-Cases, Custom Applications, Fraud Detection, etc require a non-traditional data set and logic approach - well at least non-traditional from the security administrator perspective. Having the flexibility to “compare” against user-defined fields is key to solving those use-cases. If you find yourself unable to solve a number of “Core” use-cases then it might be time to consider training, external advice or as a last resort a new solution.
• Expected Outcome: What is it that we expect to see from the system?  For example (Within “n” Minutes, we should see “x” when “y” occurs.)
• Known False Positive:  How are false positives differentiated from known bad activities and how can we tune our systems/data/environment to reduce the number of valid activities we respond to?
• Known Gaps: Relative to the problem-set described what do we expect that this solution will miss?  How can we close those gaps?
• Alternative Methods:  Within the SIEM or external to SIEM what are alternative ways to address some subset of this problem?  Do related solutions already exist?

QA:

• Performance:  is the solution Efficient?  Does it cause significant system degradation?  Have you built “content” to monitor for efficiency?
• Functionality: Is this providing an acceptable solution for the users and owners?  Are refinements required?
• Measurements: Technical effectiveness, Resource Utilization Measurements.
• Lab Validation:  Were Lab tests meaningful and successful?

Note:  You might get the sense from my wording that QA is an ongoing activity, you’d be correct.  If you lab has irrelevant data/systems your tests are meaningless.  Testing new correlation scenarios against existing data set is invaluable. Knowing how the system is going to respond before you implement into production saves time, effort and headaches.

Operations:

• Feedback:  You need a periodic feedback loop to ensure you are in touch with their needs and updating/planning around upcoming requirements.
• Monitor: Changes are inevitable, from process, people, environment to threats and data sets you will need to stay in touch with how your SIEM is supporting the evolving requirements. 
• Refine: simple refinements may be applied daily/weekly/monthly.
• Enhance: Do we need to add more/better data sets? Is there better Logic that can be applied? Do new or related use-cases offer better insight?
• Validations:  What is the “normal” operation look of this use-case look like and how would you know abnormal behavior of your solution? 

Course Summary:

So it should be clear by now that we think SIEM is a great tool, with tons of potential to identify new activities you couldn’t previously consider and to automate “definable” activities and facilitate workflow.  It should also be obvious that SIEM requires planning, testing and ongoing operational support to be most effective for you and your organization.  This guide and related articles/posts will go a long way to assist you with your efforts.  If not, reach out and we’ll find other ways to help you!

Remember that SIEM is a process not just a tool. If you aren’t making changes to your SIEM on a daily basis (or having someone make changes for you) you are not getting the most from your SIEM. Threats constantly evolve, your networks/systems/data/users are always being modified, your understanding of your environment is always changing, shouldn’t your detection techniques also be enhanced on a daily basis? The more time you spend on use-cases as identified in this post the more value you’ll receive out of your SIEM.

Disclaimer:  Not every vendor solves problems in the same manner. Due to technological differences, wildly varying skills of consultants and comprehension of actual problem and/or data you mileage will vary. That said the approach we are documenting here will work with any SIEM and should be used every time you think about solving new problems using your SIEM. It does mean effort has to be applied, but it also means you will have objective measurements of success when it comes to the value your SIEM is providing.

Just in time for “Back to School” Decurity presents “SIEM 101”: An introduction into SIEM functionality.  What is SIEM correlation? What does it deliver? What is the value to a business or organization?  What is aggregation, normalization, prioritization and how do they differ or enable correlation scenarios?

Every SIEM Vendor seems to have a different definition and marketing spiel about the functionality of SIEM “correlation”.  Some times correlation is described in a manner that evokes thoughts of a magic trick, other times it is simply labeled as “too confusing” and therefore not relevant.  Obviously, this causes confusion and an inconsistent expectations, or should I say anticipation, of the results that correlation will (or won’t) deliver. This results in the prospective customer ending up with a skewed perspective and, in all likelihood dissatisfaction.  On the other hand it may also result in the customer not knowing the full extent of the power the solution makes available to them.  Neither situation benefits anyone involved.  The purpose of this posting is to help describe common SIEM functionality so that current and prospective users of SIEM can effectively compare the capabilities of different vendors purporting to support or deliver “correlation”.

Some Basic SIEM Terminology.  Let’s start by outlining some basic terminology and functionality included in most SIEM solutions to provide some context. After that, we will be able to dive deeper into what is correlation and its related functionality.

Collection: Collection refers to the process of obtaining the logged information from various event sources. The “battle” of agent versus agent-less is meaningless should just be ignored as marketing fluff.  Things like network architecture, Network speed/latency, event source platforms, security, compliance and your environment variables all drive the decision of where is the best place to locate an agent/collector to collect information.  It is simply a matter of your use-cases and environment that drive your deployment architecture decisions. 

Event Sources: These are the devices/systems that generate events for consideration.  Inclusion of the “right” event sources, logging in the “right” way is absolutely critical to the success of your SIEM.  The SIEM can’t consider information that does not exist or is not contextually relevant with other information in the system.  I’ll spend more time on this topic in an upcoming “SIEM 201” blog post.

Normalization: This is the process, at either the collector (agent) or SIEM engine that makes sense of the event data being input into the system. The normalization process tries to map the different log event data formats into a common structure, or taxonomy, or in some cases indices, so that things common fields like names, activity type, timestamps and IP addresses, etc can be quickly compared using a simple taxonomy. Usually this means that the data is more accessible and efficiently stored for the SIEM solution. Each vendor performs this process differently in the background and the level of functionality, intelligence and capabilities associated with the process varies for each vendor, some do it well, some don’t.  Some vendor solutions don’t index/normalize on input into the system, they accomplish this task when the user requests output from the system.

Aggregation: This process summaries (counts) event data, based on (hopefully) flexible pre-defined fields. The purpose of this process is to reduce the event data load, either in terms of network traffic, data storage and/or SIEM engine efficiency.

A typical example of this process can happen if the following situation is detected:
1. “N” number of events
2. That contain the same event characteristics
3. For a given timeframe

In this situation the aggregation process could send one event record with a count inside it, instead of sending all of the individual event records.  A flexible SIEM solution should allow you to decide which fields are leveraged in the aggregation process, allow you to specify the event field characteristics that must be similar, and what information should be included in the summarized event record. The downside to aggregation, if it is incorrectly configured or designed, is loss of important information (i.e. it could cause more Aggravation then Aggregation.).

Thresholding: Some consider thresholding to be correlation.  I consider thresholding to be aggregation with alerting.  “N” events occurred in a sliding time window, then let someone know.  An example of this could be the popular “number of failed logins over a fixed period of time”.

Filtering: This is the ability to ignore, suppress or block certain event records or messages from being processed or displayed. Some consideration is required if you decide to start suppressing messages or event records. It can be a great way to reduce “noise”, but it is also a very good way to lose very important context from “previously unknown” activities. 

Intelligent Filtering is the process by which you forward events from a Log Management device to a SIEM on a per Use-Case basis. Ensuring the full data set is fully searchable and easily available within the overall solution, without overloading the SIEM. Keeps costs down, increases efficiency and enhances solution value.

Simple Prioritization: This is the process of mapping of the message priority, assigned by a particular event source vendor, for an event record to the SIEM’s message priority.
For example, IDS vendor “X” assigns an event with a priority of “1a”. The mapping process takes this value and translates it to the SIEM Vendor’s priority field and assigns a value of “10” which indicates that the priority is “High/Critical”.  All similar events will always have similar priority. This functionality is typically mapped at the agent/collector, but can also be accomplished at the engine depending on the Vendor.

Advanced Prioritization: This is similar to simple prioritization, with the addition of context from the environment or from how SIEM has been configured. This offers more dynamic prioritization model for similar type events. An example is a priority schema that takes into account, current Vulnerability information for a targeted asset. If the target has a relevant vulnerability and a corresponding IDS Event is received, then the priority of the alerts is raised (it is relevant).  On the other hand, if the vulnerability (or system) does not exist, then the priority is reduced to “Informational”, for this particular event. This functionality is typically performed at the SIEM Engine. This is one way to highlight known-bad activity and help prioritize workflow.  Advanced prioritization might be considered a form of very basic correlation by some.

Ok with that in mind, what is Correlation?

As I see it correlation included the evaluation of collected data by using one or more of the following methods:

(1) Pre-defined pattern matching
(2) Statistical analysis (anomaly detection)
(3) Basic conditional Boolean logic statements
(4) Contextually relevant and/or enhanced data set + Boolean logic

Correlation output:  the goal of event correlation is to produce a meaningful ”event of interest” that is intended to create output for use by either other correlation criteria, or to influence and/or directly enable workflow creating actionable output (potential incident identification).

Meaning either
(1) You have a higher degree of confidence that something bad has happened or,
(2) You now know something that you did not or could not know previously.

Additional functions used within Correlation:

Comparison List/Capability:  IP, Subnet, ASN, Domain Names, File Names, MAC Address, User names, Event IDs, Custom Attributes, etc. Being able to dynamically update and/or query these lists with or without Boolean logic allows your correlation scenarios to include “fresh” information all the time. Linking lists allows for even more flexibility in prioritization of events. Events can move between lists based on thresholding or other learned context. Move from suspicious to malicious or from malicious to normal based on how correlation scenarios are defined.  Decurity’s Threat Intelligence Offering keeps these current for you!

SIEM Boolean Logic: True/False and the use of IF, THEN, AND/OR, NOT variables.  This is the process where you articulate your logic statements.  More on this in the “201” blog post coming soon.

Statistical Evaluation: In my mind this is by far the most underutilized component of some SIEM solutions. Anomaly detection, Thresholding and even comparison can be accomplished in a very scalable and in most cases a low overhead manner using the correct set of statistical evaluations. The output of these evaluations can also be “events” for comparison is advanced correlation scenarios.  Expert usage only.

Contextual Comparison: Vulnerability Info, System (Computer or Network Node) Information, Application Information, User Information, or other categorized attributes describing how the network, systems, users, applications or data are used and/or organized.  The more context added to each correlation scenario the more refined (and meaningful) the output will be. In most cases, if accomplished correctly it also means the most efficient use of system resources.  A Simple example could be defining assets with PCI, PII relevance. 

Meta Correlation: Using SIEM enhanced data from previously/currently correlated events to form new correlation scenarios. This can also use the output of Statistical evaluations. The meta-correlation can be between previous correlated events and new event stream data or multiple previous correlated events.  This is also how many systems handle basic scalability or higher tier deployment scenarios.  A baseline of content is deployed at lower tiers and matching events are forwarded upward for inclusion in “enterprise-wide” correlation scenarios. 

Summary: 

Correlation is a very powerful SIEM functions that can help you refine the identification of anomalous or malicious activity.  If your (the customer) can articulate your use-cases clearly, then most vendors can find a way to solve the defined problem using existing functionality within their product set.  It is my hope that you will be able to use this blog post as a way to map the various solution offerings to a common and understandable taxonomy so you can fully comprehend what you are getting with each solution. 

In the next post in this “Back to School” series (SIEM Correlation 201) we’ll talk about Use Case Definitions, Event Sources, Performance Impact, Flexibility and Scalability.

“ring, ring” class dismissed until next week.

-Rocky

Come see Decurity at ArcSight Protect 2009.  We’ll be all over the place!  This is the first conference we’ve sponsored so we’re super excited and thrilled to partner with ArcSight for this event.  Ray, Travis, Paul and Rocky will all be at the conference and we really want to meet with you and hear how Decurity can support your ArcSight requirements.

If you are an ArcSight customer, partner or prospect and you’re not already registered you really should register and be at this conference.  It is the best opportunity to meet everyone at ArcSight, hear about what’s new and what’s coming with ArcSight and to have your opinion heard by everyone! 

Plus you’ll get to see firsthand how Decurity “enhances” ArcSight Solutions!

References: Sponsor Page: http://www.arcsight.com/protect09/sponsors/ ArcSight Protect 09: http://www.arcsight.com/protect09

Recent vendor press releases by NitroSecurity and NetWitness highlight the evolving requirement for full network packet collection, indexing and reconstruction for analysis.  These products and others (including Solera Networks) illustrate an emerging market in total network awareness.  Working in conjunction with Log Management (LogLogic, Splunk, ArcSight Logger, etc) and SIEM tools (RSA, EiQNetworks and of course ArcSight ESM) these tools provide invaluable insight into your network’s behavior (not to mention the behavior of individual users and applications over the network).  NitroSecurity updated their capabilities to include what they term as “content aware SIEM” and NetWitness announced a milestone of 15,000 active users.  Both press releases highlighted quotes from Decurity, which we appreciate, but more important to us, the emergence and rapid growth of this market segment add further credibility to Security Professionals having all of the right tools and information available.  Recent news about DHS Einstein and NSA Tutelage technologies also point towards an increased trend in better, more capable Collection tools.

Security Operations and Incident Response capabilities can’t continue to function in the dark and be expected to adequately protect the enterprise.  We need to make all of the applicable information available and apply intelligent analytical techniques against the data set so that we can more rapidly and accurately identify risks to the enterprise.  These tools when used properly can reduce analytical time required to identify incidents into time segments measured in seconds and can help understand the scope of the incident much more rapidly.  You can review the artifacts (documents, files, audio, video, web, email, chat, as well as interactive sessions (ftp, telnet, ssh, etc)) instantly and determine the legitimacy of the session.  You can extract information and search log management/SIEM for related events and set up alerts and workflow along the way.  All in a matter of clicks.  Of course you can accomplish the reverse and search for anomalies identified in SIEM/Log Management or IDS/IPS in your Network Awareness tool and understand quickly what occurred.  With this level of information available to you, the limitations of the they of analysis have more to do with the level of expertise of the user/analyst than the information.

These use of these tools in the right hands allow for much more than just security “alerts” and incident identification.  They lend themselves to true security convergence concepts and overall enterprise intelligence and security operations.  More on those concepts over the next few months.

References:
NetWitness “July 27, 2009 | Security Experts Worldwide Rely Upon NetWitness® Investigator ” Link: http://www.netwitness.com/resources/pressreleases/Jul272009.aspx

NitroSecurity “NitroSecurity Heightens Enterprise Security Information Management with Real-Time Application Content and Protocol Analysis” Link: http://www.nitrosecurity.com/information/news/pr/2009/20090722.psp

Decurity Blog:  Dec 2008:  http://blog.decurity.com/index.php/dec_template/more/netwitness_investigator_summary_1/

Today Ellen Nakashima of The Washington Post published an article about DHS USCERT, NSA and Telecommunications providers collaborating to monitor Civilian Agency Internet traffic using DHS’s planned Einstein 3 tool to help defend these civilian government entities.  The article correctly illustrates that NSA has the expertise and tools like Tutelage to know more about the context of the attacks.  It also states that DHS has the authorization to monitor using Einstein (enforced by the TIC program).  If you’ll remember a while back I talked about Trusted Internet Connection (TIC) and its role in consolidating Internet points of presence and providing chokepoints to monitor and defend for the government.
For reference see:  http://blog.decurity.com/index.php/dec_template/more/dhs_einstein_tic_overview/    and   http://blog.decurity.com/index.php/dec_template/more/dhs_blog_round_table/

In short, TIC mandated government agencies to meet very stringent requirements in order to become a TICAP (provider) or use pre-approved TICAP’s (Telecom or other Agency) for all Internet traffic.  The monitoring capabilities of these TIC’s is referenced in my earlier posts, but let’s just say its EVERYTHING.  Not that I’m complaining, from a capabilities perspective I think NSA and Cyber Command should be making the most out of this information to help protect the government and as Richard Bejtlich speculates eventually “.com” .  NSA has the expertise and intelligence data while DHS has the authorization to monitor, the framework to force everyone to play (TIC) and a toolset that is evolving (Einstein v2 is still being rolled out, v3 is in development) On a side note, I do have to wonder why the government isn’t using more capable tools like NetWitness or Solera in conjunction with NSA tools and building a META SIEM to incorporate Intelligence feeds, but that’s a topic for a later post. 

My biggest question is this…. I wonder how US-CERT and NSA are going to collaborate more effectively -  Is Einstein raw data going to be handled by NSA, if so what’s the point of US-CERT in the future?

Should be interesting to see what happens once the cyber czar is appointed, from what I can tell his/her kingdom has already layed a very clear path forward, the czar may simply be along for the ride while NSA drives over everyone else.

Update 1: (3 July 2009; 0930 EDT) SIOBHAN GORMAN of The Wall Street Journal also has an article on this topic “Troubles Plague Cyberspy Defense” .  In this article takes more conservative approach in describing what is happening across government with regards to consolidated monitoring.  According to the article Einstein v3 will be updated/rebuilt to more closely align with NSA Tutelage and is at least 18 months out.  The idea is that it would start to develop full packet inspection capabilities (Like NetWitness, Solera and a few others).

My Notes:  If this perspective is more accurate it seems US-CERT would monitor using technology enabled by NSA, instead of NSA accomplishing the monitoring.  IMHO - From what I’ve seen certain executive layers at DHS have not enabled the US-CERT to be effective enough to actually function as a true analytical center, even though USCERT has some very good people capable of executing on that misson.  In fact, I’d go as far as to say DHS is at risk of losing key staff if they don’t figure out a better way to enable their team.  The place is known as a revolving door for a reason, the people they hire are very capable and motivated, the organization itself may not be best suited for that expertise and vision. 

Earlier this month The 451 Group released a Market Insight Service Impact Report about Decurity.  Wow.  We are flattered that The 451 Group (Nick Selby and his team) took the time to to understand what our team is doing at Decurity.  From the phone calls, tweets and emails I’ve received the readership of this report is pretty amazing.  It’s an honor to have a respected entity like the 451Group take notice of our baby.  Paul and I (and the entire Decurity team) are sincere in our appreciation of all of those that have taken the time to learn about our team and our vision.  I can’t even begin to express my appreciation for the trust and partnership that our clients have extended to Decurity over the last two years.  We have amazing customers, partners and friends and we hope that we can continue to earn the trust and respect for the next several years! 

I will say this - while we are very proud of what we’ve accomplished so far, there is a lot more coming from Decurity in the very near future - keep an eye (or RSS reader) our for more about us! 

UPDATE:  We’ve secured reprint rights and will post the document on our soon to be updated webpage in the coming days.  Link to the 451 MIS Impact Report on Decurity:  451_Group_Decurity_IMPACT_REPORT_-_10_June_2009.pdf

Updated 15 Jan 2010 (see bottom)
Gartner 2009 SIEM Magic Quadrant - It’s pretty easy to digest - Everyone is in the leader quadrant or is a “leader” of some sort.  No vendor has any “real” negatives, they will all work in nearly any situation, everything is peachy.  You can register with many of the vendors to receive your copy of the Gartner Magic Quadrant for SIEM 2009 report.  A few of those links are provided here:  Q1, ArcSight, LogLogic , RSA

My obviously frustrated opinion:
If you are an executive, director, manager, technician or otherwise breath air and you use this report to make any sort of determination about SIEM vendor qualifications you should probably be seeking professional career advice.  This report is great at providing the SIEM vendors with additional fodder to confuse the sales prospects, but it does absolutely nothing for the reader seeking information about which SIEM might fit their environment.  This is the one time that I don’t blame the vendors, they have to play the game.  The game is to mitigate the effect of competitive “spin” based on the results of the report by the “leaders” listed within the report.  Along those lines I won’t use this post to congratulate or bash vendors as to their relative position in the quadrant because it’s just insane to justify anything about this report.

Every year, I review these SIEM reports and find myself hoping that the next issue will reveal something insightful or even slightly meaningful.  I scrub the “cautions” looking for anything that points to a material technical weakness in the technology and usually the most meaningful thing I find is a veiled “feature request” for some trivial item.  A review of the “strengths” of each organization shows the points to be at best highly subjective and usually just completely irrelevant.  I always leave frustrated, but hopeful that the next version will set things right. 

Gartner isn’t alone in it’s pursuit of mediocrity here.  Most (not all) of the “analyst” firms and industry magazines offer a strikingly similar lack of useful information in their reports.  Please note: this is in no way a personal attack on any author or company, it’s a rant against crappy information as a whole.  Over the years I’ve met most of the reviewers and they “seem to get it” in person, it’s just the nature of these “ranking without context” reports that simply kills the value of any insight the authors might have tried to present.  The 451Group and a few others have tried to buck that trend over the years and are making some progress, but despite their efforts the overall industry standard is still too watered down to be of any real value. 

I know I’m not endearing myself to the analyst community right now, and I expect certain vendors won’t appreciate what I’m saying but bear with me here.  I think we can make this better and everyone can benefit.  As an industry we must start expecting better from our information providers.  We need to provide specific feedback about what information these reports should provide in order to be meaningful.  I have tried to influence better context and more meaningful technical criteria through several older blog posts and through conversations with anyone that will listen.  I’ll step up my game and offer even more direct advice in the coming months - I’m just asking that everyone do the same.  Let’s encourage our information providers to pursue a higher standard. 

Maybe, next year… Hey I’m a Cubs fan - There is always next year! 

EDIT - UPDATE-1 15 Jan 2010

So after I posted this blog I pretty much went on with life.  I did run into some Gartner folks at their IT Security Forum in DC right after this post and learned alot more about how Gartner positions these MQ offerings.  I’ve got to say that with the right perspective in place I’m much more agreeable to what they are trying to do from a macro perspective.  They do offer more detailed analytical documents (critical capabilities) and they do push for more detailed requirements gathering for buyers that send them inquiries.  I was expecting something else and really in the end that was the problem, my expectations were misaligned with the intention of this MQ.  So to Mark Nicolett I apologize for not taking the time to really dig deeper into your approach.  I do respect what you are doing and my offer to help still applies.

For those seeking more information about the Gartner SIEM MQ - Mark Nicolett’s guest blog post explains the process very well—> http://blogs.gartner.com/john_pescatore/2009/06/15/guest-blogger-mark-nicolett-and-the-siem-market/