ArcSight Protect ‘09 was a whirlwind of activity for Decurity.  I would love to thank everyone that came up to the booth and gave us feedback on the blog, to all of our customers that stopped by and helped introduce us to their friends and of course to all my friends at ArcSight that made the week so enjoyable. 

Technology advances announced as part of Protect ‘09:

1. ArcSight Logger 4.0 While still technically in Beta, this product goes a long way to resolving any perceived flaws in the technology.  Unstructured search, incredible insert rates, better and much fast reporting, direct integration with ESM Console.  We got to spend some significant time with 4.0 and we were really impressed with the ability to just take data no matter how ugly it was into the system and deal with it very effectively.  A live demo conducted during the 2nd day keynote confirmed that the speed was incredible.  The fixes under the covers to how the system handles I/O means that not even RAID 5 slows down Logger.  The implications are huge!  Insert rates are just ridiculous, they prove 100K EPS on very basic hardware.  There is some pretty cool pixie dust in those appliances.  Now if we could just get them into VM’s or AMI’s…..

2. ArcSight FraudView:  This type of application integration purpose built solutions helps extend ArcSight ESM as a platform to look beyond Security in the enterprise.  Moving out of pure security thought processes and into solving core business problems is exactly what Use-Cases are all about.  This use-case took information from SAP and other applications/systems and applies various fraud detection techniques and facilitates workflow for the organization.  While not rocket science it is pretty cool to start finding real ways to leverage the power of SIEM tools in areas outside of perimeter security.

That’s all for now as I reflect back on this conference I’ll may update this post with more information. 

Rocky

Part 2 of Decurity’s “Back to School” Series:  SIEM 201: SIEM Use Case Definition 

For the full article click here

Course Prerequisites: A while back I published a diagram and associated text illustrating the benefits of a combined SIEM and Log Management architecture. This diagram/post did a good job of explaining the features and functionality of Log Management and SIEM at a very high level. If you haven’t seen that post or if you haven’t read Decurity’s SIEM 101 previously I would encourage you to go back and take a look. Basic concepts from those resources will help in understanding of Use-Cases and how they apply to SIEM .

Introduction:
In my experience I’ve noticed that SIEM customers use something like 30% of less of the functionality of the tool they bought. That number is actually probably pretty high when you consider the fact that a very high percentage of customers are only using the default content that came pre-installed or was “customized” during a professional services engagement. There are some very advanced users out there, no doubt and this post will help them as well, but it is really focused on providing a framework to advance the majority of SIEM users so they can gain better appreciation for how to maximize the value of their SIEM investment.

The process (and diagram) that follows, outlines how Decurity looks at use-cases related to SIEM. We are providing this information in the hopes that you’ll internalize it as part of your SIEM operations.  Decurity will also be announcing in the very near future an online solution using this methodology so that you can track/update/share your use-cases/solutions - contact us if you’re interested in learning more about that solution.

More...

Just in time for “Back to School” Decurity presents “SIEM 101”: An introduction into SIEM functionality.  What is SIEM correlation? What does it deliver? What is the value to a business or organization?  What is aggregation, normalization, prioritization and how do they differ or enable correlation scenarios?

Every SIEM Vendor seems to have a different definition and marketing spiel about the functionality of SIEM “correlation”.  Some times correlation is described in a manner that evokes thoughts of a magic trick, other times it is simply labeled as “too confusing” and therefore not relevant.  Obviously, this causes confusion and an inconsistent expectations, or should I say anticipation, of the results that correlation will (or won’t) deliver. This results in the prospective customer ending up with a skewed perspective and, in all likelihood dissatisfaction.  On the other hand it may also result in the customer not knowing the full extent of the power the solution makes available to them.  Neither situation benefits anyone involved.  The purpose of this posting is to help describe common SIEM functionality so that current and prospective users of SIEM can effectively compare the capabilities of different vendors purporting to support or deliver “correlation”.

Some Basic SIEM Terminology.  Let’s start by outlining some basic terminology and functionality included in most SIEM solutions to provide some context. After that, we will be able to dive deeper into what is correlation and its related functionality.

Collection: Collection refers to the process of obtaining the logged information from various event sources. The “battle” of agent versus agent-less is meaningless should just be ignored as marketing fluff.  Things like network architecture, Network speed/latency, event source platforms, security, compliance and your environment variables all drive the decision of where is the best place to locate an agent/collector to collect information.  It is simply a matter of your use-cases and environment that drive your deployment architecture decisions. 

Event Sources: These are the devices/systems that generate events for consideration.  Inclusion of the “right” event sources, logging in the “right” way is absolutely critical to the success of your SIEM.  The SIEM can’t consider information that does not exist or is not contextually relevant with other information in the system.  I’ll spend more time on this topic in an upcoming “SIEM 201” blog post.

Normalization: This is the process, at either the collector (agent) or SIEM engine that makes sense of the event data being input into the system. The normalization process tries to map the different log event data formats into a common structure, or taxonomy, or in some cases indices, so that things common fields like names, activity type, timestamps and IP addresses, etc can be quickly compared using a simple taxonomy. Usually this means that the data is more accessible and efficiently stored for the SIEM solution. Each vendor performs this process differently in the background and the level of functionality, intelligence and capabilities associated with the process varies for each vendor, some do it well, some don’t.  Some vendor solutions don’t index/normalize on input into the system, they accomplish this task when the user requests output from the system.

Aggregation: This process summaries (counts) event data, based on (hopefully) flexible pre-defined fields. The purpose of this process is to reduce the event data load, either in terms of network traffic, data storage and/or SIEM engine efficiency.

A typical example of this process can happen if the following situation is detected:
1. “N” number of events
2. That contain the same event characteristics
3. For a given timeframe

In this situation the aggregation process could send one event record with a count inside it, instead of sending all of the individual event records.  A flexible SIEM solution should allow you to decide which fields are leveraged in the aggregation process, allow you to specify the event field characteristics that must be similar, and what information should be included in the summarized event record. The downside to aggregation, if it is incorrectly configured or designed, is loss of important information (i.e. it could cause more Aggravation then Aggregation.).

Thresholding: Some consider thresholding to be correlation.  I consider thresholding to be aggregation with alerting.  “N” events occurred in a sliding time window, then let someone know.  An example of this could be the popular “number of failed logins over a fixed period of time”.

Filtering: This is the ability to ignore, suppress or block certain event records or messages from being processed or displayed. Some consideration is required if you decide to start suppressing messages or event records. It can be a great way to reduce “noise”, but it is also a very good way to lose very important context from “previously unknown” activities. 

Intelligent Filtering is the process by which you forward events from a Log Management device to a SIEM on a per Use-Case basis. Ensuring the full data set is fully searchable and easily available within the overall solution, without overloading the SIEM. Keeps costs down, increases efficiency and enhances solution value.

Simple Prioritization: This is the process of mapping of the message priority, assigned by a particular event source vendor, for an event record to the SIEM’s message priority.
For example, IDS vendor “X” assigns an event with a priority of “1a”. The mapping process takes this value and translates it to the SIEM Vendor’s priority field and assigns a value of “10” which indicates that the priority is “High/Critical”.  All similar events will always have similar priority. This functionality is typically mapped at the agent/collector, but can also be accomplished at the engine depending on the Vendor.

Advanced Prioritization: This is similar to simple prioritization, with the addition of context from the environment or from how SIEM has been configured. This offers more dynamic prioritization model for similar type events. An example is a priority schema that takes into account, current Vulnerability information for a targeted asset. If the target has a relevant vulnerability and a corresponding IDS Event is received, then the priority of the alerts is raised (it is relevant).  On the other hand, if the vulnerability (or system) does not exist, then the priority is reduced to “Informational”, for this particular event. This functionality is typically performed at the SIEM Engine. This is one way to highlight known-bad activity and help prioritize workflow.  Advanced prioritization might be considered a form of very basic correlation by some.

Ok with that in mind, what is Correlation?

As I see it correlation included the evaluation of collected data by using one or more of the following methods:

(1) Pre-defined pattern matching
(2) Statistical analysis (anomaly detection)
(3) Basic conditional Boolean logic statements
(4) Contextually relevant and/or enhanced data set + Boolean logic

Correlation output:  the goal of event correlation is to produce a meaningful ”event of interest” that is intended to create output for use by either other correlation criteria, or to influence and/or directly enable workflow creating actionable output (potential incident identification).

Meaning either
(1) You have a higher degree of confidence that something bad has happened or,
(2) You now know something that you did not or could not know previously.

Additional functions used within Correlation:

Comparison List/Capability:  IP, Subnet, ASN, Domain Names, File Names, MAC Address, User names, Event IDs, Custom Attributes, etc. Being able to dynamically update and/or query these lists with or without Boolean logic allows your correlation scenarios to include “fresh” information all the time. Linking lists allows for even more flexibility in prioritization of events. Events can move between lists based on thresholding or other learned context. Move from suspicious to malicious or from malicious to normal based on how correlation scenarios are defined.  Decurity’s Threat Intelligence Offering keeps these current for you!

SIEM Boolean Logic: True/False and the use of IF, THEN, AND/OR, NOT variables.  This is the process where you articulate your logic statements.  More on this in the “201” blog post coming soon.

Statistical Evaluation: In my mind this is by far the most underutilized component of some SIEM solutions. Anomaly detection, Thresholding and even comparison can be accomplished in a very scalable and in most cases a low overhead manner using the correct set of statistical evaluations. The output of these evaluations can also be “events” for comparison is advanced correlation scenarios.  Expert usage only.

Contextual Comparison: Vulnerability Info, System (Computer or Network Node) Information, Application Information, User Information, or other categorized attributes describing how the network, systems, users, applications or data are used and/or organized.  The more context added to each correlation scenario the more refined (and meaningful) the output will be. In most cases, if accomplished correctly it also means the most efficient use of system resources.  A Simple example could be defining assets with PCI, PII relevance. 

Meta Correlation: Using SIEM enhanced data from previously/currently correlated events to form new correlation scenarios. This can also use the output of Statistical evaluations. The meta-correlation can be between previous correlated events and new event stream data or multiple previous correlated events.  This is also how many systems handle basic scalability or higher tier deployment scenarios.  A baseline of content is deployed at lower tiers and matching events are forwarded upward for inclusion in “enterprise-wide” correlation scenarios. 

Summary: 

Correlation is a very powerful SIEM functions that can help you refine the identification of anomalous or malicious activity.  If your (the customer) can articulate your use-cases clearly, then most vendors can find a way to solve the defined problem using existing functionality within their product set.  It is my hope that you will be able to use this blog post as a way to map the various solution offerings to a common and understandable taxonomy so you can fully comprehend what you are getting with each solution. 

In the next post in this “Back to School” series (SIEM Correlation 201) we’ll talk about Use Case Definitions, Event Sources, Performance Impact, Flexibility and Scalability.

“ring, ring” class dismissed until next week.

-Rocky

Come see Decurity at ArcSight Protect 2009.  We’ll be all over the place!  This is the first conference we’ve sponsored so we’re super excited and thrilled to partner with ArcSight for this event.  Ray, Travis, Paul and Rocky will all be at the conference and we really want to meet with you and hear how Decurity can support your ArcSight requirements.

If you are an ArcSight customer, partner or prospect and you’re not already registered you really should register and be at this conference.  It is the best opportunity to meet everyone at ArcSight, hear about what’s new and what’s coming with ArcSight and to have your opinion heard by everyone! 

Plus you’ll get to see firsthand how Decurity “enhances” ArcSight Solutions!

References: Sponsor Page: http://www.arcsight.com/protect09/sponsors/ ArcSight Protect 09: http://www.arcsight.com/protect09

Recent vendor press releases by NitroSecurity and NetWitness highlight the evolving requirement for full network packet collection, indexing and reconstruction for analysis.  These products and others (including Solera Networks) illustrate an emerging market in total network awareness.  Working in conjunction with Log Management (LogLogic, Splunk, ArcSight Logger, etc) and SIEM tools (RSA, EiQNetworks and of course ArcSight ESM) these tools provide invaluable insight into your network’s behavior (not to mention the behavior of individual users and applications over the network).  NitroSecurity updated their capabilities to include what they term as “content aware SIEM” and NetWitness announced a milestone of 15,000 active users.  Both press releases highlighted quotes from Decurity, which we appreciate, but more important to us, the emergence and rapid growth of this market segment add further credibility to Security Professionals having all of the right tools and information available.  Recent news about DHS Einstein and NSA Tutelage technologies also point towards an increased trend in better, more capable Collection tools.

Security Operations and Incident Response capabilities can’t continue to function in the dark and be expected to adequately protect the enterprise.  We need to make all of the applicable information available and apply intelligent analytical techniques against the data set so that we can more rapidly and accurately identify risks to the enterprise.  These tools when used properly can reduce analytical time required to identify incidents into time segments measured in seconds and can help understand the scope of the incident much more rapidly.  You can review the artifacts (documents, files, audio, video, web, email, chat, as well as interactive sessions (ftp, telnet, ssh, etc)) instantly and determine the legitimacy of the session.  You can extract information and search log management/SIEM for related events and set up alerts and workflow along the way.  All in a matter of clicks.  Of course you can accomplish the reverse and search for anomalies identified in SIEM/Log Management or IDS/IPS in your Network Awareness tool and understand quickly what occurred.  With this level of information available to you, the limitations of the they of analysis have more to do with the level of expertise of the user/analyst than the information.

These use of these tools in the right hands allow for much more than just security “alerts” and incident identification.  They lend themselves to true security convergence concepts and overall enterprise intelligence and security operations.  More on those concepts over the next few months.

References:
NetWitness “July 27, 2009 | Security Experts Worldwide Rely Upon NetWitness® Investigator ” Link: http://www.netwitness.com/resources/pressreleases/Jul272009.aspx

NitroSecurity “NitroSecurity Heightens Enterprise Security Information Management with Real-Time Application Content and Protocol Analysis” Link: http://www.nitrosecurity.com/information/news/pr/2009/20090722.psp

Decurity Blog:  Dec 2008:  http://blog.decurity.com/index.php/dec_template/more/netwitness_investigator_summary_1/

Network World’s recent article provides additional evidence that Log Management and SIEM Vendors are still trying to evolve.

Recently, Log Management and SIEM vendors have spent a lot of time updating/fixing their products.  Over the past few months some vendors have quietly passed over other solutions in terms of market relevance and certainly the door has been opened to a whole bunch of upstarts trying to make a name for themselves.  While the majority of Log Management and SIEM business (and therefore product direction) is driven by compliance activities, I appreciate the forward movement towards enterprise security that many in the field are trying to make.  The initial execution on that product vision I’m seeing from many of the vendors this year is very welcome.  IMHO the entire space had gotten very stale with the big guys mainly focusing on compliance appliances or playing feature catch-up with one another.  Here’s my summary of what’s going on in SIEM and Log Management so far in 2009.

The following notional diagram provides some basic recommendations to consider when deploying and managing Log Management and SIEM systems together.  A well-maintained Log Management and SIEM deployment can significantly reduce the time to Incident Identification and really enhance your overall information security capability.  The diagram attempts to illustrate that all information from the Event Sources are processed through the appropriate Log Collection Mechanism and then forwarded to the Log Management System. The Log Management system eats, stores and can regurgitate everything put into it.  The Log Management Solution also can further refine the data set and forward only applicable events for analysis to the correlation engine (SIEM) through the use of intelligent “tagging” of events.  Overall data reduction is only part of the end goal, more importantly we want to ensure the right data is forwarded and evaluated so that we can gain from the overall efficiencies offered by the SIEM.  In short we’re ensuring the system has the correct information available to it so that it can respond to the questions you want to ask of it and reduce the garbage as much as possible.

Decurity’s new subscription offering works with your organization to understand your requirements and then supplies the necessary configurations and Log Management and SIEM data elements (Intelligent forwarding, correlation, reports, etc) to make this model work for you.

Decurity often has the opportunity to our customers find the right Log Management and/or SIEM solution.  We are honored that our customers trust us with that very important question so we wanted to take a moment and explain our requirements gathering/documentation process for vendor selection and hope that our explanation helps a few of more folks out there!  We also get asked by Vendors on how they can improve their products, but that’s a entirely different blog post. 

The Decurity team has been incredibly busy over the last few months cooking up new and more cost effective ways to support our mantra of “keeping security simple”.  We’re getting ready to introduce a “game changer” for our Log Management and SIEM customers.  Our newest offering will be fully described in the coming days but here is a preview of the new subscription services Decurity is offering to our clients. 

Shameless self-promotion follows:  Rocky DeStefano is presenting on SIM at the 2009 US Department of Justice Cyber Security Conference in Washington DC on 14 Jan 2009.