There are a lot of major changes going on at Decurity and soon enough we’ll be in a position to announce them to the world!  In the mean time this is just a quick note to say that Rocky DeStefano will be participating in a couple of fun information security events in the near future:

1.  NetWitness User Conference Nov 4-5 2009 in DC, Gabe Martinez and I are teaming up again and presenting some real-world examples of SIEM and NetWitness integrations in a technical training session on Nov 4th.  This integration is probably one of the most powerful enhancements you can make to a security operations center, come see it for yourself.  If you haven’t already registered, please do so - http://www.netwitness.com/userconference.html

2. SANS What Works in Incident Detection Summit Dec 9-10 2009 in DC.  I’ll be moderating a panel discussion on internal CIRT’s and MSSP’s.  This panel should just be serious fun all around.  For more information see http://www.sans.org/incident-detection-summit-2009/agenda.php and http://taosecurity.blogspot.com/2009/08/sans-incident-detection-summit-in-dc-in.html

When the rest of my dance card fills up I’ll update this post. 

ArcSight Protect ‘09 was a whirlwind of activity for Decurity.  I would love to thank everyone that came up to the booth and gave us feedback on the blog, to all of our customers that stopped by and helped introduce us to their friends and of course to all my friends at ArcSight that made the week so enjoyable. 

Technology advances announced as part of Protect ‘09:

1. ArcSight Logger 4.0 While still technically in Beta, this product goes a long way to resolving any perceived flaws in the technology.  Unstructured search, incredible insert rates, better and much fast reporting, direct integration with ESM Console.  We got to spend some significant time with 4.0 and we were really impressed with the ability to just take data no matter how ugly it was into the system and deal with it very effectively.  A live demo conducted during the 2nd day keynote confirmed that the speed was incredible.  The fixes under the covers to how the system handles I/O means that not even RAID 5 slows down Logger.  The implications are huge!  Insert rates are just ridiculous, they prove 100K EPS on very basic hardware.  There is some pretty cool pixie dust in those appliances.  Now if we could just get them into VM’s or AMI’s…..

2. ArcSight FraudView:  This type of application integration purpose built solutions helps extend ArcSight ESM as a platform to look beyond Security in the enterprise.  Moving out of pure security thought processes and into solving core business problems is exactly what Use-Cases are all about.  This use-case took information from SAP and other applications/systems and applies various fraud detection techniques and facilitates workflow for the organization.  While not rocket science it is pretty cool to start finding real ways to leverage the power of SIEM tools in areas outside of perimeter security.

That’s all for now as I reflect back on this conference I’ll may update this post with more information. 

Rocky

Part 2 of Decurity’s “Back to School” Series:  SIEM 201: SIEM Use Case Definition 

For the full article click here

Course Prerequisites: A while back I published a diagram and associated text illustrating the benefits of a combined SIEM and Log Management architecture. This diagram/post did a good job of explaining the features and functionality of Log Management and SIEM at a very high level. If you haven’t seen that post or if you haven’t read Decurity’s SIEM 101 previously I would encourage you to go back and take a look. Basic concepts from those resources will help in understanding of Use-Cases and how they apply to SIEM .

Introduction:
In my experience I’ve noticed that SIEM customers use something like 30% of less of the functionality of the tool they bought. That number is actually probably pretty high when you consider the fact that a very high percentage of customers are only using the default content that came pre-installed or was “customized” during a professional services engagement. There are some very advanced users out there, no doubt and this post will help them as well, but it is really focused on providing a framework to advance the majority of SIEM users so they can gain better appreciation for how to maximize the value of their SIEM investment.

The process (and diagram) that follows, outlines how Decurity looks at use-cases related to SIEM. We are providing this information in the hopes that you’ll internalize it as part of your SIEM operations.  Decurity will also be announcing in the very near future an online solution using this methodology so that you can track/update/share your use-cases/solutions - contact us if you’re interested in learning more about that solution.

More...

Just in time for “Back to School” Decurity presents “SIEM 101”: An introduction into SIEM functionality.  What is SIEM correlation? What does it deliver? What is the value to a business or organization?  What is aggregation, normalization, prioritization and how do they differ or enable correlation scenarios?

Every SIEM Vendor seems to have a different definition and marketing spiel about the functionality of SIEM “correlation”.  Some times correlation is described in a manner that evokes thoughts of a magic trick, other times it is simply labeled as “too confusing” and therefore not relevant.  Obviously, this causes confusion and an inconsistent expectations, or should I say anticipation, of the results that correlation will (or won’t) deliver. This results in the prospective customer ending up with a skewed perspective and, in all likelihood dissatisfaction.  On the other hand it may also result in the customer not knowing the full extent of the power the solution makes available to them.  Neither situation benefits anyone involved.  The purpose of this posting is to help describe common SIEM functionality so that current and prospective users of SIEM can effectively compare the capabilities of different vendors purporting to support or deliver “correlation”.

Some Basic SIEM Terminology.  Let’s start by outlining some basic terminology and functionality included in most SIEM solutions to provide some context. After that, we will be able to dive deeper into what is correlation and its related functionality.

Collection: Collection refers to the process of obtaining the logged information from various event sources. The “battle” of agent versus agent-less is meaningless should just be ignored as marketing fluff.  Things like network architecture, Network speed/latency, event source platforms, security, compliance and your environment variables all drive the decision of where is the best place to locate an agent/collector to collect information.  It is simply a matter of your use-cases and environment that drive your deployment architecture decisions. 

Event Sources: These are the devices/systems that generate events for consideration.  Inclusion of the “right” event sources, logging in the “right” way is absolutely critical to the success of your SIEM.  The SIEM can’t consider information that does not exist or is not contextually relevant with other information in the system.  I’ll spend more time on this topic in an upcoming “SIEM 201” blog post.

Normalization: This is the process, at either the collector (agent) or SIEM engine that makes sense of the event data being input into the system. The normalization process tries to map the different log event data formats into a common structure, or taxonomy, or in some cases indices, so that things common fields like names, activity type, timestamps and IP addresses, etc can be quickly compared using a simple taxonomy. Usually this means that the data is more accessible and efficiently stored for the SIEM solution. Each vendor performs this process differently in the background and the level of functionality, intelligence and capabilities associated with the process varies for each vendor, some do it well, some don’t.  Some vendor solutions don’t index/normalize on input into the system, they accomplish this task when the user requests output from the system.

Aggregation: This process summaries (counts) event data, based on (hopefully) flexible pre-defined fields. The purpose of this process is to reduce the event data load, either in terms of network traffic, data storage and/or SIEM engine efficiency.

A typical example of this process can happen if the following situation is detected:
1. “N” number of events
2. That contain the same event characteristics
3. For a given timeframe

In this situation the aggregation process could send one event record with a count inside it, instead of sending all of the individual event records.  A flexible SIEM solution should allow you to decide which fields are leveraged in the aggregation process, allow you to specify the event field characteristics that must be similar, and what information should be included in the summarized event record. The downside to aggregation, if it is incorrectly configured or designed, is loss of important information (i.e. it could cause more Aggravation then Aggregation.).

Thresholding: Some consider thresholding to be correlation.  I consider thresholding to be aggregation with alerting.  “N” events occurred in a sliding time window, then let someone know.  An example of this could be the popular “number of failed logins over a fixed period of time”.

Filtering: This is the ability to ignore, suppress or block certain event records or messages from being processed or displayed. Some consideration is required if you decide to start suppressing messages or event records. It can be a great way to reduce “noise”, but it is also a very good way to lose very important context from “previously unknown” activities. 

Intelligent Filtering is the process by which you forward events from a Log Management device to a SIEM on a per Use-Case basis. Ensuring the full data set is fully searchable and easily available within the overall solution, without overloading the SIEM. Keeps costs down, increases efficiency and enhances solution value.

Simple Prioritization: This is the process of mapping of the message priority, assigned by a particular event source vendor, for an event record to the SIEM’s message priority.
For example, IDS vendor “X” assigns an event with a priority of “1a”. The mapping process takes this value and translates it to the SIEM Vendor’s priority field and assigns a value of “10” which indicates that the priority is “High/Critical”.  All similar events will always have similar priority. This functionality is typically mapped at the agent/collector, but can also be accomplished at the engine depending on the Vendor.

Advanced Prioritization: This is similar to simple prioritization, with the addition of context from the environment or from how SIEM has been configured. This offers more dynamic prioritization model for similar type events. An example is a priority schema that takes into account, current Vulnerability information for a targeted asset. If the target has a relevant vulnerability and a corresponding IDS Event is received, then the priority of the alerts is raised (it is relevant).  On the other hand, if the vulnerability (or system) does not exist, then the priority is reduced to “Informational”, for this particular event. This functionality is typically performed at the SIEM Engine. This is one way to highlight known-bad activity and help prioritize workflow.  Advanced prioritization might be considered a form of very basic correlation by some.

Ok with that in mind, what is Correlation?

As I see it correlation included the evaluation of collected data by using one or more of the following methods:

(1) Pre-defined pattern matching
(2) Statistical analysis (anomaly detection)
(3) Basic conditional Boolean logic statements
(4) Contextually relevant and/or enhanced data set + Boolean logic

Correlation output:  the goal of event correlation is to produce a meaningful ”event of interest” that is intended to create output for use by either other correlation criteria, or to influence and/or directly enable workflow creating actionable output (potential incident identification).

Meaning either
(1) You have a higher degree of confidence that something bad has happened or,
(2) You now know something that you did not or could not know previously.

Additional functions used within Correlation:

Comparison List/Capability:  IP, Subnet, ASN, Domain Names, File Names, MAC Address, User names, Event IDs, Custom Attributes, etc. Being able to dynamically update and/or query these lists with or without Boolean logic allows your correlation scenarios to include “fresh” information all the time. Linking lists allows for even more flexibility in prioritization of events. Events can move between lists based on thresholding or other learned context. Move from suspicious to malicious or from malicious to normal based on how correlation scenarios are defined.  Decurity’s Threat Intelligence Offering keeps these current for you!

SIEM Boolean Logic: True/False and the use of IF, THEN, AND/OR, NOT variables.  This is the process where you articulate your logic statements.  More on this in the “201” blog post coming soon.

Statistical Evaluation: In my mind this is by far the most underutilized component of some SIEM solutions. Anomaly detection, Thresholding and even comparison can be accomplished in a very scalable and in most cases a low overhead manner using the correct set of statistical evaluations. The output of these evaluations can also be “events” for comparison is advanced correlation scenarios.  Expert usage only.

Contextual Comparison: Vulnerability Info, System (Computer or Network Node) Information, Application Information, User Information, or other categorized attributes describing how the network, systems, users, applications or data are used and/or organized.  The more context added to each correlation scenario the more refined (and meaningful) the output will be. In most cases, if accomplished correctly it also means the most efficient use of system resources.  A Simple example could be defining assets with PCI, PII relevance. 

Meta Correlation: Using SIEM enhanced data from previously/currently correlated events to form new correlation scenarios. This can also use the output of Statistical evaluations. The meta-correlation can be between previous correlated events and new event stream data or multiple previous correlated events.  This is also how many systems handle basic scalability or higher tier deployment scenarios.  A baseline of content is deployed at lower tiers and matching events are forwarded upward for inclusion in “enterprise-wide” correlation scenarios. 

Summary: 

Correlation is a very powerful SIEM functions that can help you refine the identification of anomalous or malicious activity.  If your (the customer) can articulate your use-cases clearly, then most vendors can find a way to solve the defined problem using existing functionality within their product set.  It is my hope that you will be able to use this blog post as a way to map the various solution offerings to a common and understandable taxonomy so you can fully comprehend what you are getting with each solution. 

In the next post in this “Back to School” series (SIEM Correlation 201) we’ll talk about Use Case Definitions, Event Sources, Performance Impact, Flexibility and Scalability.

“ring, ring” class dismissed until next week.

-Rocky

Come see Decurity at ArcSight Protect 2009.  We’ll be all over the place!  This is the first conference we’ve sponsored so we’re super excited and thrilled to partner with ArcSight for this event.  Ray, Travis, Paul and Rocky will all be at the conference and we really want to meet with you and hear how Decurity can support your ArcSight requirements.

If you are an ArcSight customer, partner or prospect and you’re not already registered you really should register and be at this conference.  It is the best opportunity to meet everyone at ArcSight, hear about what’s new and what’s coming with ArcSight and to have your opinion heard by everyone! 

Plus you’ll get to see firsthand how Decurity “enhances” ArcSight Solutions!

References: Sponsor Page: http://www.arcsight.com/protect09/sponsors/ ArcSight Protect 09: http://www.arcsight.com/protect09

Recent vendor press releases by NitroSecurity and NetWitness highlight the evolving requirement for full network packet collection, indexing and reconstruction for analysis.  These products and others (including Solera Networks) illustrate an emerging market in total network awareness.  Working in conjunction with Log Management (LogLogic, Splunk, ArcSight Logger, etc) and SIEM tools (RSA, EiQNetworks and of course ArcSight ESM) these tools provide invaluable insight into your network’s behavior (not to mention the behavior of individual users and applications over the network).  NitroSecurity updated their capabilities to include what they term as “content aware SIEM” and NetWitness announced a milestone of 15,000 active users.  Both press releases highlighted quotes from Decurity, which we appreciate, but more important to us, the emergence and rapid growth of this market segment add further credibility to Security Professionals having all of the right tools and information available.  Recent news about DHS Einstein and NSA Tutelage technologies also point towards an increased trend in better, more capable Collection tools.

Security Operations and Incident Response capabilities can’t continue to function in the dark and be expected to adequately protect the enterprise.  We need to make all of the applicable information available and apply intelligent analytical techniques against the data set so that we can more rapidly and accurately identify risks to the enterprise.  These tools when used properly can reduce analytical time required to identify incidents into time segments measured in seconds and can help understand the scope of the incident much more rapidly.  You can review the artifacts (documents, files, audio, video, web, email, chat, as well as interactive sessions (ftp, telnet, ssh, etc)) instantly and determine the legitimacy of the session.  You can extract information and search log management/SIEM for related events and set up alerts and workflow along the way.  All in a matter of clicks.  Of course you can accomplish the reverse and search for anomalies identified in SIEM/Log Management or IDS/IPS in your Network Awareness tool and understand quickly what occurred.  With this level of information available to you, the limitations of the they of analysis have more to do with the level of expertise of the user/analyst than the information.

These use of these tools in the right hands allow for much more than just security “alerts” and incident identification.  They lend themselves to true security convergence concepts and overall enterprise intelligence and security operations.  More on those concepts over the next few months.

References:
NetWitness “July 27, 2009 | Security Experts Worldwide Rely Upon NetWitness® Investigator ” Link: http://www.netwitness.com/resources/pressreleases/Jul272009.aspx

NitroSecurity “NitroSecurity Heightens Enterprise Security Information Management with Real-Time Application Content and Protocol Analysis” Link: http://www.nitrosecurity.com/information/news/pr/2009/20090722.psp

Decurity Blog:  Dec 2008:  http://blog.decurity.com/index.php/dec_template/more/netwitness_investigator_summary_1/

Updated 15 Jan 2010 (see bottom)
Gartner 2009 SIEM Magic Quadrant - It’s pretty easy to digest - Everyone is in the leader quadrant or is a “leader” of some sort.  No vendor has any “real” negatives, they will all work in nearly any situation, everything is peachy.  You can register with many of the vendors to receive your copy of the Gartner Magic Quadrant for SIEM 2009 report.  A few of those links are provided here:  Q1, ArcSight, LogLogic , RSA

My obviously frustrated opinion:
If you are an executive, director, manager, technician or otherwise breath air and you use this report to make any sort of determination about SIEM vendor qualifications you should probably be seeking professional career advice.  This report is great at providing the SIEM vendors with additional fodder to confuse the sales prospects, but it does absolutely nothing for the reader seeking information about which SIEM might fit their environment.  This is the one time that I don’t blame the vendors, they have to play the game.  The game is to mitigate the effect of competitive “spin” based on the results of the report by the “leaders” listed within the report.  Along those lines I won’t use this post to congratulate or bash vendors as to their relative position in the quadrant because it’s just insane to justify anything about this report.

Every year, I review these SIEM reports and find myself hoping that the next issue will reveal something insightful or even slightly meaningful.  I scrub the “cautions” looking for anything that points to a material technical weakness in the technology and usually the most meaningful thing I find is a veiled “feature request” for some trivial item.  A review of the “strengths” of each organization shows the points to be at best highly subjective and usually just completely irrelevant.  I always leave frustrated, but hopeful that the next version will set things right. 

Gartner isn’t alone in it’s pursuit of mediocrity here.  Most (not all) of the “analyst” firms and industry magazines offer a strikingly similar lack of useful information in their reports.  Please note: this is in no way a personal attack on any author or company, it’s a rant against crappy information as a whole.  Over the years I’ve met most of the reviewers and they “seem to get it” in person, it’s just the nature of these “ranking without context” reports that simply kills the value of any insight the authors might have tried to present.  The 451Group and a few others have tried to buck that trend over the years and are making some progress, but despite their efforts the overall industry standard is still too watered down to be of any real value. 

I know I’m not endearing myself to the analyst community right now, and I expect certain vendors won’t appreciate what I’m saying but bear with me here.  I think we can make this better and everyone can benefit.  As an industry we must start expecting better from our information providers.  We need to provide specific feedback about what information these reports should provide in order to be meaningful.  I have tried to influence better context and more meaningful technical criteria through several older blog posts and through conversations with anyone that will listen.  I’ll step up my game and offer even more direct advice in the coming months - I’m just asking that everyone do the same.  Let’s encourage our information providers to pursue a higher standard. 

Maybe, next year… Hey I’m a Cubs fan - There is always next year! 

EDIT - UPDATE-1 15 Jan 2010

So after I posted this blog I pretty much went on with life.  I did run into some Gartner folks at their IT Security Forum in DC right after this post and learned alot more about how Gartner positions these MQ offerings.  I’ve got to say that with the right perspective in place I’m much more agreeable to what they are trying to do from a macro perspective.  They do offer more detailed analytical documents (critical capabilities) and they do push for more detailed requirements gathering for buyers that send them inquiries.  I was expecting something else and really in the end that was the problem, my expectations were misaligned with the intention of this MQ.  So to Mark Nicolett I apologize for not taking the time to really dig deeper into your approach.  I do respect what you are doing and my offer to help still applies.

For those seeking more information about the Gartner SIEM MQ - Mark Nicolett’s guest blog post explains the process very well—> http://blogs.gartner.com/john_pescatore/2009/06/15/guest-blogger-mark-nicolett-and-the-siem-market/

Recently, I initiated a test survey using Linkedin’s Polling feature.  The survey was a quick and dirty way to help me gain a bit more perspective as to the number of resources organizations are putting towards their SIEM Projects.  There are a ton of limitations to this type of survey, including the fact you only have 75 characters to present your question to the audience.  That said, I’m sure I could have worded my survey question and responses even more clearly.  Those limitations aside, the results were very consistent with my observations over the last several years.  Put simply it requires a lot of effort, even halfway through 2009, to run a successful SIEM Project. 

Over the last month 13 organizations (75%+ Large or Enterprise Organizations) have taken the time to respond directly to the LinkedIn poll.  Quite a few more people went further and emailed me with details, questions and observations.  Just using the survey results, nearly 70% indicated that they had 2 or more FTE’s dedicated to SIEM.  With an additional 15% responding simply “Not Enough”.  100% responded with at least 1 FTE. 

My take on this initial poll - Even with all the improvements SIEM vendors are making in their products, SIEM’s are still not “plug ‘n play” systems, you need dedicated resources (internal, consultant, partner, etc) to extract the maximum value from your SIEM.  Duh!, I’ve been saying that for years, but it’s nice to see others nodding their head every once in a while.

I’m working on a better series of questions and responses with a more clear focus on for the next set of survey’s and I’ll use a more robust mechanism to accomplish that goal.  I’ll be using SurveyMonkey to get a better perspective and more clear insight into all things SIEM (and Log Management).  Look for those survey’s to begin in early July with results and analysis to follow shortly thereafter.  If you have questions or observations you’d like me to ask to the world about Log Management or SIEM - leave me a comment or email/DM me!  Thanks.

Sara Peters at Information Week recently posted an article titled “SIEM Case Study: Israeli e-government ISP” In this article, Assaf Keren, information security manager at the Israeli e-government ISP Project (called “Tehila”) calls our attention to some very important details to consider when Implementing a SIEM.  Keren’s advice is that a successful SIEM implementation requires:

1. Detailed planning,
2. Fastidious attention to detail,
3. Superb communication between concerned parties
4. Attentive oversight of vendor activity.
Another Key Point from Mr. Keren - don’t outsource this “theory phase.”

Note:  I agree with Mr. Keren that the SIEM requirements have to be driven from within your organization.  However, I believe that expert external entities can and should help drive discussions and help extract and refine requirements from your team. Obviously, the expert external entity MUST NOT be from a Vendor or reseller of any SIEM Products.

Looking back over hundreds of SIEM deployments and seeing so many consistent decisions (or indecisions) that adversely affected the success of the SIEM I felt compelled to add a bit more context to augment the lessons Mr. Keren shared.

Overview:
1. It takes a village, building planners, city inspectors, etc:  Probably, the most important takeaway from this post is that you should take the necessary time to fully comprehend and vet your requirements, as well as decide on your service delivery model, gain consensus on that approach and have realistic expectations along the way.  SIEM failures are more often the fault of poor planning, moving tactically while ignoring the strategic nature of the project, or simply misaligned expectations rather than a pure technology failure. 

2. Know what you are going to do with the Output before you make it Input:  It is tough to make sense (and therefore derive any value) out of billons of events by adding even more events to be evaluated into the mix.  Intelligent Collection, Analysis, Escalation and Remediation and workflow efforts defined before you start (and refined along the way) means that you’ll have a better idea what to do with the information your presented and a much higher chance for success in both end-user usage of the system and aligning that usage of the SIEM with the needs of your organization’s security or compliance program.

3. Purchase the “right” technology, but do it incrementally:  Quite candidly some SIEM products should be avoided at all costs, however it should be noted that most of them can at least be used to help you meet some very basic requirements.  Consider your business and technical requirements over a 24-month period, but only purchase what is necessary to deliver based on the next 6 months of work you expect to get accomplished.  The system needs to be flexible to support all of those upcoming needs, but there is no need to spend money today to support tasks you won’t even consider touching for over 12 months. 

A successful SIEM tool supporting your organization’s Security and/or Compliance needs really boils down to some very simple concepts:

Define Success
Have a strategic vision about how you want your Security Operation and/or Compliance Program to run and use that to help define requirements for how the SIEM (and Log Management) tools will provide input or drive workflow related to that Program.  Involve all the stakeholders early and keep them engaged along the way!

• If your rationale for buying a SIEM is PCI Compliance, STOP.

• If your rationale for investing in SIEM is to provide “x”,”y” and “z” data sets to business unit “a” and “b” and initiating workflow for your SOC; and you understand the event sources necessary/business logic to compile the data sets for each customer; and you fully understand how they intend to use that information the you are much closer to being ready to work with a SIEM.

Related Resources:
• SIEM: Basic Implementation Success Criteria
• SIEM: Before you Buy

Plan Accordingly
SIEM is not an overnight project, and yes even an Appliance-based SIEM’s require significant attention to work to their maximum potential for your organization.

• Gather requirements from all “stakeholders” Compliance, Legal, IT, Business Units, Security, Executive, everyone that will help you get information into the SIEM or receive information from the SIEM (or your service offering that leverages SIEM). 

• Define Event Sources based on end-user needs:  Security, IT Operations and Compliance teams all have distinct needs and therefore may require different event source information.  At a minimum they may require different “views” of similar information set available in the SIEM or Log Management Tool.  Ensure you have the proper information sets, logging at the right levels and the information is available in a logical and meaningful manner.

• End-User Requirements are the most valuable.  The more your team understands how your “customers” value the data and service offering the more you can benefit from the functionality of the SIEM. 

• Analytical and Workflow Requirements.  Security Analysts need to be able to quickly identify, analyze, prioritize and escalate the data with context in order for the SIEM to meet its most basic functions.  This functionality is not as common as you would think across different SIEM’s.  Be sure that the SIEM integrates with your workflow systems in an acceptable fashion.

Related Resources:
• SIEM: Best Practices in Collection

Vendor Selection
Now that you have your requirements documented and prioritized compare them against SIEM: Evaluation Criteria and refine them even further…

• Either partner with an expert that can tell you exactly why certain Vendors can not meet your needs (today/tomorrow) and compare those answers a n honest discussion with the vendor or invest in a Pilot in an effort to prove out ALL of your requirements (not just the top three.)
• Make sure you have data either directly from production event sources or a reasonably similar source.  If you use combined Log Management and SIEM architecture, make sure you can configure outbound events in a format the SIEM can comprehend for more than just Syslog events.  If the SIEM can natively handle ODBC but your architecture requires Log Management to be the Collection Tier and forward events to the SIEM – How does the LM reformat those events and how does the SIEM handle that data? 

• Customer Referrals are nice, but be careful.  I’ve seen this scenario too many times.  Victim asks a SIEM Reference Client about a key area of concern, say scalability and the reference client dutifully answers the questions with a resounding “Yes, the $VENDOR scales to meet my global organization’s amazing needs” in all the excitement it was overlooked that it takes 100+ systems to get there and oh yeah, by the way none of these SIEM systems can cross correlate information.  As your requirements are defined, build out testing plans if the requirement is that critical and test it prior to purchasing.

• Maximize your dollar.  Ensure the vendor is prepared to partner with you for the long haul, you both have a vested interest in the success of the program – make sure they are going to be there for you
• Find out the vendor’s fiscal period and plan your purchase accordingly.  Fiscal Quarter end and Fiscal Year end are great times to make deals (especially enterprise deals) with vendors.
• Purchase what you need not what you want.  If you don’t have a documented requirement that you can reasonably achieve in the next 6 months don’t buy it yet.  Conversely, don’t skimp on things you absolutely do need.  If you have a requirement to store 8 Billon events a day over a 10-year period and you expect to do that with local storage or even DAS, NAS.  Stop and rethink things a bit.

Focused Effort:
Ensure that you have dedicated enough time and energy to the success of your SIEM Effort.  If you are a large enterprise this is at least 2 FTE’s or an Expert Partner

Seriously, Requirements Gathering, Vendor Selection, Pilot, Implementation, Initial Operating Capability, Operational Refinements, Final Operating Capability (Formal Service Delivery), On-going Enhancements, Patches, Upgrades, Lab Testing, Additional Content Tuning, Expansion and the related Coordination, Planning, Execution, Oversight and Measurements is enough to keep an entire team busy.  Doing all of that within the framework of your overall Strategic Security Program and not just tactically solving issues as the “pop-up” on a daily basis is the key to success with SIEM and ultimately your entire security and/or compliance program.

Having the wrong team or not listening to the right team is about the same as not having resources at all.  Spend the time to ensure your SIEM team is baked into your Security/Compliance Program(s) so they can help you plan for today and tomorrow and save a lot of headaches in new hardware, storage or even total SIEM replacement.  If your not ready to dedicate the right Resources/Partner’s then you may be better off waiting and then introducing SIEM into your organization when the requirements, proposed solution and funding are more in line. 

Lifecycle Planning
This goes way beyond simple O&M tasks.  SIEM is part of your overall Security Program and as such need to stay in step with that Program.  Your SIEM Team (Partner) needs to be involved along the way to help ensure compatibility and/or flexibility as you evolve.  Service Delivery, Technology, Business and Compliance requirement changes and/or reprioritizations can all have a significant impact on the success or failure of the overall program.  The tighter the team is with the thought process around those upcoming changes the more likely your SIEM Program will meet your needs.

Network World’s recent article provides additional evidence that Log Management and SIEM Vendors are still trying to evolve.

Recently, Log Management and SIEM vendors have spent a lot of time updating/fixing their products.  Over the past few months some vendors have quietly passed over other solutions in terms of market relevance and certainly the door has been opened to a whole bunch of upstarts trying to make a name for themselves.  While the majority of Log Management and SIEM business (and therefore product direction) is driven by compliance activities, I appreciate the forward movement towards enterprise security that many in the field are trying to make.  The initial execution on that product vision I’m seeing from many of the vendors this year is very welcome.  IMHO the entire space had gotten very stale with the big guys mainly focusing on compliance appliances or playing feature catch-up with one another.  Here’s my summary of what’s going on in SIEM and Log Management so far in 2009.

The following notional diagram provides some basic recommendations to consider when deploying and managing Log Management and SIEM systems together.  A well-maintained Log Management and SIEM deployment can significantly reduce the time to Incident Identification and really enhance your overall information security capability.  The diagram attempts to illustrate that all information from the Event Sources are processed through the appropriate Log Collection Mechanism and then forwarded to the Log Management System. The Log Management system eats, stores and can regurgitate everything put into it.  The Log Management Solution also can further refine the data set and forward only applicable events for analysis to the correlation engine (SIEM) through the use of intelligent “tagging” of events.  Overall data reduction is only part of the end goal, more importantly we want to ensure the right data is forwarded and evaluated so that we can gain from the overall efficiencies offered by the SIEM.  In short we’re ensuring the system has the correct information available to it so that it can respond to the questions you want to ask of it and reduce the garbage as much as possible.

Decurity’s new subscription offering works with your organization to understand your requirements and then supplies the necessary configurations and Log Management and SIEM data elements (Intelligent forwarding, correlation, reports, etc) to make this model work for you.

Decurity often has the opportunity to our customers find the right Log Management and/or SIEM solution.  We are honored that our customers trust us with that very important question so we wanted to take a moment and explain our requirements gathering/documentation process for vendor selection and hope that our explanation helps a few of more folks out there!  We also get asked by Vendors on how they can improve their products, but that’s a entirely different blog post. 

The Decurity team has been incredibly busy over the last few months cooking up new and more cost effective ways to support our mantra of “keeping security simple”.  We’re getting ready to introduce a “game changer” for our Log Management and SIEM customers.  Our newest offering will be fully described in the coming days but here is a preview of the new subscription services Decurity is offering to our clients.