Recently, Log Management and SIEM vendors have spent a lot of time updating/fixing their products.  Over the past few months some vendors have quietly passed over other solutions in terms of market relevance and certainly the door has been opened to a whole bunch of upstarts trying to make a name for themselves.  While the majority of Log Management and SIEM business (and therefore product direction) is driven by compliance activities, I appreciate the forward movement towards enterprise security that many in the field are trying to make.  The initial execution on that product vision I’m seeing from many of the vendors this year is very welcome.  IMHO the entire space had gotten very stale with the big guys mainly focusing on compliance appliances or playing feature catch-up with one another.  Here’s my summary of what’s going on in SIEM and Log Management so far in 2009.

My observations about recent SIEM and Log Management Vendor announcements and my thoughts about what is still needed to make it more effective for most customers.

1. Log Management appliance based solutions continue to rule the land.  See ArcSight’s last quarterly results announcement and extract from that the fact that something like 46% of product revenue was based on their appliances or talk with Intellitactics about their recent growth in SAFE Logging Appliance sales if you don’t believe me.  Splunk may be exception to that appliance rule as they continue to gain ground on everyone but the fact remains if you don’t have a solid Log Management offering you’re toast in 2009.  Conversely, pure Log Management solutions have seen the importance of having a SIEM offering available to be able to effectively compete on larger accounts (see the link below about Log Logic partnership with ExaProtect as one example).

2. Some vendors are still playing feature catch-up adding things like Trouble Ticket Integration of Vulnerability Assessment Tool Integration.  It is amazing to me that these are “new feature sets” in several products, but progress is progress and I’ll take it.  I’m still not thrilled with most vendors idea of integration being unidirectional SMTP based updates but it’s a start.  The idea of Vulnerability Management and then furthering that by full CMDB integration is awesome in theory but in many cases it is very hard to scale.  This has made significant stride forward but in my mind much more is left to be accomplished.

3. Database Activity Monitoring seems to be catching on.  Interesting given how difficult this really is to accomplish in an enterprise - Admin level monitoring is easy enough but field level and/or transaction level auditing like Oracle FGA is a non-trivial task to accomplish.  Most vendors have approached this solution set through 3rd party product support (Imperva, Guardium, etc) but at least a couple have tackled this directly through development or acquisition.

4. Enterprise Visibility is a growing trend.  I’ve previously stated my appreciation for having proper visibility across the enterprise.  When NetWitness announced their free version of Investigator I posted my comments and review of the technology.  Last year in my “Best Practices in Security Operations: Collection” post I expressed my thoughts on what it takes to really do Enterprise Security Operations correctly and that the foundation of the entire process is Collection.  Part of that collection is filling in the gaps missed by other products and Full Packet Capture is an awfully powerful tool to have in your arsenal.  Several vendors have announces partnerships with 3rd party providers and/or announced network capture/replay as an internal capability through development or acquisition (see NitroSecurity).

5. Hard stuff is still hard - Windows Event Logs, Custom Applications, CMDB Integration, Database Monitoring are all hard problems to tackle.  No vendor has this perfect (how could we expect them to) but several have started to think about ways to make this easier, rather than just saying “start with critical systems” they are developing more reliable/scalable/flexible solutions.  I’m interested to see how far the vendors get with the more complex logs and more meaningful IT operations integrations in 2009.

6. Relevant content is king.  The default content (Correlation, Reports, etc) from most of the vendors is not going to work in your enterprise.  Sure some of it works at least some of the time but seriously on average how could 1500+ reports ever apply to your environment.  Content Tuning is the #1 area vendors can make these solutions more relevant and easier for customers.  I believe this so much that Decurity has re-focused much of its energy to address this issue head on hey, it’s my blog and I’ll pimp if I want to.

7. Consolidation/Contraction.  I do expect to see more industry consolidation.  High Tower won’t be the last failure and some of the big boys need to revamp their overall integrated solutions.  The Log Management side may be commodity ready but SIEM is not (yet).  I expect at least 2 more acquisitions/failures to occur in the space in the coming months and not necessarily because of bad technology.  A great technology in the wrong hands or begin put against the wrong goals is no better than a crappy technology. 

Additional Thoughts:
SIEM:   The Fortune 100 seems to be saturated with products/solutions (a lot have failed and some are actually working) and they have plenty of war stories that hey are more than willing to share.  The overall take-away is simple - you need to be ready to invest in success if you jump into SIEM, product flexibility, product expertise, documented internal requirements and perhaps most importantly internal processes/procedures for making it work.
Fortune 200-2000: I’m happy to see that many organizations are now starting to push-back on vendors (and their own management) after learning from the mistakes of others.  I think they will take a more reasonable approach of making Log Management work first before moving to SIEM and when they get to SIEM I’m encouraged by the increasing trend of documenting actual use-cases and requirements.
Channel:  Nearly all the of vendors are trying enabling better global channel programs and at least in Log Management these are very successful programs helping them gain foreign market share and SMB market in the US.

SIEM: Some idea’s for improvement:
1. Most of the SIEM products require significant expertise to deploy, manage and maintain effectively.  There is a good business there for the right partnerships and companies should leverage that expertise.  Today the number of VAR’s and Consulting organizations that really can be considered “expert” in this area can be counted on one hand.  I still think the vendors are going to have to do better to make their SIEM products more user-friendly and that will require a true partnership with their customers. 
2. If your SIEM can’t correlate events (Raw or Meta events) between engines you should be pushing your Vendor towards making that work in 2009.  Linear scalability is irrelevant you are going to need flexibility in the future.  Even if your organization is not mature enough to use that functionality today, you will eventually get there, make sure your vendors are partnered with you to help you achieve your long-term goals.

LOG MANAGEMENT:  Log Management will continue to grow across all market segments, no question.  The overall channel effectiveness seems to be growing as the acceptance of appliance based solutions grows.  Compliance is an easy driver for sales in Log MGMT but at some point the big customers will have to step up and require better and more scalable solutions from the vendors. 

Log Management: Some idea’s for improvement:
1.) Log Management solutions that attach to an in-house SAN is a great idea - but only if the SAN carve out is unlimited or at least sufficient enough that they aren’t required to purchase additional appliances to scale to the enterprise. 
2.) Better bi-directional support between Log Management and SIEM solutions is absolutely required.  You have to be able to search one/both from a single location for these to gain better traction as combined offerings.
3.) Flexibility is going to be key moving forward.  Splunk is gaining ground for a reason (and it is more than marketing).  Integration of new event sources will continue to be a key issue.
4.) Content is also an issue on the Log Management end.  Some vendors do a fantastic job and others need better partnerships or a renewed internal focus on relevant security content.

Recent Log Management and SIEM Product Update News and Links:
March 2009
Nitro Security Acquires Chronicle Solutions .
EMC RSA RSA releases enVision v4.0  And now Paul Stamp is over there helping them move the product further along!
NetForensics NetForensics releases NFX v4.1 .

Feb 2009
TenableTenable releases Database Auditing Capability
Q1Labs QRadar 6.2 was released.  .
LogLogic partners with exaprotect
NetForensics  HighTower was gobbled up by NetForensics.  I’m reserving judgment for now. 

Jan 2009
Nitro Security integrates Database Activity Monitoring (DAM) into SIEM.
eIQnetworks announces a new round of funding and management additions A lot of “buzz” around eIQ right now but I have yet to have the opportunity to see them in action in a global enterprise.  I hear good things about their approach, but I’m always cautious. 

Late 2008 News:
Symantec SIM v4.6 released” .
Trigeo seeks HighTower’s customers Trigeo offered a competitive upgrade to former HighTower customers.
LogRhythm Version 4.1 Released
Sensage version 4.0 Released
Splunk v3.4.6 released   I hear rumors of Splunk 4.0 coming this summer!
ArcSight Announces ESM 4.0 SP2 and ArcSight Logger 3.0   ArcSight Logger 3.0 was a VAST improvement over previous versions in terms of speed and capability.
Q1 Labs QRadar product was OEM’ed into Juniper STRM Products.  An interesting play, very similar to what Protego did a few years ago (now Cisco MARS).