Imagine a world where you can login to your computer once and have full access to all of the functionality in your computer, plus seamless access to all of the web sites you visit on a daily basis. No more logging into each site individually, your computer's operating system takes care of that for you.

That world may be coming quicker than you realize. I was listening to a recent episode of the PaulDotCom security podcast today. In this episode, they interviewed Jason Fossen, a SANS Security Faculty Fellow and instructor for SEC 505: Securing Windows. During the conversation, Jason mentioned some of the changes coming to the next version of Microsoft's flagship operating system, Windows 8. What he described was, in a word, horrifying…

Not much information is out there about these changes yet, but it's possible to piece together some of it. Jason mentioned that Windows 8 will have a broker system for passwords. Basically, Windows will store all of the passwords necessary to access all of the various services you interact with. Think something along the lines of 1Password or LastPass. The main difference being, this happens in the background with minimal interaction with the user. In other words, you never have to explicitly login to anything beyond your local Windows workstation.

Initially, Microsoft won't have support for all of the various login systems out there. They seem to be focusing on their own service, Windows Live, and possibly Facebook. But the API is open, allowing third-parties to provide the necessary hooks to their own systems.

I've spent some time searching for more information and what I'm finding seems to indicate that what Jason was talking about is, in fact, the plan moving forward. TechRadar has a story about the Windows 8 Credential Vault, where website passwords are stored. The credential vault appears to be a direct competitor to 1Password and LastPass. As with other technologies that Microsoft has integrated in the past, this may be the death knell for password managers.

ReadWriteWeb has a story about the Windows Azure Access Control Service that is being used for Windows 8. Interestingly, this article seems to indicate that passwords won't be stored on the Windows 8 system itself, but in a centralized "cloud" system. A system called the Access Control Service, or ACS, will store all of the actual login information, and the Windows 8 Password Broker will obtain tokens that are used for logins. This allows users to access their data from different systems, including tablets and phones, and retain full access to all of their login information.

Microsoft is positioning Azure ACS as a complete claims-based identity system. In short, this allows ACS to become a one-stop shop for single sign-on. I log into Windows and immediately have access to all of my accounts across the Internet.

Sounds great, right? In one respect, it is. But if you think about it, you're making things REALLY easy for attackers. Now they can, with a single login and password, access every system you have access to. It doesn't matter that you've used different usernames and passwords for your bank accounts. It doesn't matter that you've used longer, more secure passwords for those sensitive sites. Once an attacker gains a foothold on your machine, it's game over.

Jason also mentioned another chilling detail. You'll be able to login to your local system using your Windows Live ID. So, apparently, if you forget your password for your local user, just login with your Windows Live ID. It's all tied together. According to the TechRadar story, "if you forget your Windows password you can reset it from another PC using your Windows Live ID, so you don't need to make a password restore USB stick any more." They go on to say the following :

You'll also have to prove your identity before you can 'trust' the PC you sync them to, by giving Windows Live a second email address or a mobile number it can text a security code to, so anyone who gets your Live ID password doesn't get all your other passwords too – Windows 8 will make you set that up the first time you use your Live ID on a PC.

You can always sign in to your Windows account, even if you can't get online – or if there's a problem with your Live ID – because Windows 8 remembers the last password you signed in with successfully (again, that's encrypted in the Password Vault).

With this additional tidbit of information, it would appear that an especially crafty attacker could even go as far as compromising your entire system, without actually touching your local machine. It may not be easy, but it looks like it'll be significantly easier than it was before.

Federated identity is an interesting concept. And it definitely has its place. But, I don't think tying everything together in this manner is a good move for security. Sure, you can use your Facebook ID (or Twitter, Google, OpenID, etc) already as a single login for many disparate sites. In fact, these companies are betting on you to do so. This ties all of your activity back to one central place where the data can be mined for useful and lucrative bits. And perhaps in the realm of a social network, that's what you want. But I think there's a limit to how wide a net you want to cast. But if what Jason says is true, Microsoft may be building the equivalent of the One Ring. ACS will store them all, ACS will verify them, ACS will authenticate them all, and to the ether supply them.

Last week, another "zero-day" vulnerability was reported, this time in Adobe's Acrobat PDF reader. Anti-virus company, Symantec, reports that this vulnerability is being used as an attack vector against defense contractors, chemical companies, and others. Obviously, this is a big deal for all those being targeted, but is it really something you need to worry about? Are "zero-days" really something worth defending against?

What is a zero-day anyway? Wikipedia has this to say:

A zero-day (or zero-hour or day zero) attack or threat is a computer threat that tries to exploit computer application vulnerabilities that are unknown to others or the software developer. Zero-day exploits (actual software that uses a security hole to carry out an attack) are used or shared by attackers before the developer of the target software knows about the vulnerability.

So, in short, a zero-day is an unknown vulnerability in a piece of software. Now, how do we defend against this? We have all sorts of tools on our side, surely there's one that will catch these before they become a problem, right? IDS/IPS systems have heuristic filters for detecting anomalous activity. Of course, you wouldn't want your IPS blocking arbitrary traffic, so that might not be a good idea. Anti-virus software also has heuristic filters, so that should help, right? Well… When's the last time your heuristic filter caught something that wasn't a false positive? So yeah, that's probably not going to work either. So what's a security engineer to do?

My advice? Don't sweat it. Don't get me wrong, zero-days are dangerous and can cause all sorts of problems, but unless you have an unlimited budget with an unlimited amount of time, trying to defend against an unknown attack is a pointless exercise in futility. But don't despair, there is hope.

Turns out, if you spend your time securing your network properly, you'll defend against most attacks out there. Let's look at this latest attack, for instance. Let's assume you've spent millions and have the latest and greatest hardware with all the cutting edge signatures and software. Someone sends the CEO's secretary an innocuous PDF, which she promptly opens, and all that hard work goes out the window.

On the other hand, let's assume you spent the small budget you have defending the critical data you store and spend the time you've saved not decoding those advanced heuristics manuals on training the staff. This time the CEO's secretary looks twice, realizes this is an unsolicited email, and doesn't open the PDF. No breach, the world is saved.

Seriously, though, spending your time and effort safe-guarding your data and training your staff will get you much further than worrying about every zero-day that comes along. Of course, you should be watching for these sorts of reports. In this case, for instance, you can alert your staff that there's a critical flaw in this particular software and that they need to be extra careful. Or, if the flaw is in a web application, you can add the necessary signatures to look for it. But in the end, it's very difficult, if not impossible, to defend against something you're not aware of. Network and system security is complex and difficult enough without having to worry about the unknown.

On September 30th, 2011, over 1000 people from a variety of backgrounds descended on Louisville, Kentucky to attend the first DerbyCon. DerbyCon is a security conference put together by three security professionals, Dave Kennedy, Martin Bos, and Adrian Crenshaw. Along with a sizable crew of security and administrative staff, they hosted an absolutely amazing conference.

During the three day conference, DerbyCon sported amazing speakers such as Kevin Mitnick, HD Moore, Chris Nickerson, and others. Talks covered topics such as physical penetration testing, lock picking, and network defense techniques. There were training sessions covering Physical Penetration, Metasploit, Social Engineering, and more. A lock pick village was available to both learn and show off your skills, as well as a hardware village where you could learn how to solder among other things. And, of course, there were late-night parties.

For me, this was my first official security conference. By all accounts, I couldn't have chosen a better conference. All around me I heard unanimous praise for the conference, how it was planned, and how it was run. There were a few snafus here and there, but really nothing worth griping about.

The presentations I was able to attend were incredible and I came home with a ton of knowledge and new ideas. During the closing of the conference, Dave mentioned some ideas for next years conference such as a newbie track. This has inspired me to think about possibly presenting at next years conference. I have an idea already, something I've started working on. If all goes well, I'll have something to present.

DerbyCon was definitely one of the highlights of my year. I'm already eager to return next year.

<RANT>

It's amazing, but the deeper I dive into security, the more garbage security theater I uncover. Sure, there's insanity everywhere, but I didn't expect to come across some of this craziness…

One of the most recent activities I've been party to has been the response to an independent audit. When I inquired as to the reasoning behind the audit, the answer I've received has been that this is a recommended yearly activity. It's possible that this information is incorrect, but I suspect that it's truer than I'd like to believe.

Security audits like this are standard practice all over the US and possibly the world. Businesses are led to believe that getting audited is a good thing and that they should be repeated often. My main gripe here is that while audits can be good, they need to be done for the right reasons, not just because someone tells you they're needed. Or, even better, the audits that are forced on a company by their insurance company, or their payment processor. These sorts of audits are there to pass the blame if something bad happens.

Let's look a little deeper. The audit I participated in was a typical security audit. An auditor contacts you with a spreadsheet full of questions for you to answer. You will, of course, answer them truthfully. Questions included inquiries about the password policy, how security policies are distributed, and how logins are handled. They delve into areas such as logging, application timeouts, IDS/IPS use, and more. It's fairly in-depth, but ultimately just a checklist. The auditor goes through their list, interpreting your answers, and applying checkmarks where appropriate. The auditor then generates a list of items you "failed" to comply with and you have a chance to respond. This is all incorporated into a final report which is presented to whoever requested the audit.

Some audits will include a scanning piece as well. The one I'm most familiar with in this aspect is the SecurityMetrics PCI scan. Basically, you fill out a simplified yes/no questionnaire about your security and then they run a Nessus scan against whatever IP(s) you provide to them. It's a completely brain-dead scan, too. Here's a perfect example. I worked for a company who processed credit cards. The system they used to do this was on a private network using outbound NAT. There were both IDS and firewall systems in place. For the size of the business and the frequency of credit card transactions, this was considerable security. But, because there was a payment card processor in the mix, they were required to perform a quarterly PCI scan. The vendor of choice, SecurityMetrics.

So, the security vendor went through their checklist and requested the IP of the server. I explained that it was behind a one-way NAT and inaccessible from the outside world. They wanted the IP of the machine, which I provided to them. 10.10.10.1. Did I mention that the host in question was behind a NAT? These "security professionals" then loaded that IP into their automated scanning system. And it failed to contact the host. Go figure. Again, we went around and around until they finally said that they needed the IP of the device doing the NAT. I explained that this was a router and wouldn't provide them with any relevant information. The answer? We don't care, we just need something to scan. So, they scanned a router. For years. Hell, they could still be doing it for all I know. Like I said, brain dead security.

What's wrong with a checklist, though? The problem is, it's a list of "common" security practices not tailored to any specific company. So, for instance, the audit may require that a company uses hardware-based authentication devices in addition to standard passwords. The problem here is that this doesn't account for non-hardware solutions. The premise here is that two-factor authentication is more secure than just a username and password. Sure, I whole-heartedly agree. But, I would argue that public key authentication provides similar security. It satisfies the "What You Have" and "What You Know" portions of two-factor authentication. But it's not hardware! Fine, put your key on a USB stick. (No, really, don't. That's not very secure.)

Other examples include the standard "Password Policy" crap that I've been hearing for years. Basically, you should expire passwords every 90 days or so, passwords should be "strong", and you should prevent password reuse by remembering a history of passwords. So let's look at this a bit. Forcing password changes every 90 days results in bad password habits. The reasoning is quite simple, and there have been studies that show this. This paper (pdf) from the University of North Carolina is a good example. Another decent write up is this article from Cryptosmith. Allow me to summarize. Forcing password expiration results in people making simpler passwords, writing passwords down, or using simplistic algorithms to generate "complex" passwords. In short, cracking these "fresh" passwords is often easier than well thought out ones.

The so-called "strong" password problem can be summarized by a rather clever XKCD comic. The long and short here is that truly complex passwords that cannot be easily cracked are either horribly complex mishmashes of numbers, letters, and symbols, or they're long strings of generic words. Seriously, "correct horse battery staple" is significantly stronger than using a completely random 11 digit string.



And, of course, password history. This sort of goes hand-in-hand with password expiration, but not always. If it's used in conjunction with password expiration, then it generally results in single character variation in passwords. Your super-secure "complex" password of "Password1" (seriously, it meets the criteria.. Uppercase, lowercase, number) becomes a series of passwords where the 1 is changed to a 2, then 3, then 4, etc. until the history is exceeded and the user can return to 1 again. It's easier to remember that way and the user doesn't have to do much extra work.

So even the standard security practices on the checklist can be questioned. The real answer here is to tweak each audit to the needs of the requestor of the audit, and to properly evaluate the responses based on the security posture of the responder. There do need to be baselines, but they should be sane baselines. If you don't get all of the checkmarks on an audit, it may not mean you're not secure, it may just mean you're securing your network in a way the auditor didn't think of. There's more to security than fancy passwords and firewalls. A lot more.

</RANT>

Apple released the latest version of it's OS X operating system, Lion, on July 20th. With this release came a myriad of changes in both the UI and back-end systems. Many of these features are denounced by critics as Apple slowly killing off OS X in favor of iOS. After spending some time with Lion, I have to disagree.

Many of the new UI features are very iOS-like, but I'm convinced that this is not a move to dumb down OS X. I believe this is a move by Apple to make the OS work better with the hardware it sells. Hear me out before you declare me a fanboy and move on.

Since the advent of the unibody Macbook, Apple has been shipping buttonless input devices. The Macbook itself has a large touchpad, sans button. Later, they released the magic mouse, sort of a transition device between mice and trackpads. I'm not a fan of that particular device. And finally, they're shipping the trackpad today. No buttons, lots of room for gestures. Just check out the copy direct from their website.



If you look at a lot of the changes made in Lion, they go hand-in-hand with new gestures. Natural scrolling allows you to move the screen in the same direction your fingers are moving. Swipe three fingers to the left and right, the desktop you're on moves along with it. Explode your fingers outwards and Launchpad appears, a quick, simple way to access your applications folder. Similar gestures are available for the Magic Mouse as well.



These gestures allow for quick and simple access to many of the more advanced features of Lion. Sure, iOS had some of these features first, but just because they've moved to another platform doesn't mean that the platforms are merging.

Another really interesting feature in Lion is one that has been around for a while in iOS. When Apple first designed iOS, they likely realized that standard scrollbars chew up a significant amount of screen real estate. Sure, on a regular computer it may be a relatively small percentage, but on a small screen like a phone, it's significant. So, they designed a thinner scrollbar, minus the arrows normally seen at the top and bottom, and made it auto-hide when the screen isn't being scrolled. This saved a lot of room on the screen.



Apple has taken the scrollbar feature and integrated it into the desktop OS. And the effect is pretty significant. The amount of room saved on-screen is quite noticeable. I have seen a few complaints about this new feature, however, mostly complaining that it's difficult to grab the scrollbar with the mouse pointer, or that the arrow buttons are gone. I think the former is just a general "they changed something" complaint while the latter is truly legitimate. There have been a few situations where I've looked for the arrow buttons and their absence was noticeable., I wonder, however, whether this is a function of habit, or if their use is truly necessary. I've been able to work around this pretty easily on my Macbook, but after I install Lion on my Mac Pro, I expect that I'll have a slightly harder time. Unless, that is, I buy a trackpad. As I said, I believe Apple has built this new OS with their newer input devices in mind.

On the back end, Lion is, from what I can tell, completely 64-bit. They have removed Java and Flash, and, interestingly, banned both from their online App Store. No apps that require Java or Flash can be sold there. Interesting move. Additionally, Rosetta, the emulation software that allows older PowerPC software to run, has been removed as well.

Overall, I'm enjoying my Lion experience. I still have the power of a unix-based system with the simplicity of a well thought out GUI interface. I can still do all of the programming I'm used to as well as watch videos, listen to music, and play games. I think I'll still keep a traditional multi-button mouse around for gaming, though.

A while ago I purchased a copy of BlogPress for my iDevices.. It's pretty full-featured, and seems to work pretty well. Problem was, I couldn't get it to work with my Serendipity-based blog. Oh well, a wasted purchase.

But not so fast! Every once in a while I go back and search for a possible solution. This past week I finally hit paydirt. I came across this post on the s9y forums.

This explained why BlogPress was crashing when I used it. In short, it was expecting to see a categoryName tag in the resulting XML from the Serendipity XMLRPC plugin. Serendipity, however, used description instead, likely because Serendipity has better support for the MetaWeblog API.

Fortunately, fixing this problem is very straightforward. All you really need to do is implement both APIs and return all of the necessary data for both APIs at the same time. To fix this particular problem, it's a single line addition to the serendipity_xmlrpc.inc.php file located in $S9YHOME/plugins/serendipity_event_xmlrpc. That addition is as follows :

if ($cat['categoryid']) $xml_entries_vals[] = new XML_RPC_Value(
    array(
      'description'   => new XML_RPC_Value($cat['category_name'], 'string'),
      // XenoPhage: Add 'categoryName' to support mobile publishing (Thanks PigsLipstick)
      'categoryName'  => new XML_RPC_Value($cat['category_name'], 'string'),
      'htmlUrl'       => new XML_RPC_Value(serendipity_categoryURL($cat, 'serendipityHTTPPath'), 'string'),
      'rssUrl'        => new XML_RPC_Value(serendipity_feedCategoryURL($cat, 'serendipityHTTPPath'), 'string')
    ),
    'struct'
);

And poof, you now have the proper category support for Movable Type.

I've been pondering my choices lately, determining if I should stay with my current blogging platform or move to another one. There's nothing immediate forcing me to change, nor is there anything overly compelling to the platform I'm currently using. This is an exercise I seem to go through from time to time. It's probably for the better as it keeps me abreast of what else is out there and allows me to re-evaluate choices I've made in the past.

So, what is out there? Well, Serendipity has grown quite a bit as a blogging platform and is quite well supported. That, in its own right, makes it a worthy choice. The plugin support is quite vast and the API is simple enough that creating new plugins when the need arises is a quick task.

There are some drawbacks, however. Since it's not quite as popular as some other platforms, interoperability with some things is difficult. For instance, the offline blogging tool I'm using right now, BlogPress, doesn't work quite right with Serendipity. I believe this might be due to missing features and/or bugs in the Serendipity XMLRPC interface. Fortunately, someone in the community had already debugged the problem and provided a fix.

Wordpress is probably one of the more popular platforms right now. Starting a Wordpress blog can be as simple as creating a new account at wordpress.com. There's also the option of downloading the Wordpress distribution and hosting it on your own. As with Serendipity, Wordpress also has a vibrant community and a significant plugin collection. From what I understand, Wordpress also has the ability to be used as a static website, though that's less of an interest for me. Wordpress has wide support in a number of offline blogging tools, including custom applications for iPad and iPhone devices.

There are a number of "cloud" platforms as well. Examples include Tumblr, Live Journal, and Blogger. These platforms have a wide variety of interoperability with services such as Twitter and Flickr, but you sacrifice control. You are at the complete mercy of the platform provider with very little alternative. For instance, if a provider disagrees with you, they can easily block or delete your content. Or, the provider can go out of business, leaving you without access to your blog at all. These, in my book, are significant drawbacks.

Another possible choice is Drupal. I've been playing around with Drupal quite a bit, especially since it's the platform of choice for a lot of projects I've been involved with lately. It seems to fit the bill pretty well and is incredibly extensible. In fact, it's probably the closest I've come to actually making a switch up to this point. The one major hurdle I have at the moment is lack of API support for blogging tools. Yes, I'm aware of the BlogAPI module, but according to the project page for it, it's incomplete, unsupported, and the author isn't working on it anymore. While I was able to install it and initially connect to the Drupal site, it doesn't seem that any of the posting functionality works at this time. Drupal remains the strongest competitor at this point and has a real chance of becoming my new platform of choice.

For the time being, however, I'm content with Serendipity. The community remains strong, there's a new release on the horizon, and, most important, it just works.

I'm writing this while several thousand feet up in the air, on a flight from here to there. I won't be able to publish it until I land, but that seems to be the exception these days rather than the norm.

And yet, while preparing for takeoff, the same old announcements are made. Turn off cell phones and pagers, disable wireless communications on electronic devices. And listening around me, hurried conversations between passengers as they ensure that all of their devices are disabled. As if a stray radio signal will cause the airplane to suddenly drop from the sky, or prevent it from taking off to begin with.

Why is it that we, as a society, cannot get over these simple hurdles. Plenty of studies have shown that these devices don't interfere with planes. In fact, some airlines are offering in-flight wireless access. Many airlines have offered in-flight telephone calls. Unless my understanding of flight is severely limited, I'm fairly certain that all of these functions use radio signals to operate. And yet we are still told that stray signals may cause planes to crash, may cause interference with the pilots instrumentation.

We need to get over this hurdle. We need to start spending our time looking to the future, advancing our technology, forging new paths. We need to stop clinging to outdated ideas. Learning from our past mistakes is one thing, and there's merit in understanding history. But lets spend our energy wisely and make the simple things we take for granted even better.

OSSEC has quickly become a primary weapon in my security toolkit.  It's flexible, fast, and very easy to use.  I'd like to share a few rules I've found useful as of late.

I primarily use OSSEC in a server/client setup.  One side effect of this is that when I make changes to the agent's configuration, it takes some time for it to push out to all of the clients.  Additionally, clients don't restart automatically when a new agent config is received.  However, it's fairly easy to remedy this.

First, make sure you have syscheck enabled and that you're monitoring the OSSEC directory for changes.  I recommend monitoring all of /var/ossec and ignoring a few specific directories where files change regularly. You'll need to add this to both the ossec.conf as well as the agent.conf.

<directories check_all="yes">/var</directories>
<ignore type="sregex">^/var/ossec/queue/</ignore>
<ignore type="sregex">^/var/ossec/logs/</ignore>
<ignore type="sregex">^/var/ossec/stats/</ignore>

The first time you set this up, you'll have to manually restart the clients after the new config is pushed to them. All new clients should work fine, however.

Next, add the following rules to your local_rules.xml file (or whatever scheme you're using).

<rule level="12" id="100005">
   <if_matched_group>syscheck</if_matched_group>
   <description>agent.conf changed, restarting OSSEC</description>
   <match>/var/ossec/etc/shared/agent.conf</match>
</rule>

This rule looks for changes to the agent.conf file and triggers a level 12 alert. Now we just need to capture that alert and act on it. To do that, you need to add the following to your ossec.conf file on the server.

<command>
    <name>restart-ossec</name>
    <executable>restart-ossec.sh</executable>
    <expect>srcip</expect>
    <timeout_allowed>no</timeout_allowed>
</command>

<active-response>
    <command>restart-ossec</command>
    <location>local</location>
    <rules_id>100005</rules_id>
</active-response>

You need to add this to the top of your active response section, above any other rules. OSSEC matches the first active-response block and ignores any subsequent ones. The restart-ossec.sh script referenced in the command section should exist already in your active-response/bin directory as it's part of the distribution.

And that's all there is to it. Whenever the agent.conf file changes on a client, it'll restart the OSSEC agent, reading in the new configuration.

Next up, a simple DoS prevention rule for apache web traffic. I've had a few instances where a single IP would hammer away at a site I'm responsible for, eating up resources in the process. Generally speaking, there's no real reason for this. So, one solution is to temporarily block IPs that are abusive.

Daniel Cid, the author of OSSEC, helped me out a little on this one. It turned out to be a little less intuitive than I expected.

First, you need to group together all of the "normal" error response codes. The actual error responses (400/500 errors) are handled with other, more aggressive rules, so you can ignore most of them. For our purposes, we want to trigger on 200/300/400 error codes.

<rule id="131105" level="1">
      <if_sid>31101, 31108, 31100</if_sid>
      <description>Group of all "normal" 200/300/400 error codes.</description>
</rule>

Next, we want to create a composite rule that will fire after a set frequency and time limit. In short, we want this rule to fire if X matches are made in Y seconds.

<rule id="131106" level="10" frequency="500" timeframe="60">
      <if_matched_sid>131105</if_matched_sid>
      <same_source_ip />
      <description>Excessive access, Temporary block</description>
</rule>

That should be all you need provided you have active response already enabled. You can also add a specific active response for this rule that blocks for a shorter, or longer, period of time. That's the beauty of OSSEC, the choice is in your hands.

I hope you find these rules helpful. If you have any questions or comments, feel free to post them below.

It's always interesting to see how new technologies alter the way we do things.  Recently, I worked on firewalling for a KVM-based virtualization platform.  From the outset it seems pretty straightforward.  Set up iptables on the host and guest and move on.  But it's not that simple, and my google-fu initially failed me when searching for an answer.

The primary issue was that when iptables was enabled on the host, the guests became unavailable.  If you enable logging, you can see the traffic being blocked by the host, thus never making it to the guest.  So how do we do this?  Well, if we start with a generic iptables setup, we have something that looks like this:

# Firewall configuration written by system-config-securitylevel
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT

Adding logging to identify what's going on is pretty straightforward.  Add two logging lines, one for the INPUT chain and one for the FORWARD chain.  Make sure these are added as the first rules in the chain, otherwise you'll jump to the RH-Firewall-1-INPUT chain and never make it to the log.

-A INPUT -j LOG --log-prefix "Firewall INPUT: "
-A FORWARD -j LOG --log-prefix "Firewall FORWARD: "

 

Now, with this in place you can try sending traffic to the domU.  If you tail /var/log/messages, you'll see the blocking done by netfilter.  It should look something like this:

Apr 18 12:00:00 example kernel: Firewall FORWARD: IN=br123 OUT=br123 PHYSIN=vnet0 PHYSOUT=eth1.123 SRC=192.168.1.2 DST=192.168.1.1 LEN=56 TOS=0x00 PREC=0x00 TTL=64 ID=18137 DF PROTO=UDP SPT=56712 DPT=53 LEN=36

There are a few things of note here.  First, this occurs on the FORWARD chain only.  The INPUT chain is bypassed completely.  Second, the system recognizes that this is a bridged connection.  This makes things a bit easier to fix.

My attempt at resolving this was to put in a rule that allowed traffic to pass for the bridged interface.  I added the following:

-A FORWARD -i br123 -o br123 -j ACCEPT

This worked as expected and allowed the traffic through the FORWARD chain, making it to the domU unmolested.  However, this method means I have to add a rule for every bridge interface I create.  While explicitly adding rules for each interface should make this more secure, it means I may need to change iptables while the system is in production and running, not something I want to do.

A bit more googling led me to this post about KVM and iptables.  In short it provides two additional methods for handling this situation.  The first is a more generalized rule for bridged interfaces:

-A FORWARD -m physdev --physdev-is-bridged -j ACCEPT

Essentially, this rule tells netfilter to accept any traffic for bridged interfaces.  This removes the need to add a new rule for each bridged interface you create making management a bit simpler.  The second method is to completely remove bridged interfaces from netfilter.  Set the following sysctl variables:

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

I'm a little worried about this method as it completely bypasses iptables on dom0.  However, it appears that this is actually a more secure manner of handling bridged interfaces.  According to this bugzilla report and this post, allowing bridged traffic to pass through netfilter on dom0 can result in a possible security vulnerability.  I believe this is somewhat similar to cryptographic hash collision.  Attackers can take advantage of netfilter entries with similar IP/port combinations and possibly modify traffic or access systems.  By using the sysctl method above, the traffic completely bypasses netfilter on dom0 and these attacks are no longer possible.

More testing is required, but I believe the latter method of using sysctl is the way to go.  In addition to the security considerations, bypassing netfilter has a positive impact on throughput.  It seems like a win-win from all angles.

Back when the Chernobyl nuclear reactor in the Ukraine melted down, I was in grade school. That disaster absolutely fascinated me and I spent a bit of time researching nuclear power, drawing diagrams of reactor designs, and dreaming about being a nuclear scientist.

One thing that stuck with me about that disaster was the sheer power involved. I remember hearing about the roof of the reactor, a massive slab of concrete, having been blown off the building. From what I remember it was tossed many miles away, though I'm having trouble actually confirming that now. No doubt there was a lot of misreporting done at the time.

The reasons behind the meltdown at Chernobyl are still a point of contention ranging from operator error to design flaws in the reactor. Chances are it is more a combination of both. There's a really detailed report about what happened here. Additional supporting material can be found on Wikipedia.

Today we have the disaster at the Fukushima power plants in Japan. Of course the primary difference from the get-go is that this situation was caused by a natural disaster rather than design flaws or operator error. Honestly, when you get hit with a massive earthquake immediately followed by a devastating tsunami, you're pretty much starting at screwed.

From what I understand, there are 5 reactors at two plants that are listed as critical. In two instances, the containment structure has suffered an explosion. Whoa! An explosion? Yes, yes, calm down. It's not a nuclear explosion as most people know it. Most people equate a nuclear explosion with images of mushroom clouds, thoughts of nuclear fallout, and radiation sickness. The explosion we're talking about in this instance is a hydrogen explosion resulting from venting the inner containment chamber. Yes, it's entirely possible that radiation was released, but nothing near the high dosages most people equate with a nuclear bomb.

And herein lies a major problem with nuclear power. Not many people understand it, and a large majority are afraid of the consequences. Yes, we have had a massive meltdown as is the case with Chernobyl. We've also had a partial meltdown as is the case with Three Mile Island. Currently, the disaster in Japan is closer to Three Mile Island than it is to Chernobyl. That, of course, is subject to change. It's entirely possible that the reactor in Japan will go into a full core meltdown.

But if you look at the overall effects of nuclear power, I believe you can argue that it is cleaner and safer than many other types of power generation have been. Coal power massively pollutes the atmosphere and leaves behind some rather nasty byproducts that we just don't have a method of dealing with. Oil and gas also cause pollution in both the atmosphere as well as the area surrounding where the oil and gas are mined. Water, wind, and sun power are, generally speaking, clean, but you have to have massive amounts of each to generate sufficient power.

Nuclear power has had such a negative stigma for such a long period of time that research dollars are not being spent on improving the technology. There are severe restrictions on what scientists can research with respect to nuclear power. As a result, we haven't advanced very far as compared to other technologies. If we were to open up research we would be able to develop reactors that are significantly safer.

Unfortunately, I think this disaster will make things worse for the nuclear power industry. Despite the fact that this disaster wasn't caused by design flaws, nor was there operator error, the population at large will question the validity of this technology they know nothing about. Personally, I believe we could make the earth a much cleaner, safer place to live if we were to switch to nuclear power and spend time and effort on making it safer and more efficient.

And finally, a brief note. I'm not a nuclear physicist or engineer, but I have done some background research. I strongly encourage you to do your own research if you're in doubt about anything I've stated. And if I'm wrong about something, please, let me know! I'll happily make edits to fix incorrect facts.

It's that time of year again.  Child's Play is an incredible charity dedicated to "dedicated to improving the lives of children with toys and games in our network of over 60 hospitals worldwide."  It's an organization started by the guys from Penny Arcade and supported by you.  I encourage you to open your heart and support the Child's Play Charity.

And so a week of fun comes to an end. But before we part company, there are a few more items to discuss. There are portions of OSSEC that are often overlooked because they require little or no configuration, and they "just work." Other features are either more involved, or require external products. Let's take a look at a few of these.

Rootkit Detection

First up, rootkit detection. OSSEC ships with a set of rules and "signatures" designed to detect common rootkits. The default OSSEC installation uses two files, rootkit_files.txt and rootkit_trojans.txt, as the base of the rootkit detection.

The rootkit_files.txt file contains a list of files commonly found with rootkit infections. Using various system calls, OSSEC tries to detect if any of these files are installed on the machine and sends an alert if they are found. Multiple system calls are used because some rootkits hide themselves by altering system binaries and sometimes by altering system calls.

The rootkit_trojans.txt file contains a list of commonly trojaned files as well as patterns found in those files when they have been compromised. OSSEC will scan each file and compare it to the list of patterns. If a match is found, an alert is sent to the administrator.

There are also additional rootkit files shipped with OSSEC. For Windows clients there are three files containing signatures for common malware, signatures to detect commonly banned software packages, and signatures for checking windows policy settings. On the Linux side are a number of files for auditing system security and adhering to CIS policy. CIS policy auditing will be covered later.

Rootkit detection also goes beyond these signature-based methods. Other detection methods include scanning /dev for unusual entries, searching for hidden processes, and searching for hidden ports. Rootkit scanning is pretty in-depth and more information can be found in the OSSEC Manual.

CIS Auditing

CIS, the Center for Internet Security, publishes a benchmark tool for auditing system security on various operating systems. OSSEC can assist with compliance to the CIS guidelines by monitoring systems for non-conformity and alerting as necessary. Shipped by default with OSSEC are three cis-based signature sets for Redhat Linux and Debian. Creating new tests is fairly straightforward and the existing tests can be adapted as needed.

OSSEC Splunk

One thing that OSSEC lacks is an easy way to pour through the OSSEC logs, get visual information on alerts, etc. There was a project, the OSSEC-WUI, that aimed to resolve this, but that project has mostly died. Last I heard, there were no plans to revive this project.

There is an alternative, however. A commercial product, Splunk, can handle the heavy lifting for you. Yes, yes, Splunk is commercial. But, good news! They have a free version that can do the same thing on a smaller scale, without all of the extra shiny. There is a plugin for Splunk, specifically designed to handle OSSEC as well. It's worth checking out, you can find it over at splunkbase.

Alert Logging

And finally, alert logging. Because OSSEC is tightly secured, it can sometimes be a challenge to deal with alert logs. For instance, what if you want to put the logs in an alternate location outside of /var/ossec? There are alternatives, though. For non-application specific output you can use syslog or database output. OSSEC also supports output to Prelude and beta support exists for PicViz. I believe you can use multiple output methods if you desire, though I'd have to test that out to be sure.

The configuration for syslog output is very straightforward. You can define both the destination syslog server as well as the level of the alerts to send. Syslog output is typically what you would use in conjunction with Splunk.

Database configuration is a bit more in-depth and requires that OSSEC be compiled with certain options enabled. Both MySQL and PostgreSQL are supported. The database configuration block in the ossec.conf ile contains all of the options you would expect, database name, username and password, and hostname of the database server. Additionally you need to specify the database type, MySQL or PostgreSQL.

Prelude and PicViz support have their own specific configuration parameters. More information on this support can be found in the OSSEC Manual.

Final Thoughts

OSSEC is an incredible security product. I still haven't reached the limits of what it can do and I've been learning new techniques for using it all week. Hopefully the information provided here over the last 7 days proves to be helpful to you. There's a lot more information out there, though and an excellent place to start is the OSSEC home page. There's also the OSSEC mailing list where you can find a great deal of information as well as a number of very knowledgeable, helpful users.

The best way to get started is to get a copy of OSSEC installed and start playing. Dive right in, the water's fine.

Now that you have a functional OSSEC installation it's time to start configuring it. Proper configuration is really the heart and soul of OSSEC and will be the primary focus for the next few days. The primary OSSEC configuration files use an XML format. If you're not familiar with XML, don't worry too much. It's a pretty easy format to wrap your head around, as you'll see.

The primary configuration file is the /var/ossec/etc/ossec.conf file. The only exception to this is when you are using centralized agent configuration, which I highly recommend for large deployments. For a centralized setup, the server configuration is located in ossec.conf and the agent configuration is located in /var/ossec/etc/shared/agent.conf. All agents are configured using a single agent.conf file. I'll explain how this works in a bit.

Before I get to the standard configuration options available to ossec.conf and agent.conf, let's talk briefly about agents and centralized agent configurations. When using a centralized configuration, I recommend minimizing what you place in the agent's ossec.conf file. The centralized agent.conf file overrides any conflicting option listed in the agent's ossec.conf file. Besides, lingering configuration options in the agent's ossec.conf can result in confusion when you're not aware of those settings. To make general configuration simple and straightforward, the only option that should be present in the agent's ossec.conf is the address of the server or servers to use.

<ossec_config>
   <client>
      <server-ip>192.168.0.1</server-ip>
   </client>
</ossec_config>

The agent.conf file contains all of the configuration for every agent in the network. An agent will receive all of the configuration information relevant to that specific agent. The agent.conf file is actually sent to every agent in the network and each agent parses the file looking for configuration settings relevant to that local agent. This means that you can define an initial "generic" configuration block that will apply to all agents, and then use successive configuration blocks to define additional configuration options. Note: Configurations are cumulative.

It's a bit easier to look at a simple configuration as an example. For now, don't worry too much about what these options do, instead, concentrate on seeing how the agent builds its configuration.

<agent_config>
   <syscheck>
      <!-- Frequency that syscheck is executed -- default every 2 hours -->
      <frequency>7200</frequency>

      <!-- Directories to check (perform all possible verifications) -->
      <!-- Files/directories to ignore -->
      <directories check_all="yes">/etc</directories>
      <ignore>/etc/adjtime</ignore>
   </syscheck>
</agent_config>
<agent_config name="agent1">
   <syscheck>
      <directories check_all="yes">/usr/agent1</directories>
   </syscheck>
</agent_config> 

The above configuration is *very* tiny and very basic. However, it should help to illustrate how the agent config works. The first agent_config block defines a generic configuration that will be used for every agent in the network. The subsequent agent_config block defines an additional directory to check for agent1. So agent1 will run with the primary configuration from the first block, plus the configuration from the second block.

Each agent_config block can be defined specifically for an agent or os using the agent and os attributes. The agent attribute is simple the name (or names separates by a |) of the agent(s) to apply that agent_config block to. The os attribute is based on the uname of the agent operating system. Some examples include Linux, FreeBSD, Windows, and Darwin.

The only other real difference between a standalone or server config and an agent config is that there are some options that belong in the server config as opposed to the agent config. For instance, when we talk about active response later in the week, the active response settings are placed in the server config. Likewise, the rules are detailed in the server config and not in the agent config. This will make more sense when we cover these topics. For now, if you're unsure as to where a particular configuration option belongs, be sure to check the OSSEC manual.

That's all for now. Check in tomorrow when we'll make OSSEC do something more than just sit there!

In a previous entry we discussed OSSEC Decoders and how they work. Decoders are only the first step in the log analysis train, though. Once log entries have been decoded, we need to do something with the results. That something is to match them up with rules so that OSSEC sends alerts and/or provides the proper response.

OSSEC rules ultimately determine what log entries will be reported and what entries will trigger active responses. And, as with decoders, rules can build upon one another. You can also chain rules together to alter the ultimate response based on conditions such as frequency and time. So let's take a look at a simple rule.

<rule noalert="1" level="0" id="5700">
   <decoded_as>sshd</decoded_as>
   <description>SSHD messages grouped.</description>
</rule>

This is one of the default rules that ships with OSSEC. Each rule is defined with a unique ID. To prevent custom rules from interfering with core rules, the developers have reserved a range of IDs for custom rules. That said, there is nothing preventing you from using whatever ID numbering you desire. Just keep in mind that if you use an ID that is reserved for something else by the developers, new versions of OSSEC may interfere.

Also listed in the rule tag is a level and a noalert attribute. The noalert attribute is pretty straightforward, this rule won't send an alert when it's matched. Typically, the noalert tag is used on rules that are designed to be built upon. The level tag determines what level alert this rule will trigger when matched. Because we're combining it with the noalert tag, the level ends up not meaning a whole lot.

The decoded_as tag identifies the parent decoder for which this rule is written. Only rules with a decoded_as tag matching the decoder used to decode the current log entry will be scanned for a match. This prevents OSSEC from having to scan every rule for a match.

The description tag is a human readable description of what the rule is. When you receive and alert, or when the alert is added to the OSSEC log, this description is added along with it. In the case of this rule, the description identifies its purpose. This rule was defined purely to group together sshd alerts. The intention is that other rules will handle the alerts for any alert-worthy ssh log entries that are detected.

Now let's look at something that builds on this basic rule. Again, choosing a rule from the default ruleset, we have this:

<rule id="5702" level="5">
   <if_sid>5700</if_sid>
   <match>^reverse mapping</match>
   <regex>failed - POSSIBLE BREAK</regex>
   <description>Reverse lookup error (bad ISP or attack).</description>
</rule>

The first thing to note about this rule is the if_sid tag. The if_sid tag says if the rule number defined in this tag has already matched the incoming log entry, then we want to see if this rule matches. In other words, this rule is a child of the rule identified in the if_sid tag.

The match tag defines a string we're looking for within the log entry. The regex tag also defines a string we're trying to match, but regex can use the full set of regular expressions that OSSEC supports. If both the match and the regex are found in the log entry, then this rule matches and will alert at a level of 5.

Finally, let's look at a more advanced rule. This rule also builds on the previous rules mentioned, but contains a few extras that make it even more powerful.

<rule timeframe="360" frequency="4" level="10" id="5703">
   <if_matched_sid>5702</if_matched_sid>
   <description>Possible breakin attempt </description>
   <description>(high number of reverse lookup errors).</description>
</rule>

The intention of this rule is to identify repeated log entries that point at a more severe problem than a simple error. In this case we have multiple incoming ssh connections with bad reverse DNS entries. The frequency and timeframe attributes define how many times within a specific timespan a particular rule has to fire before this rule will kick in.

Notice that instead of the if_sid tag, we're using the if_matched_sid tag. This is because we're not necessarily making this rule a child of another, but instead making it a composite rule. In other words, this rule is fired if another rule fires multiple times based on the setting within this rule. As a result, this rule fires with a level of 10.

But now that we have rules and alerts being generated, what else can we do? The answer is that we can trigger active responses based on those rules. Generally, an active response fires when an alert comes in with a level equal to or higher than the alert level of the active response definition. There are ways to alter this, but let's keep things simple for now.

To configure active response, first you need to define the commands that active-response will use. Note: All command and active-response configuration is placed in the ossec.conf on the server.

<command>
   <name>firewall-drop</name>
   <executable>firewall-drop.sh</executable>
   <expect>srcip</expect>
   <timeout_allowed>yes</timeout_allowed>
</command>

This snippet defines a command called firewall-drop. The command depends on an executable called firewall-drop.sh. This file must exist on all agents that will run this command. Unfortunately, there is currently no mechanism to push these files out to agents automatically, but perhaps there will be in the future? (*HINT*)

Th expect tag determines what variables are required from the alert in order to fire this command. Since this particular command adds a block rule to the server firewall, the srcip is required. And finally, timeout_allowed tells OSSEC that this command supports a timeout option. In other words, the firewall block is added and then removed after a given timeout.

Once the command is in place you need to set up the active response itself.

<active-response>
   <command>firewall-drop</command>
   <location>local</location>
   <level>6</level>
   <timeout>3600</timeout>
</active-response>

The active-response block identifies the command that will be run as well as the level at which it will fire. For this example, the firewall-drop command is run for any alert with a level of 6 or higher. We have also specified a timeout of 3600 which tells OSSEC that the command needs to be run again after 3600 seconds to remove the firewall-drop.

Also included is a location tag. This tells OSSEC where to run the command. Here we have specified local, which may be slightly confusing. This means that firewall-drop is run on the local agent that triggered the alert. So, if agent 002 triggers an ssh alert with a level of 6, then OSSEC tells agent 002 to run the firewall-drop command with a timeout of 3600 seconds.

You can also specify other options for location that allow you to run commands on the server or on specific agents. For instance, what if you want to block an IP at the edge whenever you get an alert of level 10 or higher. Perhaps you create a command called edge-block and it connects to your edge router to update an ACL located on the router. Running this on every agent is unwieldy at best and probably constitutes a significant security threat. Instead, you can have this script run on the server, or even a specific agent designed to handle this connection.

And that covers the basics for rules. I encourage you to write your own rules and test them out with the ossec-logtest program located in /var/ossec/bin. Learning to write rules is essential to running and tuning an OSSEC installation.

Tune in tomorrow for the final wrap-up of this year's Week of OSSEC!

As it turns out, before you can play with OSSEC and begin learning the intricacies of host-based intrusion detection, you need to install the software. Installation itself is pretty easy and fairly straightforward, provided you know how you want to install and run the system.

What do I mean by this? Well, there are three ways to install the OSSEC software. It can be installed as standalone, as a server, or as an agent. These are pretty much what they sound like. A standalone installation is generally used when you're administering a single machine, or when circumstances prevent you from running a centralized server. A server/agent installation is for larger installations with centralized management.

For both installation types you need to start with the raw source. From there you can run the install script and choose the type of installation you want. Alternatively, if you're installing on a linux distribution that uses RPM for package management, you can grab a copy of the source RPM for the latest version here. The RPM version only supports the server/agent type, though you can probably run the server as a standalone install. Note: I make no guarantees about the RPMs I provide, you are expected to know what you are installing and take appropriate precautions such as verifying what you have downloaded.

If you're going with the raw source, grab a copy here. The current release as of this writing is version 2.5. Download the source, unpack it, and run the installation script. The installation script must be run as root. You probably want to check the md5 and/or sha1 sum prior to installation, just to make sure that the code is original.

# wget http://www.ossec.net/files/ossec-hids-2.5.tar.gz
# sha1sum ossec-hids-2.5.tar.gz
3da46b493f0e50b2453c43990b46ba43e61648bf
# tar zxvf ossec-hids-2.5.tar.gz

# cd ossec-hids-2.5
# ./install.sh

Just follow the prompts an the installer will take care of the rest, including compiling the software.

The RPM install is simplified a bit as you only have to compile the code once per server architecture you want to support. In other words, if you have both 32-bit and 64-bit installations, you'll likely want to compile the software twice. Once compiled, you have three packages, ossec-hids, ossec-hids-server, and ossec-hids-client (client = agent). The ossec-hids package is installed on every system while the server and client packages go on the appropriate systems.

In order for the agents to talk to the server, you must have port 1514 open. Additionally, you need to register the agents with the server. This is a pretty simple process, though it has to be repeated for every agent. Detailed instructions can be found here. The short and simple version is as follows:

1) Run /var/ossec/bin/manage_agents and choose to add a new agent. Follow the prompts and enter the appropriate data.
2) While still in manage_agents, select the option to extract the agent authentication key. Copy this key as you need to paste it into the agent.
3) On the agent, run /var/ossec/bin/manage_agents. Choose to import the server key. Paste in the key you copied previously.

And that's all there is to adding a new agent. There are ways to script this, but they are a bit out of scope for an introduction article. If you're interested in scripting this process, please check the OSSEC mailing list for more details.

Finally, after installation comes configuration! Tune in tomorrow for more details!

Today marks the first day of the Week of OSSEC. What is OSSEC you ask? Well, I'm glad you asked. Allow me to explain.

According to the OSSEC home page, OSSEC is :

OSSEC is an Open Source Host-based Intrusion Detection System. It performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response.

It runs on most operating systems, including Linux, MacOS, Solaris, HP-UX, AIX and Windows.

But you have an IDS already, right? One of those big fancy all-in-one units that protects your network, has tons of signatures, and spits out endless reams of data. Or maybe you're on the Open Source track and you're using Snort combined with some fancy OSS front end for reporting.

Well, OSSEC is a bit different. It's a HIDS, not an IDS. In short, this means that OSSEC is designed to monitor a single host and react to events occurring on that host. While this doesn't sound very useful from the outset, I assure you that this is significantly more useful than you think.

For instance, let's look at a typical brute-force attack. Your average IDS will allow the traffic through the network as perfectly valid traffic. There is nothing unusual happening at a network layer, so the IDS is content with passing the traffic as normal. Once it hits the server, however, the attack makes itself clear, trying to brute force its way into the server. You can ignore this and hope your password policies are strong enough to cope, but if an attacker is persistent enough, they may eventually worm their way in. Another way to deal with this is through session throttling via the server's firewall. OSSEC deals with this by identifying the attack and blocking the attacker's IP.

Sure, fail2ban and other packages offer similar capabilities, but can they also be used to monitor file integrity? How about rootkit detection? Can you centralize their capabilities, using a single host to push out configurations to remote systems? Can they be used in an agentless configuration where a host is monitored without having to install software on it?

OSSEC is extremely powerful and is gaining more capabilities over time. It is in active development with no signs of slowing. In fact, version 2.5 was just released on September 27th.

Over the next week I'll be explaining some of OSSEC's capabilities. Ultimately, I suggest you install it on a development system and start poking at it yourself. You can also join the OSSEC mailing list and join in the various on-going conversations.

One of the strongest features of OSSEC is its log analysis engine. The log collector process monitors all of the specified logs and passes each log entry off to the analysis engine for decoding and rule matching. For agents, the log entries are transmitted to the server for processing.

There are multiple steps to identifying actionable matches starting with the decoding of each log entry. First, pre-decoding breaks apart a single log entry into manageable pieces. Decoders are then used to identify specific programs and log entry types. Decoders can build on one another allowing the operator to target specific patterns within a log entry with ease.

Let's take a look at a simple example first. The following is a sample log entry from a typical /var/log/secure log. This log uses the syslog format for log entries.

Oct 21 00:01:00 dev sshd[31409]: Failed password for invalid user postfix from 189.126.97.181 port 57608 ssh2

Pre-decoding breaks apart the syslog format into three pieces, the hostname, program_name, and log. Using the ossec-logtest program provided with OSSEC, we can view the process OSSEC goes through for decoding and then rule matching. Pre-decoding this log entry produces the following :

[root@dev bin]# ./ossec-logtest
2010/10/21 00:01:00 ossec-testrule: INFO: Reading local decoder file.
2010/10/21 00:01:00 ossec-testrule: INFO: Started (pid: 1106).
ossec-testrule: Type one log per line.

Oct 21 00:01:00 dev sshd[31409]: Failed password for invalid user postfix from 189.126.97.181 port 57608 ssh2

**Phase 1: Completed pre-decoding.
full event: 'Oct 21 00:01:00 dev sshd[31409]: Failed password for invalid user postfix from 189.126.97.181 port 57608 ssh2'
hostname: 'dev'
program_name: 'sshd'
log: 'Failed password for invalid user postfix from 189.126.97.181 port 57608 ssh2'

As you can see, the log entry ends up with three parts. Further decoding uses pattern matching on these three parts to further identify and categorize log entries.

<decoder name="sshd">
   <program_name>^sshd</program_name>
</decoder>

This is about as simple as a decoder can get. The decoder block starts with an attribute identifying the name of the decoder. In this case, that name is sshd. Within the decoder block is a program_name tag that contains a regular expression used to match against the program_name from pre-decoding. If the regex matches, OSSEC will use that decoder to match against any defined rules.

As I mentioned before, however, decoders can build on each other. The first decoder above reduces the number of subsequent decoders that need to be checked before decoding is complete. For example, look at the following decoder definition :

<decoder name="ssh-invfailed">
   <parent>sshd</parent>
   <prematch>^Failed \S+ for invalid user|^Failed \S+ for illegal user</prematch>
   <regex offset="after_prematch">from (\S+) port \d+ \w+$</regex>
   <order>srcip</order>
</decoder>

This decoder builds on the first as can be seen via the parent tag. Decoders work on a first match basis. In other words, the first decoder to match is used to find secondary decoders (children of the first), the first secondary decoder is used to find tertiary (children of the second), etc. If the matching decoder has no children, then that decoder is the final decoder and the decoded information is passed on to rules processing.

There are three other tags within this decoder block worth looking at. First, the prematch tag. Prematch is used as a quick way to determine if the rest of the decoder should be run. Prematches should be written so that the portion of the entry they match can be skipped by the rest of the decoder. For instance, in the decoder example above, the prematch will match the phrase "Failed password for invalid user" in the log entry. This portion of the log contains enough information to identify the type of log entry without requiring us to parse it again to extract information. The remaining part of the log entry has the information we want to capture.

Which brings us to the regex. The regex, or regular expression, is a string used to match and pull apart a log entry. The regex expression in the example is used to extract the source ip address from the log so we can use it in an active response later. The order tag identifies what the extracted information is.

Now, using these two decoders, let's run ossec-logtest again :

2010/10/21 00:01:00 ossec-testrule: INFO: Reading local decoder file.
2010/10/21 00:01:00 ossec-testrule: INFO: Started (pid: 28358).
ossec-testrule: Type one log per line.

Oct 21 00:01:00 dev sshd[31409]: Failed password for invalid user postfix from 189.126.97.181 port 57608 ssh2

**Phase 1: Completed pre-decoding.
full event: 'Oct 21 00:01:00 dev sshd[31409]: Failed password for invalid user postfix
from 189.126.97.181 port 57608 ssh2'
hostname: 'dev'
program_name: 'sshd'
log: 'Failed password for invalid user postfix from 189.126.97.181 port 57608 ssh2'

**Phase 2: Completed decoding.
decoder: 'sshd'
srcip: '189.126.97.181'

As you can see, the decoding phase has identified the decoder as sshd. The logtest program reports the name of the parent decoder used, rather than the ultimate matching decoder.

Hopefully this has given you enough information to start writing your own decoders. The decoder.xml file that comes with OSSEC is a great place to look at examples of how to craft decoders. This is a more advanced task, however, and the existing decoders cover most of the standard log entries you'll see on a Linux or Windows machine.

For more information on decoders, please see the OSSEC manual. You might also check out chapter 4 of the OSSEC book. The book is a little outdated now, but the information within is still quite accurate. Syngress released a few chapters of the book that you can download here.

One of the simplest functions that OSSEC can perform is integrity monitoring. The premise is pretty simple, let the admin know if a file has changed. More often than not, the admin will already know that the file has changed because the admin is the one that changed it. But sometimes files change because of problems in the system that the admin doesn't know about. Or, the files may change because the server has been compromised by an outside party that has installed rogue software. Either way, the admin needs this information.

OSSEC can be configured to look at a few different aspects of a file in order to determine if it has changed or not, depending on how you configure it. But before we get to that, let's configure OSSEC to send us alerts to begin with.

There are a number of ways OSSEC can send alerts. Alerts can be sent via syslog, email, stored in a database, or sent to third-party programs such as Prelude and Picviz. To make things a bit simpler, I'm only detailing how to set up email. If you're interested in other alert setups, please check the OSSEC manual.

Setting up email alerts is as simple as adding two sections of configuration to the ossec.conf file. This configuration is set on the server, or on a standalone installation. In a server/agent setup, the server sends all alerts, including those for agents.

<global>
   <email_notification>yes</email_notification>
   <email_to>myaddress@example.com</email_to>
   <smtp_server>smtp.example.com</smtp_server>
   <email_from>ossec@example.com</email_from>
</global>

This first bit of configuration defines the To: and From: addresses as well as the SMTP server address. This configuration goes into the global config section which has a number of other options as well.

<alerts>
   <log_alert_level>1</log_alert_level>
   <email_alert_level>6</email_alert_level>
</alerts>

This portion of the configuration defines what level alerts should be sent via email and what level alerts should be logged. Don't worry too much about what a level is, you'll learn this in a later blog entry when we discuss rules and active response. For now, the config as shown above is enough.

Now that we'll receive alerts that OSSEC generates, let's set something up to send an alert!

As I mentioned before, integrity monitoring is pretty straightforward. OSSEC uses a number of different characteristics to identify when a file changes. It's pretty easy to determine that a file has changed if the owner, permissions, or size changes. OSSEC can also be configured to check the sha1 and/or md5 hash values as well. Hashing is a way of producing a "signature" for a file that is mostly unique. It is possible to create another file with the same hash, but it's very difficult. Combining all of these checks together makes it very improbable that an intruder can replace a file without you knowing.

Enabling syscheck is done on a per-host basis. What I mean by this is that the config is added to the ossec.conf file for server or standalone systems, and the config is added to the agent.conf for agents. As with other agent configurations, you can specify syscheck blocks for all agents as well as cumulatively for specific agents.

<syscheck>
   <frequency>7200</frequency>

   <directories check_all="yes">/etc</directories>
   <ignore>/etc/adjtime</ignore>
</syscheck>

The above config is an example of how to enable syscheck. The frequency block specifies the time, in seconds, between syscheck runs. The example above runs syscheck every 2 hours. Moving further down in the config is the directories tag. Simply put, this tag identifies what directories syscheck should be checking. The directories tag can define multiple directories, separated by a comma. The check_all attribute indicates that all of the previously mentioned checks should be run. The OSSEC manual details what other attributes are available. One other attribute worth mentioning is the realtime attribute. This attribute directs OSSEC to use inotify to alert when files changes, in real time, for quicker notification. Linux is the only OS, that I am aware of, that supports this option. Please check the manual for more information on realtime use.

There are times when you want to ignore files within a directory, or even certain subdirectories. The ignore tag allows you to accomplish this. Without defining any attributes, the ignore tag must define an exact match. For instance, in the example above, the file /etc/adjtime will be ignored. However, /etc/adjtime.new or /etc/adjtime0 will not be. To ignore these files, you will need to either add explicit ignore blocks for them, or you can use a regular expression to grab them all. The type attribute allows you to specify that this tag contains a regular expression. For instance :

<ignore type="sregex">^/etc/adjtime</ignore>

This ignore block will drop any file (including path) that starts with /etc/adjtime. Information on the regular expression syntax is in the OSSEC Manual.

There are a number of other tags available for syscheck that you can find in the manual. Among these are some more useful ones such as alert_new_files and, if you're in a Windows environment, windows_registry. I encourage you to check out these options and identify which, if any, would benefit your environment.

When OSSEC detects that a file has changed, it will send an alert. Alerts will be reported via the alert mechanism you have defined within the configuration. Syscheck alerts are, by default, set to a level of 7, so our alert settings will result in an email being sent. An example alert is below :

OSSEC HIDS Notification.
2010 Oct 13 08:08:05

Received From: (Example) 192.168.0.1->syscheck
Rule: 550 fired (level 7) -> "Integrity checksum changed."
Portion of the log(s):

Integrity checksum changed for: '/etc/adjtime'
Old md5sum was: '7234e9b2255e62178c5650982bae9cbc'
New md5sum is : '01210c2018146c2a9ca89505118c42f8'
Old sha1sum was: 'df60021e39119b42a4ae508ad19b65019df089af'
New sha1sum is : '694b403b74a2aa339ba323b65a6d724aa8129e3b'

--END OF NOTIFICATION

OSSEC makes some attempts at identifying false positives by automatically ignoring files that change frequently. By default, ossec will begin ignoring any file that has changed three times. You can change this behavior by creating custom rules that override the defaults. I'll be covering rules in my next blog post, so stay tuned!

I have a brain teaser for ya.. I'm looking for a way to solve a SQL problem efficiently, specifically using MySQL. The goal is to get a count of the number of unique rows returned for a complex query. It's actually for a pagination system so I can determine the limits necessary to efficiently query the database for the right amount of data rather than return everything and try to brute force it.

Let's say I have three tables as follows :

mysql> describe person;
+-------+------------------+------+-----+---------+-------+
| Field | Type             | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+-------+
| id    | int(10) unsigned | NO   | PRI | NULL    |       |
| first | char(15)         | YES  |     | NULL    |       |
| last  | char(15)         | YES  |     | NULL    |       |
+-------+------------------+------+-----+---------+-------+
3 rows in set (0.02 sec)

mysql> describe interests;
+----------+------------------+------+-----+---------+-------+
| Field    | Type             | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+-------+
| id       | int(10) unsigned | NO   | PRI | NULL    |       |
| interest | char(15)         | YES  |     | NULL    |       |
+----------+------------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

mysql> describe interest_link;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| person_id   | int(10) unsigned | NO   |     | NULL    |       |
| interest_id | int(10) unsigned | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

Simple enough. I'm mapping interests to people. I've entered data into these tables as follows :

mysql> select * from person;
+----+-------+----------+
| id | first | last     |
+----+-------+----------+
|  1 | John  | Doe      |
|  2 | Bob   | Jones    |
|  3 | Joe   | Smith    |
+----+-------+----------+
3 rows in set (0.00 sec)

mysql> select * from interests;
+----+-----------+
| id | interest  |
+----+-----------+
|  1 | Computers |
|  2 | Music     |
|  3 | Food      |
|  4 | Beer      |
|  5 | Gaming    |
+----+-----------+
5 rows in set (0.00 sec)

mysql> select * from interest_link;
+-----------+-------------+
| person_id | interest_id |
+-----------+-------------+
|         1 |           1 |
|         1 |           2 |
|         1 |           4 |
|         2 |           1 |
|         2 |           5 |
|         2 |           4 |
|         3 |           3 |
|         3 |           2 |
|         3 |           4 |
+-----------+-------------+
9 rows in set (0.00 sec)

So far, so good. Now, I want to do a search to find which users are interested in music. Simple enough search, I'd do this with a simple select statement as follows :

mysql> select * from person as p left join interest_link as il on il.person_id = p.id where interest_id = 2;
+----+-------+--------+-----------+-------------+
| id | first | last   | person_id | interest_id |
+----+-------+--------+-----------+-------------+
|  1 | John  | Doe    |         1 |           2 |
|  3 | Joe   | Smith  |         3 |           2 |
+----+-------+--------+-----------+-------------+
2 rows in set (0.00 sec)

But what if I want to find out who's interested in music *and* beer?

mysql> select * from person as p left join interest_link as il on il.person_id = p.id where interest_id in (2,4);
+----+-------+----------+-----------+-------------+
| id | first | last     | person_id | interest_id |
+----+-------+----------+-----------+-------------+
|  1 | John  | Doe      |         1 |           2 |
|  1 | John  | Doe      |         1 |           4 |
|  2 | Bob   | Jones    |         2 |           4 |
|  3 | Joe   | Smith    |         3 |           2 |
|  3 | Joe   | Smith    |         3 |           4 |
+----+-------+----------+-----------+-------------+
5 rows in set (0.00 sec)

That's a problem, now I have 5 rows.. How do I make this a unique list? Well, I'm merely interested in names and ids, so I can do this :

mysql> select p.id, p.first, p.last from person as p left join interest_link as il on il.person_id = p.id where interest_id in (2,4);
+----+-------+----------+
| id | first | last     |
+----+-------+----------+
|  1 | John  | Doe      |
|  1 | John  | Doe      |
|  2 | Bob   | Jones    |
|  3 | Joe   | Smith    |
|  3 | Joe   | Smith    |
+----+-------+----------+
5 rows in set (0.00 sec)

but that's still 5 rows.. so what now?

mysql> select distinct p.id, p.first, p.last from person as p left join interest_link as il on il.person_id = p.id where interest_id in (2,4);
+----+-------+----------+
| id | first | last     |
+----+-------+----------+
|  1 | John  | Doe      |
|  2 | Bob   | Jones    |
|  3 | Joe   | Smith    |
+----+-------+----------+
3 rows in set (0.00 sec)

Aha! perfect. That's what I need.. almost. For this particular application, I want to paginate, so I need a total number of matching rows so I can properly identify the limits as well as the upper bound on page numbers. So, I'll just replace the specific field names with a count(*) :

mysql> select distinct count(*) from person as p left join interest_link as il on il.person_id = p.id where interest_id in (2,4);
+----------+
| count(*) |
+----------+
|        5 |
+----------+
1 row in set (0.00 sec)

And here is where I'm stuck. I need the total count of DISTINCT names, not the total number of rows returned. I tried a GROUP BY, but that didn't help much :

mysql> select count(*) from person as p left join interest_link as il on il.person_id = p.id where interest_id in (2,4) group by p.id;
+----------+
| count(*) |
+----------+
|        2 |
|        1 |
|        2 |
+----------+
3 rows in set (0.00 sec)

Sure, I get 3 rows, but what I'm looking for here is a single row with the total number of items. ... So, what if I count the number of returned rows! :

mysql> select count(*) from (select count(*) from person as p left join interest_link as il on il.person_id = p.id where interest_id in (2,4) group by p.id) as foo;
+----------+
| count(*) |
+----------+
|        3 |
+----------+
1 row in set (0.00 sec)

BUT... at what cost? This seems like a rather complex query that might break down, significantly, when there's a lot of data.. And the examples above are rather simplistic. In reality, we're talking about more fields and more tables, so the simpler query gets a little complex to begin with. I'm open to ideas on how to do this properly via SQL. Yes, I am aware of indexing and how that speeds things up. I use indexing, I just eliminated it from the above example to simplify things. I'm open to ideas on how to do this properly via SQL.

I can simply return all the rows with the distinct clause, count them programmatically, and then proceed with the rest of the program, but depending on the selections made by the user, there could be a significant amount of data returned. I'm worried about both memory exhaustion on the part of the scripting language, as well as the processing and transmission time required to pass all of that data back to the program from the SQL database. Besides, this is the sort of problem that SQL was designed to solve.

I don't think this is a unique problem, so someone out there has a solution. Perhaps the subselect *is* the better solution, but I don't think so. I'm open to ideas. You can leave a comment here, or hit me up on twitter.