You got your web in my firewall

This post first appeared on Redhat’s Enable Sysadmin community. You can find the post here.

Firewalls have been around in one form or another since the beginning of networking. The first firewalls weren’t even identified as firewalls. They were nothing more than physical barriers between networks. It wasn’t until the 1980’s that the first device specifically designed to be, and named, a firewall was developed by DEC. Since then, firewalls have evolved into a myriad of forms.

But what is a firewall? At its core, a firewall is a devices designed to allow or deny traffic based on a set of rules. Those rules can be as simple as “allow http and block everything else” or can be infinitely more complex including protocols, ports, addresses, and even application fingerprinting. Some modern firewalls have even incorporated machine learning into the mix.

Like other technologies, as firewalls have evolved, some niche uses have been identified. Web Application Firewalls (WAFs) are one of those niche uses. A WAF is a firewall specifically designed to handle “web” traffic. That is, traffic using the HTTP protocol. Generally speaking, the role of a WAF is to inspect all HTTP traffic destined for a web server, discard “bad” requests, and pass “good” traffic on. The details of how this works are, as you might suspect, a bit more complicated.

Much like “normal” firewalls, a WAF is expected to block certain types of traffic. To do this, you have to provide the WAF with a list of what to block. As a result, early WAF products are very similar to other products such as anti-virus software, IDS/IPS products, and others. This is what is known as signature-based detection. Signatures typically identify a specific characteristic of an HTTP packet that you want to allow or deny.

For instance, WAFs are often used to block SQL Injection attacks. A very simplistic signature may just look for key identifying elements of a typical SQL Injection attack. For instance, it may look for something like ' AND 1=1 included as part of the GET or POST request. If this matches an incoming packet, the WAF marks this as bad and discards it.

Signatures work pretty well but require a lot of maintenance to ensure that false positives are kept to a minimum. Additionally, writing signatures is often more of an art form rather than a straight-forward programming task. And signature writing can be quite complicated as well. You’re often trying to match a general attack pattern without also matching legitimate traffic. To be blunt, this can be pretty nerveracking.

To illustrate this a bit more, let’s look at ModSecurity. The ModSecurity project is an open source WAF project. It started out as a module for the Apache web server but has since evolved into a modular package that works with IIS, nginx, and others. ModSecurity is a signature-based WAF and often ships with a default set of signatures known as the OWASP ModSecurity Core Rule Set.

The Core Rule Set (CRS) is an excellent starting point for deploying a signature-based WAF. It includes signatures for all of the OWASP Top Ten web application security risks as well as a wide variety of other attacks. The developers have done their best to ensure that the CRS has few false alerts but, inevitably, anyone deploying the CRS will need to tweak the rules. This involves learning the rules language and having a deep understanding of the HTTP protocol.

Technology evolves, however, and newer WAF providers are using other approaches to block bad traffic. There has been a pretty widespread move from static configuration approaches such as allow and block lists to more dynamic methods involving APIs and machine learning. This move has been across multiple technologies including traditional firewalls, anti-virus software, and, you guessed it, WAFs.

In the brave new world of dynamic rulesets, WAFs use more intelligent approaches to identifying good and bad traffic. One of the “easier” methods employed is to put the WAF in “learning” mode so it can monitor the traffic flowing to and from the protected web server. The objective here is to “train” the WAF to identify what good traffic looks like. This may include traffic that matches patterns labeled as bad when signatures were used. Once the WAF has been trained, it’s moved to enforcement mode.

Training a WAF like this is similar to what happens when you train an email system to identify spam. Email systems often use a Bayesian filtering algorithm to identify spam. These algorithms work relatively well but can be poisoned to allow spam. Similar issues exist with algorithms used by WAF providers, especially when the WAF is in the learning mode.

More advanced WAF providers are using proprietary techniques to allow and block traffic. These techniques include algorithms that can identify whether certain attacks will work against the target system and only blocking those that would be harmful. Advanced techniques like this, however, are typically only found in WAF SaaS providers and not in self-contained WAF appliances.

WAFs, and firewalls in general, have evolved a lot over the years, moving from static to dynamic methods for identifying and blocking traffic. These techniques will only get better in the future. There are a variety of solutions available from open-source to commercial providers. No matter what your needs, there’s a WAF out there for you.

Rising from the ashes

*cough* *cough*

Awfully dusty in here. Almost as if this place were abandoned. Of course, that was never the case, was it. Just a hiatus of sorts. A reprieve from the noise and the harshness of reality.

But it’s time, now. Time to whip this place back into shape. Time to put the pieces back together. Time to build something new and interesting.

I know it’s been a while, but it’s time to get back in the habit. I’ve learned a lot these past years and I want to start sharing it. Soon.

Boldly Gone

I have been and always shall be your friend.

It’s a sad day. We’ve lost a dear friend today, someone we grew up with, someone so iconic that he inspired generations. At the age of 83, Leonard Nimoy passed away. He will be missed.

It’s amazing to realize how much someone you’ve never met can mean to you. People larger than life, people who will live on in memory forever. I’ve been continually moved for hours at the outpouring of grief and love online for Leonard. He has meant so much for so many, and his memory will live on forever.

Of all the souls I have encountered in my travels, his was the most… human.

Programming Note

In 2012 I posted a little over a dozen entries to this blog. I like to think that each entry was well thought out and time well spent. But only a dozen? That’s about one entry a month… I’d really like to do more.

So, new year, time to make some changes.. I spent a lot of time judging whether each post was “worth the effort” and “long enough to matter.” I need to get past that. My goal is to start posting a number of smaller entries. I definitely want the quality to be there, but I want to avoid agonizing over each and every entry.

So here’s to a new year and more content!

Contemplating the Future

In 2005 I obtained a job at a regional ILEC as a Data Operations Technician. As part of this job, I took over development of one of the tools we used to diagnose customer DSL connections. Problem was, this tool was written in PHP, a programming language I was, as yet, unfamiliar with.

At the same time, I was also looking for a web-based tool I could use to keep track of various tasks. While there were a few open-source tools I could use, none had the features I was looking for. So I decided to write one myself, and to write it in PHP so I could learn the language better. In the end, I’m glad I did as PHP has become indispensable for writing web-based tools.

The tool I wrote was a web-based todo manager called phpTodo. Since the alpha release in 2005, I have released 7 more versions. Work on phpTodo has ebbed and lowed with time, often interrupted by work and life in general. In fact, the last formal release was made almost 5 years ago, bringing the current version up to 0.8.1. In 2009, I found out that phpTodo was being packaged and released with Fedora as well.

After releasing 0.8.1, I decided to switch from using categories to using tags, similar to how the blogging system I use, Serendipity, uses them. This required rewriting a good deal of the back end of the system, as well as making extensive changes to the front end. I also started using the Prototype and Scriptaculous Javascript frameworks, and then later switched to jQuery. In all, a great deal of code has been rewritten.

I’m quite happy with the general feel of the new version I’ve been working on. While there is a good deal more code to be written, I’m confident there will be a code release soon enough.

I’ve been thinking a lot about the future of phpTodo and where I want to take it. When I originally started, I wrote the system such that I could see my todo list items via an RSS feed. At the time, I had a Blackberry phone and this worked brilliantly. Of course, this was purely a one-way feed with no way to update any todo items on the go. Since that time, I started working on a mobile view for the system, but stopped quickly after I realized how horrible working with WAP was. Fortunately, technology has progressed quickly since that time and WAP is no longer necessary. So, I’m considering working on a mobile version again.

A mobile version brings new challenges, however. It should be trivial to develop a mobile view that can be used while online, but my hope was to have an offline version as well that can be synchronized with the online version. One possibility is to develop an app that can be loaded onto a phone. That, of course, severely limits the platforms it can be run on. Another possibility is an HTML5 version, though that brings challenges of its own.

Another thought was to build a web service into phpTodo. The basic premise is an XML generator that, given a set of parameters, can supply an XML feed for external systems to use as input. And an XML parser that can receive data from external systems in order to update phpTodo data. I believe this can be used as the interface for the mobile view.

A web service can also be used to power another idea I had. I stumbled across the website of Brett Terpstra a while back and found a treasure trove of interesting ideas and useful code snippets. Among these is an obsession for recording notes to keep track of projects, interesting ideas, and helpful code snippets. Brett uses a number of custom scripts and software packages, most of which are exclusive to his platform of choice, OS X. To be honest, I find this incredibly intriguing, and potentially useful. So, I’ve been thinking about developing a command-line tool I can use to interact with phpTodo. A web service could make this a great deal easier.

I have no plans to stop working on the project, and, in fact, I’m eager to keep moving forward. As I continue to rely on phpTodo itself for my daily work, I rely on improvements I can make to the system. So overall, the future of phpTodo is bright.

Mega Fail

So this happened :

Popular file-sharing website Megaupload shut down
Megaupload shut down by feds, seven charged, four arrested
Megaupload assembles worldwide criminal defense
Department of Justice shutdown of rogue site MegaUpload shows SOPA is unnecessary
And then.. This happened :

Megaupload Anonymous hacker retaliation, nobody wins

And, of course, the day before all of this happened was the SOPA/PIPA protest.

Wow.. The government, right? SOPA/PIPA isn’t even on the books, people are up in arms over it, and then they go and seize one of the largest file sharing websites on the planet! We should all band together and immediately protest this illegal seizure!

But wait.. hang on.. Since when does jumping to conclusions help? Let’s take a look and see what exactly is going on here.. According to the indictment, this case went before a grand jury before any takedown was performed. Additionally, this wasn’t an all-of-a-sudden thing. Megaupload had been contacted in the past about copyright violations and failed to deal with them as per established law.

There are a lot of people who are against this action. In fact, the hacktivist group, Anonymous, decided to display their dictate by performing DDoS attacks against high profile sites such as the US DoJ, MPAA, and RIAA. This doesn’t help things and may actually hurt the SOPA/PIPA protest in the long run.

Now I’m not going to say that the takedown was right and just, there’s just not enough information as of yet, and it may turn out that the government was dead wrong with this action. But at the moment, I have to disagree with those that point at this as an example of an illegal takedown. As a friend of mine put it, if the corner market is selling illegal bootleg videos, when they finally get raided, the store gets closed. Yes, there were legal uses of the services on the site, but the corner store sold milk too.

There are still many, many copyright and piracy issues to deal with. And it’s going to take a long time to deal with them. We need to be vigilant, and protesting when necessary does work. But jumping to conclusions like this, and then attacking sites such as the DoJ are not going to help the cause. There’s a time and a place for that, and I don’t believe we’re there yet.

Who turned the lights out?

You may have noticed that a number of websites across the Internet today have modified their look a bit. In many cases, the normal content of that site is unreachable. Why would they do such a thing, you may ask? Well, there are two proposed laws, SOPA and PIPA, that threaten what we, today, enjoy as the Internet. The short version of these laws is that, basically, if you’re found to have any material on your website that infringes copyright, you face having your website shut down, without due process, all of your advertising pulled, being stricken from search engines, and possible jail time. Pretty draconian. There are a number of places that can explain, in more detail, what the full text of the legislation says. If you’re interested, check out americancensorship.org or eff.org.

Or, you can check out this video, from ted.com, that explains the legislation and why it’s so bad.

e

If you’re coming here after the 18th of January, here are some images of the protesting.

Google

 

Wikipedia

 

Wired.com

Blacklisted!

Back in October of 2011, a bill was introduced in the House of Representatives called HR.3261, or the “Stop Online Privacy Act (SOPA).” Go take a look, I’ll wait. It’s a relatively straightforward bill, especially compared to others I’ve looked at. Hell, it’s only 15 pages long! And it’s going to kill the Internet.

Ok,ok.. It won’t *KILL* the Internet, but it has the potential to ruin what we consider to be the Internet. Personally, I believe that if this passes, it has the potential to turn the Internet into nothing more than a collection of business websites, at least in the US.

So how does this thing work? Well, it’s actually pretty straightforward. If your website is suspected of infringing on copyrighted material, your website is taken down, any advertising you have on your site is cut, and you are removed from search engines. But so what, you deserve it! You were breaking copyright law!

Not so fast. This applies to *any* content on your website. So if someone comments on a blog entry, or you innocently link to a website that infringes copyright, or other situations out of your control, you’re responsible. Basically, you have to police every single comment, link, etc. that appears on your website.

It’s even worse for service providers since they have to do the blocking. Every infringing site is blocked via DNS. And since the US doesn’t have control of all of DNS, and some infringing sites are not located in the US, this means we move into the realm of having DNS blacklist files. The ISP becomes the responsible party if they fail to block these sites, which in turn means more overhead for the ISP. Think you pay a lot for Internet access now?

So what can you do? Well, for one, you can contact your representative and tell them how insane this whole idea is. And you can protest SOPA itself by putting up a protest overlay on your site. There’s a github project with all of the source code you need to add an overlay to your website. Or, if you have a Serendipity web blog, you can download the Stop SOPA plugin I’ve written.

Get out there and protest!

In Memorium – Steve Jobs – 1955-2011

Somewhere in the early 1980’s, my father took me to a bookstore in Manhattan. I don’t remember why, exactly, we were there, but it was a defining moment in my life. On display was a new wonder, a Macintosh computer.

Being young, I wasn’t aware of social protocol. I was supposed to be awed by this machine, afraid to touch it. Instead, as my father says, I pushed my way over, grabbed the mouse, and went to town. While all of the adults around me looked on in horror, I quickly figured out the interface and was able to make the machine do what I wanted.

It would be over 20 years before I really became a Mac user, but that first experience helped define my love of computers and technology.

Thank you, Steve.