Common PHP Regular Expression Security Issue

Stefan Esser (PHP Security Blog, Suhosin) recently posted an entry on his blog titled “Holes in most preg_match() filters” about a possible security issue that apparently escapes a lot of notice.

Let me explain the situation.  PHP uses Perl Compatible Regular Expressions, PCRE, for pattern matching.  In PCRE the carat metacharacter (^) is used to match the very beginning of the string, and the dollar-sign metacharacter ($) is used to match the end of the string.  This is extremely useful to ensure that the expression you’ve written has matched the entire string.

However, PCRE_DOLLAR_ENDONLY is not used by default.  This means that the dollar-sign metacharacter still matches to the end of the string, but it also matches is a newline character is at the end of the string.  In other words, a newline character may, or may not be present at the end of the string and you won’t know either way by default.

So, how do we fix this then?  Well, there are two ways.  First, you can add a D modifier to the end of the regular expression like this :

preg_match(‘/^[a-z]+$/D’, $string);

Or, you can use the \z modifier like this :

preg_match(‘/^[a-z]+\z/’, $string);

Either method works, although from the comments at Stefan’s site, it looks like \z is more portable since Perl doesn’t support the D modifier.

Here is short script to “prove” this, as it were :

 

$badstring = urldecode(“test%0a”);

if (preg_match(‘/^[0-9a-zA-Z]+$/’, $badstring)) {

print “Test 1 MATCHES\n”

}

if (preg_match(‘/^[0-9a-zA-Z]+$/D’, $badstring)) {

print “Test 2 MATCHES\n”

}

if (preg_match(‘/^[0-9a-zA-Z]+\z/’, $badstring)) {

print “Test 3 MATCHES\n”

}

 

I’m posting this info for two reasons.  First, it’s something programmers need to know.  It’s important since security holes are a bad thing.  Second, I’m guilty of this myself.  phpTodo used the dollar-sign metacharacter without the D modifier, making my code somewhat insecure.

The good news is that I have corrected the problem and posted a new version.  This is a precautionary measure, I don’t believe this adversely affected the security of the application, but better safe than sorry.  Head over and grab the new version just to be on the safe side.

phpTodo 0.8 Beta Released

A new version of phpTodo, version 0.8 Beta, was released today.  It’s been almost six months since the last release, mostly due to lack of time.  My primary goal for this release was to add ATOM support and get all the bugs fixed.  I feel I was able to accomplish both of these goals.

I think an official 1.0 release is imminent, assuming I have time to work on the program.  I have a few features I’d like to add before 1.0 if I can.  If they do get added, a 0.9 gamma version will be released before 1.0 becomes official.

After the 1.0 release, I’d like to get group support added.  In addition, I’m thinking about switching from single category based tasks to tags.  This would allow a single todo item to be placed into several categories at the same time.  Feed support will be updated as well, keeping in-line with the current feature set.

 

Overall, I’m quite happy with this project.  It’s helped me out in numerous ways, organizing my personal todo lists as well as giving me the opportunity to work on an open-source project.  I’d love to hear some feedback concerning this project, especially if you’re using it on a daily basis.  I’m definitely open to suggestions for improvements and I’d like to get some additional CSS layouts to include with the distribution.  You can leave any comments you may have right here on this blog entry.

Thanks to everyone who has already sent me suggestions and bug reports.  I hope to hear from more of you soon!  If you’re interested in trying out phpTodo, check out the demo site.

Book Review : Pro PHP Security

I just finished reading Pro PHP Security by Chris Snyder and Michael Southwell. I’m always looking for ways to improve my programming skills and security is an area I try to focus on. Secure web applications are becoming more important every day as more and more of our lives are placed online. With that in mind, I purchased this book to increase my PHP skills.

Overall the book was quite good. The book is broken into four parts. Part one is a general overview of security and it’s importance. Nothing really new here, but a good introduction nonetheless.

Part two delves into server-side security, outside of the realm of PHP. This includes shared hosts, firewalls, software installation, and more. None of this is really PHP specific per se, but still important topics. There is a decent introduction to encryption and it’s importance in security. There are a few PHP examples throughout these chapters that show how to handle SSH, SSL, hashing, and general encryption using the mcrypt() function. Part two concludes with an overview of authentication, permissions, and restrictions. There is a decent example of a single sign-on system, as well as an overview of PHP safe mode.

Part three covers more in-depth PHP programming practices designed to prevent many of the more common security problems. This section starts with a chapter on input validation, a very important topic indeed. The authors explain what input validation is and how to accomplish it. There are several examples that show how to validate the data you need and ensure that it’s safe to use throughout your program. Subsequent chapters cover SQL Injection, Cross-Site Scripting, Remote Execution, and Session Hijacking. Throughout each chapter are dozens of examples showing how to handle each situation.

Finally, part four covers user interaction with your programs. Since the majority of the web applications you will write are intended to interact with users, this is pretty important.  The authors cover ways to ensure that your users are, in fact, human users and not scripts.  Identity verification is covered with methods ranging from simple email response to SMS messages.  And once you’ve determined that your users are real, you’ll need to provide them with a list of actions they can perform.  The authors show how roles-based authorization can help with this and allow for a scaleable system that can be extended in the future.  In the next few chapters, the authors cover data loss, safely executing system commands, and handling RPC calls.  And finally, the authors explain the value of Open Source software and the advantages of peer review.

Â

Overall this is an excellent book and I highly recommend it.  While this book is geared towards PHP programming, it does cover a wide variety of topics that are not strictly PHP specific.  While I was aware of many of the topics covered within this book, I did learn a variety of new tricks for dealing with security threats.  If you’re interested in learning more about security and how to secure your programs, I definitely recommend reading this book.

phpTodo 0.7 Released!

After 7 months a new version of phpTodo has arrived. I’ve spent the last month or so working on this and polishing it up. I think I have a pretty decent release put together.

New additions include a “Next Action” field, validated RSS feeds, UI enhancements, and tons of bug fixes. You can read all about the enhancements and bug fixes in the release itself, so head over and download it!

Future plans include adding ATOM 1.0 support, sub task support (todos for your todos!), group support (so you can assign a single task to multiple people), and more. I’d like to eventually migrate into an entire management system that can be used for project management.

 

I’d love to hear any feedback regarding this project. This is my first sourceforge project and really the first open source code I’ve released into the wild. So please feel free to leave me comments!

Phew! There’s a backup!

Network management is a field I’ve been in for the past few years. In addition to making sure that packets get from point A to point B in the most efficient manner, I’ve also had to deal with network failures and disaster recovery. Essential to the disaster recovery scenario is the concept of backups. We’ve all heard of backing up the files on your computer, and backing up the servers, and storing them off-site, etc. But sometimes people overlook other backups that need to be handled. Namely, network device configuration backups. It really sucks when you realize that the smoking router in the corner has the only copy of the configuration you need to get the network back up…

 

I’ve written a bunch of code in the past to handle backing up a bunch of different types of equipment, and I’ve decided to make it open source. This new project will be hosted at SourceForge, and there is a link to the project page in the links section of this blog.

 

The initial code release will take a little bit to put together, but I’m hoping to have an alpha release within a month or so. It’s all written in object-oriented perl, a language I find fun to code in. I hope someone out there finds this useful. I know I spent quite a bit of time looking for a solution like this, and was sadly disappointed that I did not find one…

 

phpTodo 0.6 Released!

I released version 0.6 of phpTodo last night. There were some minor bugfixes, nothing major though.

 

Added to this release were a few new fields in the database to track creation date, and last modified dates. Future releases may utilize these fields more, but at the moment they’re only being used for the sort stabilizer.. The task modify code was updated to deal with these 2 new fields. Each time a record is created, the create_date field is populated. Every change updates the last_modified field.

Which brings me to another feature I added. I noticed that if you sort by priority, status, or anything other than subject, the entries shifted when you reloaded the page. My initial thought was to just add a behind the scenes secondary sort on id. I changed my mind and decided to give that choice to the user. So, on the preferences screen, the user can choose what field to sort on. This is set to task ID by default.

 

At this point the program is pretty much feature complete. I’m adding in WML support before the 1.0 release, but that’s about it. Besides bugfixes, of course.

 

After the 1.0 release, I have some bigger plans. I read a book by David Allen about todo lists and handling tasks.. Really enlightening stuff. Basically, the idea is to process everything in your inbox (anything you need to do), and determine what the next action is. At that point, you file it away based on when it needs to be done. There’s obviously a lot more to it, and if you’re interested, you can find his book on Amazon.

Based on what I learned, I’m planning on adding a number of features to phpTodo. First, I plan on adding an email module. This will allow the user to email todo items to their list. I’ll be adding some sort of authentication schema to it to ensure the item goes to the correct list. That has yet to be worked out.

I’m also looking at updating the main screen. I’ll break it up a bit to become a dashboard of sorts like the presonalized google homepage. Essentially, there will be an inbox which will consist of un-sorted todo items, an interface to quickly go through those items, the main todo list, and a calendar with appointments.

Since I’m adding a calendar, I want to also add a tickler module that can send reminders via email, sms, IM, etc. The user can choose the method(s) they want to be notified by and the system will alert them when the time comes.

I also want to add group todo lists. In essence, another user that will “share” their list with other users. This will, I believe, add more project management capabilities. Anyone can add a todo list item, and anyone else can take it and work on it. Possibly some sort of notification feature to update all users regarding those items.

 

Overall, I think this project is working out pretty well. I’ve learned a lot about php programming and I’m working on solidifying my coding style. It’s helped me a lot with the coding I’ve been doing for work. I’ve put together a complete database system, dsl tools, and the like. Good stuff that I seem to be getting some decent praise for.. :)

 

You can download the latest version of phpTodo from the phpTodo SourceForge page.

Review – Ghost in the Shell : Stand Alone Complex (PSP)

Anticipation : 7
Expectation : 6
Initial Reaction : 7
Overall : 6
Genre : First-Person Shooter

 

I’m a big fan of the first Ghost in the Shell movie. If you haven’t seen it, I highly recommend it. So, it stands to reason that I would be interested in a game related to the movie.

 

I received GS:SAC as a Christmas present and eagerly started the game. My first impression of the game was pretty positive. It starts out with some story elements and then moves on to the mission map and character selection screens. Character selection is interesting. You can choose from 4 major characters in the movie. Motoko, Batou, Togusa, or Saito. In addition, you can choose one of 4 different Tachikoma to assist you. Think of the Tachikoma as a big robotic spider with 4 legs. You can customize the characters with 3 weapons, and the Tachikoma with up to 5.

 

This is the first FPS I’ve played on the PSP and the controls are quite good once you get used to them. The analog stick is used for moving forward and back, and sliding left and right. The square and circle buttons are used to turn left and right. The right trigger button is used to fire. The D-Pad is used to change weapons, reload, and some other stuff. Overall, I think these controls work out very well despite bad reviews from other sources. I think those bad reviews come from being used to using 2 analog sticks to control the action on the PS2…

 

I’ve played through a few missions thus far and I’m still pretty impressed. The missions, so far, have been pretty short and to the point, but fun nonetheless. I wouldn’t put this in the same category as something like Half-Life or Doom 3, but still pretty fun.

phpTodo 0.5 Released!

Yesterday I released the latest version of my phpTodo project. In a nutshell, phpTodo is a todo list manager with RSS feed capabilities. I find it extremely useful, and I hope other are finding it just as useful…

 

Since this is the first entry I’ve written about phpTodo, let me give a little background information. I have things to do. Yeah, so does everyone else. And like most people, it’s hard to keep everything straight. I had my honey-do list at home, a list of stuff that I wanted to accomplish, projects for work, tasks for different projects at work I was working on, etc.. It was all a mass of confusion..

 

So, I decided to start using the todo list manager in Lotus. Well.. it works.. It’s kinda nice, but it’s slow, and a real pain to see everything in one shot. So, I started looking online for a web based one that I could use at home and work. I found a couple, but nothing that allowed me to categorize and view by category, etc. So, since I had wanted to start a sourceforge project, and I like programming, I decided to write one. And I got to thinking.. How could I ensure that I could see my todo list from anywhere? Well, a webpage is a good start, but web pages aren’t always that great on mobile phones.. I happen to have a blackberry that I carry with me, so I looked around a bit more.. I could get RSS feeds on my blackberry, so how about that?

 

And with that, phpTodo was born.. To date, I’ve had over 400 downloads of the software (that’s all vesion combined) .. The previous version, 0.4, had approximately 170 downloads. Not too bad for a piece of beta software.. :) At least, I’m happy with it.. :)

 

This latest version contains a number of bug fixes both big and small. I’ve also added some code to redirect the user properly when the login times out. Essentially, if the user is doing something and times out, it records the current information, redirects to the login page, and then redirects them back to where they left off after they login. Works pretty well… :)

 

Work continues on the project. I’m at 0.5 now and I’d like to get a 1.0 release out pretty soon. So, for the time being, I’m in a feature freeze. At this point I want to ensure that everything works correctly and iron out any bugs that may be lingering around. Once I get 1.0 out, I’ll look at adding some new features.

 

If you’re interested in checking it out, here’s a link to the sourceforge project site. I plan on setting up a formal project site for it, but I haven’t gotten around to it yet…