Search gets … smarter?

Wolfram Research, makers of Mathematica, a leading computational software program, have developed a new search engine, Wolfram Alpha. Wolfram Alpha has been hailed by some as a “Google Killer,” and as a possible “Propaganda Machine” by others. Although, incidentally, if you type “iraq war” into Wolfram Alpha as the propaganda article mentions, you get the following :

And that seems to be the major difference between Wolfram Alpha and a typical search engine. Wolfram Alpha is more of a calculation machine rather than a search engine. Type in something that can’t really be calculated, say you’re looking for a ferrari, and you get the following:

Wolfram Alpha just doesn’t know what to do with that. Of course, that should cut down on the porn spam quite a bit…

There are some funny bits, though. For instance, ask for a calculation such as “What is the airspeed velocity of an unladen swallow,” and you may actually get an answer:

Or, perhaps, “What is the ultimate answer to life, the universe, and everything?“

Overall, Wolfram Alpha seems to be a pretty decent source for statistical and mathematical information. For instance, type in “Google” and you get a plethora of information about Google, the company:

Choose to view the information on Google as a word, and you get this:

Though, I find it surprising that it doesn’t suggest the origin of the word itself, “googol.” However, if you search for “googol” it does have an accurate answer:

Ultimately, I don’t think it’s anything close to a “Google Killer,” but it definitely has potential, both in the academic community, and with students overall. Google won’t just roll over, though, and has announced the launch of a new Google Labs project, Google Squared. Google Squared is an attempt to organize the data on the web into a format the seems to be more usable for researchers. Time will tell, though, as Squared hasn’t been launched yet.

I encourage you to take the Wolfram Alpha engine for a spin, see what you can find. I think you’ll be pleasantly surprised at the incredible amount of useful information it has. And, assuming it survives, it will only get better as time goes on.

 

The End for Duke?

Duke Nukem Forever. 12 years in the making, so far, and no end in sight. Sort of.

Last Wednesday, 3D Realms, a division of Apogee Software, closed its doors. 3D Realms was formed back in 1994, presumably as Apogee was to be phased out. The first title they published was Terminal Velocity, a futuristic combat flight simulator developed by Terminal Reality. From there, they developed their first 3D game, Duke Nukem 3D, releasing it in 1996.

The Duke Nukem franchise began earlier, in 1991, when Apogee released the first Duke Nukem side-scroller. Duke, with his irreverent humor and fast action, became an instant hit. The 3D first-person shooter furthered this popularity with excellent level design, multiplayer action, and plenty of adult humor. Duke was such a hit, the next episode in the series, Duke Nukem Forever, was announced in 1997 and expected to ship in mid-1998. However, numerous setbacks, rewrites, changes in game engines, and other behind-the-scenes action, the game remains incomplete and unpublished today.

The unfortunate demise of 3D Realms means that Duke may never be published, though Take 2 Interactive still holds the publishing rights. There are rumors that George Broussard, designer, is looking for funding to continue working on the game. After 12 years, though, is there any hope left?

Since the announcement of the studio closing, screen shots and footage have started to leak. From what has been leaked, it looks like this could have been an incredible game. There’s some gameplay here that seems fairly new to the FPS arena. Here’s the leaked game footage:


In addition to the footage, a number of screenshots have been leaked. Voodoo Extreme has a great collection of screenshots, including plot details, and design documents for the entire game. Shacknews has a collection of screenshots from a number of the artists. I’ve included some of this art below.

3D Realms had a good run, and now it’s time to say goodbye. Perhaps Duke Nukem Forever will see the light of day, but I’m not holding my breath.

phpTodo … In Fedora!!

Apparently I’m always the last to know… But.. I found out today that phpTodo, the todo list manager I wrote (and continue to write) has been included in Fedora. In fact, it seems it’s been in there since Fedora 7. It’s not in the main distribution, nor should it be, but apparently it’s a maintained package. Thanks, Marc!

Honestly, I’m truly honored. I wrote this on a whim and it has served me well. I use it every day! And since writing it, I’ve had a handful of people make suggestions and offer patches. I think it’s been pretty successful for a small project.

So, how about an update? Well, I’ve been working on phpTodo in my spare time, which, unfortunately, has been relatively lacking as of late. I have been able to add in a number of fixes and new features, however. The biggest change in the next release will be the removal of categories in favor of having tags. In using phpTodo over the years, I’ve noticed a number of times where I’d like to be able to put an item in multiple categories, and display multiple categories at once. While this may have been possible with categories as they were implemented, I think tags works a bit better. I’ve borrowed an idea from the Serendipity blogging platform to implement tags in a user-friendly manner, so I think the implementation works pretty well. I still have some more work to tie it all together, but it is coming along.

Another change is the addition of the Prototype and Scriptaculous javascript frameworks. There are a few reasons I decided to go this route. First and foremost, it significantly reduces the amount of work necessary to perform cross-platform javascript operations. To date, I’ve used relatively simple javascript functions, mostly for front-line input validation, but with the addition of tags, I wanted to move into some more advanced techniques. I’m striving to keep it simple and not overdo it, so don’t worry.

And, of course, there are the various bug fixes that need to be added. Overall, I’m excited about the next release of phpTodo. I don’t have a timetable as of now, but I’m hopeful that my free time will increase shortly, giving me more time to work on it. If so, then I’m optimistic about a new release sometime in the next 3-4 months. We’ll see what happens.

If you’re using phpTodo, I’d like to hear from you. I’m interested in what you like and what you dislike about the program, the interface, the workflow, etc. What features would you like to see? What features would you hate to see?

Thanks!

 

Slaves to Technology?

Over the past few years I have slowly moved from carrying cash to using my debit card for purchases. It’s pretty convenient for me, and reduces, somewhat, any loss I suffer from a lost wallet or something similar. I’m sure I’m not the only one doing this. However, this means I rely on technology a bit more. And when that technology fails, life becomes difficult. This bit me again this week.

I received a new debit card a few months ago and found that after just a few months, the magnetic strip on the back of the card started to rub off. I guess they’re using something different to fabricate these newer cards as my previous card lasted several years, and was still good, when it expired and I needed a new one. So, I went about ordering a new one and life went on.

Now, a mere month or so later, the strip has yet again rubbed off. Again, I’ve ordered one and I’m expecting it any day now. In the meantime, I had to run to the market the other day. I run around, gather the stuff I need, and proceed to checkout. I normally use the self-checkout, if only to avoid the usually long lines elsewhere. I go through the ritual of scanning everything, placing them into bags, etc. When I ran my card through, it failed, pretty much as I expected. I tried running it through a few times, and even tried the “bag” trick which also failed.

So what do you do in this situation? I thought there was a pretty simple solution to this, so I asked the girl at the counter to run it through by hand. This, apparently, was a big mistake. What resulted was a 20 minute ordeal as they ran to get a manual card machine, screwed it up three times and had to keep running to get new carbon sheets. Once they finally figured out how to use the manual machine, they had to enter the data into the computer. Of course, they screwed this up innumerable times. All said and done, they were finally able to get the transaction to go through.

Seriously? Come on… I do this on the Internet all the time! Enter the card number, name, expiration, and CVV. Done! I even mentioned this and was told that it was “far more complicated than that.” …. Ummm…. ok … ?

So in the end, they have a physical copy of the card (albeit a fairly crappy one… they had to hold on to my card to read the numbers because it didn’t copy well), and they have the computer transaction receipt as well. The computer receipt has the exact same information on it that a normal transaction has… So what was the problem again?

And it’s not just this particular store, I’ve had problems elsewhere. Burger King has no alternate plan if their credit card processing fails. At most, I was offered the option of running to get cash to pay with or wait for their computer to reboot… In hindsight, I should have gone to get cash.. Apparently they’re running the slowest computers on earth.

Lowes? The girl at the counter got frantic when the card wouldn’t read. She called for help, and the help got frantic too. Luckily it scanned after the umpteenth time, otherwise I may have been witness to a nervous breakdown.

Dunkin Donuts! Well, apparently they’re fairly competent there. My card failed to scan so the girl at the counter asked for it back, typed in the numbers, and ran it through manually. Took an extra few seconds. Done.

So let this be a lesson. Technology is great when it works, but you may be in trouble when it fails… At the very least, it can be incredibly inconvenient. And to think… Only a few years ago, credit cards had to be manually handled, with the carbon paper and all. And it only took a few minutes back then… How times change…

 

Digital Armageddon

April 1, 2009. The major media outlets are all over this one. Digital Armageddon. The end of computing as we know it. Again. But is it? Should we all just “Chill Out?”

So what happens April 1, 2009? Well, Conficker activates. Well, sort of. It activates the latest revision of its auto-update algorithm, switching the number of domains it can find updates on from 250 per day to 50,000 per day. Conficker, in its current form, isn’t really malicious beyond techniques to prevent detection. In order to become malicious, it will need to download an update to the base code.

There are two methods by which Conficker will update its base code. The first method is to download the code via a connection to one of the 50,000 domains it generates. However, it does not scan all 50,000 domains at once. Instead, it creates a random list of 500 of the 50,000 generated domains and scans them for an update. If no update is found, Conficker sleeps for 24 hours and starts over by generating a new list of 50,000 domains, randomly picking 500, and contacting them for an update. The overall result of this is that it becomes nearly impossible to block all of the generated domains, increasing the likelyhood that an update will get through. On the flip side, this process appears that it would result in a very slow spread of updates. It can easily take days, weeks, or months for a single machine to finally stumble upon a live domain.

The second method is to download the code via a peer-to-peer connection between infected hosts. As I understand it, the peer-to-peer mechanism has been active since revision C of Conficker has been in the wild. This mechanism allows an update to spread from system to system in a very rapid manner. Additionally, based on how the peer-to-peer mechanism works, it appears that blocking it is difficult, at best.

So what is the risk here? Seriously, is my computer destined to become a molten heap of slag, a spam factory, or possibly a zombie soldier in a botnet attack against foreign governments? Is all hope lost? Oh my , are we all going to die!

For the love of all things digital, pull it together! It’s not as bad as it looks! First off all, if you consistently update your machines and keep your anti-virus up to date, chances of you being infected are very low. If you don’t keep up to date, then perhaps you should start. At any rate, fire up a web browser and search for a Conficker scanner. Most of the major anti-virus vendors have one. Make sure you’re familiar with the company you’re downloading the scanner from, though, a large number of scam sites have popped up since Conficker hit the mainstream media.

If you’re a network admin, you have a bigger job. First, I’d recommend any windows machines you are responsible for are patched. Yes, that includes those machines on that private network that is oh-so impossible to get to. Conficker can spread via samba shares and USB keys as well. Next, try scanning your network for infections. There are a number of Conficker scanners out there now thanks to the Honeynet Project and Dan Kaminsky. I have personally used both the proof-of-concept python scanner, as well as the latest version of nmap.

If you’re using nmap, the following command line works quite well and is incredibly fast :

nmap -sC –script=smb-check-vulns –script-args=safe=1 -p139,445 \
-d -PN -n -T4 –min-hostgroup 256 –min-parallelism 64 \
-oA conficker_scan

Finally, as a network admin, you should probably have some sort of Intrusion Detection System (IDS) in place. Snort is an open source IDS that works quite well and has a large community following. IDS signatures exist to detect all known variants of Conficker.

So calm down, take a deep breath, and don’t worry. I find it extremely unlikely that April 1 will result in anything more than a blip in network activity. Instead, concentrate on detection and patching. Conficker isn’t Skynet…. Yet.

 

Bad crawler, no cookie!

My wife is a professional SEO consultant with her own business. I work with her on occasion, helping out with the server end of things. It’s fun and challenging, and I think we work pretty well together.

So, the other day she comes to me with an odd question. Why is Google Analytics suddenly showing a high bounce rate for new keywords? Interesting problem, of course. One of the first things that popped into my mind was either a blackhat SEO or a rival of some sort. It sounds paranoid, but it does happen.

So I pulled the access logs and started pouring through them. Since the bounce rate came from a keyword search, it was easy enough to locate the offending entries. There were hundreds of log entries, all coming from the same 65.55.0.0/16 address space. A couple more seconds of digging showed that 65.55.0.0/16 was owned by Microsoft. Reverse DNS on some of the IPs revealed that these IPs were all part of the MSN web crawler. MSN apparently doesn’t provide reverse DNS for all of their IPs. No matter, there were enough to prove that this was MSN. Here’s an example from the log:

65.55.110.195 – – [24/Mar/2009:03:08:05 -0400] “GET /index.html HTTP/1.0” 200 58838 “http://search.live.com/results.aspx?q=keyword” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)”

So what in the world is going on here? Why are we getting pounded by hundreds upon hundreds of requests from the MSN crawler? And why is the MSN crawler reporting itself as Internet Explorer 6.0? The referrer URL showed the source of the request to be from a live.com search, but these being crawler addresses, I’m willing to bet this was programmed in rather than a result of an actual search. It doesn’t really matter, though, because whatever it is, it’s causing a high bounce rate and really screwing up the site statistics. The high bounce rate may be affecting the Google ranking as well.

Before we blocked these requests, though, we wanted to make sure this was unwanted behavior, so we started digging for info. One of the pages we came across described the same behavior we were seeing. As it turns out, this strange activity is intended. Live.com claims they do this to detect cloaking. Of course, it was quite easy to identify these IPs as coming from Microsoft, and determine (rather quickly) that they are sourcing from a search engine. It would be very simple to broaden any cloaking to include those IPs, making this crazy technique useless.

Microsoft claims they are continuing to tune their crawler to reduce the spam and make the keywords more relevant. The point is, though, that this seems to hurt more than it helps. As a result, many webmasters are blocking the referrer spam, at risk of having MSN blacklist the site. We have followed suit, deeming both MSN and Live.com to be irrelevant search engines.

Of course, if someone out there has a better idea of how to handle this, I’m listening…

 

MacHeist – Software for Charity

This past christmas, I was made aware of a interesting site called Macheist. They go through a lot of trouble to create an elaborate story with lots of twists, turns, and some really tough puzzles, just to give away software.. All of the software given away is developed by independent developers.

The latest heist, Macheist 3, just finished the other day. Today, the bundle was released. In short, you get a ton of software for the low price of $39. 25% of that goes to various charities. You can either choose a single charity, or split it amongst all of them. Once the total sold hits a certain amount, additional apps are released for download.

This years Macheist bundle has the following apps:

World of Goo
iSale
Picturesque
SousChef
PhoneView
LittleSnapper
Acorn
Kinemac
Wiretap Studio
BoinxTV
The Hit List
Espresso
and Big Bang Board Games (For the first 25,000 buyers)

Oh, and you get EventBox free, just for checking out the site.

All said and done, if all apps are unlocked, the bundle is worth just under $1000. And there’s some great software here, too. I highly recommend World of Goo, it’s an amazing game. In addition, there are various tools for photo editing, sound editing, cooking, and more. Definitely worth the $39 purchase.

So, if you’re interested, you can use this link. Full disclosure, that’s a referral link. Why? Because if you use it and purchase the bundle, it unlocks additional software, Pop Pop and the Koingo Utility Package. The Utility Package is a collection of 20 apps. If you’re not into the referral thing, that’s cool, just use the link up towards the top of the page, or go to macheist.com yourself. But hurry, you only have until April 7th.