Automate your 404 errors

Sep 19, 2001

It's always a good idea to review the error logfile from your webserver on occasion. It will point out many of the places on your site that need fixing. Visitors will quickly leave a site that has many broken links, so keeping all of your site working is important.

The problem is that we're all WAY too busy to remember this task on a consistent basis. In fact, it probably gets shoved so far to the back of the stove that it falls off completely. There are always other more urgent tasks that need to be taken care of.

The 404 error is the web server error code for "Page Not Found" We've all run up against this a time or two. It's a good idea to create a custom 404 page for your website. A custom 404 page will preserve the look and feel of your website and reassure the user that they haven't completely left your website. It's also a good place to put hints on where that content might have gone. If you have recently reorganized portions of your website, tell the users that in the 404 page and you'll have an opportunity to keep them on your website.

Now, if you combine these two ideas, the custom 404 page and keeping track of the error file, you can provide quick feedback to your webmaster on any broken pages on the website. Most web servers run some kind of scripting language, be it PHP, ASP, ColdFusion or even good old CGI/Perl. Write your 404 page in that scripting language and have it send email to your webmaster whenever a 404 error is thrown.

The basis of this scheme is slightly sadistic in nature. If there is an error on the website, email is generated. Every time that missing file is requested, another email is sent. The annoyance of getting email over and over again will quickly cause your webmaster to fix the site. I know it certainly causes me to wake up and fix problems quickly. The problems are fixed much sooner than if I waited until the next review of the error logfile.

Most modern email programs have filters that allow you to automatically route incoming email into particular folders based on the subject or sender. Don't allow these emails to be shuffled off into a folder as that defeats the entire purpose of annoying the webmaster until the pages have been fixed.

If you really want to get gung-ho about it, you can create custom error pages for all of the server error codes. Error 500 is another one that is probably useful to trap like this. Still 404 is the most common error on a website so fixing that first will be the biggest bang for your buck.

One word of caution. This trick will also generate email to the webmaster on any improperly generated request. I recently had someone testing a web crawler on my site. It was broken and mangled all of the URLs on my site generating around 200 emails in the process. Since it was the crawler causing the problem and not my site, there wasn't anything I could do to stem the flow of email.

      Rene

If you have any comments or questions, please feel free to drop Rene an email at rene@astutecomputing.com. If you would like assistance in setting up this kind of feature on your website, please contact us for a quote.