Archive for August, 2004

SiteCheck

For probably a year now I’ve been looking at various tools to be able to verify that web services are up and working properly. Most of these tools offer the ability to “ping” a TCP port, basically to see if it is still up. The nicer ones will also let you perform a “content check” to see if a certain string of text appears on the website you are checking.. Some let you check lots of services (HTTP, SMTP, POP3, etc.) I won’t go into the reasons why, but I’ve finally decided that I absolutely need to be running one of these applications, and I need it right now…

Of course, there is a reason why I don’t have one yet… Cost. People who have written these tools are extremely proud of them.

So, I’ve decided to write my own SiteCheck tool in PHP. Here are the requirements:

1. Must work well on a PC that has a somewhat unreliable internet connection. (Check to make sure internet connection is up)
2. Must be able to check a number of sites.
3. Must be able to send an email whenever a problem is found. (One email should contain all sites that failed that attempt, not 4 emails if 4 different sites failed.)

*tick tock tick tock*
Ok, about an hour of coding and testing later, and I have my first version ready…

I was planning to post at least a portion of it here, but until I get my CMS up and running, or come up with some other “code viewer”, I guess I’ll hold off.

I used two arrays, $inet and $siteToCheck to hold data. $inet is strictly for “stable” sites that you expect to always be up and working… Basically, if you can’t get to any of these sites, your internet connection is not working right. The $siteToCheck array is for the sites you are interested in checking out. The index of these arrays is the URL of the site to check (such as http://www.yahoo.com) and the value of each element is a piece of text that is on the page, such as “Yahoo!”.

A quick function called readWebSite pulls the data in from the remote website and verifies that the “content” is there. If so, you get a true response from this function, otherwise false.

In the main loop of the program, a foreach runs against each element of the $inet array. The first “good” reply tells us that the internet is up, so we set a flag variable accordingly and “break;” out of the foreach loop. No need to check your entire list of “stable” internet sites.

If the Internet is up, we run a foreach against each site in the $siteToCheck array. Any problems are recorded in the $problems variable.

If there were any problems, an email is sent to me to let me know the extent of the issues.

Then, we simply sleep for 15 minutes and repeat…

It would not be too difficult to write a version of this to perform most of your standard communications, as PHP has other protocols that are easily checked, such as SMTP and POP3.

August 25, 2004 at 4:01 pm Leave a comment

An Important Programming Truth, re-discovered

This is mostly a tip, but it does deal directly with my FAQ project… Tonight, I re-discovered an important programming truth…

All programmers who have looked at code they wrote 6 months or more earlier know that documentation is an important thing.. I mean, I have a hard time remembering having WRITTEN code for some projects, much less remembering the specifics of how said code actually works… All serious programmers should know that good documentation is absolutely essential for all but the simplest of code.

But, the programming documentation that is perhaps the most important of all is documentation for RECURSIVE FUNCTIONS… Recursion is so powerful if used properly… In a scant few lines of code you can acomplish a lot of complex work. But recursion code is also very difficult to scan and determine what is going on. (At least, with any code that actually performs a real-world job..) I re-discovered this important programming truth tonight as I worked out a bug in my recursion code…

My recursive function consists of a grand total of 6 lines of code. After having difficulty reading this code (which I wrote only about a week ago!) I now have written seven full lines of text (consisting of 102 words total) to explain how this code works.

Of course, now that I’ve written this documentation, we all know that this will be the most bug-free portion of code in the entire project, and I won’t need to go back and look at it for years, if ever… 🙂

August 22, 2004 at 4:00 pm Leave a comment

The UI Blues

Working on a User Interface is enough to give anyone the blues… Designing the screen layout isn’t easy, and once you do know what you want it to look like, you have to figure out all the fancy SQL statements needed to be able to display the data.

Once you have the SQL statements figured out, you really run into a quandry, unless you have an extremely simple application. To produce a single page of HTML, you may end up with multiple queries per category. So, do you run the risk of performing 30 SQL queries per page-load? There has got to be a better way!

In my case, the better way has involved loading all category data into memory, then iterating through that data to determine sub-category relationships, etc. as needed, instead of performing a SQL query per category to figure that out. This “caching” of data isn’t that difficult, but does move you away from a “pure SQL” mode of thinking. I still have to perform one query per visible category to get a count of all questions in that category and the below sub-categories, but that was expected. Still, this should reduce the number of SQL queries per page-load to about 50% of what it would be otherwise.

Now, you might think that cutting out a query here and there isn’t that big of a deal, but it really is… Anyone who has every been Slashdotted will agree with me on this one. If you suddenly have 10,000 people all trying to hit your page within seconds of each other, your server will be much happier with more efficient code, since it will have to perform the code for each and every request.

August 14, 2004 at 3:58 pm Leave a comment

Image Optimization

While reviewing a site of mine, I found that I was guilty of something that I only thought everyone else was guilty of…

Non-optimized images.

There. Now you know.

Image optimization is something that I have suggested to others many times as a way to speed up their website, while conserving bandwidth at the same time. Why not kill two birds with one stone?

Well, while I was doing a bit of fiddling in my favorite image editor tonight, I discovered that I had an image on one of my sites that was not optimal in terms of size. It was a header for a page… Prior to my CSS layout, it was broken into three separate images, each about 9-10K… With CSS, I found that it would be best to have it as a single image, but it was around 40K. (I did it in a bit of hurry, though 40K seems huge in retrospect!) I messed around with various settings, looking at the final filesize and the quality of the image and finally settled on a variation that was about 20K in size. This reduction in size is enough to make a significant improvement in download speed while the image quality was only slightly lower.

The thought then crossed my mind that I should probably check some other image files, as I might be able to shave off a little more. Well, I didn’t keep an exact count, but I’m guessing that I shaved another 15-25K off of the front page of the site alone, as I was able to cut off a few hundred bytes on up to a few thousand bytes on some images.

Here are a few image optimization tips:

1. Try different formats! You may have everything on your site in GIF today, but that might not be best. In my testing, some images looked best and were most efficient in GIF format, but PNG may have an advantage with some image files. JPEG was a big favorite of mine as well. For certain types of images, JPEG is superior to GIF and PNG.

2. For GIF and PNG formats, don’t forget to lower the color depth. I found that in some cases I was able to lower the color depth down to as low as 8 colors. Simply dropping from 256 to 128 colors in many cases resulted in an almost unnoticable reduction in quality, while dropping a significant percentage of the file’s size. Depending on the imdividual image, you might be able to drop down to 64, 32, or even fewer colors.

3. For JPEG formated images, use the quality scale! Again, depending on the image you can get away with a significantly lower quality setting without significantly affecting the actual image quality. This can result in BIG size savings.

4. While some people who offer ready-to-download icons (for their products, etc) are putting out highly optimized images for others to use, not all ready-to-download images are optimized. The result? Image files that are 2-3 times larger than they need to be, if not more. So, before you use that new snazzy version of Tux you found on the web, load it into your favorite optimizer to see if it’s ready for prime time.

August 12, 2004 at 3:53 pm Leave a comment

Back to the FAQ

I’ve taken a bit of time off of programming to learn CSS. The site that I’m writing the FAQ manager for wasn’t written with CSS in mind, so I gave it an overhaul and learned a lot at the same time.

I completed the basic template with CSS (using only one table), so I’m now ready to continue working on the FAQ Manager. As a side note, CSS is really cool, but learning it can be VERY frustrating!

Enough work has been done on the back-end of my FAQ application for now… Basic functionality is complete, in that you can Add, Edit, Remove, and Sort Categories and Add, Edit, and Remove Questions. A Settings screen is also in place, which allows updating of the header and footer code (yes, I’m allowing the use of PHP in the header/footer), update the Page Title, etc. More work needs to be done on this screen, but exactly what that work is hasn’t been set in stone yet…

Which brings me to the lesson of the day…

The direction of any project can change mid-stream. Requirements documentation you wrote up will almost always go through some changes here and there. In my case, I want to modify the User Interface design of this application, but exactly what new variables (for the Settings screen) will be required isn’t known yet…

Sometimes, to firm up your requirements, you need to do a bit of coding on other parts of your application. In this case, I think that if I get started on the user interface it will tell me exactly what additional settings should be on the Settings screen.

So, it’s off to work on the User Interface… Wish me luck!

August 11, 2004 at 3:55 pm Leave a comment

Categories revisited

I recently have revisited my category code to enhance it, as I found a few things lacking…

Since I allow multi-layer categories, it was difficult to visualize your sub-categories as there was very little to display the categories above the current one in the admin panel, or when selecting a category for a question… It was simply a list of categories, such as “Email, Webmail, Mail Server Settings”, etc.. Nevermind that “Email” was a main category, with “WebMail” and “Mail Server Settings” below it.

To fix this, I wrote a routine called createCategoryString, which would recursively build a string of Categories… So, in the above example, “WebMail” would be listed as “Email > WebMail”, etc… It worked great, but this caused problems of its own…

I now understand why most web based applications don’t offer multi-layered categories… The only way to display things properly is via Recursion… For those of you who don’t know the joys of Recursion, here’s a definition from a programming site I found:

Recursion: When a function called itself, either directly or indirectly. If this isn’t clear, refer to the entry for “recursion”.

What good is that, you might ask.. Well, you can write a good recursive function in very few lines and it will do much more than the number of lines might suggest. Here’s a very simple example:

function recursionTest ($var)
{
if ($var > 0)
{
return $var . “\n” . recursionTest($var-1);
}
else
{
return $var;
}
}
echo recursionTest(100);

The above example returns a webpage containing a countdown from 100, with one number per line… Yea, not terribly useful and this particular example can be more easily done with a loop, but hey – It’s just an example…

Recursion really excels when there’s math involved, or when you have to search through data, find one result, then search through it again, etc. until you don’t turn up any new results…

The issue that I ran into with recursion is that it is possible to change the way categories are linked so that a loop is created… Very simply, using the same examples as above, if “WebMail” is a sub-category of “Email”, then you allow the admin to make “Email” a sub-category of “WebMail”… Well, next time the recursive function runs, you have an infinite loop, or in PHP’s case, a loop that runs the full length of the “max_execution_time”.

How did I solve this issue? You might have guessed it.. More recursion, of course!

I wrote a routine which will determine all sub-categories of the current category.. (And all sub-categories of those sub-categories, etc.)… This is passed to the routine which generates the drop-down list of Categories for selecting a “Parent Category”… This list is treated as an exclusion list, so the drop-down list of categories that the customer can select from will automatically have “dangerous” options removed…

You know, the deeper that I get into this, the more I think that I should probably think about writing this into a class…

August 5, 2004 at 3:47 pm Leave a comment

FAQ is coming along nicely

The FAQ project is progressing well. As of tonight, the following admin functions are operational.

Category Functions: Add, Edit, Delete, and Sorting.
Categories include ‘Parent Category’ for creating a ‘tree’ structure of categories, the ability to Hide a category from display, and a selectable Image for each category.

Question Functions: Add, Edit, Delete
Questions may be assigned to multiple categories, may be individually Hidden, and marked as ‘Pending’.
Pending Questions would be hidden from view by customers.. Basically, It is to allow for customers to ask questions. In all ways, this should be viewed as a ‘Hidden’ flag, except to note that a customer added the question. A ‘Pending Questions’ view is complete also, so you can see exactly which questions need to be answered. I’m considering a mail function to notify an email address when a Pending question has been added.

Next, I suppose, is the ‘Settings’ screen, to allow general things to be configured, though perhaps I should work on the end-user view a little, so I can see for sure what things need to be in the ‘Settings’ area.

August 2, 2004 at 3:42 pm Leave a comment

SPLIB: strip_quotes.php

Fairly early on in The PHP Anthology by Harry Fuecks, he suggests that Magic Quotes is a problem that all PHP developers has to deal with. Essentially, you must code with the idea that it is enabled, or that it is disabled… If you code as if it is enabled, you’ll run into issues when your app is running on a server that has it disabled. On the other hand, writing your application as if it is disabled will be an issue if it is enabled… Quite the quandry..

Harry’s answer? He includes a piece of code in SPLIB designed to detect if Magic Quotes is enabled, and if so, neutralize it’s effect, so it is as if it is disabled… This way, all your code can be written with the idea that Magic Quotes is off…

Good answer, I say, but I’ve ran into an issue… My test server has Magic Quotes enabled, so strip_quotes.php is active… When I include this piece of code in a script which receives an array of data from the remote web browser, the data gets basically lost… The variable is there, but it is a string variable, containing the word “Array”… I believe the specific failure is with the array_map function, which is what does the brunt of the work in strip_quotes.php.

Update:
I solved this problem with the array_map_deep function, which I found in the user notes section on php.net. Basically, it does the same thing as array_map, but it does so in a recursive fashion to any arrays it hits inside the passed array… I also posted my problem, and later my solution on the SitePoint forums. With any luck, this will be a standard part of SPLIB in the future.

August 1, 2004 at 3:41 pm Leave a comment


Calendar

August 2004
S M T W T F S
1234567
891011121314
15161718192021
22232425262728
293031  

Posts by Month

Posts by Category