More Elbow Grease

So the code from my previous journal entry parses my xml file and builds a page that contains the last ten entries. What happens when you request a specific entry? In other words, if some kind soul were to use the perma-link for this entry ( http://www.xionic.net/~jkp/index.php?weblog=20031120) on his/her site to reference this entry, what happens when they click on the link? It all happens in the index.php file. Last time I stated that if $dd was not set, then go ahead and parse the xml file and build the page. When using the permalink, $dd gets set to the permalink number field which is 20031120 for this entry. That number is in fact the date of the entry, which gets broken down like this:

$year=$dd[0].$dd[1].$dd[2].$dd[3];
$month=$dd[4].$dd[5];
if (isset($dd[6])){
$day=$dd[6].$dd[7];
}

$M = $months[$month-1];
$dir = "weblog/" . $year . "/" . $month;

if (isset($day)){
$date="$M $day, $year";
$loc = $year . "/" . $month . "/" . $day;
$permalink="http://www.xionic.net/~jkp/index.php?weblog=$year$month$day";
echo "<div id=entry class=status>";
echo "<div id=title class=status>$date</div>";
echo "<a href=$permalink>$date</a><ul>";
require($loc);
echo "</ul><br>";
if (file_exists($comments)) {
echo "<p class=status>Comments...";
require ($comments);
}
echo "<a href=\"weblog/comments.php?dn=$val\" class=status>add a comment</a></div>";
}

So my journal entries are filed in a date-tree. With the the year and month being subdirectories and a single file for each day. If you look through the code you will also note that the comments for a single day are kept in their own file for each day. This tree structure has been quite simple to maintain, and has been very stable. How well does it scale? Hmmmm… Good question. I am sure that it scales much better than several solutions that I have seen that use a flat file style database to store the information, but perhaps not quite as well as a SQL driven data store would. As long as I am not trying to do any advanced queries or searches and just display a subset of entries this system has been quite responsive and easy to maintain.

PHP + Elbow Grease…

Although I do use PHP and MySQL (as well as any other tool that fits) in many of my other projects, I have not implemented that style of solution for this website. (although the idea is quite tempting and will most likely happen in the coming months) I have tried some of the ‘canned’ blogging packages that have come out such as Movable Type, and while these packages surely fill a niche, and are very good at what they do, they do not do what this site requires. To know what this site requires, we must examine its purpose. Although ‘Intuition & Elbow Grease’ does serve as my blog and as a repository for a couple of other pieces of information, its primary reason for existence has been that of a proving ground for new code (sometimes for old code) To that end, I have tried to stay away from anyone else’s complete solution. I have surely borrowed bits and pieces along the way in order to aide in my understanding of one nuance or another, but hopefully the resulting concept is unique.

So how does this site really work? The basic building blocks are comprised of the Apache web server, PHP and of course a good smattering of Perl. No database backend has been necessary as of yet, but since I have been having quite a bit of fun designing databases and their associated web-based interfaces, one will probably arrive here soon.

To begin with, let’s look at some of the code that builds this page each time it is loaded:

if (empty($dd)) {
$data_file = "/my_home_dir/index.xml";
$list = file($data_file);

reset($list);
foreach ($list as $key => $val){
if ($key>12 && $key < 23){
$title_start=(strpos($val, "<title>")+7);
$title_end=(strpos($val, "</title>"));
$title=substr($val, $title_start, $title_end);
$link_start=(strpos($val, "<link>")+6);
$link_end=(strpos($val, "</link>"));
$permalink=substr($val, $link_start, 52);
$year=substr($permalink, -8, 4);
$month=substr($permalink, -4, 2);
$day=substr($permalink, -2);
$date=substr($permalink, -8);
$m=$month-1;

if ($title == $date){
$title="$months[$m] $day, $year";
}else{
$title="$title</a> <div id=\"date\">$months[$m] $day, $year</div>";
}

echo "<div id=entry><div id=title>";
echo "<a href=$permalink class=status>$title</a>";
echo "</div><ul>";
require ("/my_home_dir/weblog/$year/$month/$day");
echo "</ul><br>";
$comments= "weblog/" . $year . "/" . $month . "/" . $day . "-comments";
if (file_exists($comments)) {
echo "<p class=\"status\">Comments...";
require ($comments);
}
echo "<a href=\"weblog/comments.php?dn=$day&mo=$month\"";
echo "class=status>add a comment</a><br>";
echo "</div>";
}
}
}

Remember this is just part of the code, not all of it. At the beginning I check to see if the viewer is looking for a specific entry, noted as $dd. If you aren’t then I load an xml file (index.xml) to parse for the current ten entries. From the information in the xml file, I read in each of the specific entry files (and that’s what each entry is, a file in a structured hierachy to represent year, month and day) and format each entry with its permalink, date and any associated comments. When I wrote this code, php did not have any built in support for parsing xml files like it does now, so I hastily wrote the list section with all the ‘title_end’, ‘title_begin’ stuff. (This section is next on the list for a re-write) These variables help to parse the xml file into usable sections for display as this page.

There is a great deal more that goes on in displaying this page that I would like to discuss in greater detail over the next several days. Questions? Comments? Suggested improvements? Please post them below in the comments section or e-mail me directly.

Hindsight is 20/20

* As Charlie pointed out in yesterday’s comment, it has been a full year since I was ’surplussed’ by Cingular. I had spent nearly ten years there, and if it weren’t for the downsizing, I probably would have stayed there for another ten years.

Thank God for small miracles!

At first, life was quite hectic. Between wondering where my next paycheck was going to come from to worries about insurance, life was a fast-dance for a while. I struggled to work at anything I could, mostly small consulting jobs. The state of the economy was pretty much at its worst… Every company was downsizing, there were hiring freezes everywhere, and every dot-com dropout was after any job that could be had. Eventually, I decided to incorporate (see Xionic Technologies) and see where that would take me. I managed to land a very nice long-term assignment (although it was through another consulting company and not my own) and started back to the 8-5 work week in mid March. The contract was for six months, which seemed quite palatable at the time since I had been doing small one-time gigs previously. (Mostly web design, firewall implementations, basic security type stuff) I was hired as a Security/System Admin at a large company headquartered in downtown St. Louis. The people are fantastic to work with, the work I am doing is challenging and rewarding ( intrusion detection/prevention, vulnerability assessments, VPN security, etc…) and, yes the pay is notably better than before. It looks as if the contract will go through the end of the year, with a significant possibility of it turning permanent.

I still manage to take on small consulting jobs during my off hours, and continue to pursue my studies (I just finished a twelve week GIAC/SANS training class to prepare for a Security certification) I am happier with the work that I am producing than I have been in a long time. I knew I was unhappy at Cingular, but I didn’t realize just how miserable it was until I got away from there. As the memories of Cingular begin to fade, I am only now able to remember the great times that were had there.

As for the future?

The future’s so bright… I gotta wear shades!

General — entered at 22:38PM

Comments…
I am very happy that you have moved on from your last job to something better. I wish I could say the same but I can’t. Funny thing is that right after you went I was soon ‘released’ from my place of employment. Also heard over the summer about a friend in NYC who was put on probation at his high profile company. Remember how good I thought he had it before the dot-com bust. Looks like the worse may be over but still can’t say that ‘the future’s so bright…’ or as we would say out here ‘the future’s so bright I gotta wear prescription sunglasses baby!’Posted Thu Nov 13 01:21:19 CST 2003
From: P. Brahler http://paulbrahler.bravehost.com

Displaying an HTML table from MySQL data

Did you know that PHP could interface with a MySQL database? Well it can, and here’s how! The code below illustrates just how easy it is to produce a nice table with headers populated with data culled from a database. One little trick I like to use (and have used for a very long time) when displaying table data is to alternate the background color. This makes it a bit easier to read the data (at least it does for me) Notice that when the table is initialized (<table><tr bgcolor=\”#afafaf\”>), I define the background color. Later as the code iterates through the lines of data, I test to see what the current backgound color is, and then reverse it. Not rocket science, but it spruces things up just a bit. I have been using a heavily modified version of this code in several places, whether it is for displaying log files or pulling intrusion reports from a Snort database and it has worked quite well. Any comments or improvements are always welcome!

<?php
$link = mysql_connect("database_host", "user", "password");

mysql_select_db("nessus", $link);

$qry = mysql_query("SELECT * FROM table_name", $link);
echo <table><tr bgcolor=\"#afafaf\">;
if (mysql_num_rows($qry) > 0) {

for ($i = 0; $i<mysql_num_fields($qry); $i++) {

echo "<td><b>" . mysql_field_name($qry, $i) . "</b></td>";

}

}else{

echo "<td>No entries found in the database</td>";

}
echo "</tr>\n";
if (mysql_num_rows($qry) > 0) {

for ($j = 0; $j<mysql_num_rows($qry); $j++) {

if ($bgcolor == "#ffffff"){

$bgcolor="#cccccc";

}else{

$bgcolor="#ffffff";

}

echo "<tr bgcolor=\"$bgcolor\">";

for ($k = 0; $k<mysql_num_fields($qry); $k++) {

echo "<td>" . mysql_result($qry,$j, $k) . "</td>";

}

echo "</tr>\n";

}

echo "</table>";

}
?>

Comments…
I assumed that this page was driven by PHP+MySQL. Hm. In other news, I noticed that it was one year ago today that you were told you were being downsized. Any retrospective on that?Posted Tue Nov 11 18:21:41 CST 2003
From: Charles Sebold http://www.livingtorah.org/journal/

IP Conversion code

* While automating some reporting functions recently, I came across an IP address notated as a ten digit number instead of the more standard ‘dotted-quad’ notation (aaa.bbb.ccc.ddd) While technically correct, the ten digit representation wasn’t as descriptive as I would have liked it, so it needed to be converted to ‘dotted-quad’. In dotted-quad notation each octet or section represents a power of 256. The first octet represents 256 raised to the third power, the second octet is 256 raised to the second power and so on. This is important when decoding the ten-digit number. To convert the ten-digit number to dotted-quad start by dividing the ten-digit number by 256^3 (256*256*256 or 16777216) The first three digits represent the first octet. Subtract the product of 256^3 and the first octet from the original ten digit number and then divide that by 256^2. The first three digits are the second octet. Repeat the procedure for the remaining octets. Confused? Here’s some sample code:

$addr=3236053395;
$o1=substr($addr/16777216,0,3);
$addr=$addr-($o1*16777216);
$o2=substr($addr/65536,0,3);
$addr=$addr-($o2*65536);
$o3=substr($addr/256,0,3);
$o4=$addr-($o3*256);
print("$o1.".".$o2.".".$o3.".".$o4."\n");

Dang rubberneckers!

* My morning commute took an extra fifteen minutes this morning because of traffic. I figured it was most likely an accident some ways ahead on highway 44. Turns out the ‘incident’ happened on the westbound lanes (I, and everyone else was heading east). There was absolutely no reason for the eastbound lanes to slow down whatsoever. Except for the fact that everybody just had to crane their heads around (tapping their brakes in the process), for the chance to witness another human beings’ misfortune. Very sad. Especially since I couldn’t see anything by the time I got there!
General — entered at 15:51PM

Comments…
15 minutes? 40 minutes? I would like to invite all of you to try a one week daily commute between the San Gabriel Valley and Irvine Most likely this would mean taking the 5 down to OC, more importantly, taking the “Orange Crush!” down to OC Need not I mention you first need to take the 10 (and maybe 710) to get over to the 5 from SGV! When it’s busy, that can be a half hour right there! But you know, still glad I moved from St. Louis out here instead of going back home to NY. I have gotten used to the traffic out here but I was never able to get used to the NY/NJ obsessions of putting tollbooths on every few miles of highway. What did someone once say about diving in NY/NJ? “I can’t pull out of my own driveway without someone asking me for .35 cents.”Posted Mon Nov 10 21:10:58 CST 2003
From: P. Brahler http://paulbrahler.bravehost.com
Pah, 15 minutes! Once back in the olden days before the new stretch of Hwy 21 was open, I had a bad experience with rubbernecking. There was an accident on northbound (everyone is going southbound in afternoon rush hour) old Hwy 21, about half a mile away from the end of new 21. For anyone who doesn’t know, New 21 is a 4-lane highway and Old 21 is a 2-lane highway. So anyway, traffic was backed up for 2-3 miles on New 21, because of this accident on Old 21. I think it took me an extra 40 mins to get home that day. Come to find out it was a fender bender, by the time I got there there was actually nothing to see but everyone was rubbernecking at the two cars. As soon as you got past that traffic flew. I guess that’s Jefferson county for you. This is why I don’t post about traffic much in my blog. I’d be typing forever about all the annoying rush hour incidents.Posted Mon Nov 10 20:19:37 CST 2003
From: Amber http://ambysjournal.invigorated.org

A clean desktop is a happy desktop!

* After a long hiatus, I have returned, prompted by ‘Surviving Windows’ by Charles Sebold and Chouser’s ‘Tools to make Windows Livable’. Both are in a similar situation as myself. Unix admins (of one flavor or another –more likely several flavors) who find themselves confronted with the necessity of a Microsoft Windows based computer for daily business use. They have listed the tools that they each employ on their desktops in order to get their jobs done. So, like the good little web-monkey that I am, I will follow suit.

o CygWin
o Tera-Term
o Multiple browsers - Netscape, IE, Opera
o QCD Player

Not exactly a long list, but allow me to explain. Besides testing website designs in multiple browsers, using Lotus Notes for corporate messaging and playing music (QCD Player), all of my day to day work is done from within the protective familiarity of the CygWin environment. I am fortunate enough to have a dedicated SunBlade workstation at my disposal so I perform my work from there. However, I am not particularly fond of an over-crowded work environment. Having both my windows workstation (a laptop with a docking station) and a Sunblade workstation on my desk is a bit overwhelming. I much prefer the austerity of a clean desk surface. My telephone, a writing pad, my cup of water and my laptop (sans monitor- I just plug the machine into the docking station to utilize an external keyboard and mouse) are the only things on my desk. This does force me into certain workarounds in order to do my job, hence the list above.

The CygWin environment gives me local access to a multitude of unix tools- a bash interface, ssh connectivity, keychain management, vi and others. CygWin also allows me to start an X-server on my laptop, ssh into my Sunblade and execute a window manager on the Sunblade, having it display on my laptop’s X-server and giving me access to a full compliment of unix tools- perl, emacs, grep, awk, fnord, etc… So, sitting at my desk, looking at my laptop, I have full access to both the windows environment (native to the laptop with Windows2000) and to the Gnome interface from the Sunblade. Quite acceptable.

The only real exception to the list is Tera-term. It is a windows program whose sole function for me (it is capable of telnet and ssh as well) is to provide a reliable serial interface to the various pieces of hardware I may have to configure. Sure, I could use hyperterm for this, but I much prefer the logging, configurability and stability that comes from using Tera-term.