A few days ago, I decided to finally do something about my Apache logging mess. The "mess" is that I host about 15 virtual domains on a couple of colocated Linux servers. Most of the domains either belong to me or my friends. A few are business related. For the longest time, I've had apache configured to log in the typical combined log format with one log file per domain. I haven't rotated (or cleaned up) the logs for a long, long time.
Enter mod_log_mysql, an apache module that allows you to log directly to a MySQL server, optionally logging to disk as well. The module is simple and straightforward to setup. Most importantly, it Just Works. There's even a cool MySQLMassVirtualHosting option so that it'll log each domain to a separate table and even auto-create new ones as you add domains to your apache configuration. Very cool.
I did manage to find a bug along the way. The module didn't properly quote table names. So if you have a table named something like advanced-mysql_com (all dots become underscores), MySQL will barf on the INSERT and CREATE TABLE statements. But since I had the source code, it was quite easy to fix. I'll be sending a patch to the maintainer soon.
Why this is so cool.
Once everything was up and running smoothly, I had a number of MySQL tables collecting data about traffic to my various web sites. "So what?", right? The cool thing is that by adding a couple of indexes to make queries fast (there are no indexes by default), I can whip up a PHP-based application that presents interesting stats to me in real-time. The PHP app isn't done yet, but it's quite functional for providing a high-level picture of what's going on. I can see which of my blog entries are the most popular, who is visiting them, and where they came from (referer tracking).
More to do.
I'm not going to make the URL public yet, because the app needs some work and a good security audit. But now that the data is in a more accessible format, I can do tons and tons of stuff with it. I plan to have a module on my blog index page that lists the most popular entries in almost-real-time. I could have it updated every minute or two without putting much of a drain on the system at all.
The other thing I need to get around to is importing a few years worth of old log data from the existing access_log files. That's just going to take a bit of time to write a Perl script to do the job. Once that's done, I'll be able to answer a lot of interesting questions about my web sites.
The web is a funny place. Somehow I ended up on a site that was selling Buffy the Vampire Slayer DVDs. It had a link to information about DVD region encoding (they sell both region 1 and region 4 discs). That linked over to a site with a nice list of DVD player hacks.
I took the day of work to do random stuff around my apartment. Part of the fun involved two trips to Fry's for parts I needed. On the first trip I wanted to get some more rounded IDE cables. While there, I thought "what they heck..." and picked up an AirPort card for the TiBook.
After hitting the Apple web site to figure out how to make it go, I realized that (a) the card is internal and won't use the only PCMCIA slot ( yeay!), and (b) I needed a special screwdriver to open the TiBook. I didn't have said screwdriver, so I had to make a second trip to get the screwdriver. While there, I also picked up the notebook backpack that I was tempted by on my first visit.
Anyway, once I got it all setup and installed the AirPort update software, it was like magic. It found my LinkSys WAP11 gateway without any trouble and I was completely wireless. Of course, the next order of business was finding a good client-side blogging tools that runs on OS X. After a bit of looking around, I found BlogApp. I used it to post this story. It's a log like w.bloggar on Windows except that it has spell checking built-in. Excellent.
BlogApp is shareware ($6) and if it works out well, I'll click the PayPal button and make it official.
Ever wished your Web browser understood little scribble marks like your favorite handheld OS does? Then MozGest is for you.
No doubt, it's big news that Microsoft is going to be at LinuxWorld this year. As reported by The Register:
Microsoft is to exhibit at LinuxWorld Expo this August, and it appears that the company wants to be nice. Yesterday, Linux Today spotted the Beast's presence on the Expo exhibitor list, and after publicising this was contacted by an apparently kinder, gentler Microsoft.
I really hope that the Open Source "movement" can get over itself and actually talk to Microsoft rather than simply stopping by their booth to poke fun at Windows and make joke about how scared billg must be of the almighty Linux.
Look at it this way, with Microsoft at the show, LinuxWorld is certain to attract a larger than normal crowd.
Microsoft is not the enemy
As Adam Goodman has so often pointed out to me and others, Microsoft is a fact of life. They're in the business of building software and selling, just like a lot of other big companies (Oracle, SAP, etc). They're affected by the changing landscape as much as every other company. And you know what, they're a very, very smart company. But they're also very concerned about Open Source.
Why are they coming?
Why not? They're trying to figure out how to deal with Open Source. As a smart company, they're attacking the problem for multiple angles--often at the same time. They've run advertisements in Linux Magazine, attacked the GPL, appeared at the Open Source Convention, and so on. Little that they've tried so far has worked. But they're not quitting. They rarely do. They're used to wearing down their opponents until they give up or go out of business.
The Open Source movement, as we all know, is quite different. Microsoft has never really faced a challenge like this before. There is no company to buy or put out of business. They're battling a large volunteer army.
The point is that Microsoft probably still doesn't quite "get" Open Source, so they're doing everything they can to learn about it. Their shareholders would expect no less. That's why they're coming to LinuxWorld.
What should we do?
Nothing. Treat them as you would any other company seeking to learn more about Open Source. If you really believe that Microsoft is capable of destroying Open Source, think again. Despite the jokes of world domination, Open Source has never been about winning anything. It's about building great software and a great community. I don't think that attacking Microsoft helps do either of those.
Microsoft is coming to LinuxWorld. So what? No, it's more a matter of "it's about time!" They've supposedly been a leader in software innovation all these years, but they've been notably absent from one of the more important conferences year after year. It's good to see them getting involved.
Memelogging is:
the process of inventing and promoting a meme while simultaneously identifying yourself as the creator of that meme. It's an ongoing media event, and it's yours.
I saw a reference in The FuzzyBlog! to a piece of software call Drupal that seemed worth checking out.
Drupal is:
Drupal is the English pronunciation for the Dutch word 'druppel' which stands for 'drop'. Drupal is a content management/discussion engine suitable to setup or build a content driven or community driven website. We aim towards easy installation, excessive configuration and fine-grained maintenance capabilities. Due to its modular design Drupal is flexible and easy to adapt or extend. Drupal is written using PHP. The source code is available under terms of the GNU General Public License (GPL) .
Now I know this is difficult to quantify, but Drupal just feels like a much more professional project than PostNuke, PHPNuke, or similar projects. It reminds me a bit of the feeling I got when I looked at Scoop a while back.
Here are the cool things that caught my eye about Drupal:
Anyway, give it a look if you're in the market for a good CMS/discussion system. I am.
Well, sort of. In this blog entry, he explains that he switched from Radio UserLand to MovableType and AmphetaDesk for a variety of reasons. Amusingly, I made the very same switch a couple weeks ago for almost the exact same reasons. Spooky. We both ran Radio under VMWare. The only difference is that I'm running AmpetaDesk on my new XP-based laptop. But I could run it on my OS X TiBook or my older Linux-running Thinkpad. That's what's cool about AmphetaDesk (and that the author recently mailed me about some neat ideas).
My plan is to get AmphetaDesk working on one of my co-located servers this weekend. That'll be just another piece of my making the desktop irrelevant plan that I've yet to articulate in public. Pateince...
As reported in this BBC article:
So, with about 300 billion stars in our galaxy, there could be about 30 billion planetary systems in the Milky Way alone; and a great many of these systems are very likely to include Earth-like worlds, say researchers.
Thanks to Ye Olde Phart for the link.
While catching up on blogs, I found a an entry over on Windley's weblog called REST and Hyperlinks. I wasn't quite sure what to make of the title. I hadn't seen the REST before. Being rather well-versed in Web stuff, that surprised me a bit. It shouldn't have (long story). But it made reference to an article that Jon Udell had written for Infoworld (his new employer) a few months back. The article is called Hyperlinks matter.
Jon's article hit a nerve with me--one that I didn't know was there until I read the article. I've always wondered about the various protocols for so-called "Web Services", namely SOAP and XML-RPC. What bothers me is that they're called Web Services in the first place. It feels like there's no Web in them at all. The Web has never been about APIs. It has always been about information (typically documents) and URLs (the addresses of those documents). Over time applications (and URL-based APIs) naturally appeared as folks figured out how to marry databases with Web servers, write CGI scripts in Perl, and then move to things like PHP and Java. But SOAP and XML-RPC have always seemed like they're just trying to piggyback on the fact that port 80 is open on virtually every corporate firewalls. They just feel like a solution to a problem that we really don't quite have yet.
Anyway, after reading Jon's article and mentally nodding my head in agreement, I headed over to visit the other two items that Windley's weblog referenced. In Second Generation Web Services (and part two, REST and the Real World), Paul Prescod presents a compelling argument against the new Web Services standards (UDDI, SOAP, etc). His claim is that the existing HTTP infrastructure and some simple XML would go a long way toward building Web Services with tools and technology that have been around and proven for years already.
I'm inclined to agree with Paul. Having been part of the Internet in one way or another for the last ten years or so, I find that his view of what could be just feels like the right solution--for now.
Update: After posting this, the Google API plugin for MT added an interesting link to a blog I already read. Over at diveintomark, there's a good set of links and some commentary about the REST vs. SOAP debate.
Update #2: Check out the RESTwiki. Lots of good information over there.
So it seems that LinuxDailynews.net is on-line now. But I can't find the RSS feed to snag the headlines. Anyone know the url?
Update: the URL is http://linuxdailynews.net/backend.php, but there's a bug. It's not producing valid XML currnetly. (What does that say about PostNuke?)
Adam Curry's weblog noted this amusing little flash animation. What a way to start off the day. :-)
Rob Flickenger, the author of Building Wireless Community Networks, has an article on the PC Magazine web site called Broadband Block Party. Reading about the spread of wireless networks has made me realize two things.
First, I really wish I was back in college again (and had money this time). This stuff would be a blast to play with. Sure beats making ethernet cables by hand and the sort of stuff I had to deal with.
Secondly, I'm really starting to see this as the next frontier for geeks and non-geeks who can't get easy access. A few years ago, everyone was getting a computer and trying to figure out how to make a modem work so they could dial-up to this new Internet, few really knowing what it was. Now folks are able to build their own internet-like infrastructure for remarkably little cash.
Unlike a lot of computer-related activities, wireless technology actually encourages you to get out and meet your neighbors. And, of course, it's a lot of fun. Just when the Internet is starting to feel "taken over" by the corporate interests and big media giants, 802.11b provides a glimpse into the future.
I just hope it doesn't take very long for that future to get here. I'd love to be able to boot up my notebook in any moderately sized airport and find that I've got a signal. I'm a little puzzled as to why you can't already do that. It sure beats the little laptop "workstations" I've seen in some airports.
I found this article over at Newsforge to be a refreshing read. It's the story of how Snort, the Open Source Network Intrustion Detection System (IDS) came to be and the company that grew out of it. Maybe this is a sign that you still can make money in Open Source in the post-bubble era.
Yup, according to this story on Yahoo News, Apple's new Xserve (or iRack, as it ought to be called) ran right past a Dell server in a recent benchmark.
At the big Xserve announcement, Steve Jobs made it quite clear that they were going head to head with Dell, IBM, and others. This just serves to strengthen Apple's position.
The Register is running a story titled MS security patch EULA gives Billg admin privileges on your box. Yup. Microsoft isn't waiting around for Palladium or specialized hardware. They're gonna force the new restrictions on Windows users as they download critical security updates.
Microsoft has just assumed the right to attack your computer and surreptitiously install code of its choosing. You will not be warned; you will not be offered an opportunity examine the download or refuse it. MS will simply connect remotely and install what it will, or install it secretly when you contact them.
There was a link over on slashdot to a review of the NexLand ISB Pro800 Router. I had never heard of NexLand before, but the device is quite compelling. It is able to do load-balancing for dual broadband connections. If you're like me (unlikely) and have both DSL and a cable modem, this is the way to go! No more hassling with switching the default route on your machines when one provider's service dies.
At least that's what his latest pulpit says. He foresaw the recent accounting scandals as well as Microsoft's Palladium announcement. Heh.
After having installed MT 2.21, I managed to get the google integration going. Now you'll see a box on the right side of the main index page which lists matches that google has found based on the titles of my most recent blog entries. In case you're not a rocket surgeon, it's called "related links via google."
Also, I've manged to write a very simple Perl script called opml2html.pl that converts the myChannels.opml file created by AmphetaDesk (the cool RSS aggregation software I use) into an include file. That include file is sucked it by PHP and appears in the "feeds/blogs I read" box on the right side of the main index page, just below the aforementioned google box. So now you can know what I read and easily get the feed URL by clicking on the ever-popular orange XML button.
I'll probably add a "book I'm reading" and "favorite new CD" module at some point too. I just need time to do it. Remember, I do have a book to write.