After my most recent brush with Murphy (in which I lost the hard disk on my T23), I decided it was time to actually do something with my new SATA hard disk controller (the Promise Sata150 TX4 I asked about) and those three 250GB disks I bought.
Linux Archive Server
I've added the controller and three disks to the Linux box that has served as my workstation from time to time. It already has two 80GB IDE disks, a DVD-ROM drive, and a CD-RW drive. So I had to visit Fry's to pick up some power cable splitters. Luckily the case has a beefy power supply.
Using a Knoppix 3.6 CD, I've booted with the 2.6.7 kernel so that I can talk to the controller--my 2.4 kernel had no driver. I used cfdisk to put a primary partition on each, set the type to FD (Linux software RAID auto-detect), and created an /etc/raidtab that looks like this:
raiddev /dev/md1 raid-level 5 nr-raid-disks 3 nr-spare-disks 0 persistent-superblock 1 chunk-size 128 device /dev/sda1 raid-disk 0 device /dev/sdb1 raid-disk 1 device /dev/sdc1 raid-disk 2
Then I ran mkraid /dev/md1 to create the array. I built a ResierFS filesystem and let the array sync while I ate dinner.
From there, I ran an rsync to copy the data on the existing disks to the new array. With that done, I will wipe the the 80GB disks and create a RAID-1 array of them. It will house the operating sytem and home directories for the system. The larger RAID array will be mounted as /raid used to archive copies of files from my other machines.
Then I'll automate the process of using rsync to keep local copies of all my remote data on the RAID array. Given the available space, I'll likely use rsync snapshots to maintain several versions of each machine that I remotely clone.
The end result is roughly 460GB of usable space for backups and archives of those backups. Two years from now, I'll probably be able to swap in new disks to get 2TB of space for the same cost. And the old disks will still be under warranty.
Powerbook Backups
Inspired by Mark's posting on Super Duper, I finally found a solution that works well for my Powerbook. I've had an external Firewire disk: a Maxtor OneTouch 300GB model that I bought with the Powerbook. The disk came with Retrospect, but I've found the software to be rather lame.
Super Duper does exactly what I need. It provides an easy way to incrementally update a full backup of my laptop--one that I can even boot from and restore with if it dies entirely. In other circumstances, I can simply mount the backup volume and grab the files I need.
It's definitely worth the $20 registration fee and highly recommended.
The Thinkpad
After IBM replaces the disk in the Thinkpad, I'll probably sell the machine. They seem to be fetching prices in the $800 neighborhood. It's still under warranty, has 768MB of RAM, and a Wifi card with a great antenna.
That means I'll probably be running SeeYou and the software required to talk to my LX5000 flight computer on my company laptop (all praise the company!). I'll probably take two approaches to backing it up, aside from whatever the standard procedure at work is.
First, I'll use a Knoppix CD to simply copy the entire hard disk over to my archive server once in a while. Since that's a pretty brute force approach, I'll probably backup the most important data on a more regular basis. To do that, I'll probably revive my batch file scripting skills just long enough to set something up. It'll copy the important stuff over to the archive server either using Samba or WebDAV through Apache and mod_dav.
Any advice on this?
Laptop Stolen!
One thing I hadn't thought of is the possibility of my laptop being stolen. It recently happened to Rasmus.
My nice new T42p was stolen by some loser at a PHP conference in Paris. It is amazingly inconvenient to lose a laptop like this. It was from inside the conference hall and there was virtually no non-geek traffic there. If a fellow geek actually stole my laptop from a PHP conference then there is something seriously wrong with the world. You can steal my car, my money, my shoes, I don't really care, but don't steal my damn laptop!
Sigh.
I can't believe that someone at a PHP conference would steal the laptop of the guy who created PHP. Something is, indeed, wrong with this world.
It's more motivation to make sure my backups happen frequently and in a reliable manner.
Rasmus: if you need any advice on a Powebook, I know you won't be shy about asking.
Posted by jzawodn at November 20, 2004 08:25 PM
Hate to nitpick, but how do you have 'nr-raid-disks 7' with three physical disks?
I might be interested in the laptop (my mom is looking for one), let me know when you're ready to part with it.
Because it was a typo. :-(
Fixed now, thanks.
I'll ping you when the laptop is forsale.
Another great backup tool for the Mac is Carbon Copy Cloner from www.bombich.com. Not only is it a great tool, but it's free.
The version of Retrospect that comes with the external drives is indeed pathetic. But the full version of Retrospect is quite good for home/small office backups.
What I'd like is a good product for Windows that can back up to a NAS drive automatically. One that can be scheduled and is smart enough to only copy changed files (diffs of files would be even better) and is able to backup open files. I want to back up directly to the drive, no intermediate server.
Retrospect needs a server and installing the server software on each client machine seems like overkill.
If you'll be using rsync and hard-links for snapshots, take a look at http://www.rsnapshot.org/. I've used it for almost a year now, and it works pretty well.
I use an automatic hd backup unit from CMS for my Mac (http://www.cmsproducts.com/detail.aspx?ID=226). It comes with an 80GB HD that attaches via firewire and BounceBack Pro on the software side. It is great because not only is it small enough to travel with, but in case of emergency I can do a full restore of my entire hard drive (handy if my laptop goes bust or god forbid stolen). It is a little pricey at $399, but worth it for how brainless it makes the whole backup / restore process.
For your Windows box, use rsync for Windows:
http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html
You need cygwin though. Or, just use the good old ntbackup.exe.
Jeremy,
Ever considered using Subversion?
I've recently moved a whole bunch of things into a Subversion repository running over WebDAV/Apache and it's pretty much changed my life.
Tristan
Part of the hassle is that I had to go through a bunch of servers and wipe out my ssh authorized_keys entries since the laptop obviously had my private ssh key on it. Also had to change my cvs password in a couple of places. Then there is the code I was working on I hadn't committed anywhere yet, a mostly done presentation I am supposed to give this week and a good bit of a chapter for the 2nd edition Programming PHP book.
I am completely unmotivated to do any of this stuff now. Instead I have been wondering around Paris and sitting in cafes reading books. Except to make this trip even more perfect I managed to break my glasses and am now virtually blind until I find a place that can fix them. Anybody know Paris well who could tell me where I can find someone to fix a stripped tiny screw in a frame?
"wandering" too, but perhaps "wondering" was an appropriate typo. By the way, french keyboards suck. The a, q, w, and m keys are in the wrong place and you have to hit shift to get numbers. No need for shift to get an underscore or brackets although these are in the wrong place as well. Great for LISP types I guess.
Maybe one of the PHP hackers really didn't like something Rasmus wanted to commit ;).
Despite it being wrong (and I would never do this), I have to say, that geek that stole Rasmus's laptop now holds a very prized commodity(Yeah, I probably spelt that wrong) - the actual laptop of the man who made PHP. Lucky guy...but still very, very bad.
On another note, I have started taking backups more seriously now, too. I was installing a new copy of Windows 2000 (please, no anti-Windows bashing...I know, I know.) cuz my old one had gotten bagged down. I was trying to put the partitions on the letters I wanted when *POOF* the partition with all the stuff I'd been working with for the last several months disappeared. All my downloads for the last months (Including releases of PHP, of course), all my music (That's not really true, I still had it on another partition too...and it's mostly legal, I'd like to add...stuff I ripped from my own CDs), and all the web development stuff I'd been doing for the last few months all disappeared, poof.
I tried putting the partition back into the MBR, but it came up blank...and I'm afraid all my recovery efforts just screwed me more...and I can't afford any good recovery software. *sigh*
I guess it wasn't too big of a loss, but then again people could say that about Rasmus's laptop being stolen, too...at least on the part of what disappeared on it, not the laptop itself. But it still sucks, doesn't it?
Wow, this was an unbelievably long post from someone you don't even know. LOL, sorry about that. (PS: This blog is addictive, you know that?)
Doug Burkhalter
I might suggest you check out 'rdiff-backup'. It does something similar to the rsync snapshot, but different, and has all sorts of nifty things you can do and (I think) takes less disk space than snapshots because the diffs are compressed. It's a nifty python thing living at http://www.nongnu.org/rdiff-backup/
Check it out.
nice topic...
however, if I have a laptop running XP home, don't have a DVD writer but do have a CD Writer, and want to backup my files, what's the best solution?
plan 9's venti/fossil archival system is worth reading about.
and yes - french keyboards do suck.
Arcterex mentioned rdiff-backup. I second that. It works great for me on about 30 servers here (Linux & Windows) at the office. Keeps diffs of your files so if you accidently delete something or mess up a file at some point you don't mirror that to your backup.
I can't believe that someone at a PHP conference would steal the laptop of the guy who created PHP. Something is, indeed, wrong with this world.
If you ask me, I think it's just Karma.
If you ask me, I think it's just Karma.
I thought Karma was part of Vim, not PHP...
/me apologizes for the bad open-source joke. However, since the joke is open-source, you're free to use it as you wish! Maybe modify it into a decent joke.
/me apologizes for the bad open-source joke joke...
Jeremy - I'd personally stick with SAMBA over WebDAV/Apache, as I'm guessing there are way more cmd-line tools available for copying files to a UNC share than to a WebDAV server. robocopy.exe is my personal favorite file copy tool, powerful set of options built-in, restartable mode available (IIRC... - at the very least it can easily copy only the different files), and if you don't already have a copy it's d/l-able from Micro$~1's site.
You might also want to take a look at my back up article in Linux Journal, it includes a fairly simple rsync backup script that keeps a weeks worth of changes available.
bcl
