OnStream DI-30 Red Hat Backup mini-HOWTO |
About Karen JP - PGP Keys - Vossen's Law - Firefox - MythTV Photos - Deck - SCUBA - Dolphins Security - Firewall Rules - Flypaper - GNATBox - Home - Home Net Security - Principles - Snort - Snort Books - Sec Tools - GenPass - Honeypot Stats - Firewall Stats Source - Perl Networking - Time - NAT - IP Calcs Linux - apt - Edutainment - SME Server - Backup (DI-30) Windows - Win Tools - Voodoo - Win. Shell Scripting - POSIX Redirection - Winlogcheck What's New Email me Email Form |
by JP Vossen, CISSP; JPATjpsdomainDOTorg, http://www.jpsdomain.org/ $Revision: 1.5 $, $Date: 2007-11-28 02:26:46 -0500 (Wed, 28 Nov 2007) $ UTC IntroductionSee Update (2003-11-29)--OnStream Bankrupt again for important bankruptcy information about OnStream. This document describes how to use an OnStream DI30 tape drive with Red Hat Linux and several free backup utilities. It is intended for a anyone planning to use an OnStream DI-30 tape drive, or anyone trying to backup Linux, especially Red Hat. Most especially, it's intended for anyone trying to do both! It assumes some basic hardware and Linux/UNIX knowledge but little to no knowledge of tape drives and tape backup software on Linux. It provides the tools (all of which are free) and information need to implement a relatively simple rotating weekly backup scheme that is suitable for home or small business use. The most recent version of this document and the following scripts may be found at: http://www.jpsdomain.org/linux/OnStream_DI-30-RedHat_Backup_mini-HOWTO.html If you have not already purchased an OnStream drive to use with Linux,
read Part 1 to make sure this is an acceptable solution for you. You
might also want to skim the rest to make sure you are comfortable with
everything. Then check the OnStream site at I wrote this because I could not find anything already out there that answered my need. Since I had to do the research anyway, why not document it properly? I am going to be pretty specific with this mini-HOWTO, because I do not have a lot of resources (time or equipment) to spend on this. If you have different experiences, or can add information, please let me know. Contact information is included with the history section at the end. This mini-HOWTO covers both of the situations where you must reboot Linux. That is, you should only ever have to reboot Linux when adding or replacing non-hot-swappable hardware, and when you need to switch to a different kernel version. Other than that, you should never need to reboot! This documents IDE devices only. It does not cover any OnStream SCSI devices. It may still be helpful--Your Mileage May Vary. Also, I am not affiliated in any way with Red Hat, OnStream or anyone else mentioned herein. Update (2003-11-29)--OnStream Bankrupt againIt seems that all the "official" OnStream sites listed in this document are off the air! That is a Bad Thing. It you know why and can point me to new sites, please let me know. It looks like they have gone bankrupt. Again... You can find software, firmware, drivers, manuals and support at Hastec who seems to be a reseller of some kind. There are also 3 (as of 2003-11-30) files at http://www.driverguide.com/. This site requires a free membership just to search, which is highly annoying. Update (2001-11-24)I have switched to the OSST drivers, as they work much better for me. I've also updated this document to include OSST information. Update (2001-10-29)I just got a message from Jack, an OnStream Software Development Manager in the Netherlands, with some excellent up-to-date Linux information. Here are the high points. Note that this site used to be something like http://linux1.onstream.nl/.
DisclaimerCopyright © 2001-2003, JP Vossen. All rights reserved. This howto and the associated documentation and scripts are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. In no event shall the author be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or any other pecuniary loss) arising out of the use of or inability to use this documentation or scripts. If you have questions or comments, please contact me at JPATjpsdomainDOTorg. Table of ContentsIntroductionDisclaimer Part 1: The OnStream DI-30 Installing the Drive & Configuring the SystemPart 2: Your Backup Strategy Recovery StrategyPart 3: Putting It All Together KickStartAppendixes. Some Notes About Common Backup Programs Part 1: The OnStream DI-30
For our purposes, the interesting specs are:
Installing the Drive & Configuring the SystemI had a lot of trouble accomplishing this, which is largely this reason I wrote this document--I couldn't find anything to help me. Two issues particularly stick out in my mind. 1) I was not very comfortable re-compiling or installing kernels. 2) I had no idea how it should work, and what it would look like then it was, in fact, working! What I've learned is that the Linux kernel is pretty darn resilient--it's hard to screw it up (but I managed!). When you do screw it up, it's very good about telling you, "No, dumb-ass, you have to turn on packet filtering to allow DNS to run" or whatever. I've also learned how to make the drive work, and what it should look like when it does. I hope you find that the information below answers your questions! After finally getting everything to work, and writing the backup scripts included below, I am very happy with this solution. It is very inexpensive by any metric you want to use; cost of data loss, cost of alternative high-capacity solutions, cost in media and tape-swapping time for other low-capacity solutions, etc. It works quite well for me, but of course your mileage may vary. Install the Tape DriveSee also: Hardware installation manuals at Hastec. First, install the hardware as covered in the documentation that came with your drive. In my case, I had to move the master/slave jumper to master, but that was the only change I made. Otherwise, I took it out of the box and plugged it in. Note that the red stripe on the IDE cable (for pin 1) goes AWAY from the power connector in the drive, which is the opposite of hard drives and CD-ROMs! Per OnStream Tech Support, do not make your drive a slave to an IDE hard drive. Either make it a master on the second IDE interface, or make it a slave to IDE CD-ROM. Do not make an IDE hard drive a slave to an OnStream tape either. First, figure out with driver (next three sections) you are going to use. Read the sections and get a feel for everything. Then, follow the instructions for the section. Since you have to reboot anyway, don't bother installing the drive until after setting up the driver. Using OSST (Recommended)See also: http://www.torque.net/scsi/SCSI-2.4-HOWTO.html#stosst and http://www.enesbe.com.au/cgi-bin/wiki.pl?EnesbeInfoPages/TapeDrive. I originally tested this under Red Hat Linux 7.1, then upgraded to 7.2, 7.3 and 8.0. Red Hat 8.0 Just Works with OSST drivers for a DI-30!!! On a clean install I did not need to add an append to the boot loader, or create the device files. I did make sure the modprobe lines where in /etc/rc.local though. Presumably the same is true for Red Hat 9 and Fedora, but I have not tested that. So you can skip below and install the drive. The good news is that the modules you need are already built in to Red Hat 7.1 and more recent distributions, so this is pretty easy, and it seems to work a lot better for me than the IDE interface. First, you need to edit your /etc/lilo.conf file. You need to know which IDE interface your OnStream is connected to. If you don't know, cat /proc/ide/hdx/model where x is a, b, c, or d. Mine is hdc. You should see something like "OnStream DI-30" when you get the right one. Edit /etc/lilo.conf and add 'append="hdx=scsi"' where hdx is the correct IDE interface. This allows the OSST driver to grab that IDE drive and emulate OnStream SCSI on it (more or less). For example, my lilo.conf looks like this (remember, my DI-30 is on hdc): boot=/dev/hda map=/boot/map ... default=linux append="hdc=scsi" image=/boot/vmlinuz-2.4.9-12smp ... If you are using grub, edit /boot/grub/grub.conf like this instead (note, for Red Hat 8.0 I did NOT have to do this part!): ... kernel/vmlinuz-2.4.18-17.8.0 ro root=LABEL=/ hdc=scsi ... Now you need to load the correct modules. You should not need "IDE-Tape" when using OSST. You will need "ide-scsi" and "osst." Since you need to reboot so the hdx=scsi will take effect, you have two options here. You can do nothing, reboot, and load the modules manually to see what happens. Or you can add the modules to be loaded now, and then reboot. We'll do the former. But first, you need to create the device files.
Power down and installed the drive. Power back up. After some services start, and if you are using Kudzu (Red Hat's hardware recognition program), you may be asked if you want to configure your new OnStream DI-30, TAPE drive. Say yes. After logging in as root, type modprobe ide-scsi and modprobe osst. If you see something like the capture below, everything is almost working (commands you type are in bold). (After reboot) /root# dmesg > dmesg.boot.2001-11-24 /root# grep -i scsi dmesg.boot.2001-11-24 Kernel command line: auto BOOT_IMAGE=linux ro root=307 \ BOOT_FILE=/boot/vmlinuz-2.4.9-12smp hdc=scsi ide_setup: hdc=scsi -- BAD OPTION /root# grep -i onstream dmesg.boot.2001-11-24 hdc: OnStream DI-30, ATAPI TAPE drive /root# cat /proc/modules parport_pc 14736 1 (autoclean) lp 6592 0 (autoclean) parport 26848 1 (autoclean) [parport_pc lp] via-rhine 11856 1 (autoclean) ide-cd 27392 1 (autoclean) cdrom 28704 0 (autoclean) [ide-cd] /root# modprobe ide-scsi Vendor: OnStream Model: DI-30 Rev: 1.08 Type: Sequential-Access ANSI SCSI revision: 02 /root# modprobe osst osst :I: $ Id: osst.c,v 1.61 2001/06/03 21:55:12 riede Exp $ /root/onstream# cat /proc/modules sd_mod 11952 0 (autoclean) (unused) osst 42480 0 ide-scsi 8384 0 scsi_mod 99168 3 [sd_mod osst ide-scsi] parport_pc 14736 1 (autoclean) lp 6592 0 (autoclean) parport 26848 1 (autoclean) [parport_pc lp] via-rhine 11856 1 (autoclean) ide-cd 27392 1 (autoclean) cdrom 28704 0 (autoclean) [ide-cd] /root# cat /proc/scsi/scsi Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: OnStream Model: DI-30 Rev: 1.08 Type: Sequential-Access ANSI SCSI revision: 02 You should be able to access the drive at /dev/osst0 or /dev/nosst0 (assuming this is your first tape drive). You can test this with this command:
Assuming all of that worked, you now need to add the modprobe lines so they are called when the system in next rebooted. Edit /etc/rc.local and add something like this: # 2001-11-15 JPV Added modprobes for OSST stuff # 2002-05-05 JPV Upgraded to RH 7.2 # 2002-10-06 JPV Upgraded to RH 8.0 modprobe ide-scsi modprobe osst That's it! Skip down to the Tape Device Files section. Using IDE Drivers with Kernel 2.4.x (Not recommended)I have only tested this under Red Hat Linux 7.1. After you finish installing the drive and power back up, you should see something like the following. If you miss it, try dmesg | grep -i hd once you have logged in): hda: WDC WD450AA-00BAA0, ATA DISK drive hdb: IOMEGA ZIP 100, ATA DISK drive hdc: OnStream DI-30, ATAPI TAPE drive hdd: ATAPI CDROM 48X, ATAPI CDROM drive After some services start, and if you are using Kudzu (Red Hat's hardware recognition program), you will be asked if you want to configure your new OnStream DI-30, ATAPI TAPE drive. Say yes. This will create the device files you need. That's it! Once the system is completely up and you log in, you should be able to access the drive at /dev/ht0 or /dev/nht0 (assuming this is your first tape drive). You can test this with this command:
Skip down to the Tape Device Files section. Using IDE IDE Drivers with Kernel 2.2.x (REALLY Not recommended)I have only tested this under Red Hat Linux 6.2. While not trivial, this is not as bad as you think it is. Sooner or later, if you continue to use Linux, you're going to have to learn how to compile stuff, especially the kernel. Why not start now? Also, per http://www.redhat.com/support/hardware/intel/62/rh6.2-hcl-i.ld-7.html:
Obviously, they are wrong, but they are right with one thing--OnStream tape drives will not work with Red Hat 2.2.x kernels--you do have to roll your own (does not apply to 2.4.x kernels. If you are using a 2.4.x kernel, you are a reading the wrong section!).
The trick here is to enable all the stuff you need, without enabling stuff you don't need. I recommend using make menuconfig or if running X make xconfig as they are far easier to use and more tolerant of changing your mind than just make config. I had some trouble getting make menuconfig to run. It kept whining about curses so I eventually had to install all the ncurses RPMs on the Red Hat 6.2 CD. Something did the trick (I suspect the ncurses-devel package), because it worked after that. After you finish recompiling and installing the 2.2.x kernel and reboot, you should see something like the following. If you miss it, try dmesg | grep -i hd once you have logged in): hda: WDC WD450AA-00BAA0, ATA DISK drive hdb: IOMEGA ZIP 100, ATA DISK drive hdc: OnStream DI-30, ATAPI TAPE drive hdd: ATAPI CDROM 48X, ATAPI CDROM drive After some services start, and if you are using Kudzu (Red Hat's hardware recognition program), you will be asked if you want to configure your new OnStream DI-30, ATAPI TAPE drive. Say yes. This will create the device files you need. That's it! Once the system is completely up and you log in, you should be able to access the drive at /dev/ht0 or /dev/nht0 (assuming this is your first tape drive). You can test this with this command:
Tape Device FilesThere are two types of tape device files, the rewinding device and the non-rewinding device. As you can probably guess, the rewinding device rewinds the tape after each operation, the non-rewinding device doesn't. Be careful with this! If you use the rewinding device, then make two consecutive backups, the second will overwrite the first, which is probably not what you wanted to do! They are specified by the device name for the rewinding device, and the device name prefixed with "n" (for non) for the non-rewinding device. For example, a correctly installed DI-30's IDE devices are: /dev/ht0 and /dev/nht0 or OSST device are /dev/osst0 and /dev/nosst0. Well, technically the 0 indicates that this is the first device of its type. Osst1 would be the second device, etc. Some tape software looks for an environment variable, imaginatively called TAPE, to see what tape device to use if nothing is specified. TAPE is often set to /dev/tape, which may or may not actually exist on your system. If it does exist, it's quite likely to be a symbolic link to the real device. Also, note that /dev/tape may be linked to the rewinding device! Type echo $TAPE to see if it's set on your system. You can edit your /etc/profile and add TAPE=/dev/tape (or ntape) if necessary. Don't forget to add TAPE to an export line somewhere in there too. You also want to create or verify the following symbolic links: DI-30 using the OSST driver: ln -s /dev/osst0 /dev/tape Rewinding device ln -s /dev/nosst0 /dev/ntape Non-rewinding device DI-30 using the IDE driver: ln -s /dev/ht0 /dev/tape Rewinding device ln -s /dev/nht0 /dev/ntape Non-rewinding device General Tape Drive OperationMake sure you have a reasonable recent version of "mt" installed. Anything news than mt-st-0.6-1.i386.rpm should be OK. Then see the man page for the "mt" command--you're going to need it. Some highlights to get you started are the following. Note mt's default device is "/dev/tape" so you should set up the symbolic links above to whatever device you are actually using (OSST: /dev/osst0, IDE: /dev/ht0). The device may be specified or overridden using the -f switch, such as "-f /dev/ht0" or "-f /dev/osst0".
Try the tape status command from above. It looks like this if it works and there is an initialized tape in the drive: mt-st-0.7-6 and OSST Driver (nice), on Red Hat 8: /root/# mt status OnStream SC-, DI-, DP-, or USB tape drive: File number=0, block number=0. Tape block size 512 bytes. Density code 0x40 (DLT1 40 GB, or Ultrium). Soft error count since last status=131 General status bits on (41010000): BOT ONLINE IM_REP_EN mt-st-0.6-1 and OSST Driver (nice): /root/# mt status OnStream SC-, DI-, DP-, or USB tape drive: File number=0, block number=0. Tape block size 512 bytes. Density code 0x40 (no translation). Soft error count since last status=0 General status bits on (41010000): BOT ONLINE IM_REP_EN mt-st-0.5b-10 and OSST Driver (mixed results here): /root/# mt status Unknown tape drive type (type code 97) File number=0, block number=0. mt_resid: 0, mt_erreg: 0x24 mt_dsreg: 0x40000200, mt_gstat: 0x41010000 General status bits on (41010000): BOT ONLINE IM_REP_EN IDE Driver: /root# mt status SCSI 2 tape drive: File number=0, block number=0, partition=0. Tape block size 32768 bytes. Density code 0x0 (default). Soft error count since last status=0 General status bits on (41000000): BOT ONLINE And the tape is not initialized, it'll take about 10 minutes for the command to come back and fail like this: /root# mt status SCSI 2 tape drive: File number=0, block number=-1, partition=0. Tape block size 32768 bytes. Density code 0x0 (default). Soft error count since last status=0 General status bits on (1000000): ONLINE If there is no tape in the drive you'll get this: /root# mt status /dev/tape: Device or resource busy If you got the first message, congratulations. You're all set! If you got the second one, you forgot to erase the tape, which you need to do before using it for the first time. This initializes the ADRL header, about which you probably just got a bunch of console messages. OK, we now have something to backup to. Now, what data do we want to backup, and how do we want to do it? Before we get to that, however, we have one more quick tape operation issue. Finding Out What Is On An "Unknown" TapeWith a DI-30, there is no easy way to find out what is on a tape. You have to just know. Thus, having a simple system, labeling and cataloging are important. If you don't know, the best you can do is try various table of contents (ToC) commands from various tape backup programs to see what you get. You can also try using mt to fsf and try ToC commands again. Using my tape script, you could look for tar and afio (cpio) archives like this: tape rewind tape ttoc tape toc tape findnext 1 tape ttoc tape toc When you get afio: "/dev/tape": No input you are past the data (i.e. archives) on the tape. If you got nothing, then you are using the wrong program, which means the archives are not tar or afio (cpio) or there is nothing on the tape. If you get tar: This does not look like a tar archive--well, you can figure that one out. Likewise, afio will say afio: "/dev/tape": Unrecognizable archive. Part 2: Your Backup StrategyFirst, as I was recently told by a friend and UNIX guru specializing in very high-end high-availability and clustering, you have to be able to RECOVER--you do not necessarily have to backup. Think about that for a moment before you continue. What do you need to have to be able to RECOVER? Rather than rewrite what has already been well written, I now refer you to the following two chapters: The Linux System Administrators' Guide: Chapter 10. Backups and Linux Administration Made Easy: Chapter 8. Backup and Restore Procedures . The two articles above are very good, and I strongly recommend reading them. However, those authors were trying to be Linux generic, and I am being OnStream and Red Hat specific, so on with the show. Recovery StrategyYou need to have a well thought out strategy if you are to recover data successfully. There is no "one size fits all" strategy, because every environment is too different. There are, however, some guidelines:
Types Of BackupsThere are many different types of backups and thinking about them all gives me a headache. However, you have to understand at least a little about some of the types in order to decide which ones you need to implement. A full backup is just that--you backup the full system-- everything. But even that's not true, as there are always things you never want to backup. As I mentioned above, the /proc filesystem and temp directories at the best example. /proc doesn't really exist. it's a made-up filesystem containing all the details about the system. it's only a filesystem because everything in UNIX is treated as a file. Backup it up is not only useless, some of the system is recursive (it points to itself, more or less) so it can really confuse and even crash your backup. Likewise, backing up the temporary directories is pretty silly. There's a reason they are called temporary! Differential and Incremental backups are just different ways of backing up data that has been changes since the last full backup. Likewise, the "levels" used by some program (such as dump/restore) are just ways of representing data that's changed since the last higher level backup. And there are all kinds of minor variations on all of the above, especially the fact that you can use one type of backup on one day, and another type the next day. Finally, each type presents different problems to backups and more importantly, restores. As you will see below, it could easily take restores of data from 5 or mare tapes to get back you where you left off. And try to find just one or two specific versions of specific files? OK, I'll pause while you go take some aspirin. Come to think of it, I'll take two while you're at it. This figure illustrates the difference between a differential and an incremental backup. Note that in a standard Differential Backup each backup uses the same tape, while in a Modified Differential Backup each backup uses a different tape. [ F U L L ] [ B A C K U P ] { DIFFERENTIAL } { DIFFERENTIAL } { DIFFERENTIAL } { DIFFERENTIAL } [------------------------------------------------------------------] [ CHANGES TO YOUR DATA OVER TIME ] [------------------------------------------------------------------] [ F U L L ] [ B A C K U P ] {INCREMENTAL}{INCREMENTAL}{INCREMENTAL}{INCREMENTAL} [------------------------------------------------------------------] [ CHANGES TO YOUR DATA OVER TIME ] [------------------------------------------------------------------] Then of course, there's the data to consider. I think that's best explained by example. My File System (More Or Less Typical)The following table summarizes most of the important information about my environment:
See http://www.pathname.com/fhs/ for the Filesystem Hierarchy Standard, (v2.1 as of this writing). This details what things should be located where, and is an excellent reference. My Requirements
Some Possible SolutionsThe easiest thing to do is a full backup once a day, week or month, depending on your environment and then just call it a day. Depending on how much data you have, how big your tapes are and how fast your tape drive is, this may work for you. Most of the time, not all of your data will fit on one tape (less of a problem with 30 Gig DI-30 tapes), or it'll take too long to do a full backup, or something. Also, it can take a lot of tapes, which do not grow on trees. Seven (7) tapes labeled: Week1, Week2, Week3, Month1, Month2, Month3, Month4. The Week tapes are used every week, either Monday to Friday, or just Friday (note these tapes will need to be replaced most often, as they will get the most use). Month1 is used at the end of the first month, and so on. Either the previous week or the previous month tape is moved off-site. Depending on space requirements, the Monday to Friday backups could be incremental, differential or full, and could be appended to each backup set. The Month tapes are complete system backups. This strategy also gives you a 4 month window to recover data, but you may lose the weekly/daily backups of different versions of highly dynamic data, depending on exactly how you set it up. Another possible option is eleven (11) tapes labeled: Monday, Tuesday, Wednesday, Thursday, Friday1, Friday2, Friday3, Month1, Month2, Month3, Month4. The Monday to Thursday tape are used every Monday to Thursday (note these tapes will need to be replaced most often, as they will get the most use). Friday1 is used on the Friday of the first week, Friday2 at the end of the second week, and so on. Month1 is used at the end of the first month, and so on. Each week, the preceding Friday tape (which will sometimes be a Month tape) is taken to a secure off-site location, while the old off-site tape is brought back. Either the Friday or the Month tapes are complete system backups of EVERYTHING, while the Monday to Thursday tapes are full backups of "dynamic and important" data. This strategy gives you a 4 month window to recover data, plus 5 days of different versions of highly dynamic data. The most you would have to restore is two tapes, the most recent full system, and the most recent full data tapes. However, this requires a good number of tapes, and may take a while to do a full backup. The Monday to Thursday backups could also be differential or incremental, that that will substantially increase restore complexity, while substantially lowering backup time. Additional monthly tapes may be added to give any number of "archival" copies. Finally, a cheaper way to do it is four (4) tapes labeled: Tape1, Tape2, Tape3, Tape4. These are used either everyday or at the end of the week as above, with the previous tape being taken off-site. Needless to say, the above barely even scratches the surface of the possible options. If the number of tapes is not an issue, all sorts of other plans will work well. I have found the above plans to work well for me, in my environment over the last several years--your mileage may vary. My SolutionMy solution in this case is to use eight (8) tapes labeled: Week1, Week2, Week3, Month1, Month2, Month3, Month4, Month5. The weekly tapes are used once a week, early Monday morning (note these tapes will need to be replaced most often, as they will get the most use). Month1 is used at the end of the first month, and so on. Either the previous week or the previous month tape is moved off-site. The Monday backups are "full" backups of the dynamic data, the monthly tapes are complete system backups (with excepts for junk). This strategy also gives you at least a 4 month window to recover different versions. Now a problem crops up because it works out that some months have five Mondays in them. There are a couple of ways to solve that problem, but I took the easiest--I ignored it. So periodically your "Monthly" tapes will get out of sync with the last Monday of the month. Too bad. My weekly backup set is the set of: /etc /home /root /var/log /var/named /var/spool My monthly backup set is the set of: /, minus the set of: /a /cd /fpy /mnt /proc /tmp /var/tmp /zip Interestingly, it turns turn that the space and time different between my two sets is not very much. I could just use the slightly larger full or monthly set for all tapes, but keep the rotation and other part of the strategy the same. That came as a surprise to me. I expected there to be much more difference. So you'll just have to try it, and see how you make out. I'm leaving it alone as I see no compelling reason to change it. I also use the Red Hat KickStart and mkbootdisk tools with my own RDISK script. I have written a shell script that automates everything except changing the tape, and I have a 7 day windows to remember to do that. Part 3: Putting It All TogetherOK, given everything above, let's actually get into the details. KickStartKickStart is a Red Hat automated installer program. If you install and then use the "mkkickstart" program, you can create an "answer file" that allows you to automatically install everything exactly the same as you just did. That, combined with your CD-ROM, allows for a pretty cool recovery tool in case of disaster. Just replace the failed hardware (or in most cases get close enough) and you're set. See the RDISK script below and http://www.redhat.com/support/manuals/RHL-6.2-Manual/ref-guide/ch- kickstart2.html for more details. UPDATE: There is not a command line "mkkickstart" program in later versions of Red Hat Linux. mkbootdiskYou should also install and use the "mkbootdisk" command to make a recovery disk that may be able to boot your system if something goes wrong. it's not a bad idea to keep two of these, and alternate using them when you make changes. RDISKRDISK (the name comes from the similar facility on NT) uses mkkickstart and mkbootdisk, fdisk, rpm and du to capture a lot of critical information about your system, mostly your file system configuration. You can write the data to a floppy, or not (in which case mkbootdisk does not run). configbackupThis simple script just copies important files someplace else. You can copy them to a floppy, if they fit, or to another drive or server or whatever. You can keep multiple versions if you want. How you implement it is up to you. It is never used in any of the other scripts here, and is included only as a convenience. /root/updatesAnother useful strategy is to create a /root/updates directory (or whatever) and keep all the installed patches and updates in it. You do updates your system as necessary, don't you? If you need to use the KickStart file, then restore, it's amazingly easier to bring the system back up to speed when you can go into /root/updates and basically do an rpm -Fvh *.rpm. OK, it's a little more complicated than that for some updates such as the kernel, but that works 90% of the time. Also, this directory doubles as a record of how your system differs from a "stock" installation. jpbackupNOTE: the "tape" program below may actually be a lot easier to use, even though (or maybe because) it has less options and automation. I use it for ad hoc backups of file systems that change infrequently. It works much better than I would have guessed, even though I wrote it. Thanks to Robert Squire for the tip! ALSO NOTE: this script is pretty buggy. It works for me, the way I use it, but i do not recommend its use in a production environment. If you do use it, test it thoroughly and make sure you understand exactly what it's doing. jpbackup is the heart of the system. It pulls the other scripts together and actually runs the show. It implements and automates the 3 weekly and 5 monthly tape scheme above, and under normal circumstances (i.e. you do not have to do a restore) all you have to do is change the tape between every Monday. You should really look at the logs as well. In particular, I've added code that shows how long the backup took, and how big the backup set is on disk, then how big it is on tape. Given that information, you can tailor your compression settings to speed up your backups if your tapes are big enough. It uses the rewinding device, and pretty much forces you to have only one archive per tape. Conceivably, this wastes tape, but is far more simple in many ways. It keeps a catalog of what is on each tape, named "Monday_1.cat," "Monday_2.cat," etc. The afio log file is also kept, with the same name except .log. Finally, a backup.log is kept with start times, and the data sizes. Here is an outline of operation:
Sample backup.log Thu Feb 22 02:58:41 EST 2001; START: weekly backup to Monday_1... Thu Feb 22 06:38:06 EST 2001; FINISH: weekly backup to Monday_1... Thu Feb 22 06:38:06 EST 2001; START: Verify weekly Monday_1... Data size on disk: 12069397386, size on tape (GZ 4) 4269799725. Thu Feb 22 09:25:02 EST 2001; FINISH: Verify weekly Monday_1... From this you can see that the backup took under 4 hours, that 12 Gig was backed up, but using only Gzip compression level 4 (9 is best compression/slowest, 6 is default) it took up only 4.2 Gig on tape. We also see that the verify took just under three hours. If you look at Monday_1.log, you'll also see some verify errors such as the following. That's because some files CHANGED between when they got backed up and when they got verified. This is normal! For example, the backup.log file was updated by the backup itself, after it got backed up. Thus it fails the verify since the disk version is different than the tape version. afio: "var/log/backup/backup.log": Archive data and file cannot be aligned (disk 1) at Wed Feb 21 16:07:56 2001 afio: "var/log/backup/backup.log": Corrupt archive data (disk 1) at Wed Feb 21 16:07:56 2001 afiojpbackup requires afio, which is not a part of Red Hat's default install. There is a version in the Red Hat 7.1 Power Tools, but it's ancient. Just use http://www.rpmfind.net/ and grab if. I've been using afio-2.4.7-1mdk for forever. See the end of the backup script for the options I used. See the afio man page for all the options, of which there are a plethora. tapeTape is just a simple front end to keep all the tape related commands in one place. Especially since afio has so many options, it's a real pain to remember and type them all. So you can edit tape to work on your system, then pretty much just run it ad hoc if needed. RestoringThe ability to restore or recover is the entire point of this exercise, yet there is not all that much I can say about it. There are many variables, but by now you should be getting a feel for them. The following questions might help. Note that I am dealing only with restoring from tape. Rebuilding the system, using KickStart, recovering from hardware failures, etc. are all beyond the scope of this document.
To restore everything from a compressed tape archive, and to overwrite, you need to be in the root ( / ) directory. To restore everything from a compressed tape archive to a different directory, you need to be in that directory. Then: afio -ivxZ -b 32k -M 10m -L /var/log/jpbackup/jpbackup.log -@ root /dev/tape To restore just the "/root" directory from a compressed tape archive, and to overwrite, you need to be in the root ( / ) directory. To restore everything from a compressed tape archive to a different directory, you need to be in that directory. It can even handle the leading / in the path (even with the use of relative paths)! Then: afio -ivxZ -b 32k -M 10m -L /var/log/jpbackup/jpbackup.log -@ root -y "/root/" /dev/tape Scripts (Code)Read the code. It is well documented and there are more notes and tricks in it. I've removed the code from this document and just linked to it. Embedding it in here was a bad idea as it was a pain to update. jpbackup (Open shell script in new window)Backup is the script that actually backs up my system. it's called from cron every Monday, and it figures out what kind of backup (weekly or monthly) to do by itself. calcsum (Open shell script in new window)Requires /bin/zsh since the various Bourne shells can't do math correctly. calcsum takes integer input and calculates the sum. tape (Open shell script in new window)A generic front-end, so you don't have to remember the block size and other options. NOTE: this may actually be a lot easier to use than jpbackup, even though (or maybe because) it has less options and automation. I use it for ad hoc backups of file systems that change infrequently. It works much better than I would have guessed, even though I wrote it. Thanks to Robert Squire for the tip! RDISK (Open shell script in new window)Use "mkbootdisk" to make a rescue disk for this system. configbackup (Open shell script in new window)This is never used in any of the other scripts here, and is included only as a convenience. It copies various important files to some specified backup location AppendixesSome Notes About Common Backup ProgramsSee the following URLs for lists of Linux backup tools. Some of these tools are free, some are commercial, and some are in-between. I'm only going to talk about free ones.
The list below is of tools I've found to out there and interesting looking. I use afio, but I'm not making any recommendations that you should too. Just look at the list. I've quoted http://www.linux.org/apps/all/Administration/Backup.html and the home pages of some of the tools quite a bit (read, I stole their descriptions). That text remains the property of its respective owner. One interesting issue is that of relative paths. Older UNIX tar commands stored absolute paths (e.g. / home/user/mystuff by default. This is bad, because you can only restore to the exact same directory you backed up from. You may not always want to do that. Most GNU tools use relative paths (home/user/mystuff) so you can restore wherever you want. The downside is that unless you are in the root of the filesystem when you do a verify, it will fail, because it will use a relative path to find the files to compare with the tape and it won't find them. For example, if you are in /home/user, trying to verify a backup of your home directory, the tape software will be looking for /home/user/home/user, which is probably not there. The moral of the story is, cd / before doing a verify. The same goes for restores, except you might actually want to do this. There are often time when I need to restore /home/user, but I do not want to actually mess with /home/user, I just want a part of it. One solution is to do a partial restore. The other is to restore to the relative path, get what you need, then nuke the rest. Remember, this is only with the newer, usually GNU versions. The "traditional" tar does not work this way.
Backups using tarTar is the most widely known UNIX backup tool. It stands for Tape ARchive and does not have to actually use tape. You have almost certainly seen a .tar, .tar.Z or .tgz file. These all use tar. It has some problems though. Most notably, IMHO, it compresses the entire archive, so if tape is damaged, entire archive lost. That's a bit of a problem. So I don't like tar, but you pretty much have to know about it anyway.
You must use the correct block size (-b 64) or you get all kind of bazaar errors such as: ide-tape: ht0: I/O error, pc = 2b, key = 2, asc = 4, ascq = 1 ide-tape: Reached idetape_chrdev_open ide-tape: ht0: chrdev_write: use 32768 bytes as block size (10240 used) ide-tape: Reached idetape_chrdev_open ide-tape: ht0: skipping frame 21, frame type 8 ide-tape: ht0: skipping frame 21, frame type 8 Backups using dumpNext to tar, dump is another of the most widely known tools. As far as I know, it does not do compression at all. It uses "levels" from 0 to 9 to determine what to backup. You can create very complex and convoluted schemes to backup different things at different times. As I said above, thinking about this stuff gives me a headache.
Backups using cpiocpio is that last of the big three most widely known UNIX backup tools. it's interface is a bit different than tar or dump, in that it must be used as a filter (e.g. find / -print | cpio -ov --block-size=64 -C 32768 >/dev/ht0). It also suffers from the same compression issues as tar. Backups using afio
I like afio a lot. It works well with the DI-30, and I can script it to just exactly what I want. It is used as a filter, the same as cpio, and in fact uses the cpio format (as do RPMs). See my scripts in the appendix. The following examples are all very simple, and use gzip compression. Unlike tar or cpio, afio compresses each file, rather than the entire archive. That means if you have a media error, only the data where the error is are lost, instead of the entire archive.
Backups using star
See the tar command reference. Backups using Taper
Note the last line. Taper was developed for ftape (floppy tapes, like the QIC series drives). It is not recommended for use with OnStream drives. I have feedback from two different people that taper works fine: Date: Mon, 29 Apr 2002 16:03:30 +0200 From: Siegfried Heim Subject: DI-30 mini-howto Dear JP, In your mini-howto for DI-30 tape drive backup you liked to know, whether taper works with this streamer. I tested the DI-30 drive using taper 6.9a for my backup. So far it seems to work well with the following settings: rewinding device: /dev/osst0 (non-rewinding: /dev/nosst0) block size: 32k tapesize (in MB): 15000 I'm using the 2.4-18 Kernel that came with SuSE 8.0 Professional Distribution. It has built-in support for OnStream tape drives (uses ide-scsi emulation). Greetings from Germany -Siegfried Heim- Date: Mon, 26 Nov 2001 19:08:04 +0100 From: Freerk J. Subject: About Taper I updated Linux to 2.4.2-2. [Which] contains complete installation for Onstream DI-30. It is clearly visible during startup and also can it be found in /proc/ide. I also discovered that taper 6.9b was automatically installed. Just startup with taper -T ide, [but] you have to change the block size in the menu: Change Preference, tape drive Preferences, Block size is default on 28k. With arrow keys to change to 032K and it works! Momentarily testing a restore procedure........ That is OK too. I also have feedback that taper does not support backups larger than 4 Gig: Date: Mon, 7 Apr 2003 12:41:50 -0400 (EDT) From: JP Vossen To: Cor van den Berghe Subject: Re: Can you explain someting to me? On Mon, 7 Apr 2003, Cor van den Berghe wrote: > After reading the OnStream DL30 Backup mini-HOWTO on you're website I was > wondering if you could help me out with someting. I've been using an > OnStream DI30 (osst drivers) and Taper on a RedHat 8 system with no > problems, at least thats what I thought. A couple of weeks ago I tried to > restore something and Taper told me that the tape was corrupted. When I > looked on the Taper Homepage I found out that Taper does'nt support > backups > 4 Gb [...] Backups using KBackup
Backups using BRUOK, I lied. I said I would only talk about free programs, and BRU is not free. But it's one of the most popular backup system for small Linux systems, so...
Please note that you MUST use a 32k block size when writing to the DI30 drive. Also note that the tar statement uses "-b 64" due to its 512 block size e.g. bru -cvvf /dev/ht0 -b 32k /home. Get a complete BRU configuration file for this drive from http://www.estinc.com/downloads/brutabs/adr.bt Hints from OnStream Tech Support
Web ReferencesThe following are references to useful material on the Web. All the various links I've used above are here again, along with a bunch of other neat material. The manual (man) pages are provided in case you do not have access to a Linux machine to get the details, and because they are easier to read and print out. Also, see the links in the update section in the introduction. OtherOSST and block sizeDate: Wed, 12 Dec 2001 06:51:22 -0500 From: Willem Riede To: JP Vossen Subject: Re: FW: mt status question On 2001.12.12 00:45 JP Vossen wrote: > [snip] > BTW, I still have the old 32K block size hard coded in my program. Does it > matter? Could that have any effect on all the errors ("soft" errors?) I get? > No. block size is your choice to make. The driver (osst) handles all (un)packing of frame content in memory. Only entire frames go to the tape. Some frames have the ill fortune of meeting questional media, but that's totally independent of their content or how that content was constructed. The great thing about the ADR format is that most tape errors can be handled transparently and your data survives. Regards. Willem Riede IDE Configuration JumpersDate: Sun, 1 Sep 2002 13:19:48 +0200 From: Denis Faivre Subject: DI-30 Howto Hi, I just bought a DI30 and noticed that the indication engraved on the metallic case regarding Master/Slave/Cable jumpers is wrong. The right indication is that of the paper documentation. [CSM] [: : : : : : : : : : : : : : : : : : : :][o o o o] Maybe would it be useful to include this information into your HOWTO, or at least warn the reader about a possible confusion... Media ErrorsDate: Wed, 7 Nov 2001 11:35:23 +0100 From: Bombeeck, Jack Subject: RE: Beta test for ADR2.60ide? To get to your suspected media problems: one issue that repeatedly comes up is temperature related problems. They obviously show up as media problems, but not because of bad media, just because of working outside the operating range. To make sure that this is not bugging you, remove the drive's door and if need be make sure that at least one fan blows onto the back of the drive to produce an air flow. The latter is sometimes simply achieved by choosing the drive position in the cabinet carefully; otherwise you might add a fan. When the cartridge has been in the drive for a while (and been used), it should still feel cool to the touch when removed. If not, you run the risk of the above-mentioned problem, which results in write errors (not usually a problem since blocks are relocated until successfully written) and erratic unrecovered read errors (bad news, data's irretrievable!). Let me know how you fare. Document Historyhttp://www.jpsdomain.org/linux/OnStream_DI-30-RedHat_Backup_mini-HOWTO.html
|
http://www.jpsdomain.org/linux/OnStream_DI-30-RedHat_Backup_mini-HOWTO.html Copyright © 1995-2022, JP Vossen. All rights reserved. Last Modified: $Date: 2007-11-28 02:26:46 -0500 (Wed, 28 Nov 2007) $ |