TASS Discussions of archiving and data-transporting media and drives: abstracted and edited by Herb JOhnson, Jan 26 1999 My thanks to all the participants in our two-year discussion of this subject in the TASS newsgroup. My special thanks to my colleague Tim Shoppa for his inputs on 8mm and 4mm drives in the DEC world. Bottom line points: The domains: small and medium-sided file transfers on-site, short-term backups, long-term backups, sending files to others. The issues: What do you want to spend, how much time do you want to take, how important is it to keep old, raw data around and for how long? Media will get cheaper and denser over time, but the amount of data involved will also increase in amount. Data volumes: from Arne: " Also, say that we actually do get all 7 [Mark III] sites up and going, and that half of them are taking data on any given night. This means 1GB data/night coming from TASS sites, and even Michael R. is not going to give up that kind of storage space on his computer for Internet downloads!" Arne, March 1997. While many TASSers think keeping raw data around indefinitely is "unrealistic", most Mark III sites have kept it so far (Jan 1999). FOr Mark IV, it's a much more difficult task due to the larger datasets. Summary of media: 1) Removeable IDE drives. "Drawers" to plug and unplug drives are about $30 each, you need one per drive and one per PC. Great if you have loose IDE drives around. Fast transfers, no incompatibility problems. Long term storage, shelf life not discussed but unlikely. 2) ZIP drives. Cheap media, cheap drive. But very slow. Good for mailing some individual files or data sets around. Not a substitute for a network to swap files around regularly between systems. Shelf life less an issue as this is intermediate storage. But not bad for transferring small stuff around of the order of tens of megabytes. 3) "non-professional" tape drives, like Travan TR-3, TR-4, QUIXs, . They work, cheaper than professional drives, relatively dense. But there are problems with incompatibilities between drives and discomfort that one format will endure for the long term. Shelf life debatable. These tape drives are slower than professional drives. 4) "Professional" tape drives, 4mm and 8mm, DAT, DLT. They work, more reliable and more cross-compatible. But drives cost more. Less common in general use, may be better for local archiving. Shelf life debatable. These tape drives are faster than consumer tape drives. Used drives available but beware of buying older, more error and wear-prone drives. Note: 8mm video tape works but may create wear or contamination problems. 5) CD-ROM writers and CD-R media. Drives of modest expense, getting cheaper. Cross-compatibility pretty good. Most people have a CD-ROM drive, good for sending out a large chunk of data, and cheaply. Shelf life very good. It does take a long time to write and verify one. Good for transfers of hundreds of megabytes. Note: CD-R media are often on sale for VERY CHEAP. 6) DVD-writable drives. These are on their way. But prices high, standards not set in stone, availbility limited. Wait awhile. 7) Networks between systems. 10-base-T cards and hubs are very cheap. If the issue is moving data between two local computers, and they are running under Windows or Unix, then consider this. Not much help for remote sites. Other comments: From Chris: "Don't just look at the cost of the drive, look at the cost of the media....add to it the cost of about ten tapes." A number of participants noted the relative convenience - and time needed - to use one medium or another. The discussion: In summary, the participants express their experiences with the media types noted above. They name media types and models of tape drives, and discuss the merits, costs and limits of the other media types. Most of this is referenced to the Mark III survey, but by extension the issues are mostly the same but larger for the upcoming Mark IV survey. I have updated some prices where I could, my updates are noted. I have distilled the discussion to about 1500 lines of text, 75Kbytes total, including to/from/date/subject message headers. They are in date order and can be read in sequence. Herb Johnson Date: Tue, 9 Jul 1996 09:26:13 -0700 From: rhill@LPL.Arizona.EDU (Rik Hill) To: tass@wwa.com, droege@wwa.com Subject: Re: Data If [Mark III is] producing ~100Mb/night you may want to consider using a Zip Drive. This way you could keep one night's work on one disk. We have a similar problem where our Venus Lightning project piles up ~400-500 Mb/ night. Optical disks work for us. Oh for a terabyte disk! -Rik From: chris@topdog.pas1.logicon.com (Chris Albertson) Date: Thu, 17 Oct 96 13:12:15 PDT To: hjohnson@pluto.njcc.com Subject: Re: Taking Data (archive media, economics of) Cc: tass@wwa.com >> > On Wed, 16 Oct 1996 12:02:23 -0600 (CST), Tom Droege wrote: > *>Hmmmm! We seem to have two groups of people (with at least one > *>exception). There are those that have telescopes and could be > *>taking data, and for the most part they do not seem to be talking. > *>Then there are those that are talking about analysing data but don't > *>have telescopes. If any of the latter group wants to look at > *>real data, I have some that I am willing to put up on storm. I > > Speaking for myself, I help Michael Richmond from time to time, so I > "almost" have a camera. The point is well-taken. Michael and I have > talked about putting data up for others to process. The problem is > the sheer bulk of it in general, and that his camera is somewhat > inaccessable at the moment and his time is limited. > > I mention this becuase we've considered archiving data onto 150 Mb > ZIP disks. In short, this medium is relatively inexpensive ($15-$20 > US), the drives are cheap, light and portable ($150-$200 US) and in > common use for just this application - the transfer of large image-based > files. Some scheme whereby ZIP disks become a "medium of exchange" > would be most encouraging, I believe: provided that those who have both > time and resources for processing data will accept these disks, and > of course actually do the "processing". Comments supporting this > concept are encouraged. I do _not_ want to start another long archive media war but as I have been shopping lately I could not resist posting my findings. The point I want to make is to look at not just the drive but the over all "system". Remember that at one time Kodak gave away cameras. Zip drives are not bad. I may in fact buy one but let's look at the economics of using zip drives for a large data archive. Let's look at the cost of a drive _and_ media. [note; prices updated by Herb Johnson for Jan 1999] 1) Zipdrive - 1GB $100.00 + 10 * $ 9.00 = $190 2) Zipdrive - 4GB 100.00 + 40 * 9.00 = 460 3) 10GB 100.00 +100 * 9.00 = 1000 4) 4mm DAT - 1GB $600? + 1 * $ 7.00 = $??? 5) 4mm DAT - 4GB 800 + 1 * 11.00 = ??? 6) 10GB 1000 1 * 25.00 = ??? 7) Hard drives (1GB) $100 each 8) Hard drives (4GB) $150 each 9) 10GB) $250 each Notes: 7, 8, 9 are simply buying low-end 1GB IDE drives and swapping them out when they are full. Don't laugh there is a $30.00 kit that makes this an easy one handed no-tools required operation. Kind of like a ZIF socket for hard drives. 4, 5, 6, [I assume below] a 2GB DAT drive. with [multiple] 2GB tapes. We use a 4mm DAT juke box here at the office for backups. Media cost is more importent the drive cost for large datasets. We keep 100+GB in a desk drawer for less about $250 in media cost. We use 8GB tapes. An other thing to concider (for archival storage) is technology. Will there be drives for the media around in 5 or ten years? #7, 8, 9. solves that problem and I'll bet money on DAT sticking around. Still I may buy a zip drive. They are good for transporting small amounts of data. To: tass@wwa.com From: hjohnson@pluto.njcc.com (Herbert R Johnson) Subject: Re: media survey -> regulated anarchy? Date: Wed, 19 Mar 1997 12:50:06 -0500 Some comments: On Tue, 18 Mar 1997 23:20:06 -0800, Chris Albertson wrote: Chris argued for removable IDE drives, and against tapes, CD-ROMs and ZIP drives. Tapes are too much trouble, I agree there. Otherwise I disagree with his assessments. To save time, let's just tote the costs: One CD-ROM, under $10.[Jan 99 under $1] CD Rom drives are "free". Writers require some scrambling to find, they will be cheaper soon enough. Five ZIP disks, 500 MEgabytes, $75. [Jan 99 $50] Divide cost of Zip drive by number of uses, or borrow it, or ask the requestor to ship HIS/HER drive to you. Cheapest IDE drive, $120-$150 new. Add docking station, removable drawers... Tapes are slow, not standardized, and are incompatible sometimes anyway. My ranking: CD-ROMs and ZIP drives, IDE removables a distant third with tapes. I won't comment on Chris's distribution proposal as I disagree with his premises. I will say the Usenet news analogy does not hold: being an archivist or data producer is an intensive HUMAN activity and requires some particular equipment. Passing News packets around is an automatic procedure with equipment in hand. There is a kind of "who bells the cat?" flavor to this discussion. Seems to me it comes down to this: who will help the Mark III camera owners burn some CD-ROM's? And: who will offer to provide custom data subsets from the CD-ROM archives? The same people need not do both if you make more than one CD-ROM set, which is only reasonable. I would suspect the camera owners will find Zip drives convenient, but it doesn't matter HOW they *temporarily* archive their data. But its a signifigant effort and responsibility: if they aren't interested, the argument is moot. Some one or some group can act as archivist in two ways: to ARCHIVE the data and to DISTRIBUTE the data. Archivists should have access to CD-ROM writers and ZIP drives (requestors can always SEND THEIR DRIVES), support UNIX and/or MS-DOS file systems, and/or provide an Internet FTP site: how they take it in from the camera owners is between them and the owners, as Chris suggests. How they trade in all this really is idiosyncratic (as you suggest), so long as they can produce to meet demand. I guess this is a "marketplace" model: the *public* question is, "are Zip disks and CD-ROMs and FTP acceptable media to recieve TASS data?" My answer is obvious. Finally, all this is a "tempest in a teapot" if few people want all the data, and most people want a little data. Needs, interests, have not been established. There are still discussions about throwing out raw data! Chris, since it's ultimately up to the camera owners, and we have too much maillist mail, I'd suggest taking this discussion *only to them*, and also to any who want to be archivists. Has anyone volunteered to "bell the cat"? Not me! Herb Johnson Date: Wed, 19 Mar 1997 22:59:21 -0800 From: Chris Albertson To: tass@wwa.com Subject: media wars Looks like I've started an argument, I was trying to be neutral and diplomatic I was simply predicting that a grand plan for exchanging data would likely not work and that data exchanges will have to be worked out as bilateral agreements between sets of two people at a time. I was also saying that with enough of these agreements everyone can have all the data they want without over working any one person. I also do not favor one media type over an other. Each has it's uses Herbert R Johnson wrote: > > Chris argued for removable IDE drives, and against tapes, CD-ROMs and No, no, no, I was not arguing for IDE drives as any kind of standard I was just pointing out an option that was overlooked. It may work in some cases. For example if I happen to have an old 850MB drive laying around I could buy two drawers for it (at $30.00 each) and have a quick and cheap 850MB removable device. A removable setup = $30.00 for each computer. Must buy drawers all at once from the same vendor as they are not standarized. Add up the numbers and see if it works for you, may or may not. Very cheep if you have a "free" drive. Yes I expect [CD-ROM writers] will drop to the $300 level and stay about there. Cheap enough. _But_ have you ever tried to make a CD ROM? This is not something you want to do 5 or 6 times in one day. The practical aspects of CD burning are such that you just will not use CD-R in a production mode. It would be a very good idea to save up a few months of "A" and "B" field data then burn a half dozen copies to CD-R. but this would be a once a month issue. _I_ will even offer to make a few CD-ROMS now and then but I don't see it as a production mode media. Tom just e-mailed me. Says he has already had enough of using ZIP to transport data even between two computers in the same house. Says it takes stacks of disks and the process is dead slow. Looks like his use of ZIP is pushing the media past it's usfull limit (Tom, sorry for the bad paraphrase) I think ZIP has it's place but not for storing multi-gig data sets. > Tapes are slow, not standardized, and are incompatible sometimes anyway. No, **cheap tape drives** are slow, not standardized, and are incompatible. If you have $700+ to spend on a tape frive these problems go away. That is a big "if" and I assumed no one would buy one of these drives. But several people already have them. New DAT drives are as fast as SCSI disks and _are_ standardized. exchange is not a problem. Holds 8GB on a small cheap cartdridge but at $1K per drive. Also you can random access to any file in one minute. This is realy the way to go but the cost is a killer. > > My ranking: CD-ROMs and ZIP drives, IDE removables a distant third with tapes. I think it all depends on volume and frequecy of transfer and what you already own. For example if two people found that they both owned 4mm tape then they would certainly pick that, as it can write multi-gigabytes to a $9.00 reusable cartridge and is as fast as a SCSI disk. > > Some one or some group can act as archivist in two ways: to ARCHIVE the data > and to DISTRIBUTE the data. I could do the archive, but I don't see how I (or anyone else) could handle data distribution themselves. I'd spend half of every day burning CDs, copying tapes and running to the post office. Nice as it sounds I just don't think you will find one volunteer willing to be the central distribution point. I proposed a distributed system where nobody is required to send out more data then they collect. I think I've beatten this subject to death. So I promise not to post any more of this stuff to the list. -- --Chris Albertson home: chrisa@wavenet.com To: tass@wwa.com Subject: media Date: Wed, 19 Mar 97 08:25:01 -0700 From: aah@nofs.navy.mil Not to complain, but the whole purpose of the media question was for easy transport of reasonable amounts of TASS raw fits images. As I mentioned much earlier, Internet is the logical wave of the future for this, but many sites (including ours) do not have high speed data links. This means that transfers of even one night's data from one site takes a significant fraction of a day, and hogs the bandwidth from other users. Also, say that we actually do get all 7 sites up and going, and that half of them are taking data on any given night. This means 1GB data/night coming from TASS sites, and even Michael R. is not going to give up that kind of storage space on his computer for Internet downloads! Tom has a Zip drive, and it makes sense to use that media as a first choice to copy data to users with slow links and without cameras (Nick could copy the Zips to CDROM for those users without Zip drives), at least for the SMSP-A and -B fields. After that, I still say that the raw frames can be archived at the originating site in any fashion that site wants, and then send the extracted data files to Michael R.'s computer. Arne To: tass@wwa.com Cc: turner@bigbang.astro.indiana.edu Subject: Re: media wars (non-issue) Date: Thu, 20 Mar 97 08:11:49 -0500 From: turner@bigbang.astro.indiana.edu This data interchange problem (I believe) is only a short time issue; once TASS "goes into production" the only images you'll be keeping are probably random images from a night to check quality control or perhaps the software will evolve to a point where the data reduction software can trigger the camera computers to save a particular image because it found something interesting; (when it gets this far let me know & we'll tie it into RoboScope's Trigger mechanism) but we're a ways from that at this point. You're going to be producing so much data that you won't be able to hold it all. Probably what will happen is that the camera operators will archive as much data as their disks can hold and oldest data gets deleted to make room for new data. Right now the problem is getting data from the camera operators to the software folks and getting their results feedback to the camera folks. There is some need to exchange lots of images because we're studing the stability of the camera systems and an evolution of the software to deal with the various aspects we're seeing in the images. But, I think this is best handled by network servers for this volume of traffic. (in a couple of weeks I can probably help out here) At some point we'll want to test the software against large amounts of data files; one on one solutions worked out then will be the best. I think if everybody is expecting to have a duplicate copy of all the data then they've got a huge, unweildy, problem on their hands and no pratical way of doing anything with the data once they've got it; just keeping one copy of all the data would be no less challenging. (some infinities are bigger than others) The TASS system will have to be an automated system and this data will have to be dealt with much as time is; it's a fleating thing and you've got to make the most of while you have it. Snapshots are allowed; keeping all of reality is not. Just some thoughts.... - geo george wm turner turner@bigbang.astro.indiana.edu USA+ 812 855 6911 http://www.astro.indiana.edu/~turner/ Date: Sun, 20 Apr 1997 03:22:16 -0700 From: Chris Albertson To: tass@wwa.com Subject: ZIP drive speed Tom Droege wrote: > > PS I think about this kind of thing while I am moving 1.4 Mb files > around. Why does it take 7 seconds to write a file on a Zip drive > on a 50 MHz 486, but 22 seconds to read it on 100 MHz Pentium. I > seem to be moving a lot of files lately. I have run the speed up > program. > I think Tom has the Parallel port version of the ZIP drive. There is also a SCSI version of this drive. I would suggest to anyone buying a new Zip drive to read the specs on _both_ boxes before buying one. If you do will see that the SCSI version is much, much faster, but you will need a SCSI card. So there is a trade-off. Tom's problem is IMO due to the nature of the parallel port hardware. It makes a very poor interface for a disk drive. It gets worse if the port is the "standard" type as these were not bidirectional and the data must be pushed a few bits at a time over some status lines. So I suspect the printer port of the Pentium is configured as "Standard" and not ECP or EPP (Forgot the exact spelling) You should be able to change this in the BIOS setup. SCSI on the other hand is an intelegent interface. Not only is it faster but many SCSI devices have the ability to queue commands, so the device can used at its full potential. -- --Chris Albertson home: chrisa@wavenet.com Redondo Beach, California work: chris@topdog.logicon.com From hjohnson Mon Apr 21 12:00:49 1997 Received: (from hjohnson@localhost) by pluto.njcc.com (8.8.5/8.8.3) id MAA14562; Mon, 21 Apr 1997 12:00:47 -0400 (EDT) To: tass@wwa.com From: hjohnson@pluto.njcc.com (Herbert R Johnson) Subject: Re: ZIP drive speed Date: Mon, 21 Apr 1997 11:16:12 -0400 In regards to mass file transfers, Chris made another point recently about removable IDE drives that I initially scoffed at. But having caught my attention, I noticed that some of the mail-order houses (and computer sales shows and fleamarkets) offer removable IDE drive "frames" for as low as $30-$40. Using an old IDE drive and two of these frames would allow Tom et al to simply *move a drive* from one machine to another. You'd may have to use your ROM BIOS's "autodetect IDE" feature to recognize the new drive (or accept grumbles from your BIOS when it's not around) but these take little time. Chris, my apologies, this is a reasonable alterntative for two-computer non networked TASS sites. Herb To: tass@wwa.com Subject: data archival Date: Tue, 31 Mar 98 08:30:43 -0700 From: aah@nofs.navy.mil I second Chris' comment regarding CD-R drives: they are now down around $200 for the 2x variety. CD media have long-term archival quality, they can be read by just about everyone, have reasonable capacity and low cost, and are inherently random-access. In the future, I would recommend DVD, but for now I think CD-R is pretty darn good (and much better than tape). BTW - I was at an instrumentation meeting in Hawaii for the past two weeks and picked up some interesting tid-bits which I will pass on after I get caught up. The unfortunate thing was that I had no internet access for the entire period! Arne From droege@wwa.com Mon Mar 30 22:34:19 1998 Date: Mon, 30 Mar 1998 21:32:16 -0600 To: hjohnson@pluto.njcc.com From: Tom Droege Subject: Re: Satellite Tracks Presently on Travan 3 tape. Next computer I will probably put on a Travan 4 drive. Then I use zip drives to mail data. Tom Date: Tue, 31 Mar 1998 06:13:36 +0000 From: Chris Albertson To: hjohnson@pluto.njcc.com, tass@wwa.com Subject: Re: Satellite Tracks Herbert R Johnson wrote: > > On Mon, 30 Mar 1998 12:57:06 -0600, Tom Droege wrote: > *>I don't think there was a consensus to throw away any raw data. I have > *>saved all raw data for which I have calibrations, and I think everyone else > *>has also. A Traven 4 tape at $25 or so holds 20-40 nights of data. A year > *>or so at my present rate. ;^( > > Hmmm...From our previous discussions I had presumed that Mark III camera > holders were not saving their raw data. I certainly would encourage them > to retain it, and I buy into Tom's points. I'd also add: keep the data to > look for different KINDS of objects and events! > > I'd be curious as to how people are archiving their data. > One, to know they are; two, I'd like to buy a tape drive and I can't > easily choose among the many standards! As I write this I am testing out a new CD-R drive. Just hooked it up 30 minutes ago. I got I good deal, $209.00 The CD blanks are now as low as $1.40 each. Still CD-R is not so easy as just copying data. About tapes: Add up the cost of the drive +and+ about 10 blank tapes. Don't just compare the cost of the drives. Next pick a format that will be supported years down the road. I figure 8mm and 4mm are here to stay as they are not targeted at the volitle PC decktop market. Next look at the data interface. Pick one that does not have a high CPU overhead or you will be forced to wait while the tape runs. I have SCSI tape, CD, CD-R, and scanner and can recommend it. SCSI off loads the low level bit twiddling from the CPU. You will have a problem exchanging data on tape between PCs. There is no standard data format. So you will also need to have the same software at each end. On the other hand 4mm "tar" format tapes are almost universal. CDs are better IMO for data distribution but tapes are much quicker to make and hold 5X as much data. Once I get my camera set up I'd be happy to supply data via either 4mm tape, QIC-80 (Yes I still have one) or CD. I can also translate between the three media. -- --Chris Albertson Date: Tue, 31 Mar 1998 10:50:37 -0500 From: Nicholas Beser Subject: Re: data archival To: aah@nofs.navy.mil Cc: tass@wwa.com aah@nofs.navy.mil wrote: > I second Chris' comment regarding CD-R drives: they are now down around $200 We have been using CD-R to archive all of the raw APL collected data. I now have a CD-ROM writer on my desk, and I routinely copy the TASS CD's for Marty Pittinger. We also have a linux system on line with 5 CD-ROM drives that we put the last months's worth of data on. Unfortunately, Baltimore weather has not been clear enough for data collection for the last two months, (with the exception of last weekend.). Our system is coming back on line after down-time for mantenance. Hopefully we will have more data to contribute. Nick Date: Wed, 19 Aug 1998 23:35:11 +0000 From: Chris Albertson To: hjohnson@pluto.njcc.com, tass@wwa.com Subject: Re: compression/tapes Herbert R Johnson wrote: > I think it > would be helpful if you could come up with some company names or > vendors that sell older versions of the tape drives you mention above > and some prices. Herb asked for some info on tapes. This may be of general interest. Bottom line as I see it is Tr3 wins if cost is the #1 criteria. It is intended and best at backing up one isolated PC. If you have a big network you need an autoloader and an automated script to drive it. IMO DAT (dds2 or dds3) is a good middle ground for a site with a few tens of GB of data. I got lucky and found a used DAT drive in an external box for $250. It took a few months of "keeping my eyes open", posting "WTB ads on the net and so on. One other guy in the office recently picked up a refurbished (with warranty) dds2 DAT drive for $350. I have seen an older autoloader DAT dds2 drive for $400. You need to look at the price of the tape media too. Those Tr3 drives are cheap but take $25.00 tapes. For most computer stuff I just search http://www.pricewatch.com/ It gives you selected item sorted by selling price. It is the quickest way to find the best price. It used to be that the catalog vendors where the best but in the last couple years they have become overpriced. My local retailer beats them now. The guys selling on the 'net now have the best prices. Just use VISA and ask for 2nd day delivery. As for DAT, 8MM or DLT. Only DAT is personably affordable. They start at about $350 (new). The DAT drives are called "dds1", dds2 or dds3. The tapes are notched to mark them by dds#. So when you pop a tape i the drive it "knows" which format to use. A dda3 drive will accept dds1 -> dds3 media but as you'd expect a dds1 drive can only accept dds1 tapes. So dds1 tapes are the exchange media. Not to bad 2GB (uncompressed) and $6.00. dds2 holds twice as much. Speed is 5MB/sec or 10MB/sec if you pay more for the drive. dds3 drives hold 12GB (uncompressed) and cost about $850 One thing about DAT: You really need to clean the heads frequently. After every 8 hrs use. It is a helical scan system like video. One DLT I looked at for work costs about $11K but writes at 600MB/min sustained rate and holds 15 DLT cartridges in an autoloader. It allows you to backup 1TB unattended. DLT is not intended for backing up PCs unless you have an office full of them. 8mm drives are older then DAT but are still sold. I have used regular camcorder tape bought at a music store. It is not supposed to work but does if you test the brand first. These held more and where faster then DAT but now with dds3 I think for new applications (you don't already have hundreds of old tapes) dat is the way to go. The new 8mm drives hold 50GB so I guess 8MM still wins but prices are around $2200.00 Traven is still the cheapest way to go but get SCSI, not the floppy or parallel interface types as they are slow and require lots of CPU overhead. I have not seen any good Windows backup programs. What I do is boot Linux then use "tar" to copy to Win95 partition to DAT tape. --Chris Albertson From: Tom Droege Subject: DVD-RAM To: tass@wwa.com One of my friends just gave me an add for DVD-RAM drives at $699. Should we be thinking about this for the Mark IVs? I was just about to buy another tape backup system and a CD-R machine. Might not this do both jobs? Sigh! I know it is too early to jump on this device. But there is never a right time to buy computer equipment. How much can you get on one of these drives? OK people, looks like we have not been ambitious enough. When we started we were worried that we were not going to be able to find enough storage for the raw Mark III data. Now it would look easy for the Mark IV. Looks like I will have to dream up a higher rate Mark V or we will fall behind the state of the art. The decision is pretty easy, I think. Always keep all the raw data. Tom Droege From SHOPPA@trailing-edge.com Fri Aug 28 12:09:18 1998 Received: from timvax.trailing-edge.com (trailing-edge.wdn.com [198.232.144.27]) by pluto.njcc.com (8.8.7/8.8.3) with SMTP id MAA17563 for ; Fri, 28 Aug 1998 12:09:07 -0400 (EDT) Date: Fri, 28 Aug 1998 12:05:54 -0400 From: "Timothy D. Shoppa" To: hjohnson@pluto.njcc.com Message-Id: <980828120554.25400131@timvax.trailing-edge.com> Subject: Re: 4 & 8mm tape drives Status: RO X-Status: N >Typical prices for 8mm in the few to 14GB range are about $200 to $700; >some more, some less. Of course the newest drives are four-digit prices. >I'd appreciate some sage advice, which I would pass along. The low end of the 8mm world is the Exabyte 8200. Not bad when it's working, but notoriously unreliable. The next step up is the 8500, which is a little better while working, but still quite unreliable. I wouldn't pay more than a few hundred for a 8200 or 8500, and even then only if it came with some sort of guarantee against it coming DOA. The real winners in the 8mm world are the 8505 and 8700. The 8505, last time I checked (about a year ago), was more than $1000 used. The 8700 is a more recent model, and gets higher densities, and is supposed to be more reliable, but I've no actual experience. In the 4mm world, most drives are winners. The HP 35450 and 35480 were the first drives, and mine still work like champs. These get 1.2Gbytes on a 60M DDS-1 cartridge. You shouldn't have to pay more than $200 to get a decent one. More modern 4mm drives take DDS-2 (90m) tapes and get 2 Gigs without compression, 4 Gigs assuming 2:1 compression. And then there are DDS-3 drives which will take 120M carts and can get up to 8 Gigabytes with compression. For these you'll probably pay between $300 and $1000. Does this help any? What sort of software were you planning to use to talk to these tape drives? For the traditional minicomputer OS's I've always used, it's easy to to write files to ANSI labeled tapes, but from what I've seen from the PC/Mac world there is no such universal file interchange formats - just a bunch of very incompatible backup packages. For rapid access to archived data, the real winner is a CD-ROM writer. As we discussed, writing data straight to CD-ROM is usually not an option, but if you can stage on a hard drive and later (during the day) do the CD-ROM writing it'll work fine. (The reason why you probably don't want to write straight to CD-ROM during aquisition has to do with the CD-ROM writer's demand for continuous data into it; if the computer is busy with something else and can't keep up the supply, the writer will (at best) begin writing "null" records eating up disk capacity, and at worst will render the CD-ROM's directory or data unreadable.) Tim. Date: Thu, 27 Aug 1998 11:55:07 -0500 From: Tom Droege Subject: DVD To: tass@wwa.com The below press release kindly sent to me by Tom Wesson. Note the January 1998 date. They are just showing up in the adds, so there must have been some sort of legal delay as suggested by Arne. Not yet right for us, I think. It would today be cheaper to writ on CD-R media. But it is a sign of what it coming. Tom Droege ------------------------------------------------------------------ Panasonic First To Deliver DVD-RAM Drives * Available in the U.S. and Japan in January '98* NEW YORK, NY (October 28, 1997) =97Matsushita Electric, best known for its Panasonic brand products, announced today that it will be the first company to deliver state-of-the-art DVD-RAM drives and discs to market. The Panasonic LF-D101 internal drive with SCSI interface =97 which features the ability to both read from and write to discs with a maximum, double-sided capacity of 5.2 gigabytes =97 will be available in the United States and Japan in January, 1998. The DVD-RAM drive will have a suggested retail price of $799 in the U.S. Panasonic DVD-RAM discs will be available at the same time as the LF-D101 drive. The enclosed cartridge, double-sided disc, with a capacity of 5.2 gigabytes, will have a suggested retail price of $39.95. The 2.6-gigabyte capacity, single-sided disc, which is designed to allow the media to be removed from its cartridge, will have a suggested retail price of $24.95. The cost per megabyte of data storage is less than one cent ($.01). By switching between 650 nanometer (nm) and 780 nm wavelength lasers and two objective lenses, the Panasonic DVD-RAM drive can write to and read from a wide range of optical disc formats. It can write to DVD-RAM and PD discs, and can read from all of the discs currently accepted by the following devices/players: DVD-RAM, DVD-ROM, DVD video, DVD-R, PD, CD-Audio, CD-ROM, CD-R, CD-RW and video CD. In addition, all new Panasonic DVD-ROM drives will have DVD-RAM read capability. The DVD-RAM system's versatile read-compatibility is made possible by the world's first DVD integrated optical pickup device, which was developed within Matsushita Electric, and a signal processing LSI that uses a 32-bit RISC microprocessor. In Japan, Matsushita has received 620 patents for its DVD-RAM drive. In the United States and other countries, 120 patents have either been granted to the company or are pending. DVD-RAM technology has been developed in response to the need for a larger capacity removable data storage device to accommodate the increased use of multimedia applications, including full motion video, color images and digital audio. In the future, this core rewritable technology will be applied to the development of DVD video recorders. Panasonic will soon make available a variety of application software, including file back-up, MPEG1 software encoding and MPEG1 file editing software. In the U.S., Panasonic Computer Peripheral Company will begin shipping Panasonic DVD-RAM drives in January 1998. A unit of Panasonic Communications & Systems Company, it is based in Secaucus, NJ. One of the leading suppliers of CD-ROM drives to U.S. personal computer manufacturers, Panasonic Industrial Company expects to provide DVD-RAM drives to OEM customers in the computer industry following shipment of Panasonic brand drives in January. Matsushita Electric Industrial Co., Ltd. (NYSE: MC) is one of the world's largest manufacturers of CD-ROM drives and other digital electronics products. Worldwide, Matsushita has more than 270,000 employees and recorded sales of $61.9 billion during the past fiscal year. The company's principal North American subsidiary, Matsushita Electric Corp. of America, markets products under the Panasonic, Quasar and Technics brand names. Matsushita Electric subsidiaries and affiliates in North America employ over 20,000 people in the U.S., Mexico, Canada and Puerto Rico.=20 From hjohnson Thu Aug 27 17:08:40 1998 Received: (from hjohnson@localhost) by pluto.njcc.com (8.8.7/8.8.3) id RAA14856; Thu, 27 Aug 1998 17:08:37 -0400 (EDT) To: tass@wwa.com From: hjohnson@pluto.njcc.com (Herbert R Johnson) Subject: Re: dvd Date: Thu, 27 Aug 1998 15:30:23 -0400 On Thu, 27 Aug 1998 00:49:20 -0700, aah@nofs.navy.mil wrote: *>Thanks, Chris, for straightening me out. Last I heard (which was *>about 6mos ago), DVD recorders were not available, due in part to *>legal problems. I guess those have been resolved? And the format *>is ISO standard? We wouldn't get into an X2 vs. Kflex vs. V.90 *>situation, right? As I read the electronics trade literature, there is still some contention about DVD standards. There has been a series of battles over format and content, particularly on the audio side (video and music, you can imagine the issues). There is even some contention about a standard developed strictly for CHINA! [this turns out to be a video disk standard issue. - herb Jan 1999] Of course, all this is *new technology*, i.e. you are buying the right to be a beta tester. Finally, like hard drive technology the densities of the "next generation" DVD's will be greater than the previous. Keep in mind this technology is driven by *consumer* interests: that means industrial "warfare" as well as ultimate low cost. Chris and some people in the minicomputer/mainframe trades have talked to me about 8mm and 4mm formats. These are much more mature and standardized. While older drives are available cheaply, they suffer from the problems of any early-generation storage design (as well as age) and their low price reflects that. Something "not too old" and "moderately priced" may do it. It would seem the situation is reliable high-density affordable ...choose any TWO. There may be some middle ground but I would get *very* informed advice first, probably from the mini\mainframe\network folks that deal with this daily. I wish I had time.... Herb Johnson Date: Fri, 28 Aug 1998 13:49:46 -0400 From: "Timothy D. Shoppa" To: hjohnson@pluto.njcc.com Subject: Re: 4 & 8mm tape drives >CD-ROM will work for the several Mark III's in the field, but the Mark >IV will need more storage, particularly if we save raw data. It >will collect in one night's run 1.6GB - that would be three CD-ROMS. Or one 4mm or one 8mm tape - I get the idea :-). >An accumulation of 100 runs over a few years is not out of hand, so >the archiving problem is obvious. Lossy compression would reduced >the data to 1/5 to 1/10th of that size, but there is concern about >loss of information. Lossless compression is likely to only reduce >data to 2/3 or 1/2 that size. Good - you're thinking about compression :-). But you should also be thinking about interchange issues. Without a common tape format, interchange is going to be difficult! I would go for something like Unix "tar" (tape archive) format, simply because I know how to read it on all my Unix machines and PDP-11/VAX/ Alpha's. I have no idea what support there is for tar format in Macintosh/DOS/Windows world. >Would you mind if I posted your comments in our TASS mail list? Go right ahead. The group leader - if he's at Fermilab - has ready access to much more than my anecdotes about tape drive reliability. When I was at Fermilab, we were writing three 8mm tapes every hour, 24 hours a day, for a whole summer! > Of course the TASS members would have >to have some consensus as to what size and model and manufacturer >of drives to get. I would guess that "several GB" would be adequate >capacity, notwithstanding price of drive and media. I would think >the 8mm drives would be a good match for this kind of application, >with our need for "write once, read a few times, ship them around" >and cheap media (new 8mm video tapes). Actually, video grade 8mm tapes aren't good things to run through data drives. In my experience, the top of the line Sony 8mm video tapes (i.e. not data tapes) are workable, but they also shed a good amount of junk onto the tape heads. If the data means anything to you, I'd strongly advise you to think strictly in terms of data-grade only tapes. I do realize that with this much data, cost of the media does become an important issue. If I were you I'd sit down and carefully compare CD-writables with 8mm and 4mm. >I won't proceed further until I hear from you. Also, any thoughts >on that list of vendors I sent? Most of them I've heard of before, but I've never purchased used Exabytes from any of them. Tim. From: "Me" To: Subject: Re: archival storage Date: Sun, 30 Aug 1998 00:47:20 +1000 Sony makes a 4mm DAT - Digital Audio Tape - that is commonly used to back up servers. It is compact, relatively cheap and reasonably easy. It is mainly used in rotational backups so I don't think it is reliable for long term storage - being magnetic. I consider CD-R to be the most reliable, the most expedient and the cheapest method available now. Joseph Armstrong AUSTRALIA Date: Mon, 31 Aug 1998 09:42:26 +0200 (MET DST) From: BARTHOLDI Paul Subject: Re: archival storage To: hjohnson@pluto.njcc.com, tass@wwa.com Dear Herb, We have some experience with DAT (both DDS1, DDS2 and DDS3), none directely with 8mm (Exabyte). During the last 10 years, we had 8 or 9 DAT units, and used a few hundred (thousond ?) tapes. The result is mitigate. - The first units (WANGDAT) where uncompatible between themself and with other types. - The more recent have been HP DDS2 (see bellow for DDS3). They are very nice, you can put a lot of data on a very small volume, but the units are VERY FRAGILE. They were used by many students and professionals, not necessarely very carefull in their manual work. Any how after some time (one or two years), the macnanics seems to wrap and old tapes become unreadable. New one can be written but get unreadable on other units etc. - The DDS3 seems more robust and we had no problem (yet) with them. The DDS3 unit is able to read DDS2 tapes without problem. - The feedback I have from colleagues who use Exabyte units is similar. They had more compatibility problems (more different formats available ...) and the units seems also quite fragile. That is why we sticked to DAT. - One remark that may be irrelevant for you, but that caused many problems to us. All these tapes are not only erasable, but also overwritable. If you are not very cautious, you can very easely erase all your tabpe in wrting the last file just after a voluntary or unvoluntary rewind ... We now have three years experience with DLT and a few month experience with CD-R. The DLT have been very reliable, had NO problem rereading any data and we write every night between 20 and 50 GB of backup. Only once a bellt got out of wheel, but it was repaired by us in a few minutes. We have not tried robots, neither with DAT or DLT. Looking at the price and complexity of the mecanics, I believed that it was safer to get many single units than a multitape robot. If my observation that the users were responsible for the malfunctioning of the DAT, then I was wrong! Last, considering CD-R, our experience is too short, but the nicest thing about them is that in single sessesion, you cannot overwrite anything any time. When the data are written, the CD-R becomes a CDROM! The main drawback is the size, only 650 MB per disk, and the throughput (600 KBps at 4x). But when you compare the cost per MB, it becomes very interesting: It is much cheaper than DLT and just twice as expensive as DAT. I guess it is cheaper than 8mm. All our units have been and are connected to Unix machines (usually Sun Sparc Stations with Solaris, sometimes with PC and Linux). Finaly, if you have to send data, the CD-R are much less sensitive to heat or magnetic fields. Your data wont be erased so easely! I hope this note can be of some use for your choice, Best regards and good luck! Paul ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Paul Bartholdi Paul.Bartholdi@obs.unige.ch + + Observatoire de Geneve tel +41 22 755 2611 + + CH-1290 Sauverny fax +41 22 755 3983 + + Switzerland www http://obswww.unige.ch/ + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Date: Mon, 31 Aug 1998 16:23:52 +0000 From: Chris Albertson To: BARTHOLDI Paul CC: hjohnson@pluto.njcc.com, tass@wwa.com Subject: Re: archival storage I basically agree with Paul. In a heavy use environment you need to keep up on the maintenance of tape drives. With 8 or 9 drives you should assign one person to the task. For home use, the drives should last forever. BARTHOLDI Paul wrote: > > Dear Herb, > > We have some experience with DAT (both DDS1, DDS2 and DDS3), none directely > with 8mm (Exabyte). During the last 10 years, we had 8 or 9 DAT units, and > used a few hundred (thousond ?) tapes. The result is mitigate. > > - The first units (WANGDAT) where uncompatible between themself and with > other types. Yes, Standarization occured _after_ Wangdat entered the market. THey were first. Now, HP sells the the data encoding chips to everyone else. So DDSx is always really an HP chip doiing the work. > > - The more recent have been HP DDS2 (see bellow for DDS3). They are very nice, > you can put a lot of data on a very small volume, but the units are VERY > FRAGILE. They were used by many students and professionals, not necessarely > very carefull in their manual work. Any how after some time (one or two > years), the macnanics seems to wrap and old tapes become unreadable. New one > can be written but get unreadable on other units etc. What I find is you _must_ use the cleanning cartridge with DATs about every eight hours of operation. Also the tapes can become worn. This is the same as video. DAT uses a helical scan head. The heads also are only good for a few thousand hours. This could explain your problem over time. > > - The DDS3 seems more robust and we had no problem (yet) with them. The DDS3 > unit is able to read DDS2 tapes without problem. 8mm uses helical scan heads also. but the tape has more surface area so the newest 8mm holds more then the newest 4mm tape. We had the head replaced on an old 8mm drive now it wroks fine. A "few thousand" hours is a _long_ time for if used only for person use. Under industrial use expect a four year lifetime before. We use ours about 6 to 8 hours every night. > > - The feedback I have from colleagues who use Exabyte units is similar. They > had more compatibility problems (more different formats available ...) > and the units seems also quite fragile. That is why we sticked to DAT. > > - One remark that may be irrelevant for you, but that caused many problems > to us. All these tapes are not only erasable, but also overwritable. If you > are not very cautious, you can very easely erase all your tabpe in wrting > the last file just after a voluntary or unvoluntary rewind ... Yes, Use a script to do the writting and flip the write protect after you are done. Also most profesional advice you will read will tell you that for your data to be safe it needs to exist on three physical media and at two physical locatins. ANy less then this and over time you will lose something. > > We now have three years experience with DLT and a few month experience with > CDR. The DLT have been very reliable, had NO problem rereading any data > and we write every night between 20 and 50 GB of backup. Only once a bellt > got out of wheel, but it was repaired by us in a few minutes. > > We have not tried robots, neither with DAT or DLT. Looking at the price and > complexity of the mecanics, I believed that it was safer to get many single > units than a multitape robot. If my observation that the users were > responsible for the malfunctioning of the DAT, then I was wrong! We have a 12 slot DAT robotic auto loader. It's problem is the same as any other DAT drive. However that one drive sees a lot of use so it needs cleaning every two or three days. We now keep a cleaning cartridge in the 12th slot and have the script clean the drive automaticaly With DAT the cleaning cycle is automatic. pop in the cartridge it will do it's thing then eject. I do this now after writing four 120 meter tapes. End of problems. I do thing DLT is the way to go if you can justify the cost. The units I've seen are not personably afordable and I don't expect the price will every get that low. > > Last, considering CDR, our experience is too short, but the nicest thing about > them is that in single sessesion, you cannot overwrite anything any time. > When the data are written, the CDR becomes a CDROM! The main drawback is the > size, only 650 MB per disk, and the throughput (600 KBps at 4x). But when you > compare the cost per MB, it becomes very interesting: It is much cheaper > than DLT and just twice as expensive as DAT. I guess it is cheaper than 8mm. > All our units have been and are connected to Unix machines (usually Sun Sparc > Stations with Solaris, sometimes with PC and Linux). Finaly, if you have to > send data, the CDR are much less sensitive to heat or magnetic fields. Your > data wont be erased so easely! I agree about CD-R. I'd like to find an autoloader. I saw one once that held about 100 blank media. Seems that this would be perfect. > > I hope this note can be of some use for your choice, > > Best regards and good luck! > Paul > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > + Paul Bartholdi Paul.Bartholdi@obs.unige.ch + > + Observatoire de Geneve tel +41 22 755 2611 + > + CH-1290 Sauverny fax +41 22 755 3983 + > + Switzerland www http://obswww.unige.ch/ + > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ --Chris Albertson To: tass@wwa.com From: hjohnson@pluto.njcc.com (Herbert R Johnson) Subject: tape backup vs. CD-R Date: Tue, 01 Sep 1998 10:33:26 -0400 [** I had an on-line discussion with one of my colleagues in the DEC equipment world, Tim Shoppa. Tim worked until recently at TRIUMF, Canada's National Meson Facility, supporting their PDP-11 technology. He now consults on these systems for Y2K adaptions and other developement or upgrade work, and he is active in those corresponding Internet newsgroups. My comments and questions are in **'s. In summary, Tim says some of the later 8mm and 4mm drives are reliable, but watch out for the oldest ones! And CD-R is still a good choice given its stability and reliability, despite its limitations in capacity and speed. - Herb Johnson **] **A group of amateur astronmers are collecting a fair amount of data **each night, and expect to collect much more with some new instruments. **YOu mentioned some good and bad choices among 8mm tape drives: could **you review that again? I suspect a capacity of several gigs is OK, **and the ability to use plain 8mm videotapes would be a plus. This **would be archival: write once or a few times, read occasionally. The low end of the 8mm world is the Exabyte 8200. Not bad when it's working, but notoriously unreliable. The next step up is the 8500, which is a little better while working, but still quite unreliable. I wouldn't pay more than a few hundred for a 8200 or 8500, and even then only if it came with some sort of guarantee against it coming DOA. [** prices I've found in the used equipment printed publications are listed below. They will vary widely, often by 50% to 100%. REPAIR prices are typically 2/3rds of selling price. 2GB 8200 - $225 - $300 4GB 8500 - $450 8GB 8500C - $899 - $1195 **] The real winners in the 8mm world are the 8505 and 8700. The 8505, last time I checked (about a year ago), was more than $1000 used. [** prices in the publications are: 10GB 8505 - $900 -$1295 14GB 8700 - only one price listed, $655 **] The 8700 is a more recent model, and gets higher densities, and is supposed to be more reliable, but I've no actual experience. In the 4mm world, most drives are winners. The HP 35450 and 35480 were the first drives, and mine still work like champs. These get 1.2Gbytes on a 60M DDS-1 cartridge. You shouldn't have to pay more than $200 to get a decent one. [** prices in the publications are: HP35470A $265 - $495 HP35480 $295 - $595 other 2GB drives about $200-$400, 4GB about $350-$500 **] More modern 4mm drives take DDS-2 (90m) tapes and get 2 Gigs without compression, 4 Gigs assuming 2:1 compression. And then there are DDS-3 drives which will take 120M carts and can get up to 8 Gigabytes with compression. For these you'll probably pay between $300 and $1000. [** prices in the publications are: HP C1534A $265 DAT-DS HP C1536 $599 ,media $10 DDS-2 DAT HP C1599 $729 ,media $20 DDS-3 DAT Sony 7000 - $639 and $525 **] Does this help any? What sort of software were you planning to use to talk to these tape drives? For the traditional minicomputer OS's I've always used, it's easy to to write files to ANSI labeled tapes, but from what I've seen from the PC/Mac world there is no such universal file interchange formats - just a bunch of very incompatible backup packages. For rapid access to archived data, the real winner is a CD-ROM writer. As we discussed, writing data straight to CD-ROM is usually not an option, but if you can stage on a hard drive and later (during the day) do the CD-ROM writing it'll work fine. (The reason why you probably don't want to write straight to CD-ROM during aquisition has to do with the CD-ROM writer's demand for continuous data into it; if the computer is busy with something else and can't keep up the supply, the writer will (at best) begin writing "null" records eating up disk capacity, and at worst will render the CD-ROM's directory or data unreadable.) **I get "THe Processor" every month, an ad sheet for computer resellers. **Here are some tape drive repair/sales depots: any recommendations **pro or con? ** ** www.megahaus.com (MegaHaus) ** www.pacificdata.com (Pacific Data) ** www.compdisk.com (Computer Disk Service) ** www.exabyte.com ** www.i-t-s.com (Independent Technology Service Inc) ** www.gdcinc.com (Global Data Center) ** www.ezsystems.com (E Z Systems) ** www.mwtss.com (Midwest TEch Sales Inc.) Most of [these] I've heard of before, but I've never purchased used Exabytes from any of them. [** Tim previously told me he often buys these items on-line: I find prices over the Internet are often much cheaper than in the printed publications. **] **CD-ROM will work for the several Mark III's in the field, but the Mark **IV will need more storage, particularly if we save raw data. It **will collect in one night's run 1.6GB - that would be three CD-ROMS. Or one 4mm or one 8mm tape - I get the idea :-). **An accumulation of 100 runs over a few years is not out of hand, so **the archiving problem is obvious. Lossy compression would reduced **the data to 1/5 to 1/10th of that size, but there is concern about **loss of information. Lossless compression is likely to only reduce **data to 2/3 or 1/2 that size. Good - you're thinking about compression :-). But you should also be thinking about interchange issues. Without a common tape format, interchange is going to be difficult! I would go for something like Unix "tar" (tape archive) format, simply because I know how to read it on all my Unix machines and PDP-11/VAX/ Alpha's. I have no idea what support there is for tar format in Macintosh/DOS/Windows world. ** Of course the TASS members would have **to have some consensus as to what size and model and manufacturer **of drives to get. I would guess that "several GB" would be adequate **capacity, notwithstanding price of drive and media. I would think **the 8mm drives would be a good match for this kind of application, **with our need for "write once, read a few times, ship them around" **and cheap media (new 8mm video tapes). Actually, video grade 8mm tapes aren't good things to run through data drives. In my experience, the top of the line Sony 8mm video tapes (i.e. not data tapes) are workable, but they also shed a good amount of junk onto the tape heads. If the data means anything to you, I'd strongly advise you to think strictly in terms of data-grade only tapes. I do realize that with this much data, cost of the media does become an important issue. If I were you I'd sit down and carefully compare CD-writables with 8mm and 4mm. [**of course, that is what I am trying to do - Herb] To: tass@wwa.com From: hjohnson@pluto.njcc.com (Herbert R Johnson) Subject: archival storage, prices Date: Tue, 01 Sep 1998 14:10:35 -0400 Some price points in the PC world: CD-RW drives seem to be selling in the mailorder catalogs for about $300 to $350 for IDE versions. They typically read CD's at 6X speed, write CD-R at 2X speed and rewrite CD-RW at 2X speed. They include "packet writing" software for Win 95/NT: packets allow you to write incrementally during your writing session and avoid "spoiled" disks if your CPU stalls briefly. (Avoid older drives that do not support this method.) External versions add about $100-$150 to the price. SCSI versions are available for a little more or a lot more, depending. And the newest drives are much faster on read and slightly faster on write: but a new X24 or faster CD-ROM (read only) drive is about $30-$60. DVD-writable drives seem to be rare. If one waits a year or so, they may not be. But as you can ALWAYS wait a year and get something "much" better, it's generally the old problem of deciding when you are going to "waste your money" as all computer equipment "eventually" becomes obselete. But history suggests that it is expensive to be first, and DVD is still a "firsts" technology. One way to use an expensive archiving system is to network other systems to it. 10-base T Ethernet "kits" (for 10 MHz Ethernet) with two interface cards and a 10-base "hub" that supports 4 devices cost about $100 or more. Cards may be ISA or PCI, depending. Single cards are about $25 and up, in part depending on speed. Some of these kits use 10/100 interface cards, you just need a new "hub" to upgrade. Hubs are about $50 a pop, more for the 100 Mhz version or dual 10/100 hubs (either or, I don't think the cheaper ones do both). PC magazine suggests the effective data rates for a 10baseT network like these kits is about 4 Mbit/sec. A recent September issue surveys several such products. Win 95 supports these right off; some have Linux support included, some suggest Windows for Workgroups (upgraded Win 3.1) support is available. Don't know about MS-DOS but there have been some commercial file sharing network products in the DOS world. Another way to "network" is to use a removable hard drive. THe catalogs sell "kits" to turn your hard drive into a "portable" drive - actually more like transportable. Looks like about $30 a station and up will buy a docking station and a carrier for your small IDE drive. External SCSI kits cost about $75 and up, and external IDE kits about the same. These allow you to use an external box and power supply, and then you cable the drive to the system of your choice and run it. Many SCSI controllers already support external use, you just need a box with power and the right cables inside and out. (If anyone wants external SCSI boxes, I sell these for about $20-$30 plus shipping.) There are also parallel port interface kits for drives and even CD-ROM's: but I hate to think about software problems and speeds with these interfaces, as well as the added cost as noted above. One could do worse than to spend about $100 or less to take a spare 1 or 2 Gbyte drive and make it "removable" between a data collection system and an archiving system: Another $100 or less will buy a used drive also. Or, $100 or more will connect the two via 10base Ethernet; but then you have to deal with networking possibly disparate and/or old computer systems. Or, get an "adequate" CD-RW drive for $300 for each system, and "someday" a massive archiving device on one system as you accumulate CD-R disks at about $1 a pop. Herb Johnson Date: Wed, 2 Sep 1998 22:35:39 +0100 (GMT) From: Peter T Mount To: Herbert R Johnson cc: tass@wwa.com Subject: Re: archival storage, prices On Tue, 1 Sep 1998, Herbert R Johnson wrote: > Some price points in the PC world: > I've been using CD-R here for a year now. Media cost has dropped enormously (from 5 pounds per disk to just 1). Recording disks using linux has proved extremely reliable (only 4 failues during this year, all were due to dodgy media). Now in my PC at work, I have a CD-RW. That machine is running Win-NT 4 (although I prefer it's nickname of satanix ;-) ), and even though it's a more powerful machine (a PII-400 against a P-133 at home), the recording doesn't seem to be as reliable (2 failures in 2 months, and both caused by system load rather than media). > One way to use an expensive archiving system is to network other systems to > it. Over here, I've seen some kits where 2 cards + hub is selling for the same price as an identical hub. I've managed so far to get Win95, NT and Linux to file share quite easily (although Win311 had some problems with Linux). I'm not sure what's available for Dos (rarely use it now). > One could do worse than to spend about $100 or less to take a spare > 1 or 2 Gbyte drive and make it "removable"... Personally I'd use the networking option, as it tends to be the cheaper alternative in the long run, and as soon as you have it, you'll find other uses for the network. -- Peter T Mount peter@retep.org.uk From: Peter T Mount To: Herbert R Johnson cc: tass@wwa.com Subject: Re: archival storage, prices X-Status: N On Fri, 4 Sep 1998, Herbert R Johnson wrote: > I would request to anyone that is posting comments about their particular > archiving equipment to PLEASE name the brand and model of their devices!! The CD-R I'm using here is a Phillips CDD-2600, and I use mkisofs and cdrecord 1.6 running under Linux. The media I had problems with here was BASF Prod Nr 342 069 The best media I've come across is the TDK Reflex range. I tend to use these for the more important archiving (rather than the day to day stuff). Currently I have some budget unbranded disks, which haven't (yet) given me trouble. Here's what cdrecord says about a blank disk: Disk type: Cyanine, AZO or similar Manufacturer: CMC Magnetics Corporation The CD-RW at work is a Nomai (Not sure of the model number). I'll have to get the software details when I go in to work on Monday. That drive is installed in an NT 4 Workstation. We use Maxel media there. > *>Personally I'd use the networking option [for sharing], as it tends to be If the network is a busy one, then it can be a problem. However, on a network with only two or three machines, then the load is usually negligable. -- Peter T Mount peter@retep.org.uk To: tass@wwa.com Subject: Re: tape vs CD-ROM From: hjohnson@pluto.njcc.com (Herbert R Johnson) Date: Fri, 04 Sep 1998 22:45:56 -0400 More conversations with Tim Shoppa - Herb (my comments are *>'ed). *>Where are you getting your prices - I presume on-line? - Herb A mix of that and talking with actual dealers. Most of my prices are probably 6 months to a year out of date, but it looks like your research[ed prices] shows that they aren't too far out of line with present-day reality. You might also check out "misc.forsale.computers.storage" and the online auction houses like ebay.com and online.com. Again, if reliability is at all an issue, I recommend starting with a low-end DAT or a CD-ROM recorder. Pay attention to media prices too, especially if you're talking about every-day use. Finally, you've got to think about interchangability - I know that you'd be able to write easily interchange CD-ROM's, but I don't know much about interchange formats between tapes on Mac's and PC's that would be useful to you here. Most tape backup packages for micros that I've seen are completely incompatible with all other [Unix and minicomputer] tape backup packages. ---end quote--- Chris, can you speak to the transportablity of "tar" format from Linux to MS-DOS/Windows? I suspect Linux sites provide some MS-DOS utilities. Or, as I think you said, can you "mount" an MS-DOS hardrive partition under Linux and migrate tapes to and from that partition? Herb To: BARTHOLDI Paul From: hjohnson@pluto.njcc.com (Herbert R Johnson) Subject: Re: archival storage Date: Mon, 07 Sep 1998 19:26:04 -0400 Thanks for the reply. I've added a few comments. If you don't object, could you simply post this reply in the TASS mail list? It answers the questions I asked. - Herb Johnson On Mon, 07 Sep 1998 14:21:13 +0200 (MET DST), BARTHOLDI Paul wrote: *>Dear Herbert, *> *>I am not shure you got my answer, most probably it was lost here... *> *>>> DAT units, and used a few hundred (thousond ?) tapes. The result is *>>> mitigate. *>> *>> I'm not sure what you mean by "mitigate". *> *>Sorry, my knowloge in english are VERY poor. I mean 'mixte' I think you mean "mixed", as in both good and bad together. *>> It would be helpful if you could name model numbers as well as the *>> manufacturers, as I'll describe below. *> *>The WANGDAT is too old to be remembered. We had in fact 4 HP 1533A (DDS2) *>and now one HP 1537A (DDS3). We might get another 1537A soon. *> *>>>- The DDS3 seems more robust and we had no problem (yet) with them. The *>>> DDS3 unit is able to read DDS2 tapes without problem. *> *>The HP 1537A os now more than one year old, is used about twice a day *>by a very small group of 'educated' users. *> *>For the bestm they have been able to read old tapes that were unreadable *>on the 1533A on which they had been written. *> *>The old HP 1533A are still in use 'officialy', but very rarely in fcats. *> *>I can't report [Exabyte models] as we never had Exabyte units ourself, and *>users usually dont know what they are using. It is true that most of my *>informations now on Exabytes are few years old. *> *>>>- One remark that may be irrelevant for you, but that caused many *>>> problems to us. All these tapes are not only erasable, but also *>>> overwritable. If you are not very cautious, you can very easely erase *>>> all your tabpe in wrting the last file just after a voluntary or *>>> unvoluntary rewind ... *>> *>> THis is a good point. I suspect TASS use would be for writing only once or *>> twice to fill the tape, so we might not have this problem. Our use would *>> not be incremental but a "full" backup for archiving. *> *>The nice thing with CDROM is that they are by construction write once only. *>You ca not overwrite by error. But I agree, in your case the probablilty *>is rather small. *> *>>> Last, considering CDR, our experience is too short, but the nicest thing *>>> about them is that in single sessesion, you cannot overwrite anything *>>> any time. When the data are written, the CDR becomes a CDROM! The main *>>> drawback is the size, only 650 MB per disk, and the throughput (600 KBps *>>> at 4x). But when you ... *>> *>> I agree CD-ROM is a strong candidate, that is why we are looking at tape *>> to see if the benefits of tape (larger capacity) outweigh the disadvantages *>> of tape (fragile, expensive, etc.) Thanks for your information! I look *>> forward to your reply. *> *>Sorry for the delay, *> Paul *> *>PS Just a question: CD are usually in breakable plastic boxes. I have seen *>s few cardboard boxes with the same central fixation. Do you have any idea *>where these could be available? Cardboard folders to hold a CD-ROM are probably available from shipping supply stores, or from shipping companies. COsts are probably a few to several for a US dollar. I'm not sure I'd use them for storage, the plastic boxes hold the CD-ROM from the center and so they are less likely to warp. Herb Johnson Date: Tue, 08 Sep 1998 13:05:31 -0700 From: Chris Albertson To: hjohnson@pluto.njcc.com Cc: tass@wwa.com Subject: Re: tape vs CD-ROM Herbert R Johnson wrote: > First off I'll second Tim Shoppa's opinions. One thing about DAT that I don't see in other formats is that the tape cartridge itself is notched. So you never need to tell the drive what format to use or how long the tape is. Also in DAT, compression is done by the drive, not the software. HP sells the chips that define the way this works so the tapes _do_ interchange. UNIX and Linux see the tape drive as just a liner string of bytes. The tape looks a lot like a modem or for that mater a keyboard. Each just is a string of bytes. If I run the "tar" program and send the output to a tape device the OS simply pipes tar's byte stream to the tape. One could run tar under DOS but DOS does not support tape drives so you can only read/write tar files to/from disk. Same story with MS Windows. But DOS tar and UNIX tar use the same format. If you had a PC setup to run both Linux and some other OS, say Win98, OS/2 or DOS then while running Linux, reading tar format DAT tapes would be simple. Linux can also access the other OS's disk files. One way to read a UNIX tar format DAT tape would be to boot the PC up in Linux and unload the tape to a directory (/dos/C/foobar) that the other OS can "see". Then reboot to DOS and the data is in C:\foobar If you don't want Linux on your PC you can boot and run Linux entirely off a floppy disk. > Chris, can you speak to the transportablity of "tar" format from Linux > to MS-DOS/Windows? I suspect Linux sites provide some MS-DOS utilities. > Or, as I think you said, can you "mount" an MS-DOS hardrive partition > under Linux and migrate tapes to and from that partition? Did that answer the question? Maybe all you needed to know is that /dos/C/foobar = C:\foobar -- --Chris Albertson Date: Fri, 18 Sep 1998 14:47:00 -0400 (EDT) From: Herb Johnson To: Tom Droege cc: Herb Johnson Subject: Re: CD Writer My wife just bought an HP CD-ROM writer last week, and it works well under Win 95 (no DOS or Win 3.1 drivers, but most of them won't have them anyway). It's a model HP 7200e, an external parallel port X2 X2 X6 device. That's times 2 for CD-R and CD-RW, and times 6 for CD-ROM. The parallel port is no speed penality, consider the "times 2" speed. It got good reviews in PC Magazine, and we bought it at Best Buy for about $400. The internal version is cheaper by over $100 but we can share this device on a number of systems. HP has Linux/Unix drivers too I believe, and it comes with a selection of software. I'll report more as time permits, I just got back from vacation and I have many events in the next week. HErb JOhnson On Thu, 17 Sep 1998, Tom Droege wrote: > OK, I am starting to reprocess. While I am at it I might as well write the > raw files out to CDR. So I have to buy one. Chris has told me about > PriceWatch, and I can find lots (I think) of good deals. What should I buy? > In particular, once I buy it what software do I use that is good and > convenient under Windows 95? > > New 442As should get here early next week. I am optimistic that I will soon > be looking at pictures. Once that happens, I will have a busy winter > writing all the software that will make the prototype work. I will try to > pass some of it out, so you get familiar with the problems, but mostly I > have to do it myself. > > Tom Droege > > Date: Tue, 27 Oct 1998 11:02:43 -0800 From: Chris Albertson To: "Edward H. Currie" Cc: tass@wwa.com Subject: Re: Compression Edward H. Currie wrote: > > No one should ever store anything they care about on tape ... if you doubt > this ask Ampex ... I just had a 4mm tape drive fail a couple weeks ago. It munched a tape just like VCRs sometimes do. If I had been reading my only copy of a backup tape I'd have lost a few GB of data. The basic fact is "Nothing is secure". Hard disks can be counted on to fail about every five years on average. I personnally have seen examples five years on either side of this average. CD-R can be destroyed by heat or bright light Coatings can come off tape if not stored correctly and defective tape drives can destroy tapes A fire can take out all the data in an entire building. The most common cause of data loss however, is acidental overwrite. One accepted rule is that data becomes secure if it is writen on three different media at two different locations. So one copy on a hard drive and two tape backups, one kept at the office the other at home is safe. Three tape backups is safe also as long as you don't keep all three copies in the same desk drawer. There are more guidlines you can read about but yes, one tape backup copy is asking for trouble. --------------------------------------