Re: Questions about placing TASS images into on-line database

[Tass Mailing List <tass@mail.alembic.net>]

Date: Tue, 15 Jan 2008 23:22:39 -0800 (PST)
From: Chris Albertson <chrisalbertson90278@yahoo.com>
To: Tass Mailing List <tass@mail.alembic.net>
Subject: Re: Questions about placing TASS images into on-line database


>
>       - what sort of hardware would you recommend for the job?

Michael,

I doubt this will be a high volume web site.  nothing like say
100 hits per second.  This is not Amazon.com so you do not need
such a powerfull computer.  a previous generation processor should
do fine.  The bottle neck will be your Internet bandwidth.

However reliability likely maters. buy as much of that as you
can afford.  While an old surplus PC might work, they tend to
break.  But do you have budget for a machine with dual power
supplies and ECC RAM?

>       - should I simple purchase a unit which can store all the
>              information (like an NAS), or try to build one
>              by purchasing separately drives, an enclosure,
>              and perhaps a controller card?

Those low priced NAS boxs have very low end processors
in them and a DBMS system will not work well.  But if by
"NAS" you mean one of the higher end devices by a company
like NetApp they are great but at 10X the price.

Buy a PC case with room for many drives.  This eliminates
cables outside the box.  I have an Antec "sonata" case that is
first rate quality and has room for six drives each in it's
own mounting try with spaces
between for airflow and 120mm fans that move air quietly.
here is even a dust filter on the air intake.

Newer systems are nicer.  SATA drives can be plugged
directly into a backplane with no cables but watch
how close the drives are stacked.  You want (forced)
airflow between the drives.  Most of the new, low cost
SATA backplanes pack the drives to tight for good cooling
A good airflow path without any bends or constrictions
will make the drive live longer.

>
>       - hardware or software RAID?  I'm leaning towards software,
>              but seek the voices of experience

There is no "hardware raid" it is only a mater of where the software
runs.  Some RAID is done on a powerfull CPU inside the raid box.
some runs on an under powered slow processor inside the raid box.
Some 'hardware raid cards" actually only have BIOS code in a ROM
that runs on your main CPU.  Most cheap cards work this way.

You don't need  high performance.  Software raid will work fine
and will be much faster then your Internet connection.

Take a look at Solaris and ZFS.  Sun got it right. They
have compressed volume managment, raid and the
flesystem layers into one layer so the data no longer
flows through a "stack" and is faster and more flexible.
ZFS is being
ported to both BSD UNIX and Mac OS X but it is mature on
Solaris and Solaris is now free and Open Source.
http://www.sun.com/software/solaris/ds/zfs.jsp


>       - any suggestions for the software to connect a database
>              to a web browser?  Note that this database
>              will be read-only

Pick the language that you know best.  Say it's Perl.  Perl has a nice
perl modual for accessing a DBMS in an neutral way such that you
can pick MySQL or PostgreSQL later.  PHP has the same so
does Java.  Just use forms based html inside a cgi script.
it's low tech, I know.   Go with the language you know.

>
>    Suggestions?  Comments?  Guesses at an overall cost?  I welcome
> all input.  If we start to go into really gory details, it might
> make sense to take the discussion off the E-mail list and conduct
> it via private messages, but let's see if that becomes necessary.

It is good to keep this on the list so we can see other people's
good ideas.

If you have 3TB of data you can store in on four 1TB disks in
a RAID5.  You might buy five disks and tell your raid software
(or ZFS) that one drive is a hot spare

You will pay less for disks if you go with 500GB drives
RAID "pays off" better as the number of drive increases and
the hot spare is more affordable but then you will need
a big case for all those drives and likely today have to
go with ESATA.  and multiple esata cards in the PC

My gues is that the simpler design, 4 or 5 1TB drives in a
PC case will be best but I think you are on
the fence here.

That said the most imporent design is the layout of the
data tables.  I assume the FITS image files will be
so called "blobs" but you will have data and I hope
it will all be Nth order normal form.


>
>                          Michael
>



Chris Albertson
   Home:   310-376-1029  chrisalbertson90278@yahoo.com
   Office: 310-336-5189  Christopher.J.Albertson@aero.org
   KG6OMK


       ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page.
http://www.yahoo.com/r/hs