>> Not necessarily. It depends a lot on the configuration of the locally
>> attached drives vs. SAN logical disks, and the types of IO requests that
>> are made (e.g. long sequential reads or writes vs. lots of short random
>> file accesses). SANs have all kinds of caching mechanisms that can make
>> the IO actually faster than a SATA connection.
>
> SATA 3.0 connection: 6.0 GB/sec
>
> Fast Ethernet connection: 0.1 GB/sec
>
> That's, what, 6000x slower?

No, that's 60x times slower.

Besides, SAN connections are NOT fast ethernet.  Depending on the length 
and type of fibre, the speed will vary from 100 MBps to 1.6 GBps, which 
is still admittedly slower than a SATA 3 controller, but unless you are 
copying one large sequential file, you will also need to worry about 
drive seek time and file system layout to get acutal throughput 
comparisons, and Google says it's a wash.

>
> Sure, you can add caching, which might speed up write access. But it
> can't speed up read access that misses the cache.
>
>>> * Radically increased complexity compared to a direct connection.
>>
>> That depends on the point of view. From the server administrator's point
>> of view, it's much simpler. She no longer has to worry about spreading
>> her disks across multiple controllers, mirroring, etc.... on which
>> partition should the indexes go? What level of RAID to use? Is there
>> enough power in the server's power supplies to drive an additional pair
>> of disks? It's all taken care of at the other end of a pair of fibre
>> connectors.
>
> I fail to see why RAID is no longer an issue. (Unless you mean that it's
> now acceptable to lose data in the event of a hardware fault.)

RAID is an issue for the SAN administrator, not the server 
administrator.  The server admin gets assigned one (or more) virtual 
drive(s) and can safely assume that the data integrity will be 
maintained.  The SAN administrator will make sure that the logical drive 
has been allocated out of properly configured pools of RAIDs.

>
> I've yet to see any system where power consumption was even remotely
> important. Servers seem to come with power supplies that can handle
> *way* more current than you will ever need.
>

Usually, yes, but not always.

> I've also never seen a sever with more than one disk controller, come to
> mention it...
>

Then your servers are far from optimal.  In my work, It's not uncommon 
to see servers with 3 or 4 controllers.

1 for the OS, with its own set of RAID drives.
1 for the apps with its own set of RAID drives.  (you don't want log 
file or library file I/O to interfere with DB performance)
1 or more for the database, with its own sets of drives.  The indexes 
and redo logs all on different drives than the tables, to make sure that 
you don't cause bottlenecks by having the drive heads jumping from the 
index to the table back to the index...

With the exception of the OS drive (You don't want to boot off the SAN), 
all of those* can be sent to the SAN for where they will be spread 
across many disks and controllers for an optimal performance.

*Don't run your domain controllers or DNSes from a SAN either.

>> From the point of view of the backup administrator, it's much simpler
>> since he no longer has to worry about making sure that the backup agent
>> is running on all the servers and that their schedules don't actually
>> step over each other's toes, etc... it can all be done off-line in the
>> background.
>
> I don't see how that works.

The DVD or tape drives have direct backplane connections to the disk 
arrays so they can read the disks without impacting the network or 
competing with the server's own IO requests.

Just like a hardware RAID controller can rebuild a failed drive without 
the any noticeable performance degradation.

>
>> You just had a very weird problem? Just clone the DB to an new set of
>> disks, and assign those on a lab server so that the developpers can work
>> on the issue all weekend.
>
> You can trivially do that without a SAN. Disk imaging tools have existed
> for decades. Either way, you still have to take the DB server offline
> while you do this.

No.  That's the point.  The SAN administrator simply mirrors the disks 
in the background, without the server even knowing about it.

>
> Of course, if your DB server is a VM, the problem just became even more
> trivial. Depending on your virtualisation technology, maybe you don't
> even have to stop the server to copy the disk image. And you /still/
> don't need a SAN.
>
>> The biggest added level of complexity is that all these people have to
>> talk to one another. Scary. I know.
>
> Nah. Our entire IT function (including me) is about half a dozen humans.
>
>>> * If the storage network fails, ALL servers fail, so you're adding a new
>>> single point of failure.
>>
>> True, but that rarely happens.
>
> OK, fair enough.
>
>>> * All the servers now have to share the limited bandwidth available. One
>>> busy server can bring all the others to a crawl.
>>
>> What limited bandwidth? Even a SAN that's been improperly designed, and
>> that has seen 5 years of unchecked growth will have more than enough
>> bandwitdh to accomodate all the servers. Have you seen the
>> specifications of the SAN your head office wants to implement and were
>> able to determine there will be bottlenecks?
>
> My point is, if you have 10 servers, each with their own dedicated SATA
> connection (or whatever), you have 10 lots of bandwidth. And if one
> server decides to hog it all, it can't slow the others down.
>
> If you have a SAN, you have /one/ access channel, which all the servers
> have to share between them. If one server decides to hit it really hard
> (or just crashes and starts sending gibberish), all the others get
> clobbered.

Again, wrong.  Each server has dedicated access to the SAN fabric and 
the SAN switches will dispatch I/O requests to the proper drive(s) 
without impacting other servers' I/O requests.

>
>>> Oh, wait, I just found a real advantage: If you want to move a disk from
>>> one server to another, you just press a button, rather than having to
>>> physically move the device.
>>>
>>> Oh, well, that /totally/ makes it worth it. And, uh, when will you
>>> *ever* need to perform this operation? I mean, seriously, I've been
>>> working at the UK site for almost 10 years now, and I've never once
>>> wanted to do this. And even if I did, if on one day in 10 years I have
>>> to shut both servers down to physically move a drive, I think I can
>>> probably live with that.
>>
>> My current customer has two data centers with 350 servers in each. "We
>> need more disk space on server XYZ" happens on a daily basis, in fact,
>> multiple times a day, in some cases (e.g. a fast growing log file due to
>> a change in user traffic, or the activation of a debbugging feature
>> during problem troubleshooting)
>
> OK, well, perhaps if you have hundreds of servers, the massive
> performance hit /might/ be justified by the advantages of being able to
> keep more systems up more of the time. But for our 10 servers or so,
> which are almost all up almost all of the time, the advantage seems
> vanishingly small.
>
>>> Second, what is the purpose of a backup? The purpose of having a "backup
>>> copy" of your data is so that if your working copy dies somehow, you can
>>> use the backup copy to get your data back.
>>>
>>> This only works if the backup copy is more reliable than the working
>>> copy. If your working copy is on spinning disk, and you're stupid enough
>>> to put your backup copy on spinning disk as well... then it becomes
>>> equally likely that the /backup copy/ will die and you'll be left with
>>> just the working copy.
>>
>> As long as your working copy doesn't disappear before your can recreate
>> the backup copy, where's the problem? Are you saying that RAID is a
>> waste of time because there's an equal likelyhood that any of the disks
>> in the array will die?
>
> A backup copy is supposed to be an image of the system frozen at a
> particular point in time. Once that data is gone, you can't get it back.
> Because that moment in time has gone.
>
> RAID isn't the same time; it's supposed to keep the system running in
> the event of a fault, not preserve historical data.
>
>>> Third, having access only to the most recent backup is no good. There
>>> are scenarios where that would be perfectly OK, but in our line of
>>> business, we sometimes have to recover data from /months/ ago. Data that
>>> has been accidentally deleted. Data which got corrupted. Data which we
>>> thought we didn't need any more but actually we do. And so forth.
>>
>> Have they actually stated that you would lose the ability of going back
>> to older copies of your backups?
>
> No.
>
> I'm not saying we will lose this ability. I'm just noting that we need
> to be careful that we don't lose this ability. Simply mirroring the
> disks to the SAN at HQ or whatever is not sufficient. We are legally
> required to be able to restore old data.
>
> Hopefully the guys setting this up know that. As I say, they haven't
> actually decided how this is going to work yet.
>
>>> And then there's the fact that you either need to keep all this disk
>>> spinning (i.e., ever increasing power and cabling demands), or only keep
>>> the recent backups spinning (i.e., managing powering off and powering on
>>> drives, which supposedly shortens their lifetime).
>>>
>>> In all, this seems like an extremely bad idea.
>>
>> Have they actually stated that they would not be taking tape (or more
>> than likely DVD) backups of that data for archival?
>
> Bizarro. Why would you back up to a DVD that holds a piffling 4.2 GB of
> data when a single tape holds 400 GB? That makes no sense at all.
>

Tapes degrade over time faster than DVDs do.
I/O is sequential on a tape.  If you need a specific file, you need to 
read through the whole tape from the beginning.
If you only have a 10GB incremental per day, and need to sent it to 
offsite storage every day (sometimes required by law), using 400GB tape 
is a serious waste of money.

Each application has its preferred solution.  Your mileage may vary.

>> In all likelyhood, that remote copy will probably be backed up to
>> offsite storage - if not, it should be, just for the sake of business
>> continuity, but in many cases, for legal reasons as well.
>
> Nope. The idea, apparently, is that there will be two remote copies, at
> two different sites. If one dies, we've still got the other site.
> They're planning to have no tape at all, just spinning disk.
>

That _is_ silly.


-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message