>> * Tens of thousands of times more expensive than a direct connection.
>
> For a small or remote office with a couple file-servers, I agree that it
> doesn't make sense but for a data center, it's a different story. If you
> have 200 servers connected to the SAN, the "unit cost" of the SAN
> hardware becomes a lot more palatable, and you can leverage unused disk
> space that nay servers will have.

AFAIK, our data centre has about 10 physical servers. (And a slightly 
larger number of virtual servers.)

They're talking about increasing the number of virtual servers (possibly 
making /every/ end-user server virtual), which makes being able to swap 
disks around a more attractive proposition. If physical server X fails, 
move all the disks to physical server Y and start all the VMs back up. 
I'm still not convinced we need to spend 150,000 USD just to make it 
easier to move disks though.

>> * Hundreds of thousands of times reduced performance compared to a
>> direct connection.
>
> Not necessarily. It depends a lot on the configuration of the locally
> attached drives vs. SAN logical disks, and the types of IO requests that
> are made (e.g. long sequential reads or writes vs. lots of short random
> file accesses). SANs have all kinds of caching mechanisms that can make
> the IO actually faster than a SATA connection.

SATA 3.0 connection: 6.0 GB/sec

Fast Ethernet connection: 0.1 GB/sec

That's, what, 6000x slower?

Sure, you can add caching, which might speed up write access. But it 
can't speed up read access that misses the cache.

>> * Radically increased complexity compared to a direct connection.
>
> That depends on the point of view. From the server administrator's point
> of view, it's much simpler. She no longer has to worry about spreading
> her disks across multiple controllers, mirroring, etc.... on which
> partition should the indexes go? What level of RAID to use? Is there
> enough power in the server's power supplies to drive an additional pair
> of disks? It's all taken care of at the other end of a pair of fibre
> connectors.

I fail to see why RAID is no longer an issue. (Unless you mean that it's 
now acceptable to lose data in the event of a hardware fault.)

I've yet to see any system where power consumption was even remotely 
important. Servers seem to come with power supplies that can handle 
*way* more current than you will ever need.

I've also never seen a sever with more than one disk controller, come to 
mention it...

> From the point of view of the backup administrator, it's much simpler
> since he no longer has to worry about making sure that the backup agent
> is running on all the servers and that their schedules don't actually
> step over each other's toes, etc... it can all be done off-line in the
> background.

I don't see how that works.

> You just had a very weird problem? Just clone the DB to an new set of
> disks, and assign those on a lab server so that the developpers can work
> on the issue all weekend.

You can trivially do that without a SAN. Disk imaging tools have existed 
for decades. Either way, you still have to take the DB server offline 
while you do this.

Of course, if your DB server is a VM, the problem just became even more 
trivial. Depending on your virtualisation technology, maybe you don't 
even have to stop the server to copy the disk image. And you /still/ 
don't need a SAN.

> The biggest added level of complexity is that all these people have to
> talk to one another. Scary. I know.

Nah. Our entire IT function (including me) is about half a dozen humans.

>> * If the storage network fails, ALL servers fail, so you're adding a new
>> single point of failure.
>
> True, but that rarely happens.

OK, fair enough.

>> * All the servers now have to share the limited bandwidth available. One
>> busy server can bring all the others to a crawl.
>
> What limited bandwidth? Even a SAN that's been improperly designed, and
> that has seen 5 years of unchecked growth will have more than enough
> bandwitdh to accomodate all the servers. Have you seen the
> specifications of the SAN your head office wants to implement and were
> able to determine there will be bottlenecks?

My point is, if you have 10 servers, each with their own dedicated SATA 
connection (or whatever), you have 10 lots of bandwidth. And if one 
server decides to hog it all, it can't slow the others down.

If you have a SAN, you have /one/ access channel, which all the servers 
have to share between them. If one server decides to hit it really hard 
(or just crashes and starts sending gibberish), all the others get 
clobbered.

>> Oh, wait, I just found a real advantage: If you want to move a disk from
>> one server to another, you just press a button, rather than having to
>> physically move the device.
>>
>> Oh, well, that /totally/ makes it worth it. And, uh, when will you
>> *ever* need to perform this operation? I mean, seriously, I've been
>> working at the UK site for almost 10 years now, and I've never once
>> wanted to do this. And even if I did, if on one day in 10 years I have
>> to shut both servers down to physically move a drive, I think I can
>> probably live with that.
>
> My current customer has two data centers with 350 servers in each. "We
> need more disk space on server XYZ" happens on a daily basis, in fact,
> multiple times a day, in some cases (e.g. a fast growing log file due to
> a change in user traffic, or the activation of a debbugging feature
> during problem troubleshooting)

OK, well, perhaps if you have hundreds of servers, the massive 
performance hit /might/ be justified by the advantages of being able to 
keep more systems up more of the time. But for our 10 servers or so, 
which are almost all up almost all of the time, the advantage seems 
vanishingly small.

>> Second, what is the purpose of a backup? The purpose of having a "backup
>> copy" of your data is so that if your working copy dies somehow, you can
>> use the backup copy to get your data back.
>>
>> This only works if the backup copy is more reliable than the working
>> copy. If your working copy is on spinning disk, and you're stupid enough
>> to put your backup copy on spinning disk as well... then it becomes
>> equally likely that the /backup copy/ will die and you'll be left with
>> just the working copy.
>
> As long as your working copy doesn't disappear before your can recreate
> the backup copy, where's the problem? Are you saying that RAID is a
> waste of time because there's an equal likelyhood that any of the disks
> in the array will die?

A backup copy is supposed to be an image of the system frozen at a 
particular point in time. Once that data is gone, you can't get it back. 
Because that moment in time has gone.

RAID isn't the same time; it's supposed to keep the system running in 
the event of a fault, not preserve historical data.

>> Third, having access only to the most recent backup is no good. There
>> are scenarios where that would be perfectly OK, but in our line of
>> business, we sometimes have to recover data from /months/ ago. Data that
>> has been accidentally deleted. Data which got corrupted. Data which we
>> thought we didn't need any more but actually we do. And so forth.
>
> Have they actually stated that you would lose the ability of going back
> to older copies of your backups?

No.

I'm not saying we will lose this ability. I'm just noting that we need 
to be careful that we don't lose this ability. Simply mirroring the 
disks to the SAN at HQ or whatever is not sufficient. We are legally 
required to be able to restore old data.

Hopefully the guys setting this up know that. As I say, they haven't 
actually decided how this is going to work yet.

>> And then there's the fact that you either need to keep all this disk
>> spinning (i.e., ever increasing power and cabling demands), or only keep
>> the recent backups spinning (i.e., managing powering off and powering on
>> drives, which supposedly shortens their lifetime).
>>
>> In all, this seems like an extremely bad idea.
>
> Have they actually stated that they would not be taking tape (or more
> than likely DVD) backups of that data for archival?

Bizarro. Why would you back up to a DVD that holds a piffling 4.2 GB of 
data when a single tape holds 400 GB? That makes no sense at all.

> In all likelyhood, that remote copy will probably be backed up to
> offsite storage - if not, it should be, just for the sake of business
> continuity, but in many cases, for legal reasons as well.

Nope. The idea, apparently, is that there will be two remote copies, at 
two different sites. If one dies, we've still got the other site. 
They're planning to have no tape at all, just spinning disk.

>> Still, what I do know? Apparently not a lot.
>
> No offense, but I thought this was an established fact. ;)

Hey, the greatest knowledge is in knowing that you know nothing. 
Remember that...

> Don't get me wrong, you are extremely knowledgeable in many areas, but
> you seem to be unable to admit that other people may have other reasons
> to do something that you deem unnecessary because it doesn't apply to
> your situation.

I'm saying that from what I can understand about this company's 
situation, the proposed course of action doesn't make sense. That's all.

Post a reply to this message