OK this morning - the one morning that I slept in and went to work more 
'on-time' we had a server failure.

Workstations could not access the file shares.

First step of action with windows - reboot.

Rebooting the Windows server caused it to die - literally.
Nothing on the screen - just some cryptic beeps.
After some looking - memory module went dead - of course it had to be 
the 'big' one - so from 2.5 GB down to 1.5GB.....

Ok, so got that fixed.

Now bringing that back up - workstations still cannot access the file 
server.

The Linux file server uses Winbind with Samba to authenticate with 
domain credentials......  for some reason that is broken.

So I had the fun of on the fly configuring Samba to just share files 
without much security.

So now the hunt - why did winbind die......

Post a reply to this message

Tom Austin <voi### [at] voidnet> wrote:
> Workstations could not access the file shares.

> After some looking - memory module went dead - of course it had to be 
> the 'big' one - so from 2.5 GB down to 1.5GB.....

  One would think that if the server is mission-critical, it would have
redundant hardware. In other words, if for example a memory module dies,
the only consequence is that the amount of available RAM decreases and
a big-ass notification is logged somewhere, but otherwise the service
continues as usual.

  Of course this requires specialized server hardware, as well as
software support. (I don't even know if Windows supports this. I'm
assuming NT and its spawns ought to, but I have never heard either way.)

  And also I'm assuming this is not cheap, so management do not want.

-- 
                                                          - Warp

Post a reply to this message

On 1/13/2012 11:17, Warp wrote:
>    One would think that if the server is mission-critical, it would have
> redundant hardware.

Or a trivial replacement system. Turn off the power, slide the disk drive 
out, slide it into the other case, turn the power on.

Otherwise you're in the Tandem Computing realm, where the system doesn't die 
for decades at a time, and occasionally a new board shows up via FedEx with 
instructions on which one to replace.  "Oh, and by the way, we upgraded your 
kernel without you noticing, while you were out."

>    And also I'm assuming this is not cheap, so management do not want.

Making something truly zero down-time is exceedingly expensive.

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

On 13/01/2012 07:17 PM, Warp wrote:

>    One would think that if the server is mission-critical, it would have
> redundant hardware. In other words, if for example a memory module dies,
> the only consequence is that the amount of available RAM decreases and
> a big-ass notification is logged somewhere, but otherwise the service
> continues as usual.
>
>    Of course this requires specialized server hardware, as well as
> software support. (I don't even know if Windows supports this. I'm
> assuming NT and its spawns ought to, but I have never heard either way.)

For a time we had a HP ProLiant server with a memory "RAID" feature. 
(This is on top of all the memory being ECC RAM.) It's transparent to 
the OS.

For example, I might fit two 4GB RAM modules in a mirror configuration. 
The OS sees 4GB installed. If the ECC on one of them starts reporting 
uncorrectable errors, the system board will transparently fetch data 
from the other RAM module, as if nothing ever happened. In addition, an 
LED lights up on the front of the chassis, showing you exactly where on 
the motherboard the failed RAM module is, so you can replace it. (I'm 
unsure whether it was hot-swappable...)

Additionally, there were lights for EVERY INDIVIDUAL FAN (all 15 of 
them), both CPU sockets (so if one CPU dies, the server continues 
running - although I guess the OS is going to notice that one), and both 
of the redundant PSUs.

>    And also I'm assuming this is not cheap, so management do not want.

IIRC, this HP server was about £8,000. Which isn't actually as expensive 
as you might imagine. (I mean, sure, that's more money than *I* will 
ever own. But for a professional business enterprise, it's potentially 
not a lot of money.)

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

>> And also I'm assuming this is not cheap, so management do not want.
>
> Making something truly zero down-time is exceedingly expensive.

Yeah, reducing down-time isn't usually too bad, but /zero/ down-time 
requires going to absurd lengths. We're talking about backup power 
generators, multiple telecom providers, multiple physical locations, 
continuous data replication, multiple redundant systems... it gets 
expensive rapidly. As with any buying decision, you need to look at the 
cost of down-time verses the cost of preventing it.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

On 1/13/2012 2:17 PM, Warp wrote:
> Tom Austin<voi### [at] voidnet>  wrote:
>> Workstations could not access the file shares.
>
>> After some looking - memory module went dead - of course it had to be
>> the 'big' one - so from 2.5 GB down to 1.5GB.....
>
>    One would think that if the server is mission-critical, it would have
> redundant hardware. In other words, if for example a memory module dies,
> the only consequence is that the amount of available RAM decreases and
> a big-ass notification is logged somewhere, but otherwise the service
> continues as usual.
>

Yes, I agree, but we are a very small business.  We are only 7 people 
and have doubled in size int he past year.

The Windows server is an old low end Dell server machine.
I'm glad it was the memory module and not something on the MB.

We are working on migrating off of it as time allows.


>    Of course this requires specialized server hardware, as well as
> software support. (I don't even know if Windows supports this. I'm
> assuming NT and its spawns ought to, but I have never heard either way.)
>
>    And also I'm assuming this is not cheap, so management do not want.
>

We do have some methods to getting back up relatively quickly - tho they 
are not instantaneous.  Management is now willing to spend the money 
where needed - tho I don't think we will go to fail-safe quite yet.

Post a reply to this message

On 16/01/2012 12:36 PM, Tom Austin wrote:

> We do have some methods to getting back up relatively quickly - tho they
> are not instantaneous. Management is now willing to spend the money
> where needed - tho I don't think we will go to fail-safe quite yet.

Yeah, it's funny... I've noticed this strange correlation between 
expensive down-time and management willingness to invest in 
fault-tolerant equipment. ;-)

Post a reply to this message

Le 2012-01-16 08:11, Invisible a écrit :
> On 16/01/2012 12:36 PM, Tom Austin wrote:
>
>> We do have some methods to getting back up relatively quickly - tho they
>> are not instantaneous. Management is now willing to spend the money
>> where needed - tho I don't think we will go to fail-safe quite yet.
>
> Yeah, it's funny... I've noticed this strange correlation between
> expensive down-time and management willingness to invest in
> fault-tolerant equipment. ;-)

One of my all-time favorite post-mortem meetings went something like this:

Background info: there was a major catastrophe the night before.  A 
repeat of prevous incidents.  As a result, our customer made the news.

10:45am: we meet with our management to quickly explain to the big 
bosses what happened, what we did to resolve it, and how we wrote this 
nice proposal six months ago to permanently deal with the issue.  Big 
Boss says "we'll try to steer clear of We-Told-You-So, but we'll also 
remind them that this could have been avoided.  Let me do the talking 
and jump in if I mess up on my technical mumbo-jumbo"

11:00am: We walk in the customer's board room.  Before we're fully 
seated, the CIO opens up by saying "We know you told us so last time, 
but... is there something else we can do other than upgrade these 
Whatchamacallits?"

...

It took them two more major incidents - including one where they ended 
up on the cover of Time Magazine - before those old whatchamacallits 
were removed.

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

On 16/01/2012 01:51 PM, Francois Labreque wrote:

> "We know you told us so last time,
> but... is there something else we can do other than upgrade these
> Whatchamacallits?"
>
> ...
>
> It took them two more major incidents - including one where they ended
> up on the cover of Time Magazine - before those old whatchamacallits
> were removed.

One has to wonder why people are so resistant to fixing the problem. The 
solution is right there, and yet you want to go around the hard way. Why?

Post a reply to this message

On 1/16/2012 8:11 AM, Invisible wrote:
> On 16/01/2012 12:36 PM, Tom Austin wrote:
>
>> We do have some methods to getting back up relatively quickly - tho they
>> are not instantaneous. Management is now willing to spend the money
>> where needed - tho I don't think we will go to fail-safe quite yet.
>
> Yeah, it's funny... I've noticed this strange correlation between
> expensive down-time and management willingness to invest in
> fault-tolerant equipment. ;-)

As we get more people and process more data - the more down time will 
cost.  Eventually we may get to higher uptime requirements - but for now 
2 hours of downtime 2x a year is not too bad.

Post a reply to this message

>> Yeah, it's funny... I've noticed this strange correlation between
>> expensive down-time and management willingness to invest in
>> fault-tolerant equipment. ;-)
>
> As we get more people and process more data - the more down time will
> cost. Eventually we may get to higher uptime requirements - but for now
> 2 hours of downtime 2x a year is not too bad.

As with everything, it depends on just how expensive down-time actually 
is. If the answer is "not very", you don't need to worry about fixing it 
too much.

Amusing anecdote: One of our servers had a hardware RAID system. The 
idea is that if one of the drives dies, the RAID controller will keep 
the server operational. You know what died? THE RAID CONTROLLER! >_<

Post a reply to this message

On 1/16/2012 11:07 AM, Invisible wrote:
>>> Yeah, it's funny... I've noticed this strange correlation between
>>> expensive down-time and management willingness to invest in
>>> fault-tolerant equipment. ;-)
>>
>> As we get more people and process more data - the more down time will
>> cost. Eventually we may get to higher uptime requirements - but for now
>> 2 hours of downtime 2x a year is not too bad.
>
> As with everything, it depends on just how expensive down-time actually
> is. If the answer is "not very", you don't need to worry about fixing it
> too much.
>
> Amusing anecdote: One of our servers had a hardware RAID system. The
> idea is that if one of the drives dies, the RAID controller will keep
> the server operational. You know what died? THE RAID CONTROLLER! >_<


I like what Google has set up - files are stored everywhere on cheap 
hardware.  If something fails, then it just gets swapped out.  But the 
cheap hardware fails often and requires more manpower to manage and 
replace it.

Post a reply to this message

On 16/01/2012 7:12 PM, Tom Austin wrote:
> But the cheap hardware fails often and requires more manpower to manage
> and replace it.

Jobs! :-D

-- 
Regards
     Stephen

Post a reply to this message

>> Amusing anecdote: One of our servers had a hardware RAID system. The
>> idea is that if one of the drives dies, the RAID controller will keep
>> the server operational. You know what died? THE RAID CONTROLLER! >_<
>
> I like what Google has set up - files are stored everywhere on cheap
> hardware. If something fails, then it just gets swapped out. But the
> cheap hardware fails often and requires more manpower to manage and
> replace it.

Yeah, but most people can't do that. Most people don't have the space, 
ventilation or power requirements to host hundreds of boxes, nor the 
money to pay a team of twenty people to keep it all running.

On top of that, most applications are /not/ designed for distributed 
implementation. If you're Google, you can just /write/ the software you 
need. Most business buy it off the shelf.

Post a reply to this message

On 1/17/2012 1:06, Invisible wrote:
> Yeah, but most people can't do that. Most people don't have the space,
> ventilation or power requirements to host hundreds of boxes, nor the money
> to pay a team of twenty people to keep it all running.

Google has hundreds of computers in their data centers. Unfortunately for 
you, Google counts "a shipping container full of thousands of mother boards 
and disk drives" as "a computer". ;-)  It's really quite awesome. Upgrading 
a server is known as "forklifting" it.

> On top of that, most applications are /not/ designed for distributed
> implementation. If you're Google, you can just /write/ the software you
> need. Most business buy it off the shelf.

And you know, I really, really miss SQL. :-)

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

On 18/01/2012 03:04 AM, Darren New wrote:
> Google has hundreds of computers in their data centers. Unfortunately
> for you, Google counts "a shipping container full of thousands of mother
> boards and disk drives" as "a computer". ;-) It's really quite awesome.
> Upgrading a server is known as "forklifting" it.

Pictures or it didn't happen. :-P

Actually, I was just thinking about how many scrap PCs you could 
scavenge and wire together to make a cluster. I wonder how much old junk 
you would need to even come close to the processing power of a single 
high-spec PC?

> And you know, I really, really miss SQL. :-)

Even if it does violate the relational model? Aww, it must be wuv...

Post a reply to this message

Le 2012-01-16 15:03, Stephen a écrit :
> On 16/01/2012 7:12 PM, Tom Austin wrote:
>> But the cheap hardware fails often and requires more manpower to manage
>> and replace it.
>
> Jobs! :-D
>

No.  He died a while back.

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

>> Jobs! :-D
>
> No. He died a while back.

Oldies but goldies...

Post a reply to this message

On 18/01/2012 1:42 PM, Francois Labreque wrote:
> Le 2012-01-16 15:03, Stephen a écrit :
>> On 16/01/2012 7:12 PM, Tom Austin wrote:
>>> But the cheap hardware fails often and requires more manpower to manage
>>> and replace it.
>>
>> Jobs! :-D
>>
>
> No. He died a while back.
>

Jobbies! Then. :-P

-- 
Regards
     Stephen

Post a reply to this message

On 1/13/2012 2:07 PM, Tom Austin wrote:
>
> So now the hunt - why did winbind die......
>
>
>


It died because I made a small configuration change.

I did not make a backup of the config file because I was making a very 
simple change that was very easily undoable.
Just a simple x = y line.

Well, in VI I hit i too many times and that little extra 'i' got added 
to the domain for winbind - in a hard to see place.

So, lesson learned - no matter how simple the modification - always make 
a copy that you can fall back to that is UNMODIFIED from what works.

At least now I have a simple smb.conf for sharing files in an emergency.

Post a reply to this message

On Wed, 18 Jan 2012 09:06:13 +0000, Invisible wrote:

> Pictures or it didn't happen. :-P

http://www.youtube.com/watch?v=zRwPSFpLX8I perhaps?

Jim

Post a reply to this message

> Pictures or it didn't happen. :-P

They're out there.

>> And you know, I really, really miss SQL. :-)
> Even if it does violate the relational model? Aww, it must be wuv...

Far better than what you get to use at Google. :-)

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

>> Pictures or it didn't happen. :-P
>
> http://www.youtube.com/watch?v=zRwPSFpLX8I perhaps?

...the HELL?! o_O

1. I had no idea they had this many computers.

2. I had no idea it required engineering on this scale to run that many 
computers.

3. That seems like quite a lot of empty space, actually...

Post a reply to this message

On 20/01/2012 9:12 AM, Invisible wrote:
>>> Pictures or it didn't happen. :-P
>>
>> http://www.youtube.com/watch?v=zRwPSFpLX8I perhaps?
>
> ....the HELL?! o_O
>
Nor am I out of it. ;-)

> 1. I had no idea they had this many computers.
>
http://news.cnet.com/8301-1001_3-10209580-92.html

> 2. I had no idea it required engineering on this scale to run that many
> computers.
>

It is not a domestic operation.

> 3. That seems like quite a lot of empty space, actually...

It does not mean that it is wasted space, though. You need space for 
construction and maintenance. Not to mention air flow for cooling.

-- 
Regards
     Stephen

Post a reply to this message

>> 1. I had no idea they had this many computers.
>>
> http://news.cnet.com/8301-1001_3-10209580-92.html

My God. o_O

>> 2. I had no idea it required engineering on this scale to run that many
>> computers.
>
> It is not a domestic operation.

Sure. But given that computers are very low-power devices, you don't 
think of a computer plant as something requiring megawatts of power and 
entire rooms of cooling equipment.

Post a reply to this message

On 20/01/2012 10:18 AM, Invisible wrote:
>
> Sure. But given that computers are very low-power devices, you don't
> think of a computer plant as something requiring megawatts of power and
> entire rooms of cooling equipment.

I would not say that computers are *very* low-power devices. As it said 
in the article the containers have a power consumption that can reach 
250 kilowatts. So you only need 4 containers to have to supply a 
megawatt of power. Then add the cooling requirements and the utilities 
and it all adds up.

-- 
Regards
     Stephen

Post a reply to this message

On 20/01/2012 11:20 AM, Stephen wrote:
> On 20/01/2012 10:18 AM, Invisible wrote:
>>
>> Sure. But given that computers are very low-power devices, you don't
>> think of a computer plant as something requiring megawatts of power and
>> entire rooms of cooling equipment.
>
> I would not say that computers are *very* low-power devices. As it said
> in the article the containers have a power consumption that can reach
> 250 kilowatts. So you only need 4 containers to have to supply a
> megawatt of power. Then add the cooling requirements and the utilities
> and it all adds up.

Compared to an industrial cement kiln or a smelting works, a computer 
uses virtually zero power.

The difference, of course, is that we're not talking about a computer. 
We're talking about *thousands* of computers. Much like a light bulb 
uses almost no power, but the entire Blackpool illuminations probably 
use quite a bit of power...

I'm not saying it's wrong, I'm saying it's surprising.

Post a reply to this message

On 20/01/2012 11:31 AM, Invisible wrote:
> I'm not saying it's wrong, I'm saying it's surprising.

Why do you think that it is surprising?

-- 
Regards
     Stephen

Post a reply to this message

On 20/01/2012 01:08 PM, Stephen wrote:
> On 20/01/2012 11:31 AM, Invisible wrote:
>> I'm not saying it's wrong, I'm saying it's surprising.
>
> Why do you think that it is surprising?

Or, to put it another way, why do you *not* think it's surprising?

I guess once you already know something, it probably doesn't surprise 
you any more...

Post a reply to this message

On 20/01/2012 1:10 PM, Invisible wrote:
> On 20/01/2012 01:08 PM, Stephen wrote:
>> On 20/01/2012 11:31 AM, Invisible wrote:
>>> I'm not saying it's wrong, I'm saying it's surprising.
>>
>> Why do you think that it is surprising?
>
> Or, to put it another way, why do you *not* think it's surprising?
>

I have worked in heavy industry and when I worked offshore on production 
platforms I was familiar with most aspects of utilities.

> I guess once you already know something, it probably doesn't surprise
> you any more...

That is true but if you stop and think (no slur intended) you might find 
on reflection it is not really surprising. It is like doing a quick 
mental calculation to see if an answer is in the correct range.
Guesstimate power requirements for 1 server = 200 Watts
Power for 1000 servers in a container (X 1000) 200 KW
Power for 12 containers = 2.4 MW. See no surprise. ;-)

-- 
Regards
     Stephen

Post a reply to this message

>> I guess once you already know something, it probably doesn't surprise
>> you any more...
>
> That is true but if you stop and think (no slur intended) you might find
> on reflection it is not really surprising. It is like doing a quick
> mental calculation to see if an answer is in the correct range.
> Guesstimate power requirements for 1 server = 200 Watts
> Power for 1000 servers in a container (X 1000) 200 KW
> Power for 12 containers = 2.4 MW. See no surprise. ;-)

I guess the thing is, most people will never, ever see 1000 computers...

Post a reply to this message

On Fri, 20 Jan 2012 09:12:41 +0000, Invisible wrote:

>>> Pictures or it didn't happen. :-P
>>
>> http://www.youtube.com/watch?v=zRwPSFpLX8I perhaps?
> 
> ...the HELL?! o_O
> 
> 1. I had no idea they had this many computers.

It hit a number of the sites a couple years ago (see the date on the 
video)

> 2. I had no idea it required engineering on this scale to run that many
> computers.

You've got to manage them *somehow*, right?

> 3. That seems like quite a lot of empty space, actually...

You have to circulate air to keep things cool.  Overheating in such an 
environment could be a serious problem.

Jim

Post a reply to this message

On Fri, 20 Jan 2012 10:18:25 +0000, Invisible wrote:

> Sure. But given that computers are very low-power devices, you don't
> think of a computer plant as something requiring megawatts of power and
> entire rooms of cooling equipment.

I've done work in a 20,000 sq. ft. data center.  Data centers are big 
business these days.

Jim

Post a reply to this message

On 1/20/2012 2:18, Invisible wrote:
> Sure. But given that computers are very low-power devices, you don't think
> of a computer plant as something requiring megawatts of power and entire
> rooms of cooling equipment.

Each of those containers has over 1000 computers in it. There's 50 
containers in this one building. (That's a small building for Google.) This 
is from 2005 or so, when computers weren't as efficient.  So how much power 
do you think it takes to run 50,000 computers, plus air conditioning for 
them, if each has (say) 150W power supply?

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

Le 20/01/2012 18:02, Jim Henderson a écrit :
>> > 2. I had no idea it required engineering on this scale to run that many
>> > computers.
> You've got to manage them *somehow*, right?
> 


Wondering if they are running MS Windows:
 1. Did they get a rebate on number of license ?
 2. In case of security update, did they all update & reboot at the same
time ?

Obviously, they are not using Apple's computers.

Post a reply to this message

On 20/01/2012 05:03 PM, Jim Henderson wrote:
> On Fri, 20 Jan 2012 10:18:25 +0000, Invisible wrote:
>
>> Sure. But given that computers are very low-power devices, you don't
>> think of a computer plant as something requiring megawatts of power and
>> entire rooms of cooling equipment.
>
> I've done work in a 20,000 sq. ft. data center.  Data centers are big
> business these days.

The biggest data center I've ever been in was one largish room populated 
with server racks.

Heh, I still remember looking at one of the stacks, and seeing that it 
had *seven* 4.2 GB drives in it. (Remember, in 1997, those suckers where 
EXPENSIVE.) I remember feeling slightly giddy trying to compute how much 
total storage space such a monster RAID setup... Ah, the memories.

Post a reply to this message

On 21/01/2012 05:42 PM, Darren New wrote:
> On 1/20/2012 2:18, Invisible wrote:
>> Sure. But given that computers are very low-power devices, you don't
>> think
>> of a computer plant as something requiring megawatts of power and entire
>> rooms of cooling equipment.
>
> So how
> much power do you think it takes to run 50,000 computers, plus air
> conditioning for them, if each has (say) 150W power supply?

As I say, the point is that most people will never see 50,000 computers 
in one place all at once.

Post a reply to this message

On 23/01/2012 07:23 AM, Le_Forgeron wrote:

> Wondering if they are running MS Windows:
>   1. Did they get a rebate on number of license ?
>   2. In case of security update, did they all update&  reboot at the same
> time ?
>
> Obviously, they are not using Apple's computers.

I'm pretty certain it all runs Linux. (I seem to recall a Google paper 
about how they made various improvements to the Linux kernel to handle 
SCSI error conditions better or something like that...) I have no idea 
*which* Linux, of course... It's quite possible that it's something that 
does cost money.

Post a reply to this message

On 1/15/2012 6:05 AM, Orchid XP v8 wrote:
>>> And also I'm assuming this is not cheap, so management do not want.
>>
>> Making something truly zero down-time is exceedingly expensive.
>
> Yeah, reducing down-time isn't usually too bad, but /zero/ down-time
> requires going to absurd lengths.

When I was stationed at Onizuka AFB, California (the Blue Cube, for 
those who live or work near the south end of the San Francisco Bay), we 
had two large SATCOM dishes, called Sun East and Sun West.

One chronic problem was that getting downtime for preventive maintenance 
was as difficult as pulling gold teeth from a chicken.  The user 
community (who was and always will be better connected than the 
maintenance community) were as stubborn as mules about allowing any 
downtime for any reason.

"Hey, what if we took each dish down, once every six months, and do all 
of the preventive maintenance specified in the technical orders?"

"No, no, we need those dishes up 24/7.  Denied."

You'll never guess what the result was.

Regards,
John

Post a reply to this message

On Mon, 23 Jan 2012 09:31:05 +0000, Invisible wrote:

> On 21/01/2012 05:42 PM, Darren New wrote:
>> On 1/20/2012 2:18, Invisible wrote:
>>> Sure. But given that computers are very low-power devices, you don't
>>> think of a computer plant as something requiring megawatts of power
>>> and entire rooms of cooling equipment.
>>
>> So how much power do you think it takes to run 50,000 computers, plus
>> air conditioning for them, if each has (say) 150W power supply?
> 
> As I say, the point is that most people will never see 50,000 computers
> in one place all at once.

That isn't really necessary to be able to do the math....

Jim

Post a reply to this message

On Mon, 23 Jan 2012 09:29:40 +0000, Invisible wrote:

> On 20/01/2012 05:03 PM, Jim Henderson wrote:
>> On Fri, 20 Jan 2012 10:18:25 +0000, Invisible wrote:
>>
>>> Sure. But given that computers are very low-power devices, you don't
>>> think of a computer plant as something requiring megawatts of power
>>> and entire rooms of cooling equipment.
>>
>> I've done work in a 20,000 sq. ft. data center.  Data centers are big
>> business these days.
> 
> The biggest data center I've ever been in was one largish room populated
> with server racks.
> 
> Heh, I still remember looking at one of the stacks, and seeing that it
> had *seven* 4.2 GB drives in it. (Remember, in 1997, those suckers where
> EXPENSIVE.) I remember feeling slightly giddy trying to compute how much
> total storage space such a monster RAID setup... Ah, the memories.

Indeed, I still recall that the Fortune 50 company I worked for a decade 
ago had an EMC storage array with 750 GB of storage in it....

My new *laptop* has a drive that big in it.  I've got about 4 TB of 
storage here at home now.

Jim

Post a reply to this message

On Mon, 23 Jan 2012 08:23:08 +0100, Le_Forgeron wrote:

> Le 20/01/2012 18:02, Jim Henderson a écrit :
>>> > 2. I had no idea it required engineering on this scale to run that
>>> > many computers.
>> You've got to manage them *somehow*, right?
>> 
>> 
> 
> Wondering if they are running MS Windows:
>  1. Did they get a rebate on number of license ?

Google?  Run Windows?  No, they use a custom OS AFAICR.

Jim

Post a reply to this message

>> Heh, I still remember looking at one of the stacks, and seeing that it
>> had *seven* 4.2 GB drives in it. (Remember, in 1997, those suckers where
>> EXPENSIVE.) I remember feeling slightly giddy trying to compute how much
>> total storage space such a monster RAID setup... Ah, the memories.
>
> Indeed, I still recall that the Fortune 50 company I worked for a decade
> ago had an EMC storage array with 750 GB of storage in it....
>
> My new *laptop* has a drive that big in it.  I've got about 4 TB of
> storage here at home now.

The fun thing is, apparently SSD has

1. Reached price levels where Normal Humans can potentially afford them.

2. Reached capacities where you might actually buy this stuff.

Interesting times ahead, eh?

(Personally, I still can't figure out why SSD isn't several /million/ 
times faster than a mechanical spinning disk, but hey...)

Post a reply to this message

>>> So how much power do you think it takes to run 50,000 computers, plus
>>> air conditioning for them, if each has (say) 150W power supply?
>>
>> As I say, the point is that most people will never see 50,000 computers
>> in one place all at once.
>
> That isn't really necessary to be able to do the math....

Sure, that's the great thing about math; it always works right, unlike 
intuition. ;-)

But it's not a calculation most people would bother to make, because the 
power consumption of a PC is "negligible".

In a similar vein, the heat output of a normal human in a large empty 
room is also negligible. But weirdly, if you put /a lot/ of humans in a 
room, no matter how big that room is, they manage to raise the 
temperature of the whole room. Unexpected, but true...

Post a reply to this message

On Mon, 23 Jan 2012 15:41:25 +0000, Invisible wrote:

>>> Heh, I still remember looking at one of the stacks, and seeing that it
>>> had *seven* 4.2 GB drives in it. (Remember, in 1997, those suckers
>>> where EXPENSIVE.) I remember feeling slightly giddy trying to compute
>>> how much total storage space such a monster RAID setup... Ah, the
>>> memories.
>>
>> Indeed, I still recall that the Fortune 50 company I worked for a
>> decade ago had an EMC storage array with 750 GB of storage in it....
>>
>> My new *laptop* has a drive that big in it.  I've got about 4 TB of
>> storage here at home now.
> 
> The fun thing is, apparently SSD has
> 
> 1. Reached price levels where Normal Humans can potentially afford them.
> 
> 2. Reached capacities where you might actually buy this stuff.
> 
> Interesting times ahead, eh?

Indeed, I've a friend in Scotland who has a Thinkpad with an SSD in it 
and the performance is good, but the battery life is also quite good (I 
think he said he gets 10 hours out of a single battery).

Jim

Post a reply to this message

>> The fun thing is, apparently SSD has
>>
>> 1. Reached price levels where Normal Humans can potentially afford them.
>>
>> 2. Reached capacities where you might actually buy this stuff.
>>
>> Interesting times ahead, eh?
>
> Indeed, I've a friend in Scotland who has a Thinkpad with an SSD in it
> and the performance is good, but the battery life is also quite good (I
> think he said he gets 10 hours out of a single battery).

Lots of laptops have offered an SSD option for a while now. The problem 
has always been that the price is a bit insane. Looks like prices are 
starting to become reasonable now - although I see 100x differences in 
the reported I/O performance being quoted for different models. (!!)

Also, I'm quite impressed with my own laptop. Usually laptops /claim/ to 
run for 8 hours or something, but /actually/ run for about 20 minutes. 
The 8 hours figure applies only if you leave the laptop turned on but 
not *doing* anything. :-P But my current laptop /really does/ run for 
several hours without needing recharging. It's pretty neat...

(My laptop also has /actual 3D performance/ too. I had no idea laptops 
could do that now!)

Post a reply to this message

On 23/01/2012 3:43 PM, Invisible wrote:
> Unexpected, but true...

<boggle>

-- 
Regards
     Stephen

Post a reply to this message

On Mon, 23 Jan 2012 15:50:19 +0000, Invisible wrote:

>>> The fun thing is, apparently SSD has
>>>
>>> 1. Reached price levels where Normal Humans can potentially afford
>>> them.
>>>
>>> 2. Reached capacities where you might actually buy this stuff.
>>>
>>> Interesting times ahead, eh?
>>
>> Indeed, I've a friend in Scotland who has a Thinkpad with an SSD in it
>> and the performance is good, but the battery life is also quite good (I
>> think he said he gets 10 hours out of a single battery).
> 
> Lots of laptops have offered an SSD option for a while now. The problem
> has always been that the price is a bit insane. Looks like prices are
> starting to become reasonable now - although I see 100x differences in
> the reported I/O performance being quoted for different models. (!!)
> 
> Also, I'm quite impressed with my own laptop. Usually laptops /claim/ to
> run for 8 hours or something, but /actually/ run for about 20 minutes.
> The 8 hours figure applies only if you leave the laptop turned on but
> not *doing* anything. :-P But my current laptop /really does/ run for
> several hours without needing recharging. It's pretty neat...
> 
> (My laptop also has /actual 3D performance/ too. I had no idea laptops
> could do that now!)

My new laptop has 3D performance as well - it's a "desktop replacement".  
But I typically get about 2-3 hours off the battery in powersave mode.  
Most laptops that I've used for a bit, the battery typically is good for 
about an hour to an hour and a half.  But a new battery generally will do 
well.

Jim

Post a reply to this message

On Mon, 23 Jan 2012 15:43:34 +0000, Invisible wrote:

>>>> So how much power do you think it takes to run 50,000 computers, plus
>>>> air conditioning for them, if each has (say) 150W power supply?
>>>
>>> As I say, the point is that most people will never see 50,000
>>> computers in one place all at once.
>>
>> That isn't really necessary to be able to do the math....
> 
> Sure, that's the great thing about math; it always works right, unlike
> intuition. ;-)
> 
> But it's not a calculation most people would bother to make, because the
> power consumption of a PC is "negligible".

Actually not so much.  A 450 W power supply draws a reasonably 
significant amount of power.

> In a similar vein, the heat output of a normal human in a large empty
> room is also negligible. But weirdly, if you put /a lot/ of humans in a
> room, no matter how big that room is, they manage to raise the
> temperature of the whole room. Unexpected, but true...

Not unexpected at all - completely rational and in line with the laws of 
thermodynamics.

Jim

Post a reply to this message

>> But it's not a calculation most people would bother to make, because the
>> power consumption of a PC is "negligible".
>
> Actually not so much.  A 450 W power supply draws a reasonably
> significant amount of power.

Yeah, generally you only find power supplies that meaty on high-end 
"leet gamer" PCs. I'm guessing Google doesn't use those.

Even 450W is peanuts compared to what my kettle uses. (3.6 kW)

>> In a similar vein, the heat output of a normal human in a large empty
>> room is also negligible. But weirdly, if you put /a lot/ of humans in a
>> room, no matter how big that room is, they manage to raise the
>> temperature of the whole room. Unexpected, but true...
>
> Not unexpected at all - completely rational and in line with the laws of
> thermodynamics.

Sure. But humans don't think in terms of thermodynamics, they think in 
terms of common everyday experience.

Random fact: If you're dangling from a wire above a pit of molten lava, 
YOU DIE. In particular, you do /not/ have to actually *touch* the lava 
for it to kill you. An entire lake radiates easily enough heat to cook a 
small elephant, never mind your skinny arse. And that's without even 
taking into account convection and all the toxic gasses.

Really, that scene from Indiana Jones and the Temple of Doom is 
laughably implausible... But most people actually have no clue exactly 
how hot lava really is. It's not in their everyday experience.

Actually, going back to my previous point: The amount of /moisture/ 
emitted by a normal human is absurdly small. And yet, put enough of them 
in a room, and it can get astonishingly moist in there! o_O

Post a reply to this message