On Sun, 23 Nov 2008 23:15:01 -0800, Darren New wrote:

> Jim Henderson wrote:
>> I've *never* seen shared
>> libraries updated by opening them and writing a change into them, then
>> saving them out again.
> 
> So, you haven't used gcc much, then? :-)

I use it quite a bit.  A recompile of a library generates a new file, it 
doesn't modify an existing library like you would a text document = open, 
modify, save by writing to same inode.

Jim

Post a reply to this message

On Sun, 23 Nov 2008 22:57:32 -0800, Darren New wrote:

> Jim Henderson wrote:
>> Well, sure, but dealing with every possible and conceivable exception
>> gets into areas where even in professional development it's decided not
>> to go.
> 
> No, I  mean you can make it generic. Instead of looking at each of the
> 30+ possible causes of error, you can say "Any error rolls back the
> changes."

So like "an error occurs, let's ask the user what should happen"?  Which 
is what it does. :-)

>> Well, yeah, that would fall under the category of "handling the
>> exception", no? ;-)
> 
> Not really. It's correcting an error you detected. If the error
> correction step also fails, you're pretty screwed.

That would likely happen with a disk full error - if you can't commit one 
transaction, it's going to be hard to verify and write out the old state 
as well, no?

> For example, to make things work, you need to up date the executable and
> three dynamic libraries. You update the executable, and one of the
> libraries, and then your network connection to the machine hosting the
> files fails. You can't handle that exception by rolling back your
> changes manually.

You can by reinstalling the package, though.

>> Sure, and as I said, I've had that happen to me - didn't need to
>> reinstall the system, had to clean a few things up, but this sort of
>> thing can happen with any OS, not just Linux.  Ever try writing to the
>> Windows registry with C: full?  (I knew you were waiting for me to
>> bring Windows into the discussion - there you go. ;-) )
> 
> Well, yes. That's why you have file system transactions in Windows.
> That's kinda exactly my point. If you start a kernel transaction, copy
> some files, update the registry, then bomb out, your changes get rolled
> back automatically. Just like any other database system, and regardless
> of why you bombed out or over which network the files are mounted.

And on Linux you have ext3, reiser, jfs, and other journaling filesystems.

>> So you need to run a validation - but again as I've said a few times,
>> my experience has been that an *error* condition like you describe
>> stops the upgrade process, it doesn't silently say "well, that's OK"
>> and continue without notifying the user.
> 
> Unless the error is "you pulled the plug" or "the RAID fell over" or
> something like that.

If RAID falls over, I'd say you've got a more serious problem on your 
hands....I've had that happen a few times, and it generally leads to a 
system reinstall.

>> "in spite of failures" - huh?  Update a system by writing files that
>> cannot be written because of a full disk (say) even though the disk is
>> full?
> 
> No. Not having half-written files caused by the disk being full. Or
> having only (say) two out of file files updated because the disk got
> full while you were writing the third file.

Sure, and that's what package validation is for - but again it becomes 
moot because when the SUSE updater tries to update a file and can't 
because the disk is full, you get a big really obvious dialogue box that 
says "hey, we're out of disk space, what do you want to do?" - so you can 
clear up some disk space and say "try again".

Jim

Post a reply to this message

Jim Henderson wrote:
>> No, I  mean you can make it generic. Instead of looking at each of the
>> 30+ possible causes of error, you can say "Any error rolls back the
>> changes."
> 
> So like "an error occurs, let's ask the user what should happen"?  Which 
> is what it does. :-)

Uh, no. That's kind of my point. You can set things up such that it 
works *JUST LIKE* a database transaction. Something fails in the middle, 
and the system automatically puts things back to the way they were.

> That would likely happen with a disk full error - if you can't commit one 
> transaction, it's going to be hard to verify and write out the old state 
> as well, no?

No. Just like a real database system, things don't get updated 
permanently until you're sure it'll work.

>> For example, to make things work, you need to up date the executable and
>> three dynamic libraries. You update the executable, and one of the
>> libraries, and then your network connection to the machine hosting the
>> files fails. You can't handle that exception by rolling back your
>> changes manually.
> 
> You can by reinstalling the package, though.

Only after you get your network connection back. Until then, everyone 
using the machine locally is screwed.

>> Well, yes. That's why you have file system transactions in Windows.
>> That's kinda exactly my point. If you start a kernel transaction, copy
>> some files, update the registry, then bomb out, your changes get rolled
>> back automatically. Just like any other database system, and regardless
>> of why you bombed out or over which network the files are mounted.
> 
> And on Linux you have ext3, reiser, jfs, and other journaling filesystems.


Cool. What's the system call in Linux that lets me change three files 
consistemtly?  I.e., I have files /tmp/One, /tmp/Two, and /tmp/Three, 
and I want to rename them respectively to /tmp/1, /tmp/2, and /tmp/3, 
and I never want any possibility of an "ls" operation on the /tmp/ 
directory to show my /tmp/One and /tmp/3 at the same time, or /tmp/1 and 
/tmp/Three.  Is there some way to accomplish that?

>> Unless the error is "you pulled the plug" or "the RAID fell over" or
>> something like that.
> 
> If RAID falls over, I'd say you've got a more serious problem on your 
> hands....I've had that happen a few times, and it generally leads to a 
> system reinstall.

And you've never lost power either, I take it?

> Sure, and that's what package validation is for

Except that's a manual post-hoc fix that only works for package 
installs. I'm talking about something that works without intervention on 
a more generic kind of problem.

-- 
Darren New / San Diego, CA, USA (PST)

Post a reply to this message

Jim Henderson wrote:
> I use it quite a bit.  A recompile of a library generates a new file, it 
> doesn't modify an existing library 

Did you try it?  It certainly reuses the same file for me.

-- 
Darren New / San Diego, CA, USA (PST)

Post a reply to this message

On Tue, 25 Nov 2008 12:33:56 -0800, Darren New wrote:

> Jim Henderson wrote:
>> I use it quite a bit.  A recompile of a library generates a new file,
>> it doesn't modify an existing library
> 
> Did you try it?  It certainly reuses the same file for me.

Logically it doesn't make sense - since offsets change in a library and 
so on when compiling code.....

And in my test, no, it didn't use the same inode for different 
compilations of the same C file.

Jim

Post a reply to this message

On Tue, 25 Nov 2008 12:33:26 -0800, Darren New wrote:

> Jim Henderson wrote:
>>> No, I  mean you can make it generic. Instead of looking at each of the
>>> 30+ possible causes of error, you can say "Any error rolls back the
>>> changes."
>> 
>> So like "an error occurs, let's ask the user what should happen"? 
>> Which is what it does. :-)
> 
> Uh, no. That's kind of my point. You can set things up such that it
> works *JUST LIKE* a database transaction. Something fails in the middle,
> and the system automatically puts things back to the way they were.

I prefer that the system let me know about an error condition so I can 
correct the underlying issue, when it comes to things like filesystem 
errors.

>> That would likely happen with a disk full error - if you can't commit
>> one transaction, it's going to be hard to verify and write out the old
>> state as well, no?
> 
> No. Just like a real database system, things don't get updated
> permanently until you're sure it'll work.

A silent failure wouldn't be useful - no indication of success or failure 
would be even more confusing, IMO.

>>> For example, to make things work, you need to up date the executable
>>> and three dynamic libraries. You update the executable, and one of the
>>> libraries, and then your network connection to the machine hosting the
>>> files fails. You can't handle that exception by rolling back your
>>> changes manually.
>> 
>> You can by reinstalling the package, though.
> 
> Only after you get your network connection back. Until then, everyone
> using the machine locally is screwed.

A good case to be made for doing updates on the local machine rather than 
the remote machine.  Like I said, management procedures in place for non-
standard deployments.

>>> Well, yes. That's why you have file system transactions in Windows.
>>> That's kinda exactly my point. If you start a kernel transaction, copy
>>> some files, update the registry, then bomb out, your changes get
>>> rolled back automatically. Just like any other database system, and
>>> regardless of why you bombed out or over which network the files are
>>> mounted.
>> 
>> And on Linux you have ext3, reiser, jfs, and other journaling
>> filesystems.
> 
> 
> Cool. What's the system call in Linux that lets me change three files
> consistemtly?  I.e., I have files /tmp/One, /tmp/Two, and /tmp/Three,
> and I want to rename them respectively to /tmp/1, /tmp/2, and /tmp/3,
> and I never want any possibility of an "ls" operation on the /tmp/
> directory to show my /tmp/One and /tmp/3 at the same time, or /tmp/1 and
> /tmp/Three.  Is there some way to accomplish that?

Again, if it's handled as part of the system update, the updater takes 
care of that for you, not the system.

>>> Unless the error is "you pulled the plug" or "the RAID fell over" or
>>> something like that.
>> 
>> If RAID falls over, I'd say you've got a more serious problem on your
>> hands....I've had that happen a few times, and it generally leads to a
>> system reinstall.
> 
> And you've never lost power either, I take it?

Got a system here with a bad power supply, as a matter of fact - and yes, 
it screwed a VMDK file that was open at the time.

You want to implement a filesystem as a database, knock yourself 
out. ;-)  It's overkill for most applications.

>> Sure, and that's what package validation is for
> 
> Except that's a manual post-hoc fix that only works for package
> installs. I'm talking about something that works without intervention on
> a more generic kind of problem.

....

Jim

Post a reply to this message

Jim Henderson wrote:
> I prefer that the system let me know about an error condition so I can 
> correct the underlying issue, when it comes to things like filesystem 
> errors.

Of course you find out about the error. I'd just prefer the system to not 
corrupt the file system when something fails.

> A silent failure wouldn't be useful - no indication of success or failure 
> would be even more confusing, IMO.

What makes you think it would be silent??

I take it you've never used a database that has transactions. What happens 
is you get back a message saying "your transaction failed because the disk 
was full" (or whatever the reason), and you don't have to do anything to put 
your changes back to how they were before you started. So if you say (for 
example) "delete all the .doc files from this directory", and one of them 
isn't deletable, none of them get deleted, and you're told which one can't 
be deleted.

> A good case to be made for doing updates on the local machine rather than 
> the remote machine.  Like I said, management procedures in place for non-
> standard deployments.

Right. That's what I'm saying.  Management procedures to work around the 
lack of functionality.

>> Cool. What's the system call in Linux that lets me change three files
>> consistemtly?  I.e., I have files /tmp/One, /tmp/Two, and /tmp/Three,
>> and I want to rename them respectively to /tmp/1, /tmp/2, and /tmp/3,
>> and I never want any possibility of an "ls" operation on the /tmp/
>> directory to show my /tmp/One and /tmp/3 at the same time, or /tmp/1 and
>> /tmp/Three.  Is there some way to accomplish that?
> 
> Again, if it's handled as part of the system update, the updater takes 
> care of that for you, not the system.

That didn't answer the question.  How does the updater make sure that you 
never see a half-renamed collection of files, even assuming we're only 
talking about the updater here?

> You want to implement a filesystem as a database, knock yourself 
> out. ;-)  It's overkill for most applications.

Not really. ext3 and reiser and all that are already implemented with 
transactions. You just can't get to them.

Post a reply to this message

Jim Henderson wrote:
> Logically it doesn't make sense - since offsets change in a library and 
> so on when compiling code.....

I haven't the slightest idea why that would have anything to do with the 
file-system level of whether you delete the file before you recreate it or not?

There's good reasons *not* to do that too, like not messing up permissions 
and symlinks.

> And in my test, no, it didn't use the same inode for different 
> compilations of the same C file.

So you ran
   gcc xyz.c
   ls -i a.out
   gcc xyz.c
   ls -i a.out
and you got two different inodes for a.out?

Post a reply to this message

On Tue, 25 Nov 2008 17:30:32 -0800, Darren New wrote:

> Jim Henderson wrote:
>> Logically it doesn't make sense - since offsets change in a library and
>> so on when compiling code.....
> 
> I haven't the slightest idea why that would have anything to do with the
> file-system level of whether you delete the file before you recreate it
> or not?
> 
> There's good reasons *not* to do that too, like not messing up
> permissions and symlinks.
> 
>> And in my test, no, it didn't use the same inode for different
>> compilations of the same C file.
> 
> So you ran
>    gcc xyz.c
>    ls -i a.out
>    gcc xyz.c
>    ls -i a.out
> and you got two different inodes for a.out?

Modified the code inbetween.  If the compiled code doesn't change, then 
the point's moot - you'd be building the same library and there'd be no 
difference, so who cares if it gets overwritten or not?

Jim

Post a reply to this message

Jim Henderson wrote:
> Modified the code inbetween.  If the compiled code doesn't change, then 
> the point's moot - you'd be building the same library and there'd be no 
> difference, so who cares if it gets overwritten or not?

I'm not sure that answered the question, tho.  I get the same inode for both 
a.outs.  Anyway, you care if the code gets overwritten because if it's 
already running (and it has a shared text segment) you can't overwrite it, 
and if it's not already running, you can't start it when it's being written.

Post a reply to this message