On Tue, 25 Nov 2008 22:16:19 -0800, Darren New wrote:

> Jim Henderson wrote:
>> Modified the code inbetween.  If the compiled code doesn't change, then
>> the point's moot - you'd be building the same library and there'd be no
>> difference, so who cares if it gets overwritten or not?
> 
> I'm not sure that answered the question, tho.  I get the same inode for
> both a.outs.  Anyway, you care if the code gets overwritten because if
> it's already running (and it has a shared text segment) you can't
> overwrite it, and if it's not already running, you can't start it when
> it's being written.

Your point is that updating a file while it's open and has a shared text 
segment would cause a problem.  My point is that if you're *updating* the 
file, the inode changes *unless* the file is being overwritten by an 
unmodified version of that file (and that's probably not likely to happen 
often), so if the file doesn't get updated with an identical copy of 
itself, big deal.  The file is as it should be regardless of whether it 
gets overwritten with an identical copy if its previous version or not.

The SUSE updater, though, applies patches, so only files that have 
changed would get updated.......so again, not seeing what the problem is 
here.

Jim

Post a reply to this message

On Tue, 25 Nov 2008 17:28:49 -0800, Darren New wrote:

> Jim Henderson wrote:
>> I prefer that the system let me know about an error condition so I can
>> correct the underlying issue, when it comes to things like filesystem
>> errors.
> 
> Of course you find out about the error. I'd just prefer the system to
> not corrupt the file system when something fails.

Corrupting a single file isn't exactly the same thing as corrupting the 
filesystem.....cf reference to our previously crashed RAID arrays.

>> A silent failure wouldn't be useful - no indication of success or
>> failure would be even more confusing, IMO.
> 
> What makes you think it would be silent??

That seemed to be what you were promoting.  My mistake if I misread.

> I take it you've never used a database that has transactions. What
> happens is you get back a message saying "your transaction failed
> because the disk was full" (or whatever the reason), and you don't have
> to do anything to put your changes back to how they were before you
> started. So if you say (for example) "delete all the .doc files from
> this directory", and one of them isn't deletable, none of them get
> deleted, and you're told which one can't be deleted.

I've dealt with transactional systems quite a bit, now I see what you 
were saying - treat the "package" update as a single transaction.  
Somehow I missed that before.  I guess it's been a busy week trying to 
wrap things up before the holiday. :-)

>> A good case to be made for doing updates on the local machine rather
>> than the remote machine.  Like I said, management procedures in place
>> for non- standard deployments.
> 
> Right. That's what I'm saying.  Management procedures to work around the
> lack of functionality.

But there again we're talking about configurations that aren't terribly 
common these days - they used to be, sure - I can remember Sun systems 
with diskless workstations because the versions with disks were so much 
more expensive - so the system was completely remotely booted from a 
4/290.

>>> Cool. What's the system call in Linux that lets me change three files
>>> consistemtly?  I.e., I have files /tmp/One, /tmp/Two, and /tmp/Three,
>>> and I want to rename them respectively to /tmp/1, /tmp/2, and /tmp/3,
>>> and I never want any possibility of an "ls" operation on the /tmp/
>>> directory to show my /tmp/One and /tmp/3 at the same time, or /tmp/1
>>> and /tmp/Three.  Is there some way to accomplish that?
>> 
>> Again, if it's handled as part of the system update, the updater takes
>> care of that for you, not the system.
> 
> That didn't answer the question.  How does the updater make sure that
> you never see a half-renamed collection of files, even assuming we're
> only talking about the updater here?

Because it deals with the individual package updates one RPM or delta 
file at a time.

>> You want to implement a filesystem as a database, knock yourself out.
>> ;-)  It's overkill for most applications.
> 
> Not really. ext3 and reiser and all that are already implemented with
> transactions. You just can't get to them.

Where there's a will, there's a way - it is all open source code, so it 
wouldn't be impossible to implement for someone who was really determined 
to do so.  :-)

Jim

Post a reply to this message

Jim Henderson wrote:
> Your point is that updating a file while it's open and has a shared text 
> segment would cause a problem. 

No, my point is that there are numerous ways that things can fail, and if 
your OS doesn't supply the right tools, you wind up writing ad-hoc manual 
work-arounds to recover when the program tells you it failed.

> My point is that if you're *updating* the 
> file, the inode changes *unless* the file is being overwritten by an 
> unmodified version of that file 

Uh, not according to my runs of gcc. Maybe the SuSE updater works that way, 
but that just reinforces my point: they have to have an ad-hoc 
application-specific work-around for the problem.

> The SUSE updater, though, applies patches, so only files that have 
> changed would get updated.......so again, not seeing what the problem is 
> here.

That you're discussing something different from what I'm trying to talk about?

You're failing to understand that I'm pointing out something entirely 
different than what you're arguing.  I'm pointing out that UNIX has 
traditionally addressed limitations in UNIX with ad-hoc manual and/or 
application-specific work-arounds, while most other systems incorporate into 
the operating system those operations that most applications need. When I 
say "it doesn't need to be this bad," you keep answering with "in this one 
case, it's possible to work around the problems most of the time if you're 
careful."  That isn't the point, tho.

--
   Darren New, San Diego CA, USA (PST)
   The NFL should go international. I'd pay to
   see the Detroit Lions vs the Roman Catholics.

Post a reply to this message

Jim Henderson wrote:
> Corrupting a single file isn't exactly the same thing as corrupting the 
> filesystem.....cf reference to our previously crashed RAID arrays.

No. But since the file system already supports not corrupting the file 
system, why wouldn't it support not corrupting the file?

> I've dealt with transactional systems quite a bit, now I see what you 
> were saying - treat the "package" update as a single transaction.  

Yes, exactly. Which is hard to do on a file system that doesn't support 
transactions, because UNIX traditionally layered transactions and locking on 
top of the file system in an application-specific way, just like it layers 
"login" on top of the permission system in an application-specific way.

Hence, you have people creating unwritable lock files to lock things, and 
the file system can't support things like temp files that only survive until 
you log out, and things like that.

> But there again we're talking about configurations that aren't terribly 
> common these days 

Sure.  But again, you're looking at the one example and failing to generalize.

>>>> Cool. What's the system call in Linux that lets me change three files
>>>> consistemtly?  I.e., I have files /tmp/One, /tmp/Two, and /tmp/Three,
>>>> and I want to rename them respectively to /tmp/1, /tmp/2, and /tmp/3,
>>>> and I never want any possibility of an "ls" operation on the /tmp/
>>>> directory to show my /tmp/One and /tmp/3 at the same time, or /tmp/1
>>>> and /tmp/Three.  Is there some way to accomplish that?
>>> Again, if it's handled as part of the system update, the updater takes
>>> care of that for you, not the system.
>> That didn't answer the question.  How does the updater make sure that
>> you never see a half-renamed collection of files, even assuming we're
>> only talking about the updater here?
> 
> Because it deals with the individual package updates one RPM or delta 
> file at a time.

You still didn't answer the question. Look at the actual question:
     What's the system call in Linux that lets me change three files
     consistently?
You said the Linux journaling file systems let you have file system 
transactions. I believe it might be there, but I've never found it; it might 
be something new. What's the system call?

> Where there's a will, there's a way - it is all open source code, so it 
> wouldn't be impossible to implement for someone who was really determined 
> to do so.  :-)

Sure. But you implied it's already there.

I just pine for the days when it was possible to write a program that ran 
reliably even in edge conditions, rather than being impossible to ensure 
that things work correctly because the underlying systems are so half-ass. :-)

--
   Darren New, San Diego CA, USA (PST)
   The NFL should go international. I'd pay to
   see the Detroit Lions vs the Roman Catholics.

Post a reply to this message

On Wed, 26 Nov 2008 10:22:44 -0800, Darren New wrote:

> Jim Henderson wrote:
>> Your point is that updating a file while it's open and has a shared
>> text segment would cause a problem.
> 
> No, my point is that there are numerous ways that things can fail, and
> if your OS doesn't supply the right tools, you wind up writing ad-hoc
> manual work-arounds to recover when the program tells you it failed.
> 
>> My point is that if you're *updating* the file, the inode changes
>> *unless* the file is being overwritten by an unmodified version of that
>> file
> 
> Uh, not according to my runs of gcc. Maybe the SuSE updater works that
> way, but that just reinforces my point: they have to have an ad-hoc
> application-specific work-around for the problem.

You didn't modify the code between compilation runs.  I did - and got 
different inodes.

>> The SUSE updater, though, applies patches, so only files that have
>> changed would get updated.......so again, not seeing what the problem
>> is here.
> 
> That you're discussing something different from what I'm trying to talk
> about?

I'm keeping to the initial point of discussion, which was a perceived 
problem with the SUSE updater application.

Jim

Post a reply to this message

On Wed, 26 Nov 2008 10:29:51 -0800, Darren New wrote:

> Jim Henderson wrote:
>> Corrupting a single file isn't exactly the same thing as corrupting the
>> filesystem.....cf reference to our previously crashed RAID arrays.
> 
> No. But since the file system already supports not corrupting the file
> system, why wouldn't it support not corrupting the file?

<shrug>  Maybe it does.  I'm not a kernel developer.

>> I've dealt with transactional systems quite a bit, now I see what you
>> were saying - treat the "package" update as a single transaction.
> 
> Yes, exactly. Which is hard to do on a file system that doesn't support
> transactions, because UNIX traditionally layered transactions and
> locking on top of the file system in an application-specific way, just
> like it layers "login" on top of the permission system in an
> application-specific way.

Just at a guess, abstraction layers tend to impact performance.  Look at 
the performance of a fuse-based NFS server vs. a kernel-based NFS server.

>>>>> Cool. What's the system call in Linux that lets me change three
>>>>> files consistemtly?  I.e., I have files /tmp/One, /tmp/Two, and
>>>>> /tmp/Three, and I want to rename them respectively to /tmp/1,
>>>>> /tmp/2, and /tmp/3, and I never want any possibility of an "ls"
>>>>> operation on the /tmp/ directory to show my /tmp/One and /tmp/3 at
>>>>> the same time, or /tmp/1 and /tmp/Three.  Is there some way to
>>>>> accomplish that?
>>>> Again, if it's handled as part of the system update, the updater
>>>> takes care of that for you, not the system.
>>> That didn't answer the question.  How does the updater make sure that
>>> you never see a half-renamed collection of files, even assuming we're
>>> only talking about the updater here?
>> 
>> Because it deals with the individual package updates one RPM or delta
>> file at a time.
> 
> You still didn't answer the question. Look at the actual question:
>      What's the system call in Linux that lets me change three files
>      consistently?
> You said the Linux journaling file systems let you have file system
> transactions. I believe it might be there, but I've never found it; it
> might be something new. What's the system call?

I'm not a kernel developer, so I don't know what the system call is.  
That would be a question for the kernel developers. :-)

>> Where there's a will, there's a way - it is all open source code, so it
>> wouldn't be impossible to implement for someone who was really
>> determined to do so.  :-)
> 
> Sure. But you implied it's already there.

It seems to be IME.  Maybe that's because it's an application-specific 
solution implemented - but I point back to the start of the discussion 
where you cited problems with the SUSE updater application.  I was 
answering that question (at least initially), not getting into a 
discussion of kernel-level system calls for transactional dealings with 
multiple files, because I'm not a kernel guru.

> I just pine for the days when it was possible to write a program that
> ran reliably even in edge conditions, rather than being impossible to
> ensure that things work correctly because the underlying systems are so
> half-ass. :-)

Well, most development is done to the 80/20 rule rather than to "edge 
conditions" for economic reasons (at least that's what I'm told).  OSS is 
a wonderful thing, though, because those who want to develop to those 
"edge conditions" can do so.  :-)

Jim

Post a reply to this message

Jim Henderson wrote:
> You didn't modify the code between compilation runs.  I did - and got 
> different inodes.

Of course I did.

> I'm keeping to the initial point of discussion, which was a perceived 
> problem with the SUSE updater application.

OK.

-- 
   Darren New, San Diego CA, USA (PST)
   The NFL should go international. I'd pay to
   see the Detroit Lions vs the Roman Catholics.

Post a reply to this message

Jim Henderson wrote:
> I'm keeping to the initial point of discussion, which was a perceived 
> problem with the SUSE updater application.

Actually, it was along the lines of "I wish the Suse updater let me have 
multiple programs accessing the database it keeps." :-) Since BDB apparently 
allows for multiple readers and writers, I'm not sure there's any 
infrastructure reason the updater wouldn't.

-- 
   Darren New, San Diego CA, USA (PST)
   The NFL should go international. I'd pay to
   see the Detroit Lions vs the Roman Catholics.

Post a reply to this message

On Wed, 26 Nov 2008 17:46:59 -0800, Darren New wrote:

> Jim Henderson wrote:
>> I'm keeping to the initial point of discussion, which was a perceived
>> problem with the SUSE updater application.
> 
> Actually, it was along the lines of "I wish the Suse updater let me have
> multiple programs accessing the database it keeps." :-) Since BDB
> apparently allows for multiple readers and writers, I'm not sure there's
> any infrastructure reason the updater wouldn't.

On that point we are agreed. :-)

Jim

Post a reply to this message

On Wed, 26 Nov 2008 14:45:50 -0800, Darren New wrote:

> Jim Henderson wrote:
>> You didn't modify the code between compilation runs.  I did - and got
>> different inodes.
> 
> Of course I did.

That's not what I read in your post.  I duplicated with the code staying 
the same, but not with it changing.

> 
>> I'm keeping to the initial point of discussion, which was a perceived
>> problem with the SUSE updater application.
> 
> OK.

Jim

Post a reply to this message