OK, so as you guys know, I now write computer software for a living. The 
installation DVD for this thing used to be hand-crafted, but thanks to 
[mostly] my efforts it's now auto-generated by a script. Anyway, 
obviously if you completely replace the installation system, you need to 
go test it. So we're testing it.

So yesterday the tester is trying out the software on the various 
platforms we support it on. And he starts complaining that on one 
specific model of laptop, it's not working properly. He claims the 
installer gets to 60% of the first step, and then skips the rest of that 
step and skips the next step completely and just says "installation 
successful".

So I burn a copy of the same DVD and try installing it on another laptop 
of the same model. It works perfectly. But the tester is insistent; he 
hands me the laptop with the DVD still in the drive. I run the 
installer, and it appears to work fine. But then, around about 65%, it 
suddenly "completes", just like the guy said. That's odd.

The install disk is a live Linux environment that runs a Bash script to 
do the installation. It does some crazy multi-way piping and redirecting 
to get the progress display to work. (Most Linux commands helpfully 
provide absolutely no feedback whatsoever, which isn't very good for an 
operation that takes 10 minutes to complete...) With all this piping 
going on, it wouldn't surprise me if some command somewhere is emitting 
an error message and it's simply been "lost" somewhere. A bit alarming 
that the script still claims that "installation was successful" though...

At this point I'm wondering if maybe the disk has a scratch on it which 
prevents it reading past 65% of the image file. So I boot the disk and 
do an md5sum of both image files. They're both fine. Hmm. I manually run 
the key command that actually does the installation. Initially I'm 
getting 150 MB/sec - which is odd, given that the internal SSD device 
maxes out at about 40 MB/sec. After a few minutes, the speed drops to 
about 10 MB/sec - a more usual number. And then the command simply 
/stops/, with no indication as to why. It claims to have completed, but 
the amount of data copied is clearly too low.

So I take the DVD out and put in the one I've been testing with. I 
notice the DVD I took out has a slightly older version of the image 
file. But the installation script is identical, so that shouldn't matter 
at this stage.

I run the installer again, and again it fails the same way. I run it 
manually, and again it fails the same way. Now I'm wondering if I've 
somehow burned out that sector on the SSD or something. (But surely 
wear-levelling would... hmm, anyway.) So I look inside the harddrive bay 
to see how old the drive is...



At this point, things get WEIRD! I look in the drive bay, and see... 
daylight. It turns out there's NOTHING IN THERE. There *is* no harddrive!

This is not /that/ unusual; we do occasionally swap drives around or 
take drives out of machines. So somehow our test guy ended up with a 
laptop with no drive. That's not especially surprising.

But... so... if there's no drive, WHAT THE HELL IS THE INSTALLER 
INSTALLING TO?!? O_O

Why does it sit there for 5 minutes apparently working perfectly when 
there's NO DRIVE PRESENT?!

My collegue suggested that maybe someone had left an SD card in the 
internal card reader. But no. Man, that would have been fun though!

So I open up the Bash script and start reading. This is one of the few 
parts that I didn't write. (It was written early on, when I had hardly 
learned how to do Bash, so my boss else wrote it.) The programmers among 
you will appreciate this: The script reports an error if the number of 
devices connected is GREATER THAN ONE. But the script neglects to check 
for the possibility that the number of devices is LESS THAN ONE. (!) 
Because, hey, who the hell has a laptop with no harddrive in it?

OK, well that's fine I guess, but next the script checks that the 
[assumed to be] one device has sufficient space. Surely a non-existent 
device has zero space on it, right? RIGHT?? How in the HELL is this test 
passing?

Ah, but wait. You're thinking about this as if you're using a REAL 
programming language. Bash is just text-munging. It doesn't compare the 
contents of a variable to a number. It compares two bits of text to each 
other. And one of those pieces of text is now "error: the device 
/dev/sda does not exist", which is more characters than "80000000000", 
and hence is reported as being "greater than" the required disk size. 
FACEPALM!

So there's no error produced by the script. But how the HELL does it 
image a device that DOESN'T SODDING EXIST?!

Ah, but wait. This is Linux, remember? Consider the following command:

   cat Image1.raw.gz | gzip -d | dd of=/dev/sda

Slowly it dawned on my what is happening here. If there *is* a block 
device connected, then UDev will generate a special file named /dev/sda 
which represents this device, and the above command will overwrite the 
contents of that device. HOWEVER... if there is *no* block device 
connected, then this file will not exist. In that case, rather than 
produce some kind of error, the above command will *create* a regular 
file named /dev/sda and try to decompress 20GB of data into it. (!)

Once upon a time, that might not have worked. A DVD is a read-only 
device, after all. But in this modern age, you have the DVD, which is 
read-only, and you then overlay a read/write filesystem backed by RAM.

In summary, the installer is decompressing a 20GB disk image into RAM, 
hence the 150MB/sec transfer speed. (Probably limited by DVD drive speed 
and the rate at which the CPU can decompress the data.) After about 5GB, 
free RAM has become so fragmented that the transfer rate drops to 
10MB/sec as the OS desperately searches for empty pages. And eventually, 
once ALL AVAILABLE RAM HAS BEEN EXHAUSTED, the process is summarily 
terminated.

If this was written in a real programming language, some sort of 
exception would have been thrown, which would have alerted me to the 
problem (and prevented the "installation successful" message being 
shown). But this is Bash. By default, it completely ignores all errors, 
problems and malfunctions, and continues executing the next command as 
if everything worked perfectly. So when DD gets terminated, Bash simply 
executes the next line of the script - which says "installation successful"!

I told my boss, and he spent literally 15 minutes laughing 
uncontrollably. There's always a lot of banter in our office, but it's 
unusual for somebody to actually find something so funny that they're 
actually unable to speak any more. And for 15 minutes?

Damn, this is probably the most amusing bug I've ever seen.

Fortunately, the fix is very simple. You just need to add a check for 
the possibility of there being zero devices. But I love the way that 
there are three separate stages where the problem *should* have been 
caught, but wasn't. And all because we wrote the thing in Bash...

Also, somebody give that tester a medal. There's no way in hell we would 
have thought to actually *test* for such an obscure condition. (Not that 
the tester did so intentionally... It was simply a lucky accident. But 
you *know* some customer somewhere is going to do this one day, and 
saying something succeeded when it didn't is a pretty serious bug!)

Post a reply to this message

Am 23.02.2013 12:34, schrieb Orchid Win7 v1:

> Also, somebody give that tester a medal. There's no way in hell we would
> have thought to actually *test* for such an obscure condition. (Not that
> the tester did so intentionally... It was simply a lucky accident. But
> you *know* some customer somewhere is going to do this one day, and
> saying something succeeded when it didn't is a pretty serious bug!)

Heh - I once had a work colleague like that: Be it karma, a genetic 
disposition, or - as he used to put it - having "shit on his fingers" - 
he had a supernatural talent for having things go wrong on him. He 
actually took quite some pride in this mysterious gift of his, because 
yes, of course, he did work as a tester. I swear this guy was /born/ to 
test the holy crap out of things.

Post a reply to this message

On 23/02/2013 1:01 PM, clipka wrote:
> Heh - I once had a work colleague like that: Be it karma, a genetic
> disposition, or - as he used to put it - having "shit on his fingers" -
> he had a supernatural talent for having things go wrong on him. He
> actually took quite some pride in this mysterious gift of his, because
> yes, of course, he did work as a tester. I swear this guy was /born/ to
> test the holy crap out of things.

I can do that. If anything can be broken, I can break it. ;-)

-- 
Regards
     Stephen

Post a reply to this message

Am 23.02.2013 14:40, schrieb Stephen:
> On 23/02/2013 1:01 PM, clipka wrote:
>> Heh - I once had a work colleague like that: Be it karma, a genetic
>> disposition, or - as he used to put it - having "shit on his fingers" -
>> he had a supernatural talent for having things go wrong on him. He
>> actually took quite some pride in this mysterious gift of his, because
>> yes, of course, he did work as a tester. I swear this guy was /born/ to
>> test the holy crap out of things.
>
> I can do that. If anything can be broken, I can break it. ;-)

Well, he didn't have to break things. Things broke by themselves at his 
merest presence ;-)

Post a reply to this message

Orchid Win7 v1 <voi### [at] devnull> wrote:

>
> At this point, things get WEIRD! I look in the drive bay, and see...
> daylight. It turns out there's NOTHING IN THERE. There *is* no harddrive!
>

That is too funny! The best laugh of the day.

Post a reply to this message

>> At this point, things get WEIRD! I look in the drive bay, and see...
>> daylight. It turns out there's NOTHING IN THERE. There *is* no harddrive!
>
> That is too funny! The best laugh of the day.

Like I said, it had us all rolling around on the floor...

Post a reply to this message

fun indeed

I mean, to put a windoze guy on charge of Linux stuff :)

Post a reply to this message

Le 2013-02-23 06:34, Orchid Win7 v1 a écrit :
> The install disk is a live Linux environment that runs a Bash script to
> do the installation. It does some crazy multi-way piping and redirecting
> to get the progress display to work. (Most Linux commands helpfully
> provide absolutely no feedback whatsoever, which isn't very good for an
> operation that takes 10 minutes to complete...) With all this piping
> going on, it wouldn't surprise me if some command somewhere is emitting
> an error message and it's simply been "lost" somewhere. A bit alarming
> that the script still claims that "installation was successful" though...

1. Redirect stderr to a different file.  And check for that file being 
more than 0 bytes before claiming the install completed succesfully.

2. Expect the unexpected.  I am currently having issues with a "serious" 
IT company over an install script that does not error checking and where 
there are multiple error-prone steps between zeroing out the old config 
and recreating a new one, which can leave the machine pretty much 
brain-dead if Something-Bad(tm) happens in the middle of the upgrade.

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

Francois Labreque <fla### [at] videotronca> wrote:
> 1. Redirect stderr to a different file.  And check for that file being 
> more than 0 bytes before claiming the install completed succesfully.

All programs should return an error code if an error happens. If a program
ends in error but returns a success code, that program is broken (and can
subsequently break other programs, such as 'make', which relies on programs
returning non-success on error.)

-- 
                                                          - Warp

Post a reply to this message

On 24/02/2013 12:06 AM, Warp wrote:
> All programs should return an error code if an error happens. If a program
> ends in error but returns a success code, that program is broken

I agree. However, unfortunately it seems that by default Bash ignores 
all such errors and happily proceeds, unless you manually suffix every 
single command with an explicit return-code check.

Post a reply to this message

On 23/02/2013 07:56 PM, nemesis wrote:
> fun indeed
>
> I mean, to put a windoze guy on charge of Linux stuff :)

I love that everybody else in the room is LPIC-certified, yet I'm the 
one doing the crazy Linux stuff. ;-)

Post a reply to this message

On 23/02/2013 10:24 PM, Francois Labreque wrote:

> 1. Redirect stderr to a different file. And check for that file being
> more than 0 bytes before claiming the install completed succesfully.

Ooo, here's an interesting question - what is the size of /dev/sda? Is 
it the size of the special file, or the device to which it refers?

Post a reply to this message

Orchid Win7 v1 <voi### [at] devnull> wrote:
> I agree. However, unfortunately it seems that by default Bash ignores 
> all such errors and happily proceeds, unless you manually suffix every 
> single command with an explicit return-code check.

How is that different from any programming language?

-- 
                                                          - Warp

Post a reply to this message

On 24/02/2013 12:03 PM, Warp wrote:
> Orchid Win7 v1<voi### [at] devnull>  wrote:
>> I agree. However, unfortunately it seems that by default Bash ignores
>> all such errors and happily proceeds, unless you manually suffix every
>> single command with an explicit return-code check.
>
> How is that different from any programming language?

In C#, Java or similar, if you try to open a file and can't, an 
exception is thrown, and your program is halted. (Unless you thought of 
that eventuality and coded a catch{} block for it.) In Bash, the program 
continues to execute as if everything worked just fine, silently 
ignoring the error.

Post a reply to this message

Am 24.02.2013 12:52, schrieb Orchid Win7 v1:
> On 24/02/2013 12:06 AM, Warp wrote:
>> All programs should return an error code if an error happens. If a
>> program
>> ends in error but returns a success code, that program is broken
>
> I agree. However, unfortunately it seems that by default Bash ignores
> all such errors and happily proceeds, unless you manually suffix every
> single command with an explicit return-code check.

#!/bin/bash -e

Have fun!

Post a reply to this message

On Sun, 24 Feb 2013 11:53:05 +0000, Orchid Win7 v1 wrote:

> On 23/02/2013 07:56 PM, nemesis wrote:
>> fun indeed
>>
>> I mean, to put a windoze guy on charge of Linux stuff :)
> 
> I love that everybody else in the room is LPIC-certified, yet I'm the
> one doing the crazy Linux stuff. ;-)

Just goes to show that certification doesn't always mean "expert".  As 
I've said before, a cert is a measure of the minimally-qualified 
candidate. :)

Well done. :)

Jim

Post a reply to this message

On Sun, 24 Feb 2013 12:21:51 +0000, Orchid Win7 v1 wrote:

> On 24/02/2013 12:03 PM, Warp wrote:
>> Orchid Win7 v1<voi### [at] devnull>  wrote:
>>> I agree. However, unfortunately it seems that by default Bash ignores
>>> all such errors and happily proceeds, unless you manually suffix every
>>> single command with an explicit return-code check.
>>
>> How is that different from any programming language?
> 
> In C#, Java or similar, if you try to open a file and can't, an
> exception is thrown, and your program is halted. (Unless you thought of
> that eventuality and coded a catch{} block for it.) In Bash, the program
> continues to execute as if everything worked just fine, silently
> ignoring the error.

The exception handling part is a feature of OO programming languages, 
IIRC.  Bash scripting is far from OOP. ;)

Jim

Post a reply to this message

>> I agree. However, unfortunately it seems that by default Bash ignores
>> all such errors and happily proceeds, unless you manually suffix every
>> single command with an explicit return-code check.
>
> #!/bin/bash -e
>
> Have fun!

I think you mean "set -e", but anyway...

Post a reply to this message

> The exception handling part is a feature of OO programming languages,
> IIRC.  Bash scripting is far from OOP. ;)

I think exception handling predates OO. But whatever; Bash is a 
_scripting language_. It is meant for writing short, simple scripts, but 
building large, complex, mission-critical applications.

Post a reply to this message

Orchid Win7 v1 <voi### [at] devnull> wrote:
> I think exception handling predates OO.

The first object-oriented programming language was Simula, developed
in 1967. When was exception handling first used?

-- 
                                                          - Warp

Post a reply to this message

On Sun, 24 Feb 2013 20:01:56 +0000, Orchid Win7 v1 wrote:

>> The exception handling part is a feature of OO programming languages,
>> IIRC.  Bash scripting is far from OOP. ;)
> 
> I think exception handling predates OO. 

Well, I'm referring to things like catching exceptions.  That's something 
that IIRC came with OOP.

Prior to that, for example, in C, if you opened a file and then tried to 
do something with it if the open() call failed, you'd end up segfaulting 
unless you checked for an exception.

So I guess, in a way, that is "exception handling", but I'm talking about 
things like the Python "try:" method, or hooking failures.  You could 
kinda do it in various languages in various ways, but there wasn't a very 
standardized way to do it.

I may not be explaining it well. :)

> But whatever; Bash is a
> _scripting language_. It is meant for writing short, simple scripts, but
> building large, complex, mission-critical applications.

That I absolutely agree with. :)

Jim

Post a reply to this message

Am 24.02.2013 20:49, schrieb Jim Henderson:
> On Sun, 24 Feb 2013 12:21:51 +0000, Orchid Win7 v1 wrote:
>
>> On 24/02/2013 12:03 PM, Warp wrote:
>>> Orchid Win7 v1<voi### [at] devnull>  wrote:
>>>> I agree. However, unfortunately it seems that by default Bash ignores
>>>> all such errors and happily proceeds, unless you manually suffix every
>>>> single command with an explicit return-code check.
>>>
>>> How is that different from any programming language?
>>
>> In C#, Java or similar, if you try to open a file and can't, an
>> exception is thrown, and your program is halted. (Unless you thought of
>> that eventuality and coded a catch{} block for it.) In Bash, the program
>> continues to execute as if everything worked just fine, silently
>> ignoring the error.
>
> The exception handling part is a feature of OO programming languages,
> IIRC.  Bash scripting is far from OOP. ;)

Exception handling is not an OOP feature. And yes, bash /does/ have it:

   set -e
   trap "echo aborted by user || kill -INT $$" INT
   trap "echo aborted by other process || kill -TERM $$" TERM
   trap "echo error reported" EXIT

     ... critical section ...

   set +e
   trap - INT
   trap - TERM
   trap - EXIT

Post a reply to this message

Am 24.02.2013 21:01, schrieb Orchid Win7 v1:
>>> I agree. However, unfortunately it seems that by default Bash ignores
>>> all such errors and happily proceeds, unless you manually suffix every
>>> single command with an explicit return-code check.
>>
>> #!/bin/bash -e
>>
>> Have fun!
>
> I think you mean "set -e", but anyway...

Allegedly the above syntax is supposed to work, too. Provided of course 
that it's the very first line of the script.

Post a reply to this message

>>> #!/bin/bash -e
>>>
>>> Have fun!
>>
>> I think you mean "set -e", but anyway...
>
> Allegedly the above syntax is supposed to work, too. Provided of course
> that it's the very first line of the script.

According to the manpage I read, it's supposed to work anywhere in the 
script, so you can turn the feature on and off at will.

I especially love how set -e turns it ON and set +e turns it OFF. I 
think that's really special...

Post a reply to this message

Funny story, interesting that Linux works that way - I wouldn't have 
known that (but then it took me almost a week to get my bitTorrent 
client write access to an external hard drive under Linux).

> Fortunately, the fix is very simple. You just need to add a check for
> the possibility of there being zero devices.

I guess you can use this as a lesson to try and dream up some more 
possible failure modes (eg something else uses up disk space whilst you 
are installing, or ...). It only takes a half hour session and you might 
be surprised what other things you can't handle.

Also it would seem to me to make sense to have a "catch all" type check 
at the end of the installation (eg count folders/files or file sizes or 
md5 on the supposedly installed folder) to really be 100% sure the 
installation worked. At least then if it fails for some crazy reason 
you'll catch it.

> Also, somebody give that tester a medal. There's no way in hell we would
> have thought to actually *test* for such an obscure condition.

You shouldn't give him a medal then... just saying.

Post a reply to this message

Le 2013-02-24 06:53, Orchid Win7 v1 a écrit :
> On 23/02/2013 10:24 PM, Francois Labreque wrote:
>
>> 1. Redirect stderr to a different file. And check for that file being
>> more than 0 bytes before claiming the install completed succesfully.
>
> Ooo, here's an interesting question - what is the size of /dev/sda? Is
> it the size of the special file, or the device to which it refers?

It's the size of the special file.  If you want to know the size on the 
device, you'll have to parse the output of df.

But that's not what I meant.  You redirect stderr to a file and then you 
check that THAT file's size is not zero with [ -s $ErrorLog ]

Come to think of it, you should also test that /dev/sba is a block 
device before trying to dd to it using [ -b /dev/sda ]



-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

On Sun, 24 Feb 2013 23:38:16 +0100, clipka wrote:

> Am 24.02.2013 20:49, schrieb Jim Henderson:
>> On Sun, 24 Feb 2013 12:21:51 +0000, Orchid Win7 v1 wrote:
>>
>>> On 24/02/2013 12:03 PM, Warp wrote:
>>>> Orchid Win7 v1<voi### [at] devnull>  wrote:
>>>>> I agree. However, unfortunately it seems that by default Bash
>>>>> ignores all such errors and happily proceeds, unless you manually
>>>>> suffix every single command with an explicit return-code check.
>>>>
>>>> How is that different from any programming language?
>>>
>>> In C#, Java or similar, if you try to open a file and can't, an
>>> exception is thrown, and your program is halted. (Unless you thought
>>> of that eventuality and coded a catch{} block for it.) In Bash, the
>>> program continues to execute as if everything worked just fine,
>>> silently ignoring the error.
>>
>> The exception handling part is a feature of OO programming languages,
>> IIRC.  Bash scripting is far from OOP. ;)
> 
> Exception handling is not an OOP feature. And yes, bash /does/ have it:
> 
>    set -e trap "echo aborted by user || kill -INT $$" INT trap "echo
>    aborted by other process || kill -TERM $$" TERM trap "echo error
>    reported" EXIT
> 
>      ... critical section ...
> 
>    set +e trap - INT trap - TERM trap - EXIT

Interesting, I didn't know that.  Thanks!

Jim

Post a reply to this message

>> Ooo, here's an interesting question - what is the size of /dev/sda? Is
>> it the size of the special file, or the device to which it refers?
>
> It's the size of the special file.

Figures.

> If you want to know the size on the
> device, you'll have to parse the output of df.

Only if the device is *mounted*. ;-)

I believe we used sdisk to do the job.

> Come to think of it, you should also test that /dev/sba is a block
> device before trying to dd to it using [ -b /dev/sda ]

Yes, that was the solution I chose in the end. It's an easy fix.

Post a reply to this message

>> Exception handling is not an OOP feature. And yes, bash /does/ have it:
>>
>>     set -e trap "echo aborted by user || kill -INT $$" INT trap "echo
>>     aborted by other process || kill -TERM $$" TERM trap "echo error
>>     reported" EXIT
>>
>>       ... critical section ...
>>
>>     set +e trap - INT trap - TERM trap - EXIT
>
> Interesting, I didn't know that.  Thanks!

It's truly terrifying just how much stuff you /can/ do with Bash.

(At this point I must remind you that just because you /can/ does NOT, 
in any way, imply that you /should/!!)

Post a reply to this message

On 25/02/2013 09:14 AM, scott wrote:
> Funny story, interesting that Linux works that way - I wouldn't have
> known that (but then it took me almost a week to get my bitTorrent
> client write access to an external hard drive under Linux).

Yeah. In the old days of Unix, there was a folder named /dev, and you 
used to run a command called mknod ("make node") which makes "files" 
which are really just ID numbers that tell the Unix kernel to access 
device X on bus Y. When you access, say, /dev/sda, you're really 
accessing the harddrive.

Windows does a similar thing, except it uses \\?\PhysicalDisk0 as the 
name. And this "file" exists if and only if disk 0 is actually plugged in.

Linux now uses a system called UDev. Instead of the OS installer running 
mknod to generate device nodes for every device that could ever be 
plugged in, the actual /dev folder is empty. The UDev software then 
dynamically creates the appropriate device node when I device is plugged 
in, and deletes it when the device is unplugged again. In this way, only 
nodes for devices that actually exist show up.

This is all very useful. But file redirection is such that if you 
redirect to an existing file, it is overwritten (the behaviour we want 
for our disk image), but if the file does /not/ exist, it gets created. 
And /dev isn't special; it's just a normal folder. The name /dev/sda is 
also not special; it's just a regular file.

All of this makes perfect sense really. It's just that if you tried this 
on Windows (i.e., tried to create \\?\PhysicalDisk7, say) it would tell 
you to go away and die. Linux, on the other hand, lets you do it just 
fine...

>> Fortunately, the fix is very simple. You just need to add a check for
>> the possibility of there being zero devices.
>
> I guess you can use this as a lesson to try and dream up some more
> possible failure modes (eg something else uses up disk space whilst you
> are installing, or ...). It only takes a half hour session and you might
> be surprised what other things you can't handle.

Oh I'm *sure* there must be other things that could fail. If the disk 
was scratched I'd expect some non-deterministic yet spectacular failure 
(not much we can do about that), if the target drive has bad sectors 
that'll probably silently fail too, pressing Ctrl+C will probably break 
something...

> Also it would seem to me to make sense to have a "catch all" type check
> at the end of the installation (eg count folders/files or file sizes or
> md5 on the supposedly installed folder) to really be 100% sure the
> installation worked. At least then if it fails for some crazy reason
> you'll catch it.

Fortunately, the install happens running under a Linux OS that I built. 
There are no other user applications running, and the user shouldn't be 
able to start any. (There's only one text terminal open, and you can't 
open any others.) But we'll see...

Post a reply to this message

Orchid Win7 v1 <voi### [at] devnull> wrote:
> It's truly terrifying just how much stuff you /can/ do with Bash.

Bah, it doesn't even compare to zsh.

For example zsh's file completion system can be quite awesome sometimes.

For example, suppose there are several files in the current directory,
all of them with +r rights for others, except for one. You start writing
"chmod o+r " and then press tab... and it completes to the only file for
which the command has a changing effect.

Or suppose that there are several files, only one of them is a .pdf, and
you write "acroread " and press tab... and it completes to that .pdf file
(even though there could be many other possibilities.)

Or you are deep inside a directory structure that's version-controlled by
git (the .git directory being far upward in the directory structure).
Among several files in the current directory, there's only one that's not
being tracked by git. You write "git add " and press tab... and you can
guess what happens.

Or you write "mplayer " and press tab, and it lists all possible
completions, including the different protocols that mplayer supports
(such as "cdda://", "dvd://", "http://", "tv://" and so on.)

Or you write "git commit -" and press tab... and it lists all possible
options that could follow.

Well, that's nothing still. You start writing "scp somehost:" and press
tab, zsh will ask for your ssh password for that host, and then will start
completing the remote files...

-- 
                                                          - Warp

Post a reply to this message

On Mon, 25 Feb 2013 14:59:22 -0500, Warp wrote:

> Well, that's nothing still. You start writing "scp somehost:" and press
> tab, zsh will ask for your ssh password for that host, and then will
> start completing the remote files...

tsch does that as well.

Jim

Post a reply to this message

On 25/02/2013 07:59 PM, Warp wrote:

> Or you are deep inside a directory structure that's version-controlled by
> git (the .git directory being far upward in the directory structure).
> Among several files in the current directory, there's only one that's not
> being tracked by git. You write "git add " and press tab... and you can
> guess what happens.

I did this yesterday in Bash. I typed "hg add " and pressed tab. Of all 
the dozens of files in the current folder, it immediately completed with 
the single (deeply nested) file that I hadn't added.

I have literally no idea how it does that.

Annoyingly, if you type "systemctl restart x" and hit tab, it fails to 
notice that xdm.service is the only possible completion...

...none of which has much to do with /scripting/ tasks, but anyway.

Post a reply to this message

Am 25.02.2013 20:15, schrieb Orchid Win7 v1:

> Oh I'm *sure* there must be other things that could fail. If the disk
> was scratched I'd expect some non-deterministic yet spectacular failure
> (not much we can do about that), if the target drive has bad sectors
> that'll probably silently fail too, pressing Ctrl+C will probably break
> something...

You can bet on that.

That's what "trap" is good for.

Question is, which subset of all the existing signals do you need to 
trap to be safe...

Post a reply to this message

Le 2013-02-25 15:01, Jim Henderson a écrit :
> On Mon, 25 Feb 2013 14:59:22 -0500, Warp wrote:
>
>> Well, that's nothing still. You start writing "scp somehost:" and press
>> tab, zsh will ask for your ssh password for that host, and then will
>> start completing the remote files...
>
> tsch does that as well.
>
> Jim
>
Begun the shell wars have.

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

> Windows does a similar thing, except it uses \\?\PhysicalDisk0 as the
> name. And this "file" exists if and only if disk 0 is actually plugged in.

Yes, I use the \\?\{<drive_guid>} syntax to make sure my backups always 
get written to the correct drive (with several hard drives and USB 
storage devices connected I am never sure if the drive letters will be 
repeatable).

> Linux now uses a system called UDev. Instead of the OS installer running
> mknod to generate device nodes for every device that could ever be
> plugged in, the actual /dev folder is empty. The UDev software then
> dynamically creates the appropriate device node when I device is plugged
> in, and deletes it when the device is unplugged again. In this way, only
> nodes for devices that actually exist show up.
>
> This is all very useful. But file redirection is such that if you
> redirect to an existing file, it is overwritten (the behaviour we want
> for our disk image), but if the file does /not/ exist, it gets created.
> And /dev isn't special; it's just a normal folder. The name /dev/sda is
> also not special; it's just a regular file.

I only have one annoyance left with my linux setup, I have the external 
USB drive connected most of the time to it and it always appears as 
/media/EXTERNAL (EXTERNAL is the drive name). However, if I boot it up 
*without* the drive connected once, it will never appear again unless I 
manually create the folder /media/EXTERNAL and reboot. I don't know 
enough about what goes on behind the scenes to prevent this happening.

Post a reply to this message

On 26/02/2013 08:16 AM, scott wrote:
> Yes, I use the \\?\{<drive_guid>} syntax to make sure my backups always
> get written to the correct drive (with several hard drives and USB
> storage devices connected I am never sure if the drive letters will be
> repeatable).

The GUID is so huge that I'd never use that. (And if you ever reformat 
the drive, I believe it'll change.) But *much* more reliable than using 
the disk number, drive letter or similar.

> I only have one annoyance left with my linux setup, I have the external
> USB drive connected most of the time to it and it always appears as
> /media/EXTERNAL (EXTERNAL is the drive name). However, if I boot it up
> *without* the drive connected once, it will never appear again unless I
> manually create the folder /media/EXTERNAL and reboot. I don't know
> enough about what goes on behind the scenes to prevent this happening.

When the system starts up, it reads /etc/fstab and mounts everything 
listed there (except entries marked as "noauto"). But it only does this 
on startup. You can probably do "mount /media/EXTERNAL" to make it to 
mount it any time you want...

...alternatively, some fluffy end-user helpy-helper tool is doing this, 
in which case all bets are off.

This is the greatest problem with Linux. It's based on Unix, which works 
in a defined way, but it's rather unfriendly. So now a mish-mash of 
incompatible tools sit on top of it trying to make it user-friendly. And 
it works great, most of the time. But when it breaks, suddenly you have 
to know all this arcane crap again to figure out what's up.

(Oh, and your helpy-helper tool will NOT have any documentation...)

Post a reply to this message

> The GUID is so huge that I'd never use that. (And if you ever reformat
> the drive, I believe it'll change.) But *much* more reliable than using
> the disk number, drive letter or similar.

It's 24 hex digits, so yeh a bit long to put in every line. What I do is 
do a mountvol to mount the external drive using the GUID to a temp 
folder on my C drive, then instruct robocopy to backup to that folder. 
It's not ideal (for several reasons) but robocopy doesn't seem to like 
paths with the GUID directly.

> When the system starts up, it reads /etc/fstab and mounts everything
> listed there (except entries marked as "noauto").

Yes I've put it in fstab to mount the way I need, which all works fine 
so long as I never boot it without the drive connected. If I do, the 
entry in fstab stays unchanged (I assume some warning or error is thrown 
somewhere), but next time I boot with the drive connected it doesn't 
appear as /media/EXTERNAL - I have to manually create that folder and 
then reboot (or manually mount it).

I suppose one workaround is to check that folder exists before fstab is 
read. You'd also need to check the drive is actually connected though, 
and then make that folder. Otherwise I'd end up with the same bug you 
had where suddenly /media/EXTERNAL is just a normal folder on the local 
storage. I have no idea how to do any of that though.

I miss on the Acorn where you could just reference drives by the name of 
the disk in the drive. In windows why can't I rename my external drive 
"backup" and then write to "backup:\", it even comes up in the Win7 
address bar in explorer...

Post a reply to this message

Le 2013-02-27 03:17, scott a écrit :
> I miss on the Acorn where you could just reference drives by the name of
> the disk in the drive. In windows why can't I rename my external drive
> "backup" and then write to "backup:\", it even comes up in the Win7
> address bar in explorer...
>

Because DOS.


-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

>> I miss on the Acorn where you could just reference drives by the name of
>> the disk in the drive. In windows why can't I rename my external drive
>> "backup" and then write to "backup:\", it even comes up in the Win7
>> address bar in explorer...
>
> Because DOS.

Maybe that's because on DOS/Windows hard drives came along before 
multi-tasking? So you never got the situation where you were swapping 
floppies around to do several things at once in a multi-tasking 
environment. In that situation it's very useful for the OS/app to know 
which disc to use, rather than just which drive to use (otherwise you 
end up saving your word processor document to your printer driver disc 
and then can't find the latest version the next day).

Post a reply to this message

Le 2013-03-04 04:37, scott a écrit :
>>> I miss on the Acorn where you could just reference drives by the name of
>>> the disk in the drive. In windows why can't I rename my external drive
>>> "backup" and then write to "backup:\", it even comes up in the Win7
>>> address bar in explorer...
>>
>> Because DOS.
>
> Maybe that's because on DOS/Windows hard drives came along before
> multi-tasking? So you never got the situation where you were swapping
> floppies around to do several things at once in a multi-tasking
> environment. In that situation it's very useful for the OS/app to know
> which disc to use, rather than just which drive to use (otherwise you
> end up saving your word processor document to your printer driver disc
> and then can't find the latest version the next day).
>

This sounds like a "been there, done that".

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

>> Maybe that's because on DOS/Windows hard drives came along before
>> multi-tasking? So you never got the situation where you were swapping
>> floppies around to do several things at once in a multi-tasking
>> environment. In that situation it's very useful for the OS/app to know
>> which disc to use, rather than just which drive to use (otherwise you
>> end up saving your word processor document to your printer driver disc
>> and then can't find the latest version the next day).
>
> This sounds like a "been there, done that".

Nope, I never went near a DOS/Windows machine until Windows 95 when hard 
drives were standard. When I first got my Acorn it only had 1MB RAM and 
no hard drive, so everything (apart from the OS which was in ROM) was on 
floppies. If you needed to edit a spreadsheet, create a chart and put it 
in a desktop publishing document then print it out (as was quite common 
for school homework), you'd need to be using at least 3 or 4 different 
discs and then it's important you saved all the stuff on the right disc! 
I have no idea how DOS/Windows handled such situations before everyone 
had hard drives.

Post a reply to this message

On 2/27/2013 10:12 AM, Francois Labreque wrote:
> Le 2013-02-27 03:17, scott a écrit :
>> I miss on the Acorn where you could just reference drives by the name of
>> the disk in the drive. In windows why can't I rename my external drive
>> "backup" and then write to "backup:\", it even comes up in the Win7
>> address bar in explorer...
>>
>
> Because DOS.
>
>
More like, "Because lazy". You can name drives under DOS, I think you 
could even do so with a floppy, but then your application has to 
"manually" check to make sure that the name is the same, since they a) 
never added a way to reference, as far as I know, the drive name as a 
means to write to it, and b) the OS doesn't check if the name changed on 
its own, as far as I am aware.

Post a reply to this message

On 3/6/2013 10:15 AM, Patrick Elliott wrote:
> On 2/27/2013 10:12 AM, Francois Labreque wrote:
>> Le 2013-02-27 03:17, scott a écrit :
>>> I miss on the Acorn where you could just reference drives by the name of
>>> the disk in the drive. In windows why can't I rename my external drive
>>> "backup" and then write to "backup:\", it even comes up in the Win7
>>> address bar in explorer...
>>>
>>
>> Because DOS.
>>
>>
> More like, "Because lazy". You can name drives under DOS, I think you
> could even do so with a floppy, but then your application has to
> "manually" check to make sure that the name is the same, since they a)
> never added a way to reference, as far as I know, the drive name as a
> means to write to it, and b) the OS doesn't check if the name changed on
> its own, as far as I am aware.

Of course, there is a bloody stupid "DOS" way to do it:

http://stackoverflow.com/questions/47849/refer-to-select-a-drive-based-only-on-its-label-i-e-not-the-drive-letter

Or the "useless" method:

A volume GUID path, for example, 
\\?\Volume{26a21bda-a627-11d7-9931-806e6f6e6963}\.

Or.. Maybe a UNC... Hmm. Ah, hah, this would work, with a few 
adjustments. The examples given on one page is:

http://compnetworking.about.com/od/windowsnetworking/g/unc-name.htm

"Consider a standard Windows XP computer named teela. In addition to the 
built-in admin$ share, say you have also defined a share point called 
temp that is located at C:\temp. Using UNC names, you would connect to 
folders on teela as follows:

\\teela\admin$ (to reach C:\WINNT)
\\teela\admin$\system32 (to reach C:\WINNT\system32)
\\teela\temp (to reach C:\temp)"

But, its not going to let you "access" it that way normally, since its 
only directing you to C: locations. If you make it a mapped network 
drive, then, you still sort of can't do it without the drive letter. 
**However** if you create an empty folder, called backup, on the C: 
drive, then you can "mount" your external drive to that folder, and then 
reference the folder as, "\\yournetworkname\backup\", at least, from 
what I can tell applying the information on UNC above, with the 
information from here:

http://serverfault.com/questions/8832/if-i-refer-to-a-local-external-usb-drive-by-its-network-unc-name-mycomputer

But, its overly damned complicated, given something that should be 
simple. But, then, this is Windows... lol

Post a reply to this message

> **However** if you create an empty folder, called backup, on the C:
> drive, then you can "mount" your external drive to that folder,

That's exactly what I do - mounting the external drive using its GUID so 
I'm sure it's always the correct drive. Whilst this works, there are a 
few "issues". Firstly I have no idea what will happen if the folder 
"backup" already exists and has data in it, will that data be 
unavailable whilst the drive is mounted to that folder and then reappear 
once the drive is unmounted? Also it seems that you need to run the 
command prompt or script as admin to get this to work, whereas if you 
write directly to the drive via its drive letter you don't. Thirdly you 
have to check errors carefully otherwise you end up physically writing 
your entire backup to drive C rather than the external drive.

All in all it would be *way* simpler if I could just write "robocopy ... 
EXTERNAL:\Backup /MIR".

Post a reply to this message

Am 07.03.2013 07:50, schrieb Patrick Elliott:

>> More like, "Because lazy". You can name drives under DOS, I think you
>> could even do so with a floppy, but then your application has to
>> "manually" check to make sure that the name is the same, since they a)
>> never added a way to reference, as far as I know, the drive name as a
>> means to write to it, and b) the OS doesn't check if the name changed on
>> its own, as far as I am aware.
>
> Of course, there is a bloody stupid "DOS" way to do it:
>
>
http://stackoverflow.com/questions/47849/refer-to-select-a-drive-based-only-on-its-label-i-e-not-the-drive-letter

Interestingly, the top answer has /nothing/ to do with DOS whatsoever. 
(Except of course for the raw fact that drive letters and drive labels 
are a DOS legacy, and the author chose to call his visual basic script a 
"bat file" for apparently nostalgic reasons.)

Post a reply to this message

Le 2013-03-07 03:26, clipka a écrit :
> Am 07.03.2013 07:50, schrieb Patrick Elliott:
>
>>> More like, "Because lazy". You can name drives under DOS, I think you
>>> could even do so with a floppy, but then your application has to
>>> "manually" check to make sure that the name is the same, since they a)
>>> never added a way to reference, as far as I know, the drive name as a
>>> means to write to it, and b) the OS doesn't check if the name changed on
>>> its own, as far as I am aware.
>>
>> Of course, there is a bloody stupid "DOS" way to do it:
>>
>>
http://stackoverflow.com/questions/47849/refer-to-select-a-drive-based-only-on-its-label-i-e-not-the-drive-letter
>>
>
> Interestingly, the top answer has /nothing/ to do with DOS whatsoever.
> (Except of course for the raw fact that drive letters and drive labels
> are a DOS legacy, and the author chose to call his visual basic script a
> "bat file" for apparently nostalgic reasons.)
>

And the second one returns "Bad command or file name", when run from a 
DOS6.1 boot floppy, so it doesn't have much to do with DOS either.  (Not 
that I was expecting it to work...)

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

On 3/7/2013 1:20 AM, scott wrote:
>> **However** if you create an empty folder, called backup, on the C:
>> drive, then you can "mount" your external drive to that folder,
>
> That's exactly what I do - mounting the external drive using its GUID so
> I'm sure it's always the correct drive. Whilst this works, there are a
> few "issues". Firstly I have no idea what will happen if the folder
> "backup" already exists and has data in it, will that data be
> unavailable whilst the drive is mounted to that folder and then reappear
> once the drive is unmounted? Also it seems that you need to run the
> command prompt or script as admin to get this to work, whereas if you
> write directly to the drive via its drive letter you don't. Thirdly you
> have to check errors carefully otherwise you end up physically writing
> your entire backup to drive C rather than the external drive.
>
> All in all it would be *way* simpler if I could just write "robocopy ...
> EXTERNAL:\Backup /MIR".
>
Not sure, but it may just fail if you try to mount to a non-empty 
folder. Alternatively, what it might do is create a logical 
concatenation of the contents, i.e., what ever is there will stay, but 
doing a listing of the directory will give you what is there *and* what 
is in the mounted drive. But, yeah, having the contents go unavailable, 
if not empty, is one possibility.

That said.. assuming you are running the commands in a script, I suppose 
there might be some way to check if the mounted folder has some 
attribute that shows it is actually a mount, but.. no clue how you would 
do that. :p I just thought the thing was interesting enough, especially 
if I needed to do it at some point, to hunt down what, if any, solutions 
existed.

Post a reply to this message

On 3/7/2013 7:21 AM, Francois Labreque wrote:
> Le 2013-03-07 03:26, clipka a écrit :
>> Am 07.03.2013 07:50, schrieb Patrick Elliott:
>>
>>>> More like, "Because lazy". You can name drives under DOS, I think you
>>>> could even do so with a floppy, but then your application has to
>>>> "manually" check to make sure that the name is the same, since they a)
>>>> never added a way to reference, as far as I know, the drive name as a
>>>> means to write to it, and b) the OS doesn't check if the name
>>>> changed on
>>>> its own, as far as I am aware.
>>>
>>> Of course, there is a bloody stupid "DOS" way to do it:
>>>
>>>
http://stackoverflow.com/questions/47849/refer-to-select-a-drive-based-only-on-its-label-i-e-not-the-drive-letter
>>>
>>>
>>
>> Interestingly, the top answer has /nothing/ to do with DOS whatsoever.
>> (Except of course for the raw fact that drive letters and drive labels
>> are a DOS legacy, and the author chose to call his visual basic script a
>> "bat file" for apparently nostalgic reasons.)
>>
>
> And the second one returns "Bad command or file name", when run from a
> DOS6.1 boot floppy, so it doesn't have much to do with DOS either.  (Not
> that I was expecting it to work...)
>
Great.. Might as well just try to run something via windows scripting 
host, for all that those "solutions" do any good then. At least with 
that you could, in theory, find some way to get around it.

Post a reply to this message