 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
that traditional OSes hold back the more sophisticated (as in, far from
machine language) languages.
http://www.artima.com/lejava/articles/azul_pauseless_gc.html
Traditional file system interfaces probably do too. For example, if you
wanted something like a virus scanner that was watching your executables, I
suspect that "scan the file when it gets passed to exec()" is probably a
much more common implementation than "scan each block between the time it is
paged in and the time the code branches to it", for example. (Indeed, I
don't know how you'd even do that latter on Linux or Windows or whatever.)
It's interesting that this sort of stuff is starting to get to the point
where people will be willing to break with compatibility at some level.
Phones, game consoles, set-top boxes, and eventually probably "enterprise"
or "cloud" type servers will all be willing to consider a different
operating system that puts limits on compatibility with previous languages
and libraries.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] san rrcom> wrote:
> Traditional file system interfaces probably do too. For example, if you
> wanted something like a virus scanner that was watching your executables, I
> suspect that "scan the file when it gets passed to exec()" is probably a
> much more common implementation than "scan each block between the time it is
> paged in and the time the code branches to it", for example. (Indeed, I
> don't know how you'd even do that latter on Linux or Windows or whatever.)
On the subject of virus scanners in particular, I'd say that the very
need to have such scanners is a symptom of fundamentally bad OS design.
Of course this isn't an original idea of mine, as such an idea has been
expressed numerous times by people more knowledgeable than me (and probably
even linked to in this very newsgroup in the past).
The unix philosophy of OS design has always been a step or two closer
to the safer design (with respect to computer viruses and other malware)
then the typical DOS/Windows (and other similar OS's in the past) design.
The reason for this is that unixes have always been designed to be
multi-user operating systems while DOS/Windows has been designed to be
a single-user OS with no regard to security. The very need to handle
multiple user automatically brings forth the need for security: You should
not be able to access other users' data without permission, and especially
you shouldn't be able to access the superusers' data without permission.
This causes security to be built into the system from the ground up.
The DOS/Windows design always took basically the exact opposite approach:
Whatever the user wants to run or do, the OS allows. It's not the system's
task to stop the user doing what he wants. Unfortunately it took over 20
years for Microsoft to rid itself of this mentality (for some reason MS
has always been very slow to adopt certain ideas). NT had security, but
it wasn't even intended for normal users. It wasn't until XP that some
*semblance* of security was introduced (yet, nevertheless, the mentality
of the regular user being by default the superuser was still there, and
probably 99% of XP users out there still use their machine with superuser
privileges). This made the spreading of viruses and malware *trivial*.
Not that the unix design is perfect, but at least viruses, worms and
other malware have always been, and still are, extremely rare in unix
systems in comparison (basically the only relatively successful worms
in the unix world have exploited bugs in the systems to spread themselves,
rather than relying on the users; fix the bug, and the worm stops; however,
in the single-user OS's it requires a very significantly more radical
change in design than just fixing a few bugs).
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp <war### [at] tagpovrayorg> wrote:
> The DOS/Windows design always took basically the exact opposite approach:
> Whatever the user wants to run or do, the OS allows. It's not the system's
> task to stop the user doing what he wants. Unfortunately it took over 20
> years for Microsoft to rid itself of this mentality
I think the quintessential example of this are email viruses.
If in the mid and early 90's you had talked about "email viruses",
especially to unix people, they would have laughed. Well, the very
concept *should* be equally laughable even today. The very concept should
be as ludicrous today as it was in the early 90's. Advise like "never
open attachments from emails sent by unknown people" should not have to
exist.
However, then came Microsoft. Some people today might not realize this,
but the first email viruses didn't actually exploit any bugs at all. They
just abused Microsoft's software design. Back then Microsoft's ideology
was still the antiquated "whatever the user wants to do, do". This included
"if the user wants to open an email attachment and watch it in the way as
watching those files has been configured in the system, do so". And yes,
if the attachment was an executable, it would be executed by simply opening
the attachment. This was *by design*. It was not a programming error (at
most one could argue it was a design oversight).
As said, Microsoft is, for some reason, quite slow at learning certain
things, and it seems that they still hadn't developed the "what if"
instinct. (In this particular, a basic question a developer could have
asked would have been "hey, what if someone sends someone else a program
as attachment, and the program deletes everything from the hard drive?"
Even if someone at Microsoft did ask such questions, they were obviously
ignored.)
Of course when Microsoft later put a few barricades to their email program
to slow down this, the email viruses started exploiting bugs in the software.
Naturally bugs are something we have to live with, but this *still* shows
the fundamentally wrong mentality in OS design: Even if the email software
is buggy, it should *still* be impossible for a virus to spread by abusing
this bug. The OS should be designed in such way that it's just not possible.
Virus scanners are fighting the symptoms, not the fundamental problem.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> On the subject of virus scanners in particular, I'd say that the very
> need to have such scanners is a symptom of fundamentally bad OS design.
While I agree it would be nice not to need such things, I don't think many
of the common OSes don't need it. Something like Singularity, where you
can't run code that does something "unsafe", where you have to reboot after
installing a new executable, and where every executable declares in the
manifest what system resources it'll need? Sure, much less virus-prone.
> The reason for this is that unixes have always been designed to be
> multi-user operating systems while DOS/Windows has been designed to be
> a single-user OS with no regard to security.
And yet, I got a virus from a web page while running in a non-privileged
account under Vista. (The first ever malware I ever myself contracted, in
any system since 8-bit computers, I'll mention.) Nowadays, malware attacks
individual accounts. You hit a web site, some bug in Firefox lets some piece
of Javascript write some garbage to some hidden directory, and you have
malware. Bazinga.
It's not like UNIX or IBM big iron never had a worm or virus. They just
weren't trying to hide at the time. I daresay if something like the Morris
worm got a toe-hold nowadays, it would be some time before someone noticed it.
> You should
> not be able to access other users' data without permission, and especially
> you shouldn't be able to access the superusers' data without permission.
> This causes security to be built into the system from the ground up.
But this has been the case with NT forever, and since XP, people haven't
been running superuser. The problem is that people bitch about the security,
complain they can't set the clock without typing a password, etc.
> The DOS/Windows design always took basically the exact opposite approach:
> Whatever the user wants to run or do, the OS allows.
That's kind of what happens when you don't have virtual memory mapping.
> This made the spreading of viruses and malware *trivial*.
It's still pretty trivial, methinks. You don't need to be superuser to
spread the kind of malware that spreads these days. People aren't looking to
take down your machine. They're looking to install a firefox extension that
records your bank logins and posts them to a hacker's web site. No admin
privileges needed for that at all.
> rather than relying on the users;
Nah. It's in three parts: (1) unix was and still is used primarily by people
who understand how computers work at least a little; (2) when people broke
into unix systems, they got 50,000 accounts, so they didn't really need to
propagate as much; and (3) the actual problem with trojans (which is where
most malware comes from these days) was solved before networking was ubiquitous.
For example, that was the whole "NT3.5 has C2 security" stuff came from:
there was an actual keystroke (C-A-D) that would uninterruptably ensure
you're talking to a specific program, e.g. the login prompt. UNIX had no
such feature - anything the login program could do, so could a user-level
program, except for the actual logging in part. It's why "." isn't in the
path by default on UNIX (any more). There are, basically, 101 fixes in UNIX
for security that was broken in spite of being multi-user. It's just that
UNIX is old enough and had multi-user *early* enough that *those* kind of
fixes got put in place before networking was common. Otherwise, I suspect
you'd see way more worms and such in UNIX, for the same reason that even now
XP and Vista and etc have more malware for them than UNIX does.
> in the single-user OS's it requires a very significantly more radical
> change in design than just fixing a few bugs).
Sure, but Windows hasn't been single-user for 10+ years.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> The OS should be designed in such way that it's just not possible.
The problem is people, really. You put in code that says "don't accept
executables." Then you put in code that says "Hey, you downloaded this, do
you really want to run it?" Then you put in code that says "Don't accept
executables inside zip files."
If it went far enough, people would send out "here's a zip file with the
password xyz. Unpack it with that password, rename hello.jpg to hello.exe,
and run it to get an important message from your bank" and someone would do it.
> Virus scanners are fighting the symptoms, not the fundamental problem.
True. And the symptom is that software is too complex for someone to know
all its interactions in modern systems, in part because software doesn't
come as a package. This is starting to change, but it's still not enforced.
It's more of a convenience feature than a security feature.
If you said "Firefox can only run executable code that you downloaded with
firefox and was signed by the firefox development key", you'd be golden.
Unless, of course, it's an interpreter.
I think fundamentally you're up against the halting problem, and the more
you lock down the system, the more people who *do* know what they're doing
(or who want to do something regardless of whether they know what they're
doing or not) will be annoyed.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> Warp wrote:
> > The OS should be designed in such way that it's just not possible.
> The problem is people, really. You put in code that says "don't accept
> executables." Then you put in code that says "Hey, you downloaded this, do
> you really want to run it?" Then you put in code that says "Don't accept
> executables inside zip files."
> If it went far enough, people would send out "here's a zip file with the
> password xyz. Unpack it with that password, rename hello.jpg to hello.exe,
> and run it to get an important message from your bank" and someone would do it.
Didn't seem to be such a problem in the unix world. And even if that
kind of social engineering caused someone to execute a program they
received by email, it would still be limited to that user's account.
The system itself and other users would be safe. It's a whole different
mentality. It's hard to spread viruses like that.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> > You should
> > not be able to access other users' data without permission, and especially
> > you shouldn't be able to access the superusers' data without permission.
> > This causes security to be built into the system from the ground up.
> But this has been the case with NT forever, and since XP, people haven't
> been running superuser. The problem is that people bitch about the security,
> complain they can't set the clock without typing a password, etc.
They wouldn't if Microsoft hadn't taught them the bad habits.
If the very first version of DOS had had a similar account/password
system as unixes, and this strict mentality had been dragged along (and
improved) in all subsequent versions of DOS and Windows, people today
would not complain because they would take it for granted, as something
obvious.
> > The DOS/Windows design always took basically the exact opposite approach:
> > Whatever the user wants to run or do, the OS allows.
> That's kind of what happens when you don't have virtual memory mapping.
The mentality prevailed well after the 80386 became the de-facto standard.
(And, in fact, even the 80286 supported some type of memory protection.)
> > in the single-user OS's it requires a very significantly more radical
> > change in design than just fixing a few bugs).
> Sure, but Windows hasn't been single-user for 10+ years.
Well, how long it took for Microsoft to *finally* get some semblance
of security into their desktop OS? (And the major reason why it took them
so long is not because of technical difficulties, but simply because they
just didn't bother.)
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Didn't seem to be such a problem in the unix world.
No, because most people running UNIX software knew the basics of how
computers work. They'd recognise this as a scam, rather than just another
series of opaque rules you have to follow to get something done on this damn
machine.
> And even if that
> kind of social engineering caused someone to execute a program they
> received by email, it would still be limited to that user's account.
Except that's why they took "." out of the path, and why setuid files aren't
allowed to be writable and why writing to them turns off setuid, neither of
which were security features even as late as V7. Exactly because enough
people fell for it that they made it inconvenient for developers to keep
those not paying attention from running a trojan.
> The system itself and other users would be safe.
Except if there's only one user on the system, like there often is with
Windows desktop machines, you're still screwed if your web browser starts
sending your bank passwords to russian mafia guys.
> It's a whole different
> mentality. It's hard to spread viruses like that.
It only takes one hole. How widely do you think the Morris worm could have
spread if he was being paid a million dollars to make it hibernate on the
computer? He'd write an executable somewhere, call it something benign,
start it up as root, then let it sit and listen for a particular string to
show up in a spam mail to DDOS some network computer.
I think what you're seeing is more that by the time it was *profitable* to
write a virus, Windows was already the best vector both in popularity and
user-naivety. Granted, it was pretty common back in the early days of
Windows to have a virus or whatever spread, but it was also much more common
to hear of break-ins of commercial servers to be stealing credit cards and such.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> They wouldn't if Microsoft hadn't taught them the bad habits.
Probably true. But that is left over from when Windows ran on machines that
weren't even capable of running UNIX and for which having a multi-user OS
made no sense.
> If the very first version of DOS had had a similar account/password
> system as unixes,
... then it wouldn't have run on an 8086, and MS would be broke.
> would not complain because they would take it for granted, as something
> obvious.
It's hard to say. Most of the other systems of the day didn't have it either.
>>> The DOS/Windows design always took basically the exact opposite approach:
>>> Whatever the user wants to run or do, the OS allows.
>
>> That's kind of what happens when you don't have virtual memory mapping.
>
> The mentality prevailed well after the 80386 became the de-facto standard.
Backwards compatibility is a bitch, indeed.
>> Sure, but Windows hasn't been single-user for 10+ years.
>
> Well, how long it took for Microsoft to *finally* get some semblance
> of security into their desktop OS?
Backwards compatibility is a bitch.
> (And the major reason why it took them
> so long is not because of technical difficulties, but simply because they
> just didn't bother.)
Sure. Welcome to the commercial market.
But this is kind of off the topic now, methinks. Or at least drifting into
another discussion.
UNIX isn't safe because it has had multi-user stuff longer. (Safer. Far from
"safe".) One slip by the superuser can screw things up. Anyone with the root
password can do anything they want and steal any data they want from anyone
else on the machine. Tricking the superuser into running your code, or
mounting a disk you have without turning off setuid on it, or a server with
a buffer overflow, or anything like that can corrupt the system in ways just
as hard to figure out.
Contrast with something like Singularity, where you explicitly list every
program you're going to run, every file they're going to access, there's no
way to run code that isn't signed, no way to get someone to overrun a buffer
to run code that wasn't in the executable to start with, and giving each
program its own set of permissions orthogonally to the users running the
program. *That* is a significant advance in security of an operating system.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Didn't seem to be such a problem in the unix world.
Thinking on it, there are a whole bunch of "patches" made in the UNIX world
to account for bad security. The whole thing with "xhost" and the magic
cookies. Taking "." out of the default path. Sticky bits on directories.
Shadow password files. The vipw command. The -print0 option on find.
Poisoned DNS caches (which affected everyone, really). All *kinds* of stuff
in UUCP. All kinds of race conditions for files in /tmp/. Why people
switched from rcp to scp. Why people switched from http to https. Why FTP
runs in a chroot cell, and still failed that on occasion if one
misconfigured it. Why it used to be easy to find thousands of /etc/passwd
files on the web when search engines first came out. Why Kerberos used to
let you download the whole list of encrypted passwords for off-line cracking
before you identified yourself.
It's just that most of the breaks these changes fixed happened before
networking was really ubiquitous, and before taking advantage of them was
worth huge amounts of money.
Yes, MS was somewhat late to the party, but it's not like UNIX escaped being
broken into over, and over, and over again, simply because it has always
been multi-user. It's only relatively recently that people in general
started worrying about these things on a systematic basis, because it has
only been relatively recently that it was more valuable to break into it
than getting your college class grades changed. ;-)
Oh, and by the way:
http://en.wikipedia.org/wiki/Christmas_Tree_EXEC
So, yeah. Would never happen on a *real* system.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New wrote:
> This is starting to change, but it's still
> not enforced. It's more of a convenience feature than a security feature.
http://developer.android.com/guide/topics/security/security.html#declaring
You need something like that to make it work for normal people.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 12/20/2010 11:41 AM, Warp wrote:
> Darren New<dne### [at] sanrrcom> wrote:
>> Traditional file system interfaces probably do too. For example, if you
>> wanted something like a virus scanner that was watching your executables, I
>> suspect that "scan the file when it gets passed to exec()" is probably a
>> much more common implementation than "scan each block between the time it is
>> paged in and the time the code branches to it", for example. (Indeed, I
>> don't know how you'd even do that latter on Linux or Windows or whatever.)
>
> On the subject of virus scanners in particular, I'd say that the very
> need to have such scanners is a symptom of fundamentally bad OS design.
> Of course this isn't an original idea of mine, as such an idea has been
> expressed numerous times by people more knowledgeable than me (and probably
> even linked to in this very newsgroup in the past).
>
> The unix philosophy of OS design has always been a step or two closer
> to the safer design (with respect to computer viruses and other malware)
> then the typical DOS/Windows (and other similar OS's in the past) design.
> The reason for this is that unixes have always been designed to be
> multi-user operating systems while DOS/Windows has been designed to be
> a single-user OS with no regard to security. The very need to handle
> multiple user automatically brings forth the need for security: You should
> not be able to access other users' data without permission, and especially
> you shouldn't be able to access the superusers' data without permission.
> This causes security to be built into the system from the ground up.
>
> The DOS/Windows design always took basically the exact opposite approach:
> Whatever the user wants to run or do, the OS allows. It's not the system's
> task to stop the user doing what he wants. Unfortunately it took over 20
> years for Microsoft to rid itself of this mentality (for some reason MS
> has always been very slow to adopt certain ideas). NT had security, but
> it wasn't even intended for normal users. It wasn't until XP that some
> *semblance* of security was introduced (yet, nevertheless, the mentality
> of the regular user being by default the superuser was still there, and
> probably 99% of XP users out there still use their machine with superuser
> privileges). This made the spreading of viruses and malware *trivial*.
>
> Not that the unix design is perfect, but at least viruses, worms and
> other malware have always been, and still are, extremely rare in unix
> systems in comparison (basically the only relatively successful worms
> in the unix world have exploited bugs in the systems to spread themselves,
> rather than relying on the users; fix the bug, and the worm stops; however,
> in the single-user OS's it requires a very significantly more radical
> change in design than just fixing a few bugs).
>
Seems to me, the best design would be an additional layer. An
application runs "sandboxed", until a scanner can verify its not
dangerous, and only then does it get "current user" rights, never mind
admin. That way, it can only clobber itself, or the data it creates
itself, or copies of the data you opened with it. Nothing changes in the
"original", of say, a JPG, until you, or your scanner, are sure that the
program is what it says, and even any changes you make only appear in
*its* copy, until verification happens, and the original is updated with
the changes.
Or, something like that. Not unlike the idea I had with using EyeOS as a
host for fake terminals, where you could hack them, but you would be
hacking into the copy of the system, so the normal user of it could log
in, see the hack and then, on their option, restore the whole thing in a
single action, back to the unchanged state. Any changes/additions they
made would still go to *their* copy, unless the changes where to a
hacked file, in which case you could have the option to keep the changes
you made, or throw them out (i.e., restore from a backup).
--
void main () {
If Schrödingers_cat is alive or version > 98 {
if version = "Vista" {
call slow_by_half();
call DRM_everything();
}
call functional_code();
}
else
call crash_windows();
}
<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models,
3D Content, and 3D Software at DAZ3D!</A>
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Patrick Elliott wrote:
> Seems to me, the best design would be an additional layer. An
> application runs "sandboxed", until a scanner can verify its not
> dangerous, and only then does it get "current user" rights, never mind
> admin.
Begs the question. How do you know it's dangerous? And what happens if you
run two sandboxed applications that change the same file? How often do you
check? Every time you run it, or just when the code changes? If only when
the code changes, just scan it before you start it the first time, but risk
having malware modify the checksum you're using to check for valid programs.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
> (basically the only relatively successful worms
> in the unix world have exploited bugs in the systems to spread themselves,
> rather than relying on the users; fix the bug, and the worm stops;
Recently it's the same on Windows (Outlook has prevented you running
executables for years now).
There are several other important differences between viruses on Unix
type OSs and Windows. Firstly, the average "computer knowledge" of unix
users is probably way higher than windows users, this in itself prevents
a load of viruses spreading. Secondly, virus writers know this, so
target Windows. Do not underestimate a large amount of people with
malicious intent. I would be very surprised if similar exploitable bugs
and loop-holes don't exist in Unix, it's just people haven't searched
that hard for them.
Also I wonder if having an open-source OS would make it more or less
vulnerable to viruses compared to closed-source (if all other factors
were equal)? On the one hand virus writers can browse the source code
looking for exploits (surely way easier than reverse engineering or
random testing), but on the other hand people can do the same to close
the exploits.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> If it went far enough, people would send out "here's a zip file with the
>> password xyz. Unpack it with that password, rename hello.jpg to hello.exe,
>> and run it to get an important message from your bank" and someone would do it.
>
> Didn't seem to be such a problem in the unix world.
Because there are a tiny number of unix users who would follow
instructions such as "you must run this attachment as admin to regain
access to your bank account" from a random email. Yet there are
probably thousands of windows users who would simply run the attachment
and click "Yes" when asked exactly the same question by the OS. If you
were a virus writer wanting to make money, which platform would you target?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
scott <sco### [at] scottcom> wrote:
> >> If it went far enough, people would send out "here's a zip file with the
> >> password xyz. Unpack it with that password, rename hello.jpg to hello.exe,
> >> and run it to get an important message from your bank" and someone would do it.
> >
> > Didn't seem to be such a problem in the unix world.
> Because there are a tiny number of unix users who would follow
> instructions such as "you must run this attachment as admin to regain
> access to your bank account" from a random email. Yet there are
> probably thousands of windows users who would simply run the attachment
> and click "Yes" when asked exactly the same question by the OS. If you
> were a virus writer wanting to make money, which platform would you target?
Well, that's kind of my point: Microsoft made it easy for viruses and
other malware to spread. The fundamental problem is that Microsoft didn't
"teach" their users to be more conscious about security. Safety thinking
didn't become a second nature to their users because the OS wasn't hammering
it into them by its very behavior.
And by this I don't mean eg. messages popping up warning the users.
With this I am talking about the very design of the OS. The design itself
should have been such that the users who learn to use it automatically
learn an instinct that protects them from most harm.
It's a bit like the difference between a "safe" and an "unsafe" programming
language: If the programming language, by its very design, makes it very
hard to write unsafe code, programmers who learn the language will learn
to use it safely, in a natural way. When the safety is in the design, the
language doesn't need to remind the user of it eg. with warnings.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> Thinking on it, there are a whole bunch of "patches" made in the UNIX world
> to account for bad security.
I never said that unix was perfect from the start (nor even that it is
perfect now). That wasn't my point.
My point was that the *approach* at OS design was different from the
start, and this caused it to naturally grow into a safer environment.
When the internet got immensely popular, unixes were already mostly safe
(give or take some exploitable bugs), while Windows was completely open
to all kinds of attacks. And this was not solely because unix has existed
longer. It's a question of fundamental design.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> > If the very first version of DOS had had a similar account/password
> > system as unixes,
> ... then it wouldn't have run on an 8086, and MS would be broke.
Or maybe we would have much better PCs today because they would not be
based on (and mostly backwards-compatible with) a totally antiquated and
obsolete architecture designed by IBM and Intel.
Think about how the game industry has boosted the development of graphics
cards. Imagine if the same boost would have been done to the PC architecture
by OS vendors.
> > would not complain because they would take it for granted, as something
> > obvious.
> It's hard to say. Most of the other systems of the day didn't have it either.
Multi-user unix systems were certainly being used in many environments
(eg. at universities with thousands of students) back when Windows95 didn't
even exist. Back then things like logins, passwords and access rights were
a given in those system. Yes, I have personal experience.
> Contrast with something like Singularity, where you explicitly list every
> program you're going to run
I never said that unix is the perfect system. I just said that it's
*better* (in terms of safety) because the fundamental design is different
(namely, it's intended to be a multi-user system).
The point is that if operating systems had had the proper design from
the start, things like computer viruses wouldn't exist (except perhaps
ones exploiting bugs, but those would probably not get as widespread
because bugs are easier to fix than fundamental OS design).
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
scott <sco### [at] scottcom> wrote:
> Also I wonder if having an open-source OS would make it more or less
> vulnerable to viruses compared to closed-source (if all other factors
> were equal)?
At least some years ago Linux was the second OS with the most security
vulnerabilities found each year (probably no need to mention which OS the
one with the most). I wouldn't be surprised if that was the case today as
well.
It's not like systems like Linux don't have security bugs to be exploited
(either by hackers or by malware).
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> My point was that the *approach* at OS design was different from the
> start, and this caused it to naturally grow into a safer environment.
Sure, I'll grant that. It wasn't what I was talking about in the original
post, but I'll grant that.
I don't think it applies any longer, tho. Indeed, in many ways I think
Windows might have a more secure architecture than UNIX nowadays, even if in
practice it's not quite up to snuff and in practice it gets attacked more.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
scott wrote:
> Because there are a tiny number of unix users who would follow
> instructions such as "you must run this attachment as admin to regain
> access to your bank account" from a random email.
http://en.wikipedia.org/wiki/Christmas_Tree_EXEC
Enough that it made international news at the time. Granted, this was IBM
big iron, not unix, but the principle is the same.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Darren New <dne### [at] sanrrcom> wrote:
>>> If the very first version of DOS had had a similar account/password
>>> system as unixes,
>
>> ... then it wouldn't have run on an 8086, and MS would be broke.
>
> Or maybe we would have much better PCs today because they would not be
> based on (and mostly backwards-compatible with) a totally antiquated and
> obsolete architecture designed by IBM and Intel.
Maybe. But the machines had to be cheap enough for individuals to buy. More
importantly, they had to be cheap enough that they could go into the right
spot on the budget.
The only thing that made PCs "cheap" at the time was the clone makers. It's
hard to say whether things would have taken off or not, but it's not like
there weren't powerful UNIX-based machines with well-designed CPUs in the
mix at the time.
> Think about how the game industry has boosted the development of graphics
> cards. Imagine if the same boost would have been done to the PC architecture
> by OS vendors.
They had things like that. The Dolphin running Smalltalk. The Amiga's
specialized chips. LISP machines. IBM made an APL luggable computer I worked
on for a while. Heck, even the Mac was in the competition. And at the low
end you had dozens of radio shack, Apple ][, Vector Graphics, Tektroniks,
Kay-Pro, Commodore, and a dozen other brands of machines, many of which were
mostly compatible with each other at the software level via CP/M.
And the 8086/8088 was designed to run Pascal, which also didn't take off.
So... what happened?
>> It's hard to say. Most of the other systems of the day didn't have it either.
>
> Multi-user unix systems were certainly being used in many environments
> (eg. at universities with thousands of students) back when Windows95 didn't
> even exist. Back then things like logins, passwords and access rights were
> a given in those system. Yes, I have personal experience.
Sure. But the other computers you'd buy yourself, for one user, didn't have
any password stuff. I'm talking mostly the 8-bit computers that the IBM PC
wound up replacing. Sure, I used Solaris machines and even z8000-based UNIX
machines. They weren't something you'd buy for a secretary, tho.
And when you have a computer with thousands of users, you have someone
knowledgeable taking care of it, and everyone with access has a basic
understanding of computers or they wouldn't have access. It wasn't a general
purpose tool - it was a computing tool, and to use it, you needed a
fundamental understanding of how the computer worked.
>> Contrast with something like Singularity, where you explicitly list every
>> program you're going to run
>
> I never said that unix is the perfect system. I just said that it's
> *better* (in terms of safety) because the fundamental design is different
> (namely, it's intended to be a multi-user system).
Well, Windows is too, *now*. You can blame Windows users for the problems,
and blame Microsoft for Windows users. But having actually dealt with people
who don't know what they're doing, I have come to the conclusion that if you
put a general purpose tool in the hands of someone who has *no* idea how it
works, you're going to get scripted behaviors (i.e., people who take notes
on the steps they have to go thru to send a picture to their grandchild)
with no understanding or desire for understanding of the implications for
anything except "did it work?"
In other words, I don't think that having had logins from the beginning
would have taught people that mail can be forged. I mean, heck, do you think
having logins on the computer would teach people not to fall for Nigerian
scams? Do you think it would teach people not to fall for phishing scams?
Why would you think it would teach people not to fall for any other sort of
forged mail?
> The point is that if operating systems had had the proper design from
> the start, things like computer viruses wouldn't exist
I disagree. That's exactly why I listed all the security flaws that UNIX
fixed over time. Such OSes *would* have bugs, *did* have bugs, and they'd
continue to have bugs as new capabilities were introduced. Nobody has a
Morris worm before SMTP. Nobody had UUCP appending mail to /etc/passwd
before UUCP was around. Nobody stole passwords by connecting to a coworker's
X terminal before X was invented. Every operating systems has had viruses
and worms and such, including those who had multi-user access controls built
in to start with. Granted, it's hard to know exactly how many, especially
given the explosive growth in the number of machines in use and the
explosive growth in the number of computer-naive users. But, too, when a bug
was found in UNIX, it wasn't valuable to avoid reporting it, so they got
fixed instead of exploited, usually. Except Kevin Mitnick, who also made
international news by stealing stuff not from Windows machines, but from
UNIX machiens.
Now we're in a different world, where it's actually valuable to find and
exploit flaws, rather than reporting them when you come across them in
normal usage. (Sort of like now suddenly we need to protect airplanes from
terrorists, and not just the passengers. :-)
Basically, history does not bear out your statement, and the disproof is why
I listed all the UNIX flaws that had been fixed over time.
Heck, by all estimates, Mac OSX has more security holes in it than Windows
does (per installed unit, obviously), and it's based on an OS that has
always had logins.
That said, yes, certainly a system that has always had multi-user
authentication (and, more importantly, a separation of administrative duties
from daily operations) is superior to one that doesn't. But when every
system now has multi-user controls, and people try to deliver applications
over the internet yadda yadda, you wind up with viruses that don't need
administrative privs to propagate. I suspect that's the vast majority of
active viruses now - those that steal personal information or add you to a
bot-net, neither of which need (or even want) admin privileges.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> I wouldn't be surprised if that was the case today as well.
I saw an interesting article a couple weeks ago (on LWM?) where the author
looked at the security bugs fixed, and tracked back thru the archives to
figure out where they'd been introduced, to see if the number of security
holes in Linux is going up or down. Apparently, the number is down, very
slowly, but it's close to the measurement noise. (It's hard to tell where a
bug was introduced in a system where it's a number of interacting systems
that cause the bug, for example.)
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> I don't think it applies any longer, tho. Indeed, in many ways I think
> Windows might have a more secure architecture than UNIX nowadays, even if in
> practice it's not quite up to snuff and in practice it gets attacked more.
Has the amount of viruses, worms and especially malware declined with
Windows Vista and 7?
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Darren New <dne### [at] sanrrcom> wrote:
>> I don't think it applies any longer, tho. Indeed, in many ways I think
>> Windows might have a more secure architecture than UNIX nowadays, even if in
>> practice it's not quite up to snuff and in practice it gets attacked more.
>
> Has the amount of viruses, worms and especially malware declined with
> Windows Vista and 7?
Compared to XP? I would certainly think so. More importantly, they've moved
to attacking things *other* than the OS. Like java in web pages, or trojans,
or phishing scams. I don't think that the number of malware attacks says
much about the security of the systems nowadays, as much as the popularity.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 20/12/2010 06:41 PM, Warp wrote:
> On the subject of virus scanners in particular, I'd say that the very
> need to have such scanners is a symptom of fundamentally bad OS design.
I'd go along with that.
> The unix philosophy of OS design has always been a step or two closer
> to the safer design (with respect to computer viruses and other malware)
> then the typical DOS/Windows (and other similar OS's in the past) design.
> The reason for this is that unixes have always been designed to be
> multi-user operating systems while DOS/Windows has been designed to be
> a single-user OS with no regard to security.
That's pretty much it, right there.
> The DOS/Windows design always took basically the exact opposite approach:
> Whatever the user wants to run or do, the OS allows.
For a machine which is physically incapable of being networked, this is
a perfectly reasonable way to proceed. The Commodore 64 and the ZX
Spectrum had exactly the same "flaw" in their design. Only if a computer
is *capable* of being used by more than one person does any question of
"security" even exist.
> Unfortunately it took over 20
> years for Microsoft to rid itself of this mentality (for some reason MS
> has always been very slow to adopt certain ideas).
*This* is where it all went wrong.
Remind me: Is this or is this not the same Microsoft that famously took
the additude of "oh, this 'Internet' thingy is just a trendy fad; it'll
never last"? For a long, long time they seemed to believe that
networking was somehow "unimportant".
Either way, when they realised the true situation, they should have
changed their design practises. Radically.
> NT had security, but it wasn't even intended for normal users.
No, it wasn't. Which is a pitty, because it was quite a good OS.
Still, the main OS kernel lives on in some form. The Win9x kernel is
gone, and 2000, XP, Vista and 7 are all based (increasingly loosely) on
the NT kernel series.
In some ways, NT actually has *more* security features than Unix. For
example, take file security. Under Unix, I can set permissions for one
user, one group, and everybody else. Under NT, I can set permissions for
as many users and user groups as I damned well like. And this isn't a
theoretical ability; I use it extensively in my day job. Not only that,
but Unix has umask, while NT actually allows you to set the default file
permissions on a per-folder basis. And to reset the permissions on all
the files in a folder easily. Hell, I can even control who is allowed to
*see* the permissions on a file or folder.
On top of that, you make the OS log a record every single time a
particular action is performed on a given file. (Unix, on the other
hand, doesn't even provide a way for a process to be *notified* when a
file changes, much less to log such changes at the OS level.) Again, I
can control this at the file level, so I only get log entries for files
that I actually care about, not every single file in the entire
filesystem. And not for every action, and not for every user.
And that's just files. You can also set permissions controlling who is
allowed to kill a given process. Or clear the print jobs in a given
queue. And more besides. (Sadly, the OS GUI doesn't expose most of this
very useful control, and there are certainly no CLI tools for this
either. Indeed, if you want to set file permissions, you can *only* do
it via the GUI. Well, no, now there's PowerShell which can probably do
it...)
> It wasn't until XP that some
> *semblance* of security was introduced (yet, nevertheless, the mentality
> of the regular user being by default the superuser was still there
NT was where the big security changes happened. Before NT, any talk of
"security" was a nonesense. XP is just the first really well-known
consumer OS to feature these changes. (Windows 2000 didn't really get
noticed much, for whatever reason. Note that Windows 2000 /= Windows ME.)
Unfortunately, it turns out that 99% of all Windows software still
*assumes* that it has permission to do everything. And thus, you come up
with stupid glitches like, for example,
Nero claims that you do not possess a CD burner, *unless* you are
logged in as a superuser.
One could argue that it's not so much /Microsoft/ that is slow to adapt,
of even it's /users/. No, it's the people writing Windows software. If I
had a penny for every time I've had to do something stupid just to make
the buggy, barely-functional device driver for some crappy cheap-arse
piece of hardware work...
Of course, Microsoft don't really help themselves sometimes. They have
an almost obsessive tendency of making everything as scriptable,
programmable and customisable as possible. I guess because all those
extra features look good on the tin?
For example, if you use Outlook (not Outlook Express) you can create an
"email" which is actually a full-fledged application, in effect. (This
might also require Exchange, I'm not sure.) It ranges from simple stuff
like sending out appointments and questionnaires, right up to building
complex fillable forms, and having the server receive the responses and
do non-trivial processing with them, possibly producing additional
emails in response. Stuff like that.
All of which is very *powerful* and everything. But the net result is
that everything from emails to Word documents to spreadsheets,
databases, presentations, and so forth all can have arbitrary executable
code embedded in it, and more often then not executed immediately as
soon as you touch the time, often without you even realising it. This
enables you to create very "rich" documents. For example, the other day
I saw a PowerPoint presentation which is like a hyperlinked, browseable
product catelogue. Very impressive stuff. And it's no secret that you
can use Access to build what ammounts to a desktop application.
If you think in terms of shiney features, all of this sounds fanstastic.
If you think in terms of computer security, all of this sounds like a
catastrophy just waiting to happen. I mean, who thought that a *word
processed document* being able to alter the local filesystem was a good
idea?! Most users have no clue that you can do this. But I promise you,
you can. (Or, you could. These days as soon as you open a Word document,
you have to OK a dozen messages just to make it open, regardless of
whether it does anything even mildly dangerous.)
Of course, having access to the local filesystem allows a Word document
to be part of a big happy desktop application, developed using just MS
Office. Why write a GUI application when you can just customise Word a
little, and use a flat XML file as your database?
Now, if the system had been designed with security in mind from the
start, nobody would have done anything so stupid. But now all these
features exist (and, weirdly, continue to be designed), and somehow you
have to make it secure. MS's idea of "make it secure" isn't "disable
local filesystem access unless there's a damned good reason to enable
it". It isn't even "check whether the macro tries to access the local
filesystem" (which is trivially checkable). No, it's "slap a big, fat,
flashing warning on top of EVERYTHING, dangerous or not". Because, let's
face it, that's very easy to code.
And after the 198th time the user sees this annoying, unecessary error
message, they just stop paying attention. And then when a *real* threat
comes along, the user will blindly and automatically click "yes, please
run this unsigned code". Because they've had to click it a thousand
times before in order to get stuff done.
This is not "security". The whole XP Security Center is nothing more
than a nag screen that constantly whines at you. "Turn on updates.
Install a pricey AV product. Turn on the firewall." It does nothing to
actually increase security. (Automatic updates themselves may do. But
Linux has copied that idea too now...)
Really, there are several reasons why Windows (and related MS products)
are less secure than their Unix counterparts:
- Microsoft failed to realise that networks would become "important"
(and hence, security would be necessary).
- Once they did decide that networking actually was the future, they
implemented lots of security stuff but failed to really make full use of it.
- There is now a huge codebase of cheap, buggy, unsupported software
which people expect to work on Windows. If you start actually doing
things in a secure way, most of this software will break. (This is
technically a GOOD THING, but it doesn't sound very good to the people
who just want to do their stuff.)
- Microsoft thinks that endless rafts of whizzy features are more
important than computer security. (That's quite a serious problem, right
there.)
- Unix is the OS for computer experts. Windows is the OS for idiots so
stupid that arguably they shouldn't be let near a computer in the first
place. Wanna guess which one has the biggest security problems?
- Windows systems outnumber Unix systems 10^4 : 1. Wanna guess which one
most people spend their time trying to attack?
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
> Either way, when they realised the true situation, they should have
> changed their design practises. Radically.
Really, few other people understood the security flaws at the time either.
HTTP wasn't invented with encryption in mind. SMTP had no controls on who
could send mail. Basically, there wasn't even a standard encryption suite
until long after Windows was networked and HTTP started being used for
commercial applications.
> Indeed, if you want to set file permissions, you can *only* do
> it via the GUI.
cacls is so old it's deprecated in favor of icacls.
And no, Windows doesn't tend to provide command-line tools for things you
can do via scripting. That's what scripting calls are for, and why
powershell is looked on so favorably by windows admins.
When your fundamental OS interface is essentially object-oriented and
API-based, you don't get a whole lot of non-programming command-line tools
to manage stuff. It's only when most of your configuration is stored as
text files that you get text-based tools to manage it.
> Unfortunately, it turns out that 99% of all Windows software still
> *assumes* that it has permission to do everything.
I don't believe that's true any more. It might have been when XP first came out.
> One could argue that it's not so much /Microsoft/ that is slow to adapt,
> of even it's /users/. No, it's the people writing Windows software. If I
> had a penny for every time I've had to do something stupid just to make
> the buggy, barely-functional device driver for some crappy cheap-arse
> piece of hardware work...
Exactly.
> Of course, Microsoft don't really help themselves sometimes. They have
> an almost obsessive tendency of making everything as scriptable,
> programmable and customisable as possible. I guess because all those
> extra features look good on the tin?
It's in part because they have big customers who need that stuff. It's not
like UNIX isn't scriptable programmable and customizable.
> All of which is very *powerful* and everything. But the net result is
> that everything from emails to Word documents to spreadsheets,
> databases, presentations, and so forth all can have arbitrary executable
> code embedded in it,
And *that* is exactly why malware is still around in multi-user systems. And
it happens in browsers and everything too, so it's not restricted to
microsoft code.
> And it's no secret that you
> can use Access to build what ammounts to a desktop application.
Well, that's kind of what it's for.
> And after the 198th time the user sees this annoying, unecessary error
> message, they just stop paying attention. And then when a *real* threat
> comes along, the user will blindly and automatically click "yes, please
> run this unsigned code". Because they've had to click it a thousand
> times before in order to get stuff done.
That's a part of the problem. You can't ask users to be making that sort of
decision, either at the "access the file system" level or at the more
fine-grained level. The Android mechanism makes a lot of sense to users. But
in a system where you don't declare what code you're going to run or what
privileges you need (which applies to both UNIX and Windows and most other
systems), this doesn't work.
And that's why I thought the new types of hardware coming out (portables,
games, phones, and custom server hardware) might provide a chance to get
away from these legacy problems.
> - Microsoft failed to realise that networks would become "important"
> (and hence, security would be necessary).
I disagree this was unique to Microsoft.
> - There is now a huge codebase of cheap, buggy, unsupported software
> which people expect to work on Windows. If you start actually doing
> things in a secure way, most of this software will break. (This is
> technically a GOOD THING, but it doesn't sound very good to the people
> who just want to do their stuff.)
Exactly.
> - Microsoft thinks that endless rafts of whizzy features are more
> important than computer security. (That's quite a serious problem, right
> there.)
I believe starting around the time of XP SP2 they realized their lack of
security was actually hurting their business.
> - Unix is the OS for computer experts. Windows is the OS for idiots so
> stupid that arguably they shouldn't be let near a computer in the first
> place. Wanna guess which one has the biggest security problems?
>
> - Windows systems outnumber Unix systems 10^4 : 1. Wanna guess which one
> most people spend their time trying to attack?
Yes and yes.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 20/12/2010 05:53 PM, Darren New wrote:
> that traditional OSes hold back the more sophisticated (as in, far from
> machine language) languages.
Perhaps. I guess it kind of depends on what you're trying to do. If you
have an application that just does its computations in its own little
world, then it's fine. If, on the other hand, the application wants to
do arbitrary stuff with the whole machine, then... yeah.
Wasn't there an OS written in Smalltalk at some point? (Or was that only
for special hardware?) Certainly you can see that if the Smalltalk
runtime was running on the bare metal, it would have all sorts of
interesting consequences. (For example, since *everything* can be
changed, you could invent some new bizare concept of files.)
Speaking of which...
http://halvm.com/
That's Haskell running on bare metal. (Well, OK, no it isn't, it's
running under a Xen hypervisor. But that's /almost/ the same thing.)
Apparently the guys at Galios actually use this thing, so that (say)
your web server runs in one VM, all by itself, and your DB runs in
another VM, and so on. But it would be interesting to see what happens
if you tried to write an actual "operating system" in Haskell, rather
than just an application.
> http://www.artima.com/lejava/articles/azul_pauseless_gc.html
That contains a lot of "look at us, aren't we clever!" and not very much
technical detail. It seems the only novel detail is that they're using
the memory protection hardware to trap and remap accesses to moved
objects. That's a neat detail, and one which (they assert) would be too
inefficient with a normal OS. But that seems to be just about it.
Haskell has a rather interesting invariant: since *most* objects are
immutable, old objects can never point to new objects. It seems like
this should make some kind of interesting GC algorithm possible.
Personally, I thought the article on GC avoiding paging was more
interesting.
> Traditional file system interfaces probably do too.
Now, do you mean "file systems", or do you mean "interfaces to file
systems"?
> It's interesting that this sort of stuff is starting to get to the point
> where people will be willing to break with compatibility at some level.
> Phones, game consoles, set-top boxes, and eventually probably
> "enterprise" or "cloud" type servers will all be willing to consider a
> different operating system that puts limits on compatibility with
> previous languages and libraries.
Well, we'll see what happens.
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
> Perhaps. I guess it kind of depends on what you're trying to do. If you
> have an application that just does its computations in its own little
> world, then it's fine. If, on the other hand, the application wants to
> do arbitrary stuff with the whole machine, then... yeah.
Not necessarily. There are memory models that really just don't mix, for
example. GCed memory with non-GCed external resources is the classic. This
paper describes how if you assume you have a C-like/Pascal-like workload,
using thousands of page faults a second to do GC isn't going to be
efficient. Etc.
> Wasn't there an OS written in Smalltalk at some point? (Or was that only
> for special hardware?)
Yes. The "dolphin" computer. It had programmable microcode, so when you
booted a Smalltalk image, the machine code *was* smalltalk bytecodes. When
you booted a COBOL program, the machine code was appropriate for COBOL.
> Certainly you can see that if the Smalltalk
> runtime was running on the bare metal, it would have all sorts of
> interesting consequences. (For example, since *everything* can be
> changed, you could invent some new bizare concept of files.)
They had what you'd consider a bizarre concept for files. That's the point.
You didn't have to worry about "destructors" for files, precisely because
"file" was built into the language, not the OS. So to speak.
> if you tried to write an actual "operating system" in Haskell, rather
> than just an application.
But that's the point. When your application runs on the bare metal, there's
no distinction between OS and application.
>> http://www.artima.com/lejava/articles/azul_pauseless_gc.html
>
> That contains a lot of "look at us, aren't we clever!" and not very much
> technical detail. It seems the only novel detail is that they're using
> the memory protection hardware to trap and remap accesses to moved
> objects. That's a neat detail, and one which (they assert) would be too
> inefficient with a normal OS. But that seems to be just about it.
That's the only neat technical trick they used to implement their other
tricks. I think it's more interesting that they're actually getting away
from the linear memory model that C and Pascal foisted on us.
In other words, they're using an OO memory model. Objects are contiguous
blobs, but it doesn't really matter beyond that where they are in the
address space. So instead of doing a compacting garbage collection in a
linear space, they do a compacting garbage collection on a per-page basis.
*That* is really the innovation.
> Haskell has a rather interesting invariant: since *most* objects are
> immutable, old objects can never point to new objects. It seems like
> this should make some kind of interesting GC algorithm possible.
Erlang takes great advantage of that too, yes.
> Personally, I thought the article on GC avoiding paging was more
> interesting.
Yeah. Again, it's the sort of thing you have a hard time doing in an OS
tuned for something with a memory model like C's.
>> Traditional file system interfaces probably do too.
>
> Now, do you mean "file systems", or do you mean "interfaces to file
> systems"?
Interfaces. The reality of file systems nowadays isn't anywhere near what
the interface looks like. Internally, it's much closer to multic's file
system, yet it's generally presented with UNIX file system semantics.
(In Multics, the basic way of getting to a file was essentially memmap.
That's why UNIX files are all represented as an array of bytes, with no
records, no insert/delete bytes, etc. It's modeling multics' interface
without the convenience of either having it in memory *or* having the OS
provide you useful services.)
> Well, we'll see what happens.
Yeah, sadly, I don't have a whole lot of hope. Altho the thing with each
application running in its own Xen hypervisor slot sounds interesting. I'll
have to look at the implications of that a little more closely.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 22/12/2010 06:42 PM, Darren New wrote:
> Orchid XP v8 wrote:
>> Perhaps. I guess it kind of depends on what you're trying to do. If
>> you have an application that just does its computations in its own
>> little world, then it's fine. If, on the other hand, the application
>> wants to do arbitrary stuff with the whole machine, then... yeah.
>
> Not necessarily. There are memory models that really just don't mix, for
> example. GCed memory with non-GCed external resources is the classic.
And yet, code written in languages assuming GC management and languages
assuming manual management can not only interoperate, but even be linked
together in the same process image.
> This paper describes how if you assume you have a C-like/Pascal-like
> workload, using thousands of page faults a second to do GC isn't going
> to be efficient. Etc.
True enough.
>> Wasn't there an OS written in Smalltalk at some point? (Or was that
>> only for special hardware?)
>
> Yes. The "dolphin" computer. It had programmable microcode, so when you
> booted a Smalltalk image, the machine code *was* smalltalk bytecodes.
> When you booted a COBOL program, the machine code was appropriate for
> COBOL.
That sounds moderately mental. I'd hate to think what happens if you get
the microcode wrong...
>> Certainly you can see that if the Smalltalk runtime was running on the
>> bare metal, it would have all sorts of interesting consequences. (For
>> example, since *everything* can be changed, you could invent some new
>> bizare concept of files.)
>
> They had what you'd consider a bizarre concept for files. That's the
> point. You didn't have to worry about "destructors" for files, precisely
> because "file" was built into the language, not the OS. So to speak.
Not even built into the language - built into a library. And,
presumably, by changing that library, you can change how the filesystem
works...
(Does anybody remember how "Longhorn" was supposed to ship with a
filesystem that was actually a relational database? *Really* glad they
scrapped that idea!)
>> if you tried to write an actual "operating system" in Haskell, rather
>> than just an application.
>
> But that's the point. When your application runs on the bare metal,
> there's no distinction between OS and application.
Well, to some extent there /is/. By definition, an application is
something that solves a real-world problem, and an OS is something that
provides services to applications. Although in the case of something
like a Smalltalk or Haskell OS, the "OS" might be statically linked into
the application image - i.e., the "OS" might just be a runtime library.
> In other words, they're using an OO memory model. Objects are contiguous
> blobs, but it doesn't really matter beyond that where they are in the
> address space. So instead of doing a compacting garbage collection in a
> linear space, they do a compacting garbage collection on a per-page
> basis. *That* is really the innovation.
Sure. If the virtual address space supports it, why not? (Other than
"because most OS designs assume that you would never want to do such a
thing".)
>>> Traditional file system interfaces probably do too.
>>
>> Now, do you mean "file systems", or do you mean "interfaces to file
>> systems"?
>
> Interfaces. The reality of file systems nowadays isn't anywhere near
> what the interface looks like. Internally, it's much closer to multic's
> file system, yet it's generally presented with UNIX file system semantics.
>
> (In Multics, the basic way of getting to a file was essentially memmap.
> That's why UNIX files are all represented as an array of bytes, with no
> records, no insert/delete bytes, etc. It's modeling multics' interface
> without the convenience of either having it in memory *or* having the OS
> provide you useful services.)
OK, now I must know... WTF is this "memmap" that I keep hearing about?
>> Well, we'll see what happens.
>
> Yeah, sadly, I don't have a whole lot of hope. Altho the thing with each
> application running in its own Xen hypervisor slot sounds interesting.
> I'll have to look at the implications of that a little more closely.
Remember that Galios mainly does work for people who want high assurance
software. Stuff with mathematical guarantees of correctness, and so
forth. For that kind of insanity, being able to guarantee that the OS
isn't going to do something weird (because there isn't one) is probably
quite useful. I'm more dubious about how useful it is for everyone else.
The idea of experimental OS design using Haskell is quite interesting,
however.
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
> And yet, code written in languages assuming GC management and languages
> assuming manual management can not only interoperate, but even be linked
> together in the same process image.
Sure. But then you get things like "finalizers" to clean up unmanaged
resources, the "using" statement to enforce the use of finalizers, all kinds
of odd rules in the GC engine that slow it down because it has to deal with
finalizers, etc.
Sort of the same way that when a process in UNIX exits, there's a list of
things the kernel cleans up for it on its way out.
> That sounds moderately mental. I'd hate to think what happens if you get
> the microcode wrong...
The same as if you get any other program wrong. What do you mean? It's
probably far easier than writing a modern bytecode interpreter like the JVM.
> Not even built into the language - built into a library. And,
> presumably, by changing that library, you can change how the filesystem
> works...
Right. That's the idea. Basically, what's a file? It's a data structure you
access. Why is a file different from an array of bytes held by a process
that just never exits?
> Well, to some extent there /is/. By definition, an application is
> something that solves a real-world problem, and an OS is something that
> provides services to applications.
Where do you draw the distinction, if there's exactly one application
running on the custom OS?
>> In other words, they're using an OO memory model. Objects are contiguous
>> blobs, but it doesn't really matter beyond that where they are in the
>> address space. So instead of doing a compacting garbage collection in a
>> linear space, they do a compacting garbage collection on a per-page
>> basis. *That* is really the innovation.
>
> Sure. If the virtual address space supports it, why not? (Other than
> "because most OS designs assume that you would never want to do such a
> thing".)
Well, that's exactly the point. Most computer languages completely ignore
virtual memory. Only managed OO languages come close to having a concept of
memory as something other than an array of bytes, and even that is more
segments than pages.
I used to use an HP mainframe that was incapable of running C, because it
was a segmented chip. It was really old, but it would be ideal for running
an OO language nowadays, because you had tons of segments available, etc.
Basically, pointers pointed to segments, not bytes, so it was in hindsight a
very OO kind of architecture.
The Burroughs B series had typed machine code. Each memory address also had
tag bits saying what type was stored there, so the machine code just had an
"add" instruction, not an "add integers" or "add floats". The operands said
what the type was. And array accesses had to go through an array pointer,
which included the bounds of the array. Again, another machine you couldn't
run C on.
> OK, now I must know... WTF is this "memmap" that I keep hearing about?
http://lmgtfy.com/?q=what+is+memmap
> I'm more dubious about how useful it is for everyone else.
As I've said, I think that the very small and the very large will both be
using this sort of thing more and more. If you can cut $5 off the cost of
building a TiVO by using a language other than C, you'll see that happening.
If you can cache 20% more stuff on a memcached-like server by ditching
Linux, you'll see Amazon starting to do that I expect. Most large servers
are single-purpose nowadays anyway.
--
Darren New, San Diego CA, USA (PST)
Serving Suggestion:
"Don't serve this any more. It's awful."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 22/12/2010 09:27 PM, Darren New wrote:
> Sort of the same way that when a process in UNIX exits, there's a list
> of things the kernel cleans up for it on its way out.
I didn't know it actually did that. I thought that (for example) if a
Unix process exits without freeing the memory it allocated, that memory
remains allocated forever. (Presumably it gets pages out to disk fairly
soon, but even so...)
>> That sounds moderately mental. I'd hate to think what happens if you
>> get the microcode wrong...
>
> The same as if you get any other program wrong. What do you mean? It's
> probably far easier than writing a modern bytecode interpreter like the
> JVM.
I was thinking more that if you get the microcode right, you might
physically fry the processor.
>> Well, to some extent there /is/. By definition, an application is
>> something that solves a real-world problem, and an OS is something
>> that provides services to applications.
>
> Where do you draw the distinction, if there's exactly one application
> running on the custom OS?
I suppose to some extent it's a bit arbitrary. But if you have, say, a
library who's only purpose is to take graphics requests and poke the
necessary hardware registers to make pixels change colour, you could
call that part of an OS.
The "first" OS was of course the Disk Operating System, remember. ;-)
> I used to use an HP mainframe that was incapable of running C, because
> it was a segmented chip. It was really old, but it would be ideal for
> running an OO language nowadays, because you had tons of segments
> available, etc. Basically, pointers pointed to segments, not bytes, so
> it was in hindsight a very OO kind of architecture.
I'm not sure I see how.
> The Burroughs B series had typed machine code. Each memory address also
> had tag bits saying what type was stored there, so the machine code just
> had an "add" instruction, not an "add integers" or "add floats". The
> operands said what the type was. And array accesses had to go through an
> array pointer, which included the bounds of the array. Again, another
> machine you couldn't run C on.
That sounds much more interesting.
>> OK, now I must know... WTF is this "memmap" that I keep hearing about?
>
> http://lmgtfy.com/?q=what+is+memmap
I'm not sure I completely understand.
So, a memory-mapped file is a region of virtual memory which contains
the same data as a file on disk? And when you access anything in that
region, the necessary pages are read from disk? (And, presumably, saved
back to that file rather than being copied to swap when physical memory
is required.)
So... how do you change the size of the file then?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible wrote:
> On 22/12/2010 09:27 PM, Darren New wrote:
>
>> Sort of the same way that when a process in UNIX exits, there's a list
>> of things the kernel cleans up for it on its way out.
>
> I didn't know it actually did that. I thought that (for example) if a
> Unix process exits without freeing the memory it allocated, that memory
> remains allocated forever. (Presumably it gets pages out to disk fairly
> soon, but even so...)
No, that would be AmigaOS (intentionally, it turns out). :-)
Linux closes files, deallocates memory, reclaims disk space for unlinked
files that this process had the last open on, unlocks files, closes the file
holding the executable code, frees up page maps, and releases certain kinds
of semaphores. And probably more, nowadays.
>>> That sounds moderately mental. I'd hate to think what happens if you
>>> get the microcode wrong...
>>
>> The same as if you get any other program wrong. What do you mean? It's
>> probably far easier than writing a modern bytecode interpreter like the
>> JVM.
>
> I was thinking more that if you get the microcode right, you might
> physically fry the processor.
Oh. Not in any microcode I've ever seen (not that I've seen much), but I
imagine it's possible.
>> I used to use an HP mainframe that was incapable of running C, because
>> it was a segmented chip. It was really old, but it would be ideal for
>> running an OO language nowadays, because you had tons of segments
>> available, etc. Basically, pointers pointed to segments, not bytes, so
>> it was in hindsight a very OO kind of architecture.
>
> I'm not sure I see how.
Because you couldn't have pointers into the middle of segments. The memory
model was a bunch of "objects" in the sense that they were atomic lumps of
memory that you could move around without adjusting pointers everywhere.
I only read about the assembly language of the thing, tho, so I don't really
know any more details than that.
>> The Burroughs B series had typed machine code. Each memory address also
>> had tag bits saying what type was stored there, so the machine code just
>> had an "add" instruction, not an "add integers" or "add floats". The
>> operands said what the type was. And array accesses had to go through an
>> array pointer, which included the bounds of the array. Again, another
>> machine you couldn't run C on.
>
> That sounds much more interesting.
That too. You can look up the details with google.
>
>>> OK, now I must know... WTF is this "memmap" that I keep hearing about?
>>
>> http://lmgtfy.com/?q=what+is+memmap
>
> I'm not sure I completely understand.
>
> So, a memory-mapped file is a region of virtual memory which contains
> the same data as a file on disk?
You know how swap space works, right? The page file?
memmap is using the exact same mechanism, except it pages out to the file
you specify instead of "the swap space".
Or, viewed another way, all of memory is memmapped, and that system call
lets you pick which file it's memmapped into instead of the default.
> So... how do you change the size of the file then?
I don't know that you do, if that's how you're accessing it. I haven't used
it in so long that it's all probably changed by now.
--
Darren New, San Diego CA, USA (PST)
"How did he die?" "He got shot in the hand."
"That was fatal?"
"He was holding a live grenade at the time."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 31/01/2011 06:56 PM, Darren New wrote:
> Linux closes files, deallocates memory, reclaims disk space for unlinked
> files that this process had the last open on, unlocks files, closes the
> file holding the executable code, frees up page maps, and releases
> certain kinds of semaphores. And probably more, nowadays.
Man, I had no idea...
>> I was thinking more that if you get the microcode right, you might
>> physically fry the processor.
>
> Oh. Not in any microcode I've ever seen (not that I've seen much), but I
> imagine it's possible.
Well, since the microcode is just a bunch of extremely wide bitmaps that
open and close various switches, it wouldn't surprise me if doing so in
the wrong combination could short-circuit something, or do something
similarly bad.
OTOH, maybe if the microcode is user-modifyable, you'd design in some
margin for error...
(Do modern AMD64 processors let you change the microcode? Or is it wired
in permanently?)
> Because you couldn't have pointers into the middle of segments. The
> memory model was a bunch of "objects" in the sense that they were atomic
> lumps of memory that you could move around without adjusting pointers
> everywhere.
Oh, OK.
> You know how swap space works, right? The page file?
>
> memmap is using the exact same mechanism, except it pages out to the
> file you specify instead of "the swap space".
>
> Or, viewed another way, all of memory is memmapped, and that system call
> lets you pick which file it's memmapped into instead of the default.
So what you're saying is it lets you access a file like you access
memory? (And presumably avoids the data being swapped out to the swap
file when you actually want it to end up in some other file anyway...)
>> So... how do you change the size of the file then?
>
> I don't know that you do, if that's how you're accessing it. I haven't
> used it in so long that it's all probably changed by now.
OK.
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible <voi### [at] devnull> wrote:
> I thought that (for example) if a
> Unix process exits without freeing the memory it allocated, that memory
> remains allocated forever.
I really can't understand why you would think that. Processes ask the
OS for memory (how else is the OS able to allocate memory to different
processes?), but the OS would then just let that memory remain allocated
when the process ends, if it didn't ask the OS explicitly to free it?
*Why* would it do that, given that freeing the memory is, relatively
speaking, trivial?
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
> Well, since the microcode is just a bunch of extremely wide bitmaps that
> open and close various switches, it wouldn't surprise me if doing so in
> the wrong combination could short-circuit something, or do something
> similarly bad.
That's a point. I don't think that's real common, tho. There's usually
something where you're connecting an input to an output. You connect the
memory bus to the register, not the power in to the power out. :-)
> So what you're saying is it lets you access a file like you access
> memory? (And presumably avoids the data being swapped out to the swap
> file when you actually want it to end up in some other file anyway...)
Yes, precisely.
--
Darren New, San Diego CA, USA (PST)
"How did he die?" "He got shot in the hand."
"That was fatal?"
"He was holding a live grenade at the time."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> *Why* would it do that, given that freeing the memory is, relatively
> speaking, trivial?
On the Amiga, it's because you might have passed that memory to something
else, or loaded code into it to handle interrupts. But then, the Amiga was a
single-user OS with no memory protection, so you did that sort of thing. :-)
--
Darren New, San Diego CA, USA (PST)
"How did he die?" "He got shot in the hand."
"That was fatal?"
"He was holding a live grenade at the time."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 31/01/2011 09:19 PM, Warp wrote:
> Invisible<voi### [at] devnull> wrote:
>> I thought that (for example) if a
>> Unix process exits without freeing the memory it allocated, that memory
>> remains allocated forever.
>
> I really can't understand why you would think that.
Because in order to do this, the OS would have to keep track of not only
what resources are in use, but *who* is using them.
OTOH, if you have virtual memory, you presumably have to keep track of
who's using which pages anyway...
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Le 31/01/2011 14:28, Invisible a écrit :
>>> OK, now I must know... WTF is this "memmap" that I keep hearing about?
>>
>> http://lmgtfy.com/?q=what+is+memmap
>
> I'm not sure I completely understand.
>
> So, a memory-mapped file is a region of virtual memory which contains
> the same data as a file on disk? And when you access anything in that
> region, the necessary pages are read from disk? (And, presumably, saved
> back to that file rather than being copied to swap when physical memory
> is required.)
>
> So... how do you change the size of the file then?
For a memmap-ed file, usually you don't.
For a normal file, opening in append mode, and using seek to reach the
right last byte counted from the start followed by either a write, if
you want to extend it or directly a truncate call if you want it shorter.
memmap is just a "convenient" way to access the bytes in a file, just
reading and writing them as an array in memory.
seek, read & write are just as good (but file oriented, so if your huge
and smart code is expecting an array of bytes... memmap come easy for
the lazy programers)
--
Software is like dirt - it costs time and money to change it and move it
around.
Just because you can't see it, it doesn't weigh anything,
and you can't drill a hole in it and stick a rivet into it doesn't mean
it's free.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible <voi### [at] devnull> wrote:
> On 31/01/2011 09:19 PM, Warp wrote:
> > Invisible<voi### [at] devnull> wrote:
> >> I thought that (for example) if a
> >> Unix process exits without freeing the memory it allocated, that memory
> >> remains allocated forever.
> >
> > I really can't understand why you would think that.
> Because in order to do this, the OS would have to keep track of not only
> what resources are in use, but *who* is using them.
That's kind of what the OS does.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible wrote:
> Because in order to do this, the OS would have to keep track of not only
> what resources are in use, but *who* is using them.
Yes, it does that. Especially with VM, as you say, since every time the OS
switches to running a different process, it has to reassign what pages of
physical memory go with which virtual addresses.
--
Darren New, San Diego CA, USA (PST)
"How did he die?" "He got shot in the hand."
"That was fatal?"
"He was holding a live grenade at the time."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On 31/01/2011 09:31 PM, Darren New wrote:
> Orchid XP v8 wrote:
>> Well, since the microcode is just a bunch of extremely wide bitmaps
>> that open and close various switches, it wouldn't surprise me if doing
>> so in the wrong combination could short-circuit something, or do
>> something similarly bad.
>
> That's a point. I don't think that's real common, tho. There's usually
> something where you're connecting an input to an output. You connect the
> memory bus to the register, not the power in to the power out. :-)
Connect two outputs together, with one trying to drive a line high, and
the other trying to drive it low, and you could have some "interesting"
problems.
>> So what you're saying is it lets you access a file like you access
>> memory? (And presumably avoids the data being swapped out to the swap
>> file when you actually want it to end up in some other file anyway...)
>
> Yes, precisely.
OK. So what's all this stuff about "the OS loads programs by memory
mapping them"? (Hopefully this will explain why programs show up as
generating a non-zero number of "page faults", even though no swapping
is actually happening...)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible wrote:
> Connect two outputs together,
I don't think two outputs would have a gate between them like that. However,
at this point I'm really over my head. I've never *actually* written
microcode for a real computer myself. :-)
> OK. So what's all this stuff about "the OS loads programs by memory
> mapping them"? (Hopefully this will explain why programs show up as
> generating a non-zero number of "page faults", even though no swapping
> is actually happening...)
It's exactly the same thing. If you have a chunk of executable that's
supposed to be in addresses 0x1000 to 0x3000, and that's at offset zero in
the file, you memmap the file into the page tables. Basically, you use the
executable file itself as the swap file for that region of memory.
That's why in UNIX you can delete a running executable file, but you can't
write to it. Because you're actually running code right out of the file, and
not a copy of the file.
--
Darren New, San Diego CA, USA (PST)
"How did he die?" "He got shot in the hand."
"That was fatal?"
"He was holding a live grenade at the time."
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
