|
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
I don't know about Singularity and it's particular design goals, but the
other day I was thinking: What would happen if you set out to design a
new OS completely from scratch? What would that look like?
It seems to me that most OS designs today are really quite similar. For
example, in every OS I'm aware of, there is some kind of CLI that allows
you to invoke executable programs, optionally passing some arguments,
and the invoked program inherits 3 streams - stdin, stdout and stderr.
The exact details vary, but this basic story seems to apply to every
common OS.
What if we wanted to shake that up a litle? What might we decide to do
differently?
Well, currently a program's arguments are just a giant blob of text. The
OS does nothing more than hand it over to the program, which may then
interpret them in any way it pleases. This is LCD; it works for
everything, but it's not terribly sophisticated.
How about if, say, the program could somehow "tell" the OS what
arguments it actually accepts? (In the same way a program file usually
contains metadata to "tell" the OS all kinds of other stuff about it,
such as linking information.) Then the OS could report invalid argument
names without even needing to bother actually starting the program
itself. And just think of the auto-complete possibilities.
Hey, let's go one better. The majority of CLI arguments are either
on/off switches or filenames, right? Well what if we *tell* the OS what
things are on/off switches, and that their default state should be? What
if we *tell* it which things are supposed to be filenames? (And whether
the name in question *should* or *should not* exist when the program is
run? Or whether it should be a *file* or a *directory*? Or maybe even
the name of another program?)
Once you start thinking this way, you start to see that actually, if we
get the OS to interpret the arguments and pass *structured* data to the
program [rather than just a blob of textual data], suddenly all sorts of
interesting ideas become possible.
If nothing else, it means that CLI arguments now have a standardised
format, enforced by the OS, which makes it easier to learn how to
operate each new program. Maybe all the program does is somehow list a
bunch of settings it requires? Maybe then you can specify those either
by CLI arguments, or a per-user or per-machine set of defaults? Maybe
the OS has a database of these default settings somewhere? All kinds of
interesting ideas to throw around.
Similarly, on a "normal" OS, each program sets three character streams.
(And Unix programs in particular seem to do weird trickery to discover
whether the output stream "is a TTY" and behave differently if it is.)
It's also traditional to pipe data between programs.
Maybe we can do something more interesting here? Maybe we can pass
*structured* data around instead of just plain character streams?
(Although now you start having potential difficulties with finding a
data representation that everybody likes.) Maybe not every program has
to have exactly 3 such streams? Maybe piping data between programs
running on physically seperate networked machines shouldn't be too
different from piping locally? Just a thought...
Tradition dictates that when a program exists, it returns a "status
code", which is simply an integer. Zero indicates success, anything else
indicates failure or at least some kind of warning. (And every program
uses its own slightly different set of conventions here.)
Maybe we can do something better here too? Maybe we could have a small
set of standard categories like "program bug", "resource exhaustion",
"the network won't answer me", and provide a set of application-specific
codes for the actual failures that a particular program can have?
How about logging? Windoze does this slightly better than Linux in that
there are (typically 3) logs that applications can write to if they
want. But maybe we could do something better than that? Maybe
per-application logs? (If a given application wants it.) Maybe tell the
OS how to invoke different levels of logging? Just some ideas.
Of course, when you look at filesystems, most OSes provide a construct
known as a "file" which is an opaque sequence of bytes. (And a few
provide a means to specify what those bytes are supposed to represent.)
I suppose you could go down the route of having files contain structured
data - but again you're going to get people arguing over the best way of
structuring things.
I've often thought about what would happen if, say, Smalltalk was the
entire OS. Then the OS would "know about" the internal workings of each
program to a large degree, and that opens up some rather interesting
possibilities. Things like highly structured IPC and so forth. Trouble
is, now you can only run stuff implemented in Smalltalk...
In short, once you sit down and start to question the way OSes work
today, you start to see that there are actually many things we could be
doing differently - ranging from the conservative to the highly radical.
(To me, really radical ideas are interesting to think about but probably
wouldn't work too well in practice.)
Heh, if *I* had 3 years to sit and write an OS, maybe I could experiment
with a few of these ideas? ;-)
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Does anybody *else* find it ironic that Micro$oft - the corporation
internationally renowned for its poor quality, buggy products - is
interested in methods of producing high-quality software?
Surely bug-free software isn't very profitable? :-P
Regardless, Singularity has a number of interesting ideas.
- Let the compiler enforce program isolation, not the processor
hardware. (That works great if everybody uses your compiler, but I'm not
sure what happens if you allow arbitrary 3rd party code to execute...)
- Make IPC fast and use it liberally. Use IPC for plugins instead of
dynamic loading. Use statically-checked IPC protocols.
- Put almost everything outside the kernel and make all the security
decisions there.
- Assign security rights to applications as well as users. (I have often
wondered why no OS does this already...)
Some interesting ideas there...
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible <voi### [at] devnull> wrote:
> - Let the compiler enforce program isolation, not the processor
> hardware. (That works great if everybody uses your compiler, but I'm not
> sure what happens if you allow arbitrary 3rd party code to execute...)
The only way to make an OS secure is to have hardware support. Hardware
is the only thing that can stop a program from accessing what it must not
access.
(Ok, there's another alternative: Run the programs under an emulator.
Of course this is out of question because of speed issues.)
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> - Let the compiler enforce program isolation, not the processor
>> hardware. (That works great if everybody uses your compiler, but I'm not
>> sure what happens if you allow arbitrary 3rd party code to execute...)
>
> The only way to make an OS secure is to have hardware support. Hardware
> is the only thing that can stop a program from accessing what it must not
> access.
>
> (Ok, there's another alternative: Run the programs under an emulator.
> Of course this is out of question because of speed issues.)
Their approach seems to be to "verify" each program before it runs,
checking that it doesn't do any "bad" things.
Presumably verifying whether a program does or does not do something
"bad" is formally equivilent to the halting problem, so I imagine they
apply some arbitrary set of restrictions to simplify the problem.
Singularity is of course a research experiment, not a production-grade
OS. It would be interesting to see if they could make it work in the
face of hostile 3rd party code...
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible <voi### [at] devnull> wrote:
> Their approach seems to be to "verify" each program before it runs,
> checking that it doesn't do any "bad" things.
That's impossible. It can be proven that it's an unsolvable problem,
exactly for the same reason as the halting problem is unsolvable. There's
no way for any program to check if a piece of code is executed and how.
It's also impossible for it to know, for example, the addresses of all
pointers by simply examining the program (for example the address of a
pointer could be calculated from user input).
> Presumably verifying whether a program does or does not do something
> "bad" is formally equivilent to the halting problem, so I imagine they
> apply some arbitrary set of restrictions to simplify the problem.
Those restrictions could seriously hinder compiler optimizations.
For example accessing the nth element of an array can usually be done
with a simple CPU opcode. However, if the system restricts this because
it cannot prove what that n might contain, it means that the compiler
cannot generate the single opcode for accessing that array, but must
perform something much more complicated to keep the system happy.
Ah, but that's the trend nowadays: Computers get faster and the amount
of RAM grows exponentially with time. There's no need for highly optimized
code.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> Their approach seems to be to "verify" each program before it runs,
>> checking that it doesn't do any "bad" things.
>
> That's impossible. It can be proven that it's an unsolvable problem,
> exactly for the same reason as the halting problem is unsolvable.
In the general case, it's definitely unsolvable. I'm not sure precisely
what they're doing to "make" it solvable - but clearly it must involve
some kind of limitation or other.
>> Presumably verifying whether a program does or does not do something
>> "bad" is formally equivilent to the halting problem, so I imagine they
>> apply some arbitrary set of restrictions to simplify the problem.
>
> Those restrictions could seriously hinder compiler optimizations.
It's hard to say, but it *also* appears that Singularity runs some kind
of portable VM code.
From what I can gather, when you "install" an application, a verifier
checks that the VM code doesn't do any "bad" things, and then compiles
it to native code - whatever that might be. Then when you run the
application, it just runs the native code, trusting that it can't
possibly do bad things.
Apparently it "works" in their research prototype. Whether it could work
in a real-world OS is another matter entirely...
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
In a nutshell:
http://msdn.microsoft.com/en-gb/magazine/cc163603.aspx
Make of that what you will...
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp schrieb:
> Invisible <voi### [at] devnull> wrote:
>> Their approach seems to be to "verify" each program before it runs,
>> checking that it doesn't do any "bad" things.
>
> That's impossible. It can be proven that it's an unsolvable problem,
> exactly for the same reason as the halting problem is unsolvable. There's
> no way for any program to check if a piece of code is executed and how.
>
> It's also impossible for it to know, for example, the addresses of all
> pointers by simply examining the program (for example the address of a
> pointer could be calculated from user input).
If you only allow safe-mode managed code, pointer arithmethic is not
possible. I don't see a big problem to validate managed code, ensuring
it doesn't do anything "bad" for a fixed definition of "bad".
Manuel
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible wrote:
> I don't know about Singularity and it's particular design goals, but the
> other day I was thinking: What would happen if you set out to design a
> new OS completely from scratch? What would that look like?
Dude? That's Singularity. :-)
Download the release and read the design notes, too.
http://codeplex.com/singularity
> What if we wanted to shake that up a litle? What might we decide to do
> differently?
Read the Singularity papers. This is exactly the premise they started with.
> How about if, say, the program could somehow "tell" the OS what
> arguments it actually accepts?
Yep! And what if, when you installed the program, it said "Sorry, but
this program requires the ability to connect to the USB printer, and you
don't have the right USB driver installed"? Or "I can't let you install
telnet[1] until you have some sort of TCP/IP stack installed."
[1] Note that "install" means "make available for running." Of course
the uninstalled code can sit out there.
> Hey, let's go one better. The majority of CLI arguments are either
> on/off switches or filenames, right?
Only in a UNIX-like OS. The majority of arguments are instructions on
how to treat the other arguments, or references to something in the file
system namespace, there.
Once you can specify as arguments things beyond stuff represented as
strings, you get a whole nuther ball of wax. You're not stuck with
stdin, stdout, and stderr, for example. You can say "Hey, this program
needs an interrupt and two DMA channels to serve as a driver" for example.
> Once you start thinking this way, you start to see that actually, if we
> get the OS to interpret the arguments and pass *structured* data to the
> program [rather than just a blob of textual data], suddenly all sorts of
> interesting ideas become possible.
Read the bits about "compile-time reflection". You really don't even
need to "tell" the OS this info, if it's reflected in the types your
system supports.
> If nothing else, it means that CLI arguments now have a standardised
> format, enforced by the OS, which makes it easier to learn how to
> operate each new program. Maybe all the program does is somehow list a
> bunch of settings it requires? Maybe then you can specify those either
> by CLI arguments, or a per-user or per-machine set of defaults? Maybe
> the OS has a database of these default settings somewhere? All kinds of
> interesting ideas to throw around.
Yep. Singularity calls it the "application manifest". An application is
a first-class object, rather than being "a pile of files full of code".
> Maybe we can do something more interesting here? Maybe we can pass
> *structured* data around instead of just plain character streams?
Yes. Structured and typed, including a finite state machine to say when
it's OK to send and what you need to be ready to receive, checked at
compile time to ensure your code actually obeys the protocol, then
compiled down to native code and never checked again.
> Maybe we can do something better here too? Maybe we could have a small
> set of standard categories like "program bug", "resource exhaustion",
> "the network won't answer me", and provide a set of application-specific
> codes for the actual failures that a particular program can have?
Nah. You just answer back on the stream that goes to whoever invoked
you. :-) Why would only the parent want to know how you exited?
> OS how to invoke different levels of logging? Just some ideas.
SDN 14 Tracing.pdf
> I suppose you could go down the route of having files contain structured
> data - but again you're going to get people arguing over the best way of
> structuring things.
Not any more than saying "you'll have people arguing about the best way
to represent structures".
You're still thinking UNIXy. Get rid of the mindset that you have to
agree on data formats and embrace the mindset that you only have to
agree on APIs. You don't need to stick some Perl script in the middle of
a pipeline to transform your data. You present the data in a
semantically-meaningful way.
I.e., your directory isn't a file with 16-byte entries, the first two
bytes of which is a i-node number, and if non-zero, is followed by up to
14 bytes nul-terminated file name.
Your directory, instead, is a set of function calls like "read first",
"read next", "provide details". You don't have to come up with some
on-disk format to define.
> I've often thought about what would happen if, say, Smalltalk was the
> entire OS. Then the OS would "know about" the internal workings of each
> program to a large degree, and that opens up some rather interesting
> possibilities. Things like highly structured IPC and so forth. Trouble
> is, now you can only run stuff implemented in Smalltalk...
Yep. That's traditionally been the problem. Singularity does this, but
makes MSIL the bottom level for applications and such. So anything you
can compile into structured typed assembler language you can use. This
includes C#, F#, Iron Python, etc.
> In short, once you sit down and start to question the way OSes work
> today, you start to see that there are actually many things we could be
> doing differently - ranging from the conservative to the highly radical.
> (To me, really radical ideas are interesting to think about but probably
> wouldn't work too well in practice.)
It seems to be working well in practice. For example, one radical idea
(which I always thought would be a good idea) is to use safe languages
for everything. Singularity does this, and in so doing, can run
everything in Ring 0 and with no hardware memory protection. It actually
runs faster, because it takes less time to enforce array bound checks
(for example) than it does to go thru the memory mapping hardware on
every access to memory. Turning off memory protection more than makes up
for doing it in software. And then, doing things like scheduling
threads, adding space to the thread stack, allocating and freeing memory
blocks ... all that is in user space, because the compiler can inline
the kernelesque instructions.
> Heh, if *I* had 3 years to sit and write an OS, maybe I could experiment
> with a few of these ideas? ;-)
Read the papers first. It's exactly what I've been wanting to do myself,
except they figured out what seems a really good way of doing it.
I like the stuff on permissions, too. Stuff like "setuid" not being a
privileged operation is kind of funky. :-)
Really, all the stuff you're speculating about, they've written about in
detail and implemented. It's very cool. I highly suggest if the idea
"what if we started over in *this* millenium?" interests you, you read
the literature they've published. :)
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Manuel Kasten <kas### [at] gmxde> wrote:
> If you only allow safe-mode managed code, pointer arithmethic is not
> possible.
So you can't have arrays?
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|