 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
>>> I don't know about Singularity and it's particular design goals, but
>>> the other day I was thinking: What would happen if you set out to
>>> design a new OS completely from scratch? What would that look like?
>>
>> Dude? That's Singularity. :-)
>
> Not quite. They didn't do it the way *I* would do it. ;-)
Then you should have said "What would happen if *I* set out to design a
new OS completely from scratch?" ;-)
>> Download the release and read the design notes, too.
>> http://codeplex.com/singularity
>
> Uh... why? It's an experimental research prototype. It probably doesn't
> even run yet.
You say this with such confidence. Yet, surprisingly, you didn't
actually (say) read any of the papers, and noticed they're all dated
five years ago.
> They were focused specifically on "how do we increase security?" I'm
> just thinking "what widely-used but suboptimal abstractions might we
> change?"
No. You didn't read the papers, so you don't know what they're working
on. It's better to read what the authors wrote than what some blogger
says about what the authors wrote.
>> Or "I can't let you install telnet[1] until you have some sort of
>> TCP/IP stack installed."
>
> Isn't this what RPM does?
No.
>> Once you can specify as arguments things beyond stuff represented as
>> strings, you get a whole nuther ball of wax. You're not stuck with
>> stdin, stdout, and stderr, for example. You can say "Hey, this program
>> needs an interrupt and two DMA channels to serve as a driver" for
>> example.
>
> A device driver is a rather unusual type of program. I'm thinking more
> about end-user level stuff. You know - the kind of thing you might
> invoke by hand.
Right. Of course, since they're writing the OS, they're worried about
the sorts of problems drivers cause. But the same result applies to
things you invoke by hand or from other programs.
> As an aside... Whenever I compile a Haskell program, it generates a
> *.manifest file that contains some random XML. Any idea WTF that's about?
Well, a manifest is a list of what's included. Other than that, I
couldn't help you guess without seeing one.
>> Nah. You just answer back on the stream that goes to whoever invoked
>> you. :-) Why would only the parent want to know how you exited?
>
> Maybe because it's a lights-out system and you want the failed process
> to be started back up again? IDK.
No, I'm saying why would you want the exit status to *ONLY* go to the
parent process, and not to whoever you want it to go to? Why not list in
the application manifest all the applications that'll be interested in
knowing that program X failed? Wouldn't you want everyone using the TCP
stack to know that the nic driver failed?
>> You're still thinking UNIXy. Get rid of the mindset that you have to
>> agree on data formats and embrace the mindset that you only have to
>> agree on APIs.
>
> I guess if you follow all this to its logical conclusion, you end up
> with "the filesystem is a relational database" - and we all know what a
> bad idea *that* was!
Uh, no. You wind up with "everything is strongly typed", not necessarily
"everything is the same type". I am not sure I've discovered exactly
what they store in files - I'm still going thru the papers - but I'm
pretty sure it's not relational.
It's not unlike the Amiga OS in that respect, except safe and strongly
typed.
>> Yep. That's traditionally been the problem. Singularity does this, but
>> makes MSIL the bottom level for applications and such. So anything you
>> can compile into structured typed assembler language you can use. This
>> includes C#, F#, Iron Python, etc.
>
> (Or Haskell, when they fix the bitrot in the MSIL backend.)
Yep. Or most anything. I'm pretty impressed that they managed to get
functional languages doing their thing in an OO assembler language.
> One day, I'll have to sit down and find out how the Java VM or the CLR
> work.
It's ugly. I think the JVM is probably a little easier to understand.
But think of it merely as strongly typed assembler language with lots of
metadata about types and layouts.
> Yah, but this only really works if you're not going to execute arbitrary
> C code - which would be kind of a problem.
Exactly. Why do you need to execute arbitrary C code, tho? Other than
compatibility?
> Specifying access control by application seems like a perfectly logical
> thing to want to do. That whole Unixy trip with creating a user and
> group named "apache" and making sure the Apache httpd runs under that
> account just seems like a huge kludge to me...
Yes, exactly. Singularity lets you specify it as both, including the
history. So "PHP running from user Fred invoked via Apache" can have
different permissions from "PHP running from user Fred invoked via bash".
> ....and what do you think I just spent my entire afternoon doing? :-P
OK. Well, some of your assertions about how it works were at odds with
what they wrote, so I assumed you hadn't read all the way through.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Mike Raiford wrote:
> Warp wrote:
>> Manuel Kasten <kas### [at] gmx de> wrote:
>>> If you only allow safe-mode managed code, pointer arithmethic is not
>>> possible.
>>
>> So you can't have arrays?
>>
>
> You can, but not in the sense of a contiguous block of memory containing
> the data sense.
Actually, yes, you can. You just bounds-check the array. You can do
that in a C implementation, even. People just don't for some reason.
Indeed, the language they use for the OS has "representation structures"
which are specifically designed to (for example) land in certain
memory-mapped hardware bits.
There's no problem supporting arrays. Arrays are objects. The problem is
supporting arbitrary untyped pointers assigned non-pointer values -
i.e., the problem is casting an integer to a pointer.
> The question I have is what if I want to develop an application (such as
> a high performance image analysis package) against a platform that uses
> only managed code. Could it be done using the CPU the most efficiently?
Sure, why not? If the compiler can prove you're not violating the memory
constraints, why not? Note that their tests show it's actually more
efficient to check in software than in hardware, and the software checks
are pretty efficient.
> If I were restricted to "safe" code, is there a way to remove that
> restriction for that app?
No. That's the point.
I mean, I suppose, sure, you could. But it's not going to be a regular
app. You'd need to install it differently and prove you're allowed to.
That's how the kernel, for example, works. It's like asking "can I
write code in a Linux app that bypasses the memory mapping hardware?"
Sure, but it's far from normal.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Mike Raiford <mra### [at] hotmailcom> wrote:
>>> So you can't have arrays?
>
>> You can, but not in the sense of a contiguous block of memory containing
>> the data sense.
>
> But then the answer is "no".
I don't think Mike read the papers. Of course you can have an array of
contiguous memory. You declare it as an array of structs, just like you
would in C.
They even have a mechanism whereby you can declare a struct with a
definite memory layout that multiple different languages can reference,
and an operator that says "treat this as the representation of an
object", which basically adds the vtable after the fact for your
particular program. I.e., you can cast an object in memory from a flat
data structure into a full object-oriented object with inheritance and
methods and all that, without moving the memory that holds the fields.
And since you're sharing that memory with different languages, you can
have the different languages cast it into different objects without
munging it up for any one particular language.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> I have really hard time believing that if you, for example, calculate
> the sum of all the integers in an array, adding boundary checks to every
> single read operation will add only 4% of overhead.
The compiler can be pretty smart. You can actually optimize out the
bounds checking most of the time.
int x[50]; int y;
for (i = 0; i < 50; i++) x[i] = i;
for (i = 0; i < 50; i++) y += x[i];
That won't have any bounds-checking code included.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
>> Download the release and read the design notes, too.
>> http://codeplex.com/singularity
>
> Uh... why? It's an experimental research prototype. It probably doesn't
> even run yet.
BTW, the reason I told you to download the release is because there's
extensive high-level documentation included in the release. There's a
bunch of PDF files that aren't on the web site that describe how the
system works, how the verification works, the graph walking that proves
everything in a system will boot and shows what order to start drivers
and applications in, and so on.
Not because I expected you to run the code, or even read the source for
that matter.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> The compiler can be pretty smart.
If you are doing a simple linear traversal, maybe, but if it's any more
complicated than that...
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> For example accessing the nth element of an array can usually be done
> with a simple CPU opcode. However, if the system restricts this because
> it cannot prove what that n might contain, it means that the compiler
> cannot generate the single opcode for accessing that array, but must
> perform something much more complicated to keep the system happy.
Here's an example of where the 4% comes from. The compiler might
generate a single op-code, and that single op-code might take hundreds
of cycles to run, because it hits a page whose virtual address map isn't
in the cache. Or, even worse, it hits a page that isn't in memory at
all. But sure, I suppose if your program consists primarily of random
access to an array of stuff that you could do in one cycle, and your
cache coherency sucks, you might take a slight extra hit for bounds
checking. I guess things like photoshop plug-ins for distorting an image
might take something of a hit. Something like a SQL server would
probably run faster than on hardware-protected processes. The 4% was
from their compiler/verifier/code generator, IIRC.
There's another cool thing they do. Each thread starts with only a 4K
stack (i.e., one page). The installer (that compiles from MSIL to native
code, called "bartok" for some reason) will build a call map, figure out
which function calls *might* pass a page boundary, and insert in-line
code to allocate another page of memory. Then it copies the appropriate
number of arguments to the new stack frame, after including a return
address which will deallocate that new page of memory. So instead of
allocating a meg of memory for stack space for each thread, or instead
of trapping out when you run off the end and trying to rearrange things,
instead you have a bunch of randomly-allocated pages holding your stack,
linked together with compiler-generated code to allocate and deallocate
pages as needed. The compiler also makes sure there's enough space at
the top of any given page to hold the stack of any interrupt routine
that might run, so you don't even have to deal with switching pages
around for that. And when the code *does* call into the kernel, it just
allocates a new stack page for that and makes the call, and marks that
stack page as belonging to the kernel, so the GC doesn't start reaping
things it shouldn't and so the process can get cleaned up if it exits
during a call-back from the kernel. But if the compiler can look at the
call graph and figure out that either you *won't* overflow the stack
frame, or you *will* overflow the stack frame, there's no need to even
put in the check - you can just put in the code (or not) do do the right
thing.
And a lot of this gets inlined in the code, because they know what
kernel you're "linked" against, and they know you can't execute the
arbitrary code, so you're often not even "trapping" into the kernel to
allocate memory or send messages between processes or schedule threads
or whatever.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Darren New <dne### [at] sanrrcom> wrote:
>> The compiler can be pretty smart.
>
> If you are doing a simple linear traversal, maybe, but if it's any more
> complicated than that...
True. One advantage the compiler[1] has is that it has the entire source
code in front of it when it compiles. So it can check that everywhere an
index gets called, the value is within range, for example.
But yes, obviously if the compiler could prove *everything*, there
wouldn't be a 4% slow-down with the compiler adding the checks. :-)
[1] Using the term loosely here...
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>>> Dude? That's Singularity. :-)
>>
>> Not quite. They didn't do it the way *I* would do it. ;-)
>
> Then you should have said "What would happen if *I* set out to design a
> new OS completely from scratch?" ;-)
Well :-P to you.
>>> Download the release and read the design notes, too.
>>
>> Uh... why? It's an experimental research prototype. It probably
>> doesn't even run yet.
>
> You say this with such confidence. Yet, surprisingly, you didn't
> actually (say) read any of the papers, and noticed they're all dated
> five years ago.
And this makes a difference? It's not meant to be an end-user OS. It's
meant for OS hackers to play with.
>> They were focused specifically on "how do we increase security?" I'm
>> just thinking "what widely-used but suboptimal abstractions might we
>> change?"
>
> No. You didn't read the papers, so you don't know what they're working
> on.
Sure I did. They have software-enforced type and memory safety so they
can turn off hardware protection and make IPC really cheap. They also
statically check IPC. I don't see anything anywhere about, say, how end
users invoke programs.
>>> Or "I can't let you install telnet[1] until you have some sort of
>>> TCP/IP stack installed."
>>
>> Isn't this what RPM does?
>
> No.
Really? I thought that was the entire *point* of package managers. (And
also the reason that as soon as you attempt to upgrade any Linux
installation, it breaks on 50,000 instances of "wrong version of glibc"
or something similar.)
>>> Nah. You just answer back on the stream that goes to whoever invoked
>>> you. :-) Why would only the parent want to know how you exited?
>>
>> Maybe because it's a lights-out system and you want the failed process
>> to be started back up again? IDK.
>
> No, I'm saying why would you want the exit status to *ONLY* go to the
> parent process, and not to whoever you want it to go to?
Well sure, other programs might want to know as well. I was just
pointing out that there may not *be* a human sitting at the console,
that's all. ;-)
>> I guess if you follow all this to its logical conclusion, you end up
>> with "the filesystem is a relational database" - and we all know what
>> a bad idea *that* was!
>
> Uh, no. You wind up with "everything is strongly typed", not necessarily
> "everything is the same type".
You still have to get everybody to agree on what constitutes a "type".
Ask a BASIC programmer and they'll tell you a "type" is either
"integer", "float" or "string". Ask a Pascal programmer and they'll tell
you a "type" is a unique identifier that identifies an array or a
record. Ask an OOP expert and they'll tell you a "type" is a class. You
don't even wanna *know* what a Haskell programmer has to say about the
matter...
Seriously, do you have *any idea* how many standards have been put
forward for "store digital audio in a file"? ;-)
> It's not unlike the Amiga OS in that respect, except safe and strongly
> typed.
Um... AmigaDOS files are streams of octets, just like every other OS.
>>> Yep. That's traditionally been the problem. Singularity does this,
>>> but makes MSIL the bottom level for applications and such. So
>>> anything you can compile into structured typed assembler language you
>>> can use. This includes C#, F#, Iron Python, etc.
>>
>> (Or Haskell, when they fix the bitrot in the MSIL backend.)
>
> Yep. Or most anything. I'm pretty impressed that they managed to get
> functional languages doing their thing in an OO assembler language.
Uh, yeah... Haskell really doesn't fit the MSIL very well. I'm told it's
not very performant there. (I have no idea about F# - but when I
researched it, it didn't appear to be very functional.)
>> One day, I'll have to sit down and find out how the Java VM or the CLR
>> work.
>
> It's ugly. I think the JVM is probably a little easier to understand.
> But think of it merely as strongly typed assembler language with lots of
> metadata about types and layouts.
I'm just wondering how you design assembler so that it can be run
efficiently on multiple targets, that's all. (I hear it's a stack
machine rather than a register machine, for example. Aren't Wikipaths fun?)
>> Yah, but this only really works if you're not going to execute
>> arbitrary C code - which would be kind of a problem.
>
> Exactly. Why do you need to execute arbitrary C code, tho? Other than
> compatibility?
Oh, well, other than the "minor detail" of compatibility, there's no
problem at all! ;-)
(You recall that "Linux" is actually a tiny bit of software which
inherited compatibility with Unix, thus earning an instant library of
userland tools, right?)
>> Specifying access control by application seems like a perfectly
>> logical thing to want to do. That whole Unixy trip with creating a
>> user and group named "apache" and making sure the Apache httpd runs
>> under that account just seems like a huge kludge to me...
>
> Yes, exactly. Singularity lets you specify it as both, including the
> history. So "PHP running from user Fred invoked via Apache" can have
> different permissions from "PHP running from user Fred invoked via bash".
...which makes significantly more sense.
(Actually, maybe PHP is a bad example. Perhaps you want to assign
different permissions to each PHP script? Rather than just to the PHP
interpretter?)
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
> And this makes a difference? It's not meant to be an end-user OS. It's
> meant for OS hackers to play with.
It's designed to support research into building another OS that uses the
same technologies. They even tell you the name of the "production" OS,
whose name starts with an "M" but I don't remember offhand. :-)
> Sure I did. They have software-enforced type and memory safety so they
> can turn off hardware protection and make IPC really cheap. They also
> statically check IPC. I don't see anything anywhere about, say, how end
> users invoke programs.
Fair enough. They actually show it in the videos. It looks like a pretty
straightforward CLI thing.
>>>> Or "I can't let you install telnet[1] until you have some sort of
>>>> TCP/IP stack installed."
>>>
>>> Isn't this what RPM does?
>>
>> No.
>
> Really? I thought that was the entire *point* of package managers.
To some extent. Package managers tell you which dynamic libraries are
needed for which programs. They don't enforce anything, and you cannot
(for example) look at an RPM without installing it and know if it'll
work right once you're done installing it.
It's a small part of what RPMs are supposed to do, and they do it
relatively poorly. Enough so that I'd have to say "no, they're different
things."
>> No, I'm saying why would you want the exit status to *ONLY* go to the
>> parent process, and not to whoever you want it to go to?
>
> Well sure, other programs might want to know as well. I was just
> pointing out that there may not *be* a human sitting at the console,
> that's all. ;-)
Right. I'm not sure how we wound up talking past each other. :-)
>> Uh, no. You wind up with "everything is strongly typed", not
>> necessarily "everything is the same type".
>
> You still have to get everybody to agree on what constitutes a "type".
Sure. But there's a least-common-denominator that gets passed back and
forth, called the "rep types". Basically, stuff everyone can represent.
And that's why you can take the rep type and cast it to a
language-specific type.
> Seriously, do you have *any idea* how many standards have been put
> forward for "store digital audio in a file"? ;-)
Sure. But they're all of the same type by the time you talk to the codec
to get the data out of them.
>> It's not unlike the Amiga OS in that respect, except safe and strongly
>> typed.
>
> Um... AmigaDOS files are streams of octets, just like every other OS.
No they're not. Amiga OS devices are things that listen for and respond
to typed messages. Certainly the narrator isn't a "stream of octets",
nor is the clock, nor is the audio device.
Files, yes, to some extent (i.e., discounting the metadata). But you
don't access files in the Amiga OS. You access drivers. Files on disk
are one small part of it. And even the directories aren't arrays of bytes.
> Uh, yeah... Haskell really doesn't fit the MSIL very well. I'm told it's
> not very performant there. (I have no idea about F# - but when I
> researched it, it didn't appear to be very functional.)
The only thing I'd heard is that F# is apparently a port of Caml to
.NET. You now know as much about it as I do. :-)
> I'm just wondering how you design assembler so that it can be run
> efficiently on multiple targets, that's all.
One of the things .NET does, for example, is require that every path
through the MSIL that gets to the same opcode has to have the same types
on the stack at that point. For example. Apparently this makes it
easier to generate good code, because you can statically assign
addresses or registers to what's on the stack.
>>> Yah, but this only really works if you're not going to execute
>>> arbitrary C code - which would be kind of a problem.
>>
>> Exactly. Why do you need to execute arbitrary C code, tho? Other than
>> compatibility?
>
> Oh, well, other than the "minor detail" of compatibility, there's no
> problem at all! ;-)
Right. How much C is there that couldn't be ported with relative ease to
C#? Of course, if you want to maintain compatibility, you're not going
to learn much in a research system. And if your company's reason for
existence is to write software, maintaining compatibility with the other
peoples' software isn't that big a deal.
Now, if something's written in C++, it might be harder to port, yes. And
it would be hard to automate porting anything, of course. Or you could
just make a C compiler that generates safe code - if your program works,
it probably wouldn't be too difficult.
And if you're targetting a new platform, compatibility isn't too
important. How much legacy code is there for in-dash car computers, or
TiVo-like media systems?
> (You recall that "Linux" is actually a tiny bit of software which
> inherited compatibility with Unix, thus earning an instant library of
> userland tools, right?)
Sure. Most of which suck. ;-) Just look at the file system layout you
wound up with.
> (Actually, maybe PHP is a bad example. Perhaps you want to assign
> different permissions to each PHP script? Rather than just to the PHP
> interpretter?)
That too. Or different permissions to "Fred logged in via SSH with a
certificate" vs "Fred logged into the console with a password". So you
can put more trust on certificates, or smart cards, or whatever, at the
application level, but built into the ACL system. So you can look at the
static ACLs in the system, and know you can't get to see your Quicken
data unless you used the smart card to log in.
What's cool is, you can also ignore what program is claiming to be Fred,
and just say "Anyone that claims to be Fred can see this, regardless of
whether he logged in or not as Fred." It's up to the individual
programs as to which apps they trust to give good authentication.
--
Darren New / San Diego, CA, USA (PST)
Ever notice how people in a zombie movie never already know how to
kill zombies? Ask 100 random people in America how to kill someone
who has reanimated from the dead in a secret viral weapons lab,
and how many do you think already know you need a head-shot?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
