 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Optimization of dynamic languages...
http://steve-yegge.blogspot.com/2008/05/dynamic-languages-strike-back.html
--
Darren New / San Diego, CA, USA (PST)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] san rrcom> wrote:
> http://steve-yegge.blogspot.com/2008/05/dynamic-languages-strike-back.html
With all the talk about fancy new languages which are so super-efficient
and so super-fast and so super-secure and whatever, I not seen too much
discussion about one thing: What about dynamically loadable libraries?
Most of these people boast how their favorite programming language is
able to create cold fusion and solve world hunger in 0.1 seconds, yet
they usually are running their programs written in that language on an
OS which is completely based on libc and dynamically loadable libraries
to do the low-level stuff efficiently.
Most of what I have seen of these languages, especially the dynamic
ones, seems to be completely incompatible with DLLs.
It seems to me that everybody has completely forgotten or, more probably,
misunderstood the idea behind DLLs. When someone sees the acronym "DLL"
the only thing he can think of is the so-called "DLL hell" in Windows,
where your system directories are completely bloated with DLL files, with
all their dependency problems and such. In the unix side of the world,
with the DLL equivalents, ie. the .so files, it's more like they sit
there and that's it. They are not a bother, but nobody thinks why they
exist in the first place either.
DLL files in Windows don't exist because Microsoft got some fancy but
braindead idea and decided to impose it on everyone. .so files in unix
systems don't exist just because of the fun of it. They have a purpose.
The idea behind DLL files is to save memory: The vast majority of
programs use the exact same libraries. For example in linux almost 100%
of programs (regardless of the language they are written in) use libc,
plus a bunch of others. In Windows the figure is probably something
similar.
The amount of programs running at any given time can be counted in
the hundreds. The OS starts all kinds of processes and services just to
boot up. Every single one of them uses the exact same libc library, plus
a bunch of other common libraries. Given that libc is rather large, it
would be an enormous waste of memory if each single process would load
it for itself. If I'm not mistaken, libc is something like 2MB in size.
Multiply that by a couple hundred of processes.
The idea with dynamically loadable libraries is that they are loaded
only *once* into memory, and every running program which needs that
library uses that *one* instance of it. This means that even if you
have a couple hundred of processes running, libc is still only loaded
once into memory, requiring only that 2MB of RAM. Every single process
uses that one single libc instance.
Programming languages which have absolutely no support for dynamically
loadable libraries are riding on libc and the other libraries which the
OS directly supports. Without this system they would be hopeless memory
hogs. You wouldn't be able to build and entire OS created with that
programming language only. You *need* dynamically loadable libraries
for the system to work.
It just feels that people who advocate fancy languages just ignore
this fact: They are dependent on an efficient dynamically loadable
library system (almost always created with C), but they completely
ignore the fact, and instead diss C as if it was a pestilence. They
completely fail to explain how the system could work if they did not
have that pestilent C system below, running the system.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Most of what I have seen of these languages, especially the dynamic
> ones, seems to be completely incompatible with DLLs.
I don't follow. Both Java and .NET are basically 100% dynamically
loaded. Javascript and Python and such are also 100% dynamically loaded.
> DLL files in Windows don't exist because Microsoft got some fancy but
> braindead idea and decided to impose it on everyone. .so files in unix
> systems don't exist just because of the fun of it. They have a purpose.
Heh. I'm not sure why you think Microsoft and Unix have different
approaches to this. The only mistake Microsoft made was to have the
version number inside the DLL rather than part of the name of the DLL.
What lead to "DLL Hell" is two people updating the same system DLL with
incompatible bugs. Solved via the DLL Cache and side-by-side installs,
which is basically what UNIX does with .so's. I can't tell you how often
I've had to make a symlink from /usr/lib/libyadda.1.3.7.so to
/usr/lib/libyadda.1.3.8.so to make a compiled program run. So both
approaches have their flaws. :-)
> The amount of programs running at any given time can be counted in
> the hundreds. The OS starts all kinds of processes and services just to
> boot up. Every single one of them uses the exact same libc library, plus
> a bunch of other common libraries. Given that libc is rather large,
I suspect the amount of libc used by any particular program is somewhat
smaller, tho. The new Microsoft "Singularity" OS even inlines bits of
the kernel into your application - stuff like allocating pages of memory
doesn't even leave the subroutine doing it, let alone trap out.
> You wouldn't be able to build and entire OS created with that
> programming language only. You *need* dynamically loadable libraries
> for the system to work.
Check out the Singularity OS. Real live working OS, with one of the
fundamental assumptions being that once you start a process, *nothing*
in the executable can change. No dynamic loading, no plug-ins, etc.
> It just feels that people who advocate fancy languages just ignore
> this fact: They are dependent on an efficient dynamically loadable
> library system (almost always created with C), but they completely
> ignore the fact, and instead diss C as if it was a pestilence. They
> completely fail to explain how the system could work if they did not
> have that pestilent C system below, running the system.
I don't think that's what this lecture is about. It's about making the
run-time (and IDEs for that matter) of dynamic systems as efficient as
the run-time of static systems. For example, how do you inline calls to
a DLL even in a language like C? What if the overhead for calling the
floating point library in libc[1] is greater than the actual time used
to do the floating point ops?
[1] Remember - not all machines have floating point built in. Even the
8086 had the 8087 as optional.
--
Darren New / San Diego, CA, USA (PST)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp <war### [at] tagpovrayorg> wrote:
> completely fail to explain how the system could work if they did not
> have that pestilent C system below, running the system.
How does that "C system below" -- which is actually machine code below -- would
work without hardware below, running the system? How would java programs run
without the JVM?
Come on! There must always be something low-level there upon which we build
things up. It's surely important, but it doesn't mean we got to be always at
that level... I enjoy being at the top, quite entertaining a view... :)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> Warp wrote:
> > Most of what I have seen of these languages, especially the dynamic
> > ones, seems to be completely incompatible with DLLs.
> I don't follow. Both Java and .NET are basically 100% dynamically
> loaded. Javascript and Python and such are also 100% dynamically loaded.
It's not a question of whether it's dynamically loaded or not. It's
a question of whether programs which are running at the same time and
share a library use only one instance of that library, or is that library
loaded separately for both programs.
There may be speed optimization advantages when loading the library on
a per-application basis, but at the cost of enormously increased memory
consumption. If you have a couple of hundreds of programs using the same
big library, if the library is loaded for every single one of them, the
system will require huge amounts of extra RAM.
> > The amount of programs running at any given time can be counted in
> > the hundreds. The OS starts all kinds of processes and services just to
> > boot up. Every single one of them uses the exact same libc library, plus
> > a bunch of other common libraries. Given that libc is rather large,
> I suspect the amount of libc used by any particular program is somewhat
> smaller, tho. The new Microsoft "Singularity" OS even inlines bits of
> the kernel into your application - stuff like allocating pages of memory
> doesn't even leave the subroutine doing it, let alone trap out.
Loading a dynamically loadable library does not change the program being
executed (other than it's told where the functions it wants to call are
located).
Inlining kernel code into the application makes sense only if that
doesn't increase the memory footprint of the application considerably.
If 2MB of kernel code would be inlined into each single application which
is run, that would be madness. The amount of apps running in a typical OS
is quite large, usually in the hundreds.
> > You wouldn't be able to build and entire OS created with that
> > programming language only. You *need* dynamically loadable libraries
> > for the system to work.
> Check out the Singularity OS. Real live working OS, with one of the
> fundamental assumptions being that once you start a process, *nothing*
> in the executable can change. No dynamic loading, no plug-ins, etc.
Does that mean that every application has all the system libraries
statically linked to them, and this system library code exists in all
of them and are loaded into memory for all of them?
Memory hogging to the max.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> completely fail to explain how the system could work if they did not
> have that pestilent C system below, running the system.
Oh, and btw, download that there Singularity OS. Note the almost
complete lack of anything remotely approaching C in the source. The use
of C in Singularity is approximately like the use of assembler in
C-based operating systems - the stuff that's impossible to do in the
higher-level languages, like turning interrupts into appropriate stack
frames and saving off registers during context switches.
That was one of the premises they started with: what if nothing in the
OS was written in a language like C? :-)
--
Darren New / San Diego, CA, USA (PST)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
nemesis <nam### [at] gmailcom> wrote:
> How does that "C system below" -- which is actually machine code below -- would
> work without hardware below, running the system? How would java programs run
> without the JVM?
The difference is that nobody disses the hardware (ok, they do, but in a
different way) or the JVM, but they diss C.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New <dne### [at] sanrrcom> wrote:
> That was one of the premises they started with: what if nothing in the
> OS was written in a language like C? :-)
They will still need dynamically loadable libraries for the system to
be rational.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> It's not a question of whether it's dynamically loaded or not. It's
> a question of whether programs which are running at the same time and
> share a library use only one instance of that library, or is that library
> loaded separately for both programs.
I would think the read-only parts of the code could certainly be shared.
There's no good reason why things like Java .class files couldn't be
loaded into shared memory.
> There may be speed optimization advantages when loading the library on
> a per-application basis, but at the cost of enormously increased memory
> consumption. If you have a couple of hundreds of programs using the same
> big library, if the library is loaded for every single one of them, the
> system will require huge amounts of extra RAM.
Yep. Altho I would think you would need to actually measure how much of
the shared code is actually used and shared in modern applications.
> Loading a dynamically loadable library does not change the program being
> executed (other than it's told where the functions it wants to call are
> located).
Um, yes, it does. Sure. There's places you can branch to after loading
the code that you couldn't branch to before.
Or, to put it another way, say you have a C++ class, and nowhere in any
of your code do you reference the public integer field "xyz" that's in
that class. (Including nowhere do you invoke any methods of the class
that reference xyz.) The compiler Singularity uses would simply not
allocate space for that field in any of the classes. Now, if you
dynamically loaded code and passed it an instance of that class that
*did* reference the field, you'd be broken.
So, yes, dynamically loading code changes the executable *process*.
> Inlining kernel code into the application makes sense only if that
> doesn't increase the memory footprint of the application considerably.
Right. You'd have to measure it. Welcome to time/space tradeoffs. ;-)
>> Check out the Singularity OS. Real live working OS, with one of the
>> fundamental assumptions being that once you start a process, *nothing*
>> in the executable can change. No dynamic loading, no plug-ins, etc.
>
> Does that mean that every application has all the system libraries
> statically linked to them, and this system library code exists in all
> of them and are loaded into memory for all of them?
Depends on what you consider to be "system library code". The system is
broken into a large number of small processes. So there isn't really a
whole lot of "system library code" going on there. Stuff like "open()"
isn't in the kernel, so it's not really a library. The runtime is linked
in staticall, yes.
Each process does have its own garbage collector, but that's good
because each process can use a different garbage collector, depending on
the types of garbage it collects.
I think one of the things they're looking at is indeed shared-memory
libraries, but I don't think they've gone very far in that direction.
It did, yes, seem somewhat wasteful, and it's a research project, so
they may not be too worried about it. But I can't think of any
theoretical reason why having large blocks of executable code being
shared amongst multiple processes would be especially difficult in
Singularity. They just haven't done it yet.
--
Darren New / San Diego, CA, USA (PST)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp <war### [at] tagpovrayorg> wrote:
> nemesis <nam### [at] gmailcom> wrote:
> > How does that "C system below" -- which is actually machine code below -- would
> > work without hardware below, running the system? How would java programs run
> > without the JVM?
>
> The difference is that nobody disses the hardware (ok, they do, but in a
> different way) or the JVM, but they diss C.
They would if they were told to program in assembly or bytecode.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
