On 13/10/2011 07:48 PM, Darren New wrote:
> On 10/13/2011 1:28, Invisible wrote:
>> Um, why did it page it *out* in the first place?! I thought it was only
>> supposed to swap stuff out if the space was needed for something else?
>
> Other stuff, like Windows Update or disk defragger, might have run in
> the background. Note that processes like Superfetch are supposed to
> minimize this effect.

I thought that's only in newer versions of Windows. I'm using XP here.

> (I saw an interesting lecture on the changes
> they'd made to prevent "back from lunch syndrome.")

Yeah, I vaguely remember reading something like that... Can't remember 
how I found it now.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

On 13-10-2011 20:51, Darren New wrote:
> On 10/13/2011 9:12, andrel wrote:
>> Might be, but why design it this way? Why has cntrl-alt-del not a higher
>> priority than disk-IO and indeed any other program/process?
>
> Sure, but what's it going to do when it gets that keystroke? It has to
> fire up the code that handles it, presents the UI, etc. *That* is what
> takes time to page in.

So that moves the question to: why is the ctrl-alt-del handler paged 
out? And the UI: why has that to be on the GUI, with all the page faults 
that may result from swapping pixels to disk? Remember we are sending a 
low level high importance interrupt and the user knows that.

>> modern machines: why not dedicate one core to the OS and the OS alone?)
>
> The core isn't the problem.

I know. I am just asking if there is still a reason to handle the kernel 
like any other user process.

> The disk is the problem, because
> rescheduling disk is slow. You can interrupt the CPU in a few hundred
> machine cycles. You can't really interrupt a disk seek.

But, is there any reason to stop the entire kernel because of a disk seek?

> Indeed, I suspect if Windows figured out your startup programs, figured
> out that you actually have more than enough RAM to load them all, and
> would just load the entire file sequentially instead of letting them
> page in on demand, you'd cut boot time tremendously.
>
>> IIRC also the Amiga did not suffer from virtual lock up (nor do I
>> remember
>> it so badly from earlier version of MS-DOS).
>
> The Amiga didn't have demand paging, so you only suffered a long
> shut-down time when you had allocated a lot of small chunks of memory
> and needed to deallocate them before exiting. That happened on occasion.

That is true. And on many occasions it would still be sensible to load 
no more than what you have as memory. At least I am normally only using 
one large dataset at a time. Whenever it starts paging heavily I quit 
anyway to rethink what I am doing.


-- 
Apparently you can afford your own dictator for less than 10 cents per 
citizen per day.

Post a reply to this message

On 10/13/2011 12:55, andrel wrote:
> So that moves the question to: why is the ctrl-alt-del handler paged out?

How much of it do you want to keep in core? I've had very little trouble 
getting the CAD handler to come up in things like Vista, but the next step 
of picking something off the menu can cause a new program to get launched. 
However, it launches task manager or some such in XP, I forget.

> And the UI: why has that to be on the GUI, with all the page faults that may
> result from swapping pixels to disk? Remember we are sending a low level
> high importance interrupt and the user knows that.

Well, yes. That gets handled right away, most likely. Then it has to *do* 
something in response. I'm not sure what your point is. Your keystroke is of 
course taken into the kernel pretty much immediately. It's the thing that 
comes after I'm talking about.

> I know. I am just asking if there is still a reason to handle the kernel
> like any other user process.

The problem is not in the kernel's response. It's the fact that the 
complaint is the task manager takes a long time to come up, preventing you 
from killing the task.

In UNIX terms, it's like complaining that /bin/kill takes a long time to 
load when the system is thrashing, and you asking why the kill() kernel trap 
would be paged out in the first place.

> But, is there any reason to stop the entire kernel because of a disk seek?

No. Nor does that happen. Otherwise, disk seeks wouldn't continue to get in 
the way?

-- 
Darren New, San Diego CA, USA (PST)
   How come I never get only one kudo?

Post a reply to this message

Le 2011/10/13 12:12, andrel a écrit :

> IIRC also the Amiga did not suffer from virtual lock up (nor do I
> remember it so badly from earlier version of MS-DOS).
>
>

IIRC, MS-DOS did not use paging nor virtual memory.
The main reason probably been that it was made to run on a computer 
without any hard drive but just a single floppy or possibly two, at 
least, initialy.

Some applications did implement some paging using internal or external 
modules that where used only by a single application at a time.


Alain

Post a reply to this message

On 10/13/2011 15:14, Alain wrote:
> Le 2011/10/13 12:12, andrel a écrit :
>
>> IIRC also the Amiga did not suffer from virtual lock up (nor do I
>> remember it so badly from earlier version of MS-DOS).
>>
>>
>
> IIRC, MS-DOS did not use paging nor virtual memory.

It also didn't have multi-tasking, so you didn't need to deallocate just 
the 
memory for the job that was exiting. You could just delete all the memory
 
allocated since the program started.



-- 
Darren New, San Diego CA, USA (PST)
   How come I never get only one kudo?

Post a reply to this message

On 13-10-2011 23:48, Darren New wrote:
> On 10/13/2011 12:55, andrel wrote:
>> So that moves the question to: why is the ctrl-alt-del handler paged out?
>
> How much of it do you want to keep in core?

To address most issues in one: you have normal operation in a system and 
emergency situations. You should not design normal operations in such a 
way that you can not handle emergencies.
What i'd do* is have the CAD handler (and a primitive task handler) in a 
small space. Preferably in one self consistent page. Immediately suspend 
all running user level programs and switch to vga to prevent the GUI 
manager from page swapping.

*) at least without thinking all problems completely through.

> I've had very little trouble
> getting the CAD handler to come up in things like Vista, but the next
> step of picking something off the menu can cause a new program to get
> launched. However, it launches task manager or some such in XP, I forget.
>
>> And the UI: why has that to be on the GUI, with all the page faults
>> that may
>> result from swapping pixels to disk? Remember we are sending a low level
>> high importance interrupt and the user knows that.
>
> Well, yes. That gets handled right away, most likely. Then it has to
> *do* something in response. I'm not sure what your point is. Your
> keystroke is of course taken into the kernel pretty much immediately.
> It's the thing that comes after I'm talking about.
>
>> I know. I am just asking if there is still a reason to handle the kernel
>> like any other user process.
>
> The problem is not in the kernel's response. It's the fact that the
> complaint is the task manager takes a long time to come up, preventing
> you from killing the task.

Because the task manager is implemented at the same priority level as 
Blender?

> In UNIX terms, it's like complaining that /bin/kill takes a long time to
> load when the system is thrashing, and you asking why the kill() kernel
> trap would be paged out in the first place.

Can that happen?

>> But, is there any reason to stop the entire kernel because of a disk
>> seek?
>
> No. Nor does that happen. Otherwise, disk seeks wouldn't continue to get
> in the way?

What apparently happens is that a program causes a page fault, possibly 
holding up the kernel because it also needs a page. After the page is 
loaded it then gets an opportunity to create another page fault before 
the scheduler hands control back to the kernel.
My reaction as a user is: 'hello, I just pressed CAD, stop all other 
programs and process that interrupt before doing anything else!' 
Something is terribly wrong with the priorities of my machine.



-- 
Apparently you can afford your own dictator for less than 10 cents per 
citizen per day.

Post a reply to this message

On 10/14/2011 9:29, andrel wrote:
> What i'd do* is have the CAD handler (and a primitive task handler) in a
> small space.

I think this is more like what post-XP OSes do.

> Immediately suspend all running user level programs

Always mount a scratch monkey.

You really don't want the guy at the keyboard hitting CAD and suspending 
your corporate web server.

> and switch to vga to prevent the GUI manager
> from page swapping.

I don't think it goes that far, but certainly the newer CAD handlers could 
do something like that, visually speaking.

>> The problem is not in the kernel's response. It's the fact that the
>> complaint is the task manager takes a long time to come up, preventing
>> you from killing the task.
>
> Because the task manager is implemented at the same priority level as Blender?

No it isn't. The problem isn't with the CPU. The problem is with the disk. 
Now, the problem is that when the task manager pages in a page, now it's 
going to run long enough for Blender to ask for a page. Now you've started a 
disk seek, which you can't satisfy or abort before the task manager needs 
its next page.

One simple solution might be to page in the entire task-manager task rather 
than doing it on demand, but that's exactly the sort of "disk scheduling 
sucks in modern desktop OSes" that I'm talking about.

>> In UNIX terms, it's like complaining that /bin/kill takes a long time to
>> load when the system is thrashing, and you asking why the kill() kernel
>> trap would be paged out in the first place.
>
> Can that happen?

No. That's my point. You're complaining CAD is unresponsive, when the 
problem is that the user-level program you're trying to launch via CAD is 
unresponsive.

> What apparently happens is that a program causes a page fault, possibly
> holding up the kernel because it also needs a page.

The kernel doesn't need a page. That's why it's the kernel. The task manager 
needs the page. The task manager is a user-level program.

Analogously, kill(1) doesn't need a page. But /bin/kill does, because it's a 
user-level program, and it's the quickest way to actually invoke kill(1).

> After the page is loaded
> it then gets an opportunity to create another page fault before the
> scheduler hands control back to the kernel.

A page fault *is* handing control back to the kernel.

> My reaction as a user is: 'hello, I just pressed CAD, stop all other
> programs and process that interrupt before doing anything else!' Something
> is terribly wrong with the priorities of my machine.

Well, first, CAD is used for lots of stuff besides "I have too many page 
faults". Second, you don't want anyone sitting at the keyboard to shut down 
any process running on your machine. These *are* multi-user machines, you know.

-- 
Darren New, San Diego CA, USA (PST)
   How come I never get only one kudo?

Post a reply to this message

On 14-10-2011 19:09, Darren New wrote:
> On 10/14/2011 9:29, andrel wrote:
>> What i'd do* is have the CAD handler (and a primitive task handler) in a
>> small space.
>
> I think this is more like what post-XP OSes do.
>
>> Immediately suspend all running user level programs
>
> Always mount a scratch monkey.
>
> You really don't want the guy at the keyboard hitting CAD and suspending
> your corporate web server.

Nor do you want him to effectively freeze it for 10 minutes by doing 
something stupid. See also below.

>> and switch to vga to prevent the GUI manager
>> from page swapping.
>
> I don't think it goes that far, but certainly the newer CAD handlers
> could do something like that, visually speaking.
>
>>> The problem is not in the kernel's response. It's the fact that the
>>> complaint is the task manager takes a long time to come up, preventing
>>> you from killing the task.
>>
>> Because the task manager is implemented at the same priority level as
>> Blender?
>
> No it isn't. The problem isn't with the CPU. The problem is with the
> disk.

IIRC you have said that before. Actually several times and I have never 
contradicted you.

> Now, the problem is that when the task manager pages in a page,
> now it's going to run long enough for Blender to ask for a page. Now
> you've started a disk seek, which you can't satisfy or abort before the
> task manager needs its next page.
>
> One simple solution might be to page in the entire task-manager task
> rather than doing it on demand, but that's exactly the sort of "disk
> scheduling sucks in modern desktop OSes" that I'm talking about.
>
>>> In UNIX terms, it's like complaining that /bin/kill takes a long time to
>>> load when the system is thrashing, and you asking why the kill() kernel
>>> trap would be paged out in the first place.
>>
>> Can that happen?
>
> No. That's my point. You're complaining CAD is unresponsive, when the
> problem is that the user-level program you're trying to launch via CAD
> is unresponsive.

Actually my point is that it *should not* be a user program.

>> What apparently happens is that a program causes a page fault, possibly
>> holding up the kernel because it also needs a page.
>
> The kernel doesn't need a page. That's why it's the kernel. The task
> manager needs the page. The task manager is a user-level program.
>
> Analogously, kill(1) doesn't need a page. But /bin/kill does, because
> it's a user-level program, and it's the quickest way to actually invoke
> kill(1).
>
>> After the page is loaded
>> it then gets an opportunity to create another page fault before the
>> scheduler hands control back to the kernel.
>
> A page fault *is* handing control back to the kernel.

I know that (long, long time ago I studied OS's a bit). I obviously 
meant the part of the kernel (and user level program in badly designed 
systems) that handles the CAD.

>> My reaction as a user is: 'hello, I just pressed CAD, stop all other
>> programs and process that interrupt before doing anything else!'
>> Something
>> is terribly wrong with the priorities of my machine.
>
> Well, first, CAD is used for lots of stuff besides "I have too many page
> faults". Second, you don't want anyone sitting at the keyboard to shut
> down any process running on your machine. These *are* multi-user
> machines, you know.

I disagree, there is no other user on my machine than me*. And that is 
also why your corporate webserver is a wrong example. webservers and 
user machines are something different. You should not mix those. Or at 
least the OS should behave different. Specifically in the area that we 
are discussing. CAD on a webserver should leave everything running 
(probably should even be disabled), but when I press CAD on my keyboard 
because I want control, i *want control* and I know what other processes 
I am also stopping.

*) that is at least my view of the machine. Technically it is not true.

-- 
Apparently you can afford your own dictator for less than 10 cents per 
citizen per day.

Post a reply to this message

On 10/14/2011 12:36, andrel wrote:
>> You really don't want the guy at the keyboard hitting CAD and suspending
>> your corporate web server.
>
> Nor do you want him to effectively freeze it for 10 minutes by doing
> something stupid. See also below.

Agreed.

>>> Because the task manager is implemented at the same priority level as
>>> Blender?
>>
>> No it isn't. The problem isn't with the CPU. The problem is with the
>> disk.
>
> IIRC you have said that before. Actually several times and I have never
> contradicted you.

Well, sure. But the problem is that nobody implements effective disk 
priorities any more (which is what I'm saying), so just saying "priority" 
implies CPU priority.

> Actually my point is that it *should not* be a user program.

That's likely true.  However, the frequency with which this happens probably 
doesn't warrant a kernel mechanism for a UI for picking a task and killing 
it, either. Especially since many times it happens, it's nothing to do with 
a task that's still running that you could actually do anything about.

Indeed, it would seem what you'd really want is a way to indicate that if a 
job got to the point where it was paging too often, that would be enough to 
kill it or deprioritize its disk I/O, to the point of suspending it for a 
second or more each time some other process went after the disk, sort of 
like how Linux (seems to) does "low priority" disk I/O.

> I know that (long, long time ago I studied OS's a bit). I obviously meant
> the part of the kernel (and user level program in badly designed systems)
> that handles the CAD.

Oh, well, you want an entire process management system built into the 
kernel, invocable by a simple keystroke. Probably not realistic in modern OSes.

> I disagree, there is no other user on my machine than me*. And that is also
> why your corporate webserver is a wrong example. webservers and user
> machines are something different. You should not mix those.

If your corporate web server is thrashing, how are you going to stop it?

> OS should behave different. Specifically in the area that we are discussing.
> CAD on a webserver should leave everything running (probably should even be
> disabled), but when I press CAD on my keyboard because I want control, i
> *want control* and I know what other processes I am also stopping.

Well, as the Linux enthusiasts would say, "we eagerly await your patch." :-)

CAD should certainly not be disabled on a web server, as it does far more 
than just bring up the task manager. That's the other half of the problem.

In pretty much every situation other than disk thrashing, CAD works pretty 
much instantly. In many cases of disk thrashing, there's nothing you can do 
about it, like my example of sync'ing gigabytes of changes to the disk. And 
modern OSes are set up to treat file reading and page faulting as 
essentially similar in many ways (and then there's memmap, which adds 
another confusion on top), so it's not really even obvious when there's 
thrashing which process is causing it or what to do about it, because it's 
entirely possible there wouldn't be thrashing if there weren't 3 or 4 
programs all doing something at once.

> *) that is at least my view of the machine. Technically it is not true.

Well, that's kind of the point I'm making. :)

-- 
Darren New, San Diego CA, USA (PST)
   How come I never get only one kudo?

Post a reply to this message

On 15-10-2011 1:02, Darren New wrote:
> On 10/14/2011 12:36, andrel wrote:

>> Actually my point is that it *should not* be a user program.
>
> That's likely true. However, the frequency with which this happens
> probably doesn't warrant a kernel mechanism for a UI for picking a task
> and killing it, either. Especially since many times it happens, it's
> nothing to do with a task that's still running that you could actually
> do anything about.
>
> Indeed, it would seem what you'd really want is a way to indicate that
> if a job got to the point where it was paging too often, that would be
> enough to kill it or deprioritize its disk I/O, to the point of
> suspending it for a second or more each time some other process went
> after the disk, sort of like how Linux (seems to) does "low priority"
> disk I/O.
>
>> I know that (long, long time ago I studied OS's a bit). I obviously meant
>> the part of the kernel (and user level program in badly designed systems)
>> that handles the CAD.
>
> Oh, well, you want an entire process management system built into the
> kernel, invocable by a simple keystroke. Probably not realistic in
> modern OSes.

Taking both points together. Breaking out to a program that just lets 
you modify priorities would be enough. If that process runs at the 
highest level itself it would solve most of the problems.
That said, windows does have a priority mechanism, and the taskmanager 
is running at a high level, two levels higher than Blender and Matlab, 
so why does it not work?

>
>> I disagree, there is no other user on my machine than me*. And that is
>> also
>> why your corporate webserver is a wrong example. webservers and user
>> machines are something different. You should not mix those.
>
> If your corporate web server is thrashing, how are you going to stop it?

If it is because someone is using it for other things, like POVray, 
Blender or a game fire that person. If it is because of tasks that 
should run there fire the IT manager or the financial director, 
depending on who was responsible for buying the wrong infrastructure.

>> OS should behave different. Specifically in the area that we are
>> discussing.
>> CAD on a webserver should leave everything running (probably should
>> even be
>> disabled), but when I press CAD on my keyboard because I want control, i
>> *want control* and I know what other processes I am also stopping.
>
> Well, as the Linux enthusiasts would say, "we eagerly await your patch."
> :-)

Windows is open source now? I missed that.

> CAD should certainly not be disabled on a web server, as it does far
> more than just bring up the task manager. That's the other half of the
> problem.
>
> In pretty much every situation other than disk thrashing, CAD works
> pretty much instantly. In many cases of disk thrashing, there's nothing
> you can do about it, like my example of sync'ing gigabytes of changes to
> the disk. And modern OSes are set up to treat file reading and page
> faulting as essentially similar in many ways (and then there's memmap,
> which adds another confusion on top), so it's not really even obvious
> when there's thrashing which process is causing it or what to do about
> it, because it's entirely possible there wouldn't be thrashing if there
> weren't 3 or 4 programs all doing something at once.
>
>> *) that is at least my view of the machine. Technically it is not true.
>
> Well, that's kind of the point I'm making. :)


-- 
Apparently you can afford your own dictator for less than 10 cents per 
citizen per day.

Post a reply to this message