![](/i/fill.gif) |
![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Is there a way to call write() without blocking?
In more detail:
Say I'm taking a real-time stream (like live audio) in off a piece of
hardware. I want to write it to the disk as it comes in. The hardware buffer
is fairly small and needs to be emptied promptly into memory.
Obviously, I can set up a producer/consumer to write the buffer into memory
from the device, then write() that out to disk. Assume I have plenty of RAM.
But if the disk blocks for some reason (like, someone makes gigabytes of
changes to the disk then calls sync), the write() might block. And I'm
pretty sure select() or poll() isn't going to tell me this, right?
I'm thinking I allocate more memory in a circular kind of buffer, allocating
memory if the disk slows, potentially freeing it if it speeds up, but I'd
need to know when it's safe to reuse the buffer.
So I'm guessing the best way to do it is to have a thread for reading the
hardware into memory, and a thread for calling write() when each buffer is
ready, in your classic producer/consumer pattern.
But is there any other way to do it under Linux? Is there any call that (for
example) will start writing, then give you a signal or some such when the
buffer is available for refilling?
Now that I think on it, I guess a non-blocking write will write *some* bytes
of the buffer, and select() will tell you what's writable, but the write
itself will block while it copies bytes from your buffer into the kernel's
memory and such, yes? But that shouldn't be long compared to waiting on disk
revolutions or something, because it's all memory-to-memory, right? And does
that work for disk-based files? Or does that only apply to sockets, pipes, etc?
I've just never had to deal with real-time processing under Linux as such
before. The other real-time systems I've done have had message-passing type
kernels, so there wasn't blocking of buffers, but you had to be careful not
to reuse buffers before they were done. Is there anything like message
passing I/O in Linux these days, or is it just threads vs O_NONBLOCK?
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Darren New <dne### [at] san rr com> wrote:
> Is there a way to call write() without blocking?
Search for fcntl() and O_NONBLOCK.
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
clipka wrote:
> Darren New <dne### [at] san rr com> wrote:
>> Is there a way to call write() without blocking?
> Search for fcntl() and O_NONBLOCK.
I'm aware of that, but traditionally in UNIX that will still block on disk
I/O. (Like, if the disk isn't spun up or something like that.) Does it do
that in modern Linux kernels, or has that been changed?
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Darren New <dne### [at] san rr com> wrote:
> > Search for fcntl() and O_NONBLOCK.
>
> I'm aware of that, but traditionally in UNIX that will still block on disk
> I/O. (Like, if the disk isn't spun up or something like that.) Does it do
> that in modern Linux kernels, or has that been changed?
Unfortunately that's all I know about it. Maybe you'll need to do some
multithreading.
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
clipka wrote:
> Unfortunately that's all I know about it. Maybe you'll need to do some
> multithreading.
Very tl;dr of you. ;-)
It looks like the Linux *I* have at least never fails to block on disk I/O
even with O_NONBLOCK. Run time of several minutes, system time of a second,
user time of even less, and write() never returned less that writing a full
buffer, so I guess that means you get to block for disk and (grumble) learn
how to do Linux threading in C. :-) Damn.
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Darren New wrote:
> clipka wrote:
>> Unfortunately that's all I know about it. Maybe you'll need to do some
>> multithreading.
>
> Very tl;dr of you. ;-)
>
> It looks like the Linux *I* have at least never fails to block on disk
> I/O even with O_NONBLOCK. Run time of several minutes, system time of a
> second, user time of even less, and write() never returned less that
> writing a full buffer, so I guess that means you get to block for disk
> and (grumble) learn how to do Linux threading in C. :-) Damn.
>
From what I recall, 'fork' in Linux is not all that bad to use.
Direct from wikipedia:
http://en.wikipedia.org/wiki/Fork_(operating_system)
http://en.wikipedia.org/wiki/Fork-exec
I've never had to implement fork, but from reading about it, it doesn't
seem too bad.
While not exactly what you were looking for, I hope this helps some.
Tom
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Tom Austin wrote:
> From what I recall, 'fork' in Linux is not all that bad to use.
I think processes and threads in Linux are both created with the same system
call, just using different flags. Standard (v7-style) fork wouldn't work in
this case because the processes wouldn't be sharing memory.
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
On Mon, 06 Apr 2009 01:37:49 +0200, Darren New <dne### [at] san rr com> wrote:
> Is there a way to call write() without blocking?
http://lmgtfy.com/?q=asynchronous+IO+linux
Apparently there is a POSIX library for this stuff, with kernel support
from 2.6 onward. Perhaps that could be of use.
--
FE
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Fredrik Eriksson wrote:
> http://lmgtfy.com/?q=asynchronous+IO+linux
I take it you didn't actually read past the first line of the question? :-)
"A more subtle problem with non-blocking I/O is that it generally doesn't
work with regular files."
Given there are about 8 ways to do asynch I/O, none of which seem to work
with regular files, I was wondering if there was a 9th yet, or whether one
just does blocking I/O in a thread, which scales poorly but is good enough
for my application.
> Apparently there is a POSIX library for this stuff, with kernel support
> from 2.6 onward. Perhaps that could be of use.
Yeah, and the second link on the page you gave says "It sucks and its
broken." :-)
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |