 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible schrieb:
> So it seems pretty moot to me. :-P
A bit, yes. :-)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible <voi### [at] dev null> wrote:
> > In theory, POV-Ray needs to store (width*2) pixels.
Is that so even if you don't use antialiasing?
> > libpng may need to store
> > a bunch more rows at once while compressing.
> libpng only needs the current and previous row to run the pixel filter.
> I have no idea what the DEFLATE compressor needs.
Compression algorithms almost invariably use a buffer for the data, and
this buffer is often surprisingly small (like some tens of kilobytes).
(This means that most compression algorithms will *not* find and compress
repetitions which are gigabytes apart from each other. This is for the
simple reason that the compression routine has to use only a limited amount
of memory and be relatively fast.)
Some modern algorithms might use larger buffers (like some megabytes),
but they nevertheless don't achieve enormously better compression ratios
(for some reason there seems to be some kind of cutoff point after which
enlarging the compression buffer does not improve the compression of average
data significantly, but slows down the compression more than it's worth).
The problem with compressing exabytes of data is not the compression
algorithm itself, but the file format into which the compressed data is
stored. (Theoretically if the data is compressed as a stream then there
shouldn't be any theoretical limit to how much data you can compress.
However, if the file format has headers and other data expressing the
amount of compressed data, you may run into limitations.)
> I note that to store (2^31 - 1) x (2^31 - 1) pixels, where each pixel
> requires exactly 4 bytes, requires about 18 exabytes of storage. (!!)
Depends on what those pixels contain, and how smart your compression
algorithm is. If the image is completely filled with black pixels, using
a smart compression algorithm could compress it down to some tens of
bytes. (Actual compression algorithms, however, often fail to do this
because of the limited compression buffer. Some of the most naive
algorithms achieve surprisingly poor compression ratios when the input
is full of the same byte value. Other smarter algorithms are more capable
of taking advantage of this.)
Of course if your image actually contains something meaningful, the
best you can hope to compress (losslessly) is something like to 1/10th
of the original size.
I don't think many file systems even support files which are exabytes
large.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
clipka <ano### [at] anonymousorg> wrote:
> Warp schrieb:
> > As for POV-Ray itself, I bet the limit is 2147483648x2147483648 pixels
> > (or maybe even 4294967296x4294967296 pixels).
> On contemporary x64 machines, the limit will be much smaller - some 4e6
> x 4e6 pixels; more than that, and you'll crash through the 256 TeraByte
> address limit (contrary to legend, contemporary x64 processors only
> support 48-bit addresses, despite the architecture being designed for
> future extensibility to full-fledged 64-bit addresses).
> Windows will not allow more than some 7e5 x 7e5 pixels (8 TB), while
> Linux will not go beyond some 3e6 x 3e6 pixels (128 TB) due to
> per-process address space limits.
But those are not limitations of POV-Ray, they are limitations of the
hardware and/or operating system.
If POV-Ray were to be compiled on a system without those limitations,
it could probably achieve images of those sizes (in theory, of course,
because in practice it would take millenia to render such an image, as
noted elsewhere).
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp <war### [at] tagpovrayorg> wrote:
> clipka <ano### [at] anonymousorg> wrote:
> > Warp schrieb:
> > > As for POV-Ray itself, I bet the limit is 2147483648x2147483648 pixels
> > > (or maybe even 4294967296x4294967296 pixels).
> > On contemporary x64 machines, the limit will be much smaller - some 4e6
> > x 4e6 pixels; more than that, and you'll crash through the 256 TeraByte
> > address limit (contrary to legend, contemporary x64 processors only
> > support 48-bit addresses, despite the architecture being designed for
> > future extensibility to full-fledged 64-bit addresses).
> > Windows will not allow more than some 7e5 x 7e5 pixels (8 TB), while
> > Linux will not go beyond some 3e6 x 3e6 pixels (128 TB) due to
> > per-process address space limits.
> But those are not limitations of POV-Ray, they are limitations of the
> hardware and/or operating system.
> If POV-Ray were to be compiled on a system without those limitations,
> it could probably achieve images of those sizes (in theory, of course,
> because in practice it would take millenia to render such an image, as
> noted elsewhere).
Actually, thinking about it, you confused me.
POV-Ray doesn't need to keep the entire image in memory in order to render
it (after all, POV-Ray was developed on systems with limited amount of memory,
yet was able to render images larger than any conceivable RAM size back then).
Thus it's perfectly possible for POV-Ray to render a 2^31 x 2^31 pixels
large image even on a system which limits memory addresses to be smaller
than that.
(Of course you won't be able to *save* that image anywhere because you
would encounter limitations in the file system. But that doesn't stop
POV-Ray from *rendering* the image.)
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> libpng only needs the current and previous row to run the pixel filter.
>> I have no idea what the DEFLATE compressor needs. (Other than a complete
>> redesign...) All assuming you're not going to generate interlaced PNG,
>> that is.
>
> Well, with antialiasing off, it should be possible (in theory) to produce an
> interlaced PNG with O(1) memory too. You'd have to modify POV-Ray to
> compute the pixels in the same order they're stored in the PNG file.
If you generate a PNG image without any fancy filtering, then you don't
need any extra memory except for the DEFLATE compressor. But usually PNG
files use filtering to increase compression. (I don't know, but I'd
*hope* libpng provides a way to control this...) If filtering, you may
need the previous row of pixels [only].
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Invisible <voi### [at] devnull> wrote:
> If you generate a PNG image without any fancy filtering, then you don't
> need any extra memory except for the DEFLATE compressor. But usually PNG
> files use filtering to increase compression. (I don't know, but I'd
> *hope* libpng provides a way to control this...) If filtering, you may
> need the previous row of pixels [only].
I think PNG has options to filter with the previous row, the previous
column, or both. I assume that if you choose to filter with the previous
column only, you don't need to store the previous row of pixels in memory
at all. (I don't know if libpng optimizes in this way, but at least in
theory I think it could be possible to implement png compression like that.)
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> libpng only needs the current and previous row to run the pixel filter.
>> I have no idea what the DEFLATE compressor needs.
>
> Compression algorithms almost invariably use a buffer for the data, and
> this buffer is often surprisingly small (like some tens of kilobytes).
> (This means that most compression algorithms will *not* find and compress
> repetitions which are gigabytes apart from each other. This is for the
> simple reason that the compression routine has to use only a limited amount
> of memory and be relatively fast.)
>
> Some modern algorithms might use larger buffers (like some megabytes),
> but they nevertheless don't achieve enormously better compression ratios
> (for some reason there seems to be some kind of cutoff point after which
> enlarging the compression buffer does not improve the compression of average
> data significantly, but slows down the compression more than it's worth).
DEFLATE is ancient. I doubt it needs a lot of space. Most particularly,
the space required by DEFALTE is *unrelated* to the dimensions of the
PNG image. The PNG spec says the pixel data is optionally "filtered",
and then fed to DEFLATE as raw data, which DEFLATE then compresses in
the usual way.
> The problem with compressing exabytes of data is not the compression
> algorithm itself, but the file format into which the compressed data is
> stored.
>> I note that to store (2^31 - 1) x (2^31 - 1) pixels, where each pixel
>> requires exactly 4 bytes, requires about 18 exabytes of storage. (!!)
>
> Depends on what those pixels contain, and how smart your compression
> algorithm is.
I was thinking more of the amount of RAM required if you were to store
the entire thing uncompressed. I would expect any half-decent
compression algorithm to improve on this significantly.
> I don't think many file systems even support files which are exabytes
> large.
I agree.
(Maybe Google BigFile or whatever the hell it's called?)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Thus it's perfectly possible for POV-Ray to render a 2^31 x 2^31 pixels
> large image even on a system which limits memory addresses to be smaller
> than that.
>
> (Of course you won't be able to *save* that image anywhere because you
> would encounter limitations in the file system. But that doesn't stop
> POV-Ray from *rendering* the image.)
No - geological time stops you from rendering it. ;-)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> I think PNG has options to filter with the previous row, the previous
> column, or both.
Yes.
Each row can be filtered in one of these ways (or not filtered at all).
The choice is independent per row.
> I assume that if you choose to filter with the previous
> column only, you don't need to store the previous row of pixels in memory
> at all.
Yes.
> (I don't know if libpng optimizes in this way, but at least in
> theory I think it could be possible to implement png compression like that.)
Indeed - I have no idea if libpng has an option to control this...
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> I don't think many file systems even support files which are exabytes
> large.
There are a number that support 2^64 clusters, I think. People are beginning
to realize that if you actually need more than 2^64 clusters, you're not
going to want to copy that all over to a new file system when you run out of
space. :-)
Both NTFS and VFS in theory support up to 8 exabytes on a partition.
--
Darren New, San Diego CA, USA (PST)
I ordered stamps from Zazzle that read "Place Stamp Here".
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
