As a result of some discussion on the IMP
technical discussion list, there has been
some preliminary investigation into
supporting TAR archives from POV.

The advantages are:

Minimal POV installation - povray.exe, includes.tar
Modularisation of large pov scenes.
Modularisation of libraries.
Download-and-render without extraction.

Technical Issues

Support compression?
Parsers that use fseek?

I've attached more detailed discussion.
Your comments or interest are invited.

------------------------------------------------------

Date: Tue, 02 Feb 1999 15:15:40 +1100
From: Nigel Stewart <nig### [at] eisanetau>
Subject: Re: [TECH] Proposal - PVA Fileformat Specification

>>> The PVA (PovRay Archive) fileformat is proposed
>>> as a means of modularisation of POV scene
>>> files.

TarFs, ExTar
----------------------------------------------------

TarFs is a C++ class representing a tar archive.

FILE    *TarFs::file(char *filename,const char *mode = "rt")

  Create a C FILE pointer to read a file from the
  tar archive.

istream *TarFs::istream(char *filename,ios_base::openmode mode = ios::in)

  Create a C++ input stream object to read a file from
  tar archive.

----------------------------------------------------

Extar is a command-line program to extract a file
within tar archive to standard output.

For Example:

C:\Data\Nigel\devel\tarlib>tar cvf test.tar tmp
tmp/
tmp/test.txt
tmp/test2.txt

C:\Data\Nigel\devel\tarlib>extar test.tar
tmp/test.txt
tmp/test2.txt

C:\Data\Nigel\devel\tarlib>extar test.tar tmp/test.txt
Hello World


C:\Data\Nigel\devel\tarlib>

----------------------------------------------------

Applications -

  PovRay support for concept of library.
  IMP datafile encapsulation.
  Game datafile supporting read-only random access.
  Email datafile supporting arbitrary append.
  Virtual read-only filesystem for Linux.
  Tar archive browsing for WWW.  (CGI ExTar)

-------------------------------------------------------------------------

Date: Thu, 04 Feb 1999 01:17:49 +1100
From: Nigel Stewart <nig### [at] eisanetau>
Subject: [TECH] POV Support for TAR Filesystem



I had a quick-hack attempt at supporting tar
files from POV-ray.  It works nicely, except
for TTF fonts, which are parsed using fseek. (**)
POV, INC and PNG files within tar files have
worked without a hitch.

What does all this mean?

Well, it means that a 'minimalist' POV install
is one .exe file, plus one .tar file with all
the include files.  There is no dependency on
environment variables, or registry settings
to find libraries - which makes it easy to
install, zero configuration, and no
interaction with other POV installations.
Also, a renderfarmer only needs to download
ONE tar archive into the directory to
start rendering.  (Plus a command line,
which can be in the TAR as well...)

So, my conclusion is that having a custom
IMP POV compilation is feasible, has real
advantages, (we can use any patches that we
like) and is likely to have minimal impact
on render farmers.  It is also very portable.

My 0.02 for today.

(Oh, and hacking around in the POV source
 is pretty educational/interesting too!)

(**)  TarLib provides a file pointer
      offset to the correct location in
      the TAR file, but using fseek
      can depend on offset zero being
      at the start of the file, rather
      than start of Tar file.

---------------------------------
|         | scene.pov |        ...
----------------------------------
           ^
            \ beginning of "scene.pov"
 ^
  \ fseek is relative to tar file

--------------------------------------------------------------------

Date: Thu, 04 Feb 1999 20:54:29 +1100
From: Nigel Stewart <nig### [at] eisanetau>
Subject: Re: [TECH] POV Support for TAR Filesystem

TAR Files
---------

Generic.  Tar has a long history, is well supported,
          and well understood.  The actual format of
          the headers are weird, but proven.

Simple.   Tar is a "chunked" format, files are stored
          contiguously and uncompressed.

Compression
-----------

Directly supporting compressed archives is undesirable
in terms of simplicity and efficency.  Fancy languages
like Java or C++ allow you filter streams, so that
decompression is hidden from the parser.  I do not
know of a clean and efficient way of doing this in
C.  (Either parse from memory buffer, or extract file
to temporary directory on the fly).

Also, it's worth considering that POV already supports
compressed files such as PNG.

Also, do we want to spend CPU time on decompression
for every frame of animation?

Also, would it more appropriate to support POV parsing
of .pov.gz (".pgz") or .inc.gz (".igz"), independent
of tar file support?

In summary - We can do it, but do we WANT to do it?

Portability of JAR
-------------------

JAR is portable in the sense that it works on any
Java supported platform.  TAR is portable in the
sense that things like WINZIP support it.  My
impression is that the average raytracer is going
to be equally unfamiliar with TAR and JAR.

If compression is appropriate to the application,
let's use JAR.  However, I do not think there
will be a good return on the investment of time
and energy.

Perhaps there is someone from the broader
POV community who would do the work on
zlib-based compression.  Would anyone
object to making a report to pov.programming?

Post a reply to this message