![](/i/fill.gif) |
![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
I'm attempting to compile some auto-generated code, and am running into
problems with turning on optimization flags due to the large side of the
files generated (.cpp file is currently 25MB, and could be as large as
200MB later). There is no natural way to break this code into smaller
pieces and such large auto-generated files really are necessary, so I'm
stuck with having to compile some really huge C++ files.
I'm using g++ for the compilation, and it seems that if I turn on
optimization flags (even -O1) it will quickly eat through all 16 GB of
ram when I attempt to compile it.
Since I'd like to have the resulting executable run as quickly as
possible I am wonder if anyone has advice on either or all of the
following options:
1) Is there any way to specify for g++ to only to use optimizations
which do not require a significant amount of storage to compute?
2) Is there another compiler that works under linux (fedora 9) that
might handle this better? I should be able to find a version of the
Intel compiler to use if someone has reason to think it'll perform better.
3) Can I change the format of the source code so that using g++
optimizations isn't as necessary? Most of the execution time is spent
in my auto-generated code, which looks something like this:
...
double t75 = t73 - t74;
double t76 = t66 - t67;
double t77 = t74 + t73;
double t78 = t55 * t51;
double t79 = t78 + t58;
double t80 = t64 * -0.2;
....
Only for hundreds of thousands of lines with the occasional division,
sqrt, trigonometric function, etc thrown in. Since I write the code
generation functions, I can do some optimizations at the source-code level.
For example of such an optimization, I could implement a variable
allocation optimizer that would, for instance, determine that t74 isn't
used after the third line above, and then re-use that stack space to
store the value in t77 instead of allocating a new double for it.
The problem is that I don't know what (if any) of this sort of thing g++
does already if I don't specify any optimization flags, or what
source-level optimizations are likely to make the biggest improvement.
Any suggestions?
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Kevin Wampler <wampler+pov### [at] u washington edu> wrote:
> I'm attempting to compile some auto-generated code, and am running into
> problems with turning on optimization flags due to the large side of the
> files generated (.cpp file is currently 25MB, and could be as large as
> 200MB later). There is no natural way to break this code into smaller
> pieces and such large auto-generated files really are necessary, so I'm
> stuck with having to compile some really huge C++ files.
That's rather large. Are you sure it's the only way?
> 1) Is there any way to specify for g++ to only to use optimizations
> which do not require a significant amount of storage to compute?
If you run "info gcc" and the choose Invoking GCC > Optimize Options
you will get a detailed documentation of all the optimization flags gcc
supports. You can try turning then on one by one and see which ones
require significant amounts of memory and which ones don't.
> 2) Is there another compiler that works under linux (fedora 9) that
> might handle this better? I should be able to find a version of the
> Intel compiler to use if someone has reason to think it'll perform better.
I have no idea if it will perform better, but it doesn't hurt to try.
> 3) Can I change the format of the source code so that using g++
> optimizations isn't as necessary? Most of the execution time is spent
> in my auto-generated code, which looks something like this:
Are you sure you can't precalculate things in your code generator?
--
- Warp
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Warp wrote:
> Kevin Wampler <wampler+pov### [at] u washington edu> wrote:
>> I'm attempting to compile some auto-generated code, and am running into
>> problems with turning on optimization flags due to the large side of the
>> files generated (.cpp file is currently 25MB, and could be as large as
>> 200MB later). There is no natural way to break this code into smaller
>> pieces and such large auto-generated files really are necessary, so I'm
>> stuck with having to compile some really huge C++ files.
>
> That's rather large. Are you sure it's the only way?
It may be possible to re-formulate my problem to generate less code, but
unfortunately there's no way to do it that's particularly easy, or
that I'm entirely sure will work.
The auto-generated code itself is necessary because I do some
computations which involve complete knowledge and manipulation of the
function-composition graph which I eventually write to C++. It's
technically possible to avoid this, but it would be at a large cost in
code complexity and would probably result in significantly slower code
(defeating the purpose of such a change).
I can compile things perfectly well if I don't specify any optimization
flags, so it's not a disaster if I can't get g++ to cooperate -- it will
make make things somewhat less convenient.
>> 1) Is there any way to specify for g++ to only to use optimizations
>> which do not require a significant amount of storage to compute?
>
> If you run "info gcc" and the choose Invoking GCC > Optimize Options
> you will get a detailed documentation of all the optimization flags gcc
> supports. You can try turning then on one by one and see which ones
> require significant amounts of memory and which ones don't.
Ahhh, I had tried to google for that information, but with little
success. That is extremely helpful, many thanks!
>> 2) Is there another compiler that works under linux (fedora 9) that
>> might handle this better? I should be able to find a version of the
>> Intel compiler to use if someone has reason to think it'll perform better.
>
> I have no idea if it will perform better, but it doesn't hurt to try.
Fair enough. The main reason I asked was because it'll probably take
some effort to track down a version I can use (since it's not free), but
you're probably right that it's a relatively easy thing to try.
>> 3) Can I change the format of the source code so that using g++
>> optimizations isn't as necessary? Most of the execution time is spent
>> in my auto-generated code, which looks something like this:
>
> Are you sure you can't precalculate things in your code generator?
To what degree this is possible I already do, including removing any
computations which don't influence the result and precomputing anything
which only involves constants.
It's possible I could do some algebraic optimizations (factoring out
common multipliers, etc) to reduce the code size, but that's starting to
become a rather more process difficult than I'd like.
I'll start off with setting the g++ optimization flags from the info
file you pointed out, since that seems easiest.
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Kevin Wampler wrote:
> I'm attempting to compile some auto-generated code, and am running into
> problems with turning on optimization flags due to the large side of the
> files generated (.cpp file is currently 25MB, and could be as large as
> 200MB later). There is no natural way to break this code into smaller
> pieces and such large auto-generated files really are necessary, so I'm
> stuck with having to compile some really huge C++ files.
So the source code for each *function* is megabytes in size? That's the
only reason you couldn't divvy up the sources into separate files.
Regards,
John
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
John VanSickle wrote:
> So the source code for each *function* is megabytes in size? That's the
> only reason you couldn't divvy up the sources into separate files.
Yes, that is correct. Although strictly speaking there are several
functions in the file, the lion's share (maybe 60%) of the size is taken
up by a single function. So breaking the file into multiple files would
help a little, but not enough to really address the problem.
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |