C++ class with template methods having many instances consuming huge amount of memory on compilation

Question

I have a class with template methods for which I instantiate many other classes, like more than a hundred. The problem is that the compilation of the template class consumes a huge amount of memory, like 3GB. I assume this occurs because of many template instances. Just for detailing, the instantiated classes are Qt and QxOrm objects.

Has someone else had this problem too?

Some suggestions about how could I reduce the memory consumption?

Here are some parts of the code:

//The class with template methods

class SGSRequestHandler : public HttpRequestHandler
{
    public:
        SGSRequestHandler(QObject* parent = 0);
        virtual ~SGSRequestHandler();

        template<class T>
        ResponseInfo processDatabase(ODataRequestInfo<T>& odata, qx::QxSession& session) {...}

        template<class T>
        ResponseInfo httpGet(ODataRequestInfo<T> &odata, qx::QxSession& session) {...}
    ...
   }

Here is a example of what I do with the template class:

else if (className == "class1")
     rInfo = process<Class1>(request, session);
else if (className == "class2")
     rInfo = process<Class2>(request, session);
...
else if (className == "class100")
    rInfo = process<Class100>(request, session);

The class after the comment `//The template class` isn't a class template. — , Jul 14 '17 at 13:17
Ok, it is a class with template methods, like I wrote above. I'm editting the comment. — vanderdill, Jul 14 '17 at 13:21
You consume a lot of memory because the compiler has to generate many different versions of your functions. If memory during compilation is a constraint, look into type erasure techniques to reduce the number of template instantiations. — AndyG, Jul 14 '17 at 13:22
Right, AndyG, but type erasure isn't an option in this case. I'm looking for some solution like splitting the file or parallelizing the compilation of the class. — vanderdill, Jul 14 '17 at 13:25
`or parallelizing the compilation of the class` sounds like something that would multiply the memory use; which is the opposite of a solution to high memory consumption. — eerorika, Jul 14 '17 at 13:35
The class after the comment `//The class with template methods` isn't using [template method](https://en.wikipedia.org/wiki/Template_method_pattern). Anyway, using `if` statement to switch between template instances usually indicates design problems. — , Jul 14 '17 at 13:39
I'm thinking if I could design my project with polimorfism, I'll try and let you know. But, independently of this, I would to know if there is a solution for the stated problem. — vanderdill, Jul 14 '17 at 13:46
What uses 3GB? THe compiler during compilation process, or is it the final binary size, or is it the memory used by the final binary? — Mooing Duck, Jul 27 '23 at 19:21

score 2 · Answer 1 · answered Jul 18 '17 at 15:52

2

Looking at your solution suggests that really templates are the wrong answer to your problem.

Maybe you should consider making all the processing class instances register with the SGSHandler class, so that that can just do

processors_[name].process(request, session);

There is obviously a runtime cost to this but it's a lot easier to maintain

answered Jul 18 '17 at 15:52

Tom Tanner

9,244
3
33
61

1

It's less runtime cost than he's already paying, since he can eliminate the 100+ string comparisons – Mooing Duck Jul 27 '23 at 19:20

score 0 · Answer 2 · answered Jul 14 '17 at 13:32

0

The easiest solution may very well be to use extern template and move the 100 instantiations to multiple .cpp files.

answered Jul 14 '17 at 13:32

MSalters

173,980
10
155
350

This is a clever solution that I didn't know. But unfortunatly in my case I only instantiate the functions on time per class in the same file. – vanderdill Jul 14 '17 at 13:40

vanderdill · Accepted Answer · 2017-07-19T05:15:51.407

I've came with a solution. Not even close from elegant, but resolved my problem.

I simply split the class that instantiate all the template in three other classes. So, the memory consumption on compilation is a third part for each of the classes.

Remembering that I need to work with templates because I'm using QxOrm library, otherelse I could do this in a much simplier manner with polimorfism.

//processorr.h

class ProcessorR {
    void process(const QString& className) {
        if(className == "rClass1")
            rInfo = process<rClass1>(request, session);
        else if(className == "rClass2")
            rInfo = process<rClass2>(request, session);
        /// And so on
    }
}

//processort.h

class ProcessorT {

    void process(const QString& className) {
        if(className == "tClass1")
           rInfo = process<tClass1>(request, session);
        else if(className == "tClass2")
           rInfo = process<tClass1>(request, session);
        /// And so on
    }
}

//And then in usage:

if(className.startsWith("t"))
{
    ProcessorT processor;
    processor.process(className);
}
else if(className.startsWith("r"))
{
    ProcessorR processor;
    processor.process(className);
}
else if(className.startsWith("u"))
{
    ProcessorU processor;
    processor.process(className);
}

C++ class with template methods having many instances consuming huge amount of memory on compilation

3 Answers3