C++ string too big, trailing characters truncated

Question

I am making a program that when started, will write a predefined string into a file. The file is about 5 mb in size so filling the string with 5 mb of data in hex is a big variable. When I try to compile it, I get an error saying the string is too big. Is 5mb really THAT big? I split the string into 4 sections but each section is still too big. :/ what can I quickly and easily do to fix this situation.

Note: I consider myself a beginner programmer so try not to go too far over my head :P

Example of how I write the string to a file:

string file_hex("huge_a**_string_goes_here_or_in_separate_cpp");

ofstream file_out;
file_out.open("tools\\c.exe", ios::binary | ios::trunc);
string res;
res.reserve(file_hex.size() / 2);
for (int i = 0; i < file_hex.size(); i += 2)
{
    std::istringstream iss(file_hex.substr(i, 2));
    int temp;
    iss >> std::hex >> temp;
    res += static_cast<char>(temp);
}

file_out << res;
file_out.close();

Could you give some example code (obviously not the whole string)? — chris, Jun 01 '13 at 02:04
Not part of the string ,could you provide the part of code that uses the string? — Yu Hao, Jun 01 '13 at 03:02
I edited op to show how i use the string "file_hex" to write it to a new .exe file. — mrg95, Jun 01 '13 at 03:08

Captain Obvlious · Accepted Answer · 2013-06-01T03:55:22.233

18

The Standard does not specify a maximum limit of string literals but does suggest a minimum size. From Annex B - Implementation quantities

The limits may constrain quantities that include those described below or others. The bracketed number following each quantity is recommended as the minimum for that quantity. However, these quantities are only guidelines and do not determine compliance.

Characters in a string literal (after concatenation) [65 536].

However the same section in the standard also states the following

Because computers are finite, C++ implementations are inevitably limited in the size of the programs they can successfully process. Every implementation shall document those limitations where known. This documentation may cite fixed limits where they exist, say how to compute variable limits as a function of available resources, or say that fixed limits do not exist or are unknown.

It's safe to assume that a 5MB string literal is going to exceed maximum limits imposed by the compiler. You should refer to the documentation of your tool chain to determine what limits are imposed for string literals.

If you are using the Visual C++ compiler the maximum size of a string literal is 16,384 bytes. From the MSDN documentation

The maximum length of a string literal is 16,384 (16K) bytes. This limit applies to strings of type char[] and wchar_t[]. If a string literal consists of parts enclosed in double quotation marks, the preprocessor concatenates the parts into a single string, and for each line concatenated, it adds an extra byte to the total number of bytes.

edited Jun 01 '13 at 03:55

answered Jun 01 '13 at 03:36

Captain Obvlious

19,754
5
44
74

Interesting. Thanks for the info! :) Can you think of any way around this or perhaps a different way to write a .exe file? I have no idea how to use resources :/ – mrg95 Jun 01 '13 at 03:58
Adding the string to the resource file is certainly an option and would be my first recommendation. Do **not** add it to the string table though, instead add it as a custom binary resource. [MSDN](http://msdn.microsoft.com/en-us/library/zabda143(v=vs.71).aspx) has a good starting point for working with resources. – Captain Obvlious Jun 01 '13 at 04:02
@user2356609 There is also an [answer here on SO](http://stackoverflow.com/a/2740236/845568) that describes the basic steps to add the binary resource to your application. – Captain Obvlious Jun 01 '13 at 04:04
Thanks you so much for this :) Ive got the exe imported as a resource, now time to find out how to use it lol – mrg95 Jun 01 '13 at 04:11
@user2356609 You're welcome. Remember to accept the answer you find most useful by clicking on the check mark next to it ;) – Captain Obvlious Jun 01 '13 at 04:20
adding to this answer the easiest way to work around Visual C++ limit would be Noteworthy's [answer](https://stackoverflow.com/a/21284562/6162953) while the easiest way to work around the compiler limit would be to assign 65K (or 16K in case of VC++) character chunks in different variables then adding them up and releasing the temp variable's memory – Omar Abdul'Azeez Oct 05 '21 at 14:25

score 11 · Answer 2 · edited Jan 22 '14 at 14:01

11

instead of this :

char* metaDescriptor = "very long string";

it should be :

char* metaDescriptor = "very long"
                        "string";

edited Jan 22 '14 at 14:01

Alexis Pigeon

7,423
11
39
44

answered Jan 22 '14 at 13:42

Noteworthy

349
4
5

1

@Pat Actually it works, even with old compiler as VS 2005 – ggurov Jun 25 '19 at 08:06
I only recently learned this multi-line string trick. Before I only knew that you can escape newline with `\\`, but this still triggers very long string error. – ScienceDiscoverer Jul 27 '22 at 19:27

score 2 · Answer 3 · answered Jun 01 '13 at 04:23

If you're trying to just be quick and dirty, use C's level 3 IO to do it and use a static array to store the values.

#include <stdio.h>

static const unsigned char data[] = {
    1, 5, 255, 128, 50, 192,
    // all the values of your file here,
    // don't add a trailing zero, this is not a string.
};

int main(int argc, const char* argv[])
{
    FILE* fp = fopen("test.txt", "wb");
    if ( fp == NULL ) {
        printf("No dice.\n");
        return -1;
    }
    // write the entire array to the file without.
    fwrite(data, sizeof(data[0]), sizeof(data), fp);
    fclose(fp);
    return 0;
}

C++ string too big, trailing characters truncated

3 Answers3

Linked