1

Right, having looked up this issue, I believe that it is caused because I have in my class definition wchar_t downloadedText[400000]; I have read solutions about how I should deal with this by using the new operator to assign the space, ie: wchar_t *downloadedText; downloadedText = new wchar_t[400000];

However I need to write instances of the class to a file, and assigning the variable like above appears to use pointers to point to data stored in a way that does not get written to my file. It is this same reason why I cannot use std::vector.

I have read that another option I may have is that I can increase the size of the 'stack'. I use VS2010 as my IDE, and I located in my project properties > Linker > System 'Stack Commit Size', 'Stack Reserve Size', 'Heap Commit Size' and 'Heap Reserve Size' fields, but I am unsure if this is how I can deal with my issue, and if it is, what to correctly set the appropriate fields to.

Jonathan Wakely
  • 166,810
  • 27
  • 341
  • 521
Interminable
  • 1,338
  • 3
  • 21
  • 52
  • I'm not sure what you mean by pointers work differently. A pointer array will give you the elements just like a normal array would, and a vector does as well. – chris May 03 '12 at 02:08
  • Nevertheless, it doesn't write the elements to the file in the same way that it does via declaring the array as `wchar_t myArray[200]`, I believe it is just writing pointers, which is obviously not what I want. – Interminable May 03 '12 at 02:09
  • 1
    How are you accessing the element to write in each case? – chris May 03 '12 at 02:13
  • I write the entire object, of which the array is a member. I use the write() function and write in binary mode. – Interminable May 03 '12 at 02:15
  • 4
    Could you update your question with a code sample of the writing process for static arrays vs. pointer/vector? – chris May 03 '12 at 02:16
  • 2
    `fileOutput.write(reinterpret_cast (&myObjects[i]), sizeof(MyClass));` The process of writing to the file does not change when changing the members of the class (although if I wanted to read object data from the file after adding/removing member variables I would be unable to do so). I obviously cannot use `std::vector` in the same way that I cannot use `std::string` because they do not directly contain the data I want to write, instead they are more complex and contain pointers to the actual data I want, etc. – Interminable May 03 '12 at 02:21
  • 1
    how do you write when you have pointer? – deebee May 03 '12 at 02:22
  • You might want to implement a `char *` conversion operator in your class instead of using `reinterpret_cast`. – chris May 03 '12 at 02:22
  • 2
    Why not to write a proper write/serialization function, or simply use some serialization library? http://stackoverflow.com/questions/234724/how-to-serialize-in-c – aaronqli May 03 '12 at 04:05
  • I'm not familiar with the concept of serialization, what exactly is it and how can it help with this particular problem? – Interminable May 03 '12 at 10:52
  • Serialization is "writing objects to disks, network sockets, strings, or similar things." To be honest, I think Perl's right to call it "stringification," but the Java term caught on instead. It helps to use a serialization library because your question is, essentially, "how do I serialize this object?" @Polymorpher's answer is "have a library handle the details for you." – Max Lybbert May 03 '12 at 17:55
  • Do you know of ANYWHERE with a decent example of serializing an `std::string` or `std::wstring` (or an object with one of those datatypes as a member) and then writing it to a file in binary mode? I'm having difficulty finding one. – Interminable May 03 '12 at 19:28
  • Serialization doesn't mean "use `std::string`/`std::wstring`." It means "turn the object into a sequence of bytes, and write those bytes where you want, like a file in binary mode." I'm not sure how you define "binary mode." I've been assuming you meant "don't change `\n` characters.", e.g., `fopen`'s use of the term ( http://www.cplusplus.com/reference/clibrary/cstdio/fopen/ ). However, if you mean "binary data," you may be interested in Google Protocol Buffers which **does** include `std::strings` for string-like data **and** serializes to binary data ( http://code.google.com/p/protobuf/ ). – Max Lybbert May 03 '12 at 21:55
  • You may also be interested in Facebook's Thrift (now an Apache project: http://thrift.apache.org/ ), Boost Serialization ( http://www.boost.org/doc/libs/1_49_0/libs/serialization/doc/index.html ), BSON ( http://bsonspec.org/#/implementation ) and others that do the same thing. – Max Lybbert May 07 '12 at 14:26

4 Answers4

3

If you must do it this way... You could just write the array explicitly, after writing out the object. eg.

write((char*)&myObjects[i]), sizeof(MyClass));
write((char*)downloadedText, sizeof(downloadedText[0]) * 400000);

And to read it back in:

read((char*)&myObjects[i]), sizeof(MyClass));
downloadedText = new wchar_t[400000];
read((char*)downloadedText, sizeof(downloadedText[0]) * 400000);

However, this is very fragile and prone to errors. Overwriting objects in memory as done by the read is at best bad form except perhaps if you're using a struct created explicitly for that purpose which would typically contain only PODs. As a minimum, note that you have to set the downloadedText member field after the read writes over it.

dschultz
  • 485
  • 3
  • 10
  • My class does only contain PODs. In addition I would prefer to keep downloadedText as a member variable of said class. – Interminable May 03 '12 at 10:36
2

You can allocate the whole object with new operator in system heap but not on stack. In this case you can write this object to file in the same manner and have no troubles with stack overflow.

inkooboo
  • 2,904
  • 1
  • 19
  • 24
  • I thought of this too, however it does not like the statement `myObjects.push_back(new MyClass(0,0,TEXT("text\0")));` It gives me an error along the lines of `error C2664: 'void std::vector<_Ty>::push_back(_Ty &&)' : cannot convert parameter 1 from 'MyClass*' to 'MyClass &&' 1> with 1> [ 1> _Ty=MyClass 1> ] 1> Reason: cannot convert from 'MyClass *' to 'MyClass' 1> No constructor could take the source type, or constructor overload resolution was ambiguous` – Interminable May 03 '12 at 10:13
  • That error message means that `myObjects` is a `std::vector`, which you'll need to change to `std::vector`. – Max Lybbert May 04 '12 at 20:30
  • Thinking more about this, if you put your objects into a `std::vector`, then they're already allocated on the heap and off the stack. – Max Lybbert May 05 '12 at 07:04
1

Yes, you can increase the stack size in Visual Studio with the linker option /STACK. This linker option is also editable with the project properties Stack Reserve Size and Stack Commit Size. It is enough to set the reserve size, the commit size is optional. Nevertheless you should also be able to use std::vector.

Christian Ammer
  • 7,464
  • 6
  • 51
  • 108
  • I did initially try this without any success. However I then set it to a more insane value of 1024000000 (1GB?) as I didn't realise it appeared to be requesting a figure in bytes. I cannot write `std::vector` class members to my file because it is not POD. Unfortunately I can only store around 120 entries before getting some sort of bad alloc error. In addition it starts using a fair amount of memory, which is a little annoying as I'd prefer it to not use the memory for all 400000 characters unless it has to. ie if I'm not storing anything it should hopefully be taking little memory. – Interminable May 03 '12 at 10:26
0

It sounds like you're fine with your current serialization strategy (make the object POD, and write it as a POD value). In that case, your question really is "how can a keep these objects from taking up too much space on the stack?" The answer: don't put them on the stack.

Allocate them with new. Or, preferably, wrap them in smart pointers of some kind.

You have string-like data. It so happens that C++ offers a string class. In fact, since you're talking about wchar_t characters, you want to look at std::wstring (in the <string> header).

Max Lybbert
  • 19,717
  • 4
  • 46
  • 69
  • I cannot use `std::string` as a class member because it is not POD. I will have issues trying to write it to the file in binary mode. – Interminable May 03 '12 at 10:04
  • You'll only have issues if you try to write the containing `struct` or `class` to the file as if it's POD. If you write each element separately, it's trivial to use `std::wstring`. – Max Lybbert May 03 '12 at 17:51
  • I can write instances of my object just fine provided it's got only POD types like `int`, `char`, etc, and arrays declared with their exact length in the class, ie char `myArray[200]` Trying to write ONLY an `std::string` or `std::wstring` does not work because they do not themselves contain the data I want, they contain pointers to it, etc. – Interminable May 03 '12 at 19:27