C++11 Boost, how to base64 decode back to integer presentation

Question

Based on this code here, I was able to write some code that converts a vector of integers into a base64 encoded version. And I can confirm it is the right output by comparing with a separate Java implementation.

#include <iostream>
#include <iomanip>
#include <cstdlib>
#include <fstream> 
#include <iostream>
#include <cstdint>
#include <typeinfo>

#include <boost/program_options.hpp>
#include <boost/filesystem.hpp>
#include <boost/archive/iterators/base64_from_binary.hpp>
#include <boost/archive/iterators/binary_from_base64.hpp>
#include <boost/archive/iterators/transform_width.hpp>
#include <boost/archive/iterators/ostream_iterator.hpp>
#include <boost/archive/iterators/remove_whitespace.hpp>

///....

using namespace std;
namespace po = boost::program_options;
namespace fs = boost::filesystem; 
namespace bi = boost::archive::iterators;

    std::stringstream os;
    typedef 
        bi::base64_from_binary<    // convert binary values to base64 characters
            bi::transform_width<   // retrieve 6 bit integers from a sequence of 32 bit ints
                vector<int32_t>::const_iterator,
                6,
                32
            >
        > 
        base64_text; 

    copy(
         base64_text(di.cbegin()),
         base64_text(di.cend()),
         ostream_iterator<char>(os)
         );

    cout << os.str() << "\n";

Now I'm trying to write the code to decode this back into a vector of integers, but this is proving much more difficult. I tried to convert the given examples to my use case (see below), but I just get an unhelpful segfault on the copy call. Frustratingly, everything I find is assuming string input/output to encode/decode. Any help is appreciated.

typedef 
  bi::transform_width<
    bi::binary_from_base64<bi::remove_whitespace<string::const_iterator>>,
      32, 6
  > 
  base64_dec; 

  vector<int32_t> decoded_ints;
  copy(
       base64_dec(base64ints.cbegin()),
       base64_dec(base64ints.cend()),
       decoded_ints.begin()
       );

Always provide complete, minimal, working (not working in your case) example — kreuzerkrieg, Jun 07 '17 at 06:18

kreuzerkrieg · Answer 1 · 2017-06-07T06:37:23.270

You are trying to copy into empty vector, copy does not take care of inserting elements, but only copying. You have to preallocate the destination vector using resize() or just use back_inserter like this

vector<int32_t> decoded_ints;
  copy(
       base64_dec(base64ints.cbegin()),
       base64_dec(base64ints.cend()),
       std::back_inserter(decoded_ints)
       );

EDIT001: Since there is no better answer is given and despite minimal working example wasnt introduced and I have no idea why it gets segfault I will expand my answer.
Above code should work, as well as initializing your int vector when declaring it - something like vector<int32_t> decoded_ints(base64_dec(base64ints.cbegin()),base64_dec(base64ints.cend())) However, your whole code doesnt work, didnt check if it encodes the data correctly, but it definitely doesnt produce the right result when decoding. Looks like transform_width doesnt do the trick with something wider than 8 bits or there is a padding added to the encoded data which should be taken into account when decoding. In any case you could work around all these by pushing your int vector into string (or vector of int8_t's). Something like this.

#include <boost/archive/iterators/base64_from_binary.hpp>
#include <boost/archive/iterators/binary_from_base64.hpp>
#include <boost/archive/iterators/transform_width.hpp>
#include <vector>
#include <iostream>
#include <memory.h>

int main()
{
    using namespace std;
    namespace bi = boost::archive::iterators;

    using IntVec = vector<int32_t>;
    IntVec di(1024, 0);

    int32_t n = 0;
    std::generate(di.begin(), di.end(), [&n]() { return n++; });

    string input(reinterpret_cast<string::pointer>(di.data()),
                 reinterpret_cast<string::pointer>(di.data()) + (di.size() * sizeof(IntVec::value_type)));
    typedef bi::base64_from_binary<bi::transform_width<string::const_iterator, 6, 8>> base64_text;
    typedef bi::transform_width<bi::binary_from_base64<string::const_iterator>, 8, 6> base64_dec;

    std::string base64ints(base64_text(input.cbegin()), base64_text(input.cend()));
    string decoded(base64_dec(base64ints.begin()), base64_dec(base64ints.end()));
    IntVec decoded_ints(reinterpret_cast<IntVec::const_pointer>(decoded.data()),
                        reinterpret_cast<IntVec::const_pointer>(decoded.data()) +
                            (decoded.size() / sizeof(IntVec::value_type)));

    if (decoded_ints.size() == di.size())
        cout << "Size matches" << std::endl;
    else
        cout << "Size does not match" << std::endl;

    if (memcmp(decoded_ints.data(), di.data(), di.size() * sizeof(int32_t)) == 0)
        cout << "Data matches" << std::endl;
    else
        cout << "Data does not match" << std::endl;
}

Live on CoLiRu
In addition I think it is a good idea to contact Robert Ramey (boost serialization maintainer) on Boost Mailing List

This does avoid the segfault that was happening before, but does not produce the correct output. I.e., the ints returned are not the same as the original ints that were decoded. This is easy for me to check, as in my use case all the ints are in a sorted order. — Raff.Edward, Jun 07 '17 at 00:07
Using a `back_inserter` can cause multiple reallocations unless you know the size you'll need in advance and call `vector.reserve()`. — Mark Storer, Apr 28 '20 at 16:34

C++11 Boost, how to base64 decode back to integer presentation

1 Answers1