I am new to multithreading. I made program which works fine when single threaded.but when i add another thread to run asio io service "with main thread running same io service" it gives me this error :
Exception thrown at (ntdll.dll) Access violation reading location
I use vs 2017 and i tried to use call stack to locate error but it happens at ntdll.dll code which i do not have its source.
in addition ,i tried to exclude parts of code but it appears that error happens when async function returns from io service.
the program is large so i did not put code.
i want to know if there is way using vs2017 to link the ntdll.dll error to the part of my source code which lead to this error.
call stack at error onlu shows:
ntdll.dll!__except_handler4()
ntdll.dll!ExecuteHandler2@20()
ntdll.dll!ExecuteHandler@20()
if files are required ,i will put their code.
thanks
UPDATE:
this function throws alot even though i launch it after locking mutex.
i think there is something wrong regarding memory usage.it gives me access violation .
std::string HTTPRequest::output_compressed_file_2(boost::shared_ptr<unsigned char> data_bin_buffer, size_t buffer_size, const char * file_name)
{
std::string file_name_path_string = file_name;
std::ofstream ofs2(file_name_path_string, std::ios::binary);
ofs2.write(reinterpret_cast<const char*>(data_bin_buffer.get()), buffer_size);
ofs2.close();
return file_name_path_string;
}
it makes error related to memory .when i inspect call stack and threads , i found that it is called in both threads.in one thread it is just called ,in the other thread it is already called and it calls inside function sputn ,xsputn,copy and memcpy where throw happens.
i think that calling it in the other thread before it finishes in the first thread leads to error.i am trying to find any shared resource but i can not se any shared resource.this function is called inside another function which is called inside third function which is ran inside io service and this third function is member of class instance HttpRequest .
there are two instances of HttpRequest.
i can not find the reason of memcpy error.
here is the code in which this function is called :
void HTTPRequest::Execute(boost::asio::yield_context yield_r, std::string request_name, boost::shared_ptr<std::map<std::string, boost::shared_ptr<HTTPResponse>>> mHTTPClient_Responses_Map_shared_pointer)
{
std::map<std::string, boost::shared_ptr<HTTPResponse>> & mHTTPClient_Responses_Map = boost::ref(*mHTTPClient_Responses_Map_shared_pointer).get() ;
ptime startFetch = second_clock::local_time();
//??5-17-2020 isolate multithreaded error
/*
boost::unique_lock<boost::mutex> cancel_lock(mCancelMutex);
if (mWasCancelled)
{
cancel_lock.unlock();
OnFinish(boost::system::error_code(boost::asio::error::operation_aborted));
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " has been cancelled by the user at start of HTTPRequest::Execute coroutine." << std::endl;
m_formatting_ostream.flush();
////allam2020 change UniqueSignalValue to url
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "c_E", request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
//p.set_value(request_name);
//return request_name;
//5-6-2020 i think i should return when cancelled
return;
}
cancel_lock.unlock();
*/
bool iterator_failed = true;
boost::system::error_code ec;
//5-7-2020
// Compose the request message.
mRequestBuf += "GET " + mUri + " HTTP/1.1\r\n";
// Add mandatory header.
mRequestBuf += "Host: " + mHost + "\r\n";
mRequestBuf += "\r\n";
for (auto iterator_resolve : *mRequestSharedPtrVecResolverIterator)
{
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << boost::this_thread::get_id() << "\t"<<"Request #" << this->GetId() << " for " << mUrl <<" trying to send request using " << iterator_resolve->endpoint().address().to_string() << std::endl<<"\n";
//5-8-2020 changing continue to endless loop except for 404
for (int mIrange2 =1; mIrange2 < ATTEMPTS; ++mIrange2)
{
++HTTPRequest::mIrange ;
resolver_iterator iterator_connect = boost::asio::async_connect(mSock, iterator_resolve, yield_r[ec]);////allam 2020 this gets us back to io_stream_run
if (ec.value() == boost::system::errc::errc_t::success)
{
////allam 2020
iterator_failed = false;
//??5-17-2020 isolate multithreaded error
boost::unique_lock<boost::mutex> cancel_lock(mCancelMutex);
if (mWasCancelled)
{
cancel_lock.unlock();
OnFinish(boost::system::error_code(boost::asio::error::operation_aborted));
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " has been cancelled by the user after returning from async_connect inside HTTPRequest::Execute using"<< iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "cancel_async_connect_Execute", iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "c_a_c_E", request_name,m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
//p.set_value(request_name);
//return request_name;
return;
}
cancel_lock.unlock();
// Send the request message.
SendRequest(yield_r);
}
else if (ec.value() != boost::system::errc::errc_t::success)
{
OnFinish(ec);
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) <<"Request #" << this->GetId() << " for " << mUrl <<" failed after trying " << mIrange2 << "times" << " to async_connect inside HTTPRequest::Execute " << std::endl;
continue;
}
if (mContinue_for==true)
{
mContinue_for = !(mContinue_for);
//5-5-2020 async timer
mAsyncTimer.expires_from_now(boost::asio::chrono::milliseconds(mIrange));//5-5-2020
mAsyncTimer.async_wait(yield_r);
//5-7-2020
if (mSendRequest == 0)
{
if (mReadStatusLine == 0)
{
if (mHttp_1_1 == 0)
{
if (mStatusCodeNot200 == 0)
{
if (mReadResponseHeaders == 0)
{
if (mReadResponseBody == 0)
{
////allam2020 4-4-2020 no error present and response is recieved in its mHTTPResponse SO DO NOTHING
}
else if (mReadResponseBody != 0)
{
mIrange2--;
}
}
else if (mReadResponseHeaders != 0)
{
mIrange2--;
}
}
else if (mStatusCodeNot200 != 0)
{
}
}
else if (mHttp_1_1 != 0)
{
mIrange2--;
}
}
else if (mReadStatusLine != 0)
{
mIrange2--;
}
}
else if (mSendRequest != 0)
{
mIrange2--;
}
continue;
}
// Response is correct.
//??5-17-2020 isolate multithreaded error
m_formatting_ostream << boost::this_thread::get_id() << "\t" << "Request #" << this->GetId() << " for " << mUrl << " Fetched " << mUrl << " completed in : " << (second_clock::local_time() - startFetch) << "with HTTP code :" << mResponsePtr->get_status_code() << "\t" << "and the code reasonPhrase is :" << HttpStatus::reasonPhrase(static_cast<int>(mResponsePtr->get_status_code())) << "with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "http_request_completed", HttpStatus::reasonPhrase(static_cast<int>(mResponsePtr->get_status_code())), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "req_comp", HttpStatus::reasonPhrase(static_cast<int>(mResponsePtr->get_status_code())), m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
if (mResponsePtr->get_response_buf().size() <= 0)
{
//??5-17-2020 isolate multithreaded error
m_formatting_ostream << boost::this_thread::get_id() << "\t" << "Request #" << this->GetId() << " for " << mUrl << " Fetched " << mUrl << " with Buffer for " << mUrl << " is empty " << "\n" << "with HTTP code :" << mResponsePtr->get_status_code() << "\n" << "and the code reasonPhrase is :" << HttpStatus::reasonPhrase(static_cast<int>(mResponsePtr->get_status_code())) << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "http_request_completed_empty", HttpStatus::reasonPhrase(static_cast<int>(mResponsePtr->get_status_code())), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "req_comp_empty", HttpStatus::reasonPhrase(static_cast<int>(mResponsePtr->get_status_code())), m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
//continue work on response
mHTTPRequest_response_name = "response_" + request_name;
mHTTPClient_Responses_Map[mHTTPRequest_response_name] = GetResponseSharedPtr();
//4-24-2020 here i will save to file
std::size_t pos = mUrl.find("h_ticks.bi5"); // position of "h_ticks.bi5" in str
std::string mHTTPRequest_file_name = mUrl.substr(pos - 2); // get from "h_ticks.bi5" to the end
std::string mHTTPRequest_hour_name = mUrl.substr(pos - 2,2); // get from "h_ticks.bi5" to the end
date mHTTPRequest_file_name_ptime_epoch_date =mHTTPRequest_HTTPClient_shared_pointer->m_HttpClient_Day_Full_DateGet() ;
ptime mHTTPRequest_file_name_ptime_epoch(mHTTPRequest_file_name_ptime_epoch_date, pt::hours(std::stoi(mHTTPRequest_hour_name)));
std::string compressed_file_path_string =output_compressed_file(mResponsePtr->get_response_buf(), mHTTPRequest_file_name);
path compressed_file_path{ compressed_file_path_string };
read_bi5_main(compressed_file_path, mHTTPRequest_file_name_ptime_epoch);
break;
}
//??5-17-2020 isolate multithreaded error
/*
//the following conditions test the result of send request
if (mSendRequest == 0)
{
if (mReadStatusLine == 0)
{
if (mHttp_1_1 == 0)
{
if (mStatusCodeNot200 == 0)
{
if (mReadResponseHeaders == 0)
{
if (mReadResponseBody == 0)
{
////allam2020 4-4-2020 no error present and response is recieved in its mHTTPResponse SO DO NOTHING
}
else if (mReadResponseBody != 0)
{
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " has failed completely after trying" << ATTEMPTS << "times" << " to async_read inside HTTPRequest::ReadResponseBody to get ResponseBody " << "with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_async_read_inside_HTTPRequest_ReadResponseBody", "requestFailed_ReadResponseBody_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_a_r_in_RRB", "reqFail_RRB_It_" + iterator_resolve->endpoint().address().to_string(),m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_ar_RRB", "_" + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
}
else if (mReadResponseHeaders != 0)
{
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " has failed completely after trying" << ATTEMPTS << "times" << " to async_read_until inside HTTPRequest::ReadResponseHeadersto get ResponseHeaders " << "with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_async_read_until_inside_HTTPRequest_ReadResponseHeaders", "requestFailed_ReadResponseHeaders_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_a_r_until_in_RRH", "reqFail_RRH_It_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_aru_RRH", "_" + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
}
else if (mStatusCodeNot200 != 0)
{
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " has failed completely after" << ATTEMPTS << "times" << " to async_read_until inside HTTPRequest::ReadStatusLine because of status_code not 200:" << http_errors::invalid_response << "the error code is :" << mStatusCode << "\n" << "and the error reasonPhrase is :" << HttpStatus::reasonPhrase(static_cast<int>(std::stoul(mStatusCode))) << "with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_StatusCodeNot200_inside_HTTPRequest_ReadStatusLine", "requestFailed_ReadStatusLine_StatusCodeNot200:StatusCode_is_" + mStatusCode + "with_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_Not200_in_RSL", "reqFail_RSL_Not200:St_is_" + mStatusCode + "with_It_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_N200_RSL", "_" + mStatusCode + "_" + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
}
else if (mHttp_1_1 != 0)
{
////4-2-2020
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " after trying " << ATTEMPTS << "times" << " to async_read_until inside HTTPRequest::ReadStatusLine because of bad not http/1.1 version response" << mHTTP_Version << "recieved with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_Http_1_1_inside_HTTPRequest_ReadStatusLine", "requestFailed_ReadStatusLine_Http_1_1_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_Http_1_1_in_RSL", "reqFail_RSL_Http_1_1_It_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_H11_RSL", "_" + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
}
else if (mReadStatusLine != 0)
{
////4-2-2020
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " failed completely after trying " << ATTEMPTS << "times" << " to async_read_until inside HTTPRequest::ReadStatusLine with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_async_read_until_inside_HTTPRequest_ReadStatusLine", "requestFailed_ReadStatusLine_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_a_r_u_in_RSL", "reqFail_RSL_It_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_aru_RSL", "_" + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_aru_in_RSL", "_" , m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
}
else if (mSendRequest != 0)
{
////4-2-2020
m_formatting_ostream << "Request #" << this->GetId() << " for " << mUrl << " failed after trying " << ATTEMPTS << "times" << " to async_write inside HTTPRequest::SendRequest with certain resolver iterator " << iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_async_write_inside_HTTPRequest_SendRequest", "requestFailed_SendRequest_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "fail_aw_in_SR", "_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_aw_SR", "_" + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
*/
if (iterator_failed == true)
{
//??5-17-2020 isolate multithreaded error
m_formatting_ostream << "Request failed for " << mUrl << " after trying " << ATTEMPTS << "times" << " to async_connect inside HTTPRequest::Execute with certain resolver iterator "<< iterator_resolve->endpoint().address().to_string() << std::endl;
m_formatting_ostream.flush();
////allam 2020 i might need to pass iterator resolve which has failed
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_async_connect_inside_HTTPRequest_Execute", "requestFailed_Iterator_" + iterator_resolve->endpoint().address().to_string(), m_formatting_ostream_string, mBOOST_LOGMutex);
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "f_ac_E", "_" + iterator_resolve->endpoint().address().to_string()+ request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
std::string string_replace = { mSock.remote_endpoint().address().to_string() };
replace(string_replace, ".", "");
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_ac_E", "_" + string_replace + request_name, m_formatting_ostream_string, mBOOST_LOGMutex);
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
continue;////allam 2020 here i should continue for next iterator
}
}
//??5-17-2020 isolate multithreaded error
if (iterator_failed == true)
{
m_formatting_ostream << "Request failed for " << mUrl << " after trying " << ATTEMPTS << "times" << " to async_connect inside HTTPRequest::Execute with ALL resolver iterators" << std::endl;
m_formatting_ostream.flush();
////allam 2020 i might need to pass iterator resolve which has failed
//4-26-2020
//boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, "failed_async_connect_inside_HTTPRequest_Execute", "requestFailed_Iterator_" + GetmUrlLogger(), m_formatting_ostream_string, mBOOST_LOGMutex);////allam2020 ?????i might need to change this from GetmUrlLogger to request name argument of Execute???????????????????4-2-2020
boost_log_function(mHTTPRequest_LoggingInstance_shared_pointer, mHTTPRequest_Symbol_str, "f_ac_E", "_" + GetmUrlLogger()+ request_name,m_formatting_ostream_string, mBOOST_LOGMutex);////allam2020 ?????i might need to change this from GetmUrlLogger to request name argument of Execute???????????????????4-2-2020
//5-16-2020m_formatting_ostream.clear();
m_formatting_ostream_string.clear();
}
////allam2020 should i put if conditions for mSendRequest ....mReadResponseBody????? to identify final complete error at these functions and end of 5 attempts
}
and this is function read_bi5_main:
int HTTPRequest::read_bi5_main(boost::filesystem::path p, ptime epoch)
{
unsigned char *buffer;
size_t buffer_size;
int counter;
size_t raw_size = 0;
std::string filename_string = p.generic_string();
path p2 = p;
p2.replace_extension(".bin");
std::string filename_string_2_bin =p2.generic_string() ;
path p3 = p;
p3.replace_extension(".csv");
std::string filename_string_2_csv = p3.generic_string();
const char *filename = filename_string.c_str();
const char *filename_2_bin = filename_string_2_bin.c_str();
const char *filename_2_csv = filename_string_2_csv.c_str();
if (fs::exists(p) && fs::is_regular(p))
{
buffer_size = fs::file_size(p);
buffer = new unsigned char[buffer_size];
}
else {
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Error: couldn't access the data file. |"
<< filename << "|" << std::endl;
return 2;
}
std::ifstream fin(filename, std::ifstream::binary);
//fin.open(filename, std::ifstream::binary);
fin.read(reinterpret_cast<char*>(buffer), buffer_size);
fin.close();
//5-11-2020 the next line will be commented and put in HTTPCLIent constructor
//mHTTPRequest_Symbol_str= mHTTPRequest_HTTPClient_shared_pointer->Get_mHttpClient_HttpSymbolPrepareGet_shared_pointer()->mSymbol_strGet() ;
std::size_t pos = mHTTPRequest_Symbol_str.find("JPY");// position of "h_ticks.bi5" in str
double PV;
if (pos != std::string::npos)
{
PV = PV_YEN_PAIR;
}
else
{
PV = PV_DOLLAR_PAIR;
}
boost::shared_ptr<unsigned char> data_bin_buffer = boost::make_shared<unsigned char>() ;
//boost::unique_lock<boost::mutex> read_bi5_to_bin_lock(m_read_bi5_to_binMutex);
n47::tick_data *data = n47::read_bi5_to_bin(
buffer, buffer_size, epoch, PV, &raw_size, data_bin_buffer.get());
//read_bi5_to_bin_lock.unlock();
//5-11-2020 here i will save binary file
std::string file_name_path_string=output_compressed_file_2(data_bin_buffer, raw_size, filename_2_bin);
path file_name_path_2{ file_name_path_string };
buffer_size = 0;
if (fs::exists(file_name_path_2) && fs::is_regular(file_name_path_2))
{
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << boost::this_thread::get_id() <<"\t we can access the data .bin file. |"
<< filename_2_bin << "| with size ="<< fs::file_size(file_name_path_2) << std::endl;
}
else {
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Error: couldn't access the data .bin file. |"
<< filename_2_bin << "|" << std::endl;
return 2;
}
n47::tick_data_iterator iter;
//5-11-2020 here i will save file.csv from data which is pointer to vector to pointers to ticks
if (data == 0) {
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Failure: Failed to load the data!" << std::endl;
//5-14-2020 file is empty
//return 0;
}
//5-15-2020 take care that without else ,error happens with empty files because data is pointer to vector of pointers to ticks .so when data is made inside read_bi5 ,it is made as null pointer and later it is assigned to vector if file has ticks.if file does not have ticks ,then it is just returned as null pointer .so when dereferencing null pointer we got error
else if (data->size() != (raw_size / n47::ROW_SIZE)) {
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Failure: Loaded " << data->size()
<< " ticks but file size indicates we should have loaded "
<< (raw_size / n47::ROW_SIZE) << std::endl;
//5-14-2020 file is empty
//return 0;
}
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "time, bid, bid_vol, ask, ask_vol" << std::endl;
counter = 0;
std::ofstream out_csv(filename_string_2_csv);
if (data == 0)
{
}
else if (data != 0)
{
////read_bi5_to_bin_lock.lock();
for (iter = data->begin(); iter != data->end(); iter++) {
//5-11-2020 here i will save file.csv from data which is pointer to vector to pointers to ticks>>>>>>>here i should open file stream for output and save data to it
out_csv << ((*iter)->epoch + (*iter)->td) << ", "
<< (*iter)->bid << ", " << (*iter)->bidv << ", "
<< (*iter)->ask << ", " << (*iter)->askv << std::endl;
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) <<
boost::this_thread::get_id() << "\t"<<((*iter)->epoch + (*iter)->td) << ", "
<< (*iter)->bid << ", " << (*iter)->bidv << ", "
<< (*iter)->ask << ", " << (*iter)->askv << std::endl;
counter++;
}
////read_bi5_to_bin_lock.unlock();
}
out_csv.close();
//5-13-2020
//??5-17-2020 isolate multithreaded error
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << ".end." << std::endl << std::endl
<< "From " << raw_size << " bytes we read " << counter
<< " records." << std::endl
<< raw_size << " / " << n47::ROW_SIZE << " = "
<< (raw_size / n47::ROW_SIZE) << std::endl;
delete data;
delete[] buffer;
return 0;
}
this function uses lzma decompression files which i didnot include.if required i will include.
i wish you give me mind map of how to reach error source.
thanks
UPDATE:
this is read_bi5_to_bin
tick_data* read_bi5_to_bin(
unsigned char *lzma_buffer, size_t lzma_buffer_size, pt::ptime epoch,
float point_value, size_t *bytes_read, unsigned char* buffer_decompressed) {
tick_data *result = 0;
// decompress
int status;
buffer_decompressed = n47::lzma::decompress(lzma_buffer,
lzma_buffer_size, &status, bytes_read);
if (status != N47_E_OK)
{
bytes_read = 0;
}
else {
// convert to tick data (with read_bin).
result = read_bin(buffer_decompressed, *bytes_read, epoch, point_value);
//delete[] buffer;
}
return result;
}
