What preprocessing step to take before HoughLines

Question

I have written a program to

1. Combined Edge information and colour information to form the image I would need to detect straight line from
   

2. Then I use findContour and drawContour to redraw the evidence map

3. after which I use some thinning algorithm to collapse the line into one single line

4. Use HoughLinesP to calculate for line Segment. Only have traces of the line. And most importantly have missed out the horizontal line

4. From these set of line Segment, calculate the intersection point (not included in code)
5. From the intersection, draw a quadrilateral from the intersection point, vertices(called it v1 and v2, furthest from the intersection point) of the horizontal/vertical line and a vertex calculate based on the reflection of the intersection point on the line between v1 and v2

However, it is not working as i think it should. I think the problem lies with that the interior border of the rectangle is not filled. Should I use morphology eg dilation then erode. I have run out of ideas to preprocess the two "cues" image before trying to detect for intersection with Hough Transform

Need everyone help!

THanks in advance

Below is my code snippet to generate the following

int FindBoxes(cv::Mat& colorMap,cv::Mat& edgeMap)
{ 
    cv::Mat frame;
    // colorMap is a coloured filtered map while edgeMap is an edge  filter Map. frame will be the colour i want 
    cv::bitwise_and(colorMap, edgeMap, frame);

    // A trial method by using findContour to get the interior line filled up 
    std::vector<std::vector<cv::Point> > contours;
    std::vector<cv::Vec4i> hierarchy;
    cv::findContours(frame, contours, hierarchy, CV_RETR_EXTERNAL,
        cv::CHAIN_APPROX_SIMPLE);

    cv::Mat Map = cv::Mat::zeros(cueMap1.size(), CV_8U);
    cv::drawContours(Map, contours, -1, cv::Scalar(255, 255, 255), CV_FILLED);

   // thin the line to collapse it into one single line for Hough Detection
   cv::Mat thin;
   thinning(Map, Map);

   cv::GaussianBlur(Map, Map, cv::Size(5,5),1.0,1.0);

   std::vector<cv::Vec4i> lines;
   cv::HoughLinesP(Map, lines,1, CV_PI/90, 2, 30, 0);

   return 0;
}