There is already a question (with six answers) about that, however the OP there was interested in the approximate area of intersection.
If you need an exact solution, then you'll have to take into account many corner cases, which make geometrical problems so difficult to be solved correctly on a real computer with limited data precision - for example, see here.
Fortunately, there is already a number of high-quality computational geometry libraries, which can solve this kind of problems using exact computations with exact numbers. One of them is CGAL, which is a joint project of many universities, well developed and tested. However, this library doesn't support rotated rectangles as a separate entity - you'll need to work with general polygons. So, your code will look like this:
#include <iostream>
#include <vector>
#include <CGAL/Boolean_set_operations_2.h>
#include <CGAL/Exact_predicates_exact_constructions_kernel.h>
#include <CGAL/Polygon_2.h>
#include <CGAL/Polygon_with_holes_2.h>
using Kernel = CGAL::Exact_predicates_exact_constructions_kernel;
using Polygon = CGAL::Polygon_2<Kernel>;
using PolygonWithHoles = CGAL::Polygon_with_holes_2<Kernel>;
using Point = Polygon::Point_2;
int main()
{
// ------ define polygon #1
const std::vector<Point> pts1{/* ... polygon #1 vertices ... */};
const Polygon p1(pts1.cbegin(), pts1.cend());
// ------ define polygon #2
const std::vector<Point> pts2{/* ... polygon #2 vertices ... */};
const Polygon p2(pts2.cbegin(), pts2.cend());
// ------ compute intersection - you'll get a single polygon without holes!!!
std::vector<PolygonWithHoles> res;
CGAL::intersection(p1, p2, std::back_inserter(res));
// ------ output area of intersection
std::cout << res[0].outer_boundary().area() << std::endl;
}
If you don't care about robustness of geometric calculations, then this library still can help you - you can choose regular double
numbers to represent coordinates. In this case you'll get better performance with a possibility of getting wrong answers in some cases.