In short you may simply multiply by the inverse of Matrix1:
glm::mat4 Matrix2And3 = glm::inverse(Matrix1) * transMatrix;
Order of operation is important, if you wanted to remove Matrix3, you would need make transMatrix * inverse(Matrix3) instead. If it was Matrix2, you would need to remove matrix1 (or 3) and then matrix2.
However, matrix inversion should be avoided at all costs, since it's very inefficient, and for your situation it is avoidable.
What you call Matrix is actually just a 3D Pose: Position + Rotation + Size.
Assuming you are using uniform scaling (Size = float) your mat3 component of your Pose, i.e., is a Orthogonal matrix, these type of matrices have a special property which is:
Inverse(O) == Transpose(O)
Calculating the transpose of a matrix is a lot simpler than the inverse, this means that you can do the following, to achieve the same results but a lot faster:
mat4 inv = (mat4)transpose((mat3)Matrix1);
inv[3] = glm::vec4(-position1, 1.0f);
mat4 Matrix2And3 = inv * transMatrix;
If you want to go even further, I recommend you create Pose class and provide cast operators for mat4 and mat3, to take full performance at ease.