On either side, client and server, you are going to need the keystore for that side. That only really needs the certificate and private key.
Then, each side needs the trust root of the other side, but doesn't need its own. The server needs the root CA of the client, and vice-versa (unless you ignore server-certificate verification on the client). The root will be in the truststore.
Then on top of that, each side needs any intermediate CAs of the other side. They can either already have them in their truststore, or, they can receive them from the other side.
So to be polite, each side could also include their own intermediate CA certificates and send them in the chain to help the other side out. Otherwise, neither side actually has any need for their own CA or intermediate CA certificates.
So the server needs the client's root CA, and will either need any intermediate CA certificates ahead of time, or receive them from the client.
UPDATE: Responding to your comment below, if you want to filter client certificates, it is possible in some TLS implementations (openssl, for example). You can hook into the verification step and have your say as to whether or not the connection is allowed. This seems a little low-level, but it does have some advantages. For example, you could keep a concatenation of all permitted client certificates in a text file (only the public certs -- not any private keys), and read this into an openssl "stack of certificates" at start up, then run through that looking for a match on each TLS connection. That is a very specific whitelisting of clients.
I would caution against simply checking the DN for a pattern to decide if the client is allowed, as this could be a big security hole. An attacker could simply obtain a public certificate from any well-known CA, asking for a DN that fits the pattern you are looking for. This would be accepted by your server because the CA is one of hundreds of trusted CAs on the typical default truststore. Once the connection is accepted, the DN fits the expected pattern, and so the client is allowed.