Client certificate authentication should give you strong assurance that the client is whom they claim to be (as specified in the certificate subject or through the Subject Alternative Name extension). Otherwise, this would indicate a major problem with the public-key infrastructure or the way clients protect their private keys.
Of course it can be useful to consider defense-in-depth mechanisms like public-key pinning. However, the question is what you're trying to achieve, and there are some implementations challenges. As to the first aspect, server-side key pinning can prevent the server from accepting certificates that have been falsely issued by a compromised or rogue CA. This makes a lot of sense in a large infrastructure with many different CAs, but in your case, you should only deal with one CA: your own. If you want protection against the case that your CA is compromised, then pinning is valid, otherwise you may not achieve anything useful. As to the implementation, I'm not aware of any standard or common webserver software which allows the server to pin public keys of the client. The opposite is of course well-known from the (now largely obsolete) HTTP Public Key Pinning extension. So you may have to implement this mechanism yourself.
To strengthen certificate-based authentication, you should make sure the CA keys are very well protected, ideally through a hardware security module or at least a completely separate, physically isolated machine.
Access control in the sense of authorization is an entirely different topic. Client certificates only deal with authentication.
