Source of RegEx examples of Secret Detection patterns in repositories?

Question

Where can I find RegEx that can pattern match common secret strings?

I have a product that scans repos and commits and in case a developer tries to commit a secret (i.e. passwords, keys). It scans for roughly 30 patterns by default which seems insufficient given thousands of repos in over seventy languages. I can expand that scanning with RegEx. However, I don't know every common secret there is.

Is there a framework, list, or tool that can provide RegEx or patterns for likely secrets?

Where can I get comprehensive lists of secret types?

Or am I doomed to writing a metric ton of RegEx then being held responsible for when something is missed?

Answer 1

There are some existing published regular expression patterns which try to detect some categories of secrets. These usually rely on the standardized structure of private keys and tokens provided by concrete implementations.

See for example gitleaks.

Some patterns which are already available:

• For matching AWS secret keys: (?i)aws(.{0,20})?(?-i)['\"][0-9a-zA-Z\/+]{40}['\"]

• For matching a few types of asymmetric private keys: -----BEGIN ((EC|PGP|DSA|RSA|OPENSSH) )?PRIVATE KEY( BLOCK)?-----

Of course, there is no guarantee that all secrets will be found. It may be worth considering an alternative solution: education. By making developers more aware of the dangers of publishing secrets to repositories, they are less likely to commit such mistakes.