A git horror story: faking provenance with a loose nonce

Reading Time: 4 minutes

A few weeks ago, a “vulnerability” was discovered in a new security feature that shipped in Git 2.2.0: signed pushes. No need to panic though! It is vanishingly unlikely that anyone would be able to successfully exploit this particular problem. In fact, the issue was quietly fixed a few days ago in the 2.3.7 point release.

The vulnerability does however make for a fascinating story, and serves as a cautionary tale for developers on limiting and sanitizing the content received over the wire from a server or client that you don’t completely trust.

Signed pushes

Push signing allows Git servers to guarantee that a particular client updated a branch or tag to point to a particular commit. This can be used, for example, to trace the origin of malicious code that has made its way into a repository. Roughly speaking, the signing process (that introduces our vulnerability) works as follows:

  1. The client invokes git push --signed, kicking off the client-server git protocol negotiation.
  1. The server responds with the set of capabilities that it supports. If the server supports signed pushes it will include:

push-cert=<nonce>

The nonce in this case being a securely generated random string known only to the server.

  1. The client sends a push certificate containing:
    • the pusher (an identifier for the client’s GPG key)
    • the pushee (the URL of the repository)
    • the nonce
    • one or more create, update or delete operations to be performed on branches or tags
    • a GPG signature of all of the above, generated using the client’s private key
  2. The server verifies the signature using the user’s public key (typically stored in the repository) and accepts the push.

So, did you spot the problem? It’s pretty subtle.

The nonce

The nonce is used to prevent a particular man-in-the-middle (MITM) technique known as a replay attack. The server specifies a different nonce value at the beginning of each push process and will only continue when the client supplies a signed version of that nonce. The problem lies in the fact that the server specifies the nonce value to be used. If you’re familiar with other authentication systems that use nonces, for example OAuth, you might be used to the client generating a random nonce. This is not the case with the Git protocol.

The reason a server generated nonce is problematic in this case is that a malicious server can use this mechanism to force a client to sign any arbitrary value that fits into a nonce with their private key. And it turns out the Git pack protocol is very loose when it comes to what values you can use as a nonce. Most modern Git clients will accept as a nonce any collection of bytes that is:

  • less than ~64kb; and
  • doesn’t contain the null character, tabs, spaces or newlines

For example, if I initiate a git push --signed, a malicious server could respond with the capability:


push-cert:"Please_wire_$100,000_from_my_account_to_XXX-XXXX-XXX._Sincerely,_kannonboy."

And my Git client would automatically sign it with my private key and send it back to the server. This is something that only I should be able to willfully do, and observers (for example, my bank) could believe that I had authored or at least ratified the signed statement.

Of course, the underscores look a bit weird. And the nonce is embedded in a push certificate, so there would be a bunch of junk in it that would tip off my bank before they forked over the cash (I hope). To illustrate, the signed push request would look something like:


push-certside-band-64k

certificate version 0.1

pusher kannonboy <kannonboy@example.com>

pushee git@bitbucket.org:kannonboy/example

nonce Please_wire_$100,000_from_my_account_to_XXX-XXXX-XXX._Sincerely,_kannonboy.          

9be89160e7382a88e56a02bcf38f4694dd6542d6 b89363e4a5277038629491f8765c0598f366326c branch

However, some binary file formats are very forgiving about the format of a file. Jann Horn, the security researcher who first reported the vulnerability, pointed out that PDF files are particularly problematic:

If you replace all n with r and all spaces with f in a PDF file, then add lines above and below the PDF file, most PDF readers (e.g. evince and xpdf) will still treat it as a valid PDF file.

So if the server had craftily encoded the request for $100,000 as a PDF and passed it as the nonce value, they would end up with a signed PDF that my bank would be much more likely to fall for.

The solution

The vulnerability has been fixed by limiting the nonce size to 256 bytes and whitelisting the allowed characters [a-zA-Z0-9./+=_-]. It’s believed this will restrict the nonce content to the point where an attacker will be unable to force the client to produce a signed file of any exploitable value.

Why you don’t need to worry

Once again, it’s hard to label this issue a vulnerability, as the threat of exposure is so low.

You can’t use this technique to make malicious changes to the repository, as the ref operations specified in the pack protocol are rigidly defined. It is only exploitable in that it can be used to generate signatures for content that fits into a nonce, and even then the signed content will contain push certificate cruft that will look very suspicious to most consumers. In the fraudulent bank transfer above, the looseness of the PDF specification is arguably just as much to blame as the nonce.

In addition to this limitation, the attacker must have full control the Git server that the user is pushing to in order to exploit the problem. Or somehow spoof the server via MITM, which is difficult as most Git servers will be protected by TLS (particularly those frequented by Git users who sign their pushes).

While this vulnerability is a neat trick and a fun hypothetical exploit, your Git servers and bank accounts are probably safe for the time being. However, there is a slightly more important take away for developers who dabble in security: if you’re ever programmatically using GPG keys, don’t sign arbitrary data from third parties!

Keep an eye on @kannonboy for further musings about programming, git, and developer tools.

The title of this post is borrowed from Mike Gerwitz’s excellent paper illustrating why tag, commit and push signing are valuable practices for security minded professionals.