A Universal Cryptographic Signing Protocol for Git

Photo by Yancy Min on Unsplash

T̶h̶i̶s̶ ̶d̶o̶c̶u̶m̶e̶n̶t̶ ̶i̶s̶ ̶l̶i̶c̶e̶n̶s̶e̶d̶ ̶u̶n̶d̶e̶r̶ ̶t̶h̶e̶ ̶C̶r̶e̶a̶t̶i̶v̶e̶ ̶C̶o̶m̶m̶o̶n̶s̶ ̶A̶t̶t̶r̶i̶b̶u̶t̶i̶o̶n̶ ̶4̶.̶0̶ ̶I̶n̶t̶e̶r̶n̶a̶t̶i̶o̶n̶a̶l̶ ̶(̶C̶C̶ ̶B̶Y̶ ̶4̶.̶0̶)̶ ̶l̶i̶c̶e̶n̶s̶e̶.̶

This is a copy of the specification now governed as a Community Specification hosted on Github.

This project is generously sponsored by Google and the Linux Foundation.

This specification documents a new, proposed protocol Git uses when interacting with cryptographic signing and verification tools. The goal of this modification is to make Git able to use any signing and verification tool with a special emphasis on adding support for OpenSSH signing. The design eliminates all of the tool-specific code in Git, easing maintenance and increasing flexibility. The protocol is inspired by the Assuan Protocol used by GPG to link its component executables together but uses Git’s pkt-line framing.

The Protocol

The protocol uses Git’s pkt-line for framing packets as variable length binary strings. The first four bytes of the line, the pkt-len, indicates the total length of the line, in hexadecimal. The pkt-len includes the 4 bytes used to contain the length’s hexadecimal representation. The maximum length of a pkt-line’s data component is 65516 bytes and pkt-len MUST NOT exceed 65520 (65516 bytes of data + 4 bytes of length data).

Even though pkt-line supports a binary data component, the protocol Base64 encodes all binary data sent to, and received from, the signing/verifying tools. The primary reason for this is to ensure that Git can store received signatures directly in tag and commit objects without any processing. This is compatible with the assumptions about the existing GPG signing code that instructs GPG to create ASCII encoded signatures that are stored directly in tag and commit objects.

The protocol is designed as a client-server protocol with the client being the initiator of the connection and the server the receiver. In the case of Git, it uses the pipe-fork mechanism to spawn a new task and then the protocol is used over the stdin/stdout pipes between the processes. The pipe-forked child process acts as the server and the parent Git process is the client.

In the rest of this document, there are example operations demonstrating the protocol. In those examples, lines that begin with S: are lines sent by the server (the child process) and lines that begin with C: are lines sent by the client (the git process).

The Server

The server must support just four commands: OK, ERR, D, and #. Each command is documented below:

OK [<arbitrary debugging information>]

The request or operation was successful.

ERR [<human readable error description>]

The request or operation failed. The error code is used to signal the exact error.

D <raw data>

Sends raw data to the client. There must be exactly one space after the D. The values for ‘%’, carriage return (0x0d), and line feed (0x0a) must be escaped using percent escaping of their ASCII values in hexadecimal. So ‘%’ is encoded as %25 and carriage return and line feed are %0d and %0a respectively. Other characters may be percent escaped for easier debugging. All data lines are considered one data stream up to a terminating OK or ERR message.

# <string>

Comment line issued only for debugging purposes and totally ignored by clients.

In the Assuan protocol documentation there is a command called INQUIRE that is used by servers to ask clients for information required for specific operations. In this first implementation of the Git signing protocol, the INQUIRE command is specifically excluded because it is not needed for the immediate goal of supporting commit signing with GPG, GPGSM, and OpenSSH. In the future, signing tools that have interactive protocols may require the addition of the INQUIRE server command. For now though, it is left out to simplify implementation.

The Client

The client must support five basic commands: D, END, OPTION, BYE and #. In addition they must also support the signing command: SIGN. It also must support the verification commands: SIGNATURE, and VERIFY. Each of the commands are documented below:

D <raw data>

Sends raw data to the server. This command is the same as the D command described above in the server section.

END

Used by the client to mark the end of raw data.

OPTION name [[=] value]

Sets and option for the current operation. Leading and trailing spaces around the name and value are allowed but should be ignored. The use of the equal sign is optional but is suggested if value is given.

BYE

Tells the server the client is finished. The server will respond with OK, then the client can close the connection.

# <string>

Comment line issued only for debugging purposes and totally ignored by servers.

SIGN

The sign command initiates a cryptographic signing operation. Immediately following the SIGN command, the client must send one or more D commands sending the server the data to be signed. The data is encoded as hexadecimal string to simplify client implementation. The data is terminated with an END command, signaling to the server to sign the data and return the signature.

The server will respond with one or more D commands sending to the client the resulting data from the SIGN operation. The data sent by the server is terminated with either an OK command on success or an ERR command on error. The data sent from the server contains fields that are to be stored in the Git object. These fields may include one signtype, zero or more signoption, and one sign. They are significant for the verification process and described below in the section on verification.

SIGNATURE

Initiate the transfer of the sig data to the server. Immediately following
the SIGNATURE command, the client must send one or more D command sending the signature data to the server. The data is terminated with an END command. The server will respond with OK if successful or ERR if there is an error. The signature data must be sent to the server before a VERIFY command is issued.

VERIFY

Initiate the transfer of the signed object data to the server and execute the signature verification operation. Immediately after the VERIFY command, the client must send one or more D commands to send the signed object data to the server. The data is terminated with an END command, signaling the server to execute the signature verification. The server will respond with zero or more D lines with status information about the digital signature. The server will then send either an OK, if the verification was successful, or ERR if not.

Signing a Git Object

The general flow of the Git object signing process is as follows:

1. Git calculates the signing tool to execute from the config file and command line options.
2. Git pipe-forks a child process to execute the signing tool. For the purposes of the protocol, Git is the client and the signing tool is the server.
3. The signing tool starts the operation by sending an OK command.
4. Git issues zero or more OPTION commands to the signing tool to pass the options from the config file. The signing tool responds with OK in response to each OPTION if the option is valid. If the option is not valid, the signing tool responds with an ERR command and the signing operation will end following steps 9and 10below. Git passes all options to the signing tool and some may not be relevant to the signing operation. The signing tool shall ignore all options that are not relevant and only return an ERR response to relevant options with invalid values. A good example is an option to set the identifier that has an invalid value such as an unknown identifier.
5. Git issues zero or more OPTION commands to the signing tool to pass the --sign-option options from the command line. Because these come last, they override any prior options from the config file that have the same token.
6. Git issues the SIGN command followed by one or more D commands to pass the Git object data to the signing tool to be signed. Git sends and END command after the last D command to signal the end of the data.
7. If the signing tool successfully signs the data, it responds with one or more D commands containing data that must be stored in the Git object verbatim. The signing tool sends the OK command after the last D command to signal a successful signing.
8. If the signing tool fails to sign the data, it responds with zero or more D commands containing detailed error data to be output from Git’s STDERR stream. The signing tool sends the ERR command after the last D command to signal a failed signing and the optional “reason” string with the ERR command is also output to Git’s STDERR stream.
9. Git then sends the BYE command to conclude the signing operation.
10. The signing tool acknowledges the operation ending by sending an OK command and exiting execution.

An example successful signing operation is illustrated below. The lines beginning with S: are sent from the signing tool to Git and lines starting with C: are sent from Git to the signing tool. NOTE: Every byte of the Git object data passed to the signing tool is significant, this includes the line feed (0x0a) character at the end of each line. To pass line feed as data to the signing tool it must be escaped as %0a. The returned signature data also has significant line feeds and will also have escaped line feed characters.

An example of a signing operation that fails because of a bad OPTION set by Git. In this example Git passes the signing identity and the signing tool does not have an identity by that name so it responds with the ERR command and the reason for the error.

The Returned Signature Data

When a signing tool generates a successful signature, it sends to Git one or more D commands with signature related data that is intended to be stored verbatim inside of the Git object. The data is formatted so that Git can easily execute a signature verification process in the future. Each line of the signature data starts with a field name and a space followed by data. Multi-line fields have subsequent lines that start with a space to identify them as part of a multi-line field value.

There are three different fields that may be used in the signature data that are defined below:

signtype <signature scheme name>

The signtype field identifies the signing scheme used to generate and verify this signature. The signature scheme name must match the name used in the config file and also on the command line. For GPG signatures the scheme name is openpgp. For GPGSM signatures the scheme name is x509. For OpenSSH signatures the scheme name is openssh. With this design, Git no longer has to know any details specific to any signature scheme and nothing needs to be changed in Git to use new signature schemes in the future. There may only be one signtype field in a given signature and it must be the first field in the signature data.

signoption <option name> = <option value>

The signoption field specifies options that Git passes to the verification tool using the OPTION command during a signature verification operation. There may be zero or more signoption fields in the signature data.

sign <signature data>

The sign field specifies the signature data generated in the signing operation. There shall be only one sign field in the signature data and it may contain a multi-line field value.

The resulting signed Git object — in this case a tag — from the successful
signature example above is as follows:

A signed Git commit using the new tagging system looks like:

A signed Git mergetag using the tagging system looks like:

Verifying a Signed Git Object

The general flow of the signed Git object verification process is as follows:

1. Git parses the signtype field from the signed object to determine the
   signature type.
2. Git calculates the verification tool to execute from the config file and command line options using the signature type.
3. Git pipe-forks a child process to execute the verification tool. For the purposes of the protocol, Git is the client and the verification tool is the server.
4. The verification tool starts the operation by sending an OK command.
5. Git parses any signoption fields from the signed object and issues one OPTION command for each signoption field. The verification tool responds with OK in response to each OPTION if the option is valid. If the option is not valid the verification tool responds with an ERR command and the verification operation ends, executing steps 11 and 12 below. Git passes all options to the verification tool and some may not be relevant. The verification tool shall ignore options that are not relevant and only respond with ERR for relevant options with invalid values.
6. Git issues zero or more OPTION commands to the verification tool to pass the options from the config file. Because these come after the options from the signed object, these override any prior options from the signed object that have the same token.
7. Git issues zero or more OPTION commands to the verification tool to pass the options from the command line. Because these come last, they override any prior options from the signed object and the config file that have the same token.
8. Git parses the sign field and issues the SIGNATURE command followed by D commands to send the signature data to the verification tool. Git sends the END command after the last D command to signal the end of the signature data. The verification tool responds with either an OK or ERR command to signal success or failure. On failure the verification operation is terminated using steps 10 and 11 below.
9. Git sends the VERIFY command followed by one or more D commands to send the object data to the verification tool for signature verification. Git sends the END command to signal the end of the object data.
10. The verification tool responds with one or more D commands with the results of the verification process. If the verification process was successful, the verification tool sends the OK command after the last D command to signal the end of the result data. If the verification failed, the verification tool sends the ERR command after the last D command to signal the end of the result data.
11. Git then sends the BYE command to end the operation.
12. The verification tool acknowledges the ending ending by sending an OK command and then exits execution.

An example successful verification operation is illustrated below. The lines beginning with S: are sent from the verification tool to Git and lines starting with C: are sent from Git to the verification tool.

An example of a failed verification operation is illustrated below.