GRPC Core  9.0.0
Transport Explainer

Existing Transports

gRPC transports plug in below the core API (one level below the C++ or other wrapped-language API). You can write your transport in C or C++ though; currently (Nov 2017) all the transports are nominally written in C++ though they are idiomatically C. The existing transports are:

Among these, the in-process is likely the easiest to understand, though arguably also the least similar to a "real" sockets-based transport since it is only used in a single process.

Transport stream ops

In the gRPC core implementation, a fundamental struct is the grpc_transport_stream_op_batch which represents a collection of stream operations sent to a transport. (Note that in gRPC, stream and RPC are used synonymously since all RPCs are actually streams internally.) The ops in a batch can include:

The fundamental responsibility of the transport is to transform between this internal format and an actual wire format, so the processing of these operations is largely transport-specific.

One or more of these ops are grouped into a batch. Applications can start all of a call's ops in a single batch, or they can split them up into multiple batches. Results of each batch are returned asynchronously via a completion queue.

Internally, we use callbacks to indicate completion. The surface layer creates a callback when starting a new batch and sends it down the filter stack along with the batch. The transport must invoke this callback when the batch is complete, and then the surface layer returns an event to the application via the completion queue. Each batch can have up to 3 callbacks:

Timelines of transport stream op batches

The transport's job is to sequence and interpret various possible interleavings of the basic stream ops. For example, a sample timeline of batches would be:

  1. Client send_initial_metadata: Initiate an RPC with a path (method) and authority
  1. Server recv_initial_metadata: accept an RPC
  1. Client send_message: Supply the input proto for the RPC
  1. Server recv_message: Get the input proto from the RPC
  1. Client send_trailing_metadata: This is a half-close indicating that the client will not be sending any more messages
  1. Server recv_trailing_metadata: The server sees this from the client and knows that it will not get any more messages. This won't complete yet though, as described above.
  1. Server send_initial_metadata, send_message, send_trailing_metadata: A batch can contain multiple ops, and this batch provides the RPC response headers, response content, and status. Note that sending the trailing metadata will also complete the server's receive of trailing metadata.
  1. Client recv_initial_metadata: The number of ops in one side of the batch has no relation with the number of ops on the other side of the batch. In this case, the client is just collecting the response headers.
  1. Client recv_message, recv_trailing_metadata: Get the data response and status

There are other possible sample timelines. For example, for client-side streaming, a "typical" sequence would be:

  1. Server: recv_initial_metadata
    • At API-level, that would be the server requesting an RPC
  1. Server: recv_trailing_metadata
    • This is for when the server wants to know the final completion of the RPC through an AsyncNotifyWhenDone API in C++
  1. Client: send_initial_metadata, recv_message, recv_trailing_metadata
    • At API-level, that's a client invoking a client-side streaming call. The send_initial_metadata is the call invocation, the recv_message collects the final response from the server, and the recv_trailing_metadata gets the grpc::Status value that will be returned from the call
  1. Client: send_message / Server: recv_message
    • Repeat the above step numerous times; these correspond to a client issuing Write in a loop and a server doing Read in a loop until Read fails
  1. Client: send_trailing_metadata / Server: recv_message that indicates doneness (NULL)
    • These correspond to a client issuing WritesDone which causes the server's Read to fail
  1. Server: send_message, send_trailing_metadata
    • These correspond to the server doing Finish

The sends on one side will call their own callbacks when complete, and they will in turn trigger actions that cause the other side's recv operations to complete. In some transports, a send can sometimes complete before the recv on the other side (e.g., in HTTP/2 if there is sufficient flow-control buffer space available)

Other transport duties

In addition to these basic stream ops, the transport must handle cancellations of a stream at any time and pass their effects to the other side. For example, in HTTP/2, this triggers a RST_STREAM being sent on the wire. The transport must perform operations like pings and statistics that are used to shape transport-level characteristics like flow control (see, for example, their use in the HTTP/2 transport).

Putting things together with detail: Sending Metadata

Requirements for any transport

Each transport implements several operations in a vtbl (may change to actual virtual functions as transport moves to idiomatic C++).

The most important and common one is perform_stream_op. This function processes a single stream op batch on a specific stream that is associated with a specific transport:

There are other functions in the vtbl as well.

Book-keeping responsibilities of the transport layer

A given transport must keep all of its transport and streams ref-counted. This is essential to make sure that no struct disappears before it is done being used.

A transport must also preserve relevant orders for the different categories of ops on a stream, as described above. A transport must also make sure that all relevant batch operations have completed before scheduling the on_complete closure for a batch. Further examples include the idea that the server logic expects to not complete recv_trailing_metadata until after it actually sends trailing metadata since it would have already found this out by seeing a NULL’ed recv_message. This is considered part of the transport's duties in preserving orders.