Introduction
This document describes compression as implemented by the gRPC C core. See thefull compression specification" for details.
Intended Audience
Wrapped languages developers, for the purposes of supporting compression by interacting with the C core.
Criteria for GA readiness
- Be able to set compression at channel, call and message level. In principle this API should be based on compression levels as opposed to algorithms. See the discussion below.
- Have unit tests covering the cases from the spec.
- Interop tests implemented and passing on Jenkins. The two relevant interop test cases are large_compressed_unary and server_compressed_streaming.
Summary Flowcharts
The following flowcharts depict the evolution of a message, both incoming and outgoing, irrespective of the client/server character of the call. Aspects still not symmetric between clients and servers (e.g. the use of compression levels) are explicitly marked. The in-detail textual description for the different scenarios is described in subsequent sections.
Incoming Messages
Outgoing Messages
Levels vs Algorithms
As mentioned in the relevant discussion on the spec document, compression levels are the primary mechanism for compression selection at the server side. In the future, it'll also be at the client side. The use of levels abstracts away the intricacies of selecting a concrete algorithm supported by a peer, on top of removing the burden of choice from the developer. As of this writing (Q2 2016), clients can only specify compression algorithms. Clients will support levels as soon as an automatic retry/negotiation mechanism is in place.
Per Channel Settings
Compression may be configured at channel creation. This is a convenience to avoid having to repeatedly configure compression for every call. Note that any compression setting on individual calls or messages overrides channel settings.
The following aspects can be configured at channel-creation time via channel arguments:
Disable Compression <em>Algorithms</em>
Use the channel argument key GRPC_COMPRESSION_CHANNEL_ENABLED_ALGORITHMS_BITSET (from grpc/impl/codegen/compression_types.h), takes a 32 bit bitset value. A set bit means the algorithm with that enum value according to grpc_compression_algorithm is enabled. For example, GRPC_COMPRESS_GZIP currently has a numeric value of 2. To enable/disable GZIP for a channel, one would set/clear the 3rd LSB (eg, 0b100 = 0x4). Note that setting/clearing 0th position, that corresponding to GRPC_COMPRESS_NONE, has no effect, as no-compression (a.k.a. identity) is always supported. Incoming messages compressed (ie, encoded) with a disabled algorithm will result in the call being closed with GRPC_STATUS_UNIMPLEMENTED.
Default Compression <em>Level</em>
**(currently, Q2 2016, only applicable for server side channels. It's ignored for clients.)** Use the channel argument key GRPC_COMPRESSION_CHANNEL_DEFAULT_LEVEL (from grpc/impl/codegen/compression_types.h), valued by an integer corresponding to a value from the grpc_compression_level enum.
Default Compression <em>Algorithm</em>
Use the channel argument key GRPC_COMPRESSION_CHANNEL_DEFAULT_ALGORITHM (from grpc/impl/codegen/compression_types.h), valued by an integer corresponding to a value from the grpc_compression_level enum.
Per Call Settings
Compression <strong>Level</strong> in Call Responses
The server requests a compression level via initial metadata. The send_initial_metadata grpc_op contains a maybe_compression_level field with two fields, is_set and compression_level. The former must be set when actively choosing a level to disambiguate the default value of zero (no compression) from the proactive selection of no compression.
The core will receive the request for the compression level and automatically choose a compression algorithm based on its knowledge about the peer (communicated by the client via the grpc-accept-encoding header. Note that the absence of this header means no compression is supported by the client/peer).
Compression <strong>Algorithm</strong> in Call Responses
Server should avoid setting the compression algorithm directly. Prefer setting compression levels unless there's a very compelling reason to choose specific algorithms (benchmarking, testing).
Selection of concrete compression algorithms is performed by adding a (GRPC_COMPRESS_REQUEST_ALGORITHM_KEY, <algorithm-name>) key-value pair to the initial metadata, where GRPC_COMPRESS_REQUEST_ALGORITHM_KEY is defined in grpc/impl/codegen/compression_types.h), and <algorithm-name> is the human readable name of the algorithm as given in the HTTP2 spec for Message-Encoding (e.g. gzip, identity, etc.). See grpc_compression_algorithm_name for the mapping between the grpc_compression_algorithm enum values and their textual representation.
Per Message Settings
To disable compression for a specific message, the flags field of grpc_op instances of type GRPC_OP_SEND_MESSAGE must have its GRPC_WRITE_NO_COMPRESS bit set. Refer to grpc/impl/codegen/compression_types.h),
