Converters and encryption - .NET SDK

By default, Temporal payloads are stored unencrypted inside of its data store. Consequently, this means that string payloads can be read from the Temporal Web UI and CLI in plain text. When working with sensitive data, Temporal implementers may need to adopt encryption algorithms, manage encryption keys, or restrict a subset of their users from viewing payload output.

A Custom Codec allows a developer to transform the payload of a message sent or received by a Temporal Client. This ensures that the data is encrypted as it travels across the network and when it is stored in the Event History, readable only by those with access to the key.

Custom Payload Codec

How to use a custom Payload Codec using the .NET SDK

Custom Data Converters can change the default Temporal Data Conversion behavior by adding hooks, sending payloads to external storage, or performing different encoding steps. If you only need to change the encoding performed on your payloads -- by adding compression or encryption -- you can override the default Data Converter to use a new PayloadCodec.

The IPayloadCodec needs to implement EncodeAsync() and DecodeAsync() methods. These should convert the given payloads as needed into new payloads, using the "encoding" metadata field. Do not mutate the existing payloads. Here is an example of an encryption codec that just uses base64 in each direction:

public class EncryptionCodec : IPayloadCodec
{
    public Task<IReadOnlyCollection<Payload>> EncodeAsync(IReadOnlyCollection<Payload> payloads) =>
        Task.FromResult<IReadOnlyCollection<Payload>>(payloads.Select(p =>
        {
            return new Payload()
            {
                // Set our specific encoding. We may also want to add a key ID in here for use by
                // the decode side
                Metadata = { ["encoding"] = "binary/my-payload-encoding" },
                Data = ByteString.CopyFrom(Encrypt(p.ToByteArray())),
            };
        }).ToList());

    public Task<IReadOnlyCollection<Payload>> DecodeAsync(IReadOnlyCollection<Payload> payloads) =>
        Task.FromResult<IReadOnlyCollection<Payload>>(payloads.Select(p =>
        {
            // Ignore if it doesn't have our expected encoding
            if (p.Metadata.GetValueOrDefault("encoding") != "binary/my-payload-encoding")
            {
                return p;
            }
            // Decrypt
            return Payload.Parser.ParseFrom(Decrypt(p.Data.ToByteArray()));
        }).ToList());

    private byte[] Encrypt(byte[] data) => Encoding.ASCII.GetBytes(Convert.ToBase64String(data));

    private byte[] Decrypt(byte[] data) => Convert.FromBase64String(Encoding.ASCII.GetString(data));
}

Set Data Converter to use custom Payload Codec

When creating a client, the default DataConverter can be updated with the payload codec like so:

var myClient = await TemporalClient.ConnectAsync(new("localhost:7233")
{
    DataConverter = DataConverter.Default with { PayloadCodec = new EncryptionCodec() },
});

• Data encoding is performed by the client using the default converter provided by Temporal or your custom Data Converter when passing input to the Temporal Cluster. For example, plain text input is usually serialized into a JSON object, and can then be compressed or encrypted.
• Data decoding may be performed by your application logic during your Workflows or Activities as necessary, but decoded Workflow results are never persisted back to the Temporal Cluster. Instead, they are stored encoded on the Cluster, and you need to provide an additional parameter when using the temporal workflow show command or when browsing the Web UI to view output.

For reference, see the Encryption sample.

Using a Codec Server

A Codec Server is an HTTP server that uses your custom Codec logic to decode your data remotely. The Codec Server is independent of the Temporal Cluster and decodes your encrypted payloads through predefined endpoints. You create, operate, and manage access to your Codec Server in your own environment. The temporal CLI and the Web UI in turn provide built-in hooks to call the Codec Server to decode encrypted payloads on demand. Refer to the Codec Server documentation for information on how to design and deploy a Codec Server.

Payload conversion

Temporal SDKs provide a default Payload Converter that can be customized to convert a custom data type to Payload and back.

Conversion sequence

The order in which your encoding Payload Converters are applied depend on the order given to the Data Converter. You can set multiple encoding Payload Converters to run your conversions. When the Data Converter receives a value for conversion, it passes through each Payload Converter in sequence until the converter that handles the data type does the conversion.

Payload Converters can be customized independently of a Payload Codec. Temporal's Converter architecture looks like this:

Custom Payload Converter

How to use a custom Payload Converter with the .NET SDK.

Data converters are used to convert raw Temporal payloads to/from actual .NET types. A custom data converter can be set via the DataConverter option when creating a client. Data converters are a combination of payload converters, payload codecs, and failure converters. Payload converters convert .NET values to/from serialized bytes. Payload codecs convert bytes to bytes (e.g. for compression or encryption). Failure converters convert exceptions to/from serialized failures.

Data converters are in the Temporalio.Converters namespace. The default data converter uses a default payload converter, which supports the following types:

• null
• byte[]
• Google.Protobuf.IMessage instances
• Anything that System.Text.Json supports
• IRawValue as unconverted raw payloads

Custom converters can be created for all uses. For example, to create client with a data converter that converts all C# property names to camel case, you would:

using System.Text.Json;
using Temporalio.Client;
using Temporalio.Converters;

public class CamelCasePayloadConverter : DefaultPayloadConverter
{
    public CamelCasePayloadConverter()
      : base(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
    {
    }
}

var client = await TemporalClient.ConnectAsync(new()
{
    TargetHost = "localhost:7233",
    Namespace = "my-namespace",
    DataConverter = DataConverter.Default with { PayloadConverter = new CamelCasePayloadConverter() },
});