Requirement
A PGP-based encryption and decryption solution is required to establish a secure data transfer mechanism between the source platform(here Zeotap CDP) and the destination platform(customer). The scope of this document is limited to batch file-based data transfers. In this process, the destination platform (customer) would generate a private/public key , and upload the public key to the relevant batch based destination in Zeotap CDP. Zeotap would use this public key to encrypt the data and send it to the destination platform (customer). The destination platform (customer) would use the private key generated on their end to decrypt the data. Zeotap will be storing the public key shared by the customer in Google Secret manager.Solution
Supported Destinations for PGP Encryption
File-level encryption is supported for the following Zeotap destinations:- GCS
- SFTP
Supported PGP Protocol Version
The current implementation adheres to RFC 4880 (OpenPGP Message Format). Both the client and Zeotap are recommended to follow this standard for all integrations across layers. The file containing the public key which is required for encryption should be in .asc format. We suggest using the RSA algorithm to generate the public key. Sample public key is attached below:Code
Notes/References
- A pair of asymmetric (public-private) keys should be generated by the customer.
- During the destination setup in Zeotap, users can specify which data needs to be PGP encrypted, offering flexibility to enable PGP encryption only for the required and supported destination platforms.
- The decision to use RFC 4880 is based on the limited adoption of the newer RFC 9580 (released in July 2024), which is not yet widely supported by PGP clients and libraries.
- For more information on PGP compatibility, refer to Phil Zimmermann’s PGP directory.
- For Java-based implementations, refer to Bouncy Castle’s interoperability documentation.
High-Level Data Flow Architecture
Prerequisites for the customer before creating destination
- The customer would need to generate a pair of asymmetric key pairs (or Public-Private key pairs), and they would need to upload the public key while creating the destination in Zeotap.
- Zeotap would encrypt the data using the public key and send it to the customer.
- The customer would use the private key generated on their end to decrypt the data.
Destination Creation (GCS, SFTP)
While creating a GCS or SFTP destination, there is an option to select whether the data needs to be encrypted before sending from Zeotap.
The public key will be used to encrypt data in Zeotap before sending it to the destination platform.a. The file containing the public key which is required for encryption should be in .asc format.