Data Encryption

An important security service provided by the fabrication process is data encryption. Data encryption is a method where information is encoded and obscured making it unreadable by unauthorized entities.

Encryption Method

AES encryption

AES encryption is a symmetric-key algorithm which means you can use the same key to encrypt and decrypt data. To encrypt data using this algorithm you first need to generate a 128 bit (16 character) key and save it in the encrypt.properties file, then follow the example below to encrypt data using the AES algorithm.

encrypt.properties

encrypt.aes.secret.key.spec=<SOME_AES_ENCRYPTED_VALUE_HERE>

Java

import com.boozallen.aissemble.data.encryption.SimpleAesEncrypt;

AiopsEncrypt simpleAiopsEncrypt = new SimpleAesEncrypt();

//Encrypt
String encryptedData = simpleAiopsEncrypt.encryptValue("Plain text string data");

//Decrypt
String decryptedVaultData = simpleAiopsEncrypt.decryptValue(encryptedData);

Python

from aiops_encrypt.aes_encrypt import AESEncrypt

aes_encrypt = AESEncrypt()

# Encrypt
encrypted_value = aes_encrypt.encrypt('Plain text string data')

# Decrypt
decrypted_value = aes_encrypt.decrypt(encrypted_value)

Vault encryption

Vault encryption is not available at this time.

Another encryption service is provide through Hashicorp Vault. This is a well known provider of security services and the fabrication process generates the client methods for calling and encrypting data. In order to encrypt data using V ault the following properties need to be set:

encrypt.properties

secrets.host.url=[Some Url]
Example: http://vault:8217
secrets.unseal.keys=[Comma delimited set of keys]
Example (no quotes or whitespace): key1,key2,key3,key4,key5
secrets.root.key=[The root key]
Example: s.EXAMPLE

The keys in the example property file above can be retrieved from the vault console log. The keys will be printed at the top of the log when the server is started. For example:

Vault console log

vault    | ROOT KEY
vault    | s.EXAMPLE
vault    | UNSEAL KEYS
vault    | ["key1", "key2", "key3", "key4", "key5"]
vault    | TRANSIT TOKEN
vault    | {"request_id": "29d26f42-7be2-9b06-c4ce-1ecc94114393", "lease_id": "", "renewable": false, "lease_duration": 0, "data": null, "wrap_info": null, "warnings": null, "auth": {"client_token": "s.TOKEN", "accessor": "zFcMdiOHhtXUyRTUigkePpzS", "policies": ["app-aiops", "default"], "token_policies": ["app-aiops", "default"], "metadata": null, "lease_duration": 2764800, "renewable": true, "entity_id": "", "token_type": "service", "orphan": false}}

To add vault encryption to your code follow the example below. Please note that the vault Docker container needs to be running in order for this example to work.

Java

import com.boozallen.aissemble.data.encryption.VaultEncrypt;

AiopsEncrypt vaultAiopsEncrypt = new VaultEncrypt();

//Encrypt
String encryptedData = vaultAiopsEncrypt.encryptValue("Plain text string data");

//Decrypt
String decryptedVaultData = vaultAiopsEncrypt.decryptValue(encryptedData);

Python

from aiops_encrypt.vault_remote_encryption_strategy import VaultRemoteEncryptionStrategy
from aiops_encrypt.vault_local_encryption_strategy import VaultLocalEncryptionStrategy

vault_client = VaultRemoteEncryptionStrategy()
'''
 Optionally you can use local Vault encryption (vault_client = VaultLocalEncryptionStrategy())
 which will download the encryption key once and  perform encryption locally without a round trip
 to the Vault server.  This is useful for encrypting large data objects and for high volume encryption tasks.
 NOTE: If you are encrypting your data through a User Defined Function (udf) in PySpark you need to use
 the VaultLocalEncryptionStrategy.  Currently the remote version causes threading issues.  This issue will
 likely be resolved in a future update to the Hashicorp Vault client
'''

# Encrypt
encrypted_value = vault_client.encrypt('Plain text string data')

# Decrypt
decrypted_value = vault_client.decrypt(encrypted_value)

Encryption by policy

The fabrication process generates built in encryption code that can be activated through a policy file. When an encryption policy is configured the pipeline will apply encryption to the fields specified in the policy. The following example illustrates how to encrypt a field named "ssn" during the ingest step.

example-encrypt-policy.json (can be named anything as long as it’s in the correct policy directory)

[
  {
    "identifier": "encryptSSN",
    "rules": [
      {
        "className": "EncryptRule",
        "configurations": {
          "description": "Apply encryption policy"
        }
      }
    ],
    "encryptPhase": "ingest",
    "encryptFields": [
      "ssn"
    ],
    "encryptAlgorithm": "AES_ENCRYPT"
  }
]

This file should be placed in a directory which can be specified by the user in the policy-configuration.properties file (see example below).

encryptPhase - The step in the pipeline where encryption takes place. Typically, this will happen in the first step.

encryptFields - An array of field names that will be encrypted.

encryptAlgorithm - The algorithm that will be used to encrypt the data. Currently, the options are AES_ENCRYPT and VAULT_ENCRYPT. More can be added through customization.

policy-configuration.properties

policies-location=policies

This configuration defines which folder the encryption policy resides in. In the example above the policies are in the policies directory (relative to the working directory). An absolute path can also be used.