Skip to main content

Vault Integration

You can use our Vault Integration to connect to an external vault service and retrieve secrets without storing them in Cloudomation.

Use Cases

The Vault Integration offers you the possibility to authenticate to external services:

  • Authenticate yourself towards services integrated in Cloudomation
  • Use secrets (passwords, usernames, keys, ...) in flows without "hard-coding" them
  • Interact with your Vault service (for example write secrets)

Concept

Cloudomation offers the integration of a HashiCorp Vault (for more information see https://www.vaultproject.io/).

schedulesUserUserVaultVaultUser->Vaultwrite secretExecutionExecutionExecution->Vaultread secretVault\nConfigVaultConfigVault\nConfig->ExecutionConnectionConnectionConnection->Vault\nConfigreference to secretConnector type VAULTConnector type VAULTConnector type VAULT->Vaultall operations

You can interact with a Vault using:

  • Vault Configuration: configure access to your Vault service in Cloudomation.
  • Connections: configure which secrets will be used from your Vault service for a particular connection. When the saved connection is used, it will retrieve the secrets.
  • Connector type VAULT: provides an interface for exhaustive interaction with your Vault (read, write, update, versioning of secrets, change secret-metadata, ...).

Configuration

To use the Vault integration you have to

Create a Vault Configuration

You can create a Vault configuration in the User Interface by pressing the Create button and selecting Vault config:

The buttons to create a Vault configuration

The new Vault configuration opens and you can directly modify its fields.

info

Authenticate with the Vault Token or AppRole method.

Find the fields used in the Vault configuration and their meanings in the table below:

Common fields:

FieldDescriptionExample
EnabledIf unset, Cloudomation will not use this Vault configuration.
Auto-renewIf set, Cloudomation will try to renew the token before it expires. Renewal will only succeed if the MAX_TTL of the token is not reached. Please refer to token renew for details.
Vault URLThe URL to your vault installationhttps://vault.example.com:8200
CA certificateA certificate to verify the identity of the vault. Only needed if the Vault installation uses a self-signed certificate.

Specific for the Token authentication method:

FieldDescriptionExample
TokenA Vault access token which is used to fetch secrets.

Specific for the AppRole authentication method:

FieldDescriptionExample
AppRole PathPath for AppRole authentification method (final path for this authentification method will be /auth/<approle_path> and must be the same path where you enabled this method on your Vault)my-app-approle
Role-ID for AppRole workerRole-ID of worker. Associated with a role with permission to request wrapped SecretIDs.
Secret-ID for AppRole workerSecret-ID of worker.
Role name for AppRole runnerName of a role used by a runner, with permission to retrieve secrets from vault.my-database-runner
Role-ID for AppRole runnerRole-ID for the runner.
info

The worker's purpose is only fetching the wrapped secret-ID of a runner. This is the only permission it should be granted on the Vault.
The runner's role is fetching secrets.
You can have one worker configured on your Vault, but multiple runners configured with different permissions to fetch specific secrets.

Learn more about the AppRole Authentication on a HashiCorp Vault and how to set up the required Roles and Policies for your Vault: Tutorial: AppRole Pull Authentication

Attach References to Secrets in Connections

You can attach references to secrets to connections. Open a connection and press the Add vault secret button:

The add vault secret button

Find the fields you'll have to specify and their meanings in the table below:

FieldDescriptionExample
Vault config IDFrom which Vault configurartion to fetch the secret
Engine pathThe path at which secrets of a specific vault engine are storedmy-kv-engine
Secret pathPath to a secret within an engine type; can be prefixed with e.g. a methodmethod/my-secret
info

Depending on the secret engine being used, different paths and prefixes are available and required. Refer to Secrets Engines for details.

info

Required prefix for a secret in the key-value engine using version 2 is data/!

After a vault secret is attached to a connection, its values can be used in the connection value. You can map the value of a vault secret key to any value of the connection:

example

Configure a Connector and map vault secrets to connection values.

Let's assume there is a vault secret stored in a key-value version 2 secret engine below secret in the path oracle which contains two keys:

user: my-user
password: my-secret-password

Create a connection for the SQLORACLE Connector type, call it "oracle" and enter the following into the value field:

host: my-oracle-server
service_name: xe
user: vault.secret(user)
password: vault.secret(password)

Click on Add vault secret, choose the Vault configuration, enter the engine path "secret", prefix and secret path "data/oracle".

Alternatively, you can save the same connection within a flow script with (assume you have a Vault configuration with the name "default"):

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution):
# save a connection which uses vault-secrets
system.connection('my-database-connection').save(
connection_type='SQLORACLE',
value={
'host': 'my-oracle-server',
'service_name': 'xe',
'user': 'vault.secret(user)',
'password': 'vault.secret(password)',
},
vault_secrets=[
{
'vault_name': 'default',
'engine_path': 'secret',
'secret_path': 'data/my-database',
},
],
)

The example above assumes that the Vault's key-value engine is configured with version 2. You can find more information about vault secret engines at https://www.vaultproject.io/api-docs/secret

Use the Vault Integration

tip

The preferred method to read secrets is the use of a Connection.

The preferred method to write secrets is the use of VaultConfig.write_secret() method.

Read Secrets with Connections

A connection which has Vault secrets attached and mapped will automatically fetch and use the secrets every time the connection is used. The secret values are not stored in Cloudomation.

example

Use a connection which has a vault secret attached.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution):
oracle_db_version = this.connect(
'oracle',
execute='SELECT * FROM v$version',
).get('output_value')['result']
this.log(oracle_db_version)
return this.success('all done')

Using vault secrets in connections is a safe way, where secrets will not be stored or exposed within Cloudomation.

Write Secrets

Cloudomation can write secrets to a key-value Vault engine using the Flow-API method VaultConfig.write_secret().

example

Write a secret to vault.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution):
system.vault_config('my-vault').write_secret(
engine_path='my-kv-engine',
secret_path='data/my-new-secret',
data={
'username': 'cloudomation',
'password': 'super-secret',
},
)

Use the Vault Connector Type

To use the Vault Connector type it is not required to have a Vault configuration set up and saved. It is possible to create a connection of the Vault connector type with this.connect(...) (see below). All connection parameters to the Vault have to be specified.

warning

If not properly used secrets could become exposed within your flow or executions. Use this method with caution!

example

Use the Vault connector type

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution):
# create a secret
this.connect(
'VAULT',
host='https://my-vault-host:8200',
engine_path='kv',
secret_path='data/my-secret',
data={
'secret-key': 'secret-value',
},
token='my-vault-token',
)

# read a secret
secret_value = this.connect(
'VAULT',
host='https://my-vault-host:8200',
engine_path='kv',
secret_path='data/my-secret',
version=None, # read latest version
token='my-vault-token',
).get('output_value')['result']['data']['data']
assert secret_value == {'secret-key': 'secret-value'}

# destroy all versions of secret
this.connect(
'VAULT',
host='https://my-vault-host:8200',
engine_path='kv',
secret_path='data/my-secret',
mode='delete_metadata',
token='my-vault-token',
)

return this.success('all done')