mirror of
https://github.com/ceph/ceph
synced 2025-01-01 08:32:24 +00:00
rgw/kms/vault - document configuration for new transit logic
Using the new transit logic requires slightly different configuration. additionally there is some backwards compatibility support, which also needed documentation. The existing description of how to configure hashicorp vault to work with ceph was also incomplete. I've fleshed that out a bit, including considerably more information on how to use configure and use the vault secret agent with ceph. Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
This commit is contained in:
Marcus Watts
parent
eceb06022c
commit
a9edb632a5
@ -44,7 +44,7 @@ Some examples below use the Vault command line utility to interact with
|
||||
Vault. You may need to set the following environment variable with the correct
|
||||
address of your Vault server to use this utility::
|
||||
|
||||
export VAULT_ADDR='http://vault-server:8200'
|
||||
export VAULT_ADDR='https://vault-server-fqdn:8200'
|
||||
|
||||
Vault secrets engines
|
||||
=====================
|
||||
@ -88,24 +88,33 @@ Vault supports several authentication mechanisms. Currently, the Object
|
||||
Gateway can be configured to authenticate to Vault using the
|
||||
`Token authentication method`_ or a `Vault agent`_.
|
||||
|
||||
Token authentication
|
||||
--------------------
|
||||
Most tokens in Vault have limited lifetimes and powers. The only
|
||||
sort of Vault token that does not have a lifetime are root tokens.
|
||||
For all other tokens, it is necesary to periodically refresh them,
|
||||
either by performing initial authentication, or by renewing the token.
|
||||
Ceph does not have any logic to perform either operation.
|
||||
The simplest best way to use Vault tokens with ceph is to
|
||||
also run the Vault agent and have it refresh the token file.
|
||||
When the Vault agent is used in this mode, file system permissions
|
||||
can be used to restrict who has the use of tokens.
|
||||
|
||||
.. note:: Token authentication is not recommended for production environments.
|
||||
Instead of having Vault agent refresh a token file, it can be told
|
||||
to act as a proxy server. In this mode, Vault will add a token when
|
||||
necessary and add it to requests passed to it before forwarding them on
|
||||
to the real server. Vault agent will still handle token renewal just
|
||||
as it would when storing a token in the filesystem. In this mode, it
|
||||
is necessary to properly secure the network path rgw uses to reach the
|
||||
Vault agent, such as having the Vault agent listen only to localhost.
|
||||
|
||||
The token authentication method expects a Vault token to be present in a
|
||||
plaintext file. The Object Gateway can be configured to use token authentication
|
||||
with the following settings::
|
||||
Token policies for the object gateway
|
||||
-------------------------------------
|
||||
|
||||
rgw crypt vault auth = token
|
||||
rgw crypt vault token file = /etc/ceph/vault.token
|
||||
rgw crypt vault addr = http://vault-server:8200
|
||||
All Vault tokens have powers as specified by the polices attached
|
||||
to that token. Multiple policies may be associated with one
|
||||
token. You should only use the policy necessary for your
|
||||
configuration.
|
||||
|
||||
For security reasons, the token file must be readable by the Object Gateway
|
||||
only. Also, the Object Gateway should be given a Vault token with a restricted
|
||||
policy that allows it to fetch keyrings from a specific path only. Such a policy
|
||||
can be created in Vault using the command line utility as in the following
|
||||
examples::
|
||||
When using the kv secret engine with the object gateway::
|
||||
|
||||
vault policy write rgw-kv-policy -<<EOF
|
||||
path "secret/data/*" {
|
||||
@ -113,30 +122,109 @@ examples::
|
||||
}
|
||||
EOF
|
||||
|
||||
When using the transit secret engine with the object gateway::
|
||||
|
||||
vault policy write rgw-transit-policy -<<EOF
|
||||
path "transit/keys/*" {
|
||||
capabilities = [ "create", "update" ]
|
||||
denied_parameters = {"exportable" = [], "allow_plaintext_backup" = [] }
|
||||
}
|
||||
|
||||
path "transit/keys/*" {
|
||||
capabilities = ["read", "delete"]
|
||||
}
|
||||
|
||||
path "transit/keys/" {
|
||||
capabilities = ["list"]
|
||||
}
|
||||
|
||||
path "transit/keys/+/rotate" {
|
||||
capabilities = [ "update" ]
|
||||
}
|
||||
|
||||
path "transit/*" {
|
||||
capabilities = [ "update" ]
|
||||
}
|
||||
EOF
|
||||
|
||||
If you had previously used an older version of ceph with the
|
||||
transit secret engine, you might need the following policy::
|
||||
|
||||
vault policy write old-rgw-transit-policy -<<EOF
|
||||
path "transit/export/encryption-key/*" {
|
||||
capabilities = ["read"]
|
||||
}
|
||||
EOF
|
||||
|
||||
Once the policy is created, a token can be generated by a Vault administrator::
|
||||
|
||||
vault token create -policy=rgw-kv-policy
|
||||
Token authentication
|
||||
--------------------
|
||||
|
||||
Sample output::
|
||||
.. note: Never use root tokens with ceph in production environments.
|
||||
|
||||
Key Value
|
||||
--- -----
|
||||
token s.72KuPujbc065OdWB71poOmIq
|
||||
token_accessor jv95ZYBUFv6Ss84x7SCSy6lZ
|
||||
token_duration 768h
|
||||
token_renewable true
|
||||
token_policies ["default" "rgw-kv-policy"]
|
||||
identity_policies []
|
||||
policies ["default" "rgw-kv-policy"]
|
||||
The token authentication method expects a Vault token to be present in a
|
||||
plaintext file. The Object Gateway can be configured to use token authentication
|
||||
with the following settings::
|
||||
|
||||
The actual token, displayed in the ``Value`` column of the first line of the
|
||||
output, must be saved in a file as plaintext.
|
||||
rgw crypt vault auth = token
|
||||
rgw crypt vault token file = /run/.rgw-vault-token
|
||||
rgw crypt vault addr = https://vault-server-fqdn:8200
|
||||
|
||||
Adjust these settinsg to match your configuration.
|
||||
For security reasons, the token file must be readable by the Object Gateway
|
||||
only.
|
||||
|
||||
You might set up vault agent as follows::
|
||||
|
||||
vault write auth/approle/role/rgw-ap \
|
||||
token_policies=rgw-transit-policy,default \
|
||||
token_max_ttl=60m
|
||||
|
||||
Change the policy here to match your configuration.
|
||||
|
||||
Get the role-id::
|
||||
|
||||
vault read auth/approle/role/rgw-ap/role-id -format=json | \
|
||||
jq -r .data.role_id
|
||||
|
||||
Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-role-id
|
||||
|
||||
Get the secret-id::
|
||||
|
||||
vault read auth/approle/role/rgw-ap/role-id -format=json | \
|
||||
jq -r .data.role_id
|
||||
|
||||
Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-secret-id
|
||||
|
||||
Create configuration for the Vault agent, such as::
|
||||
|
||||
pid_file = "/run/rgw-vault-agent-pid"
|
||||
auto_auth {
|
||||
method "AppRole" {
|
||||
mount_path = "auth/approle"
|
||||
config = {
|
||||
role_id_file_path ="/usr/local/etc/vault/.rgw-ap-role-id"
|
||||
secret_id_file_path ="/usr/local/etc/vault/.rgw-ap-secret-id"
|
||||
remove_secret_id_file_after_reading ="false"
|
||||
}
|
||||
}
|
||||
sink "file" {
|
||||
config = {
|
||||
path = "/run/.rgw-vault-token"
|
||||
}
|
||||
}
|
||||
}
|
||||
vault {
|
||||
address = "https://vault-server-fqdn:8200"
|
||||
}
|
||||
|
||||
Then use systemctl or another method of your choice to run
|
||||
a persistent daemon with the following arguments::
|
||||
|
||||
/usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl
|
||||
|
||||
Once the vault agent is running, the token file should be populated
|
||||
with a valid token.
|
||||
|
||||
Vault agent
|
||||
-----------
|
||||
@ -150,7 +238,60 @@ The Object Gateway can be configured to use a Vault agent with the following
|
||||
settings::
|
||||
|
||||
rgw crypt vault auth = agent
|
||||
rgw crypt vault addr = http://localhost:8100
|
||||
rgw crypt vault addr = http://127.0.0.1:8100
|
||||
|
||||
You might set up vault agent as follows::
|
||||
|
||||
vault write auth/approle/role/rgw-ap \
|
||||
token_policies=rgw-transit-policy,default \
|
||||
token_max_ttl=60m
|
||||
|
||||
Change the policy here to match your configuration.
|
||||
|
||||
Get the role-id:
|
||||
vault read auth/approle/role/rgw-ap/role-id -format=json | \
|
||||
jq -r .data.role_id
|
||||
|
||||
Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-role-id
|
||||
|
||||
Get the secret-id:
|
||||
vault read auth/approle/role/rgw-ap/role-id -format=json | \
|
||||
jq -r .data.role_id
|
||||
|
||||
Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-secret-id
|
||||
|
||||
Create configuration for the Vault agent, such as::
|
||||
|
||||
pid_file = "/run/rgw-vault-agent-pid"
|
||||
auto_auth {
|
||||
method "AppRole" {
|
||||
mount_path = "auth/approle"
|
||||
config = {
|
||||
role_id_file_path ="/usr/local/etc/vault/.rgw-ap-role-id"
|
||||
secret_id_file_path ="/usr/local/etc/vault/.rgw-ap-secret-id"
|
||||
remove_secret_id_file_after_reading ="false"
|
||||
}
|
||||
}
|
||||
}
|
||||
cache {
|
||||
use_auto_auth_token = true
|
||||
}
|
||||
listener "tcp" {
|
||||
address = "127.0.0.1:8100"
|
||||
tls_disable = true
|
||||
}
|
||||
vault {
|
||||
address = "https://vault-server-fqdn:8200"
|
||||
}
|
||||
|
||||
Then use systemctl or another method of your choice to run
|
||||
a persistent daemon with the following arguments::
|
||||
|
||||
/usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl
|
||||
|
||||
Once the vault agent is running, you should find it listening
|
||||
to port 8100 on localhost, and you should be able to interact
|
||||
with it using the vault command.
|
||||
|
||||
Vault namespaces
|
||||
================
|
||||
@ -195,28 +336,25 @@ form ``key=<secret key>``.
|
||||
Using the Transit engine
|
||||
------------------------
|
||||
|
||||
Keys created with the Transit engine must be exportable in order to be used for
|
||||
server-side encryption with the Object Gateway. An exportable key can be created
|
||||
with the command line utility as follows::
|
||||
Keys created for use with the Transit engine should no longer be marked
|
||||
exportable. They can be created with::
|
||||
|
||||
vault write -f transit/keys/mybucketkey exportable=true
|
||||
vault write -f transit/keys/mybucketkey
|
||||
|
||||
The command above creates a keyring, which contains a key of type
|
||||
``aes256-gcm96`` by default. To verify that the key was correctly created, use
|
||||
the following command::
|
||||
|
||||
vault read transit/export/encryption-key/mybucketkey/1
|
||||
vault read transit/mybucketkey
|
||||
|
||||
Sample output::
|
||||
|
||||
Key Value
|
||||
--- -----
|
||||
keys map[1:-gbTI9lNpqv/V/2lDcmH2Nq1xKn6FPDWarCmFM2aNsQ=]
|
||||
name mybucketkey
|
||||
type aes256-gcm96
|
||||
|
||||
Note that in order to read the key created with the Transit engine, the full
|
||||
path must be provided including the key version.
|
||||
derived false
|
||||
exportable false
|
||||
name mybucketkey
|
||||
type aes256-gcm96
|
||||
|
||||
Configure the Ceph Object Gateway
|
||||
=================================
|
||||
@ -229,8 +367,8 @@ server-side encryption::
|
||||
Choose the Vault authentication method, e.g.::
|
||||
|
||||
rgw crypt vault auth = token
|
||||
rgw crypt vault token file = /etc/ceph/vault.token
|
||||
rgw crypt vault addr = http://vault-server:8200
|
||||
rgw crypt vault token file = /run/.rgw-vault-token
|
||||
rgw crypt vault addr = https://vault-server-fqdn:8200
|
||||
|
||||
Or::
|
||||
|
||||
@ -255,12 +393,44 @@ restricted to fetch KV keys as follows::
|
||||
|
||||
rgw crypt vault prefix = /v1/secret/data
|
||||
|
||||
Or, in the case of exportable transit keys::
|
||||
Or, when using the transit secret engine::
|
||||
|
||||
rgw crypt vault prefix = /v1/transit/export/encryption-key
|
||||
rgw crypt vault prefix = /v1/transit
|
||||
|
||||
In the example above, the Gateway would only fetch transit encryption keys under
|
||||
``http://vault-server:8200/v1/transit/export/encryption-key``.
|
||||
``https://vault-server:8200/v1/transit``.
|
||||
|
||||
Transit engine compatibility support
|
||||
------------------------------------
|
||||
The transit engine has compatibility support for previous
|
||||
versions of ceph, which used the transit engine as a simple key store.
|
||||
|
||||
There is a a "compat" option which can be given to the transit
|
||||
engine to configure the compatibility support,
|
||||
|
||||
To entirely disable backwards support, use::
|
||||
|
||||
rgw crypt vault secret engine = transit compat=0
|
||||
|
||||
This will be the default in future verisons. and is safe to use
|
||||
for new installs using the current version.
|
||||
|
||||
This is the normal default with the current version::
|
||||
|
||||
rgw crypt vault secret engine = transit compat=1
|
||||
|
||||
This enables the new engine for newly created objects,
|
||||
but still allows the old engine to be used for old objects.
|
||||
In order to access old and new objects, the vault token given
|
||||
to ceph must have both the old and new transit policies.
|
||||
|
||||
To force use of only the old engine, use::
|
||||
|
||||
rgw crypt vault secret engine = transit compat=2
|
||||
|
||||
This mode is automatically selected if the vault prefix
|
||||
ends in export/encryption-key, which was the previously
|
||||
documented setting.
|
||||
|
||||
Upload object
|
||||
=============
|
||||
@ -270,9 +440,9 @@ As an example, for the kv engine, using the AWS command-line client::
|
||||
|
||||
aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey
|
||||
|
||||
As an example, for the transit engine, using the AWS command-line client::
|
||||
As an example, for the transit engine (new flavor), using the AWS command-line client::
|
||||
|
||||
aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id mybucketkey/1
|
||||
aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id mybucketkey
|
||||
|
||||
The Object Gateway will fetch the key from Vault, encrypt the object and store
|
||||
it in the bucket. Any request to download the object will make the Gateway
|
||||
@ -286,9 +456,9 @@ In the kv engine example above, the Gateway would fetch the secret from::
|
||||
|
||||
http://vaultserver:8200/v1/secret/data/myproject/mybucketkey
|
||||
|
||||
In the transit engine example above, the Gateway would fetch the secret from::
|
||||
In the transit engine example above, the Gateway would encrypt the secret using this key::
|
||||
|
||||
http://vaultserver:8200/v1/transit/export/encryption-key/mybucketkey/1
|
||||
http://vaultserver:8200/v1/transit/mybucketkey
|
||||
|
||||
.. _Server-Side Encryption: ../encryption
|
||||
.. _Vault: https://www.vaultproject.io/docs/
|
||||
|
||||
Loading…
Reference in New Issue
Block a user