mirror of
https://github.com/ceph/ceph
synced 2025-01-11 21:50:26 +00:00
mgr/ansible: TLS Mutual Authentication
- Changes needed to allow Ansible Orchestrator to use the new authentication strategy used in Ansible Runner Service - Changes to propagate Ansible playbook errors to the completion result Addressed changes suggested by the team - Certificate and key are stored now in the mon KV store - Option server_url is now server_location - Using manager Options to have a better mgmt of MODULE_OPTIONS - Added verbosity to status command to show problems connecting with external orchestrator - lint problems fixed Addressed changes suggested by @sebastian-philipp - Improved messages and documentation Fix error in documentation - Fix error in ansible documentation - Added examples in orchestrator-cli documentation Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
This commit is contained in:
Juan Miguel Olmo Martínez
parent
ff5f4a57eb
commit
4c6a1c6c68
@ -5,14 +5,14 @@
|
||||
Ansible Orchestrator
|
||||
====================
|
||||
|
||||
This module is a :ref:`Ceph orchestrator <orchestrator-modules>` module that uses `Ansible Runner Service <https://github.com/pcuzner/ansible-runner-service>`_ (a RESTful API server) to execute Ansible playbooks in order to satisfy the different operations supported.
|
||||
This module is a :ref:`Ceph orchestrator <orchestrator-modules>` module that uses `Ansible Runner Service <https://github.com/ansible/ansible-runner-service>`_ (a RESTful API server) to execute Ansible playbooks in order to satisfy the different operations supported.
|
||||
|
||||
These operations basically (and for the moment) are:
|
||||
|
||||
- Get an inventory of the Ceph cluster nodes and all the storage devices present in each node
|
||||
- Hosts management
|
||||
- Create/remove OSD's
|
||||
- ...
|
||||
- ...
|
||||
|
||||
|
||||
Usage
|
||||
=====
|
||||
@ -41,25 +41,58 @@ Enable the Ansible orchestrator module and use it with the :ref:`CLI <orchestrat
|
||||
Configuration
|
||||
=============
|
||||
|
||||
The external Ansible Runner Service uses TLS mutual authentication to allow clients to use the API.
|
||||
A client certificate and a key files should be provided by the Administrator of the Ansible Runner Service for each manager node.
|
||||
This files should be copied in each of the manager nodes with read access for the ceph user.
|
||||
The destination folder for this files and the name of the files must be the same always in all the manager nodes,
|
||||
althought the certificate/key content of this files logically will be different in each node.
|
||||
|
||||
Configuration must be set once the module is enabled by first time.
|
||||
|
||||
This can be done in one monitor node via the configuration key facility on a
|
||||
cluster-wide level (so they apply to all manager instances) as follows::
|
||||
cluster-wide level (so they apply to all manager instances) as follows:
|
||||
|
||||
In first place, configure the Ansible Runner Service client certificate and key:
|
||||
|
||||
::
|
||||
|
||||
If the provided client certificate is usable for all servers, apply it using:
|
||||
# ceph ansible set-ssl-certificate -i <location_of_the_crt_file>
|
||||
# ceph ansible set-ssl-certificate-key -i <location_of_the_key_file>
|
||||
|
||||
|
||||
::
|
||||
|
||||
If the client certificate provided is for an especific manager server use:
|
||||
# ceph ansible set-ssl-certificate <server> -i <location_of_the_crt_file>
|
||||
# ceph ansible set-ssl-certificate-key <server> -i <location_of_the_key_file>
|
||||
|
||||
|
||||
|
||||
After setting the client certificate and key files, finish the configuration as follows:
|
||||
|
||||
::
|
||||
|
||||
# ceph config set mgr mgr/ansible/server_location <ip_address/server_name>:<port>
|
||||
# ceph config set mgr mgr/ansible/verify_server <False|True>
|
||||
# ceph config set mgr mgr/ansible/ca_bundle <path_to_ca_bundle_file>
|
||||
|
||||
|
||||
# ceph config set mgr mgr/ansible/server_addr <ip_address/server_name>
|
||||
# ceph config set mgr mgr/ansible/server_port <port>
|
||||
# ceph config set mgr mgr/ansible/username <username>
|
||||
# ceph config set mgr mgr/ansible/password <password>
|
||||
# ceph config set mgr mgr/ansible/verify_server <verify_server_value>
|
||||
|
||||
Where:
|
||||
|
||||
* <ip_address/server_name>: Is the ip address/hostname of the server where the Ansible Runner Service is available.
|
||||
* <port>: The port number where the Ansible Runner Service is listening
|
||||
* <username>: The username of one authorized user in the Ansible Runner Service
|
||||
* <password>: The password of the authorized user.
|
||||
* <verify_server_value>: Either a boolean, in which case it controls whether the server's TLS certificate is verified, or a string, in which case it must be a path to a CA bundle to use in the verification. Defaults to ``True``.
|
||||
* <verify_server_value>: boolean, it controls whether the Ansible Runner Service server's TLS certificate is verified. Defaults to ``True``.
|
||||
* <path_to_ca_bundle_file>: Path to a CA bundle to use in the verification.
|
||||
|
||||
In order to check that everything is OK, use the "status" orchestrator command.
|
||||
|
||||
# ceph orchestrator status
|
||||
Backend: ansible
|
||||
Available: True
|
||||
|
||||
Any kind of problem connecting with the external Ansible Runner Service will be reported using this command.
|
||||
|
||||
|
||||
Debugging
|
||||
@ -95,27 +128,5 @@ And use the "active" manager node: ( "ceph -s" command in one monitor give you t
|
||||
Operations
|
||||
==========
|
||||
|
||||
**Inventory:**
|
||||
|
||||
Get the list of storage devices installed in all the cluster nodes. The output format is::
|
||||
|
||||
[host:
|
||||
device_name (type_of_device , size_in_bytes)]
|
||||
|
||||
Example::
|
||||
|
||||
[root@mon0 ~]# ceph orchestrator device ls
|
||||
192.168.121.160:
|
||||
vda (hdd, 44023414784b)
|
||||
sda (hdd, 53687091200b)
|
||||
sdb (hdd, 53687091200b)
|
||||
sdc (hdd, 53687091200b)
|
||||
192.168.121.36:
|
||||
vda (hdd, 44023414784b)
|
||||
192.168.121.201:
|
||||
vda (hdd, 44023414784b)
|
||||
192.168.121.70:
|
||||
vda (hdd, 44023414784b)
|
||||
sda (hdd, 53687091200b)
|
||||
sdb (hdd, 53687091200b)
|
||||
sdc (hdd, 53687091200b)
|
||||
To see the complete list of operations, use:
|
||||
:ref:`CLI <orchestrator-cli-module>`
|
||||
|
||||
@ -113,6 +113,22 @@ filtered to a particular node:
|
||||
|
||||
ceph orchestrator device ls [--host=...] [--refresh]
|
||||
|
||||
Example::
|
||||
|
||||
# ceph orchestrator device ls
|
||||
Host 192.168.121.206:
|
||||
Device Path Type Size Rotates Available Model
|
||||
/dev/sdb hdd 50.0G True True ATA/QEMU HARDDISK
|
||||
/dev/sda hdd 50.0G True False ATA/QEMU HARDDISK
|
||||
|
||||
Host 192.168.121.181:
|
||||
Device Path Type Size Rotates Available Model
|
||||
/dev/sdb hdd 50.0G True True ATA/QEMU HARDDISK
|
||||
/dev/sda hdd 50.0G True False ATA/QEMU HARDDISK
|
||||
|
||||
.. note::
|
||||
Output form Ansible orchestrator
|
||||
|
||||
Create OSDs
|
||||
^^^^^^^^^^^
|
||||
|
||||
@ -126,6 +142,13 @@ The output of ``osd create`` is not specified and may vary between orchestrator
|
||||
|
||||
Where ``drive.group.json`` is a JSON file containing the fields defined in :class:`orchestrator.DriveGroupSpec`
|
||||
|
||||
Example::
|
||||
|
||||
# ceph orchestrator osd create 192.168.121.206:/dev/sdc
|
||||
{"status": "OK", "msg": "", "data": {"event": "playbook_on_stats", "uuid": "7082f3ba-f5b7-4b7c-9477-e74ca918afcb", "stdout": "\r\nPLAY RECAP *********************************************************************\r\n192.168.121.206 : ok=96 changed=3 unreachable=0 failed=0 \r\n", "counter": 932, "pid": 10294, "created": "2019-05-28T22:22:58.527821", "end_line": 1170, "runner_ident": "083cad3c-8197-11e9-b07a-2016b900e38f", "start_line": 1166, "event_data": {"ignored": 0, "skipped": {"192.168.121.206": 186}, "ok": {"192.168.121.206": 96}, "artifact_data": {}, "rescued": 0, "changed": {"192.168.121.206": 3}, "pid": 10294, "dark": {}, "playbook_uuid": "409364a6-9d49-4e44-8b7b-c28e5b3adf89", "playbook": "add-osd.yml", "failures": {}, "processed": {"192.168.121.206": 1}}, "parent_uuid": "409364a6-9d49-4e44-8b7b-c28e5b3adf89"}}
|
||||
|
||||
.. note::
|
||||
Output form Ansible orchestrator
|
||||
|
||||
Decommission an OSD
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
@ -136,6 +159,13 @@ Decommission an OSD
|
||||
Removes one or more OSDs from the cluster and the host, if the OSDs are marked as
|
||||
``destroyed``.
|
||||
|
||||
Example::
|
||||
|
||||
# ceph orchestrator osd rm 4
|
||||
{"status": "OK", "msg": "", "data": {"event": "playbook_on_stats", "uuid": "1a16e631-906d-48e0-9e24-fa7eb593cc0a", "stdout": "\r\nPLAY RECAP *********************************************************************\r\n192.168.121.158 : ok=2 changed=0 unreachable=0 failed=0 \r\n192.168.121.181 : ok=2 changed=0 unreachable=0 failed=0 \r\n192.168.121.206 : ok=2 changed=0 unreachable=0 failed=0 \r\nlocalhost : ok=31 changed=8 unreachable=0 failed=0 \r\n", "counter": 240, "pid": 10948, "created": "2019-05-28T22:26:09.264012", "end_line": 308, "runner_ident": "8c093db0-8197-11e9-b07a-2016b900e38f", "start_line": 301, "event_data": {"ignored": 0, "skipped": {"localhost": 37}, "ok": {"192.168.121.181": 2, "192.168.121.158": 2, "192.168.121.206": 2, "localhost": 31}, "artifact_data": {}, "rescued": 0, "changed": {"localhost": 8}, "pid": 10948, "dark": {}, "playbook_uuid": "a12ec40e-bce9-4bc9-b09e-2d8f76a5be02", "playbook": "shrink-osd.yml", "failures": {}, "processed": {"192.168.121.181": 1, "192.168.121.158": 1, "192.168.121.206": 1, "localhost": 1}}, "parent_uuid": "a12ec40e-bce9-4bc9-b09e-2d8f76a5be02"}}
|
||||
|
||||
.. note::
|
||||
Output form Ansible orchestrator
|
||||
|
||||
..
|
||||
Blink Device Lights
|
||||
@ -245,12 +275,12 @@ This is an overview of the current implementation status of the orchestrators.
|
||||
=================================== ========= ====== ========= =====
|
||||
Command Ansible Rook DeepSea SSH
|
||||
=================================== ========= ====== ========= =====
|
||||
host add ⚪ ⚪ ⚪ ✔️
|
||||
host ls ⚪ ⚪ ⚪ ✔️
|
||||
host rm ⚪ ⚪ ⚪ ✔️
|
||||
host add ✔️ ⚪ ⚪ ✔️
|
||||
host ls ✔️ ⚪ ⚪ ✔️
|
||||
host rm ✔️ ⚪ ⚪ ✔️
|
||||
mgr update ⚪ ⚪ ⚪ ✔️
|
||||
mon update ⚪ ✔️ ⚪ ✔️
|
||||
osd create ✔️ ✔️ ⚪ ✔️
|
||||
osd create ✔️ ✔️ ⚪ ✔️
|
||||
osd device {ident,fault}-{on,off} ⚪ ⚪ ⚪ ⚪
|
||||
osd rm ✔️ ⚪ ⚪ ⚪
|
||||
device {ident,fault}-(on,off} ⚪ ⚪ ⚪ ⚪
|
||||
|
||||
@ -1,19 +1,21 @@
|
||||
"""
|
||||
Client module to interact with the Ansible Runner Service
|
||||
"""
|
||||
import requests
|
||||
|
||||
import json
|
||||
import re
|
||||
from functools import wraps
|
||||
|
||||
import requests
|
||||
|
||||
# Ansible Runner service API endpoints
|
||||
API_URL = "api"
|
||||
LOGIN_URL = "api/v1/login"
|
||||
PLAYBOOK_EXEC_URL = "api/v1/playbooks"
|
||||
PLAYBOOK_EVENTS = "api/v1/jobs/%s/events"
|
||||
EVENT_DATA_URL = "api/v1/jobs/%s/events/%s"
|
||||
|
||||
class AnsibleRunnerServiceError(Exception):
|
||||
"""Generic Ansible Runner Service Exception"""
|
||||
pass
|
||||
|
||||
def handle_requests_exceptions(func):
|
||||
@ -21,10 +23,11 @@ def handle_requests_exceptions(func):
|
||||
"""
|
||||
@wraps(func)
|
||||
def inner(*args, **kwargs):
|
||||
"""Generic error mgmt decorator"""
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except requests.exceptions.RequestException as ex:
|
||||
raise AnsibleRunnerServiceError(str(ex))
|
||||
except (requests.exceptions.RequestException, IOError) as ex:
|
||||
raise AnsibleRunnerServiceError(str(ex))
|
||||
return inner
|
||||
|
||||
class ExecutionStatusCode(object):
|
||||
@ -73,8 +76,8 @@ class PlayBookExecution(object):
|
||||
|
||||
try:
|
||||
response = self.rest_client.http_post(endpoint,
|
||||
self.params,
|
||||
self.querystr_dict)
|
||||
self.params,
|
||||
self.querystr_dict)
|
||||
except AnsibleRunnerServiceError:
|
||||
self.log.exception("Error launching playbook <%s>", self.playbook)
|
||||
raise
|
||||
@ -128,10 +131,11 @@ class PlayBookExecution(object):
|
||||
self.log.info("Requested playbook execution status is: %s", status_value)
|
||||
return status_value
|
||||
|
||||
def get_result(self, event_filter=""):
|
||||
def get_result(self, event_filter):
|
||||
"""Get the data of the events filtered by a task pattern and
|
||||
a event filter
|
||||
|
||||
@event_filter: list of 0..N event names items
|
||||
@returns: the events that matches with the patterns provided
|
||||
"""
|
||||
response = None
|
||||
@ -148,16 +152,20 @@ class PlayBookExecution(object):
|
||||
else:
|
||||
events = json.loads(response.text)["data"]["events"]
|
||||
|
||||
# Filter by task
|
||||
if self.result_task_pattern:
|
||||
result_events = {event:data for event,data in events.items()
|
||||
if "task" in data and
|
||||
re.match(self.result_task_pattern, data["task"])}
|
||||
result_events = {event:data for event, data in events.items()
|
||||
if "task" in data and
|
||||
re.match(self.result_task_pattern, data["task"])}
|
||||
else:
|
||||
result_events = events
|
||||
|
||||
# Filter by event
|
||||
if event_filter:
|
||||
result_events = {event:data for event,data in result_events.items()
|
||||
if re.match(event_filter, data['event'])}
|
||||
type_of_events = "|".join(event_filter)
|
||||
|
||||
result_events = {event:data for event, data in result_events.items()
|
||||
if re.match(type_of_events, data['event'])}
|
||||
|
||||
self.log.info("Requested playbook result is: %s", json.dumps(result_events))
|
||||
return result_events
|
||||
@ -167,68 +175,41 @@ class Client(object):
|
||||
and execute easily playbooks
|
||||
"""
|
||||
|
||||
def __init__(self, server_url, user, password, verify_server, logger):
|
||||
def __init__(self, server_url, verify_server, ca_bundle, client_cert,
|
||||
client_key, logger):
|
||||
"""Provide an https client to make easy interact with the Ansible
|
||||
Runner Service"
|
||||
|
||||
:param servers_url: The base URL >server>:<port> of the Ansible Runner Service
|
||||
:param user: Username of the authorized user
|
||||
:param password: Password of the authorized user
|
||||
:param verify_server: Either a boolean, in which case it controls whether we verify
|
||||
the server's TLS certificate, or a string, in which case it must be a path
|
||||
to a CA bundle to use. Defaults to ``True``.
|
||||
:param server_url: The base URL >server>:<port> of the Ansible Runner
|
||||
Service
|
||||
:param verify_server: A boolean to specify if server authentity should
|
||||
be checked or not. (True by default)
|
||||
:param ca_bundle: If provided, an alternative Cert. Auth. bundle file
|
||||
will be used as source for checking the authentity of
|
||||
the Ansible Runner Service
|
||||
:param client_cert: Path to Ansible Runner Service client certificate
|
||||
file
|
||||
:param client_key: Path to Ansible Runner Service client certificate key
|
||||
file
|
||||
:param logger: Log file
|
||||
"""
|
||||
self.server_url = server_url
|
||||
self.user = user
|
||||
self.password = password
|
||||
self.log = logger
|
||||
self.auth = (self.user, self.password)
|
||||
if not verify_server:
|
||||
self.verify_server = True
|
||||
elif verify_server.lower().strip() == 'false':
|
||||
self.verify_server = False
|
||||
else:
|
||||
self.verify_server = verify_server
|
||||
self.client_cert = (client_cert, client_key)
|
||||
|
||||
# Once authenticated this token will be used in all the requests
|
||||
self.token = ""
|
||||
# used to provide the "verify" parameter in requests
|
||||
# a boolean that sometimes contains a string :-(
|
||||
self.verify_server = verify_server
|
||||
if ca_bundle: # This intentionallly overwrites
|
||||
self.verify_server = ca_bundle
|
||||
|
||||
self.server_url = "https://{0}".format(self.server_url)
|
||||
|
||||
# Log in the server and get a token
|
||||
self.login()
|
||||
|
||||
@handle_requests_exceptions
|
||||
def login(self):
|
||||
""" Login with user credentials to obtain a valid token
|
||||
"""
|
||||
|
||||
the_url = "%s/%s" % (self.server_url, LOGIN_URL)
|
||||
response = requests.get(the_url,
|
||||
auth = self.auth,
|
||||
verify = self.verify_server)
|
||||
|
||||
if response.status_code != requests.codes.ok:
|
||||
self.log.error("login error <<%s>> (%s):%s",
|
||||
the_url, response.status_code, response.text)
|
||||
else:
|
||||
self.log.info("login succesful <<%s>> (%s):%s",
|
||||
the_url, response.status_code, response.text)
|
||||
|
||||
if response:
|
||||
self.token = json.loads(response.text)["data"]["token"]
|
||||
self.log.info("Connection with Ansible Runner Service is operative")
|
||||
|
||||
@handle_requests_exceptions
|
||||
def is_operative(self):
|
||||
"""Indicates if the connection with the Ansible runner Server is ok
|
||||
"""
|
||||
|
||||
# No Token... this means we haven't used yet the service.
|
||||
if not self.token:
|
||||
return False
|
||||
|
||||
# Check the service
|
||||
response = self.http_get(API_URL)
|
||||
|
||||
@ -247,17 +228,19 @@ class Client(object):
|
||||
"""
|
||||
|
||||
the_url = "%s/%s" % (self.server_url, endpoint)
|
||||
|
||||
response = requests.get(the_url,
|
||||
verify = self.verify_server,
|
||||
headers = {"Authorization": self.token})
|
||||
verify=self.verify_server,
|
||||
cert=self.client_cert,
|
||||
headers={})
|
||||
|
||||
if response.status_code != requests.codes.ok:
|
||||
self.log.error("http GET %s <--> (%s - %s)\n%s",
|
||||
the_url, response.status_code, response.reason,
|
||||
response.text)
|
||||
the_url, response.status_code, response.reason,
|
||||
response.text)
|
||||
else:
|
||||
self.log.info("http GET %s <--> (%s - %s)",
|
||||
the_url, response.status_code, response.text)
|
||||
the_url, response.status_code, response.text)
|
||||
|
||||
return response
|
||||
|
||||
@ -274,19 +257,19 @@ class Client(object):
|
||||
|
||||
the_url = "%s/%s" % (self.server_url, endpoint)
|
||||
response = requests.post(the_url,
|
||||
verify = self.verify_server,
|
||||
headers = {"Authorization": self.token,
|
||||
"Content-type": "application/json"},
|
||||
json = payload,
|
||||
params = params_dict)
|
||||
verify=self.verify_server,
|
||||
cert=self.client_cert,
|
||||
headers={"Content-type": "application/json"},
|
||||
json=payload,
|
||||
params=params_dict)
|
||||
|
||||
if response.status_code != requests.codes.ok:
|
||||
self.log.error("http POST %s [%s] <--> (%s - %s:%s)\n",
|
||||
the_url, payload, response.status_code,
|
||||
response.reason, response.text)
|
||||
the_url, payload, response.status_code,
|
||||
response.reason, response.text)
|
||||
else:
|
||||
self.log.info("http POST %s <--> (%s - %s)",
|
||||
the_url, response.status_code, response.text)
|
||||
the_url, response.status_code, response.text)
|
||||
|
||||
return response
|
||||
|
||||
@ -301,16 +284,17 @@ class Client(object):
|
||||
|
||||
the_url = "%s/%s" % (self.server_url, endpoint)
|
||||
response = requests.delete(the_url,
|
||||
verify = self.verify_server,
|
||||
headers = {"Authorization": self.token})
|
||||
verify=self.verify_server,
|
||||
cert=self.client_cert,
|
||||
headers={})
|
||||
|
||||
if response.status_code != requests.codes.ok:
|
||||
self.log.error("http DELETE %s <--> (%s - %s)\n%s",
|
||||
the_url, response.status_code, response.reason,
|
||||
response.text)
|
||||
the_url, response.status_code, response.reason,
|
||||
response.text)
|
||||
else:
|
||||
self.log.info("http DELETE %s <--> (%s - %s)",
|
||||
the_url, response.status_code, response.text)
|
||||
the_url, response.status_code, response.text)
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@ -7,9 +7,13 @@ The external Orchestrator is the Ansible runner service (RESTful https service)
|
||||
# pylint: disable=abstract-method, no-member, bad-continuation
|
||||
|
||||
import json
|
||||
import os
|
||||
import errno
|
||||
import tempfile
|
||||
import requests
|
||||
from OpenSSL import crypto, SSL
|
||||
|
||||
from mgr_module import MgrModule
|
||||
from mgr_module import MgrModule, Option, CLIWriteCommand
|
||||
import orchestrator
|
||||
|
||||
from .ansible_runner_svc import Client, PlayBookExecution, ExecutionStatusCode,\
|
||||
@ -77,8 +81,8 @@ class AnsibleReadOperation(orchestrator.ReadCompletion):
|
||||
# Logger
|
||||
self.log = logger
|
||||
|
||||
# OutputWizard object used to process the result
|
||||
self.output_wizard = None
|
||||
def __str__(self):
|
||||
return "Playbook {playbook_name}".format(playbook_name=self.playbook)
|
||||
|
||||
@property
|
||||
def is_complete(self):
|
||||
@ -184,7 +188,7 @@ class PlaybookOperation(AnsibleReadOperation):
|
||||
self.playbook = playbook
|
||||
|
||||
# An aditional filter of result events based in the event
|
||||
self.event_filter = ""
|
||||
self.event_filter_list = [""]
|
||||
|
||||
# Playbook execution object
|
||||
self.pb_execution = PlayBookExecution(client,
|
||||
@ -237,8 +241,15 @@ class PlaybookOperation(AnsibleReadOperation):
|
||||
|
||||
processed_result = []
|
||||
|
||||
if self._is_complete:
|
||||
raw_result = self.pb_execution.get_result(self.event_filter)
|
||||
if self._is_errored:
|
||||
processed_result = self.pb_execution.get_result(["runner_on_failed",
|
||||
"runner_on_unreachable",
|
||||
"runner_on_no_hosts",
|
||||
"runner_on_async_failed",
|
||||
"runner_item_on_failed"])
|
||||
|
||||
elif self._is_complete:
|
||||
raw_result = self.pb_execution.get_result(self.event_filter_list)
|
||||
|
||||
if self.output_wizard:
|
||||
processed_result = self.output_wizard.process(self.pb_execution.play_uuid,
|
||||
@ -392,10 +403,12 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
"""
|
||||
|
||||
MODULE_OPTIONS = [
|
||||
{'name': 'server_url'},
|
||||
{'name': 'username'},
|
||||
{'name': 'password'},
|
||||
{'name': 'verify_server'} # Check server identity (Boolean/path to CA bundle)
|
||||
# url:port of the Ansible Runner Service
|
||||
Option(name="server_location", type="str", default=""),
|
||||
# Check server identity (True by default)
|
||||
Option(name="verify_server", type="bool", default=True),
|
||||
# Path to an alternative CA bundle
|
||||
Option(name="ca_bundle", type="str", default="")
|
||||
]
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
@ -407,11 +420,38 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
|
||||
self.ar_client = None
|
||||
|
||||
# TLS certificate and key file names used to connect with the external
|
||||
# Ansible Runner Service
|
||||
self.client_cert_fname = ""
|
||||
self.client_key_fname = ""
|
||||
|
||||
# used to provide more verbose explanation of errors in status method
|
||||
self.status_message = ""
|
||||
|
||||
def available(self):
|
||||
""" Check if Ansible Runner service is working
|
||||
"""
|
||||
# TODO
|
||||
return (True, "Everything ready")
|
||||
available = False
|
||||
msg = ""
|
||||
try:
|
||||
|
||||
if self.ar_client:
|
||||
available = self.ar_client.is_operative()
|
||||
if not available:
|
||||
msg = "No response from Ansible Runner Service"
|
||||
else:
|
||||
msg = "Not possible to initialize connection with Ansible "\
|
||||
"Runner service."
|
||||
|
||||
except AnsibleRunnerServiceError as ex:
|
||||
available = False
|
||||
msg = str(ex)
|
||||
|
||||
# Add more details to the detected problem
|
||||
if self.status_message:
|
||||
msg = "{}:\n{}".format(msg, self.status_message)
|
||||
|
||||
return (available, msg)
|
||||
|
||||
def wait(self, completions):
|
||||
"""Given a list of Completion instances, progress any which are
|
||||
@ -438,16 +478,19 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
"""
|
||||
self.log.info("Starting Ansible Orchestrator module ...")
|
||||
|
||||
# Verify config options (Just that settings are available)
|
||||
self.verify_config()
|
||||
|
||||
# Ansible runner service client
|
||||
try:
|
||||
self.ar_client = Client(server_url=self.get_module_option('server_url', ''),
|
||||
user=self.get_module_option('username', ''),
|
||||
password=self.get_module_option('password', ''),
|
||||
verify_server=self.get_module_option('verify_server', True),
|
||||
logger=self.log)
|
||||
# Verify config options and client certificates
|
||||
self.verify_config()
|
||||
|
||||
# Ansible runner service client
|
||||
self.ar_client = Client(
|
||||
server_url=self.get_module_option('server_location', ''),
|
||||
verify_server=self.get_module_option('verify_server', True),
|
||||
ca_bundle=self.get_module_option('ca_bundle', ''),
|
||||
client_cert=self.client_cert_fname,
|
||||
client_key=self.client_key_fname,
|
||||
logger=self.log)
|
||||
|
||||
except AnsibleRunnerServiceError:
|
||||
self.log.exception("Ansible Runner Service not available. "
|
||||
"Check external server status/TLS identity or "
|
||||
@ -482,7 +525,7 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
# Assign the process_output function
|
||||
playbook_operation.output_wizard = ProcessInventory(self.ar_client,
|
||||
self.log)
|
||||
playbook_operation.event_filter = "runner_on_ok"
|
||||
playbook_operation.event_filter_list = ["runner_on_ok"]
|
||||
|
||||
# Execute the playbook to obtain data
|
||||
self._launch_operation(playbook_operation)
|
||||
@ -514,7 +557,7 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
# Filter to get the result
|
||||
playbook_operation.output_wizard = ProcessPlaybookResult(self.ar_client,
|
||||
self.log)
|
||||
playbook_operation.event_filter = "playbook_on_stats"
|
||||
playbook_operation.event_filter_list = ["playbook_on_stats"]
|
||||
|
||||
# Execute the playbook
|
||||
self._launch_operation(playbook_operation)
|
||||
@ -540,7 +583,8 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
# Filter to get the result
|
||||
playbook_operation.output_wizard = ProcessPlaybookResult(self.ar_client,
|
||||
self.log)
|
||||
playbook_operation.event_filter = "playbook_on_stats"
|
||||
playbook_operation.event_filter_list = ["playbook_on_stats"]
|
||||
|
||||
|
||||
# Execute the playbook
|
||||
self._launch_operation(playbook_operation)
|
||||
@ -579,13 +623,14 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
add_url = URL_ADD_RM_HOSTS.format(host_name=host,
|
||||
inventory_group=ORCHESTRATOR_GROUP)
|
||||
|
||||
operations = [HttpOperation(add_url, "post")]
|
||||
operations = [HttpOperation(add_url, "post", "", None)]
|
||||
|
||||
except AnsibleRunnerServiceError as ex:
|
||||
# Problems with the external orchestrator.
|
||||
# Prepare the operation to return the error in a Completion object.
|
||||
self.log.exception("Error checking <orchestrator> group: %s", ex)
|
||||
operations = [HttpOperation(url_group, "post")]
|
||||
self.log.exception("Error checking <orchestrator> group: %s",
|
||||
str(ex))
|
||||
operations = [HttpOperation(url_group, "post", "", None)]
|
||||
|
||||
return ARSChangeOperation(self.ar_client, self.log, operations)
|
||||
|
||||
@ -642,49 +687,106 @@ class Module(MgrModule, orchestrator.Orchestrator):
|
||||
self.all_completions.append(ansible_operation)
|
||||
|
||||
def verify_config(self):
|
||||
""" Verify configuration options for the Ansible orchestrator module
|
||||
"""Verify mandatory settings for the module and provide help to
|
||||
configure properly the orchestrator
|
||||
"""
|
||||
client_msg = ""
|
||||
|
||||
if not self.get_module_option('server_url', ''):
|
||||
msg = "No Ansible Runner Service base URL <server_name>:<port>." \
|
||||
"Try 'ceph config set mgr mgr/{0}/server_url " \
|
||||
the_crt = None
|
||||
the_key = None
|
||||
|
||||
# Retrieve TLS content to use and check them
|
||||
# First try to get certiticate and key content for this manager instance
|
||||
# ex: mgr/ansible/mgr0/[crt/key]
|
||||
self.log.info("Tying to use configured specific certificate and key"
|
||||
"files for this server")
|
||||
the_crt = self.get_store("{}/{}".format(self.get_mgr_id(), "crt"))
|
||||
the_key = self.get_store("{}/{}".format(self.get_mgr_id(), "key"))
|
||||
if the_crt is None or the_key is None:
|
||||
# If not possible... try to get generic certificates and key content
|
||||
# ex: mgr/ansible/[crt/key]
|
||||
self.log.warning("Specific tls files for this manager not "\
|
||||
"configured, trying to use generic files")
|
||||
the_crt = self.get_store("crt")
|
||||
the_key = self.get_store("key")
|
||||
|
||||
if the_crt is None or the_key is None:
|
||||
self.status_message = "No client certificate configured. Please "\
|
||||
"set Ansible Runner Service client "\
|
||||
"certificate and key:\n"\
|
||||
"ceph ansible set-ssl-certificate-"\
|
||||
"{key,certificate} -i <file>"
|
||||
self.log.error(self.status_message)
|
||||
return
|
||||
|
||||
# generate certificate temp files
|
||||
self.client_cert_fname = generate_temp_file("crt", the_crt)
|
||||
self.client_key_fname = generate_temp_file("key", the_key)
|
||||
|
||||
self.status_message = verify_tls_files(self.client_cert_fname,
|
||||
self.client_key_fname)
|
||||
|
||||
if self.status_message:
|
||||
self.log.error(self.status_message)
|
||||
return
|
||||
|
||||
# Check module options
|
||||
if not self.get_module_option("server_location", ""):
|
||||
self.status_message = "No Ansible Runner Service base URL "\
|
||||
"<server_name>:<port>."\
|
||||
"Try 'ceph config set mgr mgr/{0}/server_location "\
|
||||
"<server name/ip>:<port>'".format(self.module_name)
|
||||
self.log.error(msg)
|
||||
client_msg += msg
|
||||
self.log.error(self.status_message)
|
||||
return
|
||||
|
||||
if not self.get_module_option('username', ''):
|
||||
msg = "No Ansible Runner Service user. " \
|
||||
"Try 'ceph config set mgr mgr/{0}/username " \
|
||||
"<string value>'".format(self.module_name)
|
||||
self.log.error(msg)
|
||||
client_msg += msg
|
||||
|
||||
if not self.get_module_option('password', ''):
|
||||
msg = "No Ansible Runner Service User password. " \
|
||||
"Try 'ceph config set mgr mgr/{0}/password " \
|
||||
"<string value>'".format(self.module_name)
|
||||
self.log.error(msg)
|
||||
client_msg += msg
|
||||
|
||||
if not self.get_module_option('verify_server', ''):
|
||||
msg = "TLS server identity verification is enabled by default." \
|
||||
"Use 'ceph config set mgr mgr/{0}/verify_server False' " \
|
||||
"to disable it. Use 'ceph config set mgr mgr/{0}/verify_server " \
|
||||
"<path>' to point the CA bundle path used for " \
|
||||
if self.get_module_option("verify_server", True):
|
||||
self.status_message = "TLS server identity verification is enabled"\
|
||||
" by default.Use 'ceph config set mgr mgr/{0}/verify_server False'"\
|
||||
"to disable it.Use 'ceph config set mgr mgr/{0}/ca_bundle <path>'"\
|
||||
"to point an alternative CA bundle path used for TLS server "\
|
||||
"verification".format(self.module_name)
|
||||
self.log.error(msg)
|
||||
client_msg += msg
|
||||
self.log.error(self.status_message)
|
||||
return
|
||||
|
||||
if client_msg:
|
||||
# Raise error
|
||||
# TODO: Use OrchestratorValidationError
|
||||
raise Exception(client_msg)
|
||||
# Everything ok
|
||||
self.status_message = ""
|
||||
|
||||
|
||||
#---------------------------------------------------------------------------
|
||||
# Ansible Orchestrator self-owned commands
|
||||
#---------------------------------------------------------------------------
|
||||
@CLIWriteCommand("ansible set-ssl-certificate",
|
||||
"name=mgr_id,type=CephString,req=false")
|
||||
def set_tls_certificate(self, mgr_id=None, inbuf=None):
|
||||
"""Load tls certificate in mon k-v store
|
||||
"""
|
||||
if inbuf is None:
|
||||
return -errno.EINVAL, \
|
||||
'Please specify the certificate file with "-i" option', ''
|
||||
if mgr_id is not None:
|
||||
self.set_store("{}/crt".format(mgr_id), inbuf)
|
||||
else:
|
||||
self.set_store("crt", inbuf)
|
||||
return 0, "SSL certificate updated", ""
|
||||
|
||||
@CLIWriteCommand("ansible set-ssl-certificate-key",
|
||||
"name=mgr_id,type=CephString,req=false")
|
||||
def set_tls_certificate_key(self, mgr_id=None, inbuf=None):
|
||||
"""Load tls certificate key in mon k-v store
|
||||
"""
|
||||
if inbuf is None:
|
||||
return -errno.EINVAL, \
|
||||
'Please specify the certificate key file with "-i" option', \
|
||||
''
|
||||
if mgr_id is not None:
|
||||
self.set_store("{}/key".format(mgr_id), inbuf)
|
||||
else:
|
||||
self.set_store("key", inbuf)
|
||||
return 0, "SSL certificate key updated", ""
|
||||
|
||||
# Auxiliary functions
|
||||
#==============================================================================
|
||||
|
||||
def dg_2_ansible(drive_group):
|
||||
""" Transform a drive group especification into:
|
||||
|
||||
@ -727,3 +829,76 @@ def dg_2_ansible(drive_group):
|
||||
#osd_spec["osd_objectstore"] = drive_group.objectstore
|
||||
|
||||
return host, osd_spec
|
||||
|
||||
|
||||
def generate_temp_file(key, content):
|
||||
""" Generates a temporal file with the content passed as parameter
|
||||
|
||||
:param key : used to build the temp file name
|
||||
:param content: the content that will be dumped to file
|
||||
:returns : the name of the generated file
|
||||
"""
|
||||
|
||||
fname = ""
|
||||
|
||||
if content is not None:
|
||||
fname = "{}/{}.tmp".format(tempfile.gettempdir(), key)
|
||||
try:
|
||||
if os.path.exists(fname):
|
||||
os.remove(fname)
|
||||
with open(fname, "w") as text_file:
|
||||
text_file.write(content)
|
||||
except IOError as ex:
|
||||
raise AnsibleRunnerServiceError("Cannot store TLS certificate/key"
|
||||
" content: {}".format(str(ex)))
|
||||
|
||||
return fname
|
||||
|
||||
def verify_tls_files(crt_file, key_file):
|
||||
"""Basic checks for TLS certificate and key files
|
||||
|
||||
:crt_file : Name of the certificate file
|
||||
:key_file : name of the certificate public key file
|
||||
|
||||
:returns : String with error description
|
||||
"""
|
||||
|
||||
# Check we have files
|
||||
if not crt_file or not key_file:
|
||||
return "no certificate/key configured"
|
||||
|
||||
if not os.path.isfile(crt_file):
|
||||
return "certificate {} does not exist".format(crt_file)
|
||||
|
||||
if not os.path.isfile(key_file):
|
||||
return "Public key {} does not exist".format(key_file)
|
||||
|
||||
# Do some validations to the private key and certificate:
|
||||
# - Check the type and format
|
||||
# - Check the certificate expiration date
|
||||
# - Check the consistency of the private key
|
||||
# - Check that the private key and certificate match up
|
||||
try:
|
||||
with open(crt_file) as fcrt:
|
||||
x509 = crypto.load_certificate(crypto.FILETYPE_PEM, fcrt.read())
|
||||
if x509.has_expired():
|
||||
return "Certificate {} has been expired".format(crt_file)
|
||||
except (ValueError, crypto.Error) as ex:
|
||||
return "Invalid certificate {}: {}".format(crt_file, str(ex))
|
||||
try:
|
||||
with open(key_file) as fkey:
|
||||
pkey = crypto.load_privatekey(crypto.FILETYPE_PEM, fkey.read())
|
||||
pkey.check()
|
||||
except (ValueError, crypto.Error) as ex:
|
||||
return "Invalid private key {}: {}".format(key_file, str(ex))
|
||||
try:
|
||||
context = SSL.Context(SSL.TLSv1_METHOD)
|
||||
context.use_certificate_file(crt_file, crypto.FILETYPE_PEM)
|
||||
context.use_privatekey_file(key_file, crypto.FILETYPE_PEM)
|
||||
context.check_privatekey()
|
||||
except crypto.Error as ex:
|
||||
return "Private key {} and certificate {} do not match up: {}".format(
|
||||
key_file, crt_file, str(ex))
|
||||
|
||||
# Everything OK
|
||||
return ""
|
||||
|
||||
@ -5,8 +5,6 @@ Output wizards are used to process results in different ways in
|
||||
completion objects
|
||||
"""
|
||||
|
||||
# pylint: disable=bad-continuation
|
||||
|
||||
import json
|
||||
|
||||
|
||||
@ -66,7 +64,7 @@ class ProcessInventory(OutputWizard):
|
||||
for event_key, dummy_data in inventory_events.items():
|
||||
|
||||
event_response = self.ar_client.http_get(EVENT_DATA_URL %
|
||||
(operation_id, event_key))
|
||||
(operation_id, event_key))
|
||||
|
||||
# self.pb_execution.play_uuid
|
||||
|
||||
@ -105,7 +103,7 @@ class ProcessPlaybookResult(OutputWizard):
|
||||
# Loop over the result events and request the data
|
||||
for event_key, dummy_data in inventory_events.items():
|
||||
event_response = self.ar_client.http_get(EVENT_DATA_URL %
|
||||
(operation_id, event_key))
|
||||
(operation_id, event_key))
|
||||
|
||||
result += event_response.text
|
||||
|
||||
|
||||
@ -8,13 +8,11 @@ import requests_mock
|
||||
from requests.exceptions import ConnectionError
|
||||
|
||||
from ..ansible_runner_svc import Client, PlayBookExecution, ExecutionStatusCode, \
|
||||
LOGIN_URL, API_URL, PLAYBOOK_EXEC_URL, \
|
||||
API_URL, PLAYBOOK_EXEC_URL, \
|
||||
PLAYBOOK_EVENTS, AnsibleRunnerServiceError
|
||||
|
||||
|
||||
SERVER_URL = "ars:5001"
|
||||
USER = "admin"
|
||||
PASSWORD = "admin"
|
||||
CERTIFICATE = ""
|
||||
|
||||
# Playbook attributes
|
||||
@ -32,30 +30,11 @@ formatter = logging.Formatter("%(levelname)s - %(message)s")
|
||||
handler.setFormatter(formatter)
|
||||
logger.addHandler(handler)
|
||||
|
||||
|
||||
def mock_login(mock_server):
|
||||
|
||||
the_login_url = "https://%s/%s" % (SERVER_URL,LOGIN_URL)
|
||||
|
||||
mock_server.register_uri("GET",
|
||||
the_login_url,
|
||||
json={"status": "OK",
|
||||
"msg": "Token returned",
|
||||
"data": {"token": "dummy_token"}},
|
||||
status_code=200)
|
||||
|
||||
the_api_url = "https://%s/%s" % (SERVER_URL,API_URL)
|
||||
mock_server.register_uri("GET",
|
||||
the_api_url,
|
||||
text="<!DOCTYPE html>api</html>",
|
||||
status_code=200)
|
||||
|
||||
def mock_get_pb(mock_server, playbook_name, return_code):
|
||||
|
||||
mock_login(mock_server)
|
||||
|
||||
ars_client = Client(SERVER_URL, USER, PASSWORD,
|
||||
CERTIFICATE, logger)
|
||||
ars_client = Client(SERVER_URL, verify_server=False, ca_bundle="",
|
||||
client_cert = "DUMMY_PATH", client_key = "DUMMY_PATH",
|
||||
logger = logger)
|
||||
|
||||
the_pb_url = "https://%s/%s/%s" % (SERVER_URL, PLAYBOOK_EXEC_URL, playbook_name)
|
||||
|
||||
@ -82,34 +61,26 @@ class ARSclientTest(unittest.TestCase):
|
||||
def test_server_not_reachable(self):
|
||||
|
||||
with self.assertRaises(AnsibleRunnerServiceError):
|
||||
ars_client = Client(SERVER_URL, USER, PASSWORD,
|
||||
CERTIFICATE, logger)
|
||||
ars_client = Client(SERVER_URL, verify_server=False, ca_bundle="",
|
||||
client_cert = "DUMMY_PATH", client_key = "DUMMY_PATH",
|
||||
logger = logger)
|
||||
|
||||
def test_server_wrong_USER(self):
|
||||
status = ars_client.is_operative()
|
||||
|
||||
with requests_mock.Mocker() as mock_server:
|
||||
the_login_url = "https://%s/%s" % (SERVER_URL,LOGIN_URL)
|
||||
mock_server.get(the_login_url,
|
||||
json={"status": "NOAUTH",
|
||||
"msg": "Access denied invalid login: unknown USER",
|
||||
"data": {}},
|
||||
status_code=401)
|
||||
|
||||
|
||||
ars_client = Client(SERVER_URL, USER, PASSWORD,
|
||||
CERTIFICATE, logger)
|
||||
|
||||
self.assertFalse(ars_client.is_operative(),
|
||||
"Operative attribute expected to be False")
|
||||
|
||||
def test_server_connection_ok(self):
|
||||
|
||||
with requests_mock.Mocker() as mock_server:
|
||||
|
||||
mock_login(mock_server)
|
||||
ars_client = Client(SERVER_URL, verify_server=False, ca_bundle="",
|
||||
client_cert = "DUMMY_PATH", client_key = "DUMMY_PATH",
|
||||
logger = logger)
|
||||
|
||||
ars_client = Client(SERVER_URL, USER, PASSWORD,
|
||||
CERTIFICATE, logger)
|
||||
the_api_url = "https://%s/%s" % (SERVER_URL,API_URL)
|
||||
mock_server.register_uri("GET",
|
||||
the_api_url,
|
||||
text="<!DOCTYPE html>api</html>",
|
||||
status_code=200)
|
||||
|
||||
self.assertTrue(ars_client.is_operative(),
|
||||
"Operative attribute expected to be True")
|
||||
@ -118,10 +89,9 @@ class ARSclientTest(unittest.TestCase):
|
||||
|
||||
with requests_mock.Mocker() as mock_server:
|
||||
|
||||
mock_login(mock_server)
|
||||
|
||||
ars_client = Client(SERVER_URL, USER, PASSWORD,
|
||||
CERTIFICATE, logger)
|
||||
ars_client = Client(SERVER_URL, verify_server=False, ca_bundle="",
|
||||
client_cert = "DUMMY_PATH", client_key = "DUMMY_PATH",
|
||||
logger = logger)
|
||||
|
||||
url = "https://%s/test" % (SERVER_URL)
|
||||
mock_server.register_uri("DELETE",
|
||||
@ -149,8 +119,6 @@ class PlayBookExecutionTests(unittest.TestCase):
|
||||
self.assertEqual(test_pb.play_uuid, PB_UUID,
|
||||
"Found Unexpected playbook uuid")
|
||||
|
||||
|
||||
|
||||
def test_playbook_execution_error(self):
|
||||
"""Check playbook id is not set when the playbook is not present
|
||||
"""
|
||||
|
||||
Loading…
Reference in New Issue
Block a user