openstack: resources hint is the max of all hints

Exactly one OpenStack resources hint can be included in a given job, as
part of an existing facet. It is error prone because it is sometimes not
trivial to figure out how a given job is composed and if two resources
hint are included only one of them will be taken into account which can
lead to problems difficult to diagnose. Another undesirable side effect
is to artificially increase resources usage. It is easier and more
reliable (from the test maintainer point of view) to increase the
resources of all jobs when a few need more RAM or disk rather than
trying to figure where to write the hints so that they are used by these
jobs and these jobs only.

Instead of being a fixed hint for a given job, the max of all hints
found in each facet is used. For instance, rados/thrash can have a facet
requiring that all jobs are given 3 devices.

cat rados/thrash/cluster/openstack.yaml
  openstack:
    - volumes:
        count: 3

If one task in rados/thrash needs 16GB RAM instead of the default of 8GB
RAM, it can have:

cat rados/thrash/tasks/bigworkunit.yaml

  task:
    - workunit: highmemoryusage.sh
  openstack:
    - machine:
        ram: 16 # GB

And a job composed of rados/thrash/{cluster/openstack.yaml
tasks/bigworkunit.yaml} is aggregated as the max of all resources,
including the default, that is:

  task:
    - workunit: highmemoryusage.sh
  openstack:
    - machine:
        disk: 20 # GB
        ram: 16 # GB
        cpu: 1
      volumes:
        count: 3
        size: 1 # GB

Signed-off-by: Loic Dachary <ldachary@redhat.com>

README.rst

@@ -487,6 +487,84 @@ test. Login wih the ssh access displayed at the end of the
     ...
     ========= 1 passed in 204.35 seconds =========
 
+Defining instances flavor and volumes
+-------------------------------------
+
+Each target (i.e. a virtual machine or instance in the OpenStack
+parlance) created by the OpenStack backend are exactly the same. By
+default they have at least 8GB RAM, 20GB disk, 1 cpus and no disk
+attached. It is equivalent to having the following in the
+`~/.teuthology.yaml <https://github.com/ceph/teuthology/blob/master/docs/siteconfig.rst>`_ file::
+
+    openstack:
+      ...
+      machine:
+        disk: 20 # GB
+        ram: 8000 # MB
+        cpus: 1
+      volumes:
+        count: 0
+        size: 1 # GB
+
+If a job needs more RAM or disk etc. the following can be included in
+an existing facet (yaml file in the teuthology parlance)::
+
+    openstack:
+      - machine:
+          disk: 100 # GB
+        volumes:
+          count: 4
+          size: 10 # GB
+
+Teuthology interprets this as the minimimum requirements, on top of
+the defaults found in the ``~/.teuthology.yaml`` file and the job will
+be given instances with at least 100GB root disk, 8GB RAM, 1 cpus and
+four 10GB volumes attached. The highest value wins: if the job claims
+to need 4GB RAM and the defaults are 8GB RAM, the targets will all
+have 8GB RAM.
+
+Note the dash before the ``machine`` key: the ``openstack`` element is
+an array with one value. If the dash is missing, it is a dictionary instead.
+It matters because there can be multiple entries per job such as::
+
+    openstack:
+      - machine:
+          disk: 40 # GB
+          ram: 8000 # MB
+
+    openstack:
+      - machine:
+          ram: 32000 # MB
+
+    openstack:
+      - volumes: # attached to each instance
+          count: 3
+          size: 200 # GB
+
+When a job is composed with these, theuthology aggregates them as::
+
+    openstack:
+      - machine:
+          disk: 40 # GB
+          ram: 8000 # MB
+      - machine:
+          ram: 32000 # MB
+      - volumes: # attached to each instance
+          count: 3
+          size: 200 # GB
+
+i.e. all entries are grouped in a list in the same fashion ``tasks`` are.
+The resource requirement is the maximum of the resources found in each
+element (including the default values). In the example above it is equivalent to::
+
+    openstack:
+      machine:
+        disk: 40 # GB
+        ram: 32000 # MB
+      volumes: # attached to each instance
+        count: 3
+        size: 200 # GB
+
 VIRTUAL MACHINE SUPPORT
 =======================
 

teuthology/openstack/__init__.py

@@ -21,6 +21,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 #
+import copy
 import json
 import logging
 import os
@@ -30,6 +31,7 @@ import socket
 import subprocess
 import tempfile
 import teuthology
+import types
 
 from teuthology.contextutil import safe_while
 from teuthology.config import config as teuth_config
@@ -149,6 +151,26 @@ class OpenStack(object):
         log.debug("sorted flavor = " + str(sorted_flavor))
         return sorted_flavor[0]['Name']
 
+    def interpret_hints(self, defaults, hints):
+        """
+        Return a hint hash which is the interpretation of a list of hints
+        """
+        result = copy.deepcopy(defaults)
+        if not hints:
+            return result
+        if type(hints) is types.DictType:
+            hints = [hints]
+            # TODO: raise after converting all ceph-qa-suite to only use arrays (oct 2015)
+            # raise Exception("openstack: " + str(hints) + " must be an array, not a dict")
+        for hint in hints:
+            for resource in ('machine', 'volumes'):
+                if resource in hint:
+                    new = hint[resource]
+                    current = result[resource]
+                    for key, value in hint[resource].iteritems():
+                        current[key] = max(current[key], new[key])
+        return result
+
     def cloud_init_wait(self, name_or_ip):
         """
         Wait for cloud-init to complete on the name_or_ip OpenStack instance.

teuthology/openstack/test/test_openstack.py

@@ -29,9 +29,64 @@ import tempfile
 
 import teuthology
 from teuthology import misc
-from teuthology.openstack import TeuthologyOpenStack
+from teuthology.openstack import TeuthologyOpenStack, OpenStack
 import scripts.openstack
 
+class TestOpenStack(object):
+
+    def test_interpret_hints(self):
+        defaults = {
+            'machine': {
+                'ram': 0,
+                'disk': 0,
+                'cpus': 0,
+            },
+            'volumes': {
+                'count': 0,
+                'size': 0,
+            },
+        }
+        expected_disk = 10 # first hint larger than the second
+        expected_ram = 20 # second hint larger than the first
+        expected_cpus = 0 # not set, hence zero by default
+        expected_count = 30 # second hint larger than the first
+        expected_size = 40 # does not exist in the first hint
+        hints = [
+            {
+                'machine': {
+                    'ram': 2,
+                    'disk': expected_disk,
+                },
+                'volumes': {
+                    'count': 9,
+                    'size': expected_size,
+                },
+            },
+            {
+                'machine': {
+                    'ram': expected_ram,
+                    'disk': 3,
+                },
+                'volumes': {
+                    'count': expected_count,
+                },
+            },
+        ]
+        hint = OpenStack().interpret_hints(defaults, hints)
+        assert hint == {
+            'machine': {
+                'ram': expected_ram,
+                'disk': expected_disk,
+                'cpus': expected_cpus,
+            },
+            'volumes': {
+                'count': expected_count,
+                'size': expected_size,
+            }
+        }
+        assert defaults == OpenStack().interpret_hints(defaults, None)
+
+
 class TestTeuthologyOpenStack(object):
 
     @classmethod

teuthology/provision.py

@@ -235,14 +235,10 @@ class ProvisionOpenStack(OpenStack):
             lab_domain=config.lab_domain)
         open(self.user_data, 'w').write(user_data)
 
-    def attach_volumes(self, name, hint):
+    def attach_volumes(self, name, volumes):
         """
         Create and attach volumes to the named OpenStack instance.
         """
-        if hint:
-            volumes = hint['volumes']
-        else:
-            volumes = config['openstack']['volumes']
         for i in range(volumes['count']):
             volume_name = name + '-' + str(i)
             try:
@@ -295,17 +291,17 @@ class ProvisionOpenStack(OpenStack):
         described in resources_hint.
         """
         log.debug('ProvisionOpenStack:create')
+        resources_hint = self.interpret_hints({
+            'machine': config['openstack']['machine'],
+            'volumes': config['openstack']['volumes'],
+        }, resources_hint)
         self.init_user_data(os_type, os_version)
         image = self.image(os_type, os_version)
         if 'network' in config['openstack']:
             net = "--nic net-id=" + str(self.net_id(config['openstack']['network']))
         else:
             net = ''
-        if resources_hint:
-            flavor_hint = resources_hint['machine']
-        else:
-            flavor_hint = config['openstack']['machine']
-        flavor = self.flavor(flavor_hint,
+        flavor = self.flavor(resources_hint['machine'],
                              config['openstack'].get('flavor-select-regexp'))
         misc.sh("openstack server create" +
                 " " + config['openstack'].get('server-create', '') +
@@ -341,7 +337,7 @@ class ProvisionOpenStack(OpenStack):
                 time.sleep(15)
                 if not self.cloud_init_wait(fqdn):
                     raise ValueError('clound_init_wait failed for ' + fqdn)
-                self.attach_volumes(name, resources_hint)
+                self.attach_volumes(name, resources_hint['volumes'])
                 fqdns.append(fqdn)
         except Exception as e:
             log.exception(str(e))