the hadoop branch rel/release-2.8.5 fails to build with:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:37 min
[INFO] Finished at: 2020-01-14T13:09:02Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) on project hadoop-aws: An Ant BuildException has occured: Unable to create javax script engine for javascript
Signed-off-by: Casey Bodley <cbodley@redhat.com>
To avoid confusion fix function names in osd-backfill-space.sh for how
they actually work.
Fixes: https://tracker.ceph.com/issues/43592
Signed-off-by: David Zafman <dzafman@redhat.com>
2020-01-13T10:48:14.235 INFO:tasks.workunit.client.0.smithi177.stderr:+ vgchange -an test_cephadm
2020-01-13T10:48:14.239 INFO:tasks.workunit.client.0.smithi177.stderr: WARNING: Running as a non-root user. Functionality may be unavailable.
2020-01-13T10:48:14.239 INFO:tasks.workunit.client.0.smithi177.stderr: /run/lvm/lvmetad.socket: access failed: Permission denied
2020-01-13T10:48:14.240 INFO:tasks.workunit.client.0.smithi177.stderr: WARNING: Failed to connect to lvmetad. Falling back to device scanning.
2020-01-13T10:48:14.260 INFO:tasks.workunit.client.0.smithi177.stderr: /dev/mapper/control: open failed: Permission denied
2020-01-13T10:48:14.261 INFO:tasks.workunit.client.0.smithi177.stderr: Failure to communicate with kernel device-mapper driver.
2020-01-13T10:48:14.261 INFO:tasks.workunit.client.0.smithi177.stderr: Incompatible libdevmapper 1.02.145 (2017-11-03) and kernel driver (unknown version).
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr: /run/lock/lvm/V_test_cephadm:aux: open failed: Permission denied
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr: Can't get lock for test_cephadm
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr: Cannot process volume group test_cephadm
Signed-off-by: Michael Fritch <mfritch@suse.com>
if mgr is not active, monitor will refuse to set any option consumed by
mgr modules.
the reason the tests pass somtimes is that, we have a racing here:
1. stop all mgr daemons
2. MgrMonitor gets updated and updates its mgr_module_options
accordingly.
3. in TestDashboard.setUp(), we reset the port number for dashboard
using "ceph config set mgr mgr/dashboard/y/ssl_server_port 7789"
4. restart all mgr daemons
but the 2nd step and 3rd step could race with each other, if the 2nd
step happens after 3rd step, the test passes. otherwise it fails.
in this change, "--force" is passed to the "ceph config set" command,
so ConfigMonitor can bypass the sanity test for the option, and just
set this option.
Signed-off-by: Kefu Chai <kchai@redhat.com>
qa/tasks/mgr/dashboard: set pg_num to 16
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
before this change, `setattr()` sets the instance specialized with a certain method
of test case, so in `MgrTestCase.setUpClass()`
assert cls.mgr_cluster is not None
fails,
after this change, instead of test case, the class of test suite is updated with the
specified params, even if we pass a certain test to test runner.
so we can
./run-backend-api-tests.sh tasks.mgr.test_dashboard.TestDashboard.test_standby
now. before this change, we can only:
./run-backend-api-tests.sh tasks.mgr.test_dashboard.TestDashboard
Signed-off-by: Kefu Chai <kchai@redhat.com>
mgr/dashboard: Throw a more meaningful exception
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
random.choice(seq) raises IndexError if seq is empty. we cannot ensure
there is always one or more pools in the cluster while using pool
related thrasher. so skip the thrasher action if there is no pools at
that moment.
Fixes: https://tracker.ceph.com/issues/43412
Signed-off-by: Kefu Chai <kchai@redhat.com>
this was added to test that admin apis forward relevent requests to the
master zone, but radosgw_admin_rest.py tries to create an admin user
with 'radosgw-admin user create'. this fails with:
Please run the command on master zone. Performing this operation on
non-master zone leads to inconsistent metadata between zones
Are you sure you want to go ahead? (requires --yes-i-really-mean-it)
Signed-off-by: Casey Bodley <cbodley@redhat.com>
"fs fail" will only fail the MDS that are part of the file system which
will generally allow us to avoid spurious MDS_INSUFFICIENT_STANDBY
warnings. Further, only restart the MDS, there's no reason to leave them
offline.
Fixes: https://tracker.ceph.com/issues/43514
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/30802/head:
cephfs-shell: Modify Quota Test
cephfs-shell: Fix permission errors on quota tests
cephfs-shell: Add tests for quota
cephfs-shell: Print max_bytes and max_files as string on get Quota
cephfs-shell: Remove extra length argument passed to setxattr()
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/32459/head:
qa/tasks: Nothing to clean up if the volume was not created
mgr/volumes: drop obsolete comment in _cmd_fs_volume_create
mgr/volumes: cleanup on fs create error
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Remove use of rules batching for upmap balancer and default for upmap_max_deviation to 5
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Choose another future_date_1 date (9 days instead of 10) to
make sure the expiration date will be between future_date_1
and future_date_2. Otherwise the expiration date might be
equal to future_date_1.
Also wait for 10 seconds before the password refresh in
test_pwd_expiration_date_update to prevent date equality.
Fixes: https://tracker.ceph.com/issues/43431
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
qa/tasks/mgr/test_orchestrator_cli: fix service action tests
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
in b77f0c74a5, "reload"
service[-instance] action was dropped. so replace "reload" with
"restart" in the related tests.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/32480/head:
qa: use -D_GNU_SOURCE when compiling fsync-tester.c
qa: add comment about fsync-tester distribution
Reviewed-by: Kefu Chai <kchai@redhat.com>
* refs/pull/31494/head:
qa: force creation of fs with EC default data pool
qa: add tests for adding EC data pools
mon/MDSMonitor: warn when creating fs with default EC data pool
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
instead of enabling PowerTools repo, we need to enable CodeReady Builder
repo for RHEL8
also, since we are moving to RHEL8, there is no need to install cmake3
specifically for CentOS. CentOS 8 comes with cmake3.
Signed-off-by: Kefu Chai <kchai@redhat.com>
In Python 3.5 json.tool was changed to produce unsorted output and
--sort-keys option was added to compensate. This wasn't caught by
4fe245cc2f ("qa: update krbd tests for python3") because it raced
with 50933b863a ("qa: krbd_exclusive_option.sh: update for recent
kernel changes").
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Otherwise loff_t is unavailable for some distributions.
Fixes: https://tracker.ceph.com/issues/43460
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
There are only 2 cases which needs cleanup:
1. The volume is successfully created
2. The volume is successfully created but create_mds fails
In either case, we could do a 'volume rm'.
Signed-off-by: Jos Collin <jcollin@redhat.com>
Changing the branch name in the section where
force-branch is defined for s3-tests does not work
because of overrides to the force-branch.
The overrides should not be there in the first place.
Signed-off-by: Ali Maredia <amaredia@redhat.com>
* Remove scope and permission check from UserPasswordPolicy controller again, otherwise the Angular UI validator can't be executed for users without that privilege.
* Remobe obsolete QA test.
Signed-off-by: Volker Theile <vtheile@suse.com>
1. Fix the test cases by using the 'assertJsonBody'
method. The '_post' method doesn't return anything.
That's why we need to use the 'assertJsonBody'
method here instead of checking the fields directly.
2. Add the missing scope and permission definition of the
'validate_password' resource. Otherwise the resource
is not restricted and 'test_validate_password_invalid_permissions'
will fail.
3. Re-word error messages. Change 'cannot' to 'must not'.
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
- Add 'api/user/validate_password' endpoint to check if a
password meets the password policy. A new controller has
to be added for 'api/user' which has NO security scope,
otherwise it wouldn't be possible for users without USER
privileges to call the endpoint.
- Add Angular async validator to check if the entered password
meets the policy.
Fixes: https://tracker.ceph.com/issues/43088
Signed-off-by: Volker Theile <vtheile@suse.com>
This patch makes following modifications quota test:
- Add additional condtion to confirm Command Failure Error due to exceeding set
quota values.
- Rename function valid to set_and_get_quota_vals.
- Use run_cephfs_shell_cmd instead of get_cephfs_shell_cmd_output.
Signed-off-by: Varsha Rao <varao@redhat.com>
Use of mount_a.run_shell() to create directory, requires Sudo access and
cephfs-shell should be run without sudo access. Instead use cephfs-shell mkdir
command to create directory.
Signed-off-by: Varsha Rao <varao@redhat.com>
we use `six.StringIO` for stdout when writing output messages from
`self.subproc.communicate()`, and `six.StringIO` accepts strings, so we
need to decode the output before sending them to `six.StringIO`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
mon: make "compact" tell command, and add deprecate/obsolete check for tell commands
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
* refs/pull/32232/head:
qa: no need to exclude ceph-mgr-diskprediction-cloud from package list to be installed
qa/packages: do not install ceph-mgr-diskprediction-cloud by default
ceph.spec.in: add runtime deps for mgr-diskprediction-cloud
Reviewed-by: Sage Weil <sage@redhat.com>
* refs/pull/32377/head:
qa/suites/rados/thrash-old-clients: configure mons in terms of addrvecs
qa/suites/rados/thrash-old-clients: hammer: fix package list
qa/tasks/cephadm: set .conf to cluster config object
qa/tasks/cephadm: archive /var/log/ceph logs too (not just cluster dir)
qa/tasks/cephadm: client keyring
qa/tasks/cephadm: setup thrashers ctx item
qa/tasks/ceph_manager: asok commands via cephadm shell
qa/suites/rados/thrash-old-clients: stick to el7
qa/tasks/cephadm: check cluster log; support log-whitelist
qa/suites/rados/thrash-old-clienets: python-foo to python3-foo
qa/suites/rados/thrash-old-clients: add new exclude_packages
qa/suites/rados/thrash-old-clients: use cephadm
mon/ConfigMonitor: make legacy mon addr/port parseable by legacy code
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
This is more explicit. More importantly, the 'mon update' command
can't handle an "ip:port"; it wants either a CIDR, bare IP, or addrvec.
Signed-off-by: Sage Weil <sage@redhat.com>
The OS image was changed in a9ee4bcf24 from Xenial to Bionic,
but the Bionic image path is incorrect.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
There's no need for mgr/cephadm to pass --mon-* arguments to
/usr/bin/cephadm just so it can add bits to the config file when the
config file is also being passed in from mgr/cephadm.
Signed-off-by: Sage Weil <sage@redhat.com>
- deploy cluster with cephadm so we can run a octopus+ cluster and also
install client packages that are ancient.
- move client.2 back onto the third node, since packages no longer
conflict.
- test on centos 7.x (i picked 6), since the old releases all built on
that release.
Signed-off-by: Sage Weil <sage@redhat.com>
In the future, we should add the EC data pool as a supplementary data
pool but that requires a mount to setup which is awkward in the code
here. When cephfs-shell is more widely available, this will be easier.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
The simple
os_type: centos
in valgrind.yaml doesn't pick a particular centos, and we end up with
the teuthology default (currently 7.6).
Signed-off-by: Sage Weil <sage@redhat.com>
quote from Sage's reply
> This is a mon-specific command--it doesn't make sense as a CLI command
> for the entire cluster--it only makes sense as a command to tell a
> specific monitor. Like ``ceph tell mon.a compact``. Back when Joao
> did #4595 these commands were all mixed together and putting it under
> 'ceph mon ...' made sense, but now you're specifially sending it to a
> mon, so the 'mon' part of the command is redundant.
so let's drop "mon compact" in favor of "compact" command
Signed-off-by: Kefu Chai <kchai@redhat.com>
"scrub" was marked deprecated in
1814d7441b. this commit
was in turn included by v10.0.0. so it's long enough for its
retirement.
the test is updated accordingly
Signed-off-by: Kefu Chai <kchai@redhat.com>
- This is an ancient swift version
- The tempest tests are newer and show provide similar coverage
- It somehow broke with the py3 transition
Signed-off-by: Sage Weil <sage@redhat.com>
* refs/pull/32252/head:
qa/cephfs/begin: libaio-devel on el8
qa/tasks: nosetests -> python -m nose
qa/tasks/rbd_fio: fio 2.21 -> 3.16
src/test/cli-integration/rbd/snap-diff.t: python -> python
qa/workunits: use nose 3
qa/tasks/cbt: install python3 deps
qa/tasks/ceph_manager.py: do not use python to write a file
test/pybind/test_rados: execute takes a bytes (not str) payload
qa/packages/packages: python[3]-ceph is no more
qa: use python3 for venvs etc
packaging: remove python3-ipaddres, as it is part of the stdlib in py3
qa/packages: python-ceph -> python3-ceph
qa/distros: centos7 -> centos8, rhel7 -> rhel8
spec: remove _python_buildid in favor of python3_pkgversion macro
spec: remove python2 packages and conditions
debian: remove python >= 2.7 requirement
debian: add mgr python versions
debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
debian: update control file to use python3 dependency names
debian: remove all python2 overrides and declarations
debian: remove all python2 install files
Reviewed-by: Alfredo Deza <adeza@redhat.com>
* refs/pull/32278/head:
qa/suites/rgw: disable valgrind for tests that require py2/ubuntu
qa/suites/rgw: disable remaining ragweed test
qa/suites/rgw: pin swift tests to py2/ubuntu
qa/suites/rgw: ragweed on ubuntu
qa/suites: run s3tests on ubuntu
Reviewed-by: Casey Bodley <cbodley@redhat.com>
The 'rbd bench' command was recently modified to print IEC units
instead of bytes/sec. This broke the handling for QoS throughput
tests since it was incorrectly evaluating the available RBD
throughput. Additionally, the QoS tests should use a "<="
comparison operator since the QoS is the upper-bound limit.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
* refs/pull/31884/head:
qa/tasks: Fix the volume ls in test_volume_rm
qa/tasks: tests for 'fs volume create' and 'fs volume ls'
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Some tests set this to a dynamic value, it'd be helpful to know how long
a test is planning to wait.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
We cannot do a traditional upgrade (install old package, start cluster,
install new package, ...) because nautilus is el7-only and octopus is
el8-only.
So, do these tests on ubuntu.
Signed-off-by: Sage Weil <sage@redhat.com>
- don't use journaling feature to chose the mode;
- provide new API function mirror_image_enable2;
- return back the old behavior to automatically enable/disable
journaling feature on enabling/disabling image mirroring.
Signed-off-by: Mykola Golub <mgolub@suse.com>
/usr/bin/python dne on el8, /usr/bin/python3 dne on el7. But
all we need to do is write a file--we can do that with tee.
Signed-off-by: Sage Weil <sage@redhat.com>
Now both CephFS quotas can be changed with a validation against the next
tree maximum in the file tree, that prevents setting the quotas in a way
that would not be usable.
Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>
To be able to catch problems with python2 *and* python3, run flake8
with both versions. From the flake8 homepage:
It is very important to install Flake8 on the correct version of
Python for your needs. If you want Flake8 to properly parse new
language features in Python 3.5 (for example), you need it to be
installed on 3.5 for Flake8 to understand those features. In many
ways, Flake8 is tied to the version of Python on which it runs.
Also fix the problems with python3 on the way.
Note: This requires now the six module for teuthology. But this is
already an install_require in teuthology itself.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
Add a 'pwd_expiration_date' field to the User object to be able
to set password expiration date per user.
There are two options to set a password expiration date for a
user:
1. Set the 'USER_PWD_EXPIRATION_SPAN' setting. If defined
the expiration date of the password will be calculated
automatically based on the given value. It will also be
updated automatically when the user changes the password.
2. Set an expiration date by setting the 'pwdExpirationDate'
field when creating a user.
Add two more settings in addition: USER_PWD_EXPIRATION_WARNING_1
and USER_PWD_EXPIRATION_WARNING_2. These two settings are
defining the amount of days to notify the user that his password
will expiration soon.
It's not possible to set the 'pwd_expiration_date' field to a
date in the past.
If the password of a user is already expired he is no longer
allowed to log into the dashboard.
Fixes: https://tracker.ceph.com/issues/40329
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
ln command needs superuser privileges, oddly that test had run
successfully before.
Fixes: https://tracker.ceph.com/issues/43250
Signed-off-by: Rishabh Dave <ridave@redhat.com>
