RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-01-12 14:10:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,688 Commits 2 Branches 394 Tags 1.3 GiB
efd5ccbc4b
Commit Graph

1468 Commits

Author SHA1 Message Date
Zack Cerza
1685de6741 Fix typo 2013-09-18 22:57:20 -05:00
Zack Cerza
d83912e0bf If duration is missing from summary.yaml, use 0 2013-09-18 14:08:23 -05:00
Zack Cerza
aaa3cf9baa Fix logs URL on Sentry pages. 2013-09-18 10:30:26 -05:00
Zack Cerza
bbda7522c6 Make sed expressions safe to run multiple times 
Also, make them ignore commented lines.
 2013-09-17 18:04:04 -05:00
Zack Cerza
61ddeb6a03 Revert "suite: don't schedule follow-on summary job for an empty run" 
This reverts commit a2b7c72896.
 2013-09-17 13:47:42 -05:00
Sage Weil
4520c5f6f5 suite: do not calculate product on an empty list (take 2) 
The original attempt to fix this with dda7954090
failed because I screwed up the indentation for the out.append(...) line.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-16 20:22:09 -07:00
Zack Cerza
e65d6e2a00 Revert "suite: do not calculate product on an empty list" 
This reverts commit dda7954090.

This commit was causing the number of jobs scheduled per suite to be
multiplied by anywhere from 2 to 7 or more.
 2013-09-16 18:33:21 -05:00
SandonV
a8e77738af Merge pull request #94 from ceph/wip-fix-sudoers 
Use teuthology-specific backup for /etc/sudoers
 2013-09-16 13:49:39 -07:00
Zack Cerza
8b63396ef6 Use teuthology-specific backup for /etc/sudoers 2013-09-16 15:25:16 -05:00
Alfredo Deza
5ca6bc3974 Merge pull request #93 from ceph/wip-runfix 
Make run_job merge job_config['config'] if needed
 2013-09-16 11:50:39 -07:00
Zack Cerza
4d2e3c2736 Make run_job merge job_config['config'] if needed 2013-09-16 13:14:52 -05:00
Zack Cerza
fad71073e2 Be less picky about trailing slashes. 2013-09-16 12:35:30 -05:00
Alfredo Deza
568eccf86e Merge pull request #91 from ceph/wip-suite 
suite: do not calculate product on an empty list
 2013-09-13 08:41:35 -07:00
Sage Weil
a2b7c72896 suite: don't schedule follow-on summary job for an empty run 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-13 08:39:30 -07:00
Sage Weil
5a083894e0 suite: be verbose about how many jobs we generate 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-13 08:39:13 -07:00
Sage Weil
dda7954090 suite: do not calculate product on an empty list 
This fixes a bug where a directory with just % generates a
bogus job item with no actual content.  (e.g.,

$ find basic
basic
basic/%
$ teuthology-suite --dry-run --base . --collections basic --name foo
INFO:teuthology.suite:Collection basic in ./basic
INFO:teuthology.suite:configs [('basic/{}', [])]

...which then blows up because the job yaml is empty.  With
the fix, there are no generated jobs:

$ teuthology-suite --dry-run --base . --collections basic --name foo
INFO:teuthology.suite:Collection basic in ./basic
INFO:teuthology.suite:configs []

(The configs print was temporarily added for debugging
purposes.)

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-13 08:22:12 -07:00
Zack Cerza
42c2b704b6 Merge pull request #90 from ceph/sentry 
Fix get_http_log_path(), update callers, add tests
 2013-09-12 15:56:06 -07:00
Zack Cerza
31f0f7ad7e Fix get_http_log_path(), update callers, add tests 2013-09-12 17:28:50 -05:00
tamil
9de651a945 Merge branch 'master' of github.com:ceph/teuthology 2013-09-12 15:07:32 -07:00
tamil
eb4c575f54 made help more readable 
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
 2013-09-12 15:03:10 -07:00
Zack Cerza
2db2ba781a Fix TypeError 
(cherry picked from commit c4591a16e1781f8da3502b38e3dc72f629478312)
 2013-09-12 16:42:59 -05:00
Zack Cerza
e83b5defe8 Use check_output() and log.exception() 
This should help us figure out why our checkouts keep getting deleted.
 2013-09-12 11:14:08 -05:00
tamil
40d6c60f13 feature # 5942. Added examples to teuthology binaries help page 
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
 2013-09-11 17:13:22 -07:00
Alfredo Deza
06be66b33f Merge pull request #87 from ceph/sentry 
Add logs URL to sentry reports
 2013-09-11 14:06:24 -07:00
Zack Cerza
1228d8cb5a Add teuthology branch to sentry report tags 2013-09-11 15:45:45 -05:00
Zack Cerza
0ad9c8751e Ensure teuthology_branch is stored in job_config 2013-09-11 15:40:14 -05:00
Zack Cerza
5b3ce84596 Add ctx.config to sentry info. 2013-09-11 15:31:31 -05:00
Zack Cerza
fe51db6fc0 Merge job_config and ctx.config 2013-09-11 15:14:58 -05:00
Zack Cerza
725ba83c04 Add test for misc.get_http_log_path() 2013-09-11 14:49:15 -05:00
Zack Cerza
38a1a6dde5 Use sentry.get_ident() again 2013-09-11 14:00:58 -05:00
Zack Cerza
4e9c5378ba Don't fail if there's no job_id 2013-09-11 13:58:19 -05:00
Zack Cerza
df4925703e Passing job_id to get_http_log_path() is optional 2013-09-11 13:47:49 -05:00
Zack Cerza
f7537f9c34 Add logs URL to sentry reports 2013-09-11 13:44:28 -05:00
Zack Cerza
713fa52455 Add job id and actual archive dir to job config 
Also add job id to info.yaml
 2013-09-11 13:44:28 -05:00
Zack Cerza
4699ad1c11 Don't "import misc as teuthology" 2013-09-11 10:50:42 -05:00
Zack Cerza
c1ba276321 Move get_http_log_path 2013-09-11 10:50:42 -05:00
Zack Cerza
c9db15e67c s/jobid/job_id/ 2013-09-11 10:03:30 -05:00
Sage Weil
89df90f68f rados: add --op copy_from ... support 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-10 16:53:18 -07:00
Sage Weil
748fa5e559 Merge pull request #85 from ceph/wip-testdir 
remove basedir/testdir distinction
 2013-09-10 12:20:29 -07:00
Sage Weil
5acc57f5ad remove basedir/testdir distinction 
We should never run with a conflicting testdir in the basedir, and the
code to do this is confusing and buggy.  Go back to a single testdir and
simple checks.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-10 10:53:41 -07:00
Joe Buck
d47ae37e93 Adding in erroneously removed args 
The 'kill' argument was accidentially removed
as part of patch
53b8e27da9

Signed-off-by: Joe Buck <jbbuck@gmail.com>
 2013-09-09 12:37:05 -07:00
Sage Weil
922aa09b4c fix a few missing adjust-ulimits users 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-08 15:12:51 -07:00
Sage Weil
6b4014452f rgw: fix valgrind when no valgrind 
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
 2013-09-06 17:38:29 -07:00
Sage Weil
9e03c73775 misc: valgrind: fix cd behavior 
The cd needs to happen at the beginning of the command, not at the end, or
else the funky wrapper scripts don't work right.

This also cleans up the command line construction a bit so that it is more
flexible, explicit, and hopefully less fragile.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 16:09:27 -07:00
Sage Weil
19324c63d2 misc: valgrind: fix valgrind.supp path 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 16:09:27 -07:00
Zack Cerza
82cfa848b4 Get rid of chdir-coredump. 2013-09-06 17:11:44 -05:00
Sage Weil
a60e84d43f ceph: fix daemon-helper typo 
Broken in edc5ef8860

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 14:49:38 -07:00
Sage Weil
2214fe1845 misc: valgrind --num-callers=50 
Default is 12, which isn't quite enough.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 14:05:29 -07:00
Sage Weil
c6d7ae0fa5 Merge pull request #78 from ceph/wip-6247 
Move helper scripts to /usr/local/bin to clean up logs.
 2013-09-06 13:24:34 -07:00
Zack Cerza
53b8e27da9 Helper scripts live in /usr/local/bin now! 2013-09-06 15:10:11 -05:00
Zack Cerza
edc5ef8860 Move helper scripts to /usr/local/bin 2013-09-06 15:09:41 -05:00
Sage Weil
1a05f9d0a2 queue: fix stderr redirect 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 09:39:30 -07:00
Alfredo Deza
a4f0551fda Merge pull request #75 from ceph/wip-tube 
queue: include tube name in worker logs
 2013-09-06 09:27:43 -07:00
Sage Weil
1a03b17568 Merge pull request #76 from ceph/wip-5954 
Be polite and restore /etc/sudoers when done
 2013-09-06 09:23:51 -07:00
Sage Weil
5cd2f08132 queue: include tube name in worker logs 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 09:19:30 -07:00
Zack Cerza
f0eabc970f Be polite and restore /etc/sudoers when done 2013-09-06 11:18:17 -05:00
Sage Weil
4857a2a0f0 Merge pull request #74 from ceph/wip-5954 
Wip 5954
 2013-09-05 15:16:32 -07:00
Sage Weil
7db9311ab7 Merge pull request #73 from ceph/wip-5954-alt 
Via automagic, only call ulimit -n if running as root.
 2013-09-05 15:15:55 -07:00
Zack Cerza
d6e8b2337f Add internal.sudo subtask. 2013-09-05 16:56:11 -05:00
Zack Cerza
6d1ed573f3 Let execute() accept a string for args. 2013-09-05 16:55:38 -05:00
Zack Cerza
6406a1ecc8 Add Cluster.write_file() 2013-09-05 16:55:37 -05:00
Zack Cerza
8f2fb9da32 Use misc.get_archive_dir() 2013-09-05 16:55:37 -05:00
Zack Cerza
90f9a6ada1 Add get_archive_dir() 2013-09-05 16:55:37 -05:00
Zack Cerza
91fd0413cc Put sudo in front of adjust-ulimits if it's used 2013-09-05 16:41:27 -05:00
Zack Cerza
8e37361b9a Via automagic, only call ulimit -n if running as root. 
Also, add comments because comments are cool.
 2013-09-05 16:30:28 -05:00
Sandon Van Ness
d34ba1647f Run yum clean all after installing new ceph-release. 
In order to get around the issue of repomd.xml being older than
the previously installed one depending on when the gitbuilder
built the repo.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
 2013-09-05 11:58:27 -07:00
Sage Weil
c155dce1b2 drop obsolete PYTHONPATH settings 
These are left over from when we were extracting a tarball in the test dir.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-04 14:21:01 -07:00
Zack Cerza
89f1139d83 Append '/' to the end of the log directory path 2013-09-04 15:16:33 -05:00
Zack Cerza
a26c831398 Merge pull request #64 from ceph/wip-nuke-info 
nuke: get pid, owner from info.yaml (if present)
 2013-09-04 09:16:04 -07:00
Zack Cerza
7dee3a1321 Merge pull request #65 from ceph/wip-schedule 
schedule: strip out targets
 2013-09-04 09:13:39 -07:00
Alfredo Deza
a9522ae245 add the missing for fedora links in install task 
Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
 2013-09-04 09:03:28 -04:00
Warren Usui
a455b16fc1 Add missing sudo from command. 
Fixes: #6218
 2013-09-03 14:35:00 -07:00
Zack Cerza
f744c4ce53 Catch CommandFailedError and print debug info 2013-09-03 14:40:31 -05:00
Sage Weil
f3db12f025 schedule: strip out targets 
If the user feeds in a yaml with targets, the worker will launch the job
but fail with

2013-09-03T11:18:34.333 CRITICAL:root:AssertionError: You cannot specify targets in a config file when using the --lock option

Just strip them out before scheduling.  This eases my personal workflow
where I have a test I'm running manually against some prelocked machines
but also want to schedule it.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-03 12:05:38 -07:00
Zack Cerza
f36f16a669 Logs and Sentry links are public now. 2013-09-03 13:46:31 -05:00
Sage Weil
4ba3d52018 Revert "Use install -d for /var/log/ceph." 
This reverts commit 8f29b3f9f1.

Reviewed-by: Warren Usui <warren.usui@inktank.com>
 2013-09-03 11:09:39 -07:00
Sage Weil
3d64322cdb Revert "get_scratch_devices(): catch CommandFailedError, log "dev in use"" 
This reverts commit 416508de9c.

NameError: global name 'CommandFailedError' is not defined
 2013-08-30 23:22:24 -07:00
tamil
35b9beab06 s/ceph-debug/ceph-debuginfo for rpms 
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
 2013-08-30 17:35:15 -07:00
Dan Mick
416508de9c get_scratch_devices(): catch CommandFailedError, log "dev in use" 
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
 2013-08-30 15:42:53 -07:00
Sage Weil
dcbf50b86c nuke: get pid, owner from info.yaml (if present) 
Fall back to the old files if info.yaml is missing.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-30 10:02:27 -07:00
Alfredo Deza
e0d42c820e Merge pull request #63 from ceph/wip-logger 
Never use 'except:' without specifying an Exception.
 2013-08-30 09:15:45 -07:00
Zack Cerza
3981a8f1af Never use 'except:' without specifying an Exception. 2013-08-30 11:10:05 -05:00
Alfredo Deza
fbd7fe466a Merge pull request #62 from ceph/wip-logger 
Add hung job description to email
 2013-08-30 07:50:30 -07:00
Sage Weil
f342baeb22 run: do not import at top level 
This makes a run with --lock hang when connecting, for some
reason.  E.g.,

 $ teuthology -v a.yaml --lock
 ...
 INFO:teuthology.task.internal:Opening connections...
 DEBUG:teuthology.task.internal:connecting to ubuntu@plana06.front.sepia.ceph.com
 <hangs>

No clue what is going on here, but this fixes it!

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-29 20:27:40 -07:00
Zack Cerza
24991c4ae0 Add hung job description to email 2013-08-29 17:08:51 -05:00
Zack Cerza
44401f9a0e Workers: only log child's stderr, not stdout 2013-08-29 16:12:36 -05:00
Zack Cerza
40f1d4fc58 Don't print colon if there's no exception message. Also, remove testing 
exception.
 2013-08-29 15:05:33 -05:00
Zack Cerza
83a5ab19b0 Pretty-up write_initial_metadata; s/run/name/ 2013-08-29 15:05:33 -05:00
Zack Cerza
9550b49921 Add exception hook 2013-08-29 15:04:50 -05:00
Sage Weil
f6430df250 run: write info.yaml on job start 
Include basic metadata bout the job.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-29 15:04:50 -05:00
Zack Cerza
a0e86c05bd Move logging setup out of main() 2013-08-29 15:03:30 -05:00
Sage Weil
27eea4ad5e test 2013-08-29 15:03:30 -05:00
Sage Weil
649bddf71b run: set up archive dir and logger output sooner 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-29 15:03:30 -05:00
Alfredo Deza
ed575251db Merge pull request #58 from ceph/queue 
Don't assume anything about the base path here.
 2013-08-29 06:07:44 -07:00
Sage Weil
4fbec83490 suite: return [] for build_matrix base case 
This gives us an empty list instead of a crash on an empty dir
or bad symlink

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-28 21:53:41 -07:00
Alfredo Deza
d3f1369812 Merge pull request #60 from ceph/sentry 
Add run name to first line of email.
 2013-08-28 18:52:29 -07:00
Zack Cerza
df73f503a3 Add run name to first line of email. 2013-08-28 15:41:14 -05:00
Sage Weil
7ce6696249 suite: build_matrix as a list of filenames, not file content 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-28 13:25:36 -07:00
Sage Weil
7163e89b83 suite: allow individual files to be scheduled 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-28 13:25:36 -07:00
Sage Weil
b829543388 suite: generalize collection expansion 
Previous, a collection was a directory like this:

 mycollection/
 mycollection/facet1/
 mycollection/facet1/1a.yaml
 mycollection/facet1/1b.yaml
 mycollection/facet2/
 mycollection/facet2/2a.yaml
 mycollection/facet3/
 mycollection/facet3/3a.yaml
 mycollection/facet3/3b.yaml

and this would expand to

 1a + 2a + 3a
 1a + 2a + 3b
 1b + 2a + 3a
 1b + 2a + 3b

The fixed directory depth and requirement for a subdir even
when there is only 1 item is annoying.  Instead, allow an
arbitrary directory structure, with the following rules:

 - a .yaml file is a taken as-is (duh); other files still
   ignored
 - a directory is normally just a way to organize files.  We
   recursively descend and build a list of what we fine.
 - a directory with a '%' file in it is special:
     - take the product of every item in the dir (much like
       we did before)
 - a directory with a '+' file in it is special:
     - concatenate everything in the dir into one job

Note that this is equivalent to the previous structure if we
do:

 for facet in mycollection/* ; do touch $facet/% ; done

We can clean up slightly be taking any dir with only one yaml
file in it and replacing the dir with the bare .yaml.

Once this is done, we can reorganize directories however we
like.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-28 13:25:36 -07:00