RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-25 21:03:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,583 Commits 211 Branches 394 Tags 1.3 GiB
5b3ce84596
Commit Graph

40 Commits

Author SHA1 Message Date
Zack Cerza
fe51db6fc0 Merge job_config and ctx.config 2013-09-11 15:14:58 -05:00
Zack Cerza
713fa52455 Add job id and actual archive dir to job config 
Also add job id to info.yaml
 2013-09-11 13:44:28 -05:00
Sage Weil
1a05f9d0a2 queue: fix stderr redirect 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 09:39:30 -07:00
Sage Weil
5cd2f08132 queue: include tube name in worker logs 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-09-06 09:19:30 -07:00
Zack Cerza
44401f9a0e Workers: only log child's stderr, not stdout 2013-08-29 16:12:36 -05:00
Zack Cerza
6175a133f4 Don't assume anything about the base path here. 2013-08-28 13:36:15 -05:00
Josh Durgin
232e3d32bc Fix undefined symbol errors 
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
 2013-08-27 15:58:14 -07:00
Sage Weil
c861e2d70b queue: only git fetch once per minute per branch 
This takes 1-2 seconds and makes launching jobs slow.  Only do it once
every 60 seconds per branch.

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-23 15:43:35 -07:00
Sage Weil
973d5aff1c queue: only let one worker update the teuthology checkouts at a time 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-23 13:43:18 -07:00
Zack Cerza
eafd591ab1 Move git stuff to fetch_teuthology_branch() 2013-08-23 10:08:01 -05:00
Zack Cerza
307284c2ed Rewrite branch fetching. 2013-08-23 09:59:48 -05:00
Sage Weil
22fc733770 queue: fetch origin, not branch 
Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-22 22:20:26 -07:00
Sage Weil
c39ec60d48 queue: only bootstrap new checkouts 
Until we figure out why bootstrap is getting stuck like this:

 9851 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 2075 pts/7    Z      0:00  \_ [git] <defunct>
 2112 pts/7    Z      0:00  \_ [git] <defunct>
 2138 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 9852 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 2153 pts/7    Z      0:00  \_ [git] <defunct>
 2177 pts/7    Z      0:00  \_ [git] <defunct>
 2264 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 9853 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 2141 pts/7    Z      0:00  \_ [git] <defunct>
 2276 pts/7    Z      0:00  \_ [git] <defunct>
 2305 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 9854 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 7448 pts/7    Z      0:00  \_ [git] <defunct>
 7449 pts/7    Z      0:00  \_ [git] <defunct>
 7450 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 7452 pts/7    Z      0:00  \_ [teuthology-resu] <defunct>
 9855 pts/7    S      0:01 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 7712 pts/7    Z      0:00  \_ [git] <defunct>
 7713 pts/7    Z      0:00  \_ [git] <defunct>
 7714 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 7716 pts/7    Z      0:00  \_ [teuthology-resu] <defunct>
 9856 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 2316 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 9857 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
 2340 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 9858 pts/7    S      0:01 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs
23188 pts/7    Z      0:00  \_ [bootstrap] <defunct>
 9859 pts/7    S      0:03 /home/teuthworker/teuthology-master/virtualenv/bin/python ./teuthology-master/virtualenv/bin/teuthology-worker -v --archive-dir /var/lib/teuthworker/archive --tube plana --log-dir /var/lib/teuthworker/archive/worker_logs

Signed-off-by: Sage Weil <sage@inktank.com>
 2013-08-22 22:14:41 -07:00
Zack Cerza
98160c5c99 Fix SyntaxError 2013-08-22 18:31:18 -05:00
Zack Cerza
a9df6c2a6a Worker shouldn't attempt to rebuild an existing virtualenv 2013-08-22 18:02:22 -05:00
Zack Cerza
c773060914 Use the ceph.com git mirror. 2013-08-22 15:51:39 -05:00
Zack Cerza
73de8b7734 Use fetch and reset --hard, not checkout and pull 2013-08-22 15:30:31 -05:00
Zack Cerza
c6293a4ee4 Make the worker obtain the correct teuthology branch 2013-08-22 14:47:18 -05:00
Alfredo Deza
5263a6f758 failed dict.get evaluates to None 2013-07-19 09:01:28 -04:00
Alfredo Deza
65f49beb35 safer get by falling back to a dictionary 2013-07-19 08:59:53 -04:00
Sandon Van Ness
5f9a1d8a0f Worker processes by machine type instead of teuthology branch. 
teuthology-suite and schedulewill now take --worker instead of
--branch. The branch is set by setting teuthology_branch in the
yaml used to schedule the job.

The teuthology branches are assumed to be in ~/teuthology-$branch
of whatever user is running the workers.
 2013-07-18 12:04:08 -07:00
Sam Lang
87b9849628 add --name option to teuthology 
Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
 2013-01-31 08:23:34 -06:00
Josh Durgin
a09153b688 Allow scheduled jobs to use different teuthology branches 
teuthology-[schedule|suite] get a parameter to specify the branch,
to put the job in a branch-specific queue. Workers running that
branch of teuthology can pull jobs from that queue.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
 2012-09-21 17:16:56 -07:00
Sage Weil
6dbbcf03fc queue: fix logging of child return code 2012-08-14 15:08:21 -07:00
Tommi Virtanen
273a43eda8 Flush data to temp file before reading it in another process. 2012-08-09 09:42:35 -07:00
Tommi Virtanen
8aaf21d537 Oops tempfile now gives us file objects not fds. 2012-08-09 09:42:13 -07:00
Tommi Virtanen
99e99758e5 In teuthology-worker, shuffle the child stdout/stderr into our log. 
Otherwise, child can suffer a failure that does not get logged by
it's own exception handling machinery, and we have no idea why.
 2012-08-08 14:48:21 -07:00
Tommi Virtanen
05007f7e0f Minimize scope of try-except. 
os.write and list.append won't raise CalledProcessError, and now
we don't need to try to contain them for temp file clean up reasons.
 2012-08-08 14:45:49 -07:00
Tommi Virtanen
4b9e17626d Use tempfile.NamedTemporaryFile instead of mkstemp. 
Simpler code, no manual cleanup needed. We see a littering of
zero-length temp files from teuthology-worker, and this seems
like a likely source.
 2012-08-08 14:44:47 -07:00
Josh Durgin
561f06cf94 suite: make email-on-success the default behavior 
This way you can tell when a run is complete, instead of wondering if
it's stuck in the queue.
 2012-01-05 17:27:31 -08:00
Josh Durgin
321381d75f teuthology-worker: remove --keep-locked-on-error 2011-10-07 14:51:46 -07:00
Josh Durgin
7b7ff6e8ce teuthology-worker: clean up last_in_suite jobs 
There's no reason not to delete them once they start.
 2011-10-04 12:32:58 -07:00
Josh Durgin
c7ff5fc38b queue: results_timeout needs to be converted to a string 2011-09-22 18:23:42 -07:00
Josh Durgin
7be9eaa030 suite: add option to send an email if the entire suite passed 2011-08-29 12:42:45 -07:00
Josh Durgin
4f4227a44d Generate coverage at the end of a suite run, 
and optionally email failures and ongoing jobs.
 2011-08-29 10:23:12 -07:00
Josh Durgin
2b66938232 queue: delete every job when it finishes, so only running jobs are buried 2011-08-29 10:23:12 -07:00
Josh Durgin
ac0a469b78 teuthology-worker: log to a file with timestamps 2011-08-03 15:28:46 -07:00
Josh Durgin
d079d51037 teuthology-worker: keep machines locked on error 
This prevents a failure to clean up in one case from affecting the
rest of the tests.
 2011-08-02 17:59:08 -07:00
Josh Durgin
731fe41022 Remove print that clutters the worker logs. 2011-07-19 17:13:13 -07:00
Josh Durgin
85c24bda7f Add teuthology-schedule and teuthology-worker. 
schedule puts jobs in a beanstalk queue, worker takes them out and runs them.
 2011-07-11 13:49:06 -07:00