2009-10-06 21:00:51 +00:00
|
|
|
v0.17
|
2009-10-09 22:15:25 +00:00
|
|
|
- kclient: fix multiple mds mdsmap decoding
|
|
|
|
|
- kclient: fix mon subscription renewal
|
|
|
|
|
- crush: fix crush map creation with empty buckets (occurs on larger clusters)
|
|
|
|
|
- osdmap: fix encoding bug (crashes kclient); make kclient not crash
|
2009-10-13 06:05:50 +00:00
|
|
|
- msgr: simplified policy, failure model
|
|
|
|
|
- mon: less push, more pull
|
|
|
|
|
- mon: request routing
|
2009-10-16 22:42:34 +00:00
|
|
|
- mon cluster expansion
|
2009-10-19 19:02:22 +00:00
|
|
|
- osd: fix pg parsing, restarts on larger clusters
|
2009-10-14 22:22:23 +00:00
|
|
|
|
2009-10-20 18:59:15 +00:00
|
|
|
v0.18
|
2009-11-21 00:22:34 +00:00
|
|
|
- osd: basic ENOSPC handling
|
2009-11-06 06:21:22 +00:00
|
|
|
- big endian fixes (required protocol/disk format change)
|
2009-11-21 00:22:34 +00:00
|
|
|
- osd: improved object -> pg hash function; selectable
|
|
|
|
|
- crush: selectable hash function(s)
|
2009-11-11 00:32:01 +00:00
|
|
|
- mds restart bug fixes
|
2009-11-21 00:22:34 +00:00
|
|
|
- kclient: mds reconnect bug fixes
|
2009-11-12 23:12:18 +00:00
|
|
|
- fixed mds log trimming bug
|
|
|
|
|
- fixed mds cap vs snap deadlock
|
2009-11-21 00:22:34 +00:00
|
|
|
- filestore: faster flushing
|
|
|
|
|
- uclient,kclient: snapshot fixes
|
|
|
|
|
- mds: fix recursive accounting bug
|
|
|
|
|
- uclient: fixes for 32bit clients
|
|
|
|
|
- auth: 'none' security framework
|
|
|
|
|
- mon: "safely" bail on write errors (e.g. ENOSPC)
|
|
|
|
|
- mds: fix replay/reconnect race (caused (fast) client reconnect to fail)
|
2009-12-01 22:59:57 +00:00
|
|
|
- mds: misc journal replay, session fixes
|
|
|
|
|
|
2009-12-10 00:06:45 +00:00
|
|
|
v0.19
|
|
|
|
|
- ms_dispatch fairness
|
2009-12-11 20:03:29 +00:00
|
|
|
- kclient: bad fsid deadlock fix
|
2009-12-10 00:06:45 +00:00
|
|
|
|
|
|
|
|
- qa: snap test. maybe walk through 2.6.* kernel trees?
|
2009-11-06 06:22:27 +00:00
|
|
|
|
2009-10-20 18:59:15 +00:00
|
|
|
- osd: rebuild pg log
|
|
|
|
|
- osd: handle storage errors
|
|
|
|
|
|
|
|
|
|
- rebuild mds hierarchy
|
|
|
|
|
|
2009-10-14 22:22:23 +00:00
|
|
|
- kclient: msgs built with a page list
|
2009-09-08 20:46:00 +00:00
|
|
|
- kclient: retry alloc on ENOMEM when reading from connection?
|
2009-10-01 20:28:30 +00:00
|
|
|
|
2009-10-09 22:15:25 +00:00
|
|
|
|
|
|
|
|
pending wire, disk format changes
|
2009-12-10 16:28:31 +00:00
|
|
|
- add v to PGMap, PGMap::Incremental
|
2009-09-30 17:24:28 +00:00
|
|
|
|
2009-06-19 04:19:25 +00:00
|
|
|
bugs
|
2009-12-22 00:46:05 +00:00
|
|
|
- mon: dup osd boot messages to log
|
|
|
|
|
09.12.21 14:09:33.634098 log 09.12.21 14:09:32.612955 mon0 10.3.14.128:6789/0/0 198 : [INF] osd6 10.3.14.133:6800/14770/0 boot
|
|
|
|
|
09.12.21 14:09:33.634125 log 09.12.21 14:09:32.614155 mon0 10.3.14.128:6789/0/0 199 : [INF] osd6 10.3.14.133:6800/14770/0 boot
|
|
|
|
|
09.12.21 14:09:33.634137 log 09.12.21 14:09:32.614726 mon0 10.3.14.128:6789/0/0 200 : [INF] osd6 10.3.14.133:6800/14770/0 boot
|
|
|
|
|
09.12.21 14:09:33.634148 log 09.12.21 14:09:32.615444 mon0 10.3.14.128:6789/0/0 201 : [INF] osd6 10.3.14.133:6800/14770/0 boot
|
|
|
|
|
|
|
|
|
|
- vi file on one (k)client, cat on another, get all zeros.
|
|
|
|
|
|
2009-12-20 05:09:23 +00:00
|
|
|
- kclient mds caps state recall deadlock?
|
|
|
|
|
[211048.250655] BUG: soft lockup - CPU#0 stuck for 61s! [ceph-msgr/0:2571]
|
|
|
|
|
[211048.250661] Modules linked in: ceph fan ac battery container uhci_hcd ehci_hcd thermal button processor
|
|
|
|
|
[211048.250661] irq event stamp: 2649905664
|
|
|
|
|
[211048.250661] hardirqs last enabled at (2649905663): [<ffffffff8100c480>] restore_args+0x0/0x30
|
|
|
|
|
[211048.250661] hardirqs last disabled at (2649905664): [<ffffffff8100b927>] save_args+0x67/0x70
|
|
|
|
|
[211048.250661] softirqs last enabled at (2649905662): [<ffffffff8104bad5>] __do_softirq+0x219/0x228
|
|
|
|
|
[211048.250661] softirqs last disabled at (2649905649): [<ffffffff8100cc1c>] call_softirq+0x1c/0x28
|
|
|
|
|
[211048.250661] CPU 0:
|
|
|
|
|
[211048.250661] Modules linked in: ceph fan ac battery container uhci_hcd ehci_hcd thermal button processor
|
|
|
|
|
[211048.250661] Pid: 2571, comm: ceph-msgr/0 Not tainted 2.6.32-rc2 #1 PDSMi
|
|
|
|
|
[211048.250661] RIP: 0010:[<ffffffff8107099c>] [<ffffffff8107099c>] lock_acquired+0x2aa/0x2b0
|
|
|
|
|
[211048.250661] RSP: 0018:ffff88011c9f3960 EFLAGS: 00000286
|
|
|
|
|
[211048.250661] RAX: ffff88011d4cc580 RBX: ffff88011c9f3970 RCX: 0000000000004f01
|
|
|
|
|
|
|
|
|
|
[211048.250661] RDX: ffff88011c9f3fd8 RSI: ffffffff816d45f8 RDI: 0000000000000001
|
|
|
|
|
[211048.250661] RBP: ffffffff8100c5ee R08: 0000000000000005 R09: 0000000000000000
|
|
|
|
|
[211048.250661] R10: 0000000000000002 R11: 0000000000000246 R12: 0000000000000004
|
|
|
|
|
[211048.250661] R13: 0000000000000001 R14: 0000000000000005 R15: 0000000000000000
|
|
|
|
|
[211048.250661] FS: 0000000000000000(0000) GS:ffff88002f000000(0000) knlGS:0000000000000000
|
|
|
|
|
[211048.250661] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
|
|
|
|
|
[211048.250661] CR2: 000000000040fc00 CR3: 000000011c3da000 CR4: 00000000000006f0
|
|
|
|
|
[211048.250661] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
|
|
|
|
|
[211048.250661] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
|
|
|
|
|
[211048.250661] Call Trace:
|
|
|
|
|
[211048.250661] [<ffffffff8146cb43>] ? _spin_lock+0x68/0x70
|
|
|
|
|
[211048.250661] [<ffffffff810ec502>] ? igrab+0x18/0x40
|
|
|
|
|
[211048.250661] [<ffffffff8146c96c>] ? _spin_unlock+0x47/0x4b
|
|
|
|
|
[211048.250661] [<ffffffff810ec502>] ? igrab+0x18/0x40
|
|
|
|
|
|
|
|
|
|
[211048.250661] [<ffffffffa007b6e0>] ? iterate_session_caps+0x78/0x111 [ceph]
|
|
|
|
|
[211048.250661] [<ffffffffa0080aee>] ? trim_caps_cb+0x0/0x1bc [ceph]
|
|
|
|
|
[211048.250661] [<ffffffffa007f6ca>] ? handle_session+0x2b3/0x40a [ceph]
|
|
|
|
|
[211048.250661] [<ffffffffa007f8c5>] ? dispatch+0xa4/0x1053 [ceph]
|
|
|
|
|
[211048.250661] [<ffffffff8106d8c8>] ? mark_held_locks+0x4d/0x6b
|
|
|
|
|
[211048.250661] [<ffffffff8146ab56>] ? __mutex_unlock_slowpath+0x10d/0x130
|
|
|
|
|
[211048.250661] [<ffffffff8106db61>] ? trace_hardirqs_on_caller+0x113/0x13e
|
|
|
|
|
[211048.250661] [<ffffffff8106db99>] ? trace_hardirqs_on+0xd/0xf
|
|
|
|
|
[211048.250661] [<ffffffffa0077a71>] ? try_read+0xebe/0x15a4 [ceph]
|
|
|
|
|
[211048.250661] [<ffffffffa0078b45>] ? con_work+0x15b/0x1679 [ceph]
|
|
|
|
|
[211048.250661] [<ffffffff8105b194>] ? worker_thread+0x229/0x398
|
|
|
|
|
[211048.250661] [<ffffffff8105b1ee>] ? worker_thread+0x283/0x398
|
|
|
|
|
[211048.250661] [<ffffffff8105b194>] ? worker_thread+0x229/0x398
|
|
|
|
|
[211048.250661] [<ffffffffa00789ea>] ? con_work+0x0/0x1679 [ceph]
|
|
|
|
|
[211048.250661] [<ffffffff8105f4d0>] ? autoremove_wake_function+0x0/0x38
|
|
|
|
|
[211048.250661] [<ffffffff8105af6b>] ? worker_thread+0x0/0x398
|
|
|
|
|
[211048.250661] [<ffffffff8105f114>] ? kthread+0x7d/0x85
|
|
|
|
|
[211048.250661] [<ffffffff8100cb1a>] ? child_rip+0xa/0x20
|
|
|
|
|
[211048.250661] [<ffffffff8100c480>] ? restore_args+0x0/0x30
|
|
|
|
|
[211048.250661] [<ffffffff810632b3>] ? __atomic_notifier_call_chain+0x0/0xb2
|
|
|
|
|
[211048.250661] [<ffffffff8105f097>] ? kthread+0x0/0x85
|
|
|
|
|
[211048.250661] [<ffffffff8100cb10>] ? child_rip+0x0/0x20
|
|
|
|
|
|
|
|
|
|
- bonnie++ -u root -d /mnt/ceph/ -s 0 -n 1
|
|
|
|
|
(03:35:29 PM) Isteriat: Using uid:0, gid:0.
|
|
|
|
|
(03:35:29 PM) Isteriat: Create files in sequential order...done.
|
|
|
|
|
(03:35:29 PM) Isteriat: Stat files in sequential order...Expected 1024 files but only got 0
|
|
|
|
|
(03:35:29 PM) Isteriat: Cleaning up test directory after error.
|
|
|
|
|
- kclient: prepare_pages vs connection reset!
|
|
|
|
|
- only do prepare_pages if reply is from the expected osd?
|
|
|
|
|
- what if we get a second reply that from a new (correct) osd?
|
|
|
|
|
- osd pg split breaks if not all osds are up...
|
2009-12-17 19:32:21 +00:00
|
|
|
- kclient calculation of expected space needed for caps during reconnect converges to incorrect value:
|
|
|
|
|
Dec 16 21:09:44 ceph4 kernel: [200451.959112] ceph: mds0 10.3.14.98:6802 socket closed
|
|
|
|
|
Dec 16 21:09:46 ceph4 kernel: [200454.456519] ceph: mds0 10.3.14.98:6802 connection failed
|
|
|
|
|
Dec 16 21:10:10 ceph4 kernel: [200478.000289] ceph: reconnect to recovering mds0
|
|
|
|
|
Dec 16 21:10:10 ceph4 kernel: [200478.005164] ceph: estimating i need 7048085 bytes for 45180 caps
|
|
|
|
|
Dec 16 21:10:10 ceph4 kernel: [200478.214756] ceph: i guessed 7048085, and did 40724 of 45180 caps, retrying with 7752893
|
|
|
|
|
Dec 16 21:10:10 ceph4 kernel: [200478.446193] ceph: i guessed 7752893, and did 44432 of 45180 caps, retrying with 7830421
|
|
|
|
|
Dec 16 21:10:10 ceph4 kernel: [200478.679594] ceph: i guessed 7830421, and did 44828 of 45180 caps, retrying with 7830421
|
|
|
|
|
Dec 16 21:10:11 ceph4 kernel: [200478.913978] ceph: i guessed 7830421, and did 44828 of 45180 caps, retrying with 7830421
|
|
|
|
|
Dec 16 21:10:11 ceph4 kernel: [200479.147611] ceph: i guessed 7830421, and did 44828 of 45180 caps, retrying with 7830421
|
|
|
|
|
Dec 16 21:10:11 ceph4 kernel: [200479.381505] ceph: i guessed 7830421, and did 44828 of 45180 caps, retrying with 7830421
|
|
|
|
|
...
|
|
|
|
|
|
|
|
|
|
- msgr local_endpoint teardown vs msg delivery race
|
|
|
|
|
==1989== Process terminating with default action of signal 11 (SIGSEGV): dumping core
|
|
|
|
|
==1989== Access not within mapped region at address 0x13C
|
|
|
|
|
==1989== at 0x660C22: SimpleMessenger::Pipe::queue_received(Message*, int) (SimpleMessenger.h:246)
|
|
|
|
|
==1989== by 0x660CF2: SimpleMessenger::Pipe::queue_received(Message*) (SimpleMessenger.h:255)
|
|
|
|
|
==1989== by 0x655045: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1478)
|
|
|
|
|
==1989== by 0x663E2C: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:159)
|
|
|
|
|
==1989== by 0x65B3EA: Thread::_entry_func(void*) (Thread.h:39)
|
|
|
|
|
==1989== by 0x5030F99: start_thread (in /lib/libpthread-2.9.so)
|
|
|
|
|
==1989== by 0x5E5555C: clone (in /lib/libc-2.9.so)
|
|
|
|
|
|
2009-12-15 22:13:23 +00:00
|
|
|
- mds recovery flag set on inode that didn't get recovered??
|
|
|
|
|
- mon delay when starting new mds, when current mds is already laggy
|
|
|
|
|
- mds file purge should truncate in place, or remove from namespace before purge. otherwise new ref can appear before inode is destroyed.
|
|
|
|
|
- mds memory leak (after some combo of client failures, mds restarts+reconnects?)
|
2009-12-11 16:36:53 +00:00
|
|
|
- osd pg split breaks if not all osds are up...
|
2009-11-16 21:35:45 +00:00
|
|
|
- mislinked directory? (cpusr.sh, mv /c/* /c/t, more cpusr, ls /c/t)
|
2009-07-08 23:30:41 +00:00
|
|
|
- premature filejournal trimming?
|
2009-08-10 23:50:26 +00:00
|
|
|
- weird osd_lock contention during osd restart?
|
2009-11-17 23:05:41 +00:00
|
|
|
- kclient: after reconnect,
|
|
|
|
|
cp: writing `/c/ceph2.2/bin/gs-gpl': Bad file descriptor
|
|
|
|
|
- need to somehow wake up unreconnected caps? hrm!!
|
|
|
|
|
- kclient: socket creation
|
2009-11-25 03:01:54 +00:00
|
|
|
|
2009-12-20 05:09:23 +00:00
|
|
|
- mds:
|
|
|
|
|
mds/MDCache.cc: In function 'void MDCache::remove_inode(CInode*)':
|
|
|
|
|
mds/MDCache.cc:217: FAILED assert(o->get_num_ref() == 0)
|
|
|
|
|
1: /tmp/cmds.20091211.084324(_Z18__ceph_assert_failPKcS0_iS0_+0x34) [0x9656ea]
|
|
|
|
|
2: /tmp/cmds.20091211.084324(_ZN7MDCache12remove_inodeEP6CInode+0x1ad) [0x7af283]
|
|
|
|
|
3: /tmp/cmds.20091211.084324(_ZN7MDCache19_purge_stray_loggedEP7CDentrymP10LogSegment+0x115) [0x7af3b5]
|
|
|
|
|
4: /tmp/cmds.20091211.084324(_ZN22C_MDC_PurgeStrayLogged6finishEi+0x34) [0x83286e]
|
|
|
|
|
5: /tmp/cmds.20091211.084324(_Z15finish_contextsRSt4listIP7ContextSaIS1_EEi+0x130) [0x736d96]
|
|
|
|
|
6: /tmp/cmds.20091211.084324(_ZN9Journaler13_finish_flushEil7utime_tb+0x873) [0x915f4d]
|
|
|
|
|
7: /tmp/cmds.20091211.084324(_ZN9Journaler7C_Flush6finishEi+0x43) [0x91d5eb]
|
|
|
|
|
8: /tmp/cmds.20091211.084324(_ZN8Objecter19handle_osd_op_replyEP11MOSDOpReply+0xcf5) [0x8e7415]
|
|
|
|
|
9: /tmp/cmds.20091211.084324(_ZN3MDS9_dispatchEP7Message+0x1f04) [0x715dda]
|
|
|
|
|
10: /tmp/cmds.20091211.084324(_ZN3MDS11ms_dispatchEP7Message+0x2f) [0x716dc1]
|
|
|
|
|
11: /tmp/cmds.20091211.084324(_ZN9Messenger19ms_deliver_dispatchEP7Message+0x54) [0x70a658]
|
|
|
|
|
12: /tmp/cmds.20091211.084324(_ZN15SimpleMessenger8Endpoint14dispatch_entryEv+0x4df) [0x6f78af]
|
|
|
|
|
13: /tmp/cmds.20091211.084324(_ZN15SimpleMessenger8Endpoint14DispatchThread5entryEv+0x19) [0x70ce77]
|
|
|
|
|
14: /tmp/cmds.20091211.084324(_ZN6Thread11_entry_funcEPv+0x20) [0x704b0c]
|
|
|
|
|
15: /lib/libpthread.so.0 [0x7f3ea5bf2fc7]
|
|
|
|
|
16: /lib/libc.so.6(clone+0x6d) [0x7f3ea4e355ad]
|
|
|
|
|
|
|
|
|
|
|
2009-12-03 00:57:07 +00:00
|
|
|
- snaprealm thing
|
|
|
|
|
ceph3:~# find /c
|
|
|
|
|
/c
|
|
|
|
|
/c/.ceph
|
|
|
|
|
/c/.ceph/mds0
|
|
|
|
|
/c/.ceph/mds0/journal
|
|
|
|
|
/c/.ceph/mds0/stray
|
|
|
|
|
[68663.397407] ceph: ceph_add_cap: couldn't find snap realm 10000491bb5
|
|
|
|
|
...
|
|
|
|
|
ceph3:/c# [68724.067160] BUG: unable to handle kernel NULL pointer dereference at 0000000000000088
|
|
|
|
|
[68724.071069] IP: [<ffffffffa00805c3>] __send_cap+0x237/0x585 [ceph]
|
|
|
|
|
[68724.078917] PGD f7a12067 PUD f688c067 PMD 0
|
|
|
|
|
[68724.082907] Oops: 0000 [#1] PREEMPT SMP
|
|
|
|
|
[68724.082907] last sysfs file: /sys/class/net/lo/operstate
|
|
|
|
|
[68724.082907] CPU 1
|
|
|
|
|
[68724.082907] Modules linked in: ceph fan ac battery psmouse ehci_hcd ohci_hcd ide_pci_generic thermal processor button
|
|
|
|
|
[68724.082907] Pid: 10, comm: events/1 Not tainted 2.6.32-rc2 #1 H8SSL
|
|
|
|
|
[68724.082907] RIP: 0010:[<ffffffffa00805c3>] [<ffffffffa00805c3>] __send_cap+0x237/0x585 [ceph]
|
|
|
|
|
[68724.114907] RSP: 0018:ffff8800f96e3a50 EFLAGS: 00010202
|
|
|
|
|
[68724.114907] RAX: 0000000000000000 RBX: 0000000000000354 RCX: 0000000000000000
|
|
|
|
|
[68724.114907] RDX: 0000000000000000 RSI: ffff8800f76e8ba8 RDI: ffff8800f581a508
|
|
|
|
|
[68724.114907] RBP: ffff8800f96e3bb0 R08: 0000000000000000 R09: 0000000000000001
|
|
|
|
|
[68724.114907] R10: ffff8800cea922b8 R11: ffffffffa0082982 R12: 0000000000000001
|
|
|
|
|
[68724.114907] R13: 0000000000000000 R14: ffff8800cea95378 R15: 0000000000000000
|
|
|
|
|
[68724.114907] FS: 00007f54be9a06e0(0000) GS:ffff880009200000(0000) knlGS:0000000000000000
|
|
|
|
|
[68724.114907] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
|
|
|
|
|
[68724.114907] CR2: 0000000000000088 CR3: 00000000f7118000 CR4: 00000000000006e0
|
|
|
|
|
[68724.178904] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
|
|
|
|
|
[68724.178904] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
|
|
|
|
|
[68724.178904] Process events/1 (pid: 10, threadinfo ffff8800f96e2000, task ffff8800f96e02c0)
|
|
|
|
|
[68724.178904] Stack:
|
|
|
|
|
[68724.178904] ffff8800f96e0980 ffff8800f96e02c0 ffff8800f96e3a80 ffffffff8106a3b9
|
|
|
|
|
[68724.178904] <0> ffff8800f96e3a80 0000000000000003 00006589ac4ca260 0000000000000004
|
|
|
|
|
[68724.178904] <0> 0cb13589944c0262 0000000000000000 ffff8800f96e3b30 ffffffff81ca7c80
|
|
|
|
|
[68724.178904] Call Trace:
|
|
|
|
|
[68724.178904] [<ffffffff8106a3b9>] ? get_lock_stats+0x19/0x4c
|
|
|
|
|
[68724.178904] [<ffffffff8106d8c8>] ? mark_held_locks+0x4d/0x6b
|
|
|
|
|
[68724.178904] [<ffffffffa0082a25>] ceph_check_caps+0x740/0xa70 [ceph]
|
|
|
|
|
[68724.178904] [<ffffffff8106a3b9>] ? get_lock_stats+0x19/0x4c
|
|
|
|
|
[68724.178904] [<ffffffff8106a964>] ? put_lock_stats+0xe/0x27
|
|
|
|
|
[68724.178904] [<ffffffffa00840b6>] ceph_check_delayed_caps+0xcb/0x14a [ceph]
|
|
|
|
|
[68724.178904] [<ffffffffa009011f>] delayed_work+0x3f/0x368 [ceph]
|
|
|
|
|
[68724.178904] [<ffffffff8105b194>] ? worker_thread+0x229/0x398
|
|
|
|
|
[68724.178904] [<ffffffff8105b1ee>] worker_thread+0x283/0x398
|
|
|
|
|
[68724.178904] [<ffffffff8105b194>] ? worker_thread+0x229/0x398
|
|
|
|
|
[68724.178904] [<ffffffffa00900e0>] ? delayed_work+0x0/0x368 [ceph]
|
|
|
|
|
[68724.178904] [<ffffffff8146a56e>] ? preempt_schedule+0x3e/0x4b
|
|
|
|
|
[68724.306901] [<ffffffff8105f4d0>] ? autoremove_ceph3:/c# [68724.067160]
|
2009-11-25 03:01:54 +00:00
|
|
|
|
|
|
|
|
|
2009-12-15 22:13:23 +00:00
|
|
|
filestore performance notes
|
|
|
|
|
- write ordering options
|
|
|
|
|
- fs only (no journal)
|
|
|
|
|
- fs, journal
|
|
|
|
|
- fs + journal in parallel
|
|
|
|
|
- journal sync, then fs
|
|
|
|
|
- and the issues
|
|
|
|
|
- latency
|
|
|
|
|
- effect of a btrfs hang
|
|
|
|
|
- unexpected error handling (EIO, ENOSPC)
|
|
|
|
|
- impact on ack, sync ordering semantics.
|
|
|
|
|
- how to throttle request stream to disk io rate
|
|
|
|
|
- rmw vs delayed mode
|
|
|
|
|
|
|
|
|
|
- if journal is on fs, then
|
|
|
|
|
- throttling isn't an issue, but
|
|
|
|
|
- fs stalls are also journal stalls
|
|
|
|
|
|
|
|
|
|
- fs only
|
|
|
|
|
- latency: commits are bad.
|
|
|
|
|
- hang: bad.
|
|
|
|
|
- errors: could be handled, aren't
|
|
|
|
|
- acks: supported
|
|
|
|
|
- throttle: fs does it
|
|
|
|
|
- rmw: pg toggles mode
|
|
|
|
|
- fs, journal
|
|
|
|
|
- latency: good, unless fs hangs
|
|
|
|
|
- hang: bad. latency spikes. overall throughput drops.
|
|
|
|
|
- errors: could probably be handled, isn't.
|
|
|
|
|
- acks: supported
|
|
|
|
|
- throttle: btrfs does it (by hanging), which leads to a (necessary) latency spike
|
|
|
|
|
- rmw: pg toggles mode
|
|
|
|
|
- fs | journal
|
|
|
|
|
- latency: good
|
|
|
|
|
- hang: no latency spike. fs throughput may drop, to the extent btrfs throughput necessarily will.
|
|
|
|
|
- errors: not detected until later. could journal addendum record. or die (like we do now)
|
|
|
|
|
- acks: could be flexible.. maybe supported, maybe not. will need some extra locking smarts?
|
|
|
|
|
- throttle: ??
|
|
|
|
|
- rmw: rmw must block on prior fs writes.
|
|
|
|
|
- journal, fs (writeahead)
|
|
|
|
|
- latency: good (commit only, no acks)
|
|
|
|
|
- hang: same as |
|
|
|
|
|
- errors: same as |
|
|
|
|
|
- acks: never.
|
|
|
|
|
- throttle: ??
|
|
|
|
|
- rmw: rmw must block on prior fs writes.
|
|
|
|
|
|
|
|
|
|
- separate reads/writes into separate op queues?
|
|
|
|
|
-
|
|
|
|
|
|
|
|
|
|
|
2009-10-29 23:24:40 +00:00
|
|
|
greg
|
|
|
|
|
- osd: error handling
|
|
|
|
|
- uclient: readdir from cache
|
|
|
|
|
- mds: basic auth checks
|
|
|
|
|
|
2009-04-28 05:34:52 +00:00
|
|
|
later
|
2009-10-06 21:00:51 +00:00
|
|
|
- document on-wire protocol
|
2009-07-01 20:50:54 +00:00
|
|
|
- authentication
|
2009-04-27 20:35:21 +00:00
|
|
|
- client reconnect after long eviction; and slow delayed reconnect
|
2008-09-15 23:01:30 +00:00
|
|
|
- repair
|
2008-05-10 23:31:14 +00:00
|
|
|
- mds security enforcement
|
|
|
|
|
- client, user authentication
|
|
|
|
|
- cas
|
2008-07-25 23:23:00 +00:00
|
|
|
- osd failure declarations
|
2009-04-08 23:29:19 +00:00
|
|
|
- rename over old files should flush data, or revert back to old contents
|
2008-07-25 23:23:00 +00:00
|
|
|
|
2009-05-06 20:51:33 +00:00
|
|
|
rados
|
2009-07-22 18:26:41 +00:00
|
|
|
- make rest interface superset of s3?
|
2009-09-15 20:14:01 +00:00
|
|
|
- create/delete snapshots
|
|
|
|
|
- list, access snapped version
|
2009-06-18 22:22:15 +00:00
|
|
|
- perl swig wrapper
|
2009-06-22 17:05:19 +00:00
|
|
|
- 'rados call foo.bar'?
|
2009-05-06 22:23:12 +00:00
|
|
|
- merge pgs
|
|
|
|
|
- destroy pg_pools
|
|
|
|
|
- autosize pg_pools?
|
|
|
|
|
- security
|
2009-05-06 20:51:33 +00:00
|
|
|
|
2008-09-18 22:39:54 +00:00
|
|
|
repair
|
2009-06-18 22:22:15 +00:00
|
|
|
- namespace reconstruction tool
|
|
|
|
|
- repair pg (rebuild log) (online or offline? ./cosd --repair_pg 1.ef?)
|
|
|
|
|
- repair file ioctl?
|
2008-09-18 22:39:54 +00:00
|
|
|
- are we concerned about
|
|
|
|
|
- scrubbing
|
|
|
|
|
- reconstruction after loss of subset of cdirs
|
|
|
|
|
- reconstruction after loss of md log
|
|
|
|
|
- data object
|
|
|
|
|
- path backpointers?
|
|
|
|
|
- parent dir pointer?
|
|
|
|
|
- mds scrubbing
|
2008-07-25 23:23:00 +00:00
|
|
|
|
2009-07-02 21:15:20 +00:00
|
|
|
kclient
|
2009-12-11 20:03:29 +00:00
|
|
|
- replace radix tree in monc with rbtree on statfs requests
|
2009-11-06 07:03:59 +00:00
|
|
|
- ENOMEM
|
|
|
|
|
- message pools
|
|
|
|
|
- sockets? (this can actual generates a lockdep warning :/)
|
2009-10-06 21:00:51 +00:00
|
|
|
- use page lists for large messages? e.g. reconnect
|
2009-11-06 07:03:59 +00:00
|
|
|
- fs-portable file layout virtual xattr (see Andreas' -fsdevel thread)
|
|
|
|
|
- statlite
|
|
|
|
|
- audit/combine/rework/whatever invalidate, writeback threads and associated invariants
|
|
|
|
|
- add cap to release if we get fouled up in fill_inode et al?
|
2009-09-17 19:04:28 +00:00
|
|
|
- make caps reservations per-client
|
2009-08-03 18:39:27 +00:00
|
|
|
- fix up ESTALE handling
|
2009-09-01 23:07:55 +00:00
|
|
|
- don't retry on ENOMEM on non-nofail requests in kick_requests
|
2009-08-24 18:41:00 +00:00
|
|
|
- make cap import/export more efficient?
|
2009-01-21 18:38:06 +00:00
|
|
|
- flock, fnctl locks
|
2008-10-16 19:13:09 +00:00
|
|
|
- ACLs
|
2009-09-28 17:56:05 +00:00
|
|
|
- init security xattrs
|
2009-04-29 19:06:42 +00:00
|
|
|
- should we try to ref CAP_PIN on special inodes that are open?
|
2008-06-12 21:23:43 +00:00
|
|
|
- fix readdir vs fragment race by keeping a separate frag pos, and ignoring dentries below it
|
2009-04-29 19:06:42 +00:00
|
|
|
- inotify for updates from other clients?
|
2007-05-25 20:10:48 +00:00
|
|
|
|
2008-06-23 13:45:49 +00:00
|
|
|
vfs issues
|
|
|
|
|
- real_lookup() race:
|
|
|
|
|
1- hash lookup find no dentry
|
|
|
|
|
2- real_lookup() takes dir i_mutex, but then finds a dentry
|
|
|
|
|
3- drops mutex, then calld d_revalidate. if that fails, we return ENOENT (instead of looping?)
|
|
|
|
|
- vfs_rename_dir()
|
2009-01-30 23:00:53 +00:00
|
|
|
- a getattr mask would be really nice
|
2007-05-25 20:10:48 +00:00
|
|
|
|
2008-12-12 05:13:02 +00:00
|
|
|
filestore
|
2009-03-05 18:22:03 +00:00
|
|
|
- make min sync interval self-tuning (ala xfs, ext3?)
|
2008-12-12 05:13:02 +00:00
|
|
|
- get file csum?
|
|
|
|
|
|
2008-12-10 00:47:40 +00:00
|
|
|
btrfs
|
|
|
|
|
- clone compressed inline extents
|
|
|
|
|
- ioctl to pull out data csum?
|
|
|
|
|
|
2009-06-22 17:05:19 +00:00
|
|
|
osd
|
2009-10-28 22:32:07 +00:00
|
|
|
- gracefully handle ENOSPC
|
|
|
|
|
- gracefully handle EIO?
|
2009-10-09 22:15:25 +00:00
|
|
|
- client session object
|
|
|
|
|
- track client's osdmap; and only share latest osdmap with them once!
|
2009-06-25 03:49:41 +00:00
|
|
|
- what to do with lost objects.. continue peering?
|
2009-06-22 22:50:37 +00:00
|
|
|
- segregate backlog from log ondisk?
|
2009-06-22 17:05:19 +00:00
|
|
|
- preserve pg logs on disk for longer period
|
2009-06-22 22:50:37 +00:00
|
|
|
- make scrub interruptible
|
2009-06-22 17:05:19 +00:00
|
|
|
- optionally separate osd interfaces (ips) for clients and osds (replication, peering, etc.)
|
|
|
|
|
- pg repair
|
|
|
|
|
- pg split should be a work queue
|
|
|
|
|
- optimize remove wrt recovery pushes?
|
2008-12-10 00:47:40 +00:00
|
|
|
|
2009-07-02 21:15:20 +00:00
|
|
|
uclient
|
2009-11-18 20:25:46 +00:00
|
|
|
- fix client_lock vs other mutex with C_SafeCond
|
2009-08-11 22:43:07 +00:00
|
|
|
- clean up check_caps to more closely mirror kclient logic
|
2009-07-02 21:15:20 +00:00
|
|
|
- readdir from cache
|
2008-09-18 22:39:54 +00:00
|
|
|
- fix readdir vs fragment race by keeping a separate frag pos, and ignoring dentries below it
|
2009-07-22 23:10:51 +00:00
|
|
|
- hadoop: clean up assert usage
|
2007-12-19 04:54:23 +00:00
|
|
|
|
2008-06-04 18:08:06 +00:00
|
|
|
mds
|
2009-12-01 22:59:57 +00:00
|
|
|
- don't sync log on every clientreplay request?
|
2009-08-20 21:48:10 +00:00
|
|
|
- pass issued, wanted into eval(lock) when eval() already has it? (and otherwise optimize eval paths..)
|
2008-12-17 22:06:15 +00:00
|
|
|
- add an up:shadow mode?
|
2008-12-15 23:30:50 +00:00
|
|
|
- tail the mds log as it is written
|
|
|
|
|
- periodically check head so that we trim, too
|
2008-06-05 14:14:45 +00:00
|
|
|
- handle slow client reconnect (i.e. after mds has gone active)
|
2008-06-04 18:08:06 +00:00
|
|
|
- anchor_destroy needs to xlock linklock.. which means it needs a Mutation wrapper?
|
|
|
|
|
- ... when it gets a caller.. someday..
|
2007-10-12 22:46:27 +00:00
|
|
|
- add FILE_CAP_EXTEND capability bit
|
2009-06-15 03:29:23 +00:00
|
|
|
- dir fragment
|
|
|
|
|
- maybe just take dftlock for now, to keep it simple.
|
|
|
|
|
- dir merge
|
2009-06-03 00:14:40 +00:00
|
|
|
- snap
|
|
|
|
|
- hard link backpointers
|
|
|
|
|
- anchor source dir
|
|
|
|
|
- build snaprealm for any hardlinked file
|
|
|
|
|
- include snaps for all (primary+remote) parents
|
|
|
|
|
- how do we properly clean up inodes when doing a snap purge?
|
|
|
|
|
- when they are mid-recover? see 136470cf7ca876febf68a2b0610fa3bb77ad3532
|
|
|
|
|
- what if a recovery is queued, or in progress, and the inode is then cowed? can that happen?
|
|
|
|
|
- proper handling of cache expire messages during rejoin phase?
|
|
|
|
|
-> i think cache expires are fine; the rejoin_ack handler just has to behave if rejoining items go missing
|
|
|
|
|
|
|
|
|
|
- clustered
|
|
|
|
|
- on replay, but dirty scatter replicas on lists so that they get flushed? or does rejoin handle that?
|
|
|
|
|
- linkage vs cdentry replicas and remote rename....
|
|
|
|
|
- rename: importing inode... also journal imported client map?
|
|
|
|
|
|
2008-12-10 00:47:40 +00:00
|
|
|
mon
|
2009-12-10 16:28:31 +00:00
|
|
|
- don't allow lpg_num expansion and osd addition at the same time?
|
2009-10-16 22:42:34 +00:00
|
|
|
- how to shrink cluster?
|
2009-10-12 17:47:11 +00:00
|
|
|
- how to tell osd to cleanly shut down
|
2009-07-01 18:43:10 +00:00
|
|
|
- mds injectargs N should take mds# or id. * should bcast to standy mds's.
|
2008-12-10 00:47:40 +00:00
|
|
|
- paxos need to clean up old states.
|
|
|
|
|
- default: simple max of (state count, min age), so that we have at least N hours of history, say?
|
|
|
|
|
- osd map: trim only old maps < oldest "in" osd up_from
|
|
|
|
|
|
2008-09-18 22:39:54 +00:00
|
|
|
osdmon
|
|
|
|
|
- monitor needs to monitor some osds...
|
|
|
|
|
|
|
|
|
|
pgmon
|
2009-01-30 23:00:53 +00:00
|
|
|
/- include osd vector with pg state
|
2008-09-18 22:39:54 +00:00
|
|
|
- check for orphan pgs
|
|
|
|
|
- monitor pg states, notify on out?
|
|
|
|
|
- watch osd utilization; adjust overload in cluster map
|
|
|
|
|
|
2008-12-10 00:47:40 +00:00
|
|
|
crush
|
|
|
|
|
- allow forcefeed for more complicated rule structures. (e.g. make force_stack a list< set<int> >)
|
2008-09-18 22:39:54 +00:00
|
|
|
|
2007-02-17 22:49:47 +00:00
|
|
|
simplemessenger
|
2008-12-10 00:47:40 +00:00
|
|
|
- close idle connections?
|
2006-08-02 18:00:24 +00:00
|
|
|
|
|
|
|
|
objectcacher
|
2008-11-07 00:14:18 +00:00
|
|
|
- read locks?
|
|
|
|
|
- maintain more explicit inode grouping instead of wonky hashes
|
2009-03-03 23:21:34 +00:00
|
|
|
|
|
|
|
|
cas
|
|
|
|
|
- chunking. see TTTD in
|
|
|
|
|
ESHGHI, K.
|
|
|
|
|
A framework for analyzing and improving content-based chunking algorithms.
|
2009-06-12 19:49:30 +00:00
|
|
|
Tech. Rep. HPL-2005-30(R.1), Hewlett Packard Laboratories, Palo Alto, 2005.
|
2009-07-01 18:58:40 +00:00
|
|
|
|
2009-10-28 22:32:07 +00:00
|
|
|
radosgw
|
2009-07-15 21:04:09 +00:00
|
|
|
- handle gracefully location related requests
|
2009-07-01 18:58:40 +00:00
|
|
|
- logging control (?)
|
2009-07-02 20:17:26 +00:00
|
|
|
- parse date/time better
|
|
|
|
|
- upload using post
|
2009-07-11 01:29:12 +00:00
|
|
|
- torrent
|
2009-07-16 00:58:14 +00:00
|
|
|
- handle gracefully PUT/GET requestPayment
|
2009-12-22 00:46:05 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
-- for nicer kclient debug output (everything but messenger, but including msg in/out)
|
|
|
|
|
echo 'module ceph +p' > /sys/kernel/debug/dynamic_debug/control ; echo 'file fs/ceph/messenger.c -p' > /sys/kernel/debug/dynamic_debug/control ; echo 'file ' `grep -- --- /sys/kernel/debug/dynamic_debug/control | grep ceph | awk '{print $1}' | sed 's/:/ line /'` +p > /sys/kernel/debug/dynamic_debug/control ; echo 'file ' `grep -- === /sys/kernel/debug/dynamic_debug/control | grep ceph | awk '{print $1}' | sed 's/:/ line /'` +p > /sys/kernel/debug/dynamic_debug/control
|
