2019-11-12T09:19:59Z k8s-node-02 /tmp/tmp.zePZzI744v/confd[16121]: INFO Backend set to etcd 2019-11-12T09:19:59Z k8s-node-02 /tmp/tmp.zePZzI744v/confd[16121]: INFO Starting confd 2019-11-12T09:19:59Z k8s-node-02 /tmp/tmp.zePZzI744v/confd[16121]: INFO Backend source(s) set to http://192.168.200.10:2379 2019-11-12 09:20:00 /entrypoint.sh: Adding bootstrap keyrings. 2019-11-12 09:20:01 /entrypoint.sh: Retrieving the admin key. 2019-11-12 09:20:03 /entrypoint.sh: SUCCESS 2019-11-12 09:20:03.614 7fe897e7f000 0 set uid:gid to 167:167 (ceph:ceph) 2019-11-12 09:20:03.614 7fe897e7f000 0 ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable), process ceph-mds, pid 16285 2019-11-12 09:20:03.618 7fe897e7f000 0 pidfile_write: ignore empty --pid-file starting mds.k8s-node-02 at - 2019-11-12 09:20:03.742 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56301 from mon.0 2019-11-12 09:20:08.406 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56302 from mon.0 2019-11-12 09:20:08.406 7fe885a11700 1 mds.k8s-node-02 Map has assigned me to become a standby 2019-11-12 09:20:08.430 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56303 from mon.0 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:boot --> up:replay 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 replay_start 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 recovery set is 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 waiting for osdmap 28661 (which blacklists prior instance) 2019-11-12 09:20:08.522 7fe87ef7a700 0 mds.0.cache creating system inode with ino:0x100 2019-11-12 09:20:08.522 7fe87ef7a700 0 mds.0.cache creating system inode with ino:0x1 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.56303 Finished replaying journal 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.56303 making mds journal writeable 2019-11-12 09:20:16.686 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56304 from mon.0 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:replay --> up:reconnect 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 reconnect_start 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 reopen_log 2019-11-12 09:20:16.694 7fe885a11700 1 mds.0.server reconnect_clients -- 14 sessions 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804200 192.168.200.30:0/979484066 after 0 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804134 192.168.200.17:0/3102681411 after 0 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804143 192.168.200.31:0/3544368504 after 0 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804137 192.168.200.19:0/2796926812 after 0 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793818 192.168.200.30:0/232332927 after 0.00400008 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804140 192.168.200.32:0/2855972843 after 0.00400008 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1792815 192.168.200.32:0/3593333399 after 0.00400008 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794668 192.168.200.30:0/2985278080 after 0.00800016 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794662 192.168.200.30:0/522103062 after 0.00800016 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793737 192.168.200.31:0/2398232289 after 0.00800016 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793815 192.168.200.15:0/4201708698 after 0.00800016 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794665 192.168.200.30:0/3506497893 after 0.0120002 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793728 192.168.200.15:0/1656050499 after 0.0120002 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794521 192.168.200.31:0/1453103486 after 0.0120002 2019-11-12 09:20:16.706 7fe885a11700 1 mds.0.56303 reconnect_done 2019-11-12 09:20:17.710 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56305 from mon.0 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:reconnect --> up:rejoin 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 rejoin_start 2019-11-12 09:20:17.714 7fe885a11700 1 mds.0.56303 rejoin_joint_start 2019-11-12 09:20:17.766 7fe87f77b700 1 mds.0.56303 rejoin_done 2019-11-12 09:20:18.779 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56306 from mon.0 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:rejoin --> up:active 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 recovery_done -- successful recovery! 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 active_start 2019-11-12 09:20:18.795 7fe885a11700 1 mds.0.56303 cluster recovered. /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: In function 'void Server::_rename_prepare(MDRequestRef&, EMetaBlob*, ceph::bufferlist*, CDentry*, CDentry*, CDentry*)' thread 7fe880f7e700 time 2019-11-12 09:20:18.803701 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: 8036: FAILED assert(oldin->first <= straydn->first) ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x7fe88f05b36b] 2: (()+0x26e4f7) [0x7fe88f05b4f7] 3: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 4: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 5: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 6: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 7: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 8: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 9: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 10: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 11: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 12: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 13: (()+0x7e65) [0x7fe88cc8ee65] 14: (clone()+0x6d) [0x7fe88bd6988d] 2019-11-12 09:20:18.803 7fe880f7e700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: In function 'void Server::_rename_prepare(MDRequestRef&, EMetaBlob*, ceph::bufferlist*, CDentry*, CDentry*, CDentry*)' thread 7fe880f7e700 time 2019-11-12 09:20:18.803701 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: 8036: FAILED assert(oldin->first <= straydn->first) ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x7fe88f05b36b] 2: (()+0x26e4f7) [0x7fe88f05b4f7] 3: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 4: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 5: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 6: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 7: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 8: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 9: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 10: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 11: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 12: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 13: (()+0x7e65) [0x7fe88cc8ee65] 14: (clone()+0x6d) [0x7fe88bd6988d] *** Caught signal (Aborted) ** in thread 7fe880f7e700 thread_name:mds_rank_progr ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (()+0xf5f0) [0x7fe88cc965f0] 2: (gsignal()+0x37) [0x7fe88bca1337] 3: (abort()+0x148) [0x7fe88bca2a28] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x248) [0x7fe88f05b468] 5: (()+0x26e4f7) [0x7fe88f05b4f7] 6: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 7: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 8: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 9: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 10: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 11: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 12: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 13: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 14: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 15: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 16: (()+0x7e65) [0x7fe88cc8ee65] 17: (clone()+0x6d) [0x7fe88bd6988d] 2019-11-12 09:20:18.803 7fe880f7e700 -1 *** Caught signal (Aborted) ** in thread 7fe880f7e700 thread_name:mds_rank_progr ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (()+0xf5f0) [0x7fe88cc965f0] 2: (gsignal()+0x37) [0x7fe88bca1337] 3: (abort()+0x148) [0x7fe88bca2a28] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x248) [0x7fe88f05b468] 5: (()+0x26e4f7) [0x7fe88f05b4f7] 6: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 7: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 8: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 9: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 10: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 11: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 12: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 13: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 14: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 15: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 16: (()+0x7e65) [0x7fe88cc8ee65] 17: (clone()+0x6d) [0x7fe88bd6988d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command abort hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perfcounters_dump hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command 1 hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf dump hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perfcounters_schema hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf histogram dump hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command 2 hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf schema hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf histogram schema hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf reset hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config show hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config help hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config set hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config unset hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config get hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config diff hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config diff get hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command log flush hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command log dump hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command log reopen hook 0x5574ef5302b0 -988> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_mempools hook 0x5574ef5662c8 -988> 2019-11-12 09:20:03.514 7fe897e7f000 10 monclient: get_monmap_and_config -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: build_initial_monmap -988> 2019-11-12 09:20:03.598 7fe888a17700 2 Event(0x5574ef543480 nevent=5000 time_id=1).set_owner idx=0 owner=140636701423360 -988> 2019-11-12 09:20:03.598 7fe887a15700 2 Event(0x5574ef6b8480 nevent=5000 time_id=1).set_owner idx=2 owner=140636684637952 -988> 2019-11-12 09:20:03.598 7fe888216700 2 Event(0x5574ef543680 nevent=5000 time_id=1).set_owner idx=1 owner=140636693030656 -988> 2019-11-12 09:20:03.598 7fe897e7f000 1 Processor -- start -988> 2019-11-12 09:20:03.598 7fe897e7f000 1 -- - start start -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: init -988> 2019-11-12 09:20:03.598 7fe897e7f000 5 adding auth protocol: cephx -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: auth_supported 2 method cephx -988> 2019-11-12 09:20:03.598 7fe897e7f000 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-k8s-node-02/keyring -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: _reopen_session rank -1 -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-00 con 0x5574ef700c00 addr 192.168.200.15:6789/0 -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-02 con 0x5574ef701200 addr 192.168.200.19:6789/0 -988> 2019-11-12 09:20:03.598 7fe897e7f000 1 -- - --> 192.168.200.15:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52afc0 con 0 -988> 2019-11-12 09:20:03.598 7fe897e7f000 1 -- - --> 192.168.200.19:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52b200 con 0 -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): _renew_subs -988> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): authenticate will time out at 2019-11-12 09:25:03.605089 -988> 2019-11-12 09:20:03.602 7fe887a15700 1 -- 192.168.200.19:0/763506544 learned_addr learned my addr 192.168.200.19:0/763506544 -988> 2019-11-12 09:20:03.602 7fe888216700 2 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:03.602 7fe887a15700 2 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:03.602 7fe888216700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15926 cs=1 l=1). rx mon.0 seq 1 0x5574ef52bb00 mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.602 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.0 192.168.200.15:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef52bb00 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.602 7fe888216700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15926 cs=1 l=1). rx mon.0 seq 2 0x5574ef52bd40 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): got monmap 323, mon.k8s-node-00 is now rank 0 -988> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -988> 2019-11-12 09:20:03.602 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.0 192.168.200.15:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (164204572 0 0) 0x5574ef52bd40 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): my global_id is 1849245 -988> 2019-11-12 09:20:03.602 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.15:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef52afc0 con 0 -988> 2019-11-12 09:20:03.606 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 1 0x5574ef7a66c0 mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a66c0 con 0x5574ef701200 -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): got monmap 323, mon.k8s-node-02 is now rank 2 -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -988> 2019-11-12 09:20:03.606 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 2 0x5574ef7a6900 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (3829032891 0 0) 0x5574ef7a6900 con 0x5574ef701200 -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): my global_id is 1841201 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.19:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef7a66c0 con 0 -988> 2019-11-12 09:20:03.606 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 3 0x5574ef7a66c0 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.606 7fe888216700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15926 cs=1 l=1). rx mon.0 seq 3 0x5574ef52afc0 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 222+0+0 (2280171065 0 0) 0x5574ef7a66c0 con 0x5574ef701200 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN pgs=15926 cs=1 l=1).mark_down -988> 2019-11-12 09:20:03.606 7fe887214700 2 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN pgs=15926 cs=1 l=1)._stop -988> 2019-11-12 09:20:03.606 7fe887214700 1 monclient: found mon.k8s-node-02 -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient: _send_mon_message to mon.k8s-node-02 at 192.168.200.19:6789/0 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ef6b9800 con 0 -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-11-12 09:19:33.612313) -988> 2019-11-12 09:20:03.606 7fe887214700 10 monclient: _send_mon_message to mon.k8s-node-02 at 192.168.200.19:6789/0 -988> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.19:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x5574ef52afc0 con 0 -988> 2019-11-12 09:20:03.606 7fe897e7f000 5 monclient: authenticate success, global_id 1841201 -988> 2019-11-12 09:20:03.610 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 4 0x5574ef79e200 config(0 keys) v1 -988> 2019-11-12 09:20:03.610 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 5 0x5574ef7a6b40 mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.610 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (0 0 0) 0x5574ef79e200 con 0x5574ef701200 -988> 2019-11-12 09:20:03.610 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 6 0x5574ef7a6d80 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: handle_config config(0 keys) v1 -988> 2019-11-12 09:20:03.610 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 5 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6b40 con 0x5574ef701200 -988> 2019-11-12 09:20:03.610 7fe897e7f000 10 monclient: get_monmap_and_config success -988> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: handle_monmap mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.610 7fe897e7f000 10 monclient: shutdown -988> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: got monmap 323, mon.k8s-node-02 is now rank 2 -988> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -988> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=31893 cs=1 l=1).mark_down -988> 2019-11-12 09:20:03.610 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (1366969319 0 0) 0x5574ef7a6d80 con 0x5574ef701200 -988> 2019-11-12 09:20:03.610 7fe897e7f000 2 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=31893 cs=1 l=1)._stop -988> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.610 7fe885a11700 4 set_mon_vals no callback set -988> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 shutdown_connections -988> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections mark down 192.168.200.19:6789/0 0x5574ef701200 -988> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections mark down 192.168.200.15:6789/0 0x5574ef700c00 -988> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections delete 0x5574ef700c00 -988> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections delete 0x5574ef701200 -988> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 shutdown_connections -988> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 wait complete. -988> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 >> 192.168.200.19:0/763506544 conn(0x5574ef700600 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -988> 2019-11-12 09:20:03.610 7fe897e7f000 2 -- 192.168.200.19:0/763506544 >> 192.168.200.19:0/763506544 conn(0x5574ef700600 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -988> 2019-11-12 09:20:03.614 7fe897e7f000 0 set uid:gid to 167:167 (ceph:ceph) -988> 2019-11-12 09:20:03.614 7fe897e7f000 0 ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable), process ceph-mds, pid 16285 -988> 2019-11-12 09:20:03.618 7fe897e7f000 0 pidfile_write: ignore empty --pid-file -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) init /var/run/ceph/ceph-mds.k8s-node-02.asok -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) bind_and_listen /var/run/ceph/ceph-mds.k8s-node-02.asok -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command 0 hook 0x5574ef52f4a0 -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command version hook 0x5574ef52f4a0 -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command git_version hook 0x5574ef52f4a0 -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command help hook 0x5574ef530510 -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command get_command_descriptions hook 0x5574ef530500 -988> 2019-11-12 09:20:03.706 7fe886a13700 5 asok(0x5574ef5585a0) entry start -988> 2019-11-12 09:20:03.706 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 _finish_bind bind my_inst.addr is 0.0.0.0:6800/1460470921 -988> 2019-11-12 09:20:03.706 7fe897e7f000 10 monclient: build_initial_monmap -988> 2019-11-12 09:20:03.706 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 start start -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 adding auth protocol: cephx -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 adding auth protocol: cephx -988> 2019-11-12 09:20:03.706 7fe897e7f000 1 Processor -- start -988> 2019-11-12 09:20:03.706 7fe897e7f000 10 monclient: init -988> 2019-11-12 09:20:03.706 7fe897e7f000 5 adding auth protocol: cephx -988> 2019-11-12 09:20:03.706 7fe897e7f000 10 monclient: auth_supported 2 method cephx -988> 2019-11-12 09:20:03.706 7fe897e7f000 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-k8s-node-02/keyring -988> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient: _reopen_session rank -1 -988> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-00 con 0x5574ef700c00 addr 192.168.200.15:6789/0 -988> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-01 con 0x5574ef701200 addr 192.168.200.17:6789/0 -988> 2019-11-12 09:20:03.710 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52b8c0 con 0 -988> 2019-11-12 09:20:03.710 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 --> 192.168.200.17:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52b680 con 0 -988> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient(hunting): _renew_subs -988> 2019-11-12 09:20:03.710 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 learned_addr learned my addr 192.168.200.19:6800/1460470921 -988> 2019-11-12 09:20:03.710 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:03.710 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:03.710 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 1 0x5574ef7a6000 mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6000 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): got monmap 323, mon.k8s-node-00 is now rank 0 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -988> 2019-11-12 09:20:03.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 2 0x5574ef7a6240 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (1092923085 0 0) 0x5574ef7a6240 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): my global_id is 1849248 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef7a6000 con 0 -988> 2019-11-12 09:20:03.714 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=29948 cs=1 l=1). rx mon.1 seq 1 0x5574ef7a6fc0 mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.1 192.168.200.17:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6fc0 con 0x5574ef701200 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): got monmap 323, mon.k8s-node-01 is now rank 1 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -988> 2019-11-12 09:20:03.714 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=29948 cs=1 l=1). rx mon.1 seq 2 0x5574ef7a7200 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.1 192.168.200.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (457588048 0 0) 0x5574ef7a7200 con 0x5574ef701200 -988> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): my global_id is 1840183 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef7a6b40 con 0 -988> 2019-11-12 09:20:03.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 3 0x5574ef7a6000 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 222+0+0 (868200785 0 0) 0x5574ef7a6000 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 2 181 bytes epoch 0) v1 -- 0x5574ef7a7200 con 0 -988> 2019-11-12 09:20:03.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=29948 cs=1 l=1). rx mon.1 seq 3 0x5574ef7a6b40 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.1 192.168.200.17:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 222+0+0 (1953234255 0 0) 0x5574ef7a6b40 con 0x5574ef701200 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6789/0 -- auth(proto 2 181 bytes epoch 0) v1 -- 0x5574ef7a6000 con 0 -988> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 4 0x5574ef7a7200 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 799+0+0 (2751031812 0 0) 0x5574ef7a7200 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=29948 cs=1 l=1).mark_down -988> 2019-11-12 09:20:03.718 7fe885a11700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=29948 cs=1 l=1)._stop -988> 2019-11-12 09:20:03.718 7fe885a11700 1 monclient: found mon.k8s-node-00 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ef6c4e00 con 0 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-11-12 09:19:33.723602) -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x5574ef7a6b40 con 0 -988> 2019-11-12 09:20:03.718 7fe897e7f000 5 monclient: authenticate success, global_id 1849248 -988> 2019-11-12 09:20:03.718 7fe897e7f000 10 monclient: wait_auth_rotating waiting (until 2019-11-12 09:20:33.723724) -988> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 5 0x5574ef542c00 config(0 keys) v1 -988> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 6 0x5574ef7a6480 mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 7 0x5574ef7a7440 auth_reply(proto 2 0 (0) Success) v1 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (0 0 0) 0x5574ef542c00 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: handle_config config(0 keys) v1 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 6 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6480 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: handle_monmap mon_map magic: 0 v1 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: got monmap 323, mon.k8s-node-00 is now rank 0 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -988> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (4270746359 0 0) 0x5574ef7a7440 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-11-12 09:19:33.725365) -988> 2019-11-12 09:20:03.718 7fe897e7f000 10 monclient: wait_auth_rotating done -988> 2019-11-12 09:20:03.718 7fe883f84700 4 set_mon_vals no callback set -988> 2019-11-12 09:20:03.718 7fe897e7f000 10 monclient: _renew_subs -988> 2019-11-12 09:20:03.722 7fe897e7f000 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:03.722 7fe897e7f000 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({mdsmap=0+,mgrmap=0+}) v3 -- 0x5574ef79f600 con 0 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command status hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_ops_in_flight hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command ops hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_blocked_ops hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_historic_ops hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_historic_ops_by_duration hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command scrub_path hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command tag path hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command flush_path hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command export dir hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump cache hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command cache status hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command cache drop hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump tree hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump loads hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump snaps hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command session evict hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command osdmap barrier hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command session ls hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command flush journal hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command force_readonly hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command get subtrees hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dirfrag split hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dirfrag merge hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dirfrag ls hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command openfiles ls hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump inode hook 0x5574ef530430 -988> 2019-11-12 09:20:03.722 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 8 0x5574ef7a78c0 mdsmap(e 56301) v1 -988> 2019-11-12 09:20:03.722 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 8 ==== mdsmap(e 56301) v1 ==== 493+0+0 (1364545111 0 0) 0x5574ef7a78c0 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.722 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 9 0x5574ef7ad080 mgrmap(e 3928) v1 -988> 2019-11-12 09:20:03.722 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:boot seq 1 -988> 2019-11-12 09:20:03.742 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:03.742 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:boot seq 1 v0) v7 -- 0x5574ef6ff180 con 0 -988> 2019-11-12 09:20:03.742 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56301 from mon.0 -988> 2019-11-12 09:20:03.742 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 9 ==== mgrmap(e 3928) v1 ==== 1318+0+0 (728182834 0 0) 0x5574ef7ad080 con 0x5574ef700c00 -988> 2019-11-12 09:20:03.742 7fe885a11700 4 mgrc handle_mgr_map Got map version 3928 -988> 2019-11-12 09:20:03.742 7fe885a11700 4 mgrc handle_mgr_map Active mgr is now 192.168.200.17:6800/91 -988> 2019-11-12 09:20:03.742 7fe885a11700 4 mgrc reconnect Starting new session with 192.168.200.17:6800/91 -988> 2019-11-12 09:20:03.746 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6800/91 conn(0x5574ef702400 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:03.746 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgropen(unknown.k8s-node-02) v3 -- 0x5574ef564c00 con 0 -988> 2019-11-12 09:20:03.746 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6800/91 conn(0x5574ef702400 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=543676 cs=1 l=1). rx mgr.1754125 seq 1 0x5574ef558f00 mgrconfigure(period=5, threshold=5) v2 -988> 2019-11-12 09:20:03.750 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mgr.1754125 192.168.200.17:6800/91 1 ==== mgrconfigure(period=5, threshold=5) v2 ==== 8+0+0 (3460719617 0 0) 0x5574ef558f00 con 0x5574ef702400 -988> 2019-11-12 09:20:03.750 7fe885a11700 4 mgrc handle_mgr_configure stats_period=5 -988> 2019-11-12 09:20:03.750 7fe885a11700 4 mgrc handle_mgr_configure updated stats threshold: 5 -988> 2019-11-12 09:20:03.750 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +0-0 packed 6) v6 -- 0x5574ef564f00 con 0 -988> 2019-11-12 09:20:07.742 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:boot seq 2 -988> 2019-11-12 09:20:07.762 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:07.762 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:boot seq 2 v56301) v7 -- 0x5574ef6ff500 con 0 -988> 2019-11-12 09:20:08.406 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 10 0x5574ef7a7b00 mdsmap(e 56302) v1 -988> 2019-11-12 09:20:08.406 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 10 ==== mdsmap(e 56302) v1 ==== 618+0+0 (519530298 0 0) 0x5574ef7a7b00 con 0x5574ef700c00 -988> 2019-11-12 09:20:08.406 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56302 from mon.0 -988> 2019-11-12 09:20:08.406 7fe885a11700 5 mds.beacon.k8s-node-02 set_want_state: up:boot -> up:standby -988> 2019-11-12 09:20:08.406 7fe885a11700 1 mds.k8s-node-02 Map has assigned me to become a standby -988> 2019-11-12 09:20:08.410 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 11 0x5574ef6ff500 mdsbeacon(1849248/k8s-node-02 up:boot seq 2 v56302) v7 -988> 2019-11-12 09:20:08.410 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 11 ==== mdsbeacon(1849248/k8s-node-02 up:boot seq 2 v56302) v7 ==== 140+0+0 (767818115 0 0) 0x5574ef6ff500 con 0x5574ef700c00 -988> 2019-11-12 09:20:08.410 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:boot seq 2 rtt 0.668014 -988> 2019-11-12 09:20:08.430 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 12 0x5574ef7a7200 mdsmap(e 56303) v1 -988> 2019-11-12 09:20:08.430 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 12 ==== mdsmap(e 56303) v1 ==== 655+0+0 (1259924393 0 0) 0x5574ef7a7200 con 0x5574ef700c00 -988> 2019-11-12 09:20:08.430 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56303 from mon.0 -988> 2019-11-12 09:20:08.430 7fe885a11700 4 mds.0.purge_queue operator(): data pool 2 not found in OSDMap -988> 2019-11-12 09:20:08.434 7fe885a11700 5 asok(0x5574ef5585a0) register_command objecter_requests hook 0x5574ef530610 -988> 2019-11-12 09:20:08.434 7fe885a11700 10 monclient: _renew_subs -988> 2019-11-12 09:20:08.434 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:08.434 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x5574ef7c4c00 con 0 -988> 2019-11-12 09:20:08.434 7fe885a11700 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) -988> 2019-11-12 09:20:08.434 7fe885a11700 4 mds.0.purge_queue operator(): data pool 2 not found in OSDMap -988> 2019-11-12 09:20:08.434 7fe885a11700 4 mds.0.0 handle_osd_map epoch 0, 0 new blacklist entries -988> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -988> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:boot --> up:replay -988> 2019-11-12 09:20:08.434 7fe885a11700 5 mds.beacon.k8s-node-02 set_want_state: up:standby -> up:replay -988> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 replay_start -988> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 recovery set is -988> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 waiting for osdmap 28661 (which blacklists prior instance) -988> 2019-11-12 09:20:08.434 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 13 0x5574ef7f4500 osd_map(28661..28661 src has 27910..28661) v4 -988> 2019-11-12 09:20:08.434 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 13 ==== osd_map(28661..28661 src has 27910..28661) v4 ==== 98323+0+0 (451166248 0 0) 0x5574ef7f4500 con 0x5574ef700c00 -988> 2019-11-12 09:20:08.438 7fe885a11700 4 mds.0.56303 handle_osd_map epoch 28661, 0 new blacklist entries -988> 2019-11-12 09:20:08.438 7fe885a11700 10 monclient: _renew_subs -988> 2019-11-12 09:20:08.438 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:08.438 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({osdmap=28662}) v3 -- 0x5574ef7c4e00 con 0 -988> 2019-11-12 09:20:08.438 7fe87f77b700 2 mds.0.56303 boot_start 0: opening inotable -988> 2019-11-12 09:20:08.438 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:1 6.13 6:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -988> 2019-11-12 09:20:08.438 7fe87f77b700 2 mds.0.56303 boot_start 0: opening sessionmap -988> 2019-11-12 09:20:08.438 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:2 6.b 6:d0630e4c:::mds0_sessionmap:head [omap-get-header,omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fcb00 con 0 -988> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: opening mds log -988> 2019-11-12 09:20:08.442 7fe87f77b700 5 mds.0.log open discovering log bounds -988> 2019-11-12 09:20:08.442 7fe87ef7a700 4 mds.0.journalpointer Reading journal pointer '400.00000000' -988> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: opening purge queue (async) -988> 2019-11-12 09:20:08.442 7fe87f77b700 4 mds.0.purge_queue open: opening -988> 2019-11-12 09:20:08.442 7fe87f77b700 1 mds.0.journaler.pq(ro) recover start -988> 2019-11-12 09:20:08.442 7fe87f77b700 1 mds.0.journaler.pq(ro) read_head -988> 2019-11-12 09:20:08.442 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:3 6.f 6:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd8c0 con 0 -988> 2019-11-12 09:20:08.442 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:4 6.55 6:aa448500:::500.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -988> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: loading open file table (async) -988> 2019-11-12 09:20:08.442 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:5 6.26 6:654134d2:::mds0_openfiles.0:head [omap-get-header,omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -988> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: opening snap table -988> 2019-11-12 09:20:08.442 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:6 6.2d 6:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fdb80 con 0 -988> 2019-11-12 09:20:08.442 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:08.442 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:08.442 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:08.446 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 1 0x5574ef7fdb80 osd_op_reply(1 mds0_inotable [read 0~34] v0'0 uv59541 ondisk = 0) v8 -988> 2019-11-12 09:20:08.446 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 1 ==== osd_op_reply(1 mds0_inotable [read 0~34] v0'0 uv59541 ondisk = 0) v8 ==== 157+0+34 (3903532989 0 4146252902) 0x5574ef7fdb80 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.446 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 2 0x5574ef7fdb80 osd_op_reply(6 mds_snaptable [read 0~70] v0'0 uv69169 ondisk = 0) v8 -988> 2019-11-12 09:20:08.446 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 2 ==== osd_op_reply(6 mds_snaptable [read 0~70] v0'0 uv69169 ondisk = 0) v8 ==== 157+0+70 (1873612932 0 336025128) 0x5574ef7fdb80 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.446 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 3 0x5574ef7fdb80 osd_op_reply(2 mds0_sessionmap [omap-get-header,omap-get-vals] v0'0 uv43122 ondisk = 0) v8 -988> 2019-11-12 09:20:08.446 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 3 ==== osd_op_reply(2 mds0_sessionmap [omap-get-header,omap-get-vals] v0'0 uv43122 ondisk = 0) v8 ==== 201+0+6124 (669162857 0 3071192317) 0x5574ef7fdb80 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.514 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 1 0x5574ef7fd8c0 osd_op_reply(3 400.00000000 [read 0~22] v0'0 uv974 ondisk = 0) v8 -988> 2019-11-12 09:20:08.514 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 1 ==== osd_op_reply(3 400.00000000 [read 0~22] v0'0 uv974 ondisk = 0) v8 ==== 156+0+22 (844539098 0 3825830296) 0x5574ef7fd8c0 con 0x5574ef703000 -988> 2019-11-12 09:20:08.514 7fe87ef7a700 1 mds.0.journaler.mdlog(ro) recover start -988> 2019-11-12 09:20:08.514 7fe87ef7a700 1 mds.0.journaler.mdlog(ro) read_head -988> 2019-11-12 09:20:08.514 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:7 6.14 6:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -988> 2019-11-12 09:20:08.514 7fe87ef7a700 4 mds.0.log Waiting for journal 0x200 to recover... -988> 2019-11-12 09:20:08.514 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 1 0x5574ef7fc840 osd_op_reply(4 500.00000000 [read 0~90] v0'0 uv121857 ondisk = 0) v8 -988> 2019-11-12 09:20:08.514 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 1 ==== osd_op_reply(4 500.00000000 [read 0~90] v0'0 uv121857 ondisk = 0) v8 ==== 156+0+90 (446579549 0 2262816634) 0x5574ef7fc840 con 0x5574ef703600 -988> 2019-11-12 09:20:08.518 7fe88077d700 1 mds.0.journaler.pq(ro) _finish_read_head loghead(trim 62914560, expire 64461072, write 64480126, stream_format 1). probing for end of log (from 64480126)... -988> 2019-11-12 09:20:08.518 7fe88077d700 1 mds.0.journaler.pq(ro) probing for end of the log -988> 2019-11-12 09:20:08.518 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:8 6.25 6:a64c6dbf:::500.0000000f:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef810000 con 0 -988> 2019-11-12 09:20:08.518 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:9 6.51 6:8a60f78d:::500.00000010:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -988> 2019-11-12 09:20:08.518 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 4 0x5574ef7fd340 osd_op_reply(7 200.00000000 [read 0~90] v0'0 uv1334401 ondisk = 0) v8 -988> 2019-11-12 09:20:08.518 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 4 ==== osd_op_reply(7 200.00000000 [read 0~90] v0'0 uv1334401 ondisk = 0) v8 ==== 156+0+90 (4261138009 0 1159998223) 0x5574ef7fd340 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.518 7fe87f77b700 1 mds.0.journaler.mdlog(ro) _finish_read_head loghead(trim 33025949696, expire 33036315012, write 33239037805, stream_format 1). probing for end of log (from 33239037805)... -988> 2019-11-12 09:20:08.518 7fe87f77b700 1 mds.0.journaler.mdlog(ro) probing for end of the log -988> 2019-11-12 09:20:08.518 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:10 6.9 6:916b6fb1:::200.00001ef4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -988> 2019-11-12 09:20:08.518 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:11 6.52 6:4b787142:::200.00001ef5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -988> 2019-11-12 09:20:08.518 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 2 0x5574ef7fd8c0 osd_op_reply(9 500.00000010 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:08.518 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 2 ==== osd_op_reply(9 500.00000010 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (2138545714 0 0) 0x5574ef7fd8c0 con 0x5574ef703000 -988> 2019-11-12 09:20:08.518 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 3 0x5574ef7fd8c0 osd_op_reply(11 200.00001ef5 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:08.518 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 3 ==== osd_op_reply(11 200.00001ef5 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (4142874598 0 0) 0x5574ef7fd8c0 con 0x5574ef703000 -988> 2019-11-12 09:20:08.518 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 2 0x5574ef810000 osd_op_reply(8 500.0000000f [stat] v0'0 uv76310 ondisk = 0) v8 -988> 2019-11-12 09:20:08.522 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 2 ==== osd_op_reply(8 500.0000000f [stat] v0'0 uv76310 ondisk = 0) v8 ==== 156+0+16 (1441095022 0 2950776802) 0x5574ef810000 con 0x5574ef703600 -988> 2019-11-12 09:20:08.518 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 5 0x5574ef8102c0 osd_op_reply(10 200.00001ef4 [stat] v0'0 uv43315 ondisk = 0) v8 -988> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 5 ==== osd_op_reply(10 200.00001ef4 [stat] v0'0 uv43315 ondisk = 0) v8 ==== 156+0+16 (586066544 0 3314252265) 0x5574ef8102c0 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.522 7fe88077d700 1 mds.0.journaler.pq(ro) _finish_probe_end write_pos = 64480126 (header had 64480126). recovered. -988> 2019-11-12 09:20:08.522 7fe88077d700 4 mds.0.purge_queue operator(): open complete -988> 2019-11-12 09:20:08.522 7fe88077d700 1 mds.0.journaler.pq(ro) set_writeable -988> 2019-11-12 09:20:08.522 7fe87f77b700 1 mds.0.journaler.mdlog(ro) _finish_probe_end write_pos = 33239061177 (header had 33239037805). recovered. -988> 2019-11-12 09:20:08.522 7fe87ef7a700 4 mds.0.log Journal 0x200 recovered. -988> 2019-11-12 09:20:08.522 7fe87ef7a700 4 mds.0.log Recovered journal 0x200 in format 1 -988> 2019-11-12 09:20:08.522 7fe87ef7a700 2 mds.0.56303 boot_start 1: loading/discovering base inodes -988> 2019-11-12 09:20:08.522 7fe87ef7a700 0 mds.0.cache creating system inode with ino:0x100 -988> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:12 6.33 6:cd5a64a3:::100.00000000:head [getxattr inode] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -988> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:13 6.3f 6:fe07bba1:::100.00000000.inode:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -988> 2019-11-12 09:20:08.522 7fe87ef7a700 0 mds.0.cache creating system inode with ino:0x1 -988> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:14 6.3f 6:ff5b34d6:::1.00000000:head [getxattr inode] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8118c0 con 0 -988> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:15 6.14 6:287034c4:::1.00000000.inode:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811b80 con 0 -988> 2019-11-12 09:20:08.522 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 3 0x5574ef810000 osd_op_reply(12 100.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 -988> 2019-11-12 09:20:08.522 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 3 ==== osd_op_reply(12 100.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 156+0+0 (3193329554 0 0) 0x5574ef810000 con 0x5574ef703600 -988> 2019-11-12 09:20:08.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 6 0x5574ef811b80 osd_op_reply(13 100.00000000.inode [read 0~544] v0'0 uv79071 ondisk = 0) v8 -988> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 6 ==== osd_op_reply(13 100.00000000.inode [read 0~544] v0'0 uv79071 ondisk = 0) v8 ==== 162+0+544 (1288455672 0 1233160524) 0x5574ef811b80 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 7 0x5574ef811b80 osd_op_reply(14 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 -988> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 7 ==== osd_op_reply(14 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 154+0+0 (37684714 0 0) 0x5574ef811b80 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 8 0x5574ef811b80 osd_op_reply(15 1.00000000.inode [read 0~544] v0'0 uv1322881 ondisk = 0) v8 -988> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 8 ==== osd_op_reply(15 1.00000000.inode [read 0~544] v0'0 uv1322881 ondisk = 0) v8 ==== 160+0+544 (2713499666 0 667298071) 0x5574ef811b80 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.522 7fe87f77b700 2 mds.0.56303 boot_start 2: replaying mds log -988> 2019-11-12 09:20:08.522 7fe87f77b700 2 mds.0.56303 boot_start 2: waiting for purge queue recovered -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:16 6.54 6:2be44c64:::200.00001ec4:head [read 1976708~2217596 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:17 6.25 6:a7e44d1f:::200.00001ec5:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:18 6.2c 6:34509b9d:::200.00001ec6:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:19 6.63 6:c60e2601:::200.00001ec7:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:20 6.7 6:e1750875:::200.00001ec8:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fdb80 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:21 6.34 6:2f48296a:::200.00001ec9:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7ad600 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:22 6.35 6:ad314bae:::200.00001eca:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:23 6.56 6:6b0e3b9d:::200.00001ecb:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:24 6.3d 6:bed7c08e:::200.00001ecc:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:25 6.5b 6:dab203d3:::200.00001ecd:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd8c0 con 0 -988> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:26 6.1f 6:f94beeba:::200.00001ece:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8118c0 con 0 -988> 2019-11-12 09:20:08.526 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 4 0x5574ef810000 osd_op_reply(5 mds0_openfiles.0 [omap-get-header,omap-get-vals] v0'0 uv83532 ondisk = 0) v8 -988> 2019-11-12 09:20:08.526 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 4 ==== osd_op_reply(5 mds0_openfiles.0 [omap-get-header,omap-get-vals] v0'0 uv83532 ondisk = 0) v8 ==== 202+0+1949 (4240281736 0 2842386935) 0x5574ef810000 con 0x5574ef703600 -988> 2019-11-12 09:20:08.534 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:08.582 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 1 0x5574ef7fd340 osd_op_reply(19 200.00001ec7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv32228 ondisk = 0) v8 -988> 2019-11-12 09:20:08.582 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 1 ==== osd_op_reply(19 200.00001ec7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv32228 ondisk = 0) v8 ==== 156+0+4194304 (255713159 0 3389336695) 0x5574ef7fd340 con 0x5574ef85c000 -988> 2019-11-12 09:20:08.750 7fe883783700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +111-0 packed 1374) v6 -- 0x5574ef565b00 con 0 -988> 2019-11-12 09:20:08.758 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 4 0x5574ef811600 osd_op_reply(16 200.00001ec4 [read 1976708~2217596 [fadvise_dontneed]] v0'0 uv32403 ondisk = 0) v8 -988> 2019-11-12 09:20:08.758 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 4 ==== osd_op_reply(16 200.00001ec4 [read 1976708~2217596 [fadvise_dontneed]] v0'0 uv32403 ondisk = 0) v8 ==== 156+0+2217596 (3528559444 0 2254027955) 0x5574ef811600 con 0x5574ef703000 -988> 2019-11-12 09:20:08.862 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 9 0x5574ef7fd340 osd_op_reply(23 200.00001ecb [read 0~4194304 [fadvise_dontneed]] v0'0 uv36300 ondisk = 0) v8 -988> 2019-11-12 09:20:08.862 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 9 ==== osd_op_reply(23 200.00001ecb [read 0~4194304 [fadvise_dontneed]] v0'0 uv36300 ondisk = 0) v8 ==== 156+0+4194304 (3043000747 0 150218771) 0x5574ef7fd340 con 0x5574ef702a00 -988> 2019-11-12 09:20:08.982 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 5 0x5574ef810000 osd_op_reply(22 200.00001eca [read 0~4194304 [fadvise_dontneed]] v0'0 uv58091 ondisk = 0) v8 -988> 2019-11-12 09:20:08.986 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 5 ==== osd_op_reply(22 200.00001eca [read 0~4194304 [fadvise_dontneed]] v0'0 uv58091 ondisk = 0) v8 ==== 156+0+4194304 (1537789602 0 1563593434) 0x5574ef810000 con 0x5574ef703600 -988> 2019-11-12 09:20:09.022 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 6 0x5574ef810000 osd_op_reply(17 200.00001ec5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74584 ondisk = 0) v8 -988> 2019-11-12 09:20:09.022 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 6 ==== osd_op_reply(17 200.00001ec5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74584 ondisk = 0) v8 ==== 156+0+4194304 (1536332796 0 4020157265) 0x5574ef810000 con 0x5574ef703600 -988> 2019-11-12 09:20:09.026 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:27 6.5d 6:ba600e0b:::200.00001ecf:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90580 con 0 -988> 2019-11-12 09:20:09.058 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 5 0x5574ef811600 osd_op_reply(20 200.00001ec8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33049 ondisk = 0) v8 -988> 2019-11-12 09:20:09.058 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 5 ==== osd_op_reply(20 200.00001ec8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33049 ondisk = 0) v8 ==== 156+0+4194304 (3747932941 0 3671309179) 0x5574ef811600 con 0x5574ef703000 -988> 2019-11-12 09:20:09.170 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 6 0x5574ef811600 osd_op_reply(18 200.00001ec6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv59925 ondisk = 0) v8 -988> 2019-11-12 09:20:09.170 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 6 ==== osd_op_reply(18 200.00001ec6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv59925 ondisk = 0) v8 ==== 156+0+4194304 (3708667789 0 915352419) 0x5574ef811600 con 0x5574ef703000 -988> 2019-11-12 09:20:09.170 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:28 6.5c 6:3ae49781:::200.00001ed0:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91b80 con 0 -988> 2019-11-12 09:20:09.202 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 7 0x5574efe91b80 osd_op_reply(24 200.00001ecc [read 0~4194304 [fadvise_dontneed]] v0'0 uv88595 ondisk = 0) v8 -988> 2019-11-12 09:20:09.202 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 7 ==== osd_op_reply(24 200.00001ecc [read 0~4194304 [fadvise_dontneed]] v0'0 uv88595 ondisk = 0) v8 ==== 156+0+4194304 (1801352771 0 3873878317) 0x5574efe91b80 con 0x5574ef703000 -988> 2019-11-12 09:20:09.326 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:29 6.2c 6:35096768:::200.00001ed1:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91600 con 0 -988> 2019-11-12 09:20:09.330 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 8 0x5574efe91b80 osd_op_reply(21 200.00001ec9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76305 ondisk = 0) v8 -988> 2019-11-12 09:20:09.334 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 8 ==== osd_op_reply(21 200.00001ec9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76305 ondisk = 0) v8 ==== 156+0+4194304 (4031564057 0 4277249194) 0x5574efe91b80 con 0x5574ef703000 -988> 2019-11-12 09:20:09.534 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 10 0x5574ef7fd340 osd_op_reply(25 200.00001ecd [read 0~4194304 [fadvise_dontneed]] v0'0 uv35517 ondisk = 0) v8 -988> 2019-11-12 09:20:09.534 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 10 ==== osd_op_reply(25 200.00001ecd [read 0~4194304 [fadvise_dontneed]] v0'0 uv35517 ondisk = 0) v8 ==== 156+0+4194304 (1298770794 0 1145050672) 0x5574ef7fd340 con 0x5574ef702a00 -988> 2019-11-12 09:20:09.538 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 7 0x5574efe90580 osd_op_reply(27 200.00001ecf [read 0~4194304 [fadvise_dontneed]] v0'0 uv42371 ondisk = 0) v8 -988> 2019-11-12 09:20:09.538 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 7 ==== osd_op_reply(27 200.00001ecf [read 0~4194304 [fadvise_dontneed]] v0'0 uv42371 ondisk = 0) v8 ==== 156+0+4194304 (1417620671 0 937541238) 0x5574efe90580 con 0x5574ef703600 -988> 2019-11-12 09:20:09.546 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:30 6.39 6:9cc82f3f:::200.00001ed2:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90b00 con 0 -988> 2019-11-12 09:20:09.650 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:31 6.a 6:50660169:::200.00001ed3:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91340 con 0 -988> 2019-11-12 09:20:09.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 9 0x5574efe91b80 osd_op_reply(28 200.00001ed0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv40793 ondisk = 0) v8 -988> 2019-11-12 09:20:09.702 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 9 ==== osd_op_reply(28 200.00001ed0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv40793 ondisk = 0) v8 ==== 156+0+4194304 (4165677614 0 2205248570) 0x5574efe91b80 con 0x5574ef703000 -988> 2019-11-12 09:20:09.738 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:32 6.26 6:642c7156:::200.00001ed4:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -988> 2019-11-12 09:20:09.766 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 10 0x5574efe91b80 osd_op_reply(29 200.00001ed1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv61317 ondisk = 0) v8 -988> 2019-11-12 09:20:09.766 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 10 ==== osd_op_reply(29 200.00001ed1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv61317 ondisk = 0) v8 ==== 156+0+4194304 (2035222751 0 1620268375) 0x5574efe91b80 con 0x5574ef703000 -988> 2019-11-12 09:20:09.822 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:33 6.24 6:26ed0d4a:::200.00001ed5:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0840 con 0 -988> 2019-11-12 09:20:09.902 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:34 6.32 6:4fa4d314:::200.00001ed6:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90dc0 con 0 -988> 2019-11-12 09:20:09.938 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 11 0x5574ef7fd340 osd_op_reply(26 200.00001ece [read 0~4194304 [fadvise_dontneed]] v0'0 uv45520 ondisk = 0) v8 -988> 2019-11-12 09:20:09.938 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 11 ==== osd_op_reply(26 200.00001ece [read 0~4194304 [fadvise_dontneed]] v0'0 uv45520 ondisk = 0) v8 ==== 156+0+4194304 (1747664234 0 852305962) 0x5574ef7fd340 con 0x5574ef702a00 -988> 2019-11-12 09:20:09.994 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:35 6.34 6:2da236a2:::200.00001ed7:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f18c0 con 0 -988> 2019-11-12 09:20:10.070 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:36 6.32 6:4da6cd2c:::200.00001ed8:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90840 con 0 -988> 2019-11-12 09:20:10.150 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:37 6.54 6:2ada770a:::200.00001ed9:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0840 con 0 -988> 2019-11-12 09:20:10.154 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 11 0x5574f17f0840 osd_op_reply(31 200.00001ed3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv34980 ondisk = 0) v8 -988> 2019-11-12 09:20:10.154 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 11 ==== osd_op_reply(31 200.00001ed3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv34980 ondisk = 0) v8 ==== 156+0+4194304 (1386494933 0 3820517588) 0x5574f17f0840 con 0x5574ef703000 -988> 2019-11-12 09:20:10.230 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:38 6.4c 6:33fe30a9:::200.00001eda:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90580 con 0 -988> 2019-11-12 09:20:10.258 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 8 0x5574f17f1340 osd_op_reply(32 200.00001ed4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv80695 ondisk = 0) v8 -988> 2019-11-12 09:20:10.258 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 8 ==== osd_op_reply(32 200.00001ed4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv80695 ondisk = 0) v8 ==== 156+0+4194304 (2138519371 0 3530924944) 0x5574f17f1340 con 0x5574ef703600 -988> 2019-11-12 09:20:10.290 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 12 0x5574f17f0840 osd_op_reply(30 200.00001ed2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90380 ondisk = 0) v8 -988> 2019-11-12 09:20:10.290 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 12 ==== osd_op_reply(30 200.00001ed2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90380 ondisk = 0) v8 ==== 156+0+4194304 (330583361 0 2656011774) 0x5574f17f0840 con 0x5574ef703000 -988> 2019-11-12 09:20:10.310 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:39 6.2f 6:f40177de:::200.00001edb:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0000 con 0 -988> 2019-11-12 09:20:10.386 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:40 6.6 6:601443b2:::200.00001edc:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe902c0 con 0 -988> 2019-11-12 09:20:10.418 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 13 0x5574f17f0840 osd_op_reply(33 200.00001ed5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv68742 ondisk = 0) v8 -988> 2019-11-12 09:20:10.418 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 13 ==== osd_op_reply(33 200.00001ed5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv68742 ondisk = 0) v8 ==== 156+0+4194304 (4128139589 0 1631669900) 0x5574f17f0840 con 0x5574ef703000 -988> 2019-11-12 09:20:10.470 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:41 6.3d 6:bc1d2bb0:::200.00001edd:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0580 con 0 -988> 2019-11-12 09:20:10.562 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:42 6.3d 6:bdbfd417:::200.00001ede:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90dc0 con 0 -988> 2019-11-12 09:20:10.638 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:43 6.3b 6:de40b110:::200.00001edf:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91340 con 0 -988> 2019-11-12 09:20:10.834 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 14 0x5574efe91340 osd_op_reply(34 200.00001ed6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv73340 ondisk = 0) v8 -988> 2019-11-12 09:20:10.834 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 14 ==== osd_op_reply(34 200.00001ed6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv73340 ondisk = 0) v8 ==== 156+0+4194304 (3979275711 0 3536306776) 0x5574efe91340 con 0x5574ef703000 -988> 2019-11-12 09:20:10.838 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:44 6.1c 6:39989f6c:::200.00001ee0:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91b80 con 0 -988> 2019-11-12 09:20:10.862 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 15 0x5574efe91340 osd_op_reply(37 200.00001ed9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33829 ondisk = 0) v8 -988> 2019-11-12 09:20:10.862 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 15 ==== osd_op_reply(37 200.00001ed9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33829 ondisk = 0) v8 ==== 156+0+4194304 (3093391751 0 342787056) 0x5574efe91340 con 0x5574ef703000 -988> 2019-11-12 09:20:10.878 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 2 0x5574efe91b80 osd_op_reply(44 200.00001ee0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv27219 ondisk = 0) v8 -988> 2019-11-12 09:20:10.878 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 2 ==== osd_op_reply(44 200.00001ee0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv27219 ondisk = 0) v8 ==== 156+0+4194304 (2562693100 0 1712739379) 0x5574efe91b80 con 0x5574ef85c000 -988> 2019-11-12 09:20:11.038 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 16 0x5574efe91340 osd_op_reply(35 200.00001ed7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv78116 ondisk = 0) v8 -988> 2019-11-12 09:20:11.038 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 16 ==== osd_op_reply(35 200.00001ed7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv78116 ondisk = 0) v8 ==== 156+0+4194304 (892522787 0 742479174) 0x5574efe91340 con 0x5574ef703000 -988> 2019-11-12 09:20:11.038 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:45 6.39 6:9f1e4369:::200.00001ee1:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1b80 con 0 -988> 2019-11-12 09:20:11.210 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 17 0x5574f17f1b80 osd_op_reply(41 200.00001edd [read 0~4194304 [fadvise_dontneed]] v0'0 uv89661 ondisk = 0) v8 -988> 2019-11-12 09:20:11.210 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 17 ==== osd_op_reply(41 200.00001edd [read 0~4194304 [fadvise_dontneed]] v0'0 uv89661 ondisk = 0) v8 ==== 156+0+4194304 (4104850994 0 1352966362) 0x5574f17f1b80 con 0x5574ef703000 -988> 2019-11-12 09:20:11.386 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 12 0x5574efe91b80 osd_op_reply(39 200.00001edb [read 0~4194304 [fadvise_dontneed]] v0'0 uv78368 ondisk = 0) v8 -988> 2019-11-12 09:20:11.386 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 12 ==== osd_op_reply(39 200.00001edb [read 0~4194304 [fadvise_dontneed]] v0'0 uv78368 ondisk = 0) v8 ==== 156+0+4194304 (827480192 0 133231095) 0x5574efe91b80 con 0x5574ef702a00 -988> 2019-11-12 09:20:11.502 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 18 0x5574f17f1b80 osd_op_reply(36 200.00001ed8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74659 ondisk = 0) v8 -988> 2019-11-12 09:20:11.502 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 18 ==== osd_op_reply(36 200.00001ed8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74659 ondisk = 0) v8 ==== 156+0+4194304 (2417045927 0 23600393) 0x5574f17f1b80 con 0x5574ef703000 -988> 2019-11-12 09:20:11.502 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:46 6.2b 6:d75ef202:::200.00001ee2:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91080 con 0 -988> 2019-11-12 09:20:11.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 13 0x5574efe91b80 osd_op_reply(38 200.00001eda [read 0~4194304 [fadvise_dontneed]] v0'0 uv31579 ondisk = 0) v8 -988> 2019-11-12 09:20:11.526 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 13 ==== osd_op_reply(38 200.00001eda [read 0~4194304 [fadvise_dontneed]] v0'0 uv31579 ondisk = 0) v8 ==== 156+0+4194304 (1167083545 0 1245400058) 0x5574efe91b80 con 0x5574ef702a00 -988> 2019-11-12 09:20:11.554 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 19 0x5574efe91080 osd_op_reply(43 200.00001edf [read 0~4194304 [fadvise_dontneed]] v0'0 uv71835 ondisk = 0) v8 -988> 2019-11-12 09:20:11.554 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 19 ==== osd_op_reply(43 200.00001edf [read 0~4194304 [fadvise_dontneed]] v0'0 uv71835 ondisk = 0) v8 ==== 156+0+4194304 (1016868077 0 408330945) 0x5574efe91080 con 0x5574ef703000 -988> 2019-11-12 09:20:11.582 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:47 6.37 6:ee598252:::200.00001ee3:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90b00 con 0 -988> 2019-11-12 09:20:11.658 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:48 6.38 6:1f27801f:::200.00001ee4:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -988> 2019-11-12 09:20:11.738 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:49 6.26 6:669afea4:::200.00001ee5:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90000 con 0 -988> 2019-11-12 09:20:11.742 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 20 0x5574efe90b00 osd_op_reply(45 200.00001ee1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv91372 ondisk = 0) v8 -988> 2019-11-12 09:20:11.742 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 20 ==== osd_op_reply(45 200.00001ee1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv91372 ondisk = 0) v8 ==== 156+0+4194304 (2648663446 0 2448890689) 0x5574efe90b00 con 0x5574ef703000 -988> 2019-11-12 09:20:11.762 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:replay seq 3 -988> 2019-11-12 09:20:11.762 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:11.762 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:replay seq 3 v56303) v7 -- 0x5574ef6ff880 con 0 -988> 2019-11-12 09:20:11.762 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 14 0x5574ef6ff180 mdsbeacon(1849248/k8s-node-02 up:replay seq 3 v56303) v7 -988> 2019-11-12 09:20:11.762 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 14 ==== mdsbeacon(1849248/k8s-node-02 up:replay seq 3 v56303) v7 ==== 136+0+0 (344388228 0 0) 0x5574ef6ff180 con 0x5574ef700c00 -988> 2019-11-12 09:20:11.762 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:replay seq 3 rtt 0 -988> 2019-11-12 09:20:11.846 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 21 0x5574efe90b00 osd_op_reply(42 200.00001ede [read 0~4194304 [fadvise_dontneed]] v0'0 uv90707 ondisk = 0) v8 -988> 2019-11-12 09:20:11.846 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 21 ==== osd_op_reply(42 200.00001ede [read 0~4194304 [fadvise_dontneed]] v0'0 uv90707 ondisk = 0) v8 ==== 156+0+4194304 (1278859041 0 4166000365) 0x5574efe90b00 con 0x5574ef703000 -988> 2019-11-12 09:20:11.862 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 9 0x5574f2fc2000 osd_op_reply(48 200.00001ee4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv87902 ondisk = 0) v8 -988> 2019-11-12 09:20:11.862 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 9 ==== osd_op_reply(48 200.00001ee4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv87902 ondisk = 0) v8 ==== 156+0+4194304 (1710263652 0 2550500820) 0x5574f2fc2000 con 0x5574ef703600 -988> 2019-11-12 09:20:11.898 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 10 0x5574f2fc2000 osd_op_reply(49 200.00001ee5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv82944 ondisk = 0) v8 -988> 2019-11-12 09:20:11.898 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 10 ==== osd_op_reply(49 200.00001ee5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv82944 ondisk = 0) v8 ==== 156+0+4194304 (2930123064 0 737151539) 0x5574f2fc2000 con 0x5574ef703600 -988> 2019-11-12 09:20:11.954 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 14 0x5574efe91b80 osd_op_reply(40 200.00001edc [read 0~4194304 [fadvise_dontneed]] v0'0 uv31386 ondisk = 0) v8 -988> 2019-11-12 09:20:11.954 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 14 ==== osd_op_reply(40 200.00001edc [read 0~4194304 [fadvise_dontneed]] v0'0 uv31386 ondisk = 0) v8 ==== 156+0+4194304 (1211016059 0 614261651) 0x5574efe91b80 con 0x5574ef702a00 -988> 2019-11-12 09:20:11.954 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:50 6.25 6:a74123a6:::200.00001ee6:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe918c0 con 0 -988> 2019-11-12 09:20:11.974 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 22 0x5574efe90b00 osd_op_reply(46 200.00001ee2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90768 ondisk = 0) v8 -988> 2019-11-12 09:20:11.974 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 22 ==== osd_op_reply(46 200.00001ee2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90768 ondisk = 0) v8 ==== 156+0+4194304 (1649087458 0 600344932) 0x5574efe90b00 con 0x5574ef703000 -988> 2019-11-12 09:20:12.006 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 23 0x5574efe90b00 osd_op_reply(47 200.00001ee3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv70914 ondisk = 0) v8 -988> 2019-11-12 09:20:12.010 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 23 ==== osd_op_reply(47 200.00001ee3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv70914 ondisk = 0) v8 ==== 156+0+4194304 (4189387006 0 3128680832) 0x5574efe90b00 con 0x5574ef703000 -988> 2019-11-12 09:20:12.034 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:51 6.27 6:e7e71572:::200.00001ee7:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7ad600 con 0 -988> 2019-11-12 09:20:12.102 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 11 0x5574efe918c0 osd_op_reply(50 200.00001ee6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76122 ondisk = 0) v8 -988> 2019-11-12 09:20:12.102 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 11 ==== osd_op_reply(50 200.00001ee6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76122 ondisk = 0) v8 ==== 156+0+4194304 (4020623717 0 1562818712) 0x5574efe918c0 con 0x5574ef703600 -988> 2019-11-12 09:20:12.106 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:52 6.3c 6:3e39cde3:::200.00001ee8:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1080 con 0 -988> 2019-11-12 09:20:12.170 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:53 6.8 6:101387a1:::200.00001ee9:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -988> 2019-11-12 09:20:12.198 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 24 0x5574efe918c0 osd_op_reply(51 200.00001ee7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76537 ondisk = 0) v8 -988> 2019-11-12 09:20:12.198 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 24 ==== osd_op_reply(51 200.00001ee7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76537 ondisk = 0) v8 ==== 156+0+4194304 (2339449442 0 2022356555) 0x5574efe918c0 con 0x5574ef703000 -988> 2019-11-12 09:20:12.238 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6802/19981 -- osd_op(unknown.0.56303:54 6.46 6:631b0aa0:::200.00001eea:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fdb80 con 0 -988> 2019-11-12 09:20:12.246 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6802/19981 conn(0x5574ef85c600 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -988> 2019-11-12 09:20:12.306 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:55 6.23 6:c571b352:::200.00001eeb:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91600 con 0 -988> 2019-11-12 09:20:12.326 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6802/19981 conn(0x5574ef85c600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5834 cs=1 l=1). rx osd.2 seq 1 0x5574efe91080 osd_op_reply(54 200.00001eea [read 0~4194304 [fadvise_dontneed]] v0'0 uv43008 ondisk = 0) v8 -988> 2019-11-12 09:20:12.326 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.2 192.168.200.17:6802/19981 1 ==== osd_op_reply(54 200.00001eea [read 0~4194304 [fadvise_dontneed]] v0'0 uv43008 ondisk = 0) v8 ==== 156+0+4194304 (1141206684 0 1871802516) 0x5574efe91080 con 0x5574ef85c600 -988> 2019-11-12 09:20:12.374 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:56 6.3d 6:be55f99a:::200.00001eec:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -988> 2019-11-12 09:20:12.438 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:57 6.15 6:a8763251:::200.00001eed:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fcb00 con 0 -988> 2019-11-12 09:20:12.506 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:58 6.5 6:a03f7ae4:::200.00001eee:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0b00 con 0 -988> 2019-11-12 09:20:12.538 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 15 0x5574f17f1080 osd_op_reply(52 200.00001ee8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv75454 ondisk = 0) v8 -988> 2019-11-12 09:20:12.538 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 15 ==== osd_op_reply(52 200.00001ee8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv75454 ondisk = 0) v8 ==== 156+0+4194304 (1339885820 0 2582243991) 0x5574f17f1080 con 0x5574ef702a00 -988> 2019-11-12 09:20:12.570 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:59 6.27 6:e42c9344:::200.00001eef:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1600 con 0 -988> 2019-11-12 09:20:12.582 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 12 0x5574efe91600 osd_op_reply(53 200.00001ee9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv37098 ondisk = 0) v8 -988> 2019-11-12 09:20:12.582 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 12 ==== osd_op_reply(53 200.00001ee9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv37098 ondisk = 0) v8 ==== 156+0+4194304 (3031359962 0 2756808336) 0x5574efe91600 con 0x5574ef703600 -988> 2019-11-12 09:20:12.638 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:60 6.3e 6:7d6c9823:::200.00001ef0:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90580 con 0 -988> 2019-11-12 09:20:12.702 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:61 6.45 6:a3b8f91c:::200.00001ef1:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90000 con 0 -988> 2019-11-12 09:20:12.766 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:62 6.50 6:0af53e21:::200.00001ef2:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1340 con 0 -988> 2019-11-12 09:20:12.802 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 3 0x5574efe902c0 osd_op_reply(62 200.00001ef2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv28322 ondisk = 0) v8 -988> 2019-11-12 09:20:12.802 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 3 ==== osd_op_reply(62 200.00001ef2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv28322 ondisk = 0) v8 ==== 156+0+4194304 (1122511463 0 79703917) 0x5574efe902c0 con 0x5574ef85c000 -988> 2019-11-12 09:20:12.834 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:63 6.20 6:059200a5:::200.00001ef3:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810000 con 0 -988> 2019-11-12 09:20:12.902 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:64 6.9 6:916b6fb1:::200.00001ef4:head [read 0~3396281 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -988> 2019-11-12 09:20:13.398 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 25 0x5574f17f1600 osd_op_reply(55 200.00001eeb [read 0~4194304 [fadvise_dontneed]] v0'0 uv35600 ondisk = 0) v8 -988> 2019-11-12 09:20:13.398 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 25 ==== osd_op_reply(55 200.00001eeb [read 0~4194304 [fadvise_dontneed]] v0'0 uv35600 ondisk = 0) v8 ==== 156+0+4194304 (3905789527 0 2657527065) 0x5574f17f1600 con 0x5574ef703000 -988> 2019-11-12 09:20:13.586 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 26 0x5574f17f1600 osd_op_reply(56 200.00001eec [read 0~4194304 [fadvise_dontneed]] v0'0 uv93869 ondisk = 0) v8 -988> 2019-11-12 09:20:13.586 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 26 ==== osd_op_reply(56 200.00001eec [read 0~4194304 [fadvise_dontneed]] v0'0 uv93869 ondisk = 0) v8 ==== 156+0+4194304 (43855582 0 2499621118) 0x5574f17f1600 con 0x5574ef703000 -988> 2019-11-12 09:20:13.710 7fe884785700 10 monclient: tick -988> 2019-11-12 09:20:13.710 7fe884785700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-11-12 09:19:43.714163) -988> 2019-11-12 09:20:13.750 7fe883783700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +0-0 packed 1374) v6 -- 0x5574f06ce000 con 0 -988> 2019-11-12 09:20:13.826 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 27 0x5574f17f1600 osd_op_reply(59 200.00001eef [read 0~4194304 [fadvise_dontneed]] v0'0 uv81570 ondisk = 0) v8 -988> 2019-11-12 09:20:13.826 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 27 ==== osd_op_reply(59 200.00001eef [read 0~4194304 [fadvise_dontneed]] v0'0 uv81570 ondisk = 0) v8 ==== 156+0+4194304 (473818481 0 1401989426) 0x5574f17f1600 con 0x5574ef703000 -988> 2019-11-12 09:20:14.462 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 28 0x5574f17f1600 osd_op_reply(58 200.00001eee [read 0~4194304 [fadvise_dontneed]] v0'0 uv42535 ondisk = 0) v8 -988> 2019-11-12 09:20:14.462 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 28 ==== osd_op_reply(58 200.00001eee [read 0~4194304 [fadvise_dontneed]] v0'0 uv42535 ondisk = 0) v8 ==== 156+0+4194304 (2933939241 0 808578468) 0x5574f17f1600 con 0x5574ef703000 -988> 2019-11-12 09:20:14.538 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 29 0x5574f17f1600 osd_op_reply(57 200.00001eed [read 0~4194304 [fadvise_dontneed]] v0'0 uv37336 ondisk = 0) v8 -988> 2019-11-12 09:20:14.538 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 29 ==== osd_op_reply(57 200.00001eed [read 0~4194304 [fadvise_dontneed]] v0'0 uv37336 ondisk = 0) v8 ==== 156+0+4194304 (910772900 0 1749242063) 0x5574f17f1600 con 0x5574ef703000 -988> 2019-11-12 09:20:14.650 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 16 0x5574f06fe000 osd_op_reply(60 200.00001ef0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv71620 ondisk = 0) v8 -988> 2019-11-12 09:20:14.650 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 16 ==== osd_op_reply(60 200.00001ef0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv71620 ondisk = 0) v8 ==== 156+0+4194304 (916184623 0 823393747) 0x5574f06fe000 con 0x5574ef702a00 -988> 2019-11-12 09:20:14.878 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 17 0x5574f06fe000 osd_op_reply(61 200.00001ef1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv60049 ondisk = 0) v8 -988> 2019-11-12 09:20:14.878 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 17 ==== osd_op_reply(61 200.00001ef1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv60049 ondisk = 0) v8 ==== 156+0+4194304 (2574875128 0 582962543) 0x5574f06fe000 con 0x5574ef702a00 -988> 2019-11-12 09:20:15.762 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:replay seq 4 -988> 2019-11-12 09:20:15.762 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:15.762 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:replay seq 4 v56303) v7 -- 0x5574ef80f180 con 0 -988> 2019-11-12 09:20:15.762 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 15 0x5574ef80f180 mdsbeacon(1849248/k8s-node-02 up:replay seq 4 v56303) v7 -988> 2019-11-12 09:20:15.762 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 15 ==== mdsbeacon(1849248/k8s-node-02 up:replay seq 4 v56303) v7 ==== 136+0+0 (3656543233 0 0) 0x5574ef80f180 con 0x5574ef700c00 -988> 2019-11-12 09:20:15.762 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:replay seq 4 rtt 0 -988> 2019-11-12 09:20:16.406 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 18 0x5574f06fe000 osd_op_reply(64 200.00001ef4 [read 0~3396281 [fadvise_dontneed]] v0'0 uv43315 ondisk = 0) v8 -988> 2019-11-12 09:20:16.406 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 18 ==== osd_op_reply(64 200.00001ef4 [read 0~3396281 [fadvise_dontneed]] v0'0 uv43315 ondisk = 0) v8 ==== 156+0+3396281 (2978750866 0 537567138) 0x5574f06fe000 con 0x5574ef702a00 -988> 2019-11-12 09:20:16.482 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 19 0x5574f06fe000 osd_op_reply(63 200.00001ef3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33434 ondisk = 0) v8 -988> 2019-11-12 09:20:16.482 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 19 ==== osd_op_reply(63 200.00001ef3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33434 ondisk = 0) v8 ==== 156+0+4194304 (2400805555 0 3258534103) 0x5574f06fe000 con 0x5574ef702a00 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.56303 Finished replaying journal -988> 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.56303 making mds journal writeable -988> 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.journaler.mdlog(ro) set_writeable -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:65 6.9 6:916b6fb1:::200.00001ef4:head [zero 3396281~798023] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:66 6.52 6:4b787142:::200.00001ef5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:67 6.3b 6:dd49a9ad:::200.00001ef6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef8118c0 con 0 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:68 6.d 6:b10d4b81:::200.00001ef7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:69 6.2a 6:55d3aaa4:::200.00001ef8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:70 6.15 6:a9ceb397:::200.00001ef9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -988> 2019-11-12 09:20:16.610 7fe87df78700 2 mds.0.56303 i am alone, moving to state reconnect -988> 2019-11-12 09:20:16.610 7fe87df78700 3 mds.0.56303 request_state up:reconnect -988> 2019-11-12 09:20:16.610 7fe87df78700 5 mds.beacon.k8s-node-02 set_want_state: up:replay -> up:reconnect -988> 2019-11-12 09:20:16.610 7fe87df78700 5 mds.beacon.k8s-node-02 Sending beacon up:reconnect seq 5 -988> 2019-11-12 09:20:16.610 7fe87df78700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:reconnect seq 5 v56303) v7 -- 0x5574ef80fc00 con 0 -988> 2019-11-12 09:20:16.634 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 30 0x5574ef811600 osd_op_reply(66 200.00001ef5 [delete] v28661'40770 uv40694 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:16.634 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 30 ==== osd_op_reply(66 200.00001ef5 [delete] v28661'40770 uv40694 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (3875959471 0 0) 0x5574ef811600 con 0x5574ef703000 -988> 2019-11-12 09:20:16.634 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 31 0x5574ef811600 osd_op_reply(70 200.00001ef9 [delete] v28661'37440 uv37342 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:16.634 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 31 ==== osd_op_reply(70 200.00001ef9 [delete] v28661'37440 uv37342 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (28087247 0 0) 0x5574ef811600 con 0x5574ef703000 -988> 2019-11-12 09:20:16.638 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 32 0x5574ef811600 osd_op_reply(67 200.00001ef6 [delete] v28661'73070 uv72994 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:16.638 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 32 ==== osd_op_reply(67 200.00001ef6 [delete] v28661'73070 uv72994 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (4041088188 0 0) 0x5574ef811600 con 0x5574ef703000 -988> 2019-11-12 09:20:16.638 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 20 0x5574f17f02c0 osd_op_reply(65 200.00001ef4 [zero 3396281~798023] v28661'43316 uv43315 ondisk = 0) v8 -988> 2019-11-12 09:20:16.638 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 20 ==== osd_op_reply(65 200.00001ef4 [zero 3396281~798023] v28661'43316 uv43315 ondisk = 0) v8 ==== 156+0+0 (106571773 0 0) 0x5574f17f02c0 con 0x5574ef702a00 -988> 2019-11-12 09:20:16.638 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 13 0x5574f17f1b80 osd_op_reply(69 200.00001ef8 [delete] v28661'80702 uv80626 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:16.638 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 13 ==== osd_op_reply(69 200.00001ef8 [delete] v28661'80702 uv80626 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (1754869307 0 0) 0x5574f17f1b80 con 0x5574ef703600 -988> 2019-11-12 09:20:16.638 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 14 0x5574f17f1b80 osd_op_reply(68 200.00001ef7 [delete] v28661'38268 uv37712 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:16.638 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 14 ==== osd_op_reply(68 200.00001ef7 [delete] v28661'38268 uv37712 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (699459646 0 0) 0x5574f17f1b80 con 0x5574ef703600 -988> 2019-11-12 09:20:16.682 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 16 0x5574f1198b40 mdsmap(e 56304) v1 -988> 2019-11-12 09:20:16.686 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef700600 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=32 - -988> 2019-11-12 09:20:16.686 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 16 ==== mdsmap(e 56304) v1 ==== 655+0+0 (1257921563 0 0) 0x5574f1198b40 con 0x5574ef700c00 -988> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56304 from mon.0 -988> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -988> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:replay --> up:reconnect -988> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 reconnect_start -988> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 reopen_log -988> 2019-11-12 09:20:16.686 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 17 0x5574ef80f180 mdsbeacon(1849248/k8s-node-02 up:reconnect seq 5 v56304) v7 -988> 2019-11-12 09:20:16.686 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 17 ==== mdsbeacon(1849248/k8s-node-02 up:reconnect seq 5 v56304) v7 ==== 136+0+0 (1350244580 0 0) 0x5574ef80f180 con 0x5574ef700c00 -988> 2019-11-12 09:20:16.686 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:reconnect seq 5 rtt 0.0760015 -988> 2019-11-12 09:20:16.686 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85d200 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=34 - -988> 2019-11-12 09:20:16.686 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85cc00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=33 - -988> 2019-11-12 09:20:16.690 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85de00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=36 - -988> 2019-11-12 09:20:16.694 7fe885a11700 4 mds.0.56303 reconnect_start: killed 0 blacklisted sessions (2132 blacklist entries, 14) -988> 2019-11-12 09:20:16.694 7fe885a11700 1 mds.0.server reconnect_clients -- 14 sessions -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 _send_message--> mds.0 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable server_ready) v1 -- ?+0 0x5574ef7c8b40 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable server_ready) v1 -- 0x5574ef7c8b40 con 0 -988> 2019-11-12 09:20:16.694 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85d800 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=35 - -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mds.0 192.168.200.19:6800/1460470921 0 ==== mds_table_request(snaptable server_ready) v1 ==== 0+0+0 (0 0 0) 0x5574ef7c8b40 con 0x5574ef701800 -988> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/979484066 conn(0x5574ef700600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4844 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.694 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:0/2796926812 conn(0x5574ef85de00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.694 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:0/3102681411 conn(0x5574ef85d200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/3544368504 conn(0x5574ef85d800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/3544368504 conn(0x5574ef85d800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 _send_message--> mds.0 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable query 1 9 bytes) v1 -- ?+0 0x5574f1198b40 -988> 2019-11-12 09:20:16.694 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:0/3102681411 conn(0x5574ef85d200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/979484066 conn(0x5574ef700600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4844 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable query 1 9 bytes) v1 -- 0x5574f1198b40 con 0 -988> 2019-11-12 09:20:16.694 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/979484066 conn(0x5574ef700600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4844 cs=1 l=0). rx client.1804200 seq 1 0x5574f1198900 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.694 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:0/3102681411 conn(0x5574ef85d200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804134 seq 1 0x5574f11986c0 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.694 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:0/2796926812 conn(0x5574ef85de00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.694 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/3544368504 conn(0x5574ef85d800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804143 seq 1 0x5574f1198fc0 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.694 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:0/2796926812 conn(0x5574ef85de00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804137 seq 1 0x5574ef7a7200 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804200 192.168.200.30:0/979484066 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1014479821) 0x5574f1198900 con 0x5574ef700600 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/979484066 -- client_session(open) v3 -- 0x5574ef7c8b40 con 0 -988> 2019-11-12 09:20:16.694 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85e400 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=37 - -988> 2019-11-12 09:20:16.694 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85f000 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=39 - -988> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804200 192.168.200.30:0/979484066 after 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804134 192.168.200.17:0/3102681411 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3890766099) 0x5574f11986c0 con 0x5574ef85d200 -988> 2019-11-12 09:20:16.694 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dc600 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=42 - -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:0/3102681411 -- client_session(open) v3 -- 0x5574f1198900 con 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804134 192.168.200.17:0/3102681411 after 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804143 192.168.200.31:0/3544368504 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 2998347668) 0x5574f1198fc0 con 0x5574ef85d800 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/3544368504 -- client_session(open) v3 -- 0x5574f11986c0 con 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804143 192.168.200.31:0/3544368504 after 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804137 192.168.200.19:0/2796926812 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1366303579) 0x5574ef7a7200 con 0x5574ef85de00 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:0/2796926812 -- client_session(open) v3 -- 0x5574f1198fc0 con 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804137 192.168.200.19:0/2796926812 after 0 -988> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mds.0 192.168.200.19:6800/1460470921 0 ==== mds_table_request(snaptable query 1 9 bytes) v1 ==== 0+0+0 (0 0 0) 0x5574f1198b40 con 0x5574ef701800 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable query_reply 1 tid 2 29 bytes) v1 -- 0x5574ef7a7200 con 0 -988> 2019-11-12 09:20:16.698 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85f600 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=40 - -988> 2019-11-12 09:20:16.698 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dcc00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=43 - -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mds.0 192.168.200.19:6800/1460470921 0 ==== mds_table_request(snaptable query_reply 1 tid 2 29 bytes) v1 ==== 0+0+0 (0 0 0) 0x5574ef7a7200 con 0x5574ef701800 -988> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4219 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1793818 seq 1 0x5574ef7c9440 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793818 192.168.200.30:0/232332927 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3089022758) 0x5574ef7c9440 con 0x5574ef85e400 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/232332927 -- client_session(open) v3 -- 0x5574ef7a7200 con 0 -988> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/2855972843 conn(0x5574ef85cc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/2855972843 conn(0x5574ef85cc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793818 192.168.200.30:0/232332927 after 0.00400008 -988> 2019-11-12 09:20:16.698 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85ea00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=38 - -988> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=7530 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dc000 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=41 - -988> 2019-11-12 09:20:16.698 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/2855972843 conn(0x5574ef85cc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804140 seq 1 0x5574ef8646c0 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4292 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4219 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4219 cs=1 l=0). rx client.1792815 seq 1 0x5574f06ce900 client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4219 cs=1 l=0). rx client.1792815 seq 2 0x5574f1199200 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5028 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4292 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4292 cs=1 l=0). rx client.1794662 seq 1 0x5574f06ce600 client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4292 cs=1 l=0). rx client.1794662 seq 2 0x5574f06cec00 client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4292 cs=1 l=0). rx client.1794662 seq 3 0x5574f1199440 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804140 192.168.200.32:0/2855972843 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3745454508) 0x5574ef8646c0 con 0x5574ef85cc00 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/2855972843 -- client_session(open) v3 -- 0x5574ef7c9440 con 0 -988> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=7530 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5362 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 1 0x5574f06ce000 client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 2 0x5574ef565b00 client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 3 0x5574f06cf200 client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 4 0x5574f06cf500 client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 5 0x5574f06cf800 client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804140 192.168.200.32:0/2855972843 after 0.00400008 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 6 0x5574f06cfb00 client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 7 0x5574f11f0000 client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 8 0x5574f11f0300 client_request(client.1794521:96231 lookup #0x100013ad07a/eula.txt 2019-11-12 08:59:14.149366 RETRY=30 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 9 0x5574f11f0600 client_request(client.1794521:96232 lookup #0x100013ad07a/eula.txt 2019-11-12 09:05:30.072151 RETRY=10 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1792815 192.168.200.32:0/3593333399 1 ==== client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 122+0+0 (1097225393 0 0) 0x5574f06ce900 con 0x5574ef85f000 -988> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 10 0x5574f11f0900 client_request(client.1794521:96233 lookup #0x100013ad07a/eula.txt 2019-11-12 09:06:44.117495 RETRY=8 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 11 0x5574f11f0c00 client_request(client.1794521:96236 lookup #0x100013ad07a/eula.txt 2019-11-12 09:17:08.464874 RETRY=2 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 1 ==== client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2264195835 0 0) 0x5574f06ce000 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 12 0x5574f11f0f00 client_request(client.1794521:96238 lookup #0x100013ad07a/eula.txt 2019-11-12 09:18:43.942618 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 13 0x5574f11f1200 client_request(client.1794521:96239 getattr As #0x100013ad07a 2019-11-12 09:19:36.755583 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 14 0x5574ef864240 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794662 192.168.200.30:0/522103062 1 ==== client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 ==== 122+0+0 (231600759 0 0) 0x5574f06ce600 con 0x5574f11dc600 -988> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1792815 192.168.200.32:0/3593333399 2 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1066103343) 0x5574f1199200 con 0x5574ef85f000 -988> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5028 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/3593333399 -- client_session(open) v3 -- 0x5574ef8646c0 con 0 -988> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/2985278080 conn(0x5574f11dc000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4285 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 1 0x5574f11f1500 client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 2 0x5574f11f1800 client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 3 0x5574f11f1b00 client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 4 0x5574f11fa000 client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1792815 192.168.200.32:0/3593333399 after 0.00400008 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 5 0x5574f11fa300 client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 6 0x5574f11fa600 client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 7 0x5574f11fa900 client_request(client.1794665:23347 lookup #0x10001346679/public 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 2 ==== client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (4027415736 0 0) 0x5574ef565b00 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 8 0x5574f11fac00 client_request(client.1794665:23348 lookup #0x1000134691b/autoload.php 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 9 0x5574f11faf00 client_request(client.1794665:23349 lookup #0x10001346679/private 2019-11-12 09:18:41.457643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 10 0x5574ef864000 client_reconnect(8 caps) v3 -988> 2019-11-12 09:20:16.702 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/2985278080 conn(0x5574f11dc000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4285 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/2985278080 conn(0x5574f11dc000 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4285 cs=1 l=0). rx client.1794668 seq 1 0x5574ef7a6900 client_reconnect(1 caps) v3 -988> 2019-11-12 09:20:16.702 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5362 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 1 0x5574f11fb500 client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 2 0x5574f11fb800 client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 3 0x5574f1198000 client_reconnect(10 caps) v3 -988> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794662 192.168.200.30:0/522103062 2 ==== client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 ==== 122+0+0 (906112753 0 0) 0x5574f06cec00 con 0x5574f11dc600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 1 ==== client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 ==== 128+0+0 (1435394209 0 0) 0x5574f11f1500 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793737 192.168.200.31:0/2398232289 1 ==== client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 ==== 122+0+0 (3576974081 0 0) 0x5574f11fb500 con 0x5574ef85ea00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 3 ==== client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (955679367 0 0) 0x5574f06cf200 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794668 192.168.200.30:0/2985278080 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3819407093) 0x5574ef7a6900 con 0x5574f11dc000 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/2985278080 -- client_session(open) v3 -- 0x5574f1199200 con 0 -988> 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794668 192.168.200.30:0/2985278080 after 0.00800016 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794662 192.168.200.30:0/522103062 3 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 2460232887) 0x5574f1199440 con 0x5574f11dc600 -988> 2019-11-12 09:20:16.702 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dd200 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=44 - -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/522103062 -- client_session(open) v3 -- 0x5574ef7a6900 con 0 -988> 2019-11-12 09:20:16.702 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dd800 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=45 - -988> 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794662 192.168.200.30:0/522103062 after 0.00800016 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 2 ==== client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (1022837727 0 0) 0x5574f11f1800 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793737 192.168.200.31:0/2398232289 2 ==== client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 ==== 124+0+0 (564909887 0 0) 0x5574f11fb800 con 0x5574ef85ea00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4244 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 4 ==== client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2539636667 0 0) 0x5574f06cf500 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4953 cs=1 l=0).handle_connect_msg accept write reply msg done -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 3 ==== client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (3472455810 0 0) 0x5574f11f1b00 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793737 192.168.200.31:0/2398232289 3 ==== client_reconnect(10 caps) v3 ==== 0+0+972 (0 0 3487394893) 0x5574f1198000 con 0x5574ef85ea00 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_session(open) v3 -- 0x5574f1199440 con 0 -988> 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793737 192.168.200.31:0/2398232289 after 0.00800016 -988> 2019-11-12 09:20:16.702 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4244 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.702 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4953 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 5 ==== client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2901537372 0 0) 0x5574f06cf800 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 4 ==== client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (134929482 0 0) 0x5574f11fa000 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 6 ==== client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2220845138 0 0) 0x5574f06cfb00 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 5 ==== client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 122+0+0 (3291907751 0 0) 0x5574f11fa300 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 1 0x5574f06cef00 client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.702 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4244 cs=1 l=0). rx client.1793815 seq 1 0x5574f11fb200 client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 7 ==== client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (305688313 0 0) 0x5574f11f0000 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 6 ==== client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (873794425 0 0) 0x5574f11fa600 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 2 0x5574f1208000 client_request(client.1793728:9471 lookup #0x1000136753a/mime_types.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4244 cs=1 l=0). rx client.1793815 seq 2 0x5574ef52b680 client_reconnect(7 caps) v3 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 1 ==== client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 136+0+0 (3106383072 0 0) 0x5574f06cef00 con 0x5574f11dd200 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 3 0x5574f1208300 client_request(client.1793728:9472 lookup #0x1000136753a/spf_dkim_whitelist.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793815 192.168.200.15:0/4201708698 1 ==== client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 ==== 168+0+0 (1137596064 0 0) 0x5574f11fb200 con 0x5574f11dd800 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 4 0x5574f1208600 client_request(client.1793728:9473 lookup #0x1000136753a/surbl-whitelist.inc.local 2019-11-12 09:18:41.736891 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 8 ==== client_request(client.1794521:96231 lookup #0x100013ad07a/eula.txt 2019-11-12 08:59:14.149366 RETRY=30 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (791300380 0 0) 0x5574f11f0300 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 5 0x5574f1208900 client_request(client.1793728:9474 getattr As #0x1000136753a 2019-11-12 09:19:29.005938 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 6 0x5574f1199680 client_reconnect(10 caps) v3 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 7 ==== client_request(client.1794665:23347 lookup #0x10001346679/public 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 128+0+0 (1953170597 0 0) 0x5574f11fa900 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 2 ==== client_request(client.1793728:9471 lookup #0x1000136753a/mime_types.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 142+0+0 (3893675223 0 0) 0x5574f1208000 con 0x5574f11dd200 -988> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793815 192.168.200.15:0/4201708698 2 ==== client_reconnect(7 caps) v3 ==== 0+0+660 (0 0 427299514) 0x5574ef52b680 con 0x5574f11dd800 -988> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/4201708698 -- client_session(open) v3 -- 0x5574f1198000 con 0 -988> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793815 192.168.200.15:0/4201708698 after 0.00800016 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 9 ==== client_request(client.1794521:96232 lookup #0x100013ad07a/eula.txt 2019-11-12 09:05:30.072151 RETRY=10 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (3260212025 0 0) 0x5574f11f0600 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 8 ==== client_request(client.1794665:23348 lookup #0x1000134691b/autoload.php 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 134+0+0 (2222831593 0 0) 0x5574f11fac00 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 3 ==== client_request(client.1793728:9472 lookup #0x1000136753a/spf_dkim_whitelist.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 150+0+0 (3844774369 0 0) 0x5574f1208300 con 0x5574f11dd200 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 10 ==== client_request(client.1794521:96233 lookup #0x100013ad07a/eula.txt 2019-11-12 09:06:44.117495 RETRY=8 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (1788368277 0 0) 0x5574f11f0900 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 9 ==== client_request(client.1794665:23349 lookup #0x10001346679/private 2019-11-12 09:18:41.457643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 129+0+0 (575778027 0 0) 0x5574f11faf00 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 4 ==== client_request(client.1793728:9473 lookup #0x1000136753a/surbl-whitelist.inc.local 2019-11-12 09:18:41.736891 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 147+0+0 (166254165 0 0) 0x5574f1208600 con 0x5574f11dd200 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 11 ==== client_request(client.1794521:96236 lookup #0x100013ad07a/eula.txt 2019-11-12 09:17:08.464874 RETRY=2 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2076245363 0 0) 0x5574f11f0c00 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 10 ==== client_reconnect(8 caps) v3 ==== 0+0+631 (0 0 907812246) 0x5574ef864000 con 0x5574f11dcc00 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_session(open) v3 -- 0x5574ef52b680 con 0 -988> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794665 192.168.200.30:0/3506497893 after 0.0120002 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 5 ==== client_request(client.1793728:9474 getattr As #0x1000136753a 2019-11-12 09:19:29.005938 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 122+0+0 (4065490981 0 0) 0x5574f1208900 con 0x5574f11dd200 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 12 ==== client_request(client.1794521:96238 lookup #0x100013ad07a/eula.txt 2019-11-12 09:18:43.942618 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (1427172407 0 0) 0x5574f11f0f00 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 6 ==== client_reconnect(10 caps) v3 ==== 0+0+1051 (0 0 3070811223) 0x5574f1199680 con 0x5574f11dd200 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_session(open) v3 -- 0x5574ef864000 con 0 -988> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793728 192.168.200.15:0/1656050499 after 0.0120002 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 13 ==== client_request(client.1794521:96239 getattr As #0x100013ad07a 2019-11-12 09:19:36.755583 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 122+0+0 (376921402 0 0) 0x5574f11f1200 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 14 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1282897092) 0x5574ef864240 con 0x5574ef85f600 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/1453103486 -- client_session(open) v3 -- 0x5574f1199680 con 0 -988> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794521 192.168.200.31:0/1453103486 after 0.0120002 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 mds.0.56303 reconnect_done -988> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.56303 request_state up:rejoin -988> 2019-11-12 09:20:16.706 7fe885a11700 5 mds.beacon.k8s-node-02 set_want_state: up:reconnect -> up:rejoin -988> 2019-11-12 09:20:16.706 7fe885a11700 5 mds.beacon.k8s-node-02 Sending beacon up:rejoin seq 6 -988> 2019-11-12 09:20:16.706 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:rejoin seq 6 v56304) v7 -- 0x5574f1216000 con 0 -988> 2019-11-12 09:20:17.710 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 18 0x5574ef864000 mdsmap(e 56305) v1 -988> 2019-11-12 09:20:17.710 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 18 ==== mdsmap(e 56305) v1 ==== 655+0+0 (1964398294 0 0) 0x5574ef864000 con 0x5574ef700c00 -988> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56305 from mon.0 -988> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -988> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:reconnect --> up:rejoin -988> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 rejoin_start -988> 2019-11-12 09:20:17.710 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 19 0x5574f1216000 mdsbeacon(1849248/k8s-node-02 up:rejoin seq 6 v56305) v7 -988> 2019-11-12 09:20:17.710 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 19 ==== mdsbeacon(1849248/k8s-node-02 up:rejoin seq 6 v56305) v7 ==== 136+0+0 (4245662320 0 0) 0x5574f1216000 con 0x5574ef700c00 -988> 2019-11-12 09:20:17.710 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:71 6.3f 6:ff5b34d6:::1.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -988> 2019-11-12 09:20:17.710 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:rejoin seq 6 rtt 1.00402 -988> 2019-11-12 09:20:17.710 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:72 6.5e 6:7a303e59:::1000001a3c6.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:73 6.b 6:d0f90662:::1000001eb17.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811b80 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:74 6.56 6:6bacae67:::1000001ffea.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:75 6.1e 6:7992850e:::1000133e06d.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f12262c0 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6802/19981 -- osd_op(unknown.0.56303:76 6.46 6:62644040:::10001366c72.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226840 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:77 6.5b 6:db1def57:::10001367140.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226dc0 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:78 6.43 6:c26089c3:::100013671a4.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0dc0 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:79 6.1c 6:38b3e7b3:::1000136753a.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7ad600 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:80 6.34 6:2c8bdca0:::100013ad077.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:81 6.50 6:0a153a6f:::100013ad078.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:82 6.26 6:66061dfa:::100013ad079.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226580 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:83 6.3d 6:bfa6ab4a:::100013b1605.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1227340 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:84 6.43 6:c3272da7:::100013b1606.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0000 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:85 6.2c 6:36da4b25:::100013b1607.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0840 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:86 6.55 6:abaeca41:::30000015b82.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1b80 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:87 6.4 6:20b05573:::30000015ba9.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f12278c0 con 0 -988> 2019-11-12 09:20:17.714 7fe885a11700 1 mds.0.56303 rejoin_joint_start -988> 2019-11-12 09:20:17.714 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 21 0x5574ef7fd600 osd_op_reply(71 1.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv79171 ondisk = 0) v8 -988> 2019-11-12 09:20:17.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 33 0x5574f1227340 osd_op_reply(80 100013ad077.00000000 [omap-get-header,omap-get-vals,getxattr (69)] v0'0 uv78120 ondisk = 0) v8 -988> 2019-11-12 09:20:17.714 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 33 ==== osd_op_reply(80 100013ad077.00000000 [omap-get-header,omap-get-vals,getxattr (69)] v0'0 uv78120 ondisk = 0) v8 ==== 248+0+827 (1746941155 0 2754020800) 0x5574f1227340 con 0x5574ef703000 -988> 2019-11-12 09:20:17.714 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 21 ==== osd_op_reply(71 1.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv79171 ondisk = 0) v8 ==== 238+0+17357 (3984365710 0 1328600032) 0x5574ef7fd600 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 34 0x5574f17f1080 osd_op_reply(83 100013b1605.00000000 [omap-get-header,omap-get-vals,getxattr (131)] v0'0 uv88597 ondisk = 0) v8 -988> 2019-11-12 09:20:17.714 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 34 ==== osd_op_reply(83 100013b1605.00000000 [omap-get-header,omap-get-vals,getxattr (131)] v0'0 uv88597 ondisk = 0) v8 ==== 248+0+898 (279877274 0 277830296) 0x5574f17f1080 con 0x5574ef703000 -988> 2019-11-12 09:20:17.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6802/19981 conn(0x5574ef85c600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5834 cs=1 l=1). rx osd.2 seq 2 0x5574f17f0840 osd_op_reply(76 10001366c72.00000000 [omap-get-header,omap-get-vals,getxattr (135)] v0'0 uv41822 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.2 192.168.200.17:6802/19981 2 ==== osd_op_reply(76 10001366c72.00000000 [omap-get-header,omap-get-vals,getxattr (135)] v0'0 uv41822 ondisk = 0) v8 ==== 248+0+6454 (1231371730 0 289533519) 0x5574f17f0840 con 0x5574ef85c600 -988> 2019-11-12 09:20:17.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 15 0x5574f17f1b80 osd_op_reply(75 1000133e06d.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv32888 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 15 ==== osd_op_reply(75 1000133e06d.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv32888 ondisk = 0) v8 ==== 248+0+13413 (2612537738 0 1454472965) 0x5574f17f1b80 con 0x5574ef703600 -988> 2019-11-12 09:20:17.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 16 0x5574f1227600 osd_op_reply(82 100013ad079.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv79212 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 16 ==== osd_op_reply(82 100013ad079.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv79212 ondisk = 0) v8 ==== 248+0+823 (707074122 0 2606661828) 0x5574f1227600 con 0x5574ef703600 -988> 2019-11-12 09:20:17.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 17 0x5574f17f1b80 osd_op_reply(86 30000015b82.00000000 [omap-get-header,omap-get-vals,getxattr (122)] v0'0 uv118734 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 17 ==== osd_op_reply(86 30000015b82.00000000 [omap-get-header,omap-get-vals,getxattr (122)] v0'0 uv118734 ondisk = 0) v8 ==== 248+0+8987 (973910231 0 4267468738) 0x5574f17f1b80 con 0x5574ef703600 -988> 2019-11-12 09:20:17.718 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 22 0x5574ef7fcb00 osd_op_reply(72 1000001a3c6.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv46880 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 22 ==== osd_op_reply(72 1000001a3c6.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv46880 ondisk = 0) v8 ==== 248+0+2748 (2985602565 0 4230017078) 0x5574ef7fcb00 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.718 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 23 0x5574ef7fd600 osd_op_reply(73 1000001eb17.00000000 [omap-get-header,omap-get-vals,getxattr (64)] v0'0 uv42866 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 23 ==== osd_op_reply(73 1000001eb17.00000000 [omap-get-header,omap-get-vals,getxattr (64)] v0'0 uv42866 ondisk = 0) v8 ==== 248+0+10888 (2827186539 0 1783259961) 0x5574ef7fd600 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.718 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 4 0x5574ef7fcb00 osd_op_reply(79 1000136753a.00000000 [omap-get-header,omap-get-vals,getxattr (252)] v0'0 uv26176 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 4 ==== osd_op_reply(79 1000136753a.00000000 [omap-get-header,omap-get-vals,getxattr (252)] v0'0 uv26176 ondisk = 0) v8 ==== 248+0+8987 (2899305789 0 1668207579) 0x5574ef7fcb00 con 0x5574ef85c000 -988> 2019-11-12 09:20:17.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 35 0x5574f17f0580 osd_op_reply(85 100013b1607.00000000 [omap-get-header,omap-get-vals,getxattr (198)] v0'0 uv59900 ondisk = 0) v8 -988> 2019-11-12 09:20:17.718 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 35 ==== osd_op_reply(85 100013b1607.00000000 [omap-get-header,omap-get-vals,getxattr (198)] v0'0 uv59900 ondisk = 0) v8 ==== 248+0+3465 (1835850951 0 2096249374) 0x5574f17f0580 con 0x5574ef703000 -988> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 24 0x5574ef7fd600 osd_op_reply(74 1000001ffea.00000000 [omap-get-header,omap-get-vals,getxattr (94)] v0'0 uv35408 ondisk = 0) v8 -988> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 24 ==== osd_op_reply(74 1000001ffea.00000000 [omap-get-header,omap-get-vals,getxattr (94)] v0'0 uv35408 ondisk = 0) v8 ==== 248+0+6097 (4183812923 0 414254997) 0x5574ef7fd600 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 25 0x5574ef7fcb00 osd_op_reply(77 10001367140.00000000 [omap-get-header,omap-get-vals,getxattr (220)] v0'0 uv35518 ondisk = 0) v8 -988> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 25 ==== osd_op_reply(77 10001367140.00000000 [omap-get-header,omap-get-vals,getxattr (220)] v0'0 uv35518 ondisk = 0) v8 ==== 248+0+3876 (2555161273 0 2568948404) 0x5574ef7fcb00 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 26 0x5574ef7fd600 osd_op_reply(78 100013671a4.00000000 [omap-get-header,omap-get-vals,getxattr (255)] v0'0 uv40089 ondisk = 0) v8 -988> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 26 ==== osd_op_reply(78 100013671a4.00000000 [omap-get-header,omap-get-vals,getxattr (255)] v0'0 uv40089 ondisk = 0) v8 ==== 248+0+4869 (3204699989 0 451091004) 0x5574ef7fd600 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 27 0x5574ef7fcb00 osd_op_reply(87 30000015ba9.00000000 [omap-get-header,omap-get-vals,getxattr (150)] v0'0 uv38565 ondisk = 0) v8 -988> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 27 ==== osd_op_reply(87 30000015ba9.00000000 [omap-get-header,omap-get-vals,getxattr (150)] v0'0 uv38565 ondisk = 0) v8 ==== 248+0+8319 (3722796679 0 2297239757) 0x5574ef7fcb00 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 28 0x5574ef7fd600 osd_op_reply(84 100013b1606.00000000 [omap-get-header,omap-get-vals,getxattr (170)] v0'0 uv40065 ondisk = 0) v8 -988> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 28 ==== osd_op_reply(84 100013b1606.00000000 [omap-get-header,omap-get-vals,getxattr (170)] v0'0 uv40065 ondisk = 0) v8 ==== 248+0+2386 (1036785619 0 3151859934) 0x5574ef7fd600 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 5 0x5574ef7fcb00 osd_op_reply(81 100013ad078.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv23726 ondisk = 0) v8 -988> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 5 ==== osd_op_reply(81 100013ad078.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv23726 ondisk = 0) v8 ==== 248+0+11231 (1837921862 0 549299108) 0x5574ef7fcb00 con 0x5574ef85c000 -988> 2019-11-12 09:20:17.726 7fe87e779700 5 mds.0.log _submit_thread 33239061177~772 : ESessions 14 opens cmapv 78165 -988> 2019-11-12 09:20:17.730 7fe87e779700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:88 6.9 6:916b6fb1:::200.00001ef4:head [write 3396281~792] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -988> 2019-11-12 09:20:17.730 7fe87e779700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:89 6.14 6:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef810000 con 0 -988> 2019-11-12 09:20:17.742 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 29 0x5574ef810000 osd_op_reply(88 200.00001ef4 [write 3396281~792] v28661'43317 uv43317 ondisk = 0) v8 -988> 2019-11-12 09:20:17.742 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 29 ==== osd_op_reply(88 200.00001ef4 [write 3396281~792] v28661'43317 uv43317 ondisk = 0) v8 ==== 156+0+0 (124758417 0 0) 0x5574ef810000 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.746 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 30 0x5574ef810000 osd_op_reply(89 200.00000000 [writefull 0~90] v28661'1334402 uv1334402 ondisk = 0) v8 -988> 2019-11-12 09:20:17.746 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 30 ==== osd_op_reply(89 200.00000000 [writefull 0~90] v28661'1334402 uv1334402 ondisk = 0) v8 ==== 156+0+0 (2410142955 0 0) 0x5574ef810000 con 0x5574ef702a00 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/3593333399 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574ef567b80 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189b80 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189900 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/4201708698 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189680 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/232332927 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189400 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/1453103486 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189180 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/522103062 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1188f00 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1188c80 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/2985278080 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1188a00 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:0/3102681411 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744000 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:0/2796926812 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744280 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/2855972843 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744500 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/3544368504 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744780 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/979484066 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744a00 con 0 -988> 2019-11-12 09:20:17.766 7fe87f77b700 1 mds.0.56303 rejoin_done -988> 2019-11-12 09:20:17.774 7fe87f77b700 3 mds.0.56303 request_state up:active -988> 2019-11-12 09:20:17.774 7fe87f77b700 5 mds.beacon.k8s-node-02 set_want_state: up:rejoin -> up:active -988> 2019-11-12 09:20:17.774 7fe87f77b700 5 mds.beacon.k8s-node-02 Sending beacon up:active seq 7 -988> 2019-11-12 09:20:17.774 7fe87f77b700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -988> 2019-11-12 09:20:17.774 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:active seq 7 v56305) v7 -- 0x5574ef80ee00 con 0 -988> 2019-11-12 09:20:17.774 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:90 6.17 6:e9c7ff81:::200.00001ec2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef811340 con 0 -988> 2019-11-12 09:20:17.774 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:91 6.42 6:424fe295:::200.00001ec3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f1226000 con 0 -988> 2019-11-12 09:20:17.794 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 18 0x5574ef811340 osd_op_reply(90 200.00001ec2 [delete] v28661'36036 uv35960 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:17.794 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 18 ==== osd_op_reply(90 200.00001ec2 [delete] v28661'36036 uv35960 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (1025874436 0 0) 0x5574ef811340 con 0x5574ef703600 -988> 2019-11-12 09:20:17.802 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 31 0x5574f1226000 osd_op_reply(91 200.00001ec3 [delete] v28661'35871 uv35683 ondisk = -2 ((2) No such file or directory)) v8 -988> 2019-11-12 09:20:17.802 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 31 ==== osd_op_reply(91 200.00001ec3 [delete] v28661'35871 uv35683 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (1997661384 0 0) 0x5574f1226000 con 0x5574ef702a00 -988> 2019-11-12 09:20:18.095 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1793818 seq 2 0x5574ef7a6240 client_session(request_renewcaps seq 28283) -988> 2019-11-12 09:20:18.095 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793818 192.168.200.30:0/232332927 2 ==== client_session(request_renewcaps seq 28283) ==== 28+0+0 (422215383 0 0) 0x5574ef7a6240 con 0x5574ef85e400 -988> 2019-11-12 09:20:18.095 7fe885a11700 3 mds.0.server handle_client_session client_session(request_renewcaps seq 28283) from client.1793818 -988> 2019-11-12 09:20:18.095 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/232332927 -- client_session(renewcaps seq 28283) v3 -- 0x5574ef864000 con 0 -988> 2019-11-12 09:20:18.751 7fe883783700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +0-0 packed 1374) v6 -- 0x5574f06ce300 con 0 -988> 2019-11-12 09:20:18.779 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 20 0x5574f122d680 mdsmap(e 56306) v1 -988> 2019-11-12 09:20:18.779 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 20 ==== mdsmap(e 56306) v1 ==== 655+0+0 (3582797354 0 0) 0x5574f122d680 con 0x5574ef700c00 -988> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56306 from mon.0 -988> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -988> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:rejoin --> up:active -988> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 recovery_done -- successful recovery! -988> 2019-11-12 09:20:18.779 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:92 6.33 6:cd5a64a3:::100.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -988> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 active_start -988> 2019-11-12 09:20:18.779 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 21 0x5574ef80ee00 mdsbeacon(1849248/k8s-node-02 up:active seq 7 v56306) v7 -988> 2019-11-12 09:20:18.779 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 21 ==== mdsbeacon(1849248/k8s-node-02 up:active seq 7 v56306) v7 ==== 136+0+0 (2666966931 0 0) 0x5574ef80ee00 con 0x5574ef700c00 -988> 2019-11-12 09:20:18.779 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:active seq 7 rtt 1.00402 -988> 2019-11-12 09:20:18.779 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 19 0x5574f17f02c0 osd_op_reply(92 100.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv82450 ondisk = 0) v8 -988> 2019-11-12 09:20:18.779 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 19 ==== osd_op_reply(92 100.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv82450 ondisk = 0) v8 ==== 240+0+5119 (3401304057 0 3700237336) 0x5574f17f02c0 con 0x5574ef703600 -988> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 7 0x5574ef5da480 client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12993 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 -988> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 8 0x5574ef5dbb00 client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12994 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 -988> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 9 0x5574ef5dbf80 client_caps(flush ino 0x100026d24a3 306632 seq 0 tid 12991 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFxcwb follows 1 size 12582912/12584448 mtime 2019-11-12 09:18:41.460885 xattrs(v=18446744071802044689 l=0)) v10 -988> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 10 0x5574ef5dc400 client_caps(flush ino 0x100026d24a3 306632 seq 0 tid 12995 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFxcwb follows 1 size 12582912/12584448 mtime 2019-11-12 09:18:41.460885 xattrs(v=18446744071802044689 l=0)) v10 -988> 2019-11-12 09:20:18.787 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 11 0x5574ef5dc880 client_caps(update ino 0x100026d24a3 306632 seq 0 caps=pAsxLsXsxFsxcrwb dirty=- wanted=pAsxXsxFxcwb follows 0 size 12582912/12584448 mtime 2019-11-12 09:18:41.460885 xattrs(v=18446744071809532324 l=0)) v10 -988> 2019-11-12 09:20:18.787 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_caps(grant ino 0x100013b15fc 1738 seq 1 caps=pAsxLsXsxFsxcrwb dirty=- wanted=pAsxXsxFxwb follows 0 size 22/4194304 ts 5974/0 mtime 2019-11-12 08:21:27.262884 tws 5970 xattrs(v=1 l=4)) v11 -- 0x5574ef5dd180 con 0 -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96231 lookup #0x100013ad07a/eula.txt 2019-11-12 08:59:14.149366 RETRY=30 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23347 lookup #0x10001346679/public 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9471 lookup #0x1000136753a/mime_types.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96232 lookup #0x100013ad07a/eula.txt 2019-11-12 09:05:30.072151 RETRY=10 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23348 lookup #0x1000134691b/autoload.php 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9472 lookup #0x1000136753a/spf_dkim_whitelist.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96233 lookup #0x100013ad07a/eula.txt 2019-11-12 09:06:44.117495 RETRY=8 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23349 lookup #0x10001346679/private 2019-11-12 09:18:41.457643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9473 lookup #0x1000136753a/surbl-whitelist.inc.local 2019-11-12 09:18:41.736891 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96236 lookup #0x100013ad07a/eula.txt 2019-11-12 09:17:08.464874 RETRY=2 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9474 getattr As #0x1000136753a 2019-11-12 09:19:29.005938 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96238 lookup #0x100013ad07a/eula.txt 2019-11-12 09:18:43.942618 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96239 getattr As #0x100013ad07a 2019-11-12 09:19:36.755583 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -988> 2019-11-12 09:20:18.795 7fe885a11700 1 mds.0.56303 cluster recovered. -988> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.56303 set_osd_epoch_barrier: epoch=28661 -988> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 7 ==== client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12993 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 ==== 236+0+0 (3677921934 0 0) 0x5574ef5da480 con 0x5574f11dd200 -988> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_caps(flush_ack ino 0x1000136718a 2423 seq 0 tid 12993 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=- follows 1 size 0/0 mtime 0.000000) v11 -- 0x5574ef5dd600 con 0 -988> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_caps(grant ino 0x1000136718a 2423 seq 1 caps=pAsxLsXsxFsxcrwb dirty=- wanted=pAsxXsxFsxcrwb follows 0 size 797696/4194304 ts 1/18446744073709551615 mtime 2019-11-12 09:13:00.157329 xattrs(v=1 l=4)) v11 -- 0x5574ef5dda80 con 0 -988> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 8 ==== client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12994 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 ==== 236+0+0 (3677921934 0 0) 0x5574ef5dbb00 con 0x5574f11dd200 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:93 6.30 6:0d82a743:::600.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f0766000 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.purge_queue activate: start work (by drain) -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:94 6.25 6:a64c6dbf:::500.0000000f:head [read 1546512~19054 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810580 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/3593333399 -- client_reply(???:329 = 0 (0) Success) v1 -- 0x5574f0766580 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:95 6.57 6:ea90f4f9:::100013ad07a.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:96 6.3d 6:bfdb3c52:::3000001e549.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:97 6.5 6:a110cbc0:::10001346b3e.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226840 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_caps(revoke ino 0x100013b15fc 1738 seq 2 caps=pAsxLsXsxFcb dirty=- wanted=pAsxXsxFxwb follows 0 size 22/4194304 ts 5974/0 mtime 2019-11-12 08:21:27.262884 tws 5970) v11 -- 0x5574f080a000 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:98 6.c 6:305695a6:::10001344421.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_reply(???:19856 = 0 (0) Success) v1 -- 0x5574f17f1600 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_reply(???:23345 = 0 (0) Success) v1 -- 0x5574f17f1080 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_reply(???:23346 = 0 (0) Success) v1 -- 0x5574f17f0580 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_reply(???:9470 = -2 (2) No such file or directory) v1 -- 0x5574f1227340 con 0 -988> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 -988> 2019-11-12 09:20:18.795 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 36 0x5574ef7fd340 osd_op_reply(96 3000001e549.00000000 [omap-get-header,omap-get-vals,getxattr (97)] v0'0 uv88617 ondisk = 0) v8 -988> 2019-11-12 09:20:18.795 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 36 ==== osd_op_reply(96 3000001e549.00000000 [omap-get-header,omap-get-vals,getxattr (97)] v0'0 uv88617 ondisk = 0) v8 ==== 248+0+2829 (838332988 0 2622649345) 0x5574ef7fd340 con 0x5574ef703000 -988> 2019-11-12 09:20:18.799 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 37 0x5574f17f0b00 osd_op_reply(98 10001344421.00000000 [omap-get-header,omap-get-vals,getxattr (125)] v0'0 uv36621 ondisk = 0) v8 -988> 2019-11-12 09:20:18.799 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 37 ==== osd_op_reply(98 10001344421.00000000 [omap-get-header,omap-get-vals,getxattr (125)] v0'0 uv36621 ondisk = 0) v8 ==== 248+0+19893 (2739227853 0 1215621712) 0x5574f17f0b00 con 0x5574ef703000 -988> 2019-11-12 09:20:18.799 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 4 0x5574ef5dd180 client_caps(update ino 0x100013b15fc 1738 seq 2 caps=pAsxLsXsxFcb dirty=- wanted=pAsxXsxFxwb follows 0 size 22/0 mtime 2019-11-12 08:21:27.262884 tws 5970) v6 -988> 2019-11-12 09:20:18.799 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 5 0x5574f11fbb00 client_request(client.1793737:19857 readdir #0x100013b1607 2019-11-12 09:20:18.796352 caller_uid=1000, caller_gid=1000{}) v2 -988> 2019-11-12 09:20:18.799 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 32 0x5574f17f0000 osd_op_reply(95 100013ad07a.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv30079 ondisk = 0) v8 -988> 2019-11-12 09:20:18.799 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 32 ==== osd_op_reply(95 100013ad07a.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv30079 ondisk = 0) v8 ==== 248+0+7185 (2976393307 0 498225518) 0x5574f17f0000 con 0x5574ef702a00 -988> 2019-11-12 09:20:18.799 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 38 0x5574ef7fd340 osd_op_reply(97 10001346b3e.00000000 [omap-get-header,omap-get-vals,getxattr (124)] v0'0 uv36678 ondisk = 0) v8 -988> 2019-11-12 09:20:18.799 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 38 ==== osd_op_reply(97 10001346b3e.00000000 [omap-get-header,omap-get-vals,getxattr (124)] v0'0 uv36678 ondisk = 0) v8 ==== 248+0+1853 (4149073175 0 3508024585) 0x5574ef7fd340 con 0x5574ef703000 -988> 2019-11-12 09:20:18.799 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 39 0x5574f17f0b00 osd_op_reply(93 600.00000000 [omap-get-header,omap-get-vals,getxattr (62)] v0'0 uv93665 ondisk = 0) v8 -988> 2019-11-12 09:20:18.799 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 39 ==== osd_op_reply(93 600.00000000 [omap-get-header,omap-get-vals,getxattr (62)] v0'0 uv93665 ondisk = 0) v8 ==== 240+0+83373 (2574217908 0 294505411) 0x5574f17f0b00 con 0x5574ef703000 -988> 2019-11-12 09:20:18.803 7fe880f7e700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: In function 'void Server::_rename_prepare(MDRequestRef&, EMetaBlob*, ceph::bufferlist*, CDentry*, CDentry*, CDentry*)' thread 7fe880f7e700 time 2019-11-12 09:20:18.803701 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: 8036: FAILED assert(oldin->first <= straydn->first) ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x7fe88f05b36b] 2: (()+0x26e4f7) [0x7fe88f05b4f7] 3: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 4: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 5: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 6: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 7: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 8: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 9: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 10: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 11: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 12: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 13: (()+0x7e65) [0x7fe88cc8ee65] 14: (clone()+0x6d) [0x7fe88bd6988d] -988> 2019-11-12 09:20:18.803 7fe880f7e700 -1 *** Caught signal (Aborted) ** in thread 7fe880f7e700 thread_name:mds_rank_progr ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (()+0xf5f0) [0x7fe88cc965f0] 2: (gsignal()+0x37) [0x7fe88bca1337] 3: (abort()+0x148) [0x7fe88bca2a28] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x248) [0x7fe88f05b468] 5: (()+0x26e4f7) [0x7fe88f05b4f7] 6: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 7: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 8: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 9: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 10: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 11: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 12: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 13: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 14: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 15: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 16: (()+0x7e65) [0x7fe88cc8ee65] 17: (clone()+0x6d) [0x7fe88bd6988d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 rgw_sync 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) 99/99 (stderr threshold) max_recent 10000 max_new 1000 log_file --- end dump of recent events --- --- begin dump of recent events --- -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command abort hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perfcounters_dump hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command 1 hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf dump hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perfcounters_schema hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf histogram dump hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command 2 hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf schema hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf histogram schema hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command perf reset hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config show hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config help hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config set hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config unset hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config get hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config diff hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command config diff get hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command log flush hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command log dump hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command log reopen hook 0x5574ef5302b0 -1010> 2019-11-12 09:20:03.494 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_mempools hook 0x5574ef5662c8 -1010> 2019-11-12 09:20:03.514 7fe897e7f000 10 monclient: get_monmap_and_config -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: build_initial_monmap -1010> 2019-11-12 09:20:03.598 7fe888a17700 2 Event(0x5574ef543480 nevent=5000 time_id=1).set_owner idx=0 owner=140636701423360 -1010> 2019-11-12 09:20:03.598 7fe887a15700 2 Event(0x5574ef6b8480 nevent=5000 time_id=1).set_owner idx=2 owner=140636684637952 -1010> 2019-11-12 09:20:03.598 7fe888216700 2 Event(0x5574ef543680 nevent=5000 time_id=1).set_owner idx=1 owner=140636693030656 -1010> 2019-11-12 09:20:03.598 7fe897e7f000 1 Processor -- start -1010> 2019-11-12 09:20:03.598 7fe897e7f000 1 -- - start start -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: init -1010> 2019-11-12 09:20:03.598 7fe897e7f000 5 adding auth protocol: cephx -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: auth_supported 2 method cephx -1010> 2019-11-12 09:20:03.598 7fe897e7f000 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-k8s-node-02/keyring -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient: _reopen_session rank -1 -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-00 con 0x5574ef700c00 addr 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-02 con 0x5574ef701200 addr 192.168.200.19:6789/0 -1010> 2019-11-12 09:20:03.598 7fe897e7f000 1 -- - --> 192.168.200.15:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52afc0 con 0 -1010> 2019-11-12 09:20:03.598 7fe897e7f000 1 -- - --> 192.168.200.19:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52b200 con 0 -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): _renew_subs -1010> 2019-11-12 09:20:03.598 7fe897e7f000 10 monclient(hunting): authenticate will time out at 2019-11-12 09:25:03.605089 -1010> 2019-11-12 09:20:03.602 7fe887a15700 1 -- 192.168.200.19:0/763506544 learned_addr learned my addr 192.168.200.19:0/763506544 -1010> 2019-11-12 09:20:03.602 7fe888216700 2 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:03.602 7fe887a15700 2 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:03.602 7fe888216700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15926 cs=1 l=1). rx mon.0 seq 1 0x5574ef52bb00 mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.602 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.0 192.168.200.15:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef52bb00 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.602 7fe888216700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15926 cs=1 l=1). rx mon.0 seq 2 0x5574ef52bd40 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): got monmap 323, mon.k8s-node-00 is now rank 0 -1010> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -1010> 2019-11-12 09:20:03.602 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.0 192.168.200.15:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (164204572 0 0) 0x5574ef52bd40 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.602 7fe887214700 10 monclient(hunting): my global_id is 1849245 -1010> 2019-11-12 09:20:03.602 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.15:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef52afc0 con 0 -1010> 2019-11-12 09:20:03.606 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 1 0x5574ef7a66c0 mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a66c0 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): got monmap 323, mon.k8s-node-02 is now rank 2 -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -1010> 2019-11-12 09:20:03.606 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 2 0x5574ef7a6900 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (3829032891 0 0) 0x5574ef7a6900 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient(hunting): my global_id is 1841201 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.19:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef7a66c0 con 0 -1010> 2019-11-12 09:20:03.606 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 3 0x5574ef7a66c0 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.606 7fe888216700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15926 cs=1 l=1). rx mon.0 seq 3 0x5574ef52afc0 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 222+0+0 (2280171065 0 0) 0x5574ef7a66c0 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN pgs=15926 cs=1 l=1).mark_down -1010> 2019-11-12 09:20:03.606 7fe887214700 2 -- 192.168.200.19:0/763506544 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN pgs=15926 cs=1 l=1)._stop -1010> 2019-11-12 09:20:03.606 7fe887214700 1 monclient: found mon.k8s-node-02 -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient: _send_mon_message to mon.k8s-node-02 at 192.168.200.19:6789/0 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ef6b9800 con 0 -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-11-12 09:19:33.612313) -1010> 2019-11-12 09:20:03.606 7fe887214700 10 monclient: _send_mon_message to mon.k8s-node-02 at 192.168.200.19:6789/0 -1010> 2019-11-12 09:20:03.606 7fe887214700 1 -- 192.168.200.19:0/763506544 --> 192.168.200.19:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x5574ef52afc0 con 0 -1010> 2019-11-12 09:20:03.606 7fe897e7f000 5 monclient: authenticate success, global_id 1841201 -1010> 2019-11-12 09:20:03.610 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 4 0x5574ef79e200 config(0 keys) v1 -1010> 2019-11-12 09:20:03.610 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 5 0x5574ef7a6b40 mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.610 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (0 0 0) 0x5574ef79e200 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.610 7fe887a15700 5 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=31893 cs=1 l=1). rx mon.2 seq 6 0x5574ef7a6d80 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: handle_config config(0 keys) v1 -1010> 2019-11-12 09:20:03.610 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 5 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6b40 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 10 monclient: get_monmap_and_config success -1010> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: handle_monmap mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 10 monclient: shutdown -1010> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: got monmap 323, mon.k8s-node-02 is now rank 2 -1010> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=31893 cs=1 l=1).mark_down -1010> 2019-11-12 09:20:03.610 7fe887214700 1 -- 192.168.200.19:0/763506544 <== mon.2 192.168.200.19:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (1366969319 0 0) 0x5574ef7a6d80 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 2 -- 192.168.200.19:0/763506544 >> 192.168.200.19:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=31893 cs=1 l=1)._stop -1010> 2019-11-12 09:20:03.610 7fe887214700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.610 7fe885a11700 4 set_mon_vals no callback set -1010> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 shutdown_connections -1010> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections mark down 192.168.200.19:6789/0 0x5574ef701200 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections mark down 192.168.200.15:6789/0 0x5574ef700c00 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections delete 0x5574ef700c00 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 5 -- 192.168.200.19:0/763506544 shutdown_connections delete 0x5574ef701200 -1010> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 shutdown_connections -1010> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 wait complete. -1010> 2019-11-12 09:20:03.610 7fe897e7f000 1 -- 192.168.200.19:0/763506544 >> 192.168.200.19:0/763506544 conn(0x5574ef700600 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -1010> 2019-11-12 09:20:03.610 7fe897e7f000 2 -- 192.168.200.19:0/763506544 >> 192.168.200.19:0/763506544 conn(0x5574ef700600 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -1010> 2019-11-12 09:20:03.614 7fe897e7f000 0 set uid:gid to 167:167 (ceph:ceph) -1010> 2019-11-12 09:20:03.614 7fe897e7f000 0 ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable), process ceph-mds, pid 16285 -1010> 2019-11-12 09:20:03.618 7fe897e7f000 0 pidfile_write: ignore empty --pid-file -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) init /var/run/ceph/ceph-mds.k8s-node-02.asok -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) bind_and_listen /var/run/ceph/ceph-mds.k8s-node-02.asok -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command 0 hook 0x5574ef52f4a0 -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command version hook 0x5574ef52f4a0 -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command git_version hook 0x5574ef52f4a0 -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command help hook 0x5574ef530510 -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 asok(0x5574ef5585a0) register_command get_command_descriptions hook 0x5574ef530500 -1010> 2019-11-12 09:20:03.706 7fe886a13700 5 asok(0x5574ef5585a0) entry start -1010> 2019-11-12 09:20:03.706 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 _finish_bind bind my_inst.addr is 0.0.0.0:6800/1460470921 -1010> 2019-11-12 09:20:03.706 7fe897e7f000 10 monclient: build_initial_monmap -1010> 2019-11-12 09:20:03.706 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 start start -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 adding auth protocol: cephx -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 adding auth protocol: cephx -1010> 2019-11-12 09:20:03.706 7fe897e7f000 1 Processor -- start -1010> 2019-11-12 09:20:03.706 7fe897e7f000 10 monclient: init -1010> 2019-11-12 09:20:03.706 7fe897e7f000 5 adding auth protocol: cephx -1010> 2019-11-12 09:20:03.706 7fe897e7f000 10 monclient: auth_supported 2 method cephx -1010> 2019-11-12 09:20:03.706 7fe897e7f000 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-k8s-node-02/keyring -1010> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient: _reopen_session rank -1 -1010> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-00 con 0x5574ef700c00 addr 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient(hunting): picked mon.k8s-node-01 con 0x5574ef701200 addr 192.168.200.17:6789/0 -1010> 2019-11-12 09:20:03.710 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52b8c0 con 0 -1010> 2019-11-12 09:20:03.710 7fe897e7f000 1 -- 0.0.0.0:6800/1460470921 --> 192.168.200.17:6789/0 -- auth(proto 0 36 bytes epoch 0) v1 -- 0x5574ef52b680 con 0 -1010> 2019-11-12 09:20:03.710 7fe897e7f000 10 monclient(hunting): _renew_subs -1010> 2019-11-12 09:20:03.710 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 learned_addr learned my addr 192.168.200.19:6800/1460470921 -1010> 2019-11-12 09:20:03.710 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:03.710 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:03.710 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 1 0x5574ef7a6000 mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6000 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): got monmap 323, mon.k8s-node-00 is now rank 0 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -1010> 2019-11-12 09:20:03.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 2 0x5574ef7a6240 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (1092923085 0 0) 0x5574ef7a6240 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): my global_id is 1849248 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef7a6000 con 0 -1010> 2019-11-12 09:20:03.714 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=29948 cs=1 l=1). rx mon.1 seq 1 0x5574ef7a6fc0 mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.1 192.168.200.17:6789/0 1 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6fc0 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): got monmap 323, mon.k8s-node-01 is now rank 1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -1010> 2019-11-12 09:20:03.714 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=29948 cs=1 l=1). rx mon.1 seq 2 0x5574ef7a7200 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.1 192.168.200.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (457588048 0 0) 0x5574ef7a7200 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.714 7fe885a11700 10 monclient(hunting): my global_id is 1840183 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x5574ef7a6b40 con 0 -1010> 2019-11-12 09:20:03.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 3 0x5574ef7a6000 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 222+0+0 (868200785 0 0) 0x5574ef7a6000 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 2 181 bytes epoch 0) v1 -- 0x5574ef7a7200 con 0 -1010> 2019-11-12 09:20:03.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=29948 cs=1 l=1). rx mon.1 seq 3 0x5574ef7a6b40 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.1 192.168.200.17:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 222+0+0 (1953234255 0 0) 0x5574ef7a6b40 con 0x5574ef701200 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6789/0 -- auth(proto 2 181 bytes epoch 0) v1 -- 0x5574ef7a6000 con 0 -1010> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 4 0x5574ef7a7200 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 799+0+0 (2751031812 0 0) 0x5574ef7a7200 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=29948 cs=1 l=1).mark_down -1010> 2019-11-12 09:20:03.718 7fe885a11700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6789/0 conn(0x5574ef701200 :-1 s=STATE_OPEN pgs=29948 cs=1 l=1)._stop -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 monclient: found mon.k8s-node-00 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ef6c4e00 con 0 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-11-12 09:19:33.723602) -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x5574ef7a6b40 con 0 -1010> 2019-11-12 09:20:03.718 7fe897e7f000 5 monclient: authenticate success, global_id 1849248 -1010> 2019-11-12 09:20:03.718 7fe897e7f000 10 monclient: wait_auth_rotating waiting (until 2019-11-12 09:20:33.723724) -1010> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 5 0x5574ef542c00 config(0 keys) v1 -1010> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 6 0x5574ef7a6480 mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 7 0x5574ef7a7440 auth_reply(proto 2 0 (0) Success) v1 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (0 0 0) 0x5574ef542c00 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: handle_config config(0 keys) v1 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 6 ==== mon_map magic: 0 v1 ==== 451+0+0 (1031779836 0 0) 0x5574ef7a6480 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: handle_monmap mon_map magic: 0 v1 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: got monmap 323, mon.k8s-node-00 is now rank 0 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: dump: epoch 323 fsid 571d4bfe-2c5d-45ca-8da1-91dcaf69942c last_changed 2019-10-31 14:07:12.621434 created 2018-04-12 14:27:23.947601 0: 192.168.200.15:6789/0 mon.k8s-node-00 1: 192.168.200.17:6789/0 mon.k8s-node-01 2: 192.168.200.19:6789/0 mon.k8s-node-02 -1010> 2019-11-12 09:20:03.718 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (4270746359 0 0) 0x5574ef7a7440 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.718 7fe885a11700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-11-12 09:19:33.725365) -1010> 2019-11-12 09:20:03.718 7fe897e7f000 10 monclient: wait_auth_rotating done -1010> 2019-11-12 09:20:03.718 7fe883f84700 4 set_mon_vals no callback set -1010> 2019-11-12 09:20:03.718 7fe897e7f000 10 monclient: _renew_subs -1010> 2019-11-12 09:20:03.722 7fe897e7f000 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({mdsmap=0+,mgrmap=0+}) v3 -- 0x5574ef79f600 con 0 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command status hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_ops_in_flight hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command ops hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_blocked_ops hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_historic_ops hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump_historic_ops_by_duration hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command scrub_path hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command tag path hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command flush_path hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command export dir hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump cache hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command cache status hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command cache drop hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump tree hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump loads hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump snaps hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command session evict hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command osdmap barrier hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command session ls hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command flush journal hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command force_readonly hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command get subtrees hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dirfrag split hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dirfrag merge hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dirfrag ls hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command openfiles ls hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe897e7f000 5 asok(0x5574ef5585a0) register_command dump inode hook 0x5574ef530430 -1010> 2019-11-12 09:20:03.722 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 8 0x5574ef7a78c0 mdsmap(e 56301) v1 -1010> 2019-11-12 09:20:03.722 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 8 ==== mdsmap(e 56301) v1 ==== 493+0+0 (1364545111 0 0) 0x5574ef7a78c0 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.722 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 9 0x5574ef7ad080 mgrmap(e 3928) v1 -1010> 2019-11-12 09:20:03.722 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:boot seq 1 -1010> 2019-11-12 09:20:03.742 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:03.742 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:boot seq 1 v0) v7 -- 0x5574ef6ff180 con 0 -1010> 2019-11-12 09:20:03.742 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56301 from mon.0 -1010> 2019-11-12 09:20:03.742 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 9 ==== mgrmap(e 3928) v1 ==== 1318+0+0 (728182834 0 0) 0x5574ef7ad080 con 0x5574ef700c00 -1010> 2019-11-12 09:20:03.742 7fe885a11700 4 mgrc handle_mgr_map Got map version 3928 -1010> 2019-11-12 09:20:03.742 7fe885a11700 4 mgrc handle_mgr_map Active mgr is now 192.168.200.17:6800/91 -1010> 2019-11-12 09:20:03.742 7fe885a11700 4 mgrc reconnect Starting new session with 192.168.200.17:6800/91 -1010> 2019-11-12 09:20:03.746 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6800/91 conn(0x5574ef702400 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:03.746 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgropen(unknown.k8s-node-02) v3 -- 0x5574ef564c00 con 0 -1010> 2019-11-12 09:20:03.746 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6800/91 conn(0x5574ef702400 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=543676 cs=1 l=1). rx mgr.1754125 seq 1 0x5574ef558f00 mgrconfigure(period=5, threshold=5) v2 -1010> 2019-11-12 09:20:03.750 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mgr.1754125 192.168.200.17:6800/91 1 ==== mgrconfigure(period=5, threshold=5) v2 ==== 8+0+0 (3460719617 0 0) 0x5574ef558f00 con 0x5574ef702400 -1010> 2019-11-12 09:20:03.750 7fe885a11700 4 mgrc handle_mgr_configure stats_period=5 -1010> 2019-11-12 09:20:03.750 7fe885a11700 4 mgrc handle_mgr_configure updated stats threshold: 5 -1010> 2019-11-12 09:20:03.750 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +0-0 packed 6) v6 -- 0x5574ef564f00 con 0 -1010> 2019-11-12 09:20:07.742 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:boot seq 2 -1010> 2019-11-12 09:20:07.762 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:07.762 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:boot seq 2 v56301) v7 -- 0x5574ef6ff500 con 0 -1010> 2019-11-12 09:20:08.406 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 10 0x5574ef7a7b00 mdsmap(e 56302) v1 -1010> 2019-11-12 09:20:08.406 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 10 ==== mdsmap(e 56302) v1 ==== 618+0+0 (519530298 0 0) 0x5574ef7a7b00 con 0x5574ef700c00 -1010> 2019-11-12 09:20:08.406 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56302 from mon.0 -1010> 2019-11-12 09:20:08.406 7fe885a11700 5 mds.beacon.k8s-node-02 set_want_state: up:boot -> up:standby -1010> 2019-11-12 09:20:08.406 7fe885a11700 1 mds.k8s-node-02 Map has assigned me to become a standby -1010> 2019-11-12 09:20:08.410 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 11 0x5574ef6ff500 mdsbeacon(1849248/k8s-node-02 up:boot seq 2 v56302) v7 -1010> 2019-11-12 09:20:08.410 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 11 ==== mdsbeacon(1849248/k8s-node-02 up:boot seq 2 v56302) v7 ==== 140+0+0 (767818115 0 0) 0x5574ef6ff500 con 0x5574ef700c00 -1010> 2019-11-12 09:20:08.410 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:boot seq 2 rtt 0.668014 -1010> 2019-11-12 09:20:08.430 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 12 0x5574ef7a7200 mdsmap(e 56303) v1 -1010> 2019-11-12 09:20:08.430 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 12 ==== mdsmap(e 56303) v1 ==== 655+0+0 (1259924393 0 0) 0x5574ef7a7200 con 0x5574ef700c00 -1010> 2019-11-12 09:20:08.430 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56303 from mon.0 -1010> 2019-11-12 09:20:08.430 7fe885a11700 4 mds.0.purge_queue operator(): data pool 2 not found in OSDMap -1010> 2019-11-12 09:20:08.434 7fe885a11700 5 asok(0x5574ef5585a0) register_command objecter_requests hook 0x5574ef530610 -1010> 2019-11-12 09:20:08.434 7fe885a11700 10 monclient: _renew_subs -1010> 2019-11-12 09:20:08.434 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x5574ef7c4c00 con 0 -1010> 2019-11-12 09:20:08.434 7fe885a11700 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) -1010> 2019-11-12 09:20:08.434 7fe885a11700 4 mds.0.purge_queue operator(): data pool 2 not found in OSDMap -1010> 2019-11-12 09:20:08.434 7fe885a11700 4 mds.0.0 handle_osd_map epoch 0, 0 new blacklist entries -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:boot --> up:replay -1010> 2019-11-12 09:20:08.434 7fe885a11700 5 mds.beacon.k8s-node-02 set_want_state: up:standby -> up:replay -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 replay_start -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 recovery set is -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 mds.0.56303 waiting for osdmap 28661 (which blacklists prior instance) -1010> 2019-11-12 09:20:08.434 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 13 0x5574ef7f4500 osd_map(28661..28661 src has 27910..28661) v4 -1010> 2019-11-12 09:20:08.434 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 13 ==== osd_map(28661..28661 src has 27910..28661) v4 ==== 98323+0+0 (451166248 0 0) 0x5574ef7f4500 con 0x5574ef700c00 -1010> 2019-11-12 09:20:08.438 7fe885a11700 4 mds.0.56303 handle_osd_map epoch 28661, 0 new blacklist entries -1010> 2019-11-12 09:20:08.438 7fe885a11700 10 monclient: _renew_subs -1010> 2019-11-12 09:20:08.438 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:08.438 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mon_subscribe({osdmap=28662}) v3 -- 0x5574ef7c4e00 con 0 -1010> 2019-11-12 09:20:08.438 7fe87f77b700 2 mds.0.56303 boot_start 0: opening inotable -1010> 2019-11-12 09:20:08.438 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:1 6.13 6:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -1010> 2019-11-12 09:20:08.438 7fe87f77b700 2 mds.0.56303 boot_start 0: opening sessionmap -1010> 2019-11-12 09:20:08.438 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:2 6.b 6:d0630e4c:::mds0_sessionmap:head [omap-get-header,omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fcb00 con 0 -1010> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: opening mds log -1010> 2019-11-12 09:20:08.442 7fe87f77b700 5 mds.0.log open discovering log bounds -1010> 2019-11-12 09:20:08.442 7fe87ef7a700 4 mds.0.journalpointer Reading journal pointer '400.00000000' -1010> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: opening purge queue (async) -1010> 2019-11-12 09:20:08.442 7fe87f77b700 4 mds.0.purge_queue open: opening -1010> 2019-11-12 09:20:08.442 7fe87f77b700 1 mds.0.journaler.pq(ro) recover start -1010> 2019-11-12 09:20:08.442 7fe87f77b700 1 mds.0.journaler.pq(ro) read_head -1010> 2019-11-12 09:20:08.442 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:3 6.f 6:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd8c0 con 0 -1010> 2019-11-12 09:20:08.442 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:4 6.55 6:aa448500:::500.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -1010> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: loading open file table (async) -1010> 2019-11-12 09:20:08.442 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:5 6.26 6:654134d2:::mds0_openfiles.0:head [omap-get-header,omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -1010> 2019-11-12 09:20:08.442 7fe87f77b700 2 mds.0.56303 boot_start 0: opening snap table -1010> 2019-11-12 09:20:08.442 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:6 6.2d 6:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fdb80 con 0 -1010> 2019-11-12 09:20:08.442 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:08.442 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:08.442 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:08.446 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 1 0x5574ef7fdb80 osd_op_reply(1 mds0_inotable [read 0~34] v0'0 uv59541 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.446 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 1 ==== osd_op_reply(1 mds0_inotable [read 0~34] v0'0 uv59541 ondisk = 0) v8 ==== 157+0+34 (3903532989 0 4146252902) 0x5574ef7fdb80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.446 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 2 0x5574ef7fdb80 osd_op_reply(6 mds_snaptable [read 0~70] v0'0 uv69169 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.446 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 2 ==== osd_op_reply(6 mds_snaptable [read 0~70] v0'0 uv69169 ondisk = 0) v8 ==== 157+0+70 (1873612932 0 336025128) 0x5574ef7fdb80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.446 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 3 0x5574ef7fdb80 osd_op_reply(2 mds0_sessionmap [omap-get-header,omap-get-vals] v0'0 uv43122 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.446 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 3 ==== osd_op_reply(2 mds0_sessionmap [omap-get-header,omap-get-vals] v0'0 uv43122 ondisk = 0) v8 ==== 201+0+6124 (669162857 0 3071192317) 0x5574ef7fdb80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.514 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 1 0x5574ef7fd8c0 osd_op_reply(3 400.00000000 [read 0~22] v0'0 uv974 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.514 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 1 ==== osd_op_reply(3 400.00000000 [read 0~22] v0'0 uv974 ondisk = 0) v8 ==== 156+0+22 (844539098 0 3825830296) 0x5574ef7fd8c0 con 0x5574ef703000 -1010> 2019-11-12 09:20:08.514 7fe87ef7a700 1 mds.0.journaler.mdlog(ro) recover start -1010> 2019-11-12 09:20:08.514 7fe87ef7a700 1 mds.0.journaler.mdlog(ro) read_head -1010> 2019-11-12 09:20:08.514 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:7 6.14 6:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -1010> 2019-11-12 09:20:08.514 7fe87ef7a700 4 mds.0.log Waiting for journal 0x200 to recover... -1010> 2019-11-12 09:20:08.514 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 1 0x5574ef7fc840 osd_op_reply(4 500.00000000 [read 0~90] v0'0 uv121857 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.514 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 1 ==== osd_op_reply(4 500.00000000 [read 0~90] v0'0 uv121857 ondisk = 0) v8 ==== 156+0+90 (446579549 0 2262816634) 0x5574ef7fc840 con 0x5574ef703600 -1010> 2019-11-12 09:20:08.518 7fe88077d700 1 mds.0.journaler.pq(ro) _finish_read_head loghead(trim 62914560, expire 64461072, write 64480126, stream_format 1). probing for end of log (from 64480126)... -1010> 2019-11-12 09:20:08.518 7fe88077d700 1 mds.0.journaler.pq(ro) probing for end of the log -1010> 2019-11-12 09:20:08.518 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:8 6.25 6:a64c6dbf:::500.0000000f:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef810000 con 0 -1010> 2019-11-12 09:20:08.518 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:9 6.51 6:8a60f78d:::500.00000010:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -1010> 2019-11-12 09:20:08.518 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 4 0x5574ef7fd340 osd_op_reply(7 200.00000000 [read 0~90] v0'0 uv1334401 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.518 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 4 ==== osd_op_reply(7 200.00000000 [read 0~90] v0'0 uv1334401 ondisk = 0) v8 ==== 156+0+90 (4261138009 0 1159998223) 0x5574ef7fd340 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.518 7fe87f77b700 1 mds.0.journaler.mdlog(ro) _finish_read_head loghead(trim 33025949696, expire 33036315012, write 33239037805, stream_format 1). probing for end of log (from 33239037805)... -1010> 2019-11-12 09:20:08.518 7fe87f77b700 1 mds.0.journaler.mdlog(ro) probing for end of the log -1010> 2019-11-12 09:20:08.518 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:10 6.9 6:916b6fb1:::200.00001ef4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -1010> 2019-11-12 09:20:08.518 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:11 6.52 6:4b787142:::200.00001ef5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -1010> 2019-11-12 09:20:08.518 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 2 0x5574ef7fd8c0 osd_op_reply(9 500.00000010 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:08.518 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 2 ==== osd_op_reply(9 500.00000010 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (2138545714 0 0) 0x5574ef7fd8c0 con 0x5574ef703000 -1010> 2019-11-12 09:20:08.518 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 3 0x5574ef7fd8c0 osd_op_reply(11 200.00001ef5 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:08.518 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 3 ==== osd_op_reply(11 200.00001ef5 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (4142874598 0 0) 0x5574ef7fd8c0 con 0x5574ef703000 -1010> 2019-11-12 09:20:08.518 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 2 0x5574ef810000 osd_op_reply(8 500.0000000f [stat] v0'0 uv76310 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.522 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 2 ==== osd_op_reply(8 500.0000000f [stat] v0'0 uv76310 ondisk = 0) v8 ==== 156+0+16 (1441095022 0 2950776802) 0x5574ef810000 con 0x5574ef703600 -1010> 2019-11-12 09:20:08.518 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 5 0x5574ef8102c0 osd_op_reply(10 200.00001ef4 [stat] v0'0 uv43315 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 5 ==== osd_op_reply(10 200.00001ef4 [stat] v0'0 uv43315 ondisk = 0) v8 ==== 156+0+16 (586066544 0 3314252265) 0x5574ef8102c0 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.522 7fe88077d700 1 mds.0.journaler.pq(ro) _finish_probe_end write_pos = 64480126 (header had 64480126). recovered. -1010> 2019-11-12 09:20:08.522 7fe88077d700 4 mds.0.purge_queue operator(): open complete -1010> 2019-11-12 09:20:08.522 7fe88077d700 1 mds.0.journaler.pq(ro) set_writeable -1010> 2019-11-12 09:20:08.522 7fe87f77b700 1 mds.0.journaler.mdlog(ro) _finish_probe_end write_pos = 33239061177 (header had 33239037805). recovered. -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 4 mds.0.log Journal 0x200 recovered. -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 4 mds.0.log Recovered journal 0x200 in format 1 -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 2 mds.0.56303 boot_start 1: loading/discovering base inodes -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 0 mds.0.cache creating system inode with ino:0x100 -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:12 6.33 6:cd5a64a3:::100.00000000:head [getxattr inode] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:13 6.3f 6:fe07bba1:::100.00000000.inode:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 0 mds.0.cache creating system inode with ino:0x1 -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:14 6.3f 6:ff5b34d6:::1.00000000:head [getxattr inode] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8118c0 con 0 -1010> 2019-11-12 09:20:08.522 7fe87ef7a700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:15 6.14 6:287034c4:::1.00000000.inode:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811b80 con 0 -1010> 2019-11-12 09:20:08.522 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 3 0x5574ef810000 osd_op_reply(12 100.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 -1010> 2019-11-12 09:20:08.522 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 3 ==== osd_op_reply(12 100.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 156+0+0 (3193329554 0 0) 0x5574ef810000 con 0x5574ef703600 -1010> 2019-11-12 09:20:08.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 6 0x5574ef811b80 osd_op_reply(13 100.00000000.inode [read 0~544] v0'0 uv79071 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 6 ==== osd_op_reply(13 100.00000000.inode [read 0~544] v0'0 uv79071 ondisk = 0) v8 ==== 162+0+544 (1288455672 0 1233160524) 0x5574ef811b80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 7 0x5574ef811b80 osd_op_reply(14 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 -1010> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 7 ==== osd_op_reply(14 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 154+0+0 (37684714 0 0) 0x5574ef811b80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 8 0x5574ef811b80 osd_op_reply(15 1.00000000.inode [read 0~544] v0'0 uv1322881 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.522 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 8 ==== osd_op_reply(15 1.00000000.inode [read 0~544] v0'0 uv1322881 ondisk = 0) v8 ==== 160+0+544 (2713499666 0 667298071) 0x5574ef811b80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.522 7fe87f77b700 2 mds.0.56303 boot_start 2: replaying mds log -1010> 2019-11-12 09:20:08.522 7fe87f77b700 2 mds.0.56303 boot_start 2: waiting for purge queue recovered -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:16 6.54 6:2be44c64:::200.00001ec4:head [read 1976708~2217596 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:17 6.25 6:a7e44d1f:::200.00001ec5:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:18 6.2c 6:34509b9d:::200.00001ec6:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:19 6.63 6:c60e2601:::200.00001ec7:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:20 6.7 6:e1750875:::200.00001ec8:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fdb80 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:21 6.34 6:2f48296a:::200.00001ec9:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7ad600 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:22 6.35 6:ad314bae:::200.00001eca:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:23 6.56 6:6b0e3b9d:::200.00001ecb:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:24 6.3d 6:bed7c08e:::200.00001ecc:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:25 6.5b 6:dab203d3:::200.00001ecd:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd8c0 con 0 -1010> 2019-11-12 09:20:08.526 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:26 6.1f 6:f94beeba:::200.00001ece:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef8118c0 con 0 -1010> 2019-11-12 09:20:08.526 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 4 0x5574ef810000 osd_op_reply(5 mds0_openfiles.0 [omap-get-header,omap-get-vals] v0'0 uv83532 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.526 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 4 ==== osd_op_reply(5 mds0_openfiles.0 [omap-get-header,omap-get-vals] v0'0 uv83532 ondisk = 0) v8 ==== 202+0+1949 (4240281736 0 2842386935) 0x5574ef810000 con 0x5574ef703600 -1010> 2019-11-12 09:20:08.534 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:08.582 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 1 0x5574ef7fd340 osd_op_reply(19 200.00001ec7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv32228 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.582 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 1 ==== osd_op_reply(19 200.00001ec7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv32228 ondisk = 0) v8 ==== 156+0+4194304 (255713159 0 3389336695) 0x5574ef7fd340 con 0x5574ef85c000 -1010> 2019-11-12 09:20:08.750 7fe883783700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +111-0 packed 1374) v6 -- 0x5574ef565b00 con 0 -1010> 2019-11-12 09:20:08.758 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 4 0x5574ef811600 osd_op_reply(16 200.00001ec4 [read 1976708~2217596 [fadvise_dontneed]] v0'0 uv32403 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.758 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 4 ==== osd_op_reply(16 200.00001ec4 [read 1976708~2217596 [fadvise_dontneed]] v0'0 uv32403 ondisk = 0) v8 ==== 156+0+2217596 (3528559444 0 2254027955) 0x5574ef811600 con 0x5574ef703000 -1010> 2019-11-12 09:20:08.862 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 9 0x5574ef7fd340 osd_op_reply(23 200.00001ecb [read 0~4194304 [fadvise_dontneed]] v0'0 uv36300 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.862 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 9 ==== osd_op_reply(23 200.00001ecb [read 0~4194304 [fadvise_dontneed]] v0'0 uv36300 ondisk = 0) v8 ==== 156+0+4194304 (3043000747 0 150218771) 0x5574ef7fd340 con 0x5574ef702a00 -1010> 2019-11-12 09:20:08.982 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 5 0x5574ef810000 osd_op_reply(22 200.00001eca [read 0~4194304 [fadvise_dontneed]] v0'0 uv58091 ondisk = 0) v8 -1010> 2019-11-12 09:20:08.986 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 5 ==== osd_op_reply(22 200.00001eca [read 0~4194304 [fadvise_dontneed]] v0'0 uv58091 ondisk = 0) v8 ==== 156+0+4194304 (1537789602 0 1563593434) 0x5574ef810000 con 0x5574ef703600 -1010> 2019-11-12 09:20:09.022 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 6 0x5574ef810000 osd_op_reply(17 200.00001ec5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74584 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.022 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 6 ==== osd_op_reply(17 200.00001ec5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74584 ondisk = 0) v8 ==== 156+0+4194304 (1536332796 0 4020157265) 0x5574ef810000 con 0x5574ef703600 -1010> 2019-11-12 09:20:09.026 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:27 6.5d 6:ba600e0b:::200.00001ecf:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90580 con 0 -1010> 2019-11-12 09:20:09.058 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 5 0x5574ef811600 osd_op_reply(20 200.00001ec8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33049 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.058 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 5 ==== osd_op_reply(20 200.00001ec8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33049 ondisk = 0) v8 ==== 156+0+4194304 (3747932941 0 3671309179) 0x5574ef811600 con 0x5574ef703000 -1010> 2019-11-12 09:20:09.170 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 6 0x5574ef811600 osd_op_reply(18 200.00001ec6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv59925 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.170 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 6 ==== osd_op_reply(18 200.00001ec6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv59925 ondisk = 0) v8 ==== 156+0+4194304 (3708667789 0 915352419) 0x5574ef811600 con 0x5574ef703000 -1010> 2019-11-12 09:20:09.170 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:28 6.5c 6:3ae49781:::200.00001ed0:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91b80 con 0 -1010> 2019-11-12 09:20:09.202 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 7 0x5574efe91b80 osd_op_reply(24 200.00001ecc [read 0~4194304 [fadvise_dontneed]] v0'0 uv88595 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.202 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 7 ==== osd_op_reply(24 200.00001ecc [read 0~4194304 [fadvise_dontneed]] v0'0 uv88595 ondisk = 0) v8 ==== 156+0+4194304 (1801352771 0 3873878317) 0x5574efe91b80 con 0x5574ef703000 -1010> 2019-11-12 09:20:09.326 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:29 6.2c 6:35096768:::200.00001ed1:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91600 con 0 -1010> 2019-11-12 09:20:09.330 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 8 0x5574efe91b80 osd_op_reply(21 200.00001ec9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76305 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.334 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 8 ==== osd_op_reply(21 200.00001ec9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76305 ondisk = 0) v8 ==== 156+0+4194304 (4031564057 0 4277249194) 0x5574efe91b80 con 0x5574ef703000 -1010> 2019-11-12 09:20:09.534 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 10 0x5574ef7fd340 osd_op_reply(25 200.00001ecd [read 0~4194304 [fadvise_dontneed]] v0'0 uv35517 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.534 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 10 ==== osd_op_reply(25 200.00001ecd [read 0~4194304 [fadvise_dontneed]] v0'0 uv35517 ondisk = 0) v8 ==== 156+0+4194304 (1298770794 0 1145050672) 0x5574ef7fd340 con 0x5574ef702a00 -1010> 2019-11-12 09:20:09.538 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 7 0x5574efe90580 osd_op_reply(27 200.00001ecf [read 0~4194304 [fadvise_dontneed]] v0'0 uv42371 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.538 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 7 ==== osd_op_reply(27 200.00001ecf [read 0~4194304 [fadvise_dontneed]] v0'0 uv42371 ondisk = 0) v8 ==== 156+0+4194304 (1417620671 0 937541238) 0x5574efe90580 con 0x5574ef703600 -1010> 2019-11-12 09:20:09.546 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:30 6.39 6:9cc82f3f:::200.00001ed2:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90b00 con 0 -1010> 2019-11-12 09:20:09.650 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:31 6.a 6:50660169:::200.00001ed3:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91340 con 0 -1010> 2019-11-12 09:20:09.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 9 0x5574efe91b80 osd_op_reply(28 200.00001ed0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv40793 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.702 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 9 ==== osd_op_reply(28 200.00001ed0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv40793 ondisk = 0) v8 ==== 156+0+4194304 (4165677614 0 2205248570) 0x5574efe91b80 con 0x5574ef703000 -1010> 2019-11-12 09:20:09.738 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:32 6.26 6:642c7156:::200.00001ed4:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -1010> 2019-11-12 09:20:09.766 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 10 0x5574efe91b80 osd_op_reply(29 200.00001ed1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv61317 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.766 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 10 ==== osd_op_reply(29 200.00001ed1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv61317 ondisk = 0) v8 ==== 156+0+4194304 (2035222751 0 1620268375) 0x5574efe91b80 con 0x5574ef703000 -1010> 2019-11-12 09:20:09.822 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:33 6.24 6:26ed0d4a:::200.00001ed5:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0840 con 0 -1010> 2019-11-12 09:20:09.902 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:34 6.32 6:4fa4d314:::200.00001ed6:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90dc0 con 0 -1010> 2019-11-12 09:20:09.938 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 11 0x5574ef7fd340 osd_op_reply(26 200.00001ece [read 0~4194304 [fadvise_dontneed]] v0'0 uv45520 ondisk = 0) v8 -1010> 2019-11-12 09:20:09.938 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 11 ==== osd_op_reply(26 200.00001ece [read 0~4194304 [fadvise_dontneed]] v0'0 uv45520 ondisk = 0) v8 ==== 156+0+4194304 (1747664234 0 852305962) 0x5574ef7fd340 con 0x5574ef702a00 -1010> 2019-11-12 09:20:09.994 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:35 6.34 6:2da236a2:::200.00001ed7:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f18c0 con 0 -1010> 2019-11-12 09:20:10.070 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:36 6.32 6:4da6cd2c:::200.00001ed8:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90840 con 0 -1010> 2019-11-12 09:20:10.150 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:37 6.54 6:2ada770a:::200.00001ed9:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0840 con 0 -1010> 2019-11-12 09:20:10.154 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 11 0x5574f17f0840 osd_op_reply(31 200.00001ed3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv34980 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.154 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 11 ==== osd_op_reply(31 200.00001ed3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv34980 ondisk = 0) v8 ==== 156+0+4194304 (1386494933 0 3820517588) 0x5574f17f0840 con 0x5574ef703000 -1010> 2019-11-12 09:20:10.230 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:38 6.4c 6:33fe30a9:::200.00001eda:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90580 con 0 -1010> 2019-11-12 09:20:10.258 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 8 0x5574f17f1340 osd_op_reply(32 200.00001ed4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv80695 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.258 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 8 ==== osd_op_reply(32 200.00001ed4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv80695 ondisk = 0) v8 ==== 156+0+4194304 (2138519371 0 3530924944) 0x5574f17f1340 con 0x5574ef703600 -1010> 2019-11-12 09:20:10.290 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 12 0x5574f17f0840 osd_op_reply(30 200.00001ed2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90380 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.290 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 12 ==== osd_op_reply(30 200.00001ed2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90380 ondisk = 0) v8 ==== 156+0+4194304 (330583361 0 2656011774) 0x5574f17f0840 con 0x5574ef703000 -1010> 2019-11-12 09:20:10.310 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:39 6.2f 6:f40177de:::200.00001edb:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0000 con 0 -1010> 2019-11-12 09:20:10.386 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:40 6.6 6:601443b2:::200.00001edc:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe902c0 con 0 -1010> 2019-11-12 09:20:10.418 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 13 0x5574f17f0840 osd_op_reply(33 200.00001ed5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv68742 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.418 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 13 ==== osd_op_reply(33 200.00001ed5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv68742 ondisk = 0) v8 ==== 156+0+4194304 (4128139589 0 1631669900) 0x5574f17f0840 con 0x5574ef703000 -1010> 2019-11-12 09:20:10.470 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:41 6.3d 6:bc1d2bb0:::200.00001edd:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0580 con 0 -1010> 2019-11-12 09:20:10.562 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:42 6.3d 6:bdbfd417:::200.00001ede:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90dc0 con 0 -1010> 2019-11-12 09:20:10.638 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:43 6.3b 6:de40b110:::200.00001edf:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91340 con 0 -1010> 2019-11-12 09:20:10.834 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 14 0x5574efe91340 osd_op_reply(34 200.00001ed6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv73340 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.834 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 14 ==== osd_op_reply(34 200.00001ed6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv73340 ondisk = 0) v8 ==== 156+0+4194304 (3979275711 0 3536306776) 0x5574efe91340 con 0x5574ef703000 -1010> 2019-11-12 09:20:10.838 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:44 6.1c 6:39989f6c:::200.00001ee0:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91b80 con 0 -1010> 2019-11-12 09:20:10.862 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 15 0x5574efe91340 osd_op_reply(37 200.00001ed9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33829 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.862 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 15 ==== osd_op_reply(37 200.00001ed9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33829 ondisk = 0) v8 ==== 156+0+4194304 (3093391751 0 342787056) 0x5574efe91340 con 0x5574ef703000 -1010> 2019-11-12 09:20:10.878 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 2 0x5574efe91b80 osd_op_reply(44 200.00001ee0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv27219 ondisk = 0) v8 -1010> 2019-11-12 09:20:10.878 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 2 ==== osd_op_reply(44 200.00001ee0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv27219 ondisk = 0) v8 ==== 156+0+4194304 (2562693100 0 1712739379) 0x5574efe91b80 con 0x5574ef85c000 -1010> 2019-11-12 09:20:11.038 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 16 0x5574efe91340 osd_op_reply(35 200.00001ed7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv78116 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.038 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 16 ==== osd_op_reply(35 200.00001ed7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv78116 ondisk = 0) v8 ==== 156+0+4194304 (892522787 0 742479174) 0x5574efe91340 con 0x5574ef703000 -1010> 2019-11-12 09:20:11.038 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:45 6.39 6:9f1e4369:::200.00001ee1:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1b80 con 0 -1010> 2019-11-12 09:20:11.210 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 17 0x5574f17f1b80 osd_op_reply(41 200.00001edd [read 0~4194304 [fadvise_dontneed]] v0'0 uv89661 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.210 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 17 ==== osd_op_reply(41 200.00001edd [read 0~4194304 [fadvise_dontneed]] v0'0 uv89661 ondisk = 0) v8 ==== 156+0+4194304 (4104850994 0 1352966362) 0x5574f17f1b80 con 0x5574ef703000 -1010> 2019-11-12 09:20:11.386 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 12 0x5574efe91b80 osd_op_reply(39 200.00001edb [read 0~4194304 [fadvise_dontneed]] v0'0 uv78368 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.386 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 12 ==== osd_op_reply(39 200.00001edb [read 0~4194304 [fadvise_dontneed]] v0'0 uv78368 ondisk = 0) v8 ==== 156+0+4194304 (827480192 0 133231095) 0x5574efe91b80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:11.502 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 18 0x5574f17f1b80 osd_op_reply(36 200.00001ed8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74659 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.502 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 18 ==== osd_op_reply(36 200.00001ed8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv74659 ondisk = 0) v8 ==== 156+0+4194304 (2417045927 0 23600393) 0x5574f17f1b80 con 0x5574ef703000 -1010> 2019-11-12 09:20:11.502 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:46 6.2b 6:d75ef202:::200.00001ee2:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91080 con 0 -1010> 2019-11-12 09:20:11.522 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 13 0x5574efe91b80 osd_op_reply(38 200.00001eda [read 0~4194304 [fadvise_dontneed]] v0'0 uv31579 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.526 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 13 ==== osd_op_reply(38 200.00001eda [read 0~4194304 [fadvise_dontneed]] v0'0 uv31579 ondisk = 0) v8 ==== 156+0+4194304 (1167083545 0 1245400058) 0x5574efe91b80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:11.554 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 19 0x5574efe91080 osd_op_reply(43 200.00001edf [read 0~4194304 [fadvise_dontneed]] v0'0 uv71835 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.554 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 19 ==== osd_op_reply(43 200.00001edf [read 0~4194304 [fadvise_dontneed]] v0'0 uv71835 ondisk = 0) v8 ==== 156+0+4194304 (1016868077 0 408330945) 0x5574efe91080 con 0x5574ef703000 -1010> 2019-11-12 09:20:11.582 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:47 6.37 6:ee598252:::200.00001ee3:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90b00 con 0 -1010> 2019-11-12 09:20:11.658 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:48 6.38 6:1f27801f:::200.00001ee4:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -1010> 2019-11-12 09:20:11.738 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:49 6.26 6:669afea4:::200.00001ee5:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90000 con 0 -1010> 2019-11-12 09:20:11.742 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 20 0x5574efe90b00 osd_op_reply(45 200.00001ee1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv91372 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.742 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 20 ==== osd_op_reply(45 200.00001ee1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv91372 ondisk = 0) v8 ==== 156+0+4194304 (2648663446 0 2448890689) 0x5574efe90b00 con 0x5574ef703000 -1010> 2019-11-12 09:20:11.762 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:replay seq 3 -1010> 2019-11-12 09:20:11.762 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:11.762 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:replay seq 3 v56303) v7 -- 0x5574ef6ff880 con 0 -1010> 2019-11-12 09:20:11.762 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 14 0x5574ef6ff180 mdsbeacon(1849248/k8s-node-02 up:replay seq 3 v56303) v7 -1010> 2019-11-12 09:20:11.762 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 14 ==== mdsbeacon(1849248/k8s-node-02 up:replay seq 3 v56303) v7 ==== 136+0+0 (344388228 0 0) 0x5574ef6ff180 con 0x5574ef700c00 -1010> 2019-11-12 09:20:11.762 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:replay seq 3 rtt 0 -1010> 2019-11-12 09:20:11.846 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 21 0x5574efe90b00 osd_op_reply(42 200.00001ede [read 0~4194304 [fadvise_dontneed]] v0'0 uv90707 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.846 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 21 ==== osd_op_reply(42 200.00001ede [read 0~4194304 [fadvise_dontneed]] v0'0 uv90707 ondisk = 0) v8 ==== 156+0+4194304 (1278859041 0 4166000365) 0x5574efe90b00 con 0x5574ef703000 -1010> 2019-11-12 09:20:11.862 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 9 0x5574f2fc2000 osd_op_reply(48 200.00001ee4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv87902 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.862 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 9 ==== osd_op_reply(48 200.00001ee4 [read 0~4194304 [fadvise_dontneed]] v0'0 uv87902 ondisk = 0) v8 ==== 156+0+4194304 (1710263652 0 2550500820) 0x5574f2fc2000 con 0x5574ef703600 -1010> 2019-11-12 09:20:11.898 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 10 0x5574f2fc2000 osd_op_reply(49 200.00001ee5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv82944 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.898 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 10 ==== osd_op_reply(49 200.00001ee5 [read 0~4194304 [fadvise_dontneed]] v0'0 uv82944 ondisk = 0) v8 ==== 156+0+4194304 (2930123064 0 737151539) 0x5574f2fc2000 con 0x5574ef703600 -1010> 2019-11-12 09:20:11.954 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 14 0x5574efe91b80 osd_op_reply(40 200.00001edc [read 0~4194304 [fadvise_dontneed]] v0'0 uv31386 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.954 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 14 ==== osd_op_reply(40 200.00001edc [read 0~4194304 [fadvise_dontneed]] v0'0 uv31386 ondisk = 0) v8 ==== 156+0+4194304 (1211016059 0 614261651) 0x5574efe91b80 con 0x5574ef702a00 -1010> 2019-11-12 09:20:11.954 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:50 6.25 6:a74123a6:::200.00001ee6:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe918c0 con 0 -1010> 2019-11-12 09:20:11.974 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 22 0x5574efe90b00 osd_op_reply(46 200.00001ee2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90768 ondisk = 0) v8 -1010> 2019-11-12 09:20:11.974 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 22 ==== osd_op_reply(46 200.00001ee2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv90768 ondisk = 0) v8 ==== 156+0+4194304 (1649087458 0 600344932) 0x5574efe90b00 con 0x5574ef703000 -1010> 2019-11-12 09:20:12.006 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 23 0x5574efe90b00 osd_op_reply(47 200.00001ee3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv70914 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.010 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 23 ==== osd_op_reply(47 200.00001ee3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv70914 ondisk = 0) v8 ==== 156+0+4194304 (4189387006 0 3128680832) 0x5574efe90b00 con 0x5574ef703000 -1010> 2019-11-12 09:20:12.034 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:51 6.27 6:e7e71572:::200.00001ee7:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7ad600 con 0 -1010> 2019-11-12 09:20:12.102 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 11 0x5574efe918c0 osd_op_reply(50 200.00001ee6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76122 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.102 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 11 ==== osd_op_reply(50 200.00001ee6 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76122 ondisk = 0) v8 ==== 156+0+4194304 (4020623717 0 1562818712) 0x5574efe918c0 con 0x5574ef703600 -1010> 2019-11-12 09:20:12.106 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:52 6.3c 6:3e39cde3:::200.00001ee8:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1080 con 0 -1010> 2019-11-12 09:20:12.170 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:53 6.8 6:101387a1:::200.00001ee9:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -1010> 2019-11-12 09:20:12.198 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 24 0x5574efe918c0 osd_op_reply(51 200.00001ee7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76537 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.198 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 24 ==== osd_op_reply(51 200.00001ee7 [read 0~4194304 [fadvise_dontneed]] v0'0 uv76537 ondisk = 0) v8 ==== 156+0+4194304 (2339449442 0 2022356555) 0x5574efe918c0 con 0x5574ef703000 -1010> 2019-11-12 09:20:12.238 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6802/19981 -- osd_op(unknown.0.56303:54 6.46 6:631b0aa0:::200.00001eea:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fdb80 con 0 -1010> 2019-11-12 09:20:12.246 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6802/19981 conn(0x5574ef85c600 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=1)._process_connection got newly_acked_seq 0 vs out_seq 0 -1010> 2019-11-12 09:20:12.306 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:55 6.23 6:c571b352:::200.00001eeb:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe91600 con 0 -1010> 2019-11-12 09:20:12.326 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6802/19981 conn(0x5574ef85c600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5834 cs=1 l=1). rx osd.2 seq 1 0x5574efe91080 osd_op_reply(54 200.00001eea [read 0~4194304 [fadvise_dontneed]] v0'0 uv43008 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.326 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.2 192.168.200.17:6802/19981 1 ==== osd_op_reply(54 200.00001eea [read 0~4194304 [fadvise_dontneed]] v0'0 uv43008 ondisk = 0) v8 ==== 156+0+4194304 (1141206684 0 1871802516) 0x5574efe91080 con 0x5574ef85c600 -1010> 2019-11-12 09:20:12.374 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:56 6.3d 6:be55f99a:::200.00001eec:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -1010> 2019-11-12 09:20:12.438 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:57 6.15 6:a8763251:::200.00001eed:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fcb00 con 0 -1010> 2019-11-12 09:20:12.506 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:58 6.5 6:a03f7ae4:::200.00001eee:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0b00 con 0 -1010> 2019-11-12 09:20:12.538 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 15 0x5574f17f1080 osd_op_reply(52 200.00001ee8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv75454 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.538 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 15 ==== osd_op_reply(52 200.00001ee8 [read 0~4194304 [fadvise_dontneed]] v0'0 uv75454 ondisk = 0) v8 ==== 156+0+4194304 (1339885820 0 2582243991) 0x5574f17f1080 con 0x5574ef702a00 -1010> 2019-11-12 09:20:12.570 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:59 6.27 6:e42c9344:::200.00001eef:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1600 con 0 -1010> 2019-11-12 09:20:12.582 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 12 0x5574efe91600 osd_op_reply(53 200.00001ee9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv37098 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.582 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 12 ==== osd_op_reply(53 200.00001ee9 [read 0~4194304 [fadvise_dontneed]] v0'0 uv37098 ondisk = 0) v8 ==== 156+0+4194304 (3031359962 0 2756808336) 0x5574efe91600 con 0x5574ef703600 -1010> 2019-11-12 09:20:12.638 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:60 6.3e 6:7d6c9823:::200.00001ef0:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90580 con 0 -1010> 2019-11-12 09:20:12.702 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:61 6.45 6:a3b8f91c:::200.00001ef1:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574efe90000 con 0 -1010> 2019-11-12 09:20:12.766 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:62 6.50 6:0af53e21:::200.00001ef2:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1340 con 0 -1010> 2019-11-12 09:20:12.802 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 3 0x5574efe902c0 osd_op_reply(62 200.00001ef2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv28322 ondisk = 0) v8 -1010> 2019-11-12 09:20:12.802 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 3 ==== osd_op_reply(62 200.00001ef2 [read 0~4194304 [fadvise_dontneed]] v0'0 uv28322 ondisk = 0) v8 ==== 156+0+4194304 (1122511463 0 79703917) 0x5574efe902c0 con 0x5574ef85c000 -1010> 2019-11-12 09:20:12.834 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:63 6.20 6:059200a5:::200.00001ef3:head [read 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810000 con 0 -1010> 2019-11-12 09:20:12.902 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:64 6.9 6:916b6fb1:::200.00001ef4:head [read 0~3396281 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -1010> 2019-11-12 09:20:13.398 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 25 0x5574f17f1600 osd_op_reply(55 200.00001eeb [read 0~4194304 [fadvise_dontneed]] v0'0 uv35600 ondisk = 0) v8 -1010> 2019-11-12 09:20:13.398 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 25 ==== osd_op_reply(55 200.00001eeb [read 0~4194304 [fadvise_dontneed]] v0'0 uv35600 ondisk = 0) v8 ==== 156+0+4194304 (3905789527 0 2657527065) 0x5574f17f1600 con 0x5574ef703000 -1010> 2019-11-12 09:20:13.586 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 26 0x5574f17f1600 osd_op_reply(56 200.00001eec [read 0~4194304 [fadvise_dontneed]] v0'0 uv93869 ondisk = 0) v8 -1010> 2019-11-12 09:20:13.586 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 26 ==== osd_op_reply(56 200.00001eec [read 0~4194304 [fadvise_dontneed]] v0'0 uv93869 ondisk = 0) v8 ==== 156+0+4194304 (43855582 0 2499621118) 0x5574f17f1600 con 0x5574ef703000 -1010> 2019-11-12 09:20:13.710 7fe884785700 10 monclient: tick -1010> 2019-11-12 09:20:13.710 7fe884785700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-11-12 09:19:43.714163) -1010> 2019-11-12 09:20:13.750 7fe883783700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +0-0 packed 1374) v6 -- 0x5574f06ce000 con 0 -1010> 2019-11-12 09:20:13.826 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 27 0x5574f17f1600 osd_op_reply(59 200.00001eef [read 0~4194304 [fadvise_dontneed]] v0'0 uv81570 ondisk = 0) v8 -1010> 2019-11-12 09:20:13.826 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 27 ==== osd_op_reply(59 200.00001eef [read 0~4194304 [fadvise_dontneed]] v0'0 uv81570 ondisk = 0) v8 ==== 156+0+4194304 (473818481 0 1401989426) 0x5574f17f1600 con 0x5574ef703000 -1010> 2019-11-12 09:20:14.462 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 28 0x5574f17f1600 osd_op_reply(58 200.00001eee [read 0~4194304 [fadvise_dontneed]] v0'0 uv42535 ondisk = 0) v8 -1010> 2019-11-12 09:20:14.462 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 28 ==== osd_op_reply(58 200.00001eee [read 0~4194304 [fadvise_dontneed]] v0'0 uv42535 ondisk = 0) v8 ==== 156+0+4194304 (2933939241 0 808578468) 0x5574f17f1600 con 0x5574ef703000 -1010> 2019-11-12 09:20:14.538 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 29 0x5574f17f1600 osd_op_reply(57 200.00001eed [read 0~4194304 [fadvise_dontneed]] v0'0 uv37336 ondisk = 0) v8 -1010> 2019-11-12 09:20:14.538 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 29 ==== osd_op_reply(57 200.00001eed [read 0~4194304 [fadvise_dontneed]] v0'0 uv37336 ondisk = 0) v8 ==== 156+0+4194304 (910772900 0 1749242063) 0x5574f17f1600 con 0x5574ef703000 -1010> 2019-11-12 09:20:14.650 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 16 0x5574f06fe000 osd_op_reply(60 200.00001ef0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv71620 ondisk = 0) v8 -1010> 2019-11-12 09:20:14.650 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 16 ==== osd_op_reply(60 200.00001ef0 [read 0~4194304 [fadvise_dontneed]] v0'0 uv71620 ondisk = 0) v8 ==== 156+0+4194304 (916184623 0 823393747) 0x5574f06fe000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:14.878 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 17 0x5574f06fe000 osd_op_reply(61 200.00001ef1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv60049 ondisk = 0) v8 -1010> 2019-11-12 09:20:14.878 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 17 ==== osd_op_reply(61 200.00001ef1 [read 0~4194304 [fadvise_dontneed]] v0'0 uv60049 ondisk = 0) v8 ==== 156+0+4194304 (2574875128 0 582962543) 0x5574f06fe000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:15.762 7fe882781700 5 mds.beacon.k8s-node-02 Sending beacon up:replay seq 4 -1010> 2019-11-12 09:20:15.762 7fe882781700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:15.762 7fe882781700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:replay seq 4 v56303) v7 -- 0x5574ef80f180 con 0 -1010> 2019-11-12 09:20:15.762 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 15 0x5574ef80f180 mdsbeacon(1849248/k8s-node-02 up:replay seq 4 v56303) v7 -1010> 2019-11-12 09:20:15.762 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 15 ==== mdsbeacon(1849248/k8s-node-02 up:replay seq 4 v56303) v7 ==== 136+0+0 (3656543233 0 0) 0x5574ef80f180 con 0x5574ef700c00 -1010> 2019-11-12 09:20:15.762 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:replay seq 4 rtt 0 -1010> 2019-11-12 09:20:16.406 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 18 0x5574f06fe000 osd_op_reply(64 200.00001ef4 [read 0~3396281 [fadvise_dontneed]] v0'0 uv43315 ondisk = 0) v8 -1010> 2019-11-12 09:20:16.406 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 18 ==== osd_op_reply(64 200.00001ef4 [read 0~3396281 [fadvise_dontneed]] v0'0 uv43315 ondisk = 0) v8 ==== 156+0+3396281 (2978750866 0 537567138) 0x5574f06fe000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:16.482 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 19 0x5574f06fe000 osd_op_reply(63 200.00001ef3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33434 ondisk = 0) v8 -1010> 2019-11-12 09:20:16.482 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 19 ==== osd_op_reply(63 200.00001ef3 [read 0~4194304 [fadvise_dontneed]] v0'0 uv33434 ondisk = 0) v8 ==== 156+0+4194304 (2400805555 0 3258534103) 0x5574f06fe000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.56303 Finished replaying journal -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.56303 making mds journal writeable -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 mds.0.journaler.mdlog(ro) set_writeable -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:65 6.9 6:916b6fb1:::200.00001ef4:head [zero 3396281~798023] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:66 6.52 6:4b787142:::200.00001ef5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:67 6.3b 6:dd49a9ad:::200.00001ef6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef8118c0 con 0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:68 6.d 6:b10d4b81:::200.00001ef7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef8102c0 con 0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:69 6.2a 6:55d3aaa4:::200.00001ef8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:70 6.15 6:a9ceb397:::200.00001ef9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 2 mds.0.56303 i am alone, moving to state reconnect -1010> 2019-11-12 09:20:16.610 7fe87df78700 3 mds.0.56303 request_state up:reconnect -1010> 2019-11-12 09:20:16.610 7fe87df78700 5 mds.beacon.k8s-node-02 set_want_state: up:replay -> up:reconnect -1010> 2019-11-12 09:20:16.610 7fe87df78700 5 mds.beacon.k8s-node-02 Sending beacon up:reconnect seq 5 -1010> 2019-11-12 09:20:16.610 7fe87df78700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:16.610 7fe87df78700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:reconnect seq 5 v56303) v7 -- 0x5574ef80fc00 con 0 -1010> 2019-11-12 09:20:16.634 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 30 0x5574ef811600 osd_op_reply(66 200.00001ef5 [delete] v28661'40770 uv40694 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:16.634 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 30 ==== osd_op_reply(66 200.00001ef5 [delete] v28661'40770 uv40694 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (3875959471 0 0) 0x5574ef811600 con 0x5574ef703000 -1010> 2019-11-12 09:20:16.634 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 31 0x5574ef811600 osd_op_reply(70 200.00001ef9 [delete] v28661'37440 uv37342 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:16.634 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 31 ==== osd_op_reply(70 200.00001ef9 [delete] v28661'37440 uv37342 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (28087247 0 0) 0x5574ef811600 con 0x5574ef703000 -1010> 2019-11-12 09:20:16.638 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 32 0x5574ef811600 osd_op_reply(67 200.00001ef6 [delete] v28661'73070 uv72994 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:16.638 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 32 ==== osd_op_reply(67 200.00001ef6 [delete] v28661'73070 uv72994 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (4041088188 0 0) 0x5574ef811600 con 0x5574ef703000 -1010> 2019-11-12 09:20:16.638 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 20 0x5574f17f02c0 osd_op_reply(65 200.00001ef4 [zero 3396281~798023] v28661'43316 uv43315 ondisk = 0) v8 -1010> 2019-11-12 09:20:16.638 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 20 ==== osd_op_reply(65 200.00001ef4 [zero 3396281~798023] v28661'43316 uv43315 ondisk = 0) v8 ==== 156+0+0 (106571773 0 0) 0x5574f17f02c0 con 0x5574ef702a00 -1010> 2019-11-12 09:20:16.638 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 13 0x5574f17f1b80 osd_op_reply(69 200.00001ef8 [delete] v28661'80702 uv80626 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:16.638 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 13 ==== osd_op_reply(69 200.00001ef8 [delete] v28661'80702 uv80626 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (1754869307 0 0) 0x5574f17f1b80 con 0x5574ef703600 -1010> 2019-11-12 09:20:16.638 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 14 0x5574f17f1b80 osd_op_reply(68 200.00001ef7 [delete] v28661'38268 uv37712 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:16.638 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 14 ==== osd_op_reply(68 200.00001ef7 [delete] v28661'38268 uv37712 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (699459646 0 0) 0x5574f17f1b80 con 0x5574ef703600 -1010> 2019-11-12 09:20:16.682 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 16 0x5574f1198b40 mdsmap(e 56304) v1 -1010> 2019-11-12 09:20:16.686 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef700600 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=32 - -1010> 2019-11-12 09:20:16.686 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 16 ==== mdsmap(e 56304) v1 ==== 655+0+0 (1257921563 0 0) 0x5574f1198b40 con 0x5574ef700c00 -1010> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56304 from mon.0 -1010> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -1010> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:replay --> up:reconnect -1010> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 reconnect_start -1010> 2019-11-12 09:20:16.686 7fe885a11700 1 mds.0.56303 reopen_log -1010> 2019-11-12 09:20:16.686 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 17 0x5574ef80f180 mdsbeacon(1849248/k8s-node-02 up:reconnect seq 5 v56304) v7 -1010> 2019-11-12 09:20:16.686 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 17 ==== mdsbeacon(1849248/k8s-node-02 up:reconnect seq 5 v56304) v7 ==== 136+0+0 (1350244580 0 0) 0x5574ef80f180 con 0x5574ef700c00 -1010> 2019-11-12 09:20:16.686 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:reconnect seq 5 rtt 0.0760015 -1010> 2019-11-12 09:20:16.686 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85d200 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=34 - -1010> 2019-11-12 09:20:16.686 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85cc00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=33 - -1010> 2019-11-12 09:20:16.690 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85de00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=36 - -1010> 2019-11-12 09:20:16.694 7fe885a11700 4 mds.0.56303 reconnect_start: killed 0 blacklisted sessions (2132 blacklist entries, 14) -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 mds.0.server reconnect_clients -- 14 sessions -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 _send_message--> mds.0 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable server_ready) v1 -- ?+0 0x5574ef7c8b40 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable server_ready) v1 -- 0x5574ef7c8b40 con 0 -1010> 2019-11-12 09:20:16.694 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85d800 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=35 - -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mds.0 192.168.200.19:6800/1460470921 0 ==== mds_table_request(snaptable server_ready) v1 ==== 0+0+0 (0 0 0) 0x5574ef7c8b40 con 0x5574ef701800 -1010> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/979484066 conn(0x5574ef700600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4844 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.694 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:0/2796926812 conn(0x5574ef85de00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.694 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:0/3102681411 conn(0x5574ef85d200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/3544368504 conn(0x5574ef85d800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/3544368504 conn(0x5574ef85d800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 _send_message--> mds.0 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable query 1 9 bytes) v1 -- ?+0 0x5574f1198b40 -1010> 2019-11-12 09:20:16.694 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:0/3102681411 conn(0x5574ef85d200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.694 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/979484066 conn(0x5574ef700600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4844 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable query 1 9 bytes) v1 -- 0x5574f1198b40 con 0 -1010> 2019-11-12 09:20:16.694 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/979484066 conn(0x5574ef700600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4844 cs=1 l=0). rx client.1804200 seq 1 0x5574f1198900 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.694 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:0/3102681411 conn(0x5574ef85d200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804134 seq 1 0x5574f11986c0 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.694 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:0/2796926812 conn(0x5574ef85de00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.694 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/3544368504 conn(0x5574ef85d800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804143 seq 1 0x5574f1198fc0 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.694 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:0/2796926812 conn(0x5574ef85de00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804137 seq 1 0x5574ef7a7200 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804200 192.168.200.30:0/979484066 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1014479821) 0x5574f1198900 con 0x5574ef700600 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/979484066 -- client_session(open) v3 -- 0x5574ef7c8b40 con 0 -1010> 2019-11-12 09:20:16.694 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85e400 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=37 - -1010> 2019-11-12 09:20:16.694 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85f000 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=39 - -1010> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804200 192.168.200.30:0/979484066 after 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804134 192.168.200.17:0/3102681411 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3890766099) 0x5574f11986c0 con 0x5574ef85d200 -1010> 2019-11-12 09:20:16.694 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dc600 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=42 - -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:0/3102681411 -- client_session(open) v3 -- 0x5574f1198900 con 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804134 192.168.200.17:0/3102681411 after 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804143 192.168.200.31:0/3544368504 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 2998347668) 0x5574f1198fc0 con 0x5574ef85d800 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/3544368504 -- client_session(open) v3 -- 0x5574f11986c0 con 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804143 192.168.200.31:0/3544368504 after 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804137 192.168.200.19:0/2796926812 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1366303579) 0x5574ef7a7200 con 0x5574ef85de00 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:0/2796926812 -- client_session(open) v3 -- 0x5574f1198fc0 con 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804137 192.168.200.19:0/2796926812 after 0 -1010> 2019-11-12 09:20:16.694 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mds.0 192.168.200.19:6800/1460470921 0 ==== mds_table_request(snaptable query 1 9 bytes) v1 ==== 0+0+0 (0 0 0) 0x5574f1198b40 con 0x5574ef701800 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6800/1460470921 -- mds_table_request(snaptable query_reply 1 tid 2 29 bytes) v1 -- 0x5574ef7a7200 con 0 -1010> 2019-11-12 09:20:16.698 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85f600 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=40 - -1010> 2019-11-12 09:20:16.698 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dcc00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=43 - -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mds.0 192.168.200.19:6800/1460470921 0 ==== mds_table_request(snaptable query_reply 1 tid 2 29 bytes) v1 ==== 0+0+0 (0 0 0) 0x5574ef7a7200 con 0x5574ef701800 -1010> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4219 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1793818 seq 1 0x5574ef7c9440 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793818 192.168.200.30:0/232332927 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3089022758) 0x5574ef7c9440 con 0x5574ef85e400 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/232332927 -- client_session(open) v3 -- 0x5574ef7a7200 con 0 -1010> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/2855972843 conn(0x5574ef85cc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/2855972843 conn(0x5574ef85cc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4212 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793818 192.168.200.30:0/232332927 after 0.00400008 -1010> 2019-11-12 09:20:16.698 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574ef85ea00 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=38 - -1010> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=7530 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dc000 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=41 - -1010> 2019-11-12 09:20:16.698 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/2855972843 conn(0x5574ef85cc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1804140 seq 1 0x5574ef8646c0 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4292 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4219 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4219 cs=1 l=0). rx client.1792815 seq 1 0x5574f06ce900 client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.32:0/3593333399 conn(0x5574ef85f000 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4219 cs=1 l=0). rx client.1792815 seq 2 0x5574f1199200 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5028 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4292 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4292 cs=1 l=0). rx client.1794662 seq 1 0x5574f06ce600 client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4292 cs=1 l=0). rx client.1794662 seq 2 0x5574f06cec00 client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/522103062 conn(0x5574f11dc600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4292 cs=1 l=0). rx client.1794662 seq 3 0x5574f1199440 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1804140 192.168.200.32:0/2855972843 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3745454508) 0x5574ef8646c0 con 0x5574ef85cc00 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/2855972843 -- client_session(open) v3 -- 0x5574ef7c9440 con 0 -1010> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=7530 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5362 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 1 0x5574f06ce000 client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 2 0x5574ef565b00 client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 3 0x5574f06cf200 client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 4 0x5574f06cf500 client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 5 0x5574f06cf800 client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1804140 192.168.200.32:0/2855972843 after 0.00400008 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 6 0x5574f06cfb00 client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 7 0x5574f11f0000 client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 8 0x5574f11f0300 client_request(client.1794521:96231 lookup #0x100013ad07a/eula.txt 2019-11-12 08:59:14.149366 RETRY=30 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 9 0x5574f11f0600 client_request(client.1794521:96232 lookup #0x100013ad07a/eula.txt 2019-11-12 09:05:30.072151 RETRY=10 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1792815 192.168.200.32:0/3593333399 1 ==== client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 122+0+0 (1097225393 0 0) 0x5574f06ce900 con 0x5574ef85f000 -1010> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 10 0x5574f11f0900 client_request(client.1794521:96233 lookup #0x100013ad07a/eula.txt 2019-11-12 09:06:44.117495 RETRY=8 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 11 0x5574f11f0c00 client_request(client.1794521:96236 lookup #0x100013ad07a/eula.txt 2019-11-12 09:17:08.464874 RETRY=2 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 1 ==== client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2264195835 0 0) 0x5574f06ce000 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 12 0x5574f11f0f00 client_request(client.1794521:96238 lookup #0x100013ad07a/eula.txt 2019-11-12 09:18:43.942618 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 13 0x5574f11f1200 client_request(client.1794521:96239 getattr As #0x100013ad07a 2019-11-12 09:19:36.755583 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/1453103486 conn(0x5574ef85f600 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7530 cs=1 l=0). rx client.1794521 seq 14 0x5574ef864240 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794662 192.168.200.30:0/522103062 1 ==== client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 ==== 122+0+0 (231600759 0 0) 0x5574f06ce600 con 0x5574f11dc600 -1010> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1792815 192.168.200.32:0/3593333399 2 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1066103343) 0x5574f1199200 con 0x5574ef85f000 -1010> 2019-11-12 09:20:16.698 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5028 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/3593333399 -- client_session(open) v3 -- 0x5574ef8646c0 con 0 -1010> 2019-11-12 09:20:16.698 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/2985278080 conn(0x5574f11dc000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4285 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 1 0x5574f11f1500 client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 2 0x5574f11f1800 client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 3 0x5574f11f1b00 client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 4 0x5574f11fa000 client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1792815 192.168.200.32:0/3593333399 after 0.00400008 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 5 0x5574f11fa300 client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 6 0x5574f11fa600 client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 7 0x5574f11fa900 client_request(client.1794665:23347 lookup #0x10001346679/public 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 2 ==== client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (4027415736 0 0) 0x5574ef565b00 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.698 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 8 0x5574f11fac00 client_request(client.1794665:23348 lookup #0x1000134691b/autoload.php 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 9 0x5574f11faf00 client_request(client.1794665:23349 lookup #0x10001346679/private 2019-11-12 09:18:41.457643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:16.698 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 10 0x5574ef864000 client_reconnect(8 caps) v3 -1010> 2019-11-12 09:20:16.702 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/2985278080 conn(0x5574f11dc000 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4285 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/2985278080 conn(0x5574f11dc000 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4285 cs=1 l=0). rx client.1794668 seq 1 0x5574ef7a6900 client_reconnect(1 caps) v3 -1010> 2019-11-12 09:20:16.702 7fe888a17700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=5362 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 1 0x5574f11fb500 client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 2 0x5574f11fb800 client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:16.702 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 3 0x5574f1198000 client_reconnect(10 caps) v3 -1010> 2019-11-12 09:20:16.698 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794662 192.168.200.30:0/522103062 2 ==== client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 ==== 122+0+0 (906112753 0 0) 0x5574f06cec00 con 0x5574f11dc600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 1 ==== client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 ==== 128+0+0 (1435394209 0 0) 0x5574f11f1500 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793737 192.168.200.31:0/2398232289 1 ==== client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 ==== 122+0+0 (3576974081 0 0) 0x5574f11fb500 con 0x5574ef85ea00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 3 ==== client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (955679367 0 0) 0x5574f06cf200 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794668 192.168.200.30:0/2985278080 1 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 3819407093) 0x5574ef7a6900 con 0x5574f11dc000 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/2985278080 -- client_session(open) v3 -- 0x5574f1199200 con 0 -1010> 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794668 192.168.200.30:0/2985278080 after 0.00800016 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794662 192.168.200.30:0/522103062 3 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 2460232887) 0x5574f1199440 con 0x5574f11dc600 -1010> 2019-11-12 09:20:16.702 7fe888216700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dd200 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=44 - -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/522103062 -- client_session(open) v3 -- 0x5574ef7a6900 con 0 -1010> 2019-11-12 09:20:16.702 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 >> - conn(0x5574f11dd800 :6800 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=45 - -1010> 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794662 192.168.200.30:0/522103062 after 0.00800016 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 2 ==== client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (1022837727 0 0) 0x5574f11f1800 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793737 192.168.200.31:0/2398232289 2 ==== client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 ==== 124+0+0 (564909887 0 0) 0x5574f11fb800 con 0x5574ef85ea00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4244 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 4 ==== client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2539636667 0 0) 0x5574f06cf500 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4953 cs=1 l=0).handle_connect_msg accept write reply msg done -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 3 ==== client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (3472455810 0 0) 0x5574f11f1b00 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793737 192.168.200.31:0/2398232289 3 ==== client_reconnect(10 caps) v3 ==== 0+0+972 (0 0 3487394893) 0x5574f1198000 con 0x5574ef85ea00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_session(open) v3 -- 0x5574f1199440 con 0 -1010> 2019-11-12 09:20:16.702 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793737 192.168.200.31:0/2398232289 after 0.00800016 -1010> 2019-11-12 09:20:16.702 7fe887a15700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4244 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.702 7fe888216700 2 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_ACCEPTING_WAIT_SEQ pgs=4953 cs=1 l=0)._process_connection accept get newly_acked_seq 0 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 5 ==== client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2901537372 0 0) 0x5574f06cf800 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 4 ==== client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (134929482 0 0) 0x5574f11fa000 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 6 ==== client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2220845138 0 0) 0x5574f06cfb00 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 5 ==== client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 122+0+0 (3291907751 0 0) 0x5574f11fa300 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 1 0x5574f06cef00 client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.702 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4244 cs=1 l=0). rx client.1793815 seq 1 0x5574f11fb200 client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 7 ==== client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (305688313 0 0) 0x5574f11f0000 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 6 ==== client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 131+0+0 (873794425 0 0) 0x5574f11fa600 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 2 0x5574f1208000 client_request(client.1793728:9471 lookup #0x1000136753a/mime_types.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/4201708698 conn(0x5574f11dd800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4244 cs=1 l=0). rx client.1793815 seq 2 0x5574ef52b680 client_reconnect(7 caps) v3 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 1 ==== client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 136+0+0 (3106383072 0 0) 0x5574f06cef00 con 0x5574f11dd200 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 3 0x5574f1208300 client_request(client.1793728:9472 lookup #0x1000136753a/spf_dkim_whitelist.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793815 192.168.200.15:0/4201708698 1 ==== client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 ==== 168+0+0 (1137596064 0 0) 0x5574f11fb200 con 0x5574f11dd800 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 4 0x5574f1208600 client_request(client.1793728:9473 lookup #0x1000136753a/surbl-whitelist.inc.local 2019-11-12 09:18:41.736891 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 8 ==== client_request(client.1794521:96231 lookup #0x100013ad07a/eula.txt 2019-11-12 08:59:14.149366 RETRY=30 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (791300380 0 0) 0x5574f11f0300 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 5 0x5574f1208900 client_request(client.1793728:9474 getattr As #0x1000136753a 2019-11-12 09:19:29.005938 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:16.702 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 6 0x5574f1199680 client_reconnect(10 caps) v3 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 7 ==== client_request(client.1794665:23347 lookup #0x10001346679/public 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 128+0+0 (1953170597 0 0) 0x5574f11fa900 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 2 ==== client_request(client.1793728:9471 lookup #0x1000136753a/mime_types.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 142+0+0 (3893675223 0 0) 0x5574f1208000 con 0x5574f11dd200 -1010> 2019-11-12 09:20:16.702 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793815 192.168.200.15:0/4201708698 2 ==== client_reconnect(7 caps) v3 ==== 0+0+660 (0 0 427299514) 0x5574ef52b680 con 0x5574f11dd800 -1010> 2019-11-12 09:20:16.702 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/4201708698 -- client_session(open) v3 -- 0x5574f1198000 con 0 -1010> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793815 192.168.200.15:0/4201708698 after 0.00800016 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 9 ==== client_request(client.1794521:96232 lookup #0x100013ad07a/eula.txt 2019-11-12 09:05:30.072151 RETRY=10 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (3260212025 0 0) 0x5574f11f0600 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 8 ==== client_request(client.1794665:23348 lookup #0x1000134691b/autoload.php 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 134+0+0 (2222831593 0 0) 0x5574f11fac00 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 3 ==== client_request(client.1793728:9472 lookup #0x1000136753a/spf_dkim_whitelist.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 150+0+0 (3844774369 0 0) 0x5574f1208300 con 0x5574f11dd200 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 10 ==== client_request(client.1794521:96233 lookup #0x100013ad07a/eula.txt 2019-11-12 09:06:44.117495 RETRY=8 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (1788368277 0 0) 0x5574f11f0900 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 9 ==== client_request(client.1794665:23349 lookup #0x10001346679/private 2019-11-12 09:18:41.457643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 ==== 129+0+0 (575778027 0 0) 0x5574f11faf00 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 4 ==== client_request(client.1793728:9473 lookup #0x1000136753a/surbl-whitelist.inc.local 2019-11-12 09:18:41.736891 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 147+0+0 (166254165 0 0) 0x5574f1208600 con 0x5574f11dd200 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 11 ==== client_request(client.1794521:96236 lookup #0x100013ad07a/eula.txt 2019-11-12 09:17:08.464874 RETRY=2 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (2076245363 0 0) 0x5574f11f0c00 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794665 192.168.200.30:0/3506497893 10 ==== client_reconnect(8 caps) v3 ==== 0+0+631 (0 0 907812246) 0x5574ef864000 con 0x5574f11dcc00 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_session(open) v3 -- 0x5574ef52b680 con 0 -1010> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794665 192.168.200.30:0/3506497893 after 0.0120002 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 5 ==== client_request(client.1793728:9474 getattr As #0x1000136753a 2019-11-12 09:19:29.005938 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 122+0+0 (4065490981 0 0) 0x5574f1208900 con 0x5574f11dd200 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 12 ==== client_request(client.1794521:96238 lookup #0x100013ad07a/eula.txt 2019-11-12 09:18:43.942618 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 ==== 130+0+0 (1427172407 0 0) 0x5574f11f0f00 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 6 ==== client_reconnect(10 caps) v3 ==== 0+0+1051 (0 0 3070811223) 0x5574f1199680 con 0x5574f11dd200 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_session(open) v3 -- 0x5574ef864000 con 0 -1010> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1793728 192.168.200.15:0/1656050499 after 0.0120002 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 13 ==== client_request(client.1794521:96239 getattr As #0x100013ad07a 2019-11-12 09:19:36.755583 RETRY=1 caller_uid=0, caller_gid=0{}) v2 ==== 122+0+0 (376921402 0 0) 0x5574f11f1200 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.server not active yet, waiting -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1794521 192.168.200.31:0/1453103486 14 ==== client_reconnect(1 caps) v3 ==== 0+0+98 (0 0 1282897092) 0x5574ef864240 con 0x5574ef85f600 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/1453103486 -- client_session(open) v3 -- 0x5574f1199680 con 0 -1010> 2019-11-12 09:20:16.706 7fe885a11700 0 log_channel(cluster) log [DBG] : reconnect by client.1794521 192.168.200.31:0/1453103486 after 0.0120002 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 mds.0.56303 reconnect_done -1010> 2019-11-12 09:20:16.706 7fe885a11700 3 mds.0.56303 request_state up:rejoin -1010> 2019-11-12 09:20:16.706 7fe885a11700 5 mds.beacon.k8s-node-02 set_want_state: up:reconnect -> up:rejoin -1010> 2019-11-12 09:20:16.706 7fe885a11700 5 mds.beacon.k8s-node-02 Sending beacon up:rejoin seq 6 -1010> 2019-11-12 09:20:16.706 7fe885a11700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:16.706 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:rejoin seq 6 v56304) v7 -- 0x5574f1216000 con 0 -1010> 2019-11-12 09:20:17.710 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 18 0x5574ef864000 mdsmap(e 56305) v1 -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 18 ==== mdsmap(e 56305) v1 ==== 655+0+0 (1964398294 0 0) 0x5574ef864000 con 0x5574ef700c00 -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56305 from mon.0 -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:reconnect --> up:rejoin -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 mds.0.56303 rejoin_start -1010> 2019-11-12 09:20:17.710 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 19 0x5574f1216000 mdsbeacon(1849248/k8s-node-02 up:rejoin seq 6 v56305) v7 -1010> 2019-11-12 09:20:17.710 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 19 ==== mdsbeacon(1849248/k8s-node-02 up:rejoin seq 6 v56305) v7 ==== 136+0+0 (4245662320 0 0) 0x5574f1216000 con 0x5574ef700c00 -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:71 6.3f 6:ff5b34d6:::1.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -1010> 2019-11-12 09:20:17.710 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:rejoin seq 6 rtt 1.00402 -1010> 2019-11-12 09:20:17.710 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:72 6.5e 6:7a303e59:::1000001a3c6.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810840 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:73 6.b 6:d0f90662:::1000001eb17.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811b80 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:74 6.56 6:6bacae67:::1000001ffea.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:75 6.1e 6:7992850e:::1000133e06d.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f12262c0 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6802/19981 -- osd_op(unknown.0.56303:76 6.46 6:62644040:::10001366c72.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226840 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:77 6.5b 6:db1def57:::10001367140.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226dc0 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:78 6.43 6:c26089c3:::100013671a4.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0dc0 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:79 6.1c 6:38b3e7b3:::1000136753a.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7ad600 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:80 6.34 6:2c8bdca0:::100013ad077.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6801/12021 -- osd_op(unknown.0.56303:81 6.50 6:0a153a6f:::100013ad078.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd600 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:82 6.26 6:66061dfa:::100013ad079.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226580 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:83 6.3d 6:bfa6ab4a:::100013b1605.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1227340 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:84 6.43 6:c3272da7:::100013b1606.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0000 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:85 6.2c 6:36da4b25:::100013b1607.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f0840 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:86 6.55 6:abaeca41:::30000015b82.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f1b80 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:87 6.4 6:20b05573:::30000015ba9.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f12278c0 con 0 -1010> 2019-11-12 09:20:17.714 7fe885a11700 1 mds.0.56303 rejoin_joint_start -1010> 2019-11-12 09:20:17.714 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 21 0x5574ef7fd600 osd_op_reply(71 1.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv79171 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 33 0x5574f1227340 osd_op_reply(80 100013ad077.00000000 [omap-get-header,omap-get-vals,getxattr (69)] v0'0 uv78120 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.714 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 33 ==== osd_op_reply(80 100013ad077.00000000 [omap-get-header,omap-get-vals,getxattr (69)] v0'0 uv78120 ondisk = 0) v8 ==== 248+0+827 (1746941155 0 2754020800) 0x5574f1227340 con 0x5574ef703000 -1010> 2019-11-12 09:20:17.714 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 21 ==== osd_op_reply(71 1.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv79171 ondisk = 0) v8 ==== 238+0+17357 (3984365710 0 1328600032) 0x5574ef7fd600 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.714 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 34 0x5574f17f1080 osd_op_reply(83 100013b1605.00000000 [omap-get-header,omap-get-vals,getxattr (131)] v0'0 uv88597 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.714 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 34 ==== osd_op_reply(83 100013b1605.00000000 [omap-get-header,omap-get-vals,getxattr (131)] v0'0 uv88597 ondisk = 0) v8 ==== 248+0+898 (279877274 0 277830296) 0x5574f17f1080 con 0x5574ef703000 -1010> 2019-11-12 09:20:17.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6802/19981 conn(0x5574ef85c600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5834 cs=1 l=1). rx osd.2 seq 2 0x5574f17f0840 osd_op_reply(76 10001366c72.00000000 [omap-get-header,omap-get-vals,getxattr (135)] v0'0 uv41822 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.2 192.168.200.17:6802/19981 2 ==== osd_op_reply(76 10001366c72.00000000 [omap-get-header,omap-get-vals,getxattr (135)] v0'0 uv41822 ondisk = 0) v8 ==== 248+0+6454 (1231371730 0 289533519) 0x5574f17f0840 con 0x5574ef85c600 -1010> 2019-11-12 09:20:17.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 15 0x5574f17f1b80 osd_op_reply(75 1000133e06d.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv32888 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 15 ==== osd_op_reply(75 1000133e06d.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv32888 ondisk = 0) v8 ==== 248+0+13413 (2612537738 0 1454472965) 0x5574f17f1b80 con 0x5574ef703600 -1010> 2019-11-12 09:20:17.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 16 0x5574f1227600 osd_op_reply(82 100013ad079.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv79212 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 16 ==== osd_op_reply(82 100013ad079.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv79212 ondisk = 0) v8 ==== 248+0+823 (707074122 0 2606661828) 0x5574f1227600 con 0x5574ef703600 -1010> 2019-11-12 09:20:17.718 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 17 0x5574f17f1b80 osd_op_reply(86 30000015b82.00000000 [omap-get-header,omap-get-vals,getxattr (122)] v0'0 uv118734 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 17 ==== osd_op_reply(86 30000015b82.00000000 [omap-get-header,omap-get-vals,getxattr (122)] v0'0 uv118734 ondisk = 0) v8 ==== 248+0+8987 (973910231 0 4267468738) 0x5574f17f1b80 con 0x5574ef703600 -1010> 2019-11-12 09:20:17.718 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 22 0x5574ef7fcb00 osd_op_reply(72 1000001a3c6.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv46880 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 22 ==== osd_op_reply(72 1000001a3c6.00000000 [omap-get-header,omap-get-vals,getxattr (65)] v0'0 uv46880 ondisk = 0) v8 ==== 248+0+2748 (2985602565 0 4230017078) 0x5574ef7fcb00 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.718 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 23 0x5574ef7fd600 osd_op_reply(73 1000001eb17.00000000 [omap-get-header,omap-get-vals,getxattr (64)] v0'0 uv42866 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 23 ==== osd_op_reply(73 1000001eb17.00000000 [omap-get-header,omap-get-vals,getxattr (64)] v0'0 uv42866 ondisk = 0) v8 ==== 248+0+10888 (2827186539 0 1783259961) 0x5574ef7fd600 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.718 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 4 0x5574ef7fcb00 osd_op_reply(79 1000136753a.00000000 [omap-get-header,omap-get-vals,getxattr (252)] v0'0 uv26176 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 4 ==== osd_op_reply(79 1000136753a.00000000 [omap-get-header,omap-get-vals,getxattr (252)] v0'0 uv26176 ondisk = 0) v8 ==== 248+0+8987 (2899305789 0 1668207579) 0x5574ef7fcb00 con 0x5574ef85c000 -1010> 2019-11-12 09:20:17.718 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 35 0x5574f17f0580 osd_op_reply(85 100013b1607.00000000 [omap-get-header,omap-get-vals,getxattr (198)] v0'0 uv59900 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.718 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 35 ==== osd_op_reply(85 100013b1607.00000000 [omap-get-header,omap-get-vals,getxattr (198)] v0'0 uv59900 ondisk = 0) v8 ==== 248+0+3465 (1835850951 0 2096249374) 0x5574f17f0580 con 0x5574ef703000 -1010> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 24 0x5574ef7fd600 osd_op_reply(74 1000001ffea.00000000 [omap-get-header,omap-get-vals,getxattr (94)] v0'0 uv35408 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 24 ==== osd_op_reply(74 1000001ffea.00000000 [omap-get-header,omap-get-vals,getxattr (94)] v0'0 uv35408 ondisk = 0) v8 ==== 248+0+6097 (4183812923 0 414254997) 0x5574ef7fd600 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 25 0x5574ef7fcb00 osd_op_reply(77 10001367140.00000000 [omap-get-header,omap-get-vals,getxattr (220)] v0'0 uv35518 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 25 ==== osd_op_reply(77 10001367140.00000000 [omap-get-header,omap-get-vals,getxattr (220)] v0'0 uv35518 ondisk = 0) v8 ==== 248+0+3876 (2555161273 0 2568948404) 0x5574ef7fcb00 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 26 0x5574ef7fd600 osd_op_reply(78 100013671a4.00000000 [omap-get-header,omap-get-vals,getxattr (255)] v0'0 uv40089 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 26 ==== osd_op_reply(78 100013671a4.00000000 [omap-get-header,omap-get-vals,getxattr (255)] v0'0 uv40089 ondisk = 0) v8 ==== 248+0+4869 (3204699989 0 451091004) 0x5574ef7fd600 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 27 0x5574ef7fcb00 osd_op_reply(87 30000015ba9.00000000 [omap-get-header,omap-get-vals,getxattr (150)] v0'0 uv38565 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 27 ==== osd_op_reply(87 30000015ba9.00000000 [omap-get-header,omap-get-vals,getxattr (150)] v0'0 uv38565 ondisk = 0) v8 ==== 248+0+8319 (3722796679 0 2297239757) 0x5574ef7fcb00 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 28 0x5574ef7fd600 osd_op_reply(84 100013b1606.00000000 [omap-get-header,omap-get-vals,getxattr (170)] v0'0 uv40065 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 28 ==== osd_op_reply(84 100013b1606.00000000 [omap-get-header,omap-get-vals,getxattr (170)] v0'0 uv40065 ondisk = 0) v8 ==== 248+0+2386 (1036785619 0 3151859934) 0x5574ef7fd600 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.722 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6801/12021 conn(0x5574ef85c000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=8677 cs=1 l=1). rx osd.3 seq 5 0x5574ef7fcb00 osd_op_reply(81 100013ad078.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv23726 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.722 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.3 192.168.200.19:6801/12021 5 ==== osd_op_reply(81 100013ad078.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv23726 ondisk = 0) v8 ==== 248+0+11231 (1837921862 0 549299108) 0x5574ef7fcb00 con 0x5574ef85c000 -1010> 2019-11-12 09:20:17.726 7fe87e779700 5 mds.0.log _submit_thread 33239061177~772 : ESessions 14 opens cmapv 78165 -1010> 2019-11-12 09:20:17.730 7fe87e779700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:88 6.9 6:916b6fb1:::200.00001ef4:head [write 3396281~792] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef810b00 con 0 -1010> 2019-11-12 09:20:17.730 7fe87e779700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:89 6.14 6:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef810000 con 0 -1010> 2019-11-12 09:20:17.742 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 29 0x5574ef810000 osd_op_reply(88 200.00001ef4 [write 3396281~792] v28661'43317 uv43317 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.742 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 29 ==== osd_op_reply(88 200.00001ef4 [write 3396281~792] v28661'43317 uv43317 ondisk = 0) v8 ==== 156+0+0 (124758417 0 0) 0x5574ef810000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.746 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 30 0x5574ef810000 osd_op_reply(89 200.00000000 [writefull 0~90] v28661'1334402 uv1334402 ondisk = 0) v8 -1010> 2019-11-12 09:20:17.746 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 30 ==== osd_op_reply(89 200.00000000 [writefull 0~90] v28661'1334402 uv1334402 ondisk = 0) v8 ==== 156+0+0 (2410142955 0 0) 0x5574ef810000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/3593333399 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574ef567b80 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189b80 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189900 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/4201708698 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189680 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/232332927 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189400 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/1453103486 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1189180 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/522103062 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1188f00 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1188c80 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/2985278080 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f1188a00 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:0/3102681411 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744000 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:0/2796926812 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744280 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/2855972843 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744500 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/3544368504 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744780 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/979484066 -- client_snap(update split=0x3 tracelen=48) v1 -- 0x5574f0744a00 con 0 -1010> 2019-11-12 09:20:17.766 7fe87f77b700 1 mds.0.56303 rejoin_done -1010> 2019-11-12 09:20:17.774 7fe87f77b700 3 mds.0.56303 request_state up:active -1010> 2019-11-12 09:20:17.774 7fe87f77b700 5 mds.beacon.k8s-node-02 set_want_state: up:rejoin -> up:active -1010> 2019-11-12 09:20:17.774 7fe87f77b700 5 mds.beacon.k8s-node-02 Sending beacon up:active seq 7 -1010> 2019-11-12 09:20:17.774 7fe87f77b700 10 monclient: _send_mon_message to mon.k8s-node-00 at 192.168.200.15:6789/0 -1010> 2019-11-12 09:20:17.774 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6789/0 -- mdsbeacon(1849248/k8s-node-02 up:active seq 7 v56305) v7 -- 0x5574ef80ee00 con 0 -1010> 2019-11-12 09:20:17.774 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:90 6.17 6:e9c7ff81:::200.00001ec2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574ef811340 con 0 -1010> 2019-11-12 09:20:17.774 7fe87f77b700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:91 6.42 6:424fe295:::200.00001ec3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f1226000 con 0 -1010> 2019-11-12 09:20:17.794 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 18 0x5574ef811340 osd_op_reply(90 200.00001ec2 [delete] v28661'36036 uv35960 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:17.794 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 18 ==== osd_op_reply(90 200.00001ec2 [delete] v28661'36036 uv35960 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (1025874436 0 0) 0x5574ef811340 con 0x5574ef703600 -1010> 2019-11-12 09:20:17.802 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 31 0x5574f1226000 osd_op_reply(91 200.00001ec3 [delete] v28661'35871 uv35683 ondisk = -2 ((2) No such file or directory)) v8 -1010> 2019-11-12 09:20:17.802 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 31 ==== osd_op_reply(91 200.00001ec3 [delete] v28661'35871 uv35683 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (1997661384 0 0) 0x5574f1226000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:18.095 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/232332927 conn(0x5574ef85e400 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4212 cs=1 l=0). rx client.1793818 seq 2 0x5574ef7a6240 client_session(request_renewcaps seq 28283) -1010> 2019-11-12 09:20:18.095 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793818 192.168.200.30:0/232332927 2 ==== client_session(request_renewcaps seq 28283) ==== 28+0+0 (422215383 0 0) 0x5574ef7a6240 con 0x5574ef85e400 -1010> 2019-11-12 09:20:18.095 7fe885a11700 3 mds.0.server handle_client_session client_session(request_renewcaps seq 28283) from client.1793818 -1010> 2019-11-12 09:20:18.095 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/232332927 -- client_session(renewcaps seq 28283) v3 -- 0x5574ef864000 con 0 -1010> 2019-11-12 09:20:18.751 7fe883783700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6800/91 -- mgrreport(unknown.k8s-node-02 +0-0 packed 1374) v6 -- 0x5574f06ce300 con 0 -1010> 2019-11-12 09:20:18.779 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 20 0x5574f122d680 mdsmap(e 56306) v1 -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 20 ==== mdsmap(e 56306) v1 ==== 655+0+0 (3582797354 0 0) 0x5574f122d680 con 0x5574ef700c00 -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.k8s-node-02 Updating MDS map to version 56306 from mon.0 -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 handle_mds_map i am now mds.0.56303 -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 handle_mds_map state change up:rejoin --> up:active -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 recovery_done -- successful recovery! -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:92 6.33 6:cd5a64a3:::100.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f17f02c0 con 0 -1010> 2019-11-12 09:20:18.779 7fe885a11700 1 mds.0.56303 active_start -1010> 2019-11-12 09:20:18.779 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6789/0 conn(0x5574ef700c00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=15927 cs=1 l=1). rx mon.0 seq 21 0x5574ef80ee00 mdsbeacon(1849248/k8s-node-02 up:active seq 7 v56306) v7 -1010> 2019-11-12 09:20:18.779 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== mon.0 192.168.200.15:6789/0 21 ==== mdsbeacon(1849248/k8s-node-02 up:active seq 7 v56306) v7 ==== 136+0+0 (2666966931 0 0) 0x5574ef80ee00 con 0x5574ef700c00 -1010> 2019-11-12 09:20:18.779 7fe888216700 5 mds.beacon.k8s-node-02 received beacon reply up:active seq 7 rtt 1.00402 -1010> 2019-11-12 09:20:18.779 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 19 0x5574f17f02c0 osd_op_reply(92 100.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv82450 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.779 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 19 ==== osd_op_reply(92 100.00000000 [omap-get-header,omap-get-vals,getxattr (30)] v0'0 uv82450 ondisk = 0) v8 ==== 240+0+5119 (3401304057 0 3700237336) 0x5574f17f02c0 con 0x5574ef703600 -1010> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 7 0x5574ef5da480 client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12993 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 -1010> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 8 0x5574ef5dbb00 client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12994 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 -1010> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 9 0x5574ef5dbf80 client_caps(flush ino 0x100026d24a3 306632 seq 0 tid 12991 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFxcwb follows 1 size 12582912/12584448 mtime 2019-11-12 09:18:41.460885 xattrs(v=18446744071802044689 l=0)) v10 -1010> 2019-11-12 09:20:18.783 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 10 0x5574ef5dc400 client_caps(flush ino 0x100026d24a3 306632 seq 0 tid 12995 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFxcwb follows 1 size 12582912/12584448 mtime 2019-11-12 09:18:41.460885 xattrs(v=18446744071802044689 l=0)) v10 -1010> 2019-11-12 09:20:18.787 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:0/1656050499 conn(0x5574f11dd200 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=4953 cs=1 l=0). rx client.1793728 seq 11 0x5574ef5dc880 client_caps(update ino 0x100026d24a3 306632 seq 0 caps=pAsxLsXsxFsxcrwb dirty=- wanted=pAsxXsxFxcwb follows 0 size 12582912/12584448 mtime 2019-11-12 09:18:41.460885 xattrs(v=18446744071809532324 l=0)) v10 -1010> 2019-11-12 09:20:18.787 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_caps(grant ino 0x100013b15fc 1738 seq 1 caps=pAsxLsXsxFsxcrwb dirty=- wanted=pAsxXsxFxwb follows 0 size 22/4194304 ts 5974/0 mtime 2019-11-12 08:21:27.262884 tws 5970 xattrs(v=1 l=4)) v11 -- 0x5574ef5dd180 con 0 -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.787 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.787 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.791 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.791 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96231 lookup #0x100013ad07a/eula.txt 2019-11-12 08:59:14.149366 RETRY=30 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23347 lookup #0x10001346679/public 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9471 lookup #0x1000136753a/mime_types.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96232 lookup #0x100013ad07a/eula.txt 2019-11-12 09:05:30.072151 RETRY=10 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23348 lookup #0x1000134691b/autoload.php 2019-11-12 09:18:41.409643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9472 lookup #0x1000136753a/spf_dkim_whitelist.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96233 lookup #0x100013ad07a/eula.txt 2019-11-12 09:06:44.117495 RETRY=8 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794665:23349 lookup #0x10001346679/private 2019-11-12 09:18:41.457643 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9473 lookup #0x1000136753a/surbl-whitelist.inc.local 2019-11-12 09:18:41.736891 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96236 lookup #0x100013ad07a/eula.txt 2019-11-12 09:17:08.464874 RETRY=2 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1793728:9474 getattr As #0x1000136753a 2019-11-12 09:19:29.005938 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96238 lookup #0x100013ad07a/eula.txt 2019-11-12 09:18:43.942618 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.server handle_client_request client_request(client.1794521:96239 getattr As #0x100013ad07a 2019-11-12 09:19:36.755583 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe885a11700 5 mds.0.server waiting for root -1010> 2019-11-12 09:20:18.795 7fe885a11700 1 mds.0.56303 cluster recovered. -1010> 2019-11-12 09:20:18.795 7fe885a11700 4 mds.0.56303 set_osd_epoch_barrier: epoch=28661 -1010> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 7 ==== client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12993 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 ==== 236+0+0 (3677921934 0 0) 0x5574ef5da480 con 0x5574f11dd200 -1010> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_caps(flush_ack ino 0x1000136718a 2423 seq 0 tid 12993 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=- follows 1 size 0/0 mtime 0.000000) v11 -- 0x5574ef5dd600 con 0 -1010> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_caps(grant ino 0x1000136718a 2423 seq 1 caps=pAsxLsXsxFsxcrwb dirty=- wanted=pAsxXsxFsxcrwb follows 0 size 797696/4194304 ts 1/18446744073709551615 mtime 2019-11-12 09:13:00.157329 xattrs(v=1 l=4)) v11 -- 0x5574ef5dda80 con 0 -1010> 2019-11-12 09:20:18.795 7fe885a11700 1 -- 192.168.200.19:6800/1460470921 <== client.1793728 192.168.200.15:0/1656050499 8 ==== client_caps(flush ino 0x1000136718a 2423 seq 0 tid 12994 caps=pAsxLsXsxFsxcrwb dirty=Fw wanted=pAsxXsxFsxcrwb follows 1 size 797696/0 mtime 2019-11-12 09:13:00.157329 xattrs(v=18446744071802044689 l=0)) v10 ==== 236+0+0 (3677921934 0 0) 0x5574ef5dbb00 con 0x5574f11dd200 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:93 6.30 6:0d82a743:::600.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f0766000 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.purge_queue activate: start work (by drain) -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1792815:329 getattr pAsLsXsFs #0x10000025367 2019-11-12 09:19:42.723279 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:94 6.25 6:a64c6dbf:::500.0000000f:head [read 1546512~19054 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef810580 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.32:0/3593333399 -- client_reply(???:329 = 0 (0) Success) v1 -- 0x5574f0766580 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96218 lookup #0x100013ad07a/eula.txt 2019-11-12 08:28:11.070817 RETRY=67 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:95 6.57 6:ea90f4f9:::100013ad07a.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fc840 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794662:952065 readdir #0x3000001e549 2019-11-12 08:37:22.860969 RETRY=55 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:96 6.3d 6:bfdb3c52:::3000001e549.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef811600 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96220 lookup #0x100013ad07a/eula.txt 2019-11-12 08:29:41.944559 RETRY=65 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794662:952066 readdir #0x3000001e549 2019-11-12 09:10:51.957957 RETRY=6 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:22967 lookup #0x10001346b3e/public 2019-11-12 08:23:30.359288 RETRY=73 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:97 6.5 6:a110cbc0:::10001346b3e.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574f1226840 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793737:19795 setattr size=0 mtime=2019-11-12 08:23:02.984801 #0x100013b15fc 2019-11-12 08:23:02.984801 RETRY=74 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_caps(revoke ino 0x100013b15fc 1738 seq 2 caps=pAsxLsXsxFcb dirty=- wanted=pAsxXsxFxwb follows 0 size 22/4194304 ts 5974/0 mtime 2019-11-12 08:21:27.262884 tws 5970) v11 -- 0x5574f080a000 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96222 lookup #0x100013ad07a/eula.txt 2019-11-12 08:36:00.047709 RETRY=57 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:22975 lookup #0x10001344421/bookmarks 2019-11-12 08:26:09.157921 RETRY=70 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:98 6.c 6:305695a6:::10001344421.00000000:head [omap-get-header,omap-get-vals,getxattr parent] snapc 0=[] ondisk+read+known_if_redirected+full_force e28661) v8 -- 0x5574ef7fd340 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793737:19856 getattr pAs #0x100013b1606/db 2019-11-12 09:18:41.418572 RETRY=1 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.31:0/2398232289 -- client_reply(???:19856 = 0 (0) Success) v1 -- 0x5574f17f1600 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96224 lookup #0x100013ad07a/eula.txt 2019-11-12 08:37:28.357363 RETRY=55 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23185 lookup #0x10001344421/bookmarks 2019-11-12 08:48:12.519545 RETRY=41 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96226 lookup #0x100013ad07a/eula.txt 2019-11-12 08:44:11.084864 RETRY=47 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23218 open #0x10001346b3e/.htaccess 2019-11-12 08:57:40.980883 RETRY=35 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96227 lookup #0x100013ad07a/eula.txt 2019-11-12 08:45:24.114219 RETRY=45 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23345 getattr As #0x1000133e06d 2019-11-12 09:18:35.453545 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_reply(???:23345 = 0 (0) Success) v1 -- 0x5574f17f1080 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794521:96230 lookup #0x100013ad07a/eula.txt 2019-11-12 08:57:57.351991 RETRY=34 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1794665:23346 lookup #0x1000133e06d/index.php 2019-11-12 09:18:40.585629 RETRY=1 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.30:0/3506497893 -- client_reply(???:23346 = 0 (0) Success) v1 -- 0x5574f17f0580 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793728:9470 lookup #0x1000136753a/2tld.inc.local 2019-11-12 09:18:41.416884 RETRY=1 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:0/1656050499 -- client_reply(???:9470 = -2 (2) No such file or directory) v1 -- 0x5574f1227340 con 0 -1010> 2019-11-12 09:20:18.795 7fe880f7e700 4 mds.0.server handle_client_request client_request(client.1793815:22873 rename #0x30000015ba9/20160904_113956.jpg #0x30000015ba9/.20160904_113956.jpg.NIh764 2019-11-10 15:46:08.760660 RETRY=113 caller_uid=0, caller_gid=0{}) v2 -1010> 2019-11-12 09:20:18.795 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 36 0x5574ef7fd340 osd_op_reply(96 3000001e549.00000000 [omap-get-header,omap-get-vals,getxattr (97)] v0'0 uv88617 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.795 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 36 ==== osd_op_reply(96 3000001e549.00000000 [omap-get-header,omap-get-vals,getxattr (97)] v0'0 uv88617 ondisk = 0) v8 ==== 248+0+2829 (838332988 0 2622649345) 0x5574ef7fd340 con 0x5574ef703000 -1010> 2019-11-12 09:20:18.799 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 37 0x5574f17f0b00 osd_op_reply(98 10001344421.00000000 [omap-get-header,omap-get-vals,getxattr (125)] v0'0 uv36621 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.799 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 37 ==== osd_op_reply(98 10001344421.00000000 [omap-get-header,omap-get-vals,getxattr (125)] v0'0 uv36621 ondisk = 0) v8 ==== 248+0+19893 (2739227853 0 1215621712) 0x5574f17f0b00 con 0x5574ef703000 -1010> 2019-11-12 09:20:18.799 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 4 0x5574ef5dd180 client_caps(update ino 0x100013b15fc 1738 seq 2 caps=pAsxLsXsxFcb dirty=- wanted=pAsxXsxFxwb follows 0 size 22/0 mtime 2019-11-12 08:21:27.262884 tws 5970) v6 -1010> 2019-11-12 09:20:18.799 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.31:0/2398232289 conn(0x5574ef85ea00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5362 cs=1 l=0). rx client.1793737 seq 5 0x5574f11fbb00 client_request(client.1793737:19857 readdir #0x100013b1607 2019-11-12 09:20:18.796352 caller_uid=1000, caller_gid=1000{}) v2 -1010> 2019-11-12 09:20:18.799 7fe888a17700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.15:6805/24523 conn(0x5574ef702a00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=11977 cs=1 l=1). rx osd.1 seq 32 0x5574f17f0000 osd_op_reply(95 100013ad07a.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv30079 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.799 7fe888a17700 1 -- 192.168.200.19:6800/1460470921 <== osd.1 192.168.200.15:6805/24523 32 ==== osd_op_reply(95 100013ad07a.00000000 [omap-get-header,omap-get-vals,getxattr (95)] v0'0 uv30079 ondisk = 0) v8 ==== 248+0+7185 (2976393307 0 498225518) 0x5574f17f0000 con 0x5574ef702a00 -1010> 2019-11-12 09:20:18.799 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 38 0x5574ef7fd340 osd_op_reply(97 10001346b3e.00000000 [omap-get-header,omap-get-vals,getxattr (124)] v0'0 uv36678 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.799 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 38 ==== osd_op_reply(97 10001346b3e.00000000 [omap-get-header,omap-get-vals,getxattr (124)] v0'0 uv36678 ondisk = 0) v8 ==== 248+0+1853 (4149073175 0 3508024585) 0x5574ef7fd340 con 0x5574ef703000 -1010> 2019-11-12 09:20:18.799 7fe888216700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.19:6805/12258 conn(0x5574ef703000 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=12849 cs=1 l=1). rx osd.5 seq 39 0x5574f17f0b00 osd_op_reply(93 600.00000000 [omap-get-header,omap-get-vals,getxattr (62)] v0'0 uv93665 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.799 7fe888216700 1 -- 192.168.200.19:6800/1460470921 <== osd.5 192.168.200.19:6805/12258 39 ==== osd_op_reply(93 600.00000000 [omap-get-header,omap-get-vals,getxattr (62)] v0'0 uv93665 ondisk = 0) v8 ==== 240+0+83373 (2574217908 0 294505411) 0x5574f17f0b00 con 0x5574ef703000 -1010> 2019-11-12 09:20:18.803 7fe880f7e700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: In function 'void Server::_rename_prepare(MDRequestRef&, EMetaBlob*, ceph::bufferlist*, CDentry*, CDentry*, CDentry*)' thread 7fe880f7e700 time 2019-11-12 09:20:18.803701 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.6/rpm/el7/BUILD/ceph-13.2.6/src/mds/Server.cc: 8036: FAILED assert(oldin->first <= straydn->first) ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x7fe88f05b36b] 2: (()+0x26e4f7) [0x7fe88f05b4f7] 3: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 4: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 5: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 6: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 7: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 8: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 9: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 10: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 11: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 12: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 13: (()+0x7e65) [0x7fe88cc8ee65] 14: (clone()+0x6d) [0x7fe88bd6988d] -1010> 2019-11-12 09:20:18.803 7fe880f7e700 -1 *** Caught signal (Aborted) ** in thread 7fe880f7e700 thread_name:mds_rank_progr ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable) 1: (()+0xf5f0) [0x7fe88cc965f0] 2: (gsignal()+0x37) [0x7fe88bca1337] 3: (abort()+0x148) [0x7fe88bca2a28] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x248) [0x7fe88f05b468] 5: (()+0x26e4f7) [0x7fe88f05b4f7] 6: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::list*, CDentry*, CDentry*, CDentry*)+0x276d) [0x5574ed818c0d] 7: (Server::handle_client_rename(boost::intrusive_ptr&)+0x19bb) [0x5574ed81a6cb] 8: (Server::handle_client_request(MClientRequest*)+0x49e) [0x5574ed82dc2e] 9: (Server::dispatch(Message*)+0x2db) [0x5574ed83192b] 10: (MDSRank::handle_deferrable_message(Message*)+0x434) [0x5574ed7a4234] 11: (MDSRank::_dispatch(Message*, bool)+0x88b) [0x5574ed7baffb] 12: (MDSRank::retry_dispatch(Message*)+0x12) [0x5574ed7bb6e2] 13: (MDSInternalContextBase::complete(int)+0x67) [0x5574eda144a7] 14: (MDSRank::_advance_queues()+0xd1) [0x5574ed7b9fa1] 15: (MDSRank::ProgressThread::entry()+0x43) [0x5574ed7ba633] 16: (()+0x7e65) [0x7fe88cc8ee65] 17: (clone()+0x6d) [0x7fe88bd6988d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. -1010> 2019-11-12 09:20:18.807 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 11 0x5574f06ce300 client_request(client.1794665:23350 lookup #0x1000133e06d/remote.php 2019-11-12 09:20:18.807234 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.807 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.30:0/3506497893 conn(0x5574f11dcc00 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=5028 cs=1 l=0). rx client.1794665 seq 12 0x5574f1209b00 client_request(client.1794665:23351 open #0x1000133e06d/.htaccess 2019-11-12 09:20:18.807234 caller_uid=33, caller_gid=33{}) v2 -1010> 2019-11-12 09:20:18.823 7fe887a15700 5 -- 192.168.200.19:6800/1460470921 >> 192.168.200.17:6806/20250 conn(0x5574ef703600 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=16763 cs=1 l=1). rx osd.4 seq 20 0x5574f17f0580 osd_op_reply(94 500.0000000f [read 1546512~19054 [fadvise_dontneed]] v0'0 uv76310 ondisk = 0) v8 -1010> 2019-11-12 09:20:18.823 7fe887a15700 1 -- 192.168.200.19:6800/1460470921 <== osd.4 192.168.200.17:6806/20250 20 ==== osd_op_reply(94 500.0000000f [read 1546512~19054 [fadvise_dontneed]] v0'0 uv76310 ondisk = 0) v8 ==== 156+0+19054 (1887848830 0 2063325954) 0x5574f17f0580 con 0x5574ef703600 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:99 2.27 2:e47eb51a:::100026d4cfc.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f082f600 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:100 2.33 2:ce5abf8f:fsvolumens_kubernetes-dynamic-pvc-54a06738-0c74-11e9-9748-6e9a8c583412::100026d248f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f082f8c0 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:101 2.33 2:ce877a67:fsvolumens_kubernetes-dynamic-pvc-54a06738-0c74-11e9-9748-6e9a8c583412::100026d248f.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f082fb80 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:102 2.0 2:03503df5:::100026d248f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0854000 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:103 2.26 2:65674777:::100026d4d07.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f08542c0 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:104 2.1c 2:3897a2ea:::100026d4d00.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0854580 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:105 2.c 2:30e0a575:fsvolumens_kubernetes-dynamic-pvc-54a06738-0c74-11e9-9748-6e9a8c583412::100026d2490.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0854840 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:106 2.32 2:4ef45668:::100026d2490.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0854b00 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.19:6805/12258 -- osd_op(unknown.0.56303:107 2.21 2:86a3858f:::100026d4d11.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0854dc0 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:108 2.3a 2:5d66a153:fsvolumens_kubernetes-dynamic-pvc-54a06738-0c74-11e9-9748-6e9a8c583412::100026d248c.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0855080 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:109 2.26 2:655bd393:::100026d248c.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0855340 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:110 2.2e 2:741866b6:::100026d4d04.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0855600 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:111 2.16 2:686929be:fsvolumens_kubernetes-dynamic-pvc-54a06738-0c74-11e9-9748-6e9a8c583412::100026d2491.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f08558c0 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:112 2.14 2:2b55b030:::100026d2491.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f0855b80 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:113 2.18 2:18f38e99:::100026d4ce7.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f085e000 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:114 2.29 2:94b42518:::100026d4cee.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f085e2c0 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.17:6806/20250 -- osd_op(unknown.0.56303:115 2.9 2:91599790:fsvolumens_kubernetes-dynamic-pvc-54a06738-0c74-11e9-9748-6e9a8c583412::100026d2486.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f085e580 con 0 -1010> 2019-11-12 09:20:18.823 7fe88077d700 1 -- 192.168.200.19:6800/1460470921 --> 192.168.200.15:6805/24523 -- osd_op(unknown.0.56303:116 2.17 2:ea15fcba:::100026d2486.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e28661) v8 -- 0x5574f085e840 con 0 --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 rgw_sync 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) 99/99 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/lib/ceph/crash/2019-11-12_09:20:18.807853Z_6f30738c-6eb6-4a44-8a05-dec5f867fb8b/log --- end dump of recent events --- start_mds.sh: line 4: 16285 Aborted /usr/bin/ceph-mds "${DAEMON_OPTS[@]}" -i "${MDS_NAME}"