本文转载自一位专家朋友Bean Li的博客,原文链接见文末。
前言
本文记录multipath相关指令, Multipath的path path\_grouping\_policy – Paths are grouped into path groups. The policy determines how path groups are formed. There are five different policies.
- failover: One path per priority group
- multibus: All paths in one priority group. This is the default.
- group\_by\_serial: One priority group per storage controller (serial number)
- group\_by\_prio: One priority group per priority value
- group\_by\_node\_name: One priority group per target node name
有时候使用failover,需要切换主路径,本文记录切换主路径的方法
查看当前配置
root@host243:~# multipath -ll2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scalsize=5.0T features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=active| `- 49:0:0:0 sdg 8:96 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 48:0:0:0 sdf 8:80 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 46:0:0:0 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 47:0:0:0 sde 8:64 active ready running
灰色部分按住可左右拖动,以下同
切换主路径
========
从上面的输出可以看出,sdd/sde/sdf/sdg 都是同一个设备的4条不同的路径。如果设备是分布式存储提供的iSCSI设备,每条路径可能对应一个Gateway。有时候可能需要切换主路径,比如某个存储节点要关机维护,这种情况,要怎么做呢?
首先执行如下语句,进入交互页面:
multipathd -k
通过如下指令可以实时查看某个multipath的多个group信息:
multipathd> show topology2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scalsize=5.0T features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=active| `- 49:0:0:0 sdg 8:96 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 48:0:0:0 sdf 8:80 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 46:0:0:0 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 47:0:0:0 sde 8:64 active ready runningmultipathd>
对于我这个环境比较简单,如果存在多个,可以通过如下指令指定:
multipathd> show map 2c2c5d201aba719ee topology2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scalsize=5.0T features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=active| `- 49:0:0:0 sdg 8:96 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 48:0:0:0 sdf 8:80 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 46:0:0:0 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 47:0:0:0 sde 8:64 active ready runningmultipathd>
从上面的输出可以看出,我们设备下有多条路径,第一条sdg 为group 1,第二个sdf为group 2, 如果想切换主路径到sdd,那么需要执行:
执行完毕后,可以执行quit 退出multipathd的交互,然后执行multipath -ll来确认主路径是否发生了变化。
有些时候,没有I/O流量的情况下,multipath -ll的输出并不会及时发生变化,不过没关系,引入一些IO,就可以看到active 路径发生了变化。
oot@host243:~# multipath -ll2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scalsize=5.0T features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=1 status=enabled| `- 49:0:0:0 sdg 8:96 active ready running|-+- policy='round-robin 0' prio=1 status=enabled| `- 48:0:0:0 sdf 8:80 active ready running|-+- policy='round-robin 0' prio=1 status=active| `- 46:0:0:0 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=1 status=enabled `- 47:0:0:0 sde 8:64 active ready running
验证
对于分布式存储的iSCSI,每一条路径对应某一个存储Gateway,如何判断sdX和存储Gateway的映射关系呢?
root@host243:~# iscsiadm -m session -P 3iSCSI Transport Class version 2.0-870version 2.0-873Target: iqn.2020-01.lenovo:test Current Portal: 10.16.172.105:3260,1 Persistent Portal: 10.16.172.105:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498 Iface IPaddress: 10.16.172.243 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 16 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ********* Timeouts: ********* Recovery Timeout: 120 Target Reset Timeout: 30 LUN Reset Timeout: 30 Abort Timeout: 15 ***** CHAP: ***** username: <empty> password: ******** username_in: <empty> password_in: ******** ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 1048576 FirstBurstLength: 262144 MaxBurstLength: 1048576 ImmediateData: Yes InitialR2T: No MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 46 State: running scsi46 Channel 00 Id 0 Lun: 0 Attached scsi disk sdd State: running Current Portal: 10.16.172.106:3260,1 Persistent Portal: 10.16.172.106:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498 Iface IPaddress: 10.16.172.243 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 17 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ********* Timeouts: ********* Recovery Timeout: 120 Target Reset Timeout: 30 LUN Reset Timeout: 30 Abort Timeout: 15 ***** CHAP: ***** username: <empty> password: ******** username_in: <empty> password_in: ******** ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 1048576 FirstBurstLength: 262144 MaxBurstLength: 1048576 ImmediateData: Yes InitialR2T: No MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 47 State: running scsi47 Channel 00 Id 0 Lun: 0 Attached scsi disk sde State: running Current Portal: 10.16.172.107:3260,1 Persistent Portal: 10.16.172.107:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498 Iface IPaddress: 10.16.172.243 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 18 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ********* Timeouts: ********* Recovery Timeout: 120 Target Reset Timeout: 30 LUN Reset Timeout: 30 Abort Timeout: 15 ***** CHAP: ***** username: <empty> password: ******** username_in: <empty> password_in: ******** ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 1048576 FirstBurstLength: 262144 MaxBurstLength: 1048576 ImmediateData: Yes InitialR2T: No MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 48 State: running scsi48 Channel 00 Id 0 Lun: 0 Attached scsi disk sdf State: running Current Portal: 10.16.172.108:3260,1 Persistent Portal: 10.16.172.108:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498 Iface IPaddress: 10.16.172.243 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 19 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ********* Timeouts: ********* Recovery Timeout: 120 Target Reset Timeout: 30 LUN Reset Timeout: 30 Abort Timeout: 15 ***** CHAP: ***** username: <empty> password: ******** username_in: <empty> password_in: ******** ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 1048576 FirstBurstLength: 262144 MaxBurstLength: 1048576 ImmediateData: Yes InitialR2T: No MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 49 State: running scsi49 Channel 00 Id 0 Lun: 0 Attached scsi disk sdg State: runningroot@host243:~#
通过上面的输出,可以看到 /dev/sdd对应的是10.16.172.105:3260 ,因此找到了对应关系。
修改完毕了主路径,如何从网路上判定已经发生了切换呢?
iftop -i bond0
通过iftop,查看网络流向,确定是否流向了正确的Gateway。
点开后双击放大显示
原文链接:https://bean-li.github.io/mul...
推荐阅读
本文转载自企业存储技术微信公众号原文链接点这里注:本文只代表作者个人观点,与任何组织机构无关,如有错误和不足之处欢迎在留言中批评指正。 进一步交流技术可以加我的微信/QQ:490834312。
尊重知识,转载时请保留全文,并包括本行及如下二维码。感谢您的阅读和支持!《企业存储技术》微信公众号:HL_Storage,也欢迎关注企业存储技术极术专栏,定期更新。