转载

JBOD磁盘配置和StorCLI命令使用小结

前言

已经实施过2次Hadoop大数据平台的基础环境搭建，对于硬盘的RAID模式都不约而同选择了JBOD，毕竟大数据对于文件系统的管理本身就是基于分布式，由至少3份副本来保障数据可靠性，只是不做RAID后的SAS硬盘读写性能就有点惨不忍睹。这次Hadoop大数据平台工具选择的是Cloudera，大家感兴趣的化也可以浏览下它的官网。本篇文章主要介绍如何实现在不停机状态下在线更换JBOD硬盘。

利用StorCLI命令不停机在线更换JBOD硬盘

更新历史

2016年12月31日 - 初稿

阅读原文 - https://wsgzao.github.io/post/jbod/

扩展阅读

Cloudera - http://www.cloudera.com/

JBOD原理

JBOD （ Just a Bunch Of Disks ）不是标准的 RAID 等级，它通常用来表示一个没有控制软件提供协调控制的磁盘集合。 JBOD 将多个物理磁盘串联起来，提供一个巨大的逻辑磁盘。 JBOD 的数据存放机制是由第一块磁盘开始按顺序往后存储，当前磁盘存储空间用完后，再依次往后面的磁盘存储数据。 JBOD 存储性能完全等同于单块磁盘，而且也不提供数据安全保护。它只是简单提供一种扩展存储空间的机制， JBOD 可用存储容量等于所有成员磁盘的存储空间之和。目前 JBOD 常指磁盘柜，而不论其是否提供 RAID 功能。

RAID磁盘阵列配置和调优小结 - https://wsgzao.github.io/post/raid/

参考文档和工具

StorCLI_RefMan_revf

https://www.broadcom.com/support/download-search/?dk=StorCLI

StorCLI (Command Line) Utility for Storage Management v1.07.07 on Linux

https://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5092950

使用方式

#查询硬件RAID卡信息
[root@localhost Linux]# cat /proc/scsi/scsi |more

Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: IBM      Model: ServeRAID M5210  Rev: 4.62
  Type:   Direct-Access                    ANSI  SCSI revision: 05

#解压安装storcli
unzip ibm_utl_sraidmr_storcli-1.07.07_linux_32-64.zip
rpm -Uvh storcli-1.07.07-1.noarch.rpm

#设置软连接
ln -s /opt/MegaRAID/storcli/storcli64 /bin/storcli
ln -s /opt/MegaRAID/storcli/storcli64 /sbin/storcli

#Raid数量及当前状态
[root@localhost Linux]# storcli show ctrlcount
Status Code = 0
Status = Success
Description = None

Controller Count = 2

#查询虚拟磁盘大小，raid结构等
[root@localhost Linux]# storcli /c0 /v0 show
Controller = 0
Status = Success
Description = None


Virtual Drives :
==============

-----------------------------------------------------------
DG/VD TYPE  State Access Consist Cache sCC       Size Name 
-----------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     RWTD  -   278.465 GB      
-----------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency
#可以看出此为RAID1，一个虚拟磁盘，大小为278.465 GB 

#查看control0的物理硬盘
[root@localhost Linux]# storcli /c0 /eall /sall show
Controller = 0
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

-------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model            Sp 
-------------------------------------------------------------------------
252:16    8 Onln   0 278.465 GB SAS  HDD N   Y  512B ST300MP0035    X U  
252:17    9 Onln   0 278.465 GB SAS  HDD N   Y  512B ST300MP0035    X U  
-------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

#可以看到总计有两颗硬盘，每一颗大小278.465GB，是否在线等

#查看control1的物理硬盘
[root@localhost Linux]# storcli /c1 /eall /sall show
Controller = 1
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model            Sp 
------------------------------------------------------------------------
8:0      23 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:1      24 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:2      22 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:3      19 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:4      13 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:5      14 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:6      18 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:7      12 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:8      10 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:9      16 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:10     11 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:11     21 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:12     15 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:13      9 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

#可以看到总计有两颗硬盘，每一颗大小1.0810 TB，是否在线等


#设置硬盘状态
storcli /c1/e8/s0 show all

storcli /c1/e8/s0 start locate
storcli /c1/e8/s0 set offline
storcli /c1/e8/s0 set online
storcli /c1/e8/s0 set jbod

StorCLI Help

#查看StorCLI帮助信息
[root@localhost Linux]# storcli -h
     Storage Command Line Tool  Ver 1.07.07 Nov 15, 2013

     (c)Copyright 2013, LSI Corporation, All Rights Reserved.


storcli -v 
storcli -h| -help| ? 
storcli -h| -help| ? legacy
storcli show 
storcli show all
storcli show ctrlcount
storcli /cx add vd r[0|1|5|6|00|10|50|60]
    [Size=<VD1_Sz>,<VD2_Sz>,..|all] [name=<VDNAME1>,..] 
    drives=e:s|e:s-x|e:s-x,y,e:s-x,y,z [PDperArray=x][SED]
    [pdcache=on|off|default][pi][DimmerSwitch(ds)=default|automatic(auto)|
    none|maximum(max)|MaximumWithoutCaching(maxnocache)]
    [wt|wb|awb][nora|ra] [direct|cached] [cachevd]
    [Strip=<8|16|32|64|128|256|512|1024>] [AfterVd=X] 
    [Spares = [e:]s|[e:]s-x|[e:]s-x,y] [force][ExclusiveAccess]
storcli /cx add vd each r0 [name=<VDNAME1>,..] [drives=e:s|e:s-x|e:s-x,y]
    [SED] [pdcache=on|off|default][pi] [DimmerSwitch(ds)=default|automatic(auto)|
    none|maximum(max)|MaximumWithoutCaching(maxnocache)] 
    [wt|wb|awb] [nora|ra] [direct|cached] 
    [Strip=<8|16|32|64|128|256|512|1024>] [ExclusiveAccess]
storcli /cx add VD cachecade|nytrocache r[aid][0,1,10, 1EC] 
    drives = [e:]s|[e:]s-x|[e:]s-x,y [WT| WB] [assignvds = 0,1,2] [BOOTVOLSIZE=x]
storcli /cx/ex show all 
storcli /cx/ex show status 
storcli /cx/vx del [cachecade] [force]
storcli /cx delete events
storcli /cx show events [[type= <sincereboot| sinceshutdown| includedeleted|
latest=x| ccincon vd=<0,1,...>] filter=<info|warning|critical|fatal>]
file=<filepath>
storcli /cx show eventloginfo
storcli /cx show health [all]
storcli /cx delete securitykey
storcli /cx set securitykey=xxxxxxxx {passphrase=xxxx} {keyid=xxx}
storcli /cx start Diag Duration=<Val> 
storcli /cx set securitykey keyid=xxx
storcli /cx compare securitykey=xxxxxxxxxx
storcli /cx set termlog[=on|off|offthisboot]
storcli /cx show termlog [type=config|contents]
storcli /cx delete termlog
storcli /cx Compare factorysettings file=<filepath>
storcli /cx Compare factorysettings raw file=<filepath>
storcli /cx set securitykey=xxxxxxxx oldsecuritykey=xxxxxxxx
    {passphrase=xxxx} {keyid=xxx}
storcli /cx/dx show 
storcli /cx/dall show cachecade 
storcli /cx/dx show all 
storcli /cx show freespace
storcli /cx/fall show [all] [securityKey = xxx]
storcli /cx/fall del|delete [securityKey = xxx]
storcli /cx/fall import [preview] [securityKey = xxx]
storcli /cx/vx set ssdcaching=on|off
storcli /cx/dx set security=on
storcli /cx/vx show expansion
storcli /cx show fshinting
storcli /cx/vx set fshinting=<value>
storcli /cx/vx expand Size=<xx> [expandarray]
storcli /cx[/ex]/sx show 
storcli /cx[/ex]/sx show all
storcli /cx[/ex]/sx start rebuild 
storcli /cx[/ex]/sx stop rebuild 
storcli /cx[/ex]/sx pause rebuild 
storcli /cx[/ex]/sx resume rebuild 
storcli /cx[/ex]/sx show rebuild 
storcli /cx[/ex]/sx show health 
storcli /cx[/ex]/sx start copyback target=e:s 
storcli /cx[/ex]/sx stop copyback 
storcli /cx[/ex]/sx pause copyback 
storcli /cx[/ex]/sx resume copyback 
storcli /cx[/ex]/sx show copyback 
storcli /cx[/ex]/sx start initialization  
storcli /cx[/ex]/sx stop initialization  
storcli /cx[/ex]/sx show initialization  
storcli /cx[/ex]/sx start locate  
storcli /cx[/ex]/sx stop locate  
storcli /cx[/ex]/sx show securitykey keyid 
storcli /cx[/ex]/sx add hotsparedrive [DGs=<N|0,1,2...>] [enclaffinity]
    [nonrevertible]
storcli /cx[/ex]/sx delete hotsparedrive
storcli /cx[/ex]/sx spinup
storcli /cx[/ex]/sx spindown
storcli /cx[/ex]/sx set online 
storcli /cx[/ex]/sx set offline 
storcli /cx[/ex]/sx set missing 
storcli /cx[/ex]/sx set jbod 
storcli /cx[/ex]/sx set good [force]  
storcli /cx[/ex]/sx insert dg=A array=B row=C 
storcli /cx/vx set emulationType=0|1
storcli /cx/vx set wrcache=WT|WB|AWB
storcli /cx/vx set rdcache=RA|NoRA
storcli /cx/vx set iopolicy=Cached|Direct
storcli /cx/vx set accesspolicy=RW|RO|Blocked|RmvBlkd
storcli /cx/vx set pdcache=On|Off|Default
storcli /cx/vx set name=<NameString>
storcli /cx/vx set HostAccess=ExclusiveAccess|SharedAccess
storcli /cx/vx set ds=Default|Auto|None|Max|MaxNoCache
storcli /cx/vx set autobgi=On|Off
storcli /cx/vx set pi=Off
storcli /cx/vx show
storcli /cx/vx show all
storcli /cx/vx show init
storcli /cx/vx show cc
storcli /cx/vx show erase
storcli /cx/vx show migrate
storcli /cx/vx show bgi
storcli /cx/vx show autobgi
storcli /cx/vx show trim
storcli /cx set consistencycheck|cc=[off|seq|conc] [delay=value]
    starttime=yyyy/mm/dd hh] [excludevd=x-y,z]
storcli /cx show cc|consistencycheck
storcli /cx show ocr 
storcli /cx set ocr=<on|off>
storcli /cx/vx start init[Full][Force]
storcli /cx/vx start erase [simple|normal|thorough|standard][patternA=<val>]
[patternB=<val>]
storcli /cx/vx start cc[Force]
storcli /cx/vx start migrate type=raidx [option=add|remove 
    drives=[e:]s|[e:]s-x|[e:]s-x,y] [Force] 
storcli /cx/vx stop init
storcli /cx/vx stop erase
storcli /cx/vx stop cc
storcli /cx/vx stop bgi
storcli /cx/vx pause cc
storcli /cx/vx pause bgi
storcli /cx/vx resume cc
storcli /cx/vx resume bgi
storcli /cx show 
storcli /cx show all 
storcli /cx show preservedcache 
storcli /cx/vx delete preservedcache[force] 
storcli /cx[/ex]/sx download src=<filepath> [satabridge] [mode= 5|7]
storcli /cx/ex download src=<filepath> [forceActivate] 
storcli /cx[/ex]/sx secureerase [force]  
storcli /cx[/ex]/sx start erase [simple| normal| thorough | standard| threepass]
   [patternA=<val>] [patternB=<val>]
storcli /cx[/ex]/sx stop erase 
storcli /cx[/ex]/sx show erase 
storcli /cx[/ex]/sx show rawdata pageaddr=<pageaddress in hex> file=<filename> 
storcli /cx[/ex]/sx set bootdrive=<on|off> 
storcli /cx/vx set bootdrive=<on|off>
storcli /cx show bootdrive
storcli /cx show bootwithpinnedcache
storcli /cx set bootwithpinnedcache=<on|off>
storcli /cx show activityforlocate
storcli /cx set activityforlocate=<on|off>
storcli /cx show copyback
storcli /cx set copyback=<on|off> type=smartssd|smarthdd|all 
storcli /cx show jbod
storcli /cx set jbod=<on|off> 
storcli /cx set autorebuild=<on|off> 
storcli /cx show autorebuild
storcli /cx show cachebypass
storcli /cx set cachebypass=<on|off> 
storcli /cx show usefdeonlyencrypt
storcli /cx set usefdeonlyencrypt=<on|off>
storcli /cx show prcorrectunconfiguredareas
storcli /cx set prcorrectunconfiguredareas=<on|off> 
storcli /cx show batterywarning
storcli /cx set batterywarning=<on|off> 
storcli /cx show abortcconerror
storcli /cx set abortcconerror=<on|off> 
storcli /cx show ncq
storcli /cx set ncq=<on|off> 
storcli /cx show maintainpdfailhistory
storcli /cx set maintainpdfailhistory=<on|off> 
storcli /cx show restorehotspare
storcli /cx set restorehotspare=<on|off> 
storcli /cx set bios [state=<on|off>] [Mode=<SOE|BE|HCOE|HSM>] [abs=<on|off>]
storcli /cx show bios 
storcli /cx show alarm
storcli /cx set alarm=<on|off|silence> 
storcli /cx show foreignautoimport
storcli /cx set foreignautoimport=<on|off> 
storcli /cx show directpdmapping
storcli /cx set directpdmapping=<on|off> 
storcli /cx show rebuildrate
storcli /cx set rebuildrate=<value> 
storcli /cx show loadbalancemode
storcli /cx set loadbalancemode=<on|off> 
storcli /cx show eghs
storcli /cx set eghs [state=<on|off>] [eug=<on|off>] [smarter=<on|off>] 
storcli /cx show cacheflushint
storcli /cx set cacheflushint=<value> 
storcli /cx show prrate
storcli /cx set prrate=<value> 
storcli /cx show ccrate
storcli /cx set ccrate=<value> 
storcli /cx show bgirate
storcli /cx set bgirate =<value> 
storcli /cx show dpm 
storcli /cx set dpm =<on|off> 
storcli /cx show sgpioforce 
storcli /cx set sgpioforce =<on|off> 
storcli /cx set supportssdpatrolread =<on|off> 
storcli /cx show migraterate
storcli /cx set migraterate=<value> 
storcli /cx show spinupdrivecount
storcli /cx show wbsupport
storcli /cx set spinupdrivecount=<value> 
storcli /cx show spinupdelay
storcli /cx set spinupdelay=<value> 
storcli /cx show coercion
storcli /cx set coercion=<value> 
storcli /cx show smartpollinterval
storcli /cx set smartpollinterval=<value> 
storcli /cx show eccbucketsize
storcli /cx set eccbucketsize=<value> 
storcli /cx show eccbucketleakrate
storcli /cx set eccbucketleakrate=<value> 
storcli /cx show backplane
storcli /cx set backplane mode=<value> expose=<on/off> 
storcli /cx show perfmode
storcli /cx set perfmode=<value> [maxflushlines=<value> numiostoorder=<value>]
storcli /cx show perfmodevalues
storcli /cx show pi
storcli /cx set pi [state=<on|off>] [import=<on|off>]
storcli /cx show time
storcli /cx set time=<yyyymmdd hh:mm:ss | systemtime> 
storcli /cx show ds
storcli /cx set ds=OFF type=1|2|3|4
storcli /cx set ds=ON type=1|2 [properties]
storcli /cx set ds=ON type=3|4 DefaultLdType=<val> [properties]
storcli /cx set ds [properties] 
storcli /cx show safeid
storcli /cx show rehostinfo
storcli /cx show pci
storcli /cx show ASO
storcli /cx set aso key=<key value> preview
storcli /cx set aso key=<key value>
storcli /cx set aso transfertovault
storcli /cx set aso rehostcomplete
storcli /cx set aso deactivatetrialkey
storcli /cx set factory defaults
storcli /cx download file=<filepath> [fwtype=<val>] [ResetNow] [nosigchk] 
    [noverchk]
storcli /cx flush|flushcache
storcli /cx/px show 
storcli /cx/px show all
storcli /cx/px set linkspeed=0|1.5|3|6|12 
storcli /cx/bbu show 
storcli /cx/bbu set SCAPVPD file=<input file path> VPDPage=<SCapVPDFixed>
storcli /cx/bbu show all 
storcli /cx/bbu show status 
storcli /cx/bbu show properties 
storcli /cx/bbu show learn 
storcli /cx/bbu show gasgauge Offset=xxxx Numbytes=n 
storcli /cx/bbu start learn 
storcli /cx/bbu show modes 
storcli /cx/bbu set [ learnDelayInterval=<val> | bbuMode=<val>  
    |learnStartTime=[DDD HH | off] | autolearnmode=<val> | powermode=sleep | writeaccess=sealed ]
storcli /cx/cv show 
storcli /cx/cv show all 
storcli /cx/cv show status 
storcli /cx/cv show learn 
storcli /cx/cv start learn 
storcli /cx show securitykey keyid 
storcli /cx start patrolread
storcli /cx stop patrolread
storcli /cx pause patrolread
storcli /cx resume patrolread
storcli /cx show patrolRead
storcli /cx show powermonitoringinfo
storcli /cx set patrolread = {{on mode=<auto|manual> }|{off}}
storcli /cx set patrolread [starttime=< yyyy/mm/dd hh>] 
    [maxconcurrentpd =<value>] [includessds=<on|onlymixed|off>]
    [uncfgareas=on|off]
storcli /cx set patrolread delay = <value>
storcli /cx[/ex]/sx show diag paniclog [Query] | [ExtractSlot=x] 
                                         | [EraseSlot=x] [file=filepath]
storcli /cx[/ex]/sx show diag smartlog [file=filepath] 
storcli /cx[/ex]/sx show diag errorlog [file=filepath] 
storcli /cx del Nytrocache [force]
storcli /cx show badblocks
storcli /cx flasherase 
storcli /cx shutdown 
storcli /cx/mx set mode=<Internal | External | Auto> 
storcli /cx/mx show 
storcli /cx transform iMR 
storcli /cx restart 
storcli /cx/vx show BBMT
storcli /cx/vx delete BBMT
storcli /xd add type=CacheDev 
                <devices=comma separated cache devices  | ALL> [force] 
storcli /xd add type=VirtualDrive 
                <devices=comma separated Virtual drives | ALL> 
storcli /xd delete type=<CacheDev | VirtualDrive> 
                        <devices=comma separated cache devices or 
                        Virtual drives as applicable | ALL> 
storcli /xd show type=<CacheDev | VirtualDrive> [state=<assigned|unassigned>] 
storcli /xd show 
storcli /xd show perfmon 
storcli /xd/wd<X|ALL> show 
storcli /xd/wd<X|ALL> show safeid 
storcli /xd set online [Vds=comma separated VD ids] 
storcli /xd/wdX apply key=value 
storcli /cx[/ex]/sx start format [thorough] 
storcli /cx show dequeuelog file=<filepath>


Note:Use 'page=[x]'as the last option in all the commands to set the page break.
      X=lines per page. E.g. 'storcli help page=10'
原文  https://wsgzao.github.io/post/jbod/
正文到此结束
所属分类： 编程技术

热门推荐

相关文章

阿里云首购8折
说给你听

本文目录
随机标签
书籍教程
springboot-demo Java入门教程 bootstrap3 CSS Apache基础教程 php ionic 教程 Python mysql教程 eclipse Ubuntu VPS系统配置 AngularJS 教程 MongoDB教程 Struts2教程 Redis教程 Spring教程 Git教程 Jenkins进阶系列 openfire参考指南 Java设计模式 HBase教程 Maven教程 hibernate教程 Docker 教程 memcached教程 Quartz指南 Hive教程 Ant教程 ANTLR教程 SpringCloud java实例教程 springcloud-demo XStream教程 Hazelcast教程 Elastic-Job-Lite 深入浅出MyBatis SVN教程 ibaties教程 rabittmq教程 solr教程 Hadoop教程 WebService CXF学习 JPA教程 ActiveMQ中文指南 java-demo Java内存模型 dubbo教程 python3-demo Linux入门视频教程
近期评论
  1. 1 Spring Cloud Consul实现选举机制
  2. 2 Spring Boot集成ShedLock实现分布式定时任务
  3. 3 利用oss进行数据库和网站图片备份
  4. 4 Spring Cloud Stream实现数据流处理
  5. 5 Python3访问MySQL数据库快速入门Demo
  6. 6 Github开源项目作者可以免费申请 JetBrains 全家桶
  7. 7 Spring Cloud Vault快速入门Demo
  8. 8 Spring Cloud Gateway快速入门Demo
  9. 9 Spring Cloud Contract快速入门Demo
  10. 10 Spring Cloud Consul快速入门Demo
网站信息
  • 文章总数：82,714 篇
  • 文件总数：284,287 个
  • 标签总数：2,393 个
  • 分类总数：85 个
  • 留言数量：2,560 条
  • 在线人数：616 人
  • 运行天数：4,409天
  • 最后更新：2024年11月22日03点
Loading...

其他链接

关于本站

本站定位：个人技术类博客

本站作用：写博客、记日志、闲聊扯淡鼓捣技术。

问题交流

[HBLOG]公众号 HBLOG HBLOG