转载

JBOD磁盘配置和StorCLI命令使用小结

前言

已经实施过2次Hadoop大数据平台的基础环境搭建，对于硬盘的RAID模式都不约而同选择了JBOD，毕竟大数据对于文件系统的管理本身就是基于分布式，由至少3份副本来保障数据可靠性，只是不做RAID后的SAS硬盘读写性能就有点惨不忍睹。这次Hadoop大数据平台工具选择的是Cloudera，大家感兴趣的化也可以浏览下它的官网。本篇文章主要介绍如何实现在不停机状态下在线更换JBOD硬盘。

利用StorCLI命令不停机在线更换JBOD硬盘

更新历史

2016年12月31日 - 初稿

阅读原文 - https://wsgzao.github.io/post/jbod/

扩展阅读

Cloudera - http://www.cloudera.com/

JBOD原理

JBOD （ Just a Bunch Of Disks ）不是标准的 RAID 等级，它通常用来表示一个没有控制软件提供协调控制的磁盘集合。 JBOD 将多个物理磁盘串联起来，提供一个巨大的逻辑磁盘。 JBOD 的数据存放机制是由第一块磁盘开始按顺序往后存储，当前磁盘存储空间用完后，再依次往后面的磁盘存储数据。 JBOD 存储性能完全等同于单块磁盘，而且也不提供数据安全保护。它只是简单提供一种扩展存储空间的机制， JBOD 可用存储容量等于所有成员磁盘的存储空间之和。目前 JBOD 常指磁盘柜，而不论其是否提供 RAID 功能。

RAID磁盘阵列配置和调优小结 - https://wsgzao.github.io/post/raid/

参考文档和工具

StorCLI_RefMan_revf

https://www.broadcom.com/support/download-search/?dk=StorCLI

StorCLI (Command Line) Utility for Storage Management v1.07.07 on Linux

https://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5092950

使用方式

#查询硬件RAID卡信息
[root@localhost Linux]# cat /proc/scsi/scsi |more

Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: IBM      Model: ServeRAID M5210  Rev: 4.62
  Type:   Direct-Access                    ANSI  SCSI revision: 05

#解压安装storcli
unzip ibm_utl_sraidmr_storcli-1.07.07_linux_32-64.zip
rpm -Uvh storcli-1.07.07-1.noarch.rpm

#设置软连接
ln -s /opt/MegaRAID/storcli/storcli64 /bin/storcli
ln -s /opt/MegaRAID/storcli/storcli64 /sbin/storcli

#Raid数量及当前状态
[root@localhost Linux]# storcli show ctrlcount
Status Code = 0
Status = Success
Description = None

Controller Count = 2

#查询虚拟磁盘大小，raid结构等
[root@localhost Linux]# storcli /c0 /v0 show
Controller = 0
Status = Success
Description = None


Virtual Drives :
==============

-----------------------------------------------------------
DG/VD TYPE  State Access Consist Cache sCC       Size Name 
-----------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     RWTD  -   278.465 GB      
-----------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency
#可以看出此为RAID1，一个虚拟磁盘，大小为278.465 GB 

#查看control0的物理硬盘
[root@localhost Linux]# storcli /c0 /eall /sall show
Controller = 0
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

-------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model            Sp 
-------------------------------------------------------------------------
252:16    8 Onln   0 278.465 GB SAS  HDD N   Y  512B ST300MP0035    X U  
252:17    9 Onln   0 278.465 GB SAS  HDD N   Y  512B ST300MP0035    X U  
-------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

#可以看到总计有两颗硬盘，每一颗大小278.465GB，是否在线等

#查看control1的物理硬盘
[root@localhost Linux]# storcli /c1 /eall /sall show
Controller = 1
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model            Sp 
------------------------------------------------------------------------
8:0      23 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:1      24 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:2      22 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:3      19 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:4      13 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:5      14 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:6      18 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:7      12 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:8      10 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:9      16 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:10     11 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:11     21 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:12     15 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:13      9 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

#可以看到总计有两颗硬盘，每一颗大小1.0810 TB，是否在线等


#设置硬盘状态
storcli /c1/e8/s0 show all

storcli /c1/e8/s0 start locate
storcli /c1/e8/s0 set offline
storcli /c1/e8/s0 set online
storcli /c1/e8/s0 set jbod

StorCLI Help

#查看StorCLI帮助信息
[root@localhost Linux]# storcli -h
     Storage Command Line Tool  Ver 1.07.07 Nov 15, 2013

     (c)Copyright 2013, LSI Corporation, All Rights Reserved.


storcli -v 
storcli -h| -help| ? 
storcli -h| -help| ? legacy
storcli show 
storcli show all
storcli show ctrlcount
storcli /cx add vd r[0|1|5|6|00|10|50|60]
    [Size=<VD1_Sz>,<VD2_Sz>,..|all] [name=<VDNAME1>,..] 
    drives=e:s|e:s-x|e:s-x,y,e:s-x,y,z [PDperArray=x][SED]
    [pdcache=on|off|default][pi][DimmerSwitch(ds)=default|automatic(auto)|
    none|maximum(max)|MaximumWithoutCaching(maxnocache)]
    [wt|wb|awb][nora|ra] [direct|cached] [cachevd]
    [Strip=<8|16|32|64|128|256|512|1024>] [AfterVd=X] 
    [Spares = [e:]s|[e:]s-x|[e:]s-x,y] [force][ExclusiveAccess]
storcli /cx add vd each r0 [name=<VDNAME1>,..] [drives=e:s|e:s-x|e:s-x,y]
    [SED] [pdcache=on|off|default][pi] [DimmerSwitch(ds)=default|automatic(auto)|
    none|maximum(max)|MaximumWithoutCaching(maxnocache)] 
    [wt|wb|awb] [nora|ra] [direct|cached] 
    [Strip=<8|16|32|64|128|256|512|1024>] [ExclusiveAccess]
storcli /cx add VD cachecade|nytrocache r[aid][0,1,10, 1EC] 
    drives = [e:]s|[e:]s-x|[e:]s-x,y [WT| WB] [assignvds = 0,1,2] [BOOTVOLSIZE=x]
storcli /cx/ex show all 
storcli /cx/ex show status 
storcli /cx/vx del [cachecade] [force]
storcli /cx delete events
storcli /cx show events [[type= <sincereboot| sinceshutdown| includedeleted|
latest=x| ccincon vd=<0,1,...>] filter=<info|warning|critical|fatal>]
file=<filepath>
storcli /cx show eventloginfo
storcli /cx show health [all]
storcli /cx delete securitykey
storcli /cx set securitykey=xxxxxxxx {passphrase=xxxx} {keyid=xxx}
storcli /cx start Diag Duration=<Val> 
storcli /cx set securitykey keyid=xxx
storcli /cx compare securitykey=xxxxxxxxxx
storcli /cx set termlog[=on|off|offthisboot]
storcli /cx show termlog [type=config|contents]
storcli /cx delete termlog
storcli /cx Compare factorysettings file=<filepath>
storcli /cx Compare factorysettings raw file=<filepath>
storcli /cx set securitykey=xxxxxxxx oldsecuritykey=xxxxxxxx
    {passphrase=xxxx} {keyid=xxx}
storcli /cx/dx show 
storcli /cx/dall show cachecade 
storcli /cx/dx show all 
storcli /cx show freespace
storcli /cx/fall show [all] [securityKey = xxx]
storcli /cx/fall del|delete [securityKey = xxx]
storcli /cx/fall import [preview] [securityKey = xxx]
storcli /cx/vx set ssdcaching=on|off
storcli /cx/dx set security=on
storcli /cx/vx show expansion
storcli /cx show fshinting
storcli /cx/vx set fshinting=<value>
storcli /cx/vx expand Size=<xx> [expandarray]
storcli /cx[/ex]/sx show 
storcli /cx[/ex]/sx show all
storcli /cx[/ex]/sx start rebuild 
storcli /cx[/ex]/sx stop rebuild 
storcli /cx[/ex]/sx pause rebuild 
storcli /cx[/ex]/sx resume rebuild 
storcli /cx[/ex]/sx show rebuild 
storcli /cx[/ex]/sx show health 
storcli /cx[/ex]/sx start copyback target=e:s 
storcli /cx[/ex]/sx stop copyback 
storcli /cx[/ex]/sx pause copyback 
storcli /cx[/ex]/sx resume copyback 
storcli /cx[/ex]/sx show copyback 
storcli /cx[/ex]/sx start initialization  
storcli /cx[/ex]/sx stop initialization  
storcli /cx[/ex]/sx show initialization  
storcli /cx[/ex]/sx start locate  
storcli /cx[/ex]/sx stop locate  
storcli /cx[/ex]/sx show securitykey keyid 
storcli /cx[/ex]/sx add hotsparedrive [DGs=<N|0,1,2...>] [enclaffinity]
    [nonrevertible]
storcli /cx[/ex]/sx delete hotsparedrive
storcli /cx[/ex]/sx spinup
storcli /cx[/ex]/sx spindown
storcli /cx[/ex]/sx set online 
storcli /cx[/ex]/sx set offline 
storcli /cx[/ex]/sx set missing 
storcli /cx[/ex]/sx set jbod 
storcli /cx[/ex]/sx set good [force]  
storcli /cx[/ex]/sx insert dg=A array=B row=C 
storcli /cx/vx set emulationType=0|1
storcli /cx/vx set wrcache=WT|WB|AWB
storcli /cx/vx set rdcache=RA|NoRA
storcli /cx/vx set iopolicy=Cached|Direct
storcli /cx/vx set accesspolicy=RW|RO|Blocked|RmvBlkd
storcli /cx/vx set pdcache=On|Off|Default
storcli /cx/vx set name=<NameString>
storcli /cx/vx set HostAccess=ExclusiveAccess|SharedAccess
storcli /cx/vx set ds=Default|Auto|None|Max|MaxNoCache
storcli /cx/vx set autobgi=On|Off
storcli /cx/vx set pi=Off
storcli /cx/vx show
storcli /cx/vx show all
storcli /cx/vx show init
storcli /cx/vx show cc
storcli /cx/vx show erase
storcli /cx/vx show migrate
storcli /cx/vx show bgi
storcli /cx/vx show autobgi
storcli /cx/vx show trim
storcli /cx set consistencycheck|cc=[off|seq|conc] [delay=value]
    starttime=yyyy/mm/dd hh] [excludevd=x-y,z]
storcli /cx show cc|consistencycheck
storcli /cx show ocr 
storcli /cx set ocr=<on|off>
storcli /cx/vx start init[Full][Force]
storcli /cx/vx start erase [simple|normal|thorough|standard][patternA=<val>]
[patternB=<val>]
storcli /cx/vx start cc[Force]
storcli /cx/vx start migrate type=raidx [option=add|remove 
    drives=[e:]s|[e:]s-x|[e:]s-x,y] [Force] 
storcli /cx/vx stop init
storcli /cx/vx stop erase
storcli /cx/vx stop cc
storcli /cx/vx stop bgi
storcli /cx/vx pause cc
storcli /cx/vx pause bgi
storcli /cx/vx resume cc
storcli /cx/vx resume bgi
storcli /cx show 
storcli /cx show all 
storcli /cx show preservedcache 
storcli /cx/vx delete preservedcache[force] 
storcli /cx[/ex]/sx download src=<filepath> [satabridge] [mode= 5|7]
storcli /cx/ex download src=<filepath> [forceActivate] 
storcli /cx[/ex]/sx secureerase [force]  
storcli /cx[/ex]/sx start erase [simple| normal| thorough | standard| threepass]
   [patternA=<val>] [patternB=<val>]
storcli /cx[/ex]/sx stop erase 
storcli /cx[/ex]/sx show erase 
storcli /cx[/ex]/sx show rawdata pageaddr=<pageaddress in hex> file=<filename> 
storcli /cx[/ex]/sx set bootdrive=<on|off> 
storcli /cx/vx set bootdrive=<on|off>
storcli /cx show bootdrive
storcli /cx show bootwithpinnedcache
storcli /cx set bootwithpinnedcache=<on|off>
storcli /cx show activityforlocate
storcli /cx set activityforlocate=<on|off>
storcli /cx show copyback
storcli /cx set copyback=<on|off> type=smartssd|smarthdd|all 
storcli /cx show jbod
storcli /cx set jbod=<on|off> 
storcli /cx set autorebuild=<on|off> 
storcli /cx show autorebuild
storcli /cx show cachebypass
storcli /cx set cachebypass=<on|off> 
storcli /cx show usefdeonlyencrypt
storcli /cx set usefdeonlyencrypt=<on|off>
storcli /cx show prcorrectunconfiguredareas
storcli /cx set prcorrectunconfiguredareas=<on|off> 
storcli /cx show batterywarning
storcli /cx set batterywarning=<on|off> 
storcli /cx show abortcconerror
storcli /cx set abortcconerror=<on|off> 
storcli /cx show ncq
storcli /cx set ncq=<on|off> 
storcli /cx show maintainpdfailhistory
storcli /cx set maintainpdfailhistory=<on|off> 
storcli /cx show restorehotspare
storcli /cx set restorehotspare=<on|off> 
storcli /cx set bios [state=<on|off>] [Mode=<SOE|BE|HCOE|HSM>] [abs=<on|off>]
storcli /cx show bios 
storcli /cx show alarm
storcli /cx set alarm=<on|off|silence> 
storcli /cx show foreignautoimport
storcli /cx set foreignautoimport=<on|off> 
storcli /cx show directpdmapping
storcli /cx set directpdmapping=<on|off> 
storcli /cx show rebuildrate
storcli /cx set rebuildrate=<value> 
storcli /cx show loadbalancemode
storcli /cx set loadbalancemode=<on|off> 
storcli /cx show eghs
storcli /cx set eghs [state=<on|off>] [eug=<on|off>] [smarter=<on|off>] 
storcli /cx show cacheflushint
storcli /cx set cacheflushint=<value> 
storcli /cx show prrate
storcli /cx set prrate=<value> 
storcli /cx show ccrate
storcli /cx set ccrate=<value> 
storcli /cx show bgirate
storcli /cx set bgirate =<value> 
storcli /cx show dpm 
storcli /cx set dpm =<on|off> 
storcli /cx show sgpioforce 
storcli /cx set sgpioforce =<on|off> 
storcli /cx set supportssdpatrolread =<on|off> 
storcli /cx show migraterate
storcli /cx set migraterate=<value> 
storcli /cx show spinupdrivecount
storcli /cx show wbsupport
storcli /cx set spinupdrivecount=<value> 
storcli /cx show spinupdelay
storcli /cx set spinupdelay=<value> 
storcli /cx show coercion
storcli /cx set coercion=<value> 
storcli /cx show smartpollinterval
storcli /cx set smartpollinterval=<value> 
storcli /cx show eccbucketsize
storcli /cx set eccbucketsize=<value> 
storcli /cx show eccbucketleakrate
storcli /cx set eccbucketleakrate=<value> 
storcli /cx show backplane
storcli /cx set backplane mode=<value> expose=<on/off> 
storcli /cx show perfmode
storcli /cx set perfmode=<value> [maxflushlines=<value> numiostoorder=<value>]
storcli /cx show perfmodevalues
storcli /cx show pi
storcli /cx set pi [state=<on|off>] [import=<on|off>]
storcli /cx show time
storcli /cx set time=<yyyymmdd hh:mm:ss | systemtime> 
storcli /cx show ds
storcli /cx set ds=OFF type=1|2|3|4
storcli /cx set ds=ON type=1|2 [properties]
storcli /cx set ds=ON type=3|4 DefaultLdType=<val> [properties]
storcli /cx set ds [properties] 
storcli /cx show safeid
storcli /cx show rehostinfo
storcli /cx show pci
storcli /cx show ASO
storcli /cx set aso key=<key value> preview
storcli /cx set aso key=<key value>
storcli /cx set aso transfertovault
storcli /cx set aso rehostcomplete
storcli /cx set aso deactivatetrialkey
storcli /cx set factory defaults
storcli /cx download file=<filepath> [fwtype=<val>] [ResetNow] [nosigchk] 
    [noverchk]
storcli /cx flush|flushcache
storcli /cx/px show 
storcli /cx/px show all
storcli /cx/px set linkspeed=0|1.5|3|6|12 
storcli /cx/bbu show 
storcli /cx/bbu set SCAPVPD file=<input file path> VPDPage=<SCapVPDFixed>
storcli /cx/bbu show all 
storcli /cx/bbu show status 
storcli /cx/bbu show properties 
storcli /cx/bbu show learn 
storcli /cx/bbu show gasgauge Offset=xxxx Numbytes=n 
storcli /cx/bbu start learn 
storcli /cx/bbu show modes 
storcli /cx/bbu set [ learnDelayInterval=<val> | bbuMode=<val>  
    |learnStartTime=[DDD HH | off] | autolearnmode=<val> | powermode=sleep | writeaccess=sealed ]
storcli /cx/cv show 
storcli /cx/cv show all 
storcli /cx/cv show status 
storcli /cx/cv show learn 
storcli /cx/cv start learn 
storcli /cx show securitykey keyid 
storcli /cx start patrolread
storcli /cx stop patrolread
storcli /cx pause patrolread
storcli /cx resume patrolread
storcli /cx show patrolRead
storcli /cx show powermonitoringinfo
storcli /cx set patrolread = {{on mode=<auto|manual> }|{off}}
storcli /cx set patrolread [starttime=< yyyy/mm/dd hh>] 
    [maxconcurrentpd =<value>] [includessds=<on|onlymixed|off>]
    [uncfgareas=on|off]
storcli /cx set patrolread delay = <value>
storcli /cx[/ex]/sx show diag paniclog [Query] | [ExtractSlot=x] 
                                         | [EraseSlot=x] [file=filepath]
storcli /cx[/ex]/sx show diag smartlog [file=filepath] 
storcli /cx[/ex]/sx show diag errorlog [file=filepath] 
storcli /cx del Nytrocache [force]
storcli /cx show badblocks
storcli /cx flasherase 
storcli /cx shutdown 
storcli /cx/mx set mode=<Internal | External | Auto> 
storcli /cx/mx show 
storcli /cx transform iMR 
storcli /cx restart 
storcli /cx/vx show BBMT
storcli /cx/vx delete BBMT
storcli /xd add type=CacheDev 
                <devices=comma separated cache devices  | ALL> [force] 
storcli /xd add type=VirtualDrive 
                <devices=comma separated Virtual drives | ALL> 
storcli /xd delete type=<CacheDev | VirtualDrive> 
                        <devices=comma separated cache devices or 
                        Virtual drives as applicable | ALL> 
storcli /xd show type=<CacheDev | VirtualDrive> [state=<assigned|unassigned>] 
storcli /xd show 
storcli /xd show perfmon 
storcli /xd/wd<X|ALL> show 
storcli /xd/wd<X|ALL> show safeid 
storcli /xd set online [Vds=comma separated VD ids] 
storcli /xd/wdX apply key=value 
storcli /cx[/ex]/sx start format [thorough] 
storcli /cx show dequeuelog file=<filepath>


Note:Use 'page=[x]'as the last option in all the commands to set the page break.
      X=lines per page. E.g. 'storcli help page=10'
原文  https://wsgzao.github.io/post/jbod/
正文到此结束
所属分类： 编程技术

热门推荐

相关文章

阿里云首购8折
说给你听

本文目录
随机标签
书籍教程
springboot-demo Java入门教程 bootstrap3 CSS Apache基础教程 php ionic 教程 Python mysql教程 eclipse Ubuntu VPS系统配置 AngularJS 教程 MongoDB教程 Struts2教程 Redis教程 Spring教程 Git教程 Jenkins进阶系列 openfire参考指南 Java设计模式 HBase教程 java-demo Maven教程 springcloud-demo hibernate教程 Docker 教程 memcached教程 Quartz指南 Hive教程 ANTLR教程 SpringCloud Ant教程 java实例教程 Hazelcast教程 XStream教程 Elastic-Job-Lite 深入浅出MyBatis ibaties教程 SVN教程 rabittmq教程 Hadoop教程 solr教程 WebService CXF学习 JPA教程 ActiveMQ中文指南 Java内存模型 dubbo教程 python3-demo Linux入门视频教程
近期评论
  1. 1 使用 Docker 打包和运行 Vue 应用
  2. 2 Dependency Check命令行方式扫描jar包的安全漏洞
  3. 3 maven使用Dependency-Check来扫描安全漏洞
  4. 4 使用sonarqube分析代码质量
  5. 5 使用screw来对比数据库表和字段差异
  6. 6 Spring Boot 实现防盗链功能
  7. 7 在Spring Boot中的实现国际化（i18n）
  8. 8 一个用于 SMB/CIFS 网络的 Java 库-JCIFS
  9. 9 Spring Boot 3.x嵌入MongoDB 进行测试
  10. 10 java如何解析和生成yaml文件？
网站信息
  • 文章总数：82,737 篇
  • 文件总数：284,312 个
  • 标签总数：2,416 个
  • 分类总数：85 个
  • 留言数量：2,559 条
  • 在线人数：607 人
  • 运行天数：4,439天
  • 最后更新：2024年12月22日19点
Loading...

其他链接

关于本站

本站定位：个人技术类博客

本站作用：写博客、记日志、闲聊扯淡鼓捣技术。

问题交流

[HBLOG]公众号 HBLOG HBLOG