PCIE的简单配置

最近测试了一下PCIE-SSD在数据库环境的迁移和加压情况，IOPS无可置疑，比起机械硬盘确实是高了很多个量级，在数据环境中的IO方面确实有很稳定的提升，目前使用闪存产品系列，看官网最新的发布已经支持6T了，这个着实让我想起曾经的U盘，几十M,几百兆都曾经风光一时。
基本的软件要求如下，摘自用户文档中，各个厂商应该都差不多。

尽管官网的发布是支持Linux 5,6其实安装包里已经有了7的支持版本了，这个绑定是内核粒度，确实非常细。

1.1在官网下载rpm包，解压

如果是Linux 6版本，基本就是参考CentOS，可以看到版本6有以下内核版本的rpm

./CentOS/6:

total 5816

-rw-rw-r-- 1 1000 1000 559112 Mar 31 08:18 shannon-module-2.6.32-131.0.15.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 563396 Mar 31 08:18 shannon-module-2.6.32-220.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 563028 Mar 31 08:18 shannon-module-2.6.32-279.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 528616 Mar 31 08:18 shannon-module-2.6.32-358.23.2.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 528456 Mar 31 08:18 shannon-module-2.6.32-358.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 528204 Mar 31 08:18 shannon-module-2.6.32-431.11.2.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 527836 Mar 31 08:18 shannon-module-2.6.32-431.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 527528 Mar 31 08:18 shannon-module-2.6.32-504.12.2.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 527732 Mar 31 08:18 shannon-module-2.6.32-504.8.1.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 527756 Mar 31 08:18 shannon-module-2.6.32-504.el6.x86_64-2.8-6.1.x86_64.rpm

-rw-rw-r-- 1 1000 1000 555616 Mar 31 08:18 shannon-module-2.6.32-71.el6.x86_64-2.8-6.1.x86_64.rpm

1.2选择适合的内核版本，安装rpm

[root@db117 6]# rpm -ivh shannon-module-2.6.32-431.el6.x86_64-2.8-6.1.x86_64.rpm

Preparing...                ########################################### [100%]

   1:shannon-module-2.6.32-4########################################### [100%]

1.3验证是否安装完成

[root@db117 rpm]# ll /lib/modules/2.6.32-431.el6.x86_64/extra

total 2448

-rw-r--r-- 1 root root 2502739 Mar 13 17:28 shannon.ko

安装shannon-utils

[root@db117 rpm]# rpm -ivh shannon-utils-2.8-6.1.x86_64.rpm

Preparing...                ########################################### [100%]

   1:shannon-utils          ########################################### [100%]

1.4检查安装情况：

[root@db117 rpm]#fdisk -l

Disk /dev/dfa: 800.0 GB, 800000573440 bytes
32 heads, 32 sectors/track, 1525880 cylinders
Units = cylinders of 1024 * 512 = 524288 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 32768 bytes
Disk identifier: 0x00000000

[root@db117 rpm]# ls /dev/df*

/dev/dfa

[root@db117 rpm]# rpm -qa | grep shannon

shannon-utils-2.8-6.1.x86_64

shannon-module-2.6.32-431.el6.x86_64-2.8-6.1.x86_64

1.5查看PCIE-SSD状态

[root@db117 rpm]# shannon-status /dev/scta

Basic Information:

Control Device Node:                /dev/scta

Driver Mode:                        Block

Block Device Node:                  /dev/dfa

Device State:                       Attached

Access Mode:                        ReadWrite

Product Model:                      Direct-IO G3i

Serial Number:                      SH15608K734181

UDID:                               1CBO0275-1CB00030-04815608-4B734181

PCI VendorID:                       1CB0

PCI DeviceID:                       0275

PCI Bus Address:                    05:00:0

PCI Link Speed:                     pcie 2.0 x 8

Firmware Version:                   3.2

Firmware Build:                     f2796a40

Driver Version:                     2.8.6.1

FPGA Reconfig Support:              Yes

Logical Sector:                     512

Physical Sector:                    4096

Disk Capacity:                      800.00 GB

Physical Capacity:                  1094.72 GB

Overprovision:                      26.93%

Atomic Write:                       Disabled

Prioritize Write:                   Disabled

Work Status:

Controller Temperature:             45 degC, Max 45 degC

Board Temperature:                  30 degC, Max 31 degC

Flash Temperature:                  32 degC, Max 32 degC

Internal Voltage:                   1022 mV, Max 1028 mV

Auxiliary Voltage:                  1833 mV, Max 1842 mV

Power Cycles:                       8

Power On Time:                      46 hours 12 minutes

Dynamic Bad Blocks:                 0

Media Status:                       Healthy

SEU Flag:                           Normal

Lifetime Data Volumes:

Host Write Data:                    73555.63 GB

Host Read Data:                     72660.33 GB

Total Write Data:                   80719.31 GB

Lifetime Write Amplifier:           1.090

Estimated Life Left:                98.560%

Realtime IO Statistics:

Write Bandwith:                     0.000 MB/s

Write IOPS:                         0.000 K

Avg Write Latency:                  0.000 ms

Total Write Bandwith:               0.000 MB/s

Read Bandwith:                      0.001 MB/s

Read IOPS:                          0.000 K

Avg Read Latency:                   0.042 ms

Write Amplifier:                    1.000

Buffer Write Percentage:            0%

[root@db117 rpm]#

1.6监控性能

[root@db117 rpm]# shannon-status -m /dev/scta

        Direct-IO PCIe Flash Card Monitor Program

Commands: q|Q exit; g|G general info; m|M main window

Monitoring device '/dev/scta' at PCI:05:00:0

 Device State:                  Attached

 Access Mode:                   ReadWrite

 Power Cycles:                  8

 Power On Time:                 46 hours 12 minutes

 Controller Temperature:        45 degC, Max 45 degC

 Board Temperature:             30 degC, Max 31 degC

 Flash Temperature:             31 degC, Max 32 degC

 Internal Voltage:              1019 mV, Max 1028 mV

 Auxiliary Voltage:             1833 mV, Max 1842 mV

 Free Block Count:              2102

 Host Write Data:               73555.63 GB

 Write Bandwith:                0.000 MB/s

 Write IOPS:                    0.000 K

 Avg Write Latency:             0.000 ms

 Host Read Data:                72660.33 GB

 Read Bandwith:                 0.000 MB/s

 Read IOPS:                     0.000 K

 Avg Read Latency:              0.000 ms

 Total Write Data:              80719.31 GB

 Total Write Bandwith:          0.000 MB/s

 Write Amplifier:               1.000, lifetime 1.090

 Buffer Write Percentage:       0%

1.7分区格式化：

[root@db117 rpm]# fdisk /dev/dfa

Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-1525880, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-1525880, default 1525880):

Using default value 1525880

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@db117 rpm]# mkfs -t ext4 /dev/dfa1

1.8挂载分区

mkdir /U02

mount /dev/dfa1 /U02

1.9配置/etc/fstab

/dev/dfa1               /U02                    ext4    defaults,noatime,nodiratime 1 2