转载

Docker 容器的IO基准

［编者的话］本次分享主要介绍Docker 容器IO基准的创建与测试。

本周我需要一个快速而且简单的IO 基准工具，因此决定创建一个Docker 容器来实现我的目标。Docker 容器它本身是比较简单的，在你闲暇的时候就可以使用。Docker 容器基于Ubuntu的最新版本，安装FIO包，并从我的网站抓取一个简单的测试文件来生成负载和基准测试结果。

Dockerfile

1 # FIO benchmark on Ubuntu:latest
 2 
 3 FROM ubuntu:latest
 4 MAINTAINER Chris Mutchler <chris@virtualelephant.com>
 5 
 6 RUN apt-get update
 7 RUN apt-get -y install fio wget
 8 RUN wget http://virtualelephant.com/wp-content/uploads/2015/10/threads.txt
 9 
 10 CMD [ "/bin/bash" ]

一旦Docker镜像上传到我的repo，我就会创建一个简单的JSON文件来部置容器到我的Mesos集群。

iobench.json

{
 "container": {
 "type": "DOCKER",
 "docker": {
   "image": "chrismutchler/iobench"
 }    
 },
 "id": "iobench",
 "instances": 1,
 "cpus": 0.25,
 "mem": 256,
 "uris": [],
 "cmd": "while true; do date; /usr/bin/fio /threads.txt; sleep 10; done"
 }

如上所示，Docker容器将继续无限地运行FIO测试，直到这个应用程序在这次马拉松式的测试中被杀死。该应用程序也可以扩展到跨多个apache Mesos节点上运行，它们各自独立运行FIO测试。运行任何类型的负载或者在您的环境中做基准测试时要小心，它可能有不利影响。

通过命令行在Ｍarathon启动应用程序是很简单的。

curl -A POST http://mesos.local.domain:8080/v2/apps -d @iobench.json -H "Content-Type: application/json"

我曾用这个测试在我的环境中创建一个嘈杂的邻居的问题去测试VMware Storage IO Control (SIOC)的设置，它非常适合这个角色的工作。一旦应用程序已经被启动，它的结果可以下载或者在Mesos 用户界面查看，通过选择’沙盒’，然后就会有STDOUT （标准输出）日志文件。为了理解FIO对于IO 基准做了些什么，请阅读 Ben Martin的博客 ,FIO测试文件来是从那里拷贝的。

stdout

--container="mesos-20151009-234723-2583865536-5050-2901-S1.c0052620-2322-4d12-aa49-1e3b24402f50" --docker="docker" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/tmp/mesos/slaves/20151009-234723-2583865536-5050-2901-S1/frameworks/20151009-234723-2583865536-5050-2901-0001/executors/iobench.cb93c618-6eea-11e5-9209-0050569a4da8/runs/c0052620-2322-4d12-aa49-1e3b24402f50" --stop_timeout="0ns"
 --container="mesos-20151009-234723-2583865536-5050-2901-S1.c0052620-2322-4d12-aa49-1e3b24402f50" --docker="docker" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/tmp/mesos/slaves/20151009-234723-2583865536-5050-2901-S1/frameworks/20151009-234723-2583865536-5050-2901-0001/executors/iobench.cb93c618-6eea-11e5-9209-0050569a4da8/runs/c0052620-2322-4d12-aa49-1e3b24402f50" --stop_timeout="0ns"
 Registered docker executor on dhcp2-157.local.domain
 Starting task iobench.cb93c618-6eea-11e5-9209-0050569a4da8
 Sat Oct 10 01:04:40 UTC 2015
 bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
 queryA: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=mmap, iodepth=1
 queryB: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=mmap, iodepth=1
 bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
 fio-2.1.3
 Starting 4 processes
 bgwriter: Laying out IO file(s) (1 file(s) / 256MB)
 queryA: Laying out IO file(s) (1 file(s) / 256MB)
 queryB: Laying out IO file(s) (1 file(s) / 256MB)
 bgupdater: Laying out IO file(s) (1 file(s) / 32MB)
 
 bgwriter: (groupid=0, jobs=1): err= 0: pid=8: Sat Oct 10 01:05:32 2015
 write: io=262144KB, bw=11578KB/s, iops=2894, runt= 22641msec
 slat (usec): min=16, max=4713.1K, avg=334.93, stdev=20011.69
 clat (usec): min=9, max=4750.2K, avg=10711.15, stdev=112034.10
  lat (usec): min=75, max=4750.6K, avg=11048.59, stdev=113823.09
 clat percentiles (msec):
  |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    6],
  | 30.00th=[    6], 40.00th=[    7], 50.00th=[    7], 60.00th=[    7],
  | 70.00th=[    8], 80.00th=[    8], 90.00th=[    9], 95.00th=[    9],
  | 99.00th=[   16], 99.50th=[   31], 99.90th=[  947], 99.95th=[ 1582],
  | 99.99th=[ 4752]
 bw (KB  /s): min=    5, max=39208, per=100.00%, avg=15073.13, stdev=7536.85
 lat (usec) : 10=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
 lat (usec) : 1000=0.01%
 lat (msec) : 2=0.05%, 4=12.47%, 10=85.23%, 20=1.46%, 50=0.34%
 lat (msec) : 250=0.05%, 500=0.19%, 750=0.05%, 1000=0.05%, 2000=0.05%
 lat (msec) : >=2000=0.05%
 cpu          : usr=2.60%, sys=8.83%, ctx=126969, majf=0, minf=26
 IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  issued    : total=r=0/w=65536/d=0, short=r=0/w=0/d=0
 queryA: (groupid=0, jobs=1): err= 0: pid=9: Sat Oct 10 01:05:32 2015
 read : io=262144KB, bw=12106KB/s, iops=3026, runt= 21654msec
 clat (usec): min=26, max=4713.2K, avg=322.55, stdev=20018.65
  lat (usec): min=26, max=4713.2K, avg=322.77, stdev=20018.66
 clat percentiles (usec):
  |  1.00th=[   31],  5.00th=[   71], 10.00th=[   77], 20.00th=[  115],
  | 30.00th=[  126], 40.00th=[  155], 50.00th=[  159], 60.00th=[  165],
  | 70.00th=[  183], 80.00th=[  207], 90.00th=[  253], 95.00th=[  314],
  | 99.00th=[  724], 99.50th=[ 1012], 99.90th=[ 2576], 99.95th=[ 4384],
  | 99.99th=[272384]
 bw (KB  /s): min=    7, max=24840, per=67.92%, avg=16686.43, stdev=7338.70
 lat (usec) : 50=4.04%, 100=11.27%, 250=74.04%, 500=8.29%, 750=1.42%
 lat (usec) : 1000=0.45%
 lat (msec) : 2=0.36%, 4=0.09%, 10=0.03%, 20=0.01%, 50=0.01%
 lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
 lat (msec) : 2000=0.01%, >=2000=0.01%
 cpu          : usr=2.93%, sys=7.36%, ctx=131487, majf=65536, minf=31
 IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  issued    : total=r=65536/w=0/d=0, short=r=0/w=0/d=0
 queryB: (groupid=0, jobs=1): err= 0: pid=10: Sat Oct 10 01:05:32 2015
 read : io=262144KB, bw=11910KB/s, iops=2977, runt= 22010msec
 clat (usec): min=27, max=4714.3K, avg=325.85, stdev=20020.01
  lat (usec): min=27, max=4714.3K, avg=326.11, stdev=20020.01
 clat percentiles (usec):
  |  1.00th=[   32],  5.00th=[   72], 10.00th=[   82], 20.00th=[  117],
  | 30.00th=[  139], 40.00th=[  155], 50.00th=[  159], 60.00th=[  169],
  | 70.00th=[  187], 80.00th=[  213], 90.00th=[  262], 95.00th=[  318],
  | 99.00th=[  700], 99.50th=[  980], 99.90th=[ 2608], 99.95th=[ 4768],
  | 99.99th=[272384]
 bw (KB  /s): min=    2, max=23384, per=65.75%, avg=16152.90, stdev=6882.55
 lat (usec) : 50=2.90%, 100=9.19%, 250=76.15%, 500=9.50%, 750=1.34%
 lat (usec) : 1000=0.43%
 lat (msec) : 2=0.32%, 4=0.09%, 10=0.03%, 20=0.01%, 50=0.01%
 lat (msec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%, 2000=0.01%
 lat (msec) : >=2000=0.01%
 cpu          : usr=3.81%, sys=6.74%, ctx=131493, majf=65536, minf=30
 IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  issued    : total=r=65536/w=0/d=0, short=r=0/w=0/d=0
 bgupdater: (groupid=0, jobs=1): err= 0: pid=11: Sat Oct 10 01:05:32 2015
 read : io=16416KB, bw=3611.1KB/s, iops=902, runt=  4545msec
 slat (usec): min=81, max=285023, avg=520.41, stdev=5536.65
 clat (usec): min=1, max=2841, avg= 5.40, stdev=80.88
  lat (usec): min=115, max=285031, avg=528.39, stdev=5537.20
 clat percentiles (usec):
  |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
  | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
  | 70.00th=[    2], 80.00th=[    3], 90.00th=[    3], 95.00th=[    3],
  | 99.00th=[    3], 99.50th=[    6], 99.90th=[ 1448], 99.95th=[ 2384],
  | 99.99th=[ 2832]
 bw (KB  /s): min= 2096, max= 4472, per=14.60%, avg=3587.00, stdev=1065.00
 write: io=16352KB, bw=3597.9KB/s, iops=899, runt=  4545msec
 slat (usec): min=74, max=254089, avg=489.47, stdev=3971.19
 clat (usec): min=1, max=2233, avg= 5.13, stdev=66.52
  lat (usec): min=77, max=254096, avg=497.09, stdev=3971.57
 clat percentiles (usec):
  |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
  | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
  | 70.00th=[    2], 80.00th=[    3], 90.00th=[    3], 95.00th=[    3],
  | 99.00th=[    3], 99.50th=[    5], 99.90th=[ 1256], 99.95th=[ 1816],
  | 99.99th=[ 2224]
 bw (KB  /s): min= 2024, max= 4504, per=28.58%, avg=3514.88, stdev=1085.81
 lat (usec) : 2=0.04%, 4=99.26%, 10=0.24%, 50=0.18%, 100=0.04%
 lat (usec) : 250=0.06%, 500=0.02%, 750=0.01%, 1000=0.01%
 lat (msec) : 2=0.07%, 4=0.06%
 cpu          : usr=8.30%, sys=4.71%, ctx=16460, majf=0, minf=25
 IO depths    : 1=99.8%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.0%, >=64=0.0%
  submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  issued    : total=r=4104/w=4088/d=0, short=r=0/w=0/d=0
 
 Run status group 0 (all jobs):
 READ: io=540704KB, aggrb=24566KB/s, minb=3611KB/s, maxb=12106KB/s, mint=4545msec, maxt=22010msec
 WRITE: io=278496KB, aggrb=12300KB/s, minb=3597KB/s, maxb=11578KB/s, mint=4545msec, maxt=22641msec
 
 Disk stats (read/write):
 dm-2: ios=135188/67188, merge=0/0, ticks=39356/28285, in_queue=67640, util=97.96%, aggrios=135217/70245, aggrmerge=0/0, aggrticks=1492/4673, aggrin_queue=6169, aggrutil=15.06%
 dm-0: ios=135217/70245, merge=0/0, ticks=1492/4673, in_queue=6169, util=15.06%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
 loop1: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
 loop0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%