docker registry_v2 部署及错误总结
docker registry_v2
docker registry_v2的搭建和排错文档，nginx+registry源码搭建，有别于网上类docker的搭建方法，方便registry日后调优
搭建过程
- CA证书的制作(openssl)
- nginx的搭建及配置
- registry源码编译及配置
- 验证及排错
CA证书的制作
1.首先我们去 /etc/ssl/openssl.cnf 下修改下参数，必须在生成证书之前修改，否则无意义
[ CA_default ]
dir = /etc/ssl/demoCA # Where everything is kept
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
#unique_subject = no # Set to 'no' to allow creation of
# several ctificates with same subject.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/certs/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crlnumber = $dir/crlnumber # the current crl number
# must be commented out to leave a V1 CRL
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem# The private key
RANDFILE = $dir/private/.rand # private random number file
[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
#这个很重要，否则在后面会报registry endpoint...x509: cannot validate certificate for ... because it doesn't contain any IP SANs
subjectAltName=IP:192.168.172.150
2.制作证书
证书的配置文件都在 Ubuntu 的路径在/etc/ssl下
cd /etc/ssl
mkdir demoCA demoCA/certs demoCA/crl demoCA/newcerts demoCA/private
touch /etc/ssl/demoCA/index.txt
echo 01 > /etc/ssl/demoCA/serial
cd /etc/ssl/demoCA
openssl req -newkey rsa:4096 -nodes -sha256 -keyout cakey.pem -x509 -days 365 -out cacert.pem
mv cacert.pem certs/ && mv cakey.pem private/
注意这里的domain设置成自己的域名即可，比如我的是*.192.168.172.150.xip.io
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) [Default City]:beijing
Organization Name (eg, company) [Default Company Ltd]:self
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:*.192.168.172.150.xip.io
Email Address []:jackyuan@126.com
OK,至此，根证书等制作完成
nginx的搭建及配置
1.选择版本安装,最好是高版本，否则add header功能没法使用
cd ~
wget http://nginx.org/download/nginx-1.9.4.tar.gz
tar zxvf nginx-1.9.4.tar.gz
cd ./nginx-1.4.6 && /
./configure --user=www --group=www --prefix=/opt/nginx /
--with-pcre /
--with-http_stub_status_module /
--with-http_ssl_module /
--with-http_addition_module /
--with-http_realip_module /
--with-http_flv_module && /
make && /
make install
2.生成nginx的ssl证书,并加入进openssl本身的证书数据库
mkdir -p /etc/nginx/ssl
cd /etc/nginx/ssl
openssl genrsa -out nginx.key 4096
openssl req -new -key nginx.key -out nginx.csr
#上面这一步的配置要和跟设置的一样，尤其是domain那块
openssl ca -in nginx.csr -out nginx.crt
在这里如果不在之前配置好CA的配置，则会出现demoCA无法打开等错误，所以要注意。
3.生成htpassword,用户名和密码都为admin
htpasswd -cb /opt/nginx/conf/.htpasswd admin admin
4.修改nginx配置
user www www;
worker_processes auto;
error_log /var/log/nginx_error.log error;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
worker_rlimit_nofile 51200;
events {
use epoll;
worker_connections 51200;
multi_accept on;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx_access.log main;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
upstream registry {
server 192.168.172.150:5000;
}
server {
listen 443;
server_name 192.168.172.150;
ssl on;
ssl_certificate /etc/nginx/ssl/nginx.crt;
ssl_certificate_key /etc/nginx/ssl/nginx.key;
client_max_body_size 0;
chunked_transfer_encoding on;
location /v2/ {
auth_basic "Registry realm";
auth_basic_user_file /opt/nginx/conf/.htpasswd;
add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;
proxy_pass http://registry;
proxy_set_header Host /$http_host; # required for docker client's sake
proxy_set_header X-Real-IP /$remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For /$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
}
# location / {
# auth_basic "docker registry";
# auth_basic_user_file /opt/nginx/conf/.htpasswd;
#
# root html;
# index index.html index.htm;
#
# proxy_pass http://registry;
# proxy_set_header Host $http_host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header Authorization "";
# client_body_buffer_size 128k;
# proxy_connect_timeout 90;
# proxy_send_timeout 90;
# proxy_read_timeout 90;
# proxy_buffer_size 8k;
# proxy_buffers 4 32k;
# proxy_busy_buffers_size 64k;
# proxy_temp_file_write_size 64k;
# }
# location /_ping {
# auth_basic off;
# proxy_pass http://registry;
# }
# location /v1/_ping {
# auth_basic off;
# proxy_pass http://registry;
# }
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
这里要说明两个地方：proxy_set_header X-Forwarded-Proto $scheme;注意这里没有/$scheme，如果配置则会报： unsupported protocol scheme ""
PS:认证是LDAP的可以在nginx里设置一下
user www www;
worker_processes auto;
error_log /var/log/nginx_error.log error;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
worker_rlimit_nofile 51200;
events {
use epoll;
worker_connections 51200;
multi_accept on;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx_access.log main;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
upstream registry {
server 192.168.172.150:5000;
}
ldap_server ldapserver {
url ldap://xxxxxxxxx:389/OU=Users,DC=xxxxx,DC=it?samaccountname?sub?(objectClass=user);
binddn xxxxx@xxxxxxxxxx.it;
binddn_passwd xxxxxxxxxxxx
group_attribute uniquemember;
group_attribute_is_dn on;
}
server {
listen 443;
server_name 192.168.172.150;
ssl on;
ssl_certificate /etc/nginx/ssl/nginx.crt;
ssl_certificate_key /etc/nginx/ssl/nginx.key;
client_max_body_size 0;
chunked_transfer_encoding on;
location /v2/ {
auth_ldap "Forbidden";
auth_ldap_servers ldapserver;
add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;
proxy_pass http://registry;
proxy_set_header Host /$http_host; # required for docker client's sake
proxy_set_header X-Real-IP /$remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For /$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
还有一个地方：如果之前有配置过V1的，可以保留，但最好分离出来，我是直接注掉了。
5.验证及启动启动nginx
/opt/nginx/sbin/nginx -t
/opt/nginx/sbin/nginx -s /opt/nginx/conf/nginx.conf
registry源码编译及配置
1.checkout 源码：
git clone https://github.com/docker/distribution
cd distribution
git checkout v2.1.1
godep restore ./...
2.安装registry,这里图省事，要编译重新设定下gopath即可
go get github.com/docker/distribution
3.配置registry,config-example.yml
version: 0.1
log:
fields:
service: registry
storage:
cache:
layerinfo: inmemory
filesystem:
rootdirectory: /home/jojo/registry
http:
addr: :5000
secret: admin
# tls:
# certificate: /etc/ssl/demoCA/certs/cacert.pem
# key: /etc/ssl/demoCA/private/cakey.pem
#proxy:
# remoteurl: https://api.192.168.172.150.xip.io
# username: admin
# password: admin
这里注意我注掉的部分，因为前方已经有一层代理了，我们这里就没有必要设置tls了，否则，后端 会报 tls: first record does not look like a TLS handshake
4.配置docker,增加一行
vi /etc/default/docker
DOCKER_OPTS="--insecure-registry api.192.168.172.150.xip.io --tlsverify --tlscacert /etc/ssl/demoCA/certs/cacert.pem"
5.启动docker和registry
service docker start
registry /home/jojo/register/config-example.yml
验证及排错
1.验证联通性：
curl -i -k -v https://admin:admin@api.192.168.172.150.xip.io/v2/
* Hostname was NOT found in DNS cache
* Trying 192.168.172.150...
* Connected to api.192.168.172.150.xip.io (192.168.172.150) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
* subject: C=CN; ST=beijing; O=self; OU=self; CN=*.192.168.172.150.xip.io; emailAddress=jack@126.com
* start date: 2015-09-18 13:54:11 GMT
* expire date: 2016-09-17 13:54:11 GMT
* issuer: C=CN; ST=beijing; L=beijing; O=self; OU=self; CN=*.192.168.172.150.xip.io; emailAddress=jack@126.com
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Server auth using Basic with user 'admin'
> GET /v2/ HTTP/1.1
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.35.0
> Host: api.192.168.172.150.xip.io
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
* Server nginx/1.9.4 is not blacklisted
< Server: nginx/1.9.4
Server: nginx/1.9.4
< Date: Fri, 18 Sep 2015 17:02:00 GMT
Date: Fri, 18 Sep 2015 17:02:00 GMT
< Content-Type: application/json; charset=utf-8
Content-Type: application/json; charset=utf-8
< Content-Length: 2
Content-Length: 2
< Connection: keep-alive
Connection: keep-alive
< Docker-Distribution-Api-Version: registry/2.0
Docker-Distribution-Api-Version: registry/2.0
< Docker-Distribution-Api-Version: registry/2.0
Docker-Distribution-Api-Version: registry/2.0
2.验证docker login
docker login -u admin -p admin -e jackyuan@126 https://api.192.168.172.150.xip.io/v2/
WARNING: login credentials saved in /root/.docker/config.json
Login Succeeded
3.push镜像到私有registry
docker tag 91e54dfb1179 api.192.168.172.150.xip.io/ubuntu:trusty
docker push api.192.168.172.150.xip.io/ubuntu:trusty
The push refers to a repository [api.192.168.172.150.xip.io/ubuntu] (len: 1)
91e54dfb1179: Image already exists
d74508fb6632: Image successfully pushed
c22013c84729: Image successfully pushed
d3a1f33e8a5a: Image successfully pushed
Digest: sha256:a731c12a4d21af384c4659666f177cd1e871646b95b9440d709ec4ee176145b2
4.查看是否上传
curl -i -k -v https://admin:admin@api.192.168.172.150.xip.io/v2/_catalog
{"repositories":["ubuntu"]}
cd /home/jojo/registry/docker/registry/v2 && ls
blobs repositories
这里就不往里面看了，深入registry后可以继续看里面的数据结构。
正文到此结束
热门推荐
相关文章
近期评论
-
文章和留言都翻到11页了 没有OOM
-
-
朋友，翻页到11页，及以后，会出现OOM，无法访问
-
-
搞个gitee的项目
-
-
版本号是多少，你可以下载哪个代码仓库，jdk选1.8 直接跑就行
-
-
-
Loading...
![[HBLOG]公众号](https://www.liuhaihua.cn/img/qrcode_gzh.jpg)
