Docker Java App Load Balanced By Nginx Or Apache HTTP, Mongo Replica Set And Hazelcast Cluster
为了发挥Docker在跨多个服务器的分布式应用程序的部署(甚至是跨区域)的能力,人们不应该限制哪些服务进到哪个服务器。动态可扩展性(或自动缩放)的关键环境的要求(如生产环境)不只是适用于新的微服务架构设计。也适用于典型的单片应用程序的部署。如果这些服务绑定在特定的服务器上,那么这些程序的未来的可扩展性是很困难的。
为了能够实现服务发现,我们需要满足以下条件:
我们还要实现强的服务,以满足企业级对服务发现的要求。包括以下这些问题:
大多数典型的服务发现工具拥有某种高度可用的分布式(key/value)存储,大家可以参考阅读这篇博客 http://technologyconversations.com/2015/09/08/service-discovery-zookeeper-vs-etcd-vs-consul/
这些工具的主要缺点是他们自己在容器中运行对第三方工具的依赖。为了使用Consul,打个比方。一个用户需要在程序中使用Consul 和Registrator 容器-这一定会使容器的数量增加。那么用户就需要管理这些容器。
DCHQ,从另一方面说。如果使用代理来协调服务注册和服务发现。把信息存储在底层DCHQ数据库。这意味着我们不需要附加的容器。另外,服务发现框架,允许用户自定义某些事件需要被执行的同时,利用的不只是IP和应用程序中的其他容器的主机名的信息提供了更大的灵活性。在部署的时候使用的也是环境变量的值。
这并不等于说服务发现框架使用DHCQ就能完全替代其它工具,最好的工具还是能够满足业务的需求。
在这篇blog中,我们将介绍使用Docker部署3个服务发现的程序,包括: Nginx & Apache HTTP Server load balancing , Mongo Replica Set and Hazelcast Cluster .
包括以下内容:
用户一旦登录到DCHQ (either the hosted DCHQ.io or on-premise version),用户可以导航到 Manage > App/Machine,然后可以点击 +创建新的 Docker Compose模板
我们使用 Docker Hub官方镜像创建了5 个程序的模板
这些模板包括了一下程序栈:
更多关于Docker application modeling in DCHQ的资料可以访问: http://dchq.co/docker-compose.html
更多关于Docker service discovery in DCHQ的资料可以访问: http://dchq.co/docker-service-discovery.html
通过这些程序的模板,你会发现一些容器为了配置容器会调用Bash脚本插件。这些脚本对post-provision操作执行的非常好。
这些插件可以在 Manage > Plug-ins导航中创建,一旦这些脚本启动是,DHCQ代理服务会执行这些容器里面的脚本。用户可以指定一个能被在请求之后被覆盖的参数。在$符号之前的都可以被成为参数,比如: $file_url 可以是一个参数,它允许开发者指定的网址下载一个WAR文件,还可以在当用户希望更新运行在容器上例如Java WAR文件请求时间和后规定被覆盖:
该插件ID需要在定义YAML应用程序模板时候提供,例如:调用Nginxbash脚本插件。我们可以参照下面的plug-in ID:
Nginx: image: nginx:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: 0H1Nk restart: true lifecycle: on_create, post_scale_out:AppServer, post_scale_in:AppServer, post_stop:AppServer, post_start:AppServer arguments: # Use container_private_ip if you're using Docker networking - servers=server {{AppServer | container_private_ip}}:8080; # Use container_hostname if you're using Weave networking #- servers=server {{AppServer | container_hostname}}:8080;
Nginx: image: nginx:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: 0H1Nk restart: true lifecycle: on_create, post_scale_out:AppServer, post_scale_in:AppServer, post_stop:AppServer, post_start:AppServer arguments: # Use container_private_ip if you're using Docker networking - servers=server {{AppServer | container_private_ip}}:8080; # Use container_hostname if you're using Weave networking #- servers=server {{AppServer | container_hostname}}:8080;
你会发现,同样的Nginx插件,在不同的阶段得到了执行。
然而这里的服务发现框架同时做着服务发现和服务注册的事情。
Nginx: image: dchq/nginx-consul:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: GINmu restart: true lifecycle: on_create arguments: - APPSERVER_IP={{AppServer | container_private_ip}} - SERVICE_NAME={{AppServer | SERVICE_NAME}} - SERVICE_TAGS={{AppServer | SERVICE_TAGS}} - CONSUL_IP={{Consul | container_private_ip}} AppServer: image: tomcat:8.0.21-jre8 mem_min: 600m host: host1 cluster_size: 1 environment: - mongo_url={{Mongo|container_private_ip}}:27017/dchq - solr_host={{Solr|container_private_ip}} - solr_port=8983 - SERVICE_NAME=app - SERVICE_TAGS=production plugins: - !plugin id: oncXN restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/dbconnect.war - dir=/usr/local/tomcat/webapps/ROOT.war - delete_dir=/usr/local/tomcat/webapps/ROOT Consul: image: progrium/consul:latest host: host1 ports: - "8300:8300" - "8400:8400" - "8500:8500" - "8600:53/udp" command: -server -bootstrap -advertise 10.0.2.15 Registrator: image: gliderlabs/registrator:latest host: host1 command: consul://<HOST_IP>:8500 volumes: - "/var/run/docker.sock:/tmp/docker.sock" Solr: image: solr:latest mem_min: 300m host: host1 publish_all: false plugins: - !plugin id: doX8s restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/names.zip Mongo: image: mongo:latest host: host1 mem_min: 400m
Nginx: image: dchq/nginx-consul:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: GINmu restart: true lifecycle: on_create arguments: - APPSERVER_IP={{AppServer | container_private_ip}} - SERVICE_NAME={{AppServer | SERVICE_NAME}} - SERVICE_TAGS={{AppServer | SERVICE_TAGS}} - CONSUL_IP={{Consul | container_private_ip}} AppServer: image: tomcat:8.0.21-jre8 mem_min: 600m host: host1 cluster_size: 1 environment: - mongo_url={{Mongo|container_private_ip}}:27017/dchq - solr_host={{Solr|container_private_ip}} - solr_port=8983 - SERVICE_NAME=app - SERVICE_TAGS=production plugins: - !plugin id: oncXN restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/dbconnect.war - dir=/usr/local/tomcat/webapps/ROOT.war - delete_dir=/usr/local/tomcat/webapps/ROOT Consul: image: progrium/consul:latest host: host1 ports: - "8300:8300" - "8400:8400" - "8500:8500" - "8600:53/udp" command: -server -bootstrap -advertise 10.0.2.15 Registrator: image: gliderlabs/registrator:latest host: host1 command: consul://<HOST_IP>:8500 volumes: - "/var/run/docker.sock:/tmp/docker.sock" Solr: image: solr:latest mem_min: 300m host: host1 publish_all: false plugins: - !plugin id: doX8s restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/names.zip Mongo: image: mongo:latest host: host1 mem_min: 400m
HTTP-LB: image: httpd:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: uazUi restart: true lifecycle: on_create, post_scale_out:AppServer, post_scale_in:AppServer arguments: # Use container_private_ip if you're using Docker networking - BalancerMembers=BalancerMember http://{{AppServer | container_private_ip}}:8080 # Use container_hostname if you're using Weave networking #- BalancerMembers=BalancerMember http://{{AppServer | container_hostname}}:8080 AppServer: image: tomcat:8.0.21-jre8 mem_min: 600m host: host1 cluster_size: 1 environment: - mongo_url={{Mongo|container_private_ip}}:27017/dchq - solr_host={{Solr|container_private_ip}} - solr_port=8983 plugins: - !plugin id: oncXN restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/dbconnect.war - dir=/usr/local/tomcat/webapps/ROOT.war - delete_dir=/usr/local/tomcat/webapps/ROOT Solr: image: solr:latest mem_min: 300m host: host1 publish_all: false plugins: - !plugin id: doX8s restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/names.zip Mongo: image: mongo:latest host: host1 mem_min: 400m
HTTP-LB: image: httpd:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: uazUi restart: true lifecycle: on_create, post_scale_out:AppServer, post_scale_in:AppServer arguments: # Use container_private_ip if you're using Docker networking - BalancerMembers=BalancerMemberhttp://{{AppServer | container_private_ip}}:8080 # Use container_hostname if you're using Weave networking #- BalancerMembers=BalancerMember http://{{AppServer | container_hostname}}:8080 AppServer: image: tomcat:8.0.21-jre8 mem_min: 600m host: host1 cluster_size: 1 environment: - mongo_url={{Mongo|container_private_ip}}:27017/dchq - solr_host={{Solr|container_private_ip}} - solr_port=8983 plugins: - !plugin id: oncXN restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/dbconnect.war - dir=/usr/local/tomcat/webapps/ROOT.war - delete_dir=/usr/local/tomcat/webapps/ROOT Solr: image: solr:latest mem_min: 300m host: host1 publish_all: false plugins: - !plugin id: doX8s restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/names.zip Mongo: image: mongo:latest host: host1 mem_min: 400m
Nginx: image: nginx:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: 0H1Nk restart: true lifecycle: on_create, post_scale_out:AppServer, post_scale_in:AppServer arguments: # Use container_private_ip if you're using Docker networking - servers=server {{AppServer | container_private_ip}}:8080; # Use container_hostname if you're using Weave networking #- servers=server {{AppServer | container_hostname}}:8080; AppServer: image: tomcat:8.0.21-jre8 mem_min: 600m host: host1 cluster_size: 1 environment: - database_driverClassName=com.mysql.jdbc.Driver - database_url=jdbc:mysql://{{MySQL|container_hostname}}:3306/{{MySQL|MYSQL_DATABASE}} - database_username={{MySQL|MYSQL_USER}} - database_password={{MySQL|MYSQL_ROOT_PASSWORD}} - solr_host={{Solr|container_private_ip}} - solr_port=8983 plugins: - !plugin id: oncXN restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/dbconnect.war - dir=/usr/local/tomcat/webapps/ROOT.war - delete_dir=/usr/local/tomcat/webapps/ROOT Solr: image: solr:latest mem_min: 300m host: host1 publish_all: false plugins: - !plugin id: doX8s restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/names.zip MySQL: image: mysql:latest host: host1 mem_min: 400m environment: - MYSQL_USER=root - MYSQL_DATABASE=names - MYSQL_ROOT_PASSWORD={{alphanumeric|8}}
Nginx: image: nginx:latest publish_all: true mem_min: 50m host: host1 plugins: - !plugin id: 0H1Nk restart: true lifecycle: on_create, post_scale_out:AppServer, post_scale_in:AppServer arguments: # Use container_private_ip if you're using Docker networking - servers=server {{AppServer | container_private_ip}}:8080; # Use container_hostname if you're using Weave networking #- servers=server {{AppServer | container_hostname}}:8080; AppServer: image: tomcat:8.0.21-jre8 mem_min: 600m host: host1 cluster_size: 1 environment: - database_driverClassName=com.mysql.jdbc.Driver - database_url=jdbc:mysql://{{MySQL|container_hostname}}:3306/{{MySQL|MYSQL_DATABASE}} - database_username={{MySQL|MYSQL_USER}} - database_password={{MySQL|MYSQL_ROOT_PASSWORD}} - solr_host={{Solr|container_private_ip}} - solr_port=8983 plugins: - !plugin id: oncXN restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/dbconnect.war - dir=/usr/local/tomcat/webapps/ROOT.war - delete_dir=/usr/local/tomcat/webapps/ROOT Solr: image: solr:latest mem_min: 300m host: host1 publish_all: false plugins: - !plugin id: doX8s restart: true arguments: - file_url=https://github.com/dchqinc/dchq-docker-java-solr-mongo-cassandra-example/raw/master/names.zip MySQL: image: mysql:latest host: host1 mem_min: 400m environment: - MYSQL_USER=root - MYSQL_DATABASE=names - MYSQL_ROOT_PASSWORD={{alphanumeric|8}}
mongo_rs1_srv1: image: mongo:latest mem_min: 400m publish_all: true command: --replSet rs1 environment: # inject clustered node ips - MONGO_RS1_SRV2_IP={{mongo_rs1_srv2|container_private_ip}} # The above mapping will insert comma separated ip list e.g. 10.1.1.1,10.1.12 plugins: - !plugin # This plugin initializes the Replica Set lifecycle: on_create id: QX25F - !plugin # This plug-in re-balances the Replica Set post scale out lifecycle: post_scale_out:mongo_rs1_srv2 id: sxKM9 - !plugin # This plug-in re-balances the Replica Set post scale in lifecycle: post_scale_in:mongo_rs1_srv2 id: YIALH # Define this node as clustered mongo_rs1_srv2: image: mongo:latest mem_min: 400m cpu_shares: 1 cluster_size: 2 command: --replSet rs1
mongo_rs1_srv1: image: mongo:latest mem_min: 400m publish_all: true command: --replSetrs1 environment: # inject clustered node ips - MONGO_RS1_SRV2_IP={{mongo_rs1_srv2|container_private_ip}} # The above mapping will insert comma separated ip list e.g. 10.1.1.1,10.1.12 plugins: - !plugin # This plugin initializes the Replica Set lifecycle: on_create id: QX25F - !plugin # This plug-in re-balances the Replica Set post scale out lifecycle: post_scale_out:mongo_rs1_srv2 id: sxKM9 - !plugin # This plug-in re-balances the Replica Set post scale in lifecycle: post_scale_in:mongo_rs1_srv2 id: YIALH # Define this node as clustered mongo_rs1_srv2: image: mongo:latest mem_min: 400m cpu_shares: 1 cluster_size: 2 command: --replSetrs1
Hazelcast-Management-Center: image: hazelcast/management-center:latest # Use this configuration if deploying to a shared VM publish_all: true # (Recommended) Use this configuration if deploying to a dedicated VM #ports: # - 8080:8080 host: host1 environment: - HAZELCAST_IP={{Hazelcast | container_private_ip}} Hazelcast: image: hazelcast/hazelcast:latest # Use this configuration if deploying to a shared VM publish_all: true # (Recommended) Use this configuration if deploying to a dedicated VM #ports: # - 5701:5701 cluster_size: 1 host: host1 plugins: - !plugin id: Qgp4H lifecycle: post_create, post_scale_out:Hazelcast, post_scale_in:Hazelcast restart: true arguments: # Use this configuration if deploying to a shared VM - Hazelcast_IP=<member>{{Hazelcast | container_private_ip}}</member> # (Recommended) Use this configuration if deplying to a dedicated VM #- Hazelcast_IP=<member>{{Hazelcast | ip}}</member> - Management_Center_URL=http://{{Hazelcast-Management-Center | ip}}:{{Hazelcast-Management-Center | port_8080}}/mancenter environment: # Uncomment the line below to specify the heap size #- MIN_HEAP_SIZE="1g" # Uncomment the line below to provide your own hazelcast.xml file - JAVA_OPTS=-Dhazelcast.config=/opt/hazelcast/hazelcast.xml volumes: # Uncomment the line below if you plan to use your own hazelcast.xml file #- ./configFolder:./configFolder
Hazelcast-Management-Center: image: hazelcast/management-center:latest # Use this configuration if deploying to a shared VM publish_all: true # (Recommended) Use this configuration if deploying to a dedicated VM #ports: # - 8080:8080 host: host1 environment: - HAZELCAST_IP={{Hazelcast | container_private_ip}} Hazelcast: image: hazelcast/hazelcast:latest # Use this configuration if deploying to a shared VM publish_all: true # (Recommended) Use this configuration if deploying to a dedicated VM #ports: # - 5701:5701 cluster_size: 1 host: host1 plugins: - !plugin id: Qgp4H lifecycle: post_create, post_scale_out:Hazelcast, post_scale_in:Hazelcast restart: true arguments: # Use this configuration if deploying to a shared VM - Hazelcast_IP=<member>{{Hazelcast | container_private_ip}}</member> # (Recommended) Use this configuration if deplying to a dedicated VM #- Hazelcast_IP=<member>{{Hazelcast | ip}}</member> - Management_Center_URL=http://{{Hazelcast-Management-Center | ip}}:{{Hazelcast-Management-Center | port_8080}}/mancenter environment: # Uncomment the line below to specify the heap size #- MIN_HEAP_SIZE="1g" # Uncomment the line below to provide your own hazelcast.xml file - JAVA_OPTS=-Dhazelcast.config=/opt/hazelcast/hazelcast.xml volumes: # Uncomment the line below if you plan to use your own hazelcast.xml file #- ./configFolder:./configFolder
一旦应用程序被保存,用户可以注册一个云服务提供商自动Docker集群的配置和自动缩放的功能。在不止18种云服务上,包括VMware vSphere, OpenStack, CloudStack, Amazon Web Services, Rackspace, Microsoft Azure, DigitalOcean, IBM SoftLayer, Google Compute Engine等
这里有一些主流云平台的资料:
英文原文: https://www.voxxed.com/blog/2016/01/docker-service-discovery-microsoft-azure/?ref=dzone
微服务实践——Docker与服务发现