作者:青木,工程师,DevOps践行者,微服务化,容器化业务实践者。
在 之前文章 中( 点击查看 ),我们搭建好了基本的网关的架子。这篇文章中
我们继续剩下的部分,将网关和部署到k8s中,并结合Ingress完成域名访问我们的网关,通过Secret完成网关HTTPS的安全加密。
我们可以使用阿里云的SLB作为IP入口,可以将域名直接用*.xxx.com解析到该IP上。
IngressController采用DS+HostNetwork方式运行,管理所有的二级域名,以及HTTPS。
最后层就是我们的Gateway了,负责自动发现我们的业务服务,并通过Ribbon进行LB。
SLB访问Ingress走HostNetwork,Ingress访问Gateway走IPVS,Gateway访问业务服务走pod网络。
这里我们直接使用maven的docker插件完成一下流程.
<span style="font-size: 15px;">buildFatjar --> buildDockerImage --> pushDockerImage</span>
一个完整的maven插件配置如下。这个插件在我们前面的文章中已经配置好了,关于这个插件相关的详细你可以参考我的这篇文章 https://qingmu.io/2018/08/07/How-to-run-springcloud-in-docker/
<span style="font-size: 15px;"><plugin></span>
<span style="font-size: 15px;"><groupId>com.spotify</groupId></span>
<span style="font-size: 15px;"><artifactId>docker-maven-plugin</artifactId></span>
<span style="font-size: 15px;"><configuration></span>
<span style="font-size: 15px;"><imageName></span>
<span style="font-size: 15px;"> freemanliu/demo-gatway:v1.0.0</span>
<span style="font-size: 15px;"></imageName></span>
<span style="font-size: 15px;"><registryUrl></registryUrl></span>
<span style="font-size: 15px;"><workdir>/work</workdir></span>
<span style="font-size: 15px;"><rm>true</rm></span>
<span style="font-size: 15px;"><env></span>
<span style="font-size: 15px;"><TZ>Asia/Shanghai</TZ></span>
<span style="font-size: 15px;"><JAVA_OPTS></span>
<span style="font-size: 15px;"> -XX:+UnlockExperimentalVMOptions /</span>
<span style="font-size: 15px;"> -XX:+UseCGroupMemoryLimitForHeap /</span>
<span style="font-size: 15px;"> -XX:MaxRAMFraction=2 /</span>
<span style="font-size: 15px;"> -XX:CICompilerCount=8 /</span>
<span style="font-size: 15px;"> -XX:ActiveProcessorCount=8 /</span>
<span style="font-size: 15px;"> -XX:+UseG1GC /</span>
<span style="font-size: 15px;"> -XX:+AggressiveOpts /</span>
<span style="font-size: 15px;"> -XX:+UseFastAccessorMethods /</span>
<span style="font-size: 15px;"> -XX:+UseStringDeduplication /</span>
<span style="font-size: 15px;"> -XX:+UseCompressedOops /</span>
<span style="font-size: 15px;"> -XX:+OptimizeStringConcat</span>
<span style="font-size: 15px;"></JAVA_OPTS></span>
<span style="font-size: 15px;"></env></span>
<span style="font-size: 15px;"><baseImage>freemanliu/openjre:8.212</baseImage></span>
<span style="font-size: 15px;"><cmd></span>
<span style="font-size: 15px;"> /sbin/tini java ${JAVA_OPTS} -jar ${project.build.finalName}.jar</span>
<span style="font-size: 15px;"></cmd></span>
<span style="font-size: 15px;"><!--是否推送image--></span>
<span style="font-size: 15px;"><pushImage>true</pushImage></span>
<span style="font-size: 15px;"><resources></span>
<span style="font-size: 15px;"><resource></span>
<span style="font-size: 15px;"><directory>${project.build.directory}</directory></span>
<span style="font-size: 15px;"><include>${project.build.finalName}.jar</include></span>
<span style="font-size: 15px;"></resource></span>
<span style="font-size: 15px;"></resources></span>
<span style="font-size: 15px;"><serverId>docker-hub</serverId></span>
<span style="font-size: 15px;"></configuration></span>
<span style="font-size: 15px;"><executions></span>
<span style="font-size: 15px;"><execution></span>
<span style="font-size: 15px;"><phase>package</phase></span>
<span style="font-size: 15px;"><goals></span>
<span style="font-size: 15px;"><goal>build</goal></span>
<span style="font-size: 15px;"></goals></span>
<span style="font-size: 15px;"></execution></span>
<span style="font-size: 15px;"></executions></span>
<span style="font-size: 15px;"></plugin></span>
配置好以上插件之后我们执行以下命令.
$ mvn clean package -DskipTests[INFO] [INFO] -----------------------< io.qingmu:demo-gateway >-----------------------[INFO] Building demo-gateway 0.0.1-SNAPSHOT[INFO] --------------------------------[ jar ]---------------------------------[INFO] [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ demo-gateway ---[INFO] Deleting /Users/freeman/GithubProjects/demo-gateway/target[INFO] [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ demo-gateway ---[INFO] Using 'UTF-8' encoding to copy filtered resources.[INFO] Copying 1 resource[INFO] Copying 2 resources[INFO] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ demo-gateway ---[INFO] Changes detected - recompiling the module![INFO] Compiling 6 source files to /Users/freeman/GithubProjects/demo-gateway/target/classes[INFO] [INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ demo-gateway ---[INFO] Using 'UTF-8' encoding to copy filtered resources.[INFO] skip non existing resourceDirectory /Users/freeman/GithubProjects/demo-gateway/src/test/resources[INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ demo-gateway ---[INFO] Changes detected - recompiling the module![INFO] Compiling 1 source file to /Users/freeman/GithubProjects/demo-gateway/target/test-classes[INFO] [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ demo-gateway ---[INFO] Tests are skipped.[INFO] [INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ demo-gateway ---[INFO] Building jar: /Users/freeman/GithubProjects/demo-gateway/target/demo-gateway-0.0.1-SNAPSHOT.jar[INFO] [INFO] --- spring-boot-maven-plugin:2.1.7.RELEASE:repackage (repackage) @ demo-gateway ---[INFO] Replacing main artifact with repackaged archive[INFO] [INFO] --- docker-maven-plugin:1.2.1:build (default) @ demo-gateway ---[INFO] Using authentication suppliers: [ConfigFileRegistryAuthSupplier, FixedRegistryAuthSupplier][INFO] Copying /Users/freeman/GithubProjects/demo-gateway/target/demo-gateway-0.0.1-SNAPSHOT.jar -> /Users/freeman/GithubProjects/demo-gateway/target/docker/demo-gateway-0.0.1-SNAPSHOT.jar[INFO] Building image freemanliu/demo-gateway:v1.0.0Step 1/6 : FROM freemanliu/openjre:8.212 ---> 8a33da3f701eStep 2/6 : ENV JAVA_OPTS -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=2 -XX:CICompilerCount=8 -XX:ActiveProcessorCount=8 -XX:+UseG1GC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+UseStringDeduplication -XX:+UseCompressedOops -XX:+OptimizeStringConcat ---> Using cache ---> 0f65d8120eb4Step 3/6 : ENV TZ Asia/Shanghai ---> Using cache ---> 698b12f89dd2Step 4/6 : WORKDIR /work ---> Using cache ---> 45e2bbd5eb93Step 5/6 : ADD demo-gateway-0.0.1-SNAPSHOT.jar . ---> 0a76e61162bbStep 6/6 : CMD /sbin/tini java ${JAVA_OPTS} -jar demo-gateway-0.0.1-SNAPSHOT.jar ---> Running in 034aee2b1d56Removing intermediate container 034aee2b1d56 ---> 19591ef9767bProgressMessage{id=null, status=null, stream=null, error=null, progress=null, progressDetail=null}Successfully built 19591ef9767bSuccessfully tagged freemanliu/demo-gateway:v1.0.0[INFO] Built freemanliu/demo-gateway:v1.0.0[INFO] Pushing freemanliu/demo-gateway:v1.0.0The push refers to repository [docker.io/freemanliu/demo-gateway]854550ddc678: Pushed 607d6bc26ce8: Pushed e02c5cc56276: Pushed f1b5933fe4b5: Layer already exists v1.0.0: digest: sha256:bcc45e9b9edde415523c7e742df55486eb9d109c6864729fdf68eeefd13b3b9e size: 1158null: null[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESS[INFO] ------------------------------------------------------------------------[INFO] Total time: 59.288 s[INFO] Finished at: 2019-09-15T20:26:42+08:00[INFO] ------------------------------------------------------------------------
完整的执行完打包命令之后,我们的image就被推送到了docker的中央仓库
完成了我们的前置工作,我们就可以编写我们的yaml描述文件
<span style="font-size: 15px;">apiVersion: v1</span>
<span style="font-size: 15px;">kind: Service</span>
<span style="font-size: 15px;">metadata:</span>
<span style="font-size: 15px;">name: demo-gateway</span>
<span style="font-size: 15px;">namespace: default</span>
<span style="font-size: 15px;">labels:</span>
<span style="font-size: 15px;">app: demo-gateway</span>
<span style="font-size: 15px;">spec:</span>
<span style="font-size: 15px;">ports:</span>
<span style="font-size: 15px;">- port: 8084</span>
<span style="font-size: 15px;">name: tcp</span>
<span style="font-size: 15px;">selector:</span>
<span style="font-size: 15px;">app: demo-gateway</span>
<span style="font-size: 15px;">---</span>
<span style="font-size: 15px;">apiVersion: apps/v1</span>
<span style="font-size: 15px;">kind: Deployment</span>
<span style="font-size: 15px;">metadata:</span>
<span style="font-size: 15px;">name: demo-gateway</span>
<span style="font-size: 15px;">namespace: default</span>
<span style="font-size: 15px;">spec:</span>
<span style="font-size: 15px;">revisionHistoryLimit: 10</span>
<span style="font-size: 15px;">strategy:</span>
<span style="font-size: 15px;">type: RollingUpdate</span>
<span style="font-size: 15px;">rollingUpdate:</span>
<span style="font-size: 15px;">maxUnavailable: 25%</span>
<span style="font-size: 15px;">maxSurge: 25%</span>
<span style="font-size: 15px;">replicas: 1</span>
<span style="font-size: 15px;">selector:</span>
<span style="font-size: 15px;">matchLabels:</span>
<span style="font-size: 15px;">app: demo-gateway</span>
<span style="font-size: 15px;">template:</span>
<span style="font-size: 15px;">metadata:</span>
<span style="font-size: 15px;">labels:</span>
<span style="font-size: 15px;">app: demo-gateway</span>
<span style="font-size: 15px;">spec:</span>
<span style="font-size: 15px;">affinity:</span>
<span style="font-size: 15px;">podAntiAffinity:</span>
<span style="font-size: 15px;">preferredDuringSchedulingIgnoredDuringExecution:</span>
<span style="font-size: 15px;">- podAffinityTerm:</span>
<span style="font-size: 15px;">topologyKey: kubernetes.io/hostname</span>
<span style="font-size: 15px;">labelSelector:</span>
<span style="font-size: 15px;">matchExpressions:</span>
<span style="font-size: 15px;">- key: app</span>
<span style="font-size: 15px;">operator: In</span>
<span style="font-size: 15px;">values:</span>
<span style="font-size: 15px;">- app-gateway # 反亲和,尽量让gateway分散到不同的节点上</span>
<span style="font-size: 15px;">weight: 1</span>
<span style="font-size: 15px;">containers:</span>
<span style="font-size: 15px;">- name: demo-gateway</span>
<span style="font-size: 15px;">image: freemanliu/demo-gateway:v1.0.0</span>
<span style="font-size: 15px;">imagePullPolicy: Always</span>
<span style="font-size: 15px;">lifecycle:</span>
<span style="font-size: 15px;">preStop:</span>
<span style="font-size: 15px;">httpGet:</span>
<span style="font-size: 15px;">port: 8084</span>
<span style="font-size: 15px;">path: /spring/shutdown</span>
<span style="font-size: 15px;">livenessProbe:</span>
<span style="font-size: 15px;">httpGet:</span>
<span style="font-size: 15px;">path: /actuator/health</span>
<span style="font-size: 15px;">port: 8084</span>
<span style="font-size: 15px;">periodSeconds: 5</span>
<span style="font-size: 15px;">timeoutSeconds: 10</span>
<span style="font-size: 15px;">successThreshold: 1</span>
<span style="font-size: 15px;">failureThreshold: 5</span>
<span style="font-size: 15px;">readinessProbe:</span>
<span style="font-size: 15px;">httpGet:</span>
<span style="font-size: 15px;">path: /actuator/health</span>
<span style="font-size: 15px;">port: 8084</span>
<span style="font-size: 15px;">periodSeconds: 5</span>
<span style="font-size: 15px;">timeoutSeconds: 10</span>
<span style="font-size: 15px;">successThreshold: 1</span>
<span style="font-size: 15px;">failureThreshold: 5</span>
<span style="font-size: 15px;">resources:</span>
<span style="font-size: 15px;">requests:</span>
<span style="font-size: 15px;">memory: 1Gi</span>
<span style="font-size: 15px;">limits:</span>
<span style="font-size: 15px;">memory: 1Gi</span>
<span style="font-size: 15px;">ports:</span>
<span style="font-size: 15px;">- containerPort: 8084</span>
<span style="font-size: 15px;">env:</span>
<span style="font-size: 15px;">- name: EUREKA_SERVER</span>
<span style="font-size: 15px;">value: "http://eureka-0.eureka:8761/eureka/,http://eureka-1.eureka:8761/eureka/,http://eureka-2.eureka:8761/eureka/"</span>
<span style="font-size: 15px;">- name: SAMPLER_PROBABILITY</span>
<span style="font-size: 15px;">value: "1.0"</span>
<span style="font-size: 15px;">- name: ZIPKIN</span>
<span style="font-size: 15px;">value: "http://10.96.0.13:9411/"</span>
<span style="font-size: 15px;">$ kubectl apply -f demo.gateway.deployment.yaml</span>
<span style="font-size: 15px;">$ kubectl get po -owide | grep demo-gateway</span>
<span style="font-size: 15px;">demo-gateway-c94bc5679-8ld2v 1/1 Running 0 3m35s 172.224.3.23 node1 <none> <none></span>
$ curl -i 172.224.3.23:8084/demo2-service/worldHTTP/1.1 200 OKContent-Type: text/plain;charset=UTF-8Content-Length: 5Date: Tue, 17 Sep 2019 06:53:46 GMT
<span style="font-size: 15px;">$ kubectl get svc | grep demo-gateway</span>
<span style="font-size: 15px;">demo-gateway ClusterIP 10.111.213.174 <none> 8084/TCP 5m10s</span>
<span style="font-size: 15px;">$ curl -i 10.111.213.174:8084/demo2-service/world</span>
<span style="font-size: 15px;">HTTP/1.1 200 OK</span>
<span style="font-size: 15px;">Content-Type: text/plain;charset=UTF-8</span>
<span style="font-size: 15px;">Content-Length: 5</span>
<span style="font-size: 15px;">Date: Tue, 17 Sep 2019 06:55:25 GMT</span>
到此我们就将完整的demo部署到了kubernetes中。接下来我们处理ingress,以支持通过域名和https访问我们的服务。
这里我们使用域名gateway.qingmu.io访问我们的服务。首先需要申请一个HTTPS证书,自己测试使用可以在阿里云的证书服务中申请一个为期一年的,生产使用建议购买一个通配证书。
在kubernetes中专门抽象了一个资源类型叫Secret,专门用户存储我们的上传的信息,存在这里的信息,将会被我们的ingress-controller使用,并自动配置https。
我们申请完的证书,选择下载nginx,下载之后的zip包中解压开会有两个文件2232293__gateway.qingmu.io.key,2232293__gateway.qingmu.io.pem
<span style="font-size: 15px;">2822276_gateway.qingmu.io.key 2822276_gateway.qingmu.io.pem</span>
<span><br /></span>
这里-n代表存储在哪个namespace中,这取决于你的ingress存在哪个namespace中,这里我们都是在default中。
<span style="font-size: 15px;">$ kubectl -n default create secret tls qingmu-secret /</span>
<span style="font-size: 15px;">--key 2822276_gateway.qingmu.io.key /</span>
<span style="font-size: 15px;">--cert 2822276_gateway.qingmu.io.pem</span>
<span style="font-size: 15px;">secret/qingmu-secret created</span>
<span style="font-size: 15px;">$ kubectl get secret -ndefault qingmu-secret</span>
<span style="font-size: 15px;">NAME TYPE DATA AGE</span>
<span style="font-size: 15px;">qingmu-secret kubernetes.io/tls 2 84s</span>
<span style="font-size: 15px;">apiVersion: networking.k8s.io/v1beta1</span>
<span style="font-size: 15px;">kind: Ingress</span>
<span style="font-size: 15px;">metadata:</span>
<span style="font-size: 15px;">name: gateway-ingress</span>
<span style="font-size: 15px;">namespace: default</span>
<span style="font-size: 15px;">annotations:</span>
<span style="font-size: 15px;"> # 是否强制重定向到https</span>
<span style="font-size: 15px;">ingress.kubernetes.io/force-ssl-redirect: "true"</span>
<span style="font-size: 15px;">spec:</span>
<span style="font-size: 15px;">tls:</span>
<span style="font-size: 15px;">- hosts:</span>
<span style="font-size: 15px;">- gateway.qingmu.io</span>
<span style="font-size: 15px;"> # 这里需要和我们上面的一致</span>
<span style="font-size: 15px;">secretName: qingmu-secret</span>
<span style="font-size: 15px;">rules:</span>
<span style="font-size: 15px;">- host: gateway.qingmu.io</span>
<span style="font-size: 15px;">http:</span>
<span style="font-size: 15px;">paths:</span>
<span style="font-size: 15px;">- backend:</span>
<span style="font-size: 15px;">serviceName: demo-gateway</span>
<span style="font-size: 15px;">servicePort: 8084</span>
<span style="font-size: 15px;">$ kubectl apply -f demo.gateway.ingress.yaml </span>
<span style="font-size: 15px;">ingress.networking.k8s.io/gateway-ingress created</span>
<span><br /></span>
<span style="font-size: 15px;">$ kubectl get ingress gateway-ingress</span>
<span style="font-size: 15px;">NAME HOSTS ADDRESS PORTS AGE</span>
<span style="font-size: 15px;">gateway-ingress gateway.qingmu.io 80, 443 53s</span>
<span><br /></span>
访问http端口的时候,服务器会返回重定向308(永久重定向)
<span style="font-size: 15px;">$ curl -i gateway.qingmu.io/demo2-service/world</span>
<span style="font-size: 15px;">HTTP/1.1 308 Permanent Redirect</span>
<span style="font-size: 15px;">Server: openresty/1.15.8.1</span>
<span style="font-size: 15px;">Date: Tue, 17 Sep 2019 07:41:08 GMT</span>
<span style="font-size: 15px;">Content-Type: text/html</span>
<span style="font-size: 15px;">Content-Length: 177</span>
<span style="font-size: 15px;">Connection: keep-alive</span>
<span style="font-size: 15px;">Location: https://gateway.qingmu.io/demo2-service/world</span>
<span><br /></span>
<span style="font-size: 15px;"><html></span>
<span style="font-size: 15px;"><head><title>308 Permanent Redirect</title></head></span>
<span style="font-size: 15px;"><body></span>
<span style="font-size: 15px;"><center><h1>308 Permanent Redirect</h1></center></span>
<span style="font-size: 15px;"><hr><center>openresty/1.15.8.1</center></span>
<span style="font-size: 15px;"></body></span>
<span style="font-size: 15px;"></html></span>
<span><br /></span>
<span style="font-size: 15px;">$ curl -i https://gateway.qingmu.io/demo2-service/world</span>
<span style="font-size: 15px;">HTTP/2 200 </span>
<span style="font-size: 15px;">server: openresty/1.15.8.1</span>
<span style="font-size: 15px;">date: Tue, 17 Sep 2019 07:40:06 GMT</span>
<span style="font-size: 15px;">content-type: text/plain;charset=UTF-8</span>
<span style="font-size: 15px;">content-length: 5</span>
<span style="font-size: 15px;">strict-transport-security: max-age=15724800; includeSubDomains</span>
<span><br /></span>
到此我们就完整的spring cloud gateway部署到了kubernetes中。
END
Kubernetes CKA线下班
Kubernetes 线上直播班