微服务架构是一项在云中部署应用和服务的新技术。大部分围绕微服务的争论都集中在容器或其他技术是否能很好的实施微服务,而红帽说API应该是重点。
微服务可以在"自己的程序"中运行,并通过"轻量级设备与HTTP型API进行沟通"。关键在于该服务可以在自己的程序中运行。通过这一点我们就可以将服务公开与微服务架构(在现有系统中分布一个API)区分开来。在服务公开中,许多服务都可以被内部独立进程所限制。如果其中任何一个服务需要增加某种功能,那么就必须缩小进程范围。在微服务架构中,只需要在特定的某种服务中增加所需功能,而不影响整体进程的架构。
这里只做演示,以及在k8s中如何部署微服务,以及实现步骤和思路,希望对你有所帮助
主机使用ip:
192.168.30.21 k8s-master 192.168.30.22 k8s-node1 192.168.30.23 k8s-node2 192.168.30.24 harbor-mysql
Dev1 交付代码
Dev2 编写Dockerfile构建镜像
Dev3 k8s资源编排
Dev4 微服务链路监控
Dev5 新功能测试
Master 最终上线
一、代码编译构建(Maven)
拉取项目代码
代码分支:
[root@k8s-master ~]# git clone -b dev https://github.com/xxxxx/simple-microservice
声明:交付的分支
simple-microservice-dev1
[root@k8s-master ~]# ls anaconda-ks.cfg simple-microservice-dev2 simple-microservice-master 模板 文档 桌面 initial-setup-ks.cfg simple-microservice-dev3 yum.sh 视频 下载 simple-microservice-dev1 simple-microservice-dev4 公共 图片 音乐 编译需要安装jdk , jdk版本要看开发那边使用什么,注意一下。我这里用的1.8.0
用maven去编译项目
Centos7默认有openjdk的包
[root@k8s-master ~]# yum install java-1.8.0-openjdk maven -y 软件包 1:java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64 已安装
并且是最新版本
软件包 maven-3.0.5-17.el7.noarch 已安装并且是最新版本
查看安装的jdk版本
[root@k8s-master ~]# java -version openjdk version "1.8.0_222" OpenJDK Runtime Environment (build 1.8.0_222-b10) OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
查看maven启动文件
[root@k8s-master ~]# mvn mvn mvnDebug mvnyjp
进行编译:
进入源代码里,我们先进入交付的分支,开发工程师原封不动的把源代码交给我们,然后我们运维工程师进行编译构建,把源代码打成jar包
[root@k8s-master ~]# cd simple-microservice-dev1 [root@k8s-master simple-microservice-dev1]# ls basic-common eureka-service LICENSE order-service portal-service README.md db gateway-service lombok.config pom.xml product-service stock-service
Maven项目对象模型(POM),可以通过一小段描述信息来观念里项目的构建,报告和文档的项目管理工具软件
mvn clean package -D maven.test.skip=true -P prod mvn clean package:清除目录中生成的结果,做一个清除,重新打新的包。 -D maven.test.skip: 跳过单元测试,写的测试用例,如果写的有问题,是编译不过去的 -P prod: 使用哪一套配置文件 [root@k8s-master simple-microservice-dev1]# mvn clean package -D maven.test.skip=true
如果代码没问题就会以下面的呈现,有问题就可以去找研发那边
运行很多的输出,构建中会在网上拉取需要的依赖包,
[INFO]
[INFO] --- spring-boot-maven-plugin:2.0.9.RELEASE:repackage (default) @ portal-service ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] simple-microservice ............................... SUCCESS [13.208s]
[INFO] basic-common ...................................... SUCCESS [0.002s]
[INFO] basic-common-core ................................. SUCCESS [6:34.870s]
[INFO] gateway-service ................................... SUCCESS [5:15.194s]
[INFO] eureka-service .................................... SUCCESS [30.681s]
[INFO] product-service ................................... SUCCESS [0.014s]
[INFO] product-service-api ............................... SUCCESS [0.677s]
[INFO] stock-service ..................................... SUCCESS [0.001s]
[INFO] stock-service-api ................................. SUCCESS [0.545s]
[INFO] product-service-biz ............................... SUCCESS [10.321s]
[INFO] stock-service-biz ................................. SUCCESS [1.787s]
[INFO] order-service ..................................... SUCCESS [0.002s]
[INFO] order-service-api ................................. SUCCESS [0.697s]
[INFO] order-service-biz ................................. SUCCESS [2.029s]
[INFO] basic-common-bom .................................. SUCCESS [0.001s]
[INFO] portal-service .................................... SUCCESS [2.454s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:16.229s
[INFO] Finished at: Fri Aug 02 09:35:01 CST 2019
[INFO] Final Memory: 78M/392M
[INFO] ------------------------------------------------------------------------
构建完成会多出一个target,在源码中是没有这个target
[root@k8s-master simple-microservice-dev1]# ls basic-common eureka-service LICENSE order-service portal-service README.md db gateway-service lombok.config pom.xml product-service stock-service [root@k8s-master simple-microservice-dev1]# ls gateway-service/ pom.xml src target
order-service-api:是相关调用; order-service-biz:具体是我们部署的
这几个里面都有
[root@k8s-master simple-microservice-dev1]# ls order-service/ order-service-api order-service-biz pom.xml [root@k8s-master simple-microservice-dev1]# ls product-service/ pom.xml product-service-api product-service-biz [root@k8s-master simple-microservice-dev1]# ls stock-service/ pom.xml stock-service-api stock-service-biz
查看order-service/order-service-biz/里面有这个target;
[root@k8s-master simple-microservice-dev1]# ls pom.xml src target
target里面有一个jar包,这个就是我们部署的包,打包到镜像中部署到k8s中,其他目录页一样都这个jar包,传统项目也会生成target,目录下有可部署的jar包
[root@k8s-master simple-microservice-dev1]# ls order-service/order-service-biz/target/ classes maven-archiver order-service-biz.jar generated-sources maven-status order-service-biz.jar.original
二、构建项目镜像并推送到镜像仓库
构建镜像使用Docker和结合Dockerfile
镜像三类:
1、基础镜像,基本操作系统
2、运行环境,java就会有tomcat
3、项目镜像,应用程序和应用程序环境,最终的镜像,部署到任何docker上或者K8s中
将我们构建完的jar包,打包到镜像中
将我们的Dockerfile放入dev1分支中,这个我没放Dockerfile,现在把dev2分支的拿过来
[root@k8s-master ~]# cd simple-microservice-dev1 [root@k8s-master simple-microservice-dev1]# cd gateway-service/ [root@k8s-master gateway-service]# cp /root/simple-microservice-dev2/gateway-service/Dockerfile ./ [root@k8s-master gateway-service]# ls Dockerfile pom.xml src target
Dockerfile声明:
java:8-jdk-alpine:这里使用的是一个alpine的小Linux命令操作系统的镜像,大约300条命令,以便测试构建用
apk add -U tzdata:这是一个时区,默认jdk中没有时区,然后通过软连接,放到系统里面
./target/gateway-service.jar ./:拷贝dev1我们分支下target/gateway-service.jar包放到根目录下
EXPOSE 9999:暴露端口:9999
CMD java -jar /gateway-service.jar:启动镜像
[root@k8s-master gateway-service]# vim Dockerfile FROM java:8-jdk-alpine LABEL maintainer xxx/www.ctnrs.com RUN apk add -U tzdata && / ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime COPY ./target/gateway-service.jar ./ EXPOSE 9999 CMD java -jar /gateway-service.jar
使用docker命令构建镜像:安装docker,这里不做演示了
[root@k8s-master gateway-service]# docker build -t gateway .
查看我们构建的镜像
[root@k8s-master gateway-service]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE gateway latest c9429e09fe42 34 seconds ago 191MB java 8-jdk-alpine 3fd9dd82815c 2 years ago 145MB
然后开始将我们构建好的镜像推送到我们的镜像仓库,这里我用的是Harbor,安装步骤这里不演示了
创建一个microservice的项目名称
打个标签,安装上面的格式,把我们的gateway打个标签
[root@k8s-master gateway-service]# docker tag gateway 192.168.30.24/microservice/gateway:latest
如果不是https的Harbor需要在docker里面添加信任才能访问到镜像
[root@k8s-master ~]# vim /etc/docker/daemon.json { "registry-mirrors": ["http://f1361db2.m.daocloud.io"], "insecure-registries": ["192.168.30.24"] }
测试登录,因为我在页面上登录了,所有显示已经登录了
[root@k8s-master ~]# docker login 192.168.30.24 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded
推送镜像到Harbor仓库中
[root@k8s-master ~]# docker push 192.168.30.24/microservice/gateway:latest The push refers to repository [192.168.30.24/microservice/gateway] 6912928d45d8: Pushed e7917d042778: Pushed a1e7033f082e: Pushed 78075328e0da: Pushed 9f8566ee5135: Pushed latest: digest:
查看我们的Harbor仓库已经上传成功
接下来就是部署k8s,然后将spring cloud放入我们k8s中
这里需要注意的是,用二进制或者kubeadm都可以
K8s平台环境的要求如下:
跨主机网络:使用flannel或者Calico ,需要网络来打通主机之间资源的通信
CoreDNS: k8s内部的DNS ,用于对pod对service做记录的,好让其他的pod做访问
Harbor镜像仓库:这个我们已经准备好了,并将项目镜像推送上去了
Ingress Controller:同一暴露我们的应用,写yaml文件实现
K8s部署这里不做演示了,flannel也已经部署好了,k8s-master节点部署coredns和ingress-nginx 也部署好了
这里portal门户网站,前端,用户访问www.baidu.com的页面,通过域名访问之后,进行的一个页面展示,我们我们通过pod来进行实现,拿ingress来定义我们的域名,域名定义哪个service,来定义到某个pod上,来影响静态页面,下订单请求交给网关api,采用异步调用,
暴露网关,进行来用户访问,ingress也来调用,来service来实现pod副本
gateway网关,通过一些前端页面的页面功能,同gateway来调用实现,用户点击某个功能gateway拿到这个请求之后,通过路由转发规则,到后端的业务程序,比如商品信息(product)
库存,订单,他会根据不同的业务需要来处理,库存服务会根据订单的使用来和内部的调用接口来实现,pod直接调用,需要跨主机网络,怎么找到这个服务,就需要这个注册中心,谁找谁就需要这个注册中心,所有的服务都会放在这里,来进行消息通信,现在比较流行的就是erueka,订单服务都会放入到我们的mysql数据库中的,mysql是部署在外部的,有状态的,这个部署在k8s中是比较麻烦大的,erueka是部署在k8s集群内的,只需要保证他的id是唯一性就可以了,不需要考虑他的存储。
三、spring cloud部署到k8s集群中
1、服务编排
2、在k8s平台部署Erueka
3、导入数据库文件到Mysql
4、部署网关gateway
5、部署业务程序(product、stock、order)
6、部署前端(portal)
编译打成jar包,在dev3分支
创建命名空间
kubectl create ns ms [root@k8s-master simple-microservice-dev3]# mvn clean package -D maven.test.skip=true
[INFO] --- spring-boot-maven-plugin:2.0.9.RELEASE:repackage (default) @ portal-service ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] simple-microservice ............................... SUCCESS [0.732s]
[INFO] basic-common ...................................... SUCCESS [0.002s]
[INFO] basic-common-core ................................. SUCCESS [41.516s]
[INFO] gateway-service ................................... SUCCESS [34.127s]
[INFO] eureka-service .................................... SUCCESS [8.062s]
[INFO] product-service ................................... SUCCESS [0.001s]
[INFO] product-service-api ............................... SUCCESS [3.029s]
[INFO] stock-service ..................................... SUCCESS [0.001s]
[INFO] stock-service-api ................................. SUCCESS [0.371s]
[INFO] product-service-biz ............................... SUCCESS [6.170s]
[INFO] stock-service-biz ................................. SUCCESS [10.095s]
[INFO] order-service ..................................... SUCCESS [0.002s]
[INFO] order-service-api ................................. SUCCESS [0.372s]
[INFO] order-service-biz ................................. SUCCESS [4.396s]
[INFO] basic-common-bom .................................. SUCCESS [0.001s]
[INFO] portal-service .................................... SUCCESS [4.379s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:05.137s
[INFO] Finished at: Wed Aug 07 13:38:55 CST 2019
[INFO] Final Memory: 70M/391M
[INFO] ------------------------------------------------------------------------
查看生成的jar包
[root@k8s-master simple-microservice-dev3]# cd eureka-service/ [root@k8s-master eureka-service]# ls Dockerfile pom.xml src target [root@k8s-master eureka-service]# cd target/ [root@k8s-master target]# ls classes eureka-service.jar.original maven-archiver eureka-service.jar generated-sources maven-status [root@k8s-master eureka-service]# ls Dockerfile pom.xml src target
查看dockerfile
[root@k8s-master eureka-service]# cat Dockerfile FROM java:8-jdk-alpine LABEL maintainer lizhenliang/www.ctnrs.com RUN apk add -U tzdata && / ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime COPY ./target/eureka-service.jar ./ EXPOSE 8888 CMD java -jar -Deureka.instance.hostname=${MY_POD_NAME}.eureka.ms /eureka-service.jar
构建镜像
[root@k8s-master eureka-service]# docker build -t eureka . [root@k8s-master eureka-service]# docker tag eureka 192.168.30.24/microservice/eureka:latest 上传到我们的harbor上 [root@k8s-master eureka-service]# docker push 192.168.30.24/microservice/eureka:latest The push refers to repository [192.168.30.24/microservice/eureka] d0a47606affb: Layer already exists d37aee246f9e: Layer already exists a1e7033f082e: Layer already exists 78075328e0da: Layer already exists 9f8566ee5135: Layer already exists
修改我们的euerka的yaml,这样他就能调用我们的eureka.yaml的标签了
[root@k8s-master ~]# vim simple-microservice-dev3/k8s/eureka.yaml spec: imagePullSecrets: - name: registry-pull-secret containers: - name: eureka image: 192.168.30.24/microservice/eureka:latest ports: [root@k8s-master ~]# cd simple-microservice-dev3 [root@k8s-master simple-microservice-dev3]# cd k8s/
创建k8s登录harbor信息认证
[root@k8s-master k8s]# kubectl create secret docker-registry registry-pull-secret --docker-server=192.168.30.24 --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@ctnrs.com -n ms
修改register-with-eureka: true为true
[root@k8s-master resources]# vim application-fat.yml register-with-eureka: true serviceUrl: defaultZone: http://eureka-0.eureka.ms:${server.port}/eureka/,http://eureka-1.eureka.ms:${server.port}/eureka/,http://eureka-2.eureka.ms:${server.port}/eureka/ fetch-registry: true
创建eurekayaml
[root@k8s-master k8s]# kubectl create -f eureka.yaml ingress.extensions/eureka created service/eureka created statefulset.apps/eureka created
查看详细信息,这里有可能会报错,kubelet node节点登录harbor需要认证这里在daemon.json里面写进去就可以了
查看日志信息
[root@k8s-master k8s]# kubectl describe pod eureka-0 -n ms
查看状态
[root@k8s-master k8s]# kubectl get pod eureka-0 -n ms NAME READY STATUS RESTARTS AGE eureka-0 1/1 Running 1 4m18s [root@k8s-master ~]# kubectl get pod,svc,ing -n ms NAME READY STATUS RESTARTS AGE pod/eureka-0 1/1 Running 1 16h pod/eureka-1 1/1 Running 0 16h pod/eureka-2 1/1 Running 1 16h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/eureka ClusterIP None <none> 8888/TCP 16h NAME HOSTS ADDRESS PORTS AGE ingress.extensions/eureka eureka.ctnrs.com 80 16h
将域名写入本机windows的hosts文件里面进行ingress的访问
1、打开路径C:/Windows/System32/drivers/etc。
2、将hosts文件拖拽到桌面上。
3、以笔记本程序打开,完成修改后保存。
4、修改完成后重新拖回etc目录下完成修改。
192.168.30.23 eureka.ctnrs.com portal.ctnrs.com gateway.ctnrs.com
这里确保ingress和coredns集群状态正常,当我们去访问eureka.ctnrs.com的时候就可以正常访问的我们的页面,按F5刷新就可以切换不同的节点上
然后说一下用云服务器或者服务器来怎么设置域名访问
这里也是一样,注意的是写外网ip,这是访问外部的
导入数据库到Mysql
[root@k8s-master1 ~]# cd simple-microservice-dev3 [root@k8s-master1 simple-microservice-dev3]# ls basic-common gateway-service lombok.config portal-service stock-service db k8s order-service product-service eureka-service LICENSE pom.xml README.md [root@k8s-master1 simple-microservice-dev3]# cd db/ [root@k8s-master1 db]# ls order.sql product.sql stock.sql [root@k8s-master1 db]# scp * root@192.168.30.24:~
这里我安装了一个mariadb的实例进行测试
[root@harbor-mysql ~]# yum install mariadb mariadb-server mariadb-devel [root@harbor-mysql ~]# systemctl start mariadb [root@harbor-mysql ~]# netstat -anpt |grep 3306 tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 6226/mysqld
修改mariadb的密码,默认没有密码
[root@harbor-mysql ~]# mysql -u root -p MariaDB [(none)]> set password for root@localhost = password('666666'); MariaDB [(none)]> flush privileges;
创建数据库
MariaDB [(none)]> create database tb_order; MariaDB [(none)]> create database tb_product; MariaDB [(none)]> create database tb_stock;
进入order数据库,把我们根目录下的sql语句导入我们的数据库中
MariaDB [(none)]> use tb_order; MariaDB [tb_order]> source /root/order.sql;
进入product数据库,把我们根目录下的sql语句导入我们的数据库中
MariaDB [tb_order]> use tb_product ; MariaDB [tb_product]> source /root/product.sql
进入stock数据库,把我们根目录下的sql语句导入我们的数据库中
MariaDB [tb_product]> use tb_stock ; Database changed MariaDB [tb_stock]> source /root/stock.sql; MariaDB [tb_stock]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | tb_order | | tb_product | | tb_stock | | test | +--------------------+ 7 rows in set (0.00 sec)
部署网关gateway
[root@k8s-master1 ~]# cd simple-microservice-dev3 [root@k8s-master1 simple-microservice-dev3]# cd k8s/ [root@k8s-master1 k8s]# ls docker_build.sh eureka.yaml gateway.yaml order.yaml portal.yaml product.yaml stock.yaml [root@k8s-master1 k8s]# vim docker_build.sh #!/bin/bash docker_registry=192.168.30.24 kubectl create secret docker-registry registry-pull-secret --docker-server=$docker_registry --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@ctnrs.com -n ms service_list="eureka-service gateway-service order-service product-service stock-service portal-service" service_list=${1:-${service_list}} work_dir=$(dirname $PWD) current_dir=$PWD cd $work_dir mvn clean package -Dmaven.test.skip=true for service in $service_list; do cd $work_dir/$service if ls |grep biz &>/dev/null; then cd ${service}-biz fi service=${service%-*} image_name=$docker_registry/microservice/${service}:$(date +%F-%H-%M-%S) docker build -t ${image_name} . docker push ${image_name} sed -i -r "s#(image: )(.*)#/1$image_name#" ${current_dir}/${service}.yaml kubectl apply -f ${current_dir}/${service}.yaml Done
[root@k8s-master1 k8s]# ./docker_build.sh gateway-service [root@k8s-master1 k8s]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.30.24/microservice/gateway 2019-08-10-14-54-28 395e59fcaf17 4 minutes ago 191MB 192.168.30.24/microservice/eureka latest 72d2811b9d61 3 hours ago 194MB eureka latest 72d2811b9d61 3 hours ago 194MB 192.168.30.24/microservice/gateway latest e7af6a3ae487 27 hours ago 191MB gateway latest e7af6a3ae487 27 hours ago 191MB java 8-jdk-alpine 3fd9dd82815c 2 years ago
[root@k8s-master1 ~]# kubectl get pod -n ms NAME READY STATUS RESTARTS AGE eureka-0 1/1 Running 1 114m eureka-1 1/1 Running 1 113m eureka-2 1/1 Running 1 112m gateway-59f86bfffb-7lglm 1/1 Running 0 5m50s gateway-59f86bfffb-s4qzl 1/1 Running 0 5m50s [root@k8s-master1 ~]# kubectl get pod,svc,ing -n ms NAME READY STATUS RESTARTS AGE pod/eureka-0 1/1 Running 1 115m pod/eureka-1 1/1 Running 1 114m pod/eureka-2 1/1 Running 1 113m pod/gateway-59f86bfffb-7lglm 1/1 Running 0 6m19s pod/gateway-59f86bfffb-s4qzl 1/1 Running 0 6m19s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/eureka ClusterIP None <none> 8888/TCP 115m service/gateway ClusterIP 10.0.0.3 <none> 9999/TCP 6m20s NAME HOSTS ADDRESS PORTS AGE ingress.extensions/eureka eureka.ctnrs.com 80 115m ingress.extensions/gateway gateway.ctnrs.com 80 6m20s
查看我们的eureka的注册中心,已经把gateway注册进来了
部署微服务业务程序与前端
对我们的脚本的mvn注释,我们这里不需要构建了
后端的服务我们只需要部署deployment就可以
[root@k8s-master1 k8s]# ./docker_build.sh product-service [root@k8s-master1 k8s]# kubectl get pods -n ms NAME READY STATUS RESTARTS AGE eureka-0 1/1 Running 1 137m eureka-1 1/1 Running 1 136m eureka-2 1/1 Running 1 135m gateway-59f86bfffb-7lglm 1/1 Running 0 28m gateway-59f86bfffb-s4qzl 1/1 Running 0 28m product-7fcd7646d-kpfxm 1/1 Running 0 87s product-7fcd7646d-lgfzq 1/1 Running 0 87s product也部署成功了
[root@k8s-master1 k8s]# ./docker_build.sh stock-service [root@k8s-master1 k8s]# ./docker_build.sh order-service [root@k8s-master k8s]# ./docker_build.sh portal-service
查看我们的微服务正常运行,这个根据自己的业务量启动副本数
[root@k8s-master k8s]# kubectl get pod,svc,ing -n ms NAME READY STATUS RESTARTS AGE pod/eureka-0 1/1 Running 7 56m pod/eureka-1 1/1 Running 5 55m pod/eureka-2 1/1 Running 7 54m pod/gateway-8b67dc9d4-8g2s9 1/1 Running 1 38m pod/gateway-8b67dc9d4-d6996 1/1 Running 1 38m pod/order-89f7bd55b-6t7wf 1/1 Running 1 12m pod/order-89f7bd55b-n74cv 1/1 Running 0 12m pod/portal-5f6d96ffdd-2smtj 1/1 Running 0 3m33s pod/portal-5f6d96ffdd-s7wf9 1/1 Running 0 3m33s pod/product-65d9bb865-9h7z7 1/1 Running 1 20m pod/product-65d9bb865-khvjp 1/1 Running 0 20m pod/stock-cb7fd87b5-6bqfp 1/1 Running 1 14m pod/stock-cb7fd87b5-7mbg9 1/1 Running 2 14m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/eureka ClusterIP None <none> 8888/TCP 56m service/gateway ClusterIP 10.0.0.74 <none> 9999/TCP 38m service/portal ClusterIP 10.0.0.62 <none> 8080/TCP 3m33s NAME HOSTS ADDRESS PORTS AGE ingress.extensions/eureka eureka.ctnrs.com 80 56m ingress.extensions/gateway gateway.ctnrs.com 80 38m ingress.extensions/portal portal.ctnrs.com 80 3m33s
访问portal.ctnrs.com