开一个系列坑,记录使用Go语言练习实现微服务工具链的过程,第一篇是蓝绿部署的实现。
蓝绿部署是不停老版本,部署新版本然后进行测试,确认OK,将流量切到新版本。
项目的git地址为 github.com/mikellxy/mk… (蓝绿部署的实现在api目录的deploy目录下)
在蓝绿部署中,上线的过程中,不会停用老版本,而是另外部署新的服务来运行新版本,并导入测试流量进行测试。下文中,把一个项目正在对外提供服务的称作production服务,部署了新版本正在进行测试的称作staging服务。当测试通过后,我们执行swtich操作,把staging和production的身份对调,此时运行新版本代码的服务变成production对外提供服务。
由此可见,我们在描述一个项目的时候,必须要指定两套部署环境,因为需要支持同时运行production和staging的服务。
type Project struct { Model Name string `gorm:"not null;unique_index:uix_project_name;type:varchar(64)" json:"name" binding:"required"` BlueIP string `json:"blue_ip"` BluePort uint32 `json:"blue_port"` GreenIP string `json:"green_ip"` GreenPort uint32 `json:"green_port"` Deployments []*Deployment `gorm:"association foreign:project_id"` } 复制代码
创建一个叫test_project的项目指定两套部署环境
+----+--------------+-----------+-----------+-----------+------------+ | id | name | blue_ip | blue_port | green_ip | green_port | +----+--------------+-----------+-----------+-----------+------------+ | 1 | test_project | 127.0.0.1 | 8001 | 127.0.0.1 | 8002 | +----+--------------+-----------+-----------+-----------+------------+ 复制代码
在这个练习中,会把项目部署在本机的docker swarm上,所以两套部署环境的ip指定为localhost,docker swarm代理项目的端口分别是本机的8001和8002端口。
在给项目定义了描述项目部署环境的参数之后,下一步需要定义一个数据结构来描述两个部署环境的状态。
type Deployment struct { Model ProjectID uint `gorm:"not null;unique_index:uix_deployment_project_id_color" json:"project_id"` Color string `gorm:"type:varchar(16);not null;unique_index:uix_deployment_project_id_color" json:"color"` // production or staging Status string `gorm:"type:varchar(32);not null;default:'staging'" json:"status"` // stop, pending or running Stage string `gorm:"type:varchar(32);not null;default:'stop'" json:"stage"` PackageTag string `json:"package_tag"` } 复制代码
当需要上线新版本代码的时候,首先我们需要找到一个合适的部署环境,来部署staging服务。选择部署环境的逻辑如下:
首先根据project id查出对应的deployment数据,然后进行部署环境的选择,实现代码如下:
func GetStaging(project *models.Project) (*models.Deployment, error) { var blue, green *models.Deployment for _, d := range project.Deployments { if d.Color == GREEN { green = d } else { blue = d } } if blue != nil { // blue可用于staging if blue.Status == STATUS_STAG { if blue.Stage != STAGE_PENDING { return blue, nil } // 正在部署,返回错误 return nil, errors.New("deploying") } if green != nil { // green可用于staging if green.Stage != STAGE_PENDING { return green, nil } return nil, errors.New("deploying") } // 第一使用green, 创建数据 green, err := (&models.Deployment{}).Create(project.ID, GREEN) if err != nil { return nil, err } return green, nil } else if green != nil { // green可用于staging if green.Status == STATUS_STAG { if green.Stage != STAGE_PENDING { return green, nil } return nil, errors.New("deploying") } // 第一使用blue, 创建数据 blue, err := (&models.Deployment{}).Create(project.ID, BLUE) if err != nil { return nil, err } return blue, nil } else { // 新项目第一次部署,创建blue,用于部署 blue, err := (&models.Deployment{}).Create(project.ID, BLUE) if err != nil { return nil, err } return blue, nil } } 复制代码
一般在进行部署之前,新版本的代码已经通过ci工具打包成了docker image。当确定了用于部署staging服务的环境之后,我们需要获得最新的docker image的信息,然后通过docker api来部署docker image。
type Package struct { Model ProjectID uint `gorm:"not null;index" json:"project_id"` Tag string `gorm:"not null;unique;" json:"tag"` Port uint32 `gorm:"not null" json:"port"` } 复制代码
部署的api实现如下,调用docker api使用的是官方的Go SDK:
type IntID struct { ID int `uri:"id" binding:"required"` } func deploy(c *gin.Context) { var ip string var port uint32 param := IntID{} if err := c.BindUri(¶m); err != nil { c.JSON(http.StatusUnprocessableEntity, err.Error()) return } // 获取project project := &models.Project{} project, err := project.FindOneByID(uint(param.ID), true) if err != nil { c.JSON(http.StatusNotFound, err.Error()) return } // 选择要部署的环境 deployment, err := service.GetStaging(project) if err != nil { c.JSON(http.StatusInternalServerError, err.Error()) return } // 根据color确定项目部署的ip和port if deployment.Color == "green" { ip, port = project.GreenIP, project.GreenPort } else { ip, port = project.BlueIP, project.BluePort } // 获取项目最新的docker image版本 pkg := &models.Package{} pkg, err = pkg.FindOneByProjectID(project.ID) if err != nil { c.JSON(http.StatusNotFound, err.Error()) return } // 获取连接相应部署环境的docker client,使用docker api进行部署 conf := config.Conf.DockerClient dockerClient, err := docker_api.NewDockerClient(fmt.Sprintf(conf.Host, ip)) if err != nil { c.JSON(http.StatusInternalServerError, err.Error()) return } err = dockerClient.CreateSwarmService(fmt.Sprintf("%s_%s", project.Name, deployment.Color), pkg.Tag, 4, map[uint32]uint32{pkg.Port: port}) if err != nil { c.JSON(http.StatusInternalServerError, err.Error()) return } // 部署成功之后,把deployment的stage改成running db, _ := database.GetDB() deployment.Stage = "running" deployment.PackageTag = pkg.Tag db.Save(deployment) if db.Error != nil { c.JSON(http.StatusInternalServerError, db.Error.Error()) return } c.JSON(http.StatusCreated, deployment) } 复制代码
当staging环境的代码测试通过之后,可以把它修改成production(信息同时同步到api网关,后续文章再讨论api网关的实现,这里先不展开)对外提供服务。
func deploymentSwitch(c *gin.Context) { param := IntID{} if err := c.BindUri(¶m); err != nil { c.JSON(http.StatusUnprocessableEntity, err.Error()) return } // 获取project project := &models.Project{} project, err := project.FindOneByID(uint(param.ID), true) if err != nil { c.JSON(http.StatusNotFound, err.Error()) return } // 获取staging和production(第一次上线,还没有production) deployment var staging, other *models.Deployment for _, d := range project.Deployments { if d.Status == "staging" && d.Stage == "running" { staging = d } else { other = d } } if staging == nil { c.JSON(http.StatusInternalServerError, "no staging project is running") return } // 把staging的身份转换成staging,把原先的production的身份转换成staging,合适的时候可以停用老版本代码(调用docker api删除service,并把stage改成stop) db, _ := database.GetDB() staging.Status = "production" if other != nil { other.Status = "staging" other.Stage = "stop" db.Save(other) } db.Save(staging) c.JSON(http.StatusOK, project) } 复制代码
mysql> select * from package where project_id = 1 order by id desc; +----+------------+-----------------------------------------------------------+------+ | id | project_id | tag | port | +----+------------+-----------------------------------------------------------+------+ | 1 | 1 | mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f | 0 | +----+------------+-----------------------------------------------------------+------+ 复制代码
mysql> select * from deployment where project_id = 1; +----+------------+-------+---------+---------+-----------------------------------------------------------+ | id | project_id | color | status | stage | package_tag | +----+------------+-------+---------+---------+-----------------------------------------------------------+ | 1 | 1 | blue | staging | running | mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f | +----+------------+-------+---------+---------+-----------------------------------------------------------+ 复制代码
mysql> select * from deployment where project_id = 1; +----+------------+-------+------------+---------+-----------------------------------------------------------+ | id | project_id | color | status | stage | package_tag | +----+------------+-------+------------+---------+-----------------------------------------------------------+ | 1 | 1 | blue | production | running | mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f | +----+------------+-------+------------+---------+-----------------------------------------------------------+ 复制代码
mysql> select * from package where project_id = 1 order by id desc; +----+------------+-----------------------------------------------------------+------+ | id | project_id | tag | port | +----+------------+-----------------------------------------------------------+------+ | 2 | 1 | mikellxy/test-ci:9ed308ff2e483a873e1acfd1baa7e66ae232ce19 | 8090 | | 1 | 1 | mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f | 8090 | +----+------------+-----------------------------------------------------------+------+ 复制代码
mysql> select * from deployment where project_id = 1 order by updated_at desc; +----+------------+-------+------------+---------+-----------------------------------------------------------+ | id | project_id | color | status | stage | package_tag | +----+------------+-------+------------+---------+-----------------------------------------------------------+ | 2 | 1 | green | staging | running | mikellxy/test-ci:9ed308ff2e483a873e1acfd1baa7e66ae232ce19 | | 1 | 1 | blue | production | running | mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f | +----+------------+-------+------------+---------+-----------------------------------------------------------+ 复制代码
看一下docker service,两个环境上服务运行的docker iamge跟描述的也是一致的
docker service ls ID NAME MODE REPLICAS IMAGE PORTS go2nobdfy41b test_project_blue replicated 4/4 mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f *:8001->8090/tcp es8x3npaobhk test_project_green replicated 4/4 mikellxy/test-ci:9ed308ff2e483a873e1acfd1baa7e66ae232ce19 *:8002->8090/tcp 复制代码
mysql> select * from deployment where project_id = 1 order by updated_at desc; +----+------------+-------+------------+---------+-----------------------------------------------------------+ | id | project_id | color | status | stage | package_tag | +----+------------+-------+------------+---------+-----------------------------------------------------------+ | 1 | 1 | blue | staging | stop | mikellxy/test-ci:ef95f0d98da9d5f76cb75c49e3541f8c98a9327f | | 2 | 1 | green | production | running | mikellxy/test-ci:9ed308ff2e483a873e1acfd1baa7e66ae232ce19 | +----+------------+-------+------------+---------+-----------------------------------------------------------+ 复制代码
当服务被部署到blue或green环境之后,会把自己的ip和port注册到注册中心。在使用蓝绿部署的时候,部署工具可以把每对ip/port当前是production还是staging同步给网关。API网关基于服务注册发现和部署工作同步过来的信息,即可知道测试流量和外部正常流量分别应该转发到哪个ip/port。