不使用pvc的方式在K8S中部署apisix-gateway

不使用pvc的方式在K8S中部署apisix-gateway

简介

我的apisix使用etcd作为数据存储服务器，官方的使用pvc方式或者docker-compose的方式，对于新手不太友好，本篇是从etcd的安装到apisix的打通都会涉及。

部署etcd

etcd 是一个分布式键值对存储，设计用来可靠而快速的保存关键数据并提供访问。通过分布式锁，leader选举和写屏障(write barriers)来实现可靠的分布式协作。etcd集群是为高可用，持久性数据存储和检索而准备。

• ubuntu部署etcd

  • ubuntu中部署etcd的两种方式：

    一种是去GitHub下载二进制的安装包，还有一种是apt-get install etcd，第二种方式我也尝试过，可能是我软件源的问题，版本有点老，所以我就换成了使用第一种方式，而且也比较推荐使用第一种方式。

  • 我用的etcd下载的版本是3.5.2，废话不多说直接看步骤：

    1、将etcd etcdctl etcdutl 二进制文件拷贝到/usr/local/bin目录下

    /usr/local/bin

    2、创建一个etcd的etcd.conf.yml，将下面代码拷贝进去，我这里etcd就简单的配置了一下，没有做集群，所以yml很简单。

    name: etcd-1data-dir: /home/etcd/datalisten-client-urls: http://0.0.0.0:2379advertise-client-urls: http://0.0.0.0:2379

    3、通过etcd --config-file etcd.conf.yml的路径运行，如下图就是成功了，也可以使用etcd manager客户端来测试。
    
    

    4、如果使用etcd直接启动的话没有办法后台运行，所以我们需要在/etc/systemd/system目录下创建一个etcd.service来进行后台运行。

    [Unit]Description=ETCD ServerDocumentation=https://github.com/coreos/etcdAfter=network-online.targetWants=network-online.target[Service]User=rootGroup=rootExecStart= etcd --config-file /home/etcd/etcd.conf.yml[Install]WantedBy=multi-user.target

    5、创建好之后可以通过以下命令来确定运行状态如下图：

    # 启动sudo systemctl start etcd.service# 查看状态sudo systemctl status etcd.service# 开机自启sudo systemctl enable etcd.service

    
    6、设置用户名和密码

    # 设置版本为V3export ETCDCTL_API=3# 添加用户etcdctl user add root# 开启认证etcdctl auth enable

K8S部署apisix

apisix-gateway在部署的时候分为两块，分别是apisix和apisix-dashboard面板，所以看起来比较绕，不过apisix在部署的时候使用的是yaml文件覆盖的方式，所以我这里是将yaml存储到configmap中了，方便进行统一管理。我使用的k8s是Ubuntu出品的microk8s，用它的主要原因是因为配置简单。

• apisix是服务端，用来进行网络请求转发。
• apisix-dashboard是他的控制面板，用来进行可视化配置。

  • apisix介绍

    apisix是基于 OpenResty + etcd 实现的云原生、高性能、可扩展的微服务 API 网关。它是国人开源，目前已经进入 Apache 进行孵化。APISIX 通过插件机制，提供了动态负载平衡、身份验证、限流限速等等功能，当然我们也可以自己开发插件进行拓展。

    • 动态负载均衡：跨多个上游服务的动态负载均衡，目前已支持 round-robin 轮询和一致性哈希算法。
    • 身份验证：支持 key-auth、JWT、basic-auth、wolf-rbac 等多种认证方式。
    • 限流限速：可以基于速率、请求数、并发等维度限制。

  • 部署apisix

    1、创建apisix.conf.yaml，并存储到configmap中，

    apisix:node_listen: 9080              # APISIX listening portenable_ipv6: falseallow_admin:                  # http://nginx.org/en/docs/http/ngx_http_access_module.html#allow    - 0.0.0.0/0              # We need to restrict ip access rules for security. 0.0.0.0/0 is for test.admin_key:- name: "admin"    key: edd1c9f034335f136f87ad84b625c8f1    role: admin                 # admin: manage all configuration data                                # viewer: only can view configuration data- name: "viewer"    key: 4054f7cf07e344346cd3f287985e76a2    role: viewerenable_control: truecontrol:    ip: "0.0.0.0"    port: 9092etcd:host:          # supports defining multiple etcd host addresses for an etcd cluster    - "http://192.168.31.170:2379"user: "root"    # ignore etcd username if not enable etcd authpassword: "root"  # ignore etcd password if not enable etcd authdiscovery:nacos:    host:    - "http://47.100.213.49:8848"    prefix: "/nacos/v1/"    fetch_interval: 30    # default 30 sec    weight: 100           # default 100    timeout:    connect: 2000       # default 2000 ms    send: 2000          # default 2000 ms    read: 5000          # default 5000 msplugin_attr:prometheus:    export_addr:    ip: "0.0.0.0"    port: 9091plugins:- client-control- ext-plugin-pre-req- zipkin- request-id- fault-injection- serverless-pre-function- batch-requests- cors- ip-restriction- ua-restriction- referer-restriction- uri-blocker- request-validation- openid-connect- wolf-rbac- hmac-auth- basic-auth- jwt-auth- key-auth- consumer-restriction- authz-keycloak- proxy-mirror- proxy-cache- proxy-rewrite- api-breaker- limit-conn- limit-count- limit-req- gzip- server-info- traffic-split- redirect- response-rewrite- grpc-transcode- prometheus- echo- http-logger- sls-logger- tcp-logger- kafka-logger- syslog- udp-logger- serverless-post-function- ext-plugin-post-reqstream_plugins:- ip-restriction- limit-conn- mqtt-proxy

    2、使用kubectl命令创建configmap

    kubectl create configmap sukt-apisix-gateway-config --from-file=config.yaml=/home/sukt-platform/apisix/apisix-gateway-config.yaml -n sukt-platform

    3、新建apisix-deployment.yaml

    apiVersion: apps/v1kind: Deploymentmetadata:name: sukt-apisix-gatewaynamespace: sukt-platformspec:selector:    matchLabels:    app: sukt-apisix-gatewaytemplate:    metadata:    labels:        app: sukt-apisix-gateway    spec:    containers:    - name: sukt-apisix-gateway        image: apache/apisix:2.10.3-alpine        imagePullPolicy: IfNotPresent        resources:        limits:            cpu: 500m            memory: 1Gi        requests:            cpu: 250m            memory: 256Mi        securityContext:        privileged: false        terminationMessagePath: /dev/termination-log        terminationMessagePolicy: File        volumeMounts:        - mountPath: /usr/local/apisix/conf/config.yaml        name: config        subPath: config.yaml        ports:        - containerPort: 9080        - containerPort: 9443    dnsPolicy: ClusterFirst    restartPolicy: Always    schedulerName: default-scheduler    securityContext: {}    terminationGracePeriodSeconds: 30    volumes:        - configMap:            defaultMode: 420            name: sukt-apisix-gateway-config        name: config

    4、新建apisix-service.yaml

    apiVersion: v1kind: Servicemetadata:name: sukt-apisix-gateway-nodetypelabels:    app: sukt-apisix-gateway-nodetypenamespace: sukt-platformspec:type: NodePortselector:    app: sukt-apisix-gatewayports:- port: 9080    name: transfer1    targetPort: 9080    nodePort: 30107- port: 9443    name: transfer2    targetPort: 9443    nodePort: 30108

  • 部署apisix-dashboard

    1、创建apisix-dashboard-config.yaml，并存储到configmap中，

    ## Licensed to the Apache Software Foundation (ASF) under one or more# contributor license agreements.  See the NOTICE file distributed with# this work for additional information regarding copyright ownership.# The ASF licenses this file to You under the Apache License, Version 2.0# (the "License"); you may not use this file except in compliance with# the License.  You may obtain a copy of the License at##     http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.#conf:listen:    host: 0.0.0.0     # `manager api` listening ip or host name    port: 9000          # `manager api` listening portallow_list:           # If we don't set any IP list, then any IP access is allowed by default.    - 0.0.0.0/0etcd:    endpoints:          # supports defining multiple etcd host addresses for an etcd cluster    - "http://192.168.31.170:2379"                        # yamllint disable rule:comments-indentation                        # etcd basic auth info    username: "root"    # ignore etcd username if not enable etcd auth    password: "root"  # ignore etcd password if not enable etcd auth    mtls:    key_file: ""          # Path of your self-signed client side key    cert_file: ""         # Path of your self-signed client side cert    ca_file: ""           # Path of your self-signed ca cert, the CA is used to sign callers' certificates    # prefix: /apisix     # apisix config's prefix in etcd, /apisix by defaultlog:    error_log:    level: warn       # supports levels, lower to higher: debug, info, warn, error, panic, fatal    file_path:        logs/error.log  # supports relative path, absolute path, standard output                        # such as: logs/error.log, /tmp/logs/error.log, /dev/stdout, /dev/stderr    access_log:    file_path:        logs/access.log  # supports relative path, absolute path, standard output                        # such as: logs/access.log, /tmp/logs/access.log, /dev/stdout, /dev/stderr                        # log example: 2020-12-09T16:38:09.039+0800	INFO	filter/logging.go:46	/apisix/admin/routes/r1	{"status": 401, "host": "127.0.0.1:9000", "query": "asdfsafd=adf&a=a", "requestId": "3d50ecb8-758c-46d1-af5b-cd9d1c820156", "latency": 0, "remoteIP": "127.0.0.1", "method": "PUT", "errs": []}authentication:secret:    secret              # secret for jwt token generation.                        # NOTE: Highly recommended to modify this value to protect `manager api`.                        # if it's default value, when `manager api` start, it will generate a random string to replace it.expire_time: 3600     # jwt token expire time, in secondusers:                # yamllint enable rule:comments-indentation    - username: admin   # username and password for login `manager api`    password: P@ssW0rd    - username: user    password: P@ssW0rdplugins:                          # plugin list (sorted in alphabetical order)- api-breaker- authz-keycloak- basic-auth- batch-requests- consumer-restriction- cors# - dubbo-proxy- echo# - error-log-logger# - example-plugin- fault-injection- grpc-transcode- hmac-auth- http-logger- ip-restriction- jwt-auth- kafka-logger- key-auth- limit-conn- limit-count- limit-req# - log-rotate# - node-status- openid-connect- prometheus- proxy-cache- proxy-mirror- proxy-rewrite- redirect- referer-restriction- request-id- request-validation- response-rewrite- serverless-post-function- serverless-pre-function# - skywalking- sls-logger- syslog- tcp-logger- udp-logger- uri-blocker- wolf-rbac- zipkin- server-info- traffic-split

    2、使用kubectl命令创建configmap

    kubectl create configmap sukt-apisix-dashboard-config --from-file=config.yaml=/home/sukt-platform/apisix/apisix-dashboard-config.yaml -n sukt-platform

    3、新建apisix-dashboard-deployment.yaml

    apiVersion: apps/v1kind: Deploymentmetadata:name: sukt-apisix-dashboardnamespace: sukt-platformspec:selector:    matchLabels:    app: sukt-apisix-dashboardtemplate:    metadata:    labels:        app: sukt-apisix-dashboard    spec:    nodeName: microk8sslave1 # 部署到指定的node节点    containers:    - name: sukt-apisix-dashboard        image: apache/apisix-dashboard:2.10.1-alpine        imagePullPolicy: IfNotPresent        resources:        limits:            cpu: 500m            memory: 1Gi        requests:            cpu: 250m            memory: 256Mi        securityContext:        privileged: false        terminationMessagePath: /dev/termination-log        terminationMessagePolicy: File        volumeMounts:        - mountPath: /usr/local/apisix-dashboard/conf/conf.yaml        name: config        subPath: config.yaml #这个位置对应的是comfigmap中的名字，不是 /usr/local/apisix-dashboard/conf/conf.yaml        ports:        - containerPort: 9000    dnsPolicy: ClusterFirst    restartPolicy: Always    schedulerName: default-scheduler    securityContext: {}    terminationGracePeriodSeconds: 30    volumes:        - configMap:            defaultMode: 420            name: sukt-apisix-dashboard-config        name: config

    4、新建apisix-dashboard-service.yaml

    apiVersion: v1kind: Servicemetadata:name: sukt-apisix-dashboard-nodetypelabels:    app: sukt-apisix-dashboard-nodetypenamespace: sukt-platformspec:type: NodePortselector:    app: sukt-apisix-dashboardports:- port: 9000    name: transfer1    targetPort: 9000    nodePort: 30109

  • 运行效果图

    可以通过dashboard面板的系统信息查看apisix-gateway的运行信息
    
    

结语

apisix-gateway文章分为两篇，本篇只是讲解了如何在k8s中安装以及启动，下一章讲解如何进行转发以及其他功能测试等。