背景
最近想学习一下elasticsearch和fluentd的配合使用， fluentd比logstash节省太多资源了，所以就有了如下文章
Elasticsearch快捷安装(使用ECK方式)
参考文章
https://www.elastic.co/guide/en/cloud-on-k8s/1.8/k8s-deploy-eck.html
先安装一个eck的operator
kubectl create -f https://download.elastic.co/downloads/eck/1.8.0/crds.yaml kubectl apply -f https://download.elastic.co/downloads/eck/1.8.0/operator.yaml
等命令介绍，输入下面命令查看日志
kubectl -n elastic-system logs -f statefulset.apps/elastic-operator
安装elasticsearch
cat <<EOF | kubectl apply -f - apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: quickstart spec: version: 7.15.2 nodeSets: - name: default count: 1 config: node.store.allow_mmap: false EOF
安装完成后，输入命令，获得es的密码，默认账户是elastic
PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
部署完毕后，可以通过port-forward转发elasticsearch的端口到外部进行测试
kubectl port-forward service/quickstart-es-http 9200
再安装一个kibana
cat <<EOF | kubectl apply -f - apiVersion: kibana.k8s.elastic.co/v1 kind: Kibana metadata: name: quickstart spec: version: 7.15.2 count: 1 elasticsearchRef: name: quickstart EOF
可以通过port-forward转发kibana的端口到外部进行测试
kubectl port-forward service/quickstart-kb-http 5601
fluentd安装
编写一个fluentd.yaml ，
编写完毕后kubectl apply -f fluentd.yaml 。内容如下,注意替换密码：
--- apiVersion: v1 kind: ServiceAccount metadata: name: fluentd namespace: kube-logging labels: app: fluentd --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: fluentd labels: app: fluentd rules: - apiGroups: - "" resources: - pods - namespaces verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: fluentd roleRef: kind: ClusterRole name: fluentd apiGroup: rbac.authorization.k8s.io subjects: - kind: ServiceAccount name: fluentd namespace: kube-logging --- apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd namespace: kube-logging labels: app: fluentd spec: selector: matchLabels: app: fluentd template: metadata: labels: app: fluentd spec: serviceAccount: fluentd serviceAccountName: fluentd tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule containers: - name: fluentd image: fluent/fluentd-kubernetes-daemonset:v1.4.2-debian-elasticsearch-1.1 env: - name: FLUENT_ELASTICSEARCH_HOST value: "quickstart-es-http.default.svc.cluster.local" - name: FLUENT_ELASTICSEARCH_PORT value: "9200" - name: FLUENT_ELASTICSEARCH_SCHEME value: "http" - name: FLUENT_ELASTICSEARCH_USER value: "elastic" - name: FLUENT_ELASTICSEARCH_PASSWORD value: "我是密码！注意替换" - name: FLUENT_ELASTICSEARCH_SSL_VERSION value: "TLSv1_2" - name: FLUENTD_SYSTEMD_CONF value: disable - name: FLUENT_UID value: "0" resources: limits: memory: 512Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: varlog mountPath: /var/log - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true terminationGracePeriodSeconds: 30 volumes: - name: varlog hostPath: path: /var/log - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers
部署一个测试程序（用完之后可以删除）
kubectl -n logging apply -f - <<"EOF" apiVersion: apps/v1 kind: Deployment metadata: name: log-generator spec: selector: matchLabels: app.kubernetes.io/name: log-generator replicas: 1 template: metadata: labels: app.kubernetes.io/name: log-generator spec: containers: - name: nginx image: banzaicloud/log-generator:0.3.2 EOF
kibana里添加index和查看
直接看图说话
参考文章
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-on-kuberneteshttps://docs.fluentd.org/output/elasticsearchhttps://github.com/fluent/fluentd-kubernetes-daemonsethttps://medium.com/kubernetes-tutorials/cluster-level-logging-in-kubernetes-with-fluentd-e59aa2b6093a
