Loki简介

Loki相比EFK/ELK，它不对原始日志进行索引，只对日志的标签进行索引，而日志通过压缩进行存储，通常是文件系统存储，所以其操作成本更低，数量级效率更高

由于Loki的存储都是基于文件系统的，所以它的日志搜索时基于内容即日志行中的文本，所以它的查询支持LogQL，在搜索窗口中通过过滤标签的方式进行搜索和查询

Loki分两部分，Loki是日志引擎部分，Promtail是收集日志端，然后通过Grafana进行展示

Promtail: 代理，负责收集日志并将其发送给 loki

Loki: 日志记录引擎，负责存储日志和处理查询

Grafana: UI 界面

upload successful

Loki的官方文档

1
https://grafana.com/docs/loki/latest/installation

从官方文档看，Loki支持源码安装、Docker、Helm、Local、Tanka

Docker-Compose部署

本章只用一台服务器部署，如需收集其他节点日志，只需在被收集端部署Promtail Agent即可

Loki配置文件下载路径

1
https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml

Promtail配置文件下载路径

1
https://raw.githubusercontent.com/grafana/loki/main/clients/cmd/promtail/promtail-local-config.yaml

配置yaml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
[root@pool1 loki]# tree
.
├── image
│   ├── grafana.tar.gz
│   ├── loki-2.4.tar.gz
│   └── promtail-2.4.tar.gz
├── pkgs
│   ├── docker-19.03.9.tgz
│   └── InstallDocker.py
└── yaml
    └── docker-compose.yaml
[root@pool1 loki]# vi yaml/docker-compose.yaml
version: "3"

services:
  loki:
    container_name: loki
    image: grafana/loki:2.4.0
    restart: always
    ports:
      - "3100:3100"
    volumes:
      - /opt/loki/conf:/etc/loki
      - /etc/localtime:/etc/localtime
    command: -config.file=/etc/loki/loki-local-config.yaml
    networks:
      - loki

  promtail:
    container_name: promtail
    image: grafana/promtail:2.4.0
    restart: always
    depends_on:
      - loki
    volumes:
      - /var/log:/var/log
      - /opt/promtail/conf:/etc/promtail
      - /etc/localtime:/etc/localtime
    command: -config.file=/etc/promtail/promtail-local-config.yaml
    networks:
      - loki

  grafana:
    container_name: grafana
    image: grafana/grafana:latest
    restart: always
    depends_on:
      - loki
      - promtail
    ports:
      - "3200:3000"
    volumes:
      - /etc/localtime:/etc/localtime
    networks:
      - loki
      
networks:
  loki:
    driver: bridge

创建持久化目录

1
mkdir -p /opt/{loki,promtail}/conf

导入loki及promtail的配置文件

1
2
3
4
5
6
7
8
[root@pool1 opt]# tree
.
├── loki
│   └── conf
│       └── loki-local-config.yaml
└── promtail
    └── conf
        └── promtail-local-config.yaml

Loki配置文件解析

  • 官方下载的文件默认即可，不用修改
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    instance_addr: 127.0.0.1
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

ruler:
  alertmanager_url: http://localhost:9093

Promtail配置文件解析

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://10.99.2.164:3100/loki/api/v1/push
  # 这里要修改为loki的ip

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost # 代表收集promtail本机的日志目录
    labels:
    # 将会作为索引查询，除了"__path__"外，其他的"key: value"可以自定义
      job: varlogs
      # 收集日志的目录
      __path__: /var/log/*log

# 其他服务日志实例，可按照以下模板修改
- job_name: nginx
  static_configs:
  - targets:
      - localhost
    labels:
      job: "70.60-nginx"
      __path__: /var/log/nginx/*log

- job_name: db
  static_configs:
  - targets:
      - localhost
    labels:
      job: "70.60-mysql"
      __path__: /var/log/mysqld.log

启动服务

1
2
3
4
5
6
7
8
9
10
[root@pool1 loki]# docker-compose up -d
Creating loki ... done
Creating promtail ... done
Creating grafana  ... done
[root@pool1 yaml]# docker-compose ps
  Name                Command               State           Ports         
--------------------------------------------------------------------------
grafana    /run.sh                          Up      0.0.0.0:3000->3000/tcp
loki       /usr/bin/loki -config.file ...   Up      0.0.0.0:3100->3100/tcp
promtail   /usr/bin/promtail -config. ...   Up

Grafana 与 Loki 结合

访问 http://10.99.2.164:3000，默认用户密码为admin，登录后进行如下操作

添加数据源

upload successful

upload successful

upload successful

upload successful

upload successful

安装wget演示

1
2
3
4
5
6
7
8
[root@pool1 yaml]# yum -y install wget
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-15.el7 will be updated
---> Package wget.x86_64 0:1.14-18.el7_6.1 will be an update
--> Finished Dependency Resolution

upload successful