test: add pika benchmark tools by luky116 · Pull Request #2663 · OpenAtomFoundation/pikiwidb

luky116 · 2024-05-21T09:33:20Z

添加 pika benchark 工具，旨在提升压测效率，并输出可视化的统计图表。目前支持了以下的压测 case：
1、常用命令的百分位延时统计图：

2、常用命令的 OPS 统计图

3、不同读写场景下的百分位延时统计图

4、不同读写场景下的 OPS 统计图

baerwang · 2024-05-21T11:08:42Z

+```
+
+3. After the stress test is completed, parse and format the stress test data. First execute go build to compile and convert the program:
+```go


baerwang · 2024-05-21T11:08:56Z

+```
+
+Next, execute the program to format the stress test data. If the output folder does not exist, it needs to be created manually in advance:
+```go


ditto

done

baerwang · 2024-05-21T11:09:00Z

+./parser -in_dir=$(pwd)/bench_data -out_dir=$(pwd)/parsed_data
+```
+4. Use a python script to generate a statistical chart image from the data:
+```go


ditto

done

baerwang · 2024-05-21T11:09:18Z

+本工具提供了常用指标的压测命令，并支持将压测结果一键生成可视化的 svg 格式的统计图，提升压测效率。使用步骤如下：
+1、首先，压测工具使用了 Redis 官方的 memtier_benchmark，所以要选把这个工具安装好，安装流程参考官方文档：[https://github.com/RedisLabs/memtier_benchmark](https://github.com/RedisLabs/memtier_benchmark)。
+2、接下来启动一个 Pika 进程，然后执行压测脚本进行压测：
+```go


ditto

done

baerwang · 2024-05-21T11:09:22Z

+sh pika_benchmark.sh -host 127.0.0.1 -port 9221
+```
+目前可支持以下测试参数：
+```go


ditto

done

baerwang · 2024-05-21T11:09:53Z

+1. First of all, the stress testing tool uses Redis’ official memtier_benchmark, so you must choose to install this tool. For the installation process, please refer to the official documentation: [https://github.com/RedisLabs/memtier_benchmark](https://github.com /RedisLabs/memtier_benchmark).
+2. Next, start a Pika process, and then execute the stress test script to perform the stress test:
+
+```go


ditto

done

baerwang · 2024-05-21T11:09:58Z

+sh pika_benchmark.sh -host 127.0.0.1 -port 9221
+```
+Currently the following test parameters are supported:
+```go


ditto

done

baerwang · 2024-05-21T11:12:04Z

+}
+
+func writeLantencyFile(path string, data ParsedLatencyData) {
+	fileName := path + "/" + data.Title


filepath.Join(path,data.Title)

filepath.Join(path,data.Title)

done

baerwang · 2024-05-21T11:12:35Z

+}
+
+func writeOpsFile(path string, data ParsedOpsData) {
+	fileName := path + "/" + data.Title


ditto

done

baerwang · 2024-05-21T11:13:21Z

+	}
+	defer file.Close()
+
+	var (


var ( hasAllStatsLine bool hasRequestLatencyDistributionLine bool hasDividingLineBegin bool res = newBenchResult(name, usePrefix) )

var ( hasAllStatsLine bool hasRequestLatencyDistributionLine bool hasDividingLineBegin bool res = newBenchResult(name, usePrefix) )

done

baerwang · 2024-05-21T12:23:02Z

+cmd_zadd=$pwd/bench_data/cmd_zadd.txt
+cmd_range=$pwd/bench_data/cmd_zrange.txt
+
+memtier_benchmark --server=$server --port=$port --clients=10 --requests=$requests --data-size=32 --threads=20 --ratio=0:10 --select-db=0 > $rw_0r10w


这个是不是没用传参数的值？

这个是不是没用传参数的值？

done

* add pika benchmark --------- Co-authored-by: liuyuecai <liuyuecai@360.cn>

add pika benchmark

d3835a8

github-actions Bot added Invalid PR Title ✏️ Feature New feature or request 📒 Documentation Improvements or additions to documentation labels May 21, 2024

remove unsed conf

5a76d03

luky116 changed the title ~~add pika benchmark~~ test: add pika benchmark May 21, 2024

github-actions Bot removed the Invalid PR Title label May 21, 2024

reformat

271f364

github-actions Bot added the 🧹 Updates This will not be worked on label May 21, 2024

baerwang reviewed May 21, 2024

View reviewed changes

reformat code

ef52afb

baerwang reviewed May 21, 2024

View reviewed changes

liuyuecai added 7 commits May 21, 2024 20:23

reformat code

4a8f01f

reformat code

770f044

reformat pika_benchmark.sh

07d63d0

reformat charts

dfef9e5

reformat charts

3893185

reformat charts

f479898

reformat charts

82982d2

baerwang approved these changes May 21, 2024

View reviewed changes

liuyuecai added 6 commits May 21, 2024 22:05

change filepath

5d70511

change /usr/bin/python3 to python3

4715e06

reformat readme

e524b80

reformat readme

77ee9b2

reformat readme

880c3d6

reformat gen_chart.py

c428bd4

luky116 changed the title ~~test: add pika benchmark~~ test: add pika benchmark tools May 21, 2024

liuyuecai added 2 commits May 21, 2024 22:38

reformat code

349b2b5

reformat code

f007504

luky116 removed the ✏️ Feature New feature or request label May 23, 2024

luky116 added test benchmark and removed 🧹 Updates This will not be worked on 📒 Documentation Improvements or additions to documentation labels May 23, 2024

AlexStocks added the 4.0.0 label May 29, 2024

AlexStocks approved these changes May 29, 2024

View reviewed changes

AlexStocks merged commit 0e903fb into OpenAtomFoundation:unstable May 29, 2024

chejinge pushed a commit that referenced this pull request Aug 1, 2024

test: add pika benchmark tools (#2663)

ca05211

* add pika benchmark --------- Co-authored-by: liuyuecai <liuyuecai@360.cn>

cheniujh pushed a commit to cheniujh/pika that referenced this pull request Sep 24, 2024

test: add pika benchmark tools (OpenAtomFoundation#2663)

a464130

* add pika benchmark --------- Co-authored-by: liuyuecai <liuyuecai@360.cn>

cheniujh pushed a commit to cheniujh/pika that referenced this pull request Sep 24, 2024

test: add pika benchmark tools (OpenAtomFoundation#2663)

6b688a4

* add pika benchmark --------- Co-authored-by: liuyuecai <liuyuecai@360.cn>

byseea11 pushed a commit to byseea11/pikiwidb that referenced this pull request Sep 27, 2025

test: add pika benchmark tools (OpenAtomFoundation#2663)

46bbbc6

* add pika benchmark --------- Co-authored-by: liuyuecai <liuyuecai@360.cn>

byseea11 pushed a commit to byseea11/pikiwidb that referenced this pull request Sep 29, 2025

test: add pika benchmark tools (OpenAtomFoundation#2663)

271cc8d

* add pika benchmark --------- Co-authored-by: liuyuecai <liuyuecai@360.cn>

Uh oh!

Conversation

luky116 commented May 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

baerwang May 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

baerwang May 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

luky116 commented May 21, 2024 •

edited

Loading

baerwang May 21, 2024 •

edited

Loading

baerwang May 21, 2024 •

edited

Loading