Skip to content

Commit

Permalink
Update "Performance Testing" section in README.md and README_zh.md
Browse files Browse the repository at this point in the history 
Signed-off-by: Daniel Hu <[email protected]>
  • Loading branch information
@daniel-hutao
daniel-hutao committed Aug 14, 2023
1 parent b88e50e commit 9882ff4
Show file tree
Hide file tree
Showing 5 changed files with 177 additions and 111 deletions.
111 changes: 58 additions & 53 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,70 @@

</div>

Welcome to GoPool, **a project where 95% of the code is generated by GPT**. You can find the corresponding list of Commit and Prompt at [pro.devchat.ai](https://pro.devchat.ai).
Welcome to GoPool, **a project with 95% of the code is generated by GPT**. You can find the corresponding list of Commit and Prompt at [pro.devchat.ai](https://pro.devchat.ai).

GoPool is a high-performance, feature-rich, and easy-to-use worker pool library for Golang. It is designed to manage and recycle a pool of goroutines to complete tasks concurrently, improving the efficiency and performance of your applications.
GoPool is a **high-performance**, **feature-rich**, and **easy-to-use** worker pool library for Golang. It is designed to manage and recycle a pool of goroutines to complete tasks concurrently, improving the efficiency and performance of your applications.

<div align="center">
<img src="./logo/gopool.png" width="750">
</div>

## Performance Testing

This table shows the performance testing results for three Go libraries: GoPool, [ants](https://github.com/panjf2000/ants), and [pond](https://github.com/alitto/pond). The table includes the time it takes for each library to process 1 million tasks and the memory consumption in MB.

| Project | Time to Process 1M Tasks (s) | Memory Consumption (MB) |
|----------------|:----------------------------:|:-----------------------:|
| GoPool | 1.13 | 1.23 |
| [ants](https://github.com/panjf2000/ants) | 1.43 | 9.49 |
| [pond](https://github.com/alitto/pond) | 3.51 | 1.88 |

You can run the following commands to test the performance of GoPool, ants, and pond on your machine:

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool
$ go test -benchmem -run=^$ -bench ^BenchmarkAnts$ github.com/devchat-ai/gopool
$ go test -benchmem -run=^$ -bench ^BenchmarkPond$ github.com/devchat-ai/gopool
```

The results of the performance testing on my machine are as follows:

- GoPool

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool
goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
BenchmarkGoPoolWithMutex-10 1 1131753125 ns/op 1966192 B/op 13609 allocs/op
PASS
ok github.com/devchat-ai/gopool 1.5085s
```

- ants

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkAnts$ github.com/devchat-ai/gopool
goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
BenchmarkAnts-10 1 1425282750 ns/op 9952656 B/op 74068 allocs/op
PASS
ok github.com/devchat-ai/gopool 1.730s
```

- pond

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkPond$ github.com/devchat-ai/gopool
goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
BenchmarkPond-10 1 3512323792 ns/op 1288984 B/op 11106 allocs/op
PASS
ok github.com/devchat-ai/gopool 3.946s
```

## Features

- [x] **Task Queue**: GoPool uses a thread-safe task queue to store tasks waiting to be processed. Multiple workers can simultaneously fetch tasks from this queue.
Expand Down Expand Up @@ -264,54 +320,3 @@ func main() {
```

In this example, if a task fails, it will be retried up to 3 times.

## Performance Testing

We have conducted several performance tests to evaluate the efficiency and performance of GoPool. Here are the results:

- **TestGoPoolWithMutex**:

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN BenchmarkGoPoolWithMutex
BenchmarkGoPoolWithMutex
BenchmarkGoPoolWithMutex-10 2 803105167 ns/op 17416408 B/op 1017209 allocs/op
PASS
ok github.com/devchat-ai/gopool 2.586s
```

- **TestGoPoolWithSpinLock**:

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithSpinLock$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN BenchmarkGoPoolWithSpinLock
BenchmarkGoPoolWithSpinLock
BenchmarkGoPoolWithSpinLock-10 2 662952562 ns/op 17327176 B/op 1016087 allocs/op
PASS
ok github.com/devchat-ai/gopool 2.322s
```

- **BenchmarkGoroutines**:

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoroutines$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN BenchmarkGoroutines
BenchmarkGoroutines
BenchmarkGoroutines-10 3 371622847 ns/op 96642458 B/op 2005219 allocs/op
PASS
ok github.com/devchat-ai/gopool 2.410s
```

Please note that the actual performance may vary depending on the specific use case and system environment.
109 changes: 57 additions & 52 deletions README_zh.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,68 @@

欢迎来到 GoPool，这是**一个95%的代码由GPT生成的项目**。你可以在[pro.devchat.ai](https://pro.devchat.ai)找到相应的 Commit 和 Prompt 列表。

GoPool 是一个用 Golang 实现的高性能、功能丰富、易于使用的工作池库。它会管理和回收一组 goroutine 来并发完成任务，从而提高你的应用程序的效率和性能。
GoPool 是一个用 Golang 实现的**高性能**、**功能丰富**、**简单易用**的工作池库。它会管理和回收一组 goroutine 来并发完成任务，从而提高你的应用程序的效率和性能。

<div align="center">
<img src="./logo/gopool.png" width="750">
</div>

## 性能测试

这个表格展示了三个 Go 库 GoPool、[ants](https://github.com/panjf2000/ants) 和 [pond](https://github.com/alitto/pond)的性能测试结果。表格包括每个库处理 100 万个任务所需的时间和内存消耗（以 MB 为单位）。

| 项目 | 处理一百万任务耗时 (s) | 内存消耗 (MB) |
|----------------|:----------------------------:|:-----------------------:|
| GoPool | 1.13 | 1.23 |
| [ants](https://github.com/panjf2000/ants) | 1.43 | 9.49 |
| [pond](https://github.com/alitto/pond) | 3.51 | 1.88 |

你可以通过运行下列命令在你的机器上测试 GoPool、ants 和 pond 的性能：

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool
$ go test -benchmem -run=^$ -bench ^BenchmarkAnts$ github.com/devchat-ai/gopool
$ go test -benchmem -run=^$ -bench ^BenchmarkPond$ github.com/devchat-ai/gopool
```

在我的电脑上，性能测试的结果如下：

- GoPool

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool
goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
BenchmarkGoPoolWithMutex-10 1 1131753125 ns/op 1966192 B/op 13609 allocs/op
PASS
ok github.com/devchat-ai/gopool 1.5085s
```

- ants

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkAnts$ github.com/devchat-ai/gopool
goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
BenchmarkAnts-10 1 1425282750 ns/op 9952656 B/op 74068 allocs/op
PASS
ok github.com/devchat-ai/gopool 1.730s
```

- pond

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkPond$ github.com/devchat-ai/gopool
goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
BenchmarkPond-10 1 3512323792 ns/op 1288984 B/op 11106 allocs/op
PASS
ok github.com/devchat-ai/gopool 3.946s
```

## 特性

- [x] **任务队列**：GoPool 使用一个线程安全的任务队列来存储等待处理的任务。多个工作器可以同时从这个队列中获取任务。
Expand Down Expand Up @@ -264,54 +320,3 @@ func main() {
```

在这个示例中，如果任务失败，它将重试最多3次。

## 性能测试

我们进行了几个性能测试来评估 GoPool 的效率和性能。以下是结果：

- **TestGoPoolWithMutex**：

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN BenchmarkGoPoolWithMutex
BenchmarkGoPoolWithMutex
BenchmarkGoPoolWithMutex-10 2 803105167 ns/op 17416408 B/op 1017209 allocs/op
PASS
ok github.com/devchat-ai/gopool 2.586s
```

- **TestGoPoolWithSpinLock**：

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithSpinLock$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN BenchmarkGoPoolWithSpinLock
BenchmarkGoPoolWithSpinLock
BenchmarkGoPoolWithSpinLock-10 2 662952562 ns/op 17327176 B/op 1016087 allocs/op
PASS
ok github.com/devchat-ai/gopool 2.322s
```

- **BenchmarkGoroutines**：

```bash
$ go test -benchmem -run=^$ -bench ^BenchmarkGoroutines$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN BenchmarkGoroutines
BenchmarkGoroutines
BenchmarkGoroutines-10 3 371622847 ns/op 96642458 B/op 2005219 allocs/op
PASS
ok github.com/devchat-ai/gopool 2.410s
```

请注意，实际性能可能会根据具体的使用情况和系统环境而变化。
2 changes: 2 additions & 0 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ module github.com/devchat-ai/gopool
go 1.20

require (
github.com/alitto/pond v1.8.3
github.com/daniel-hutao/spinlock v0.1.0
github.com/onsi/ginkgo/v2 v2.11.0
github.com/onsi/gomega v1.27.10
github.com/panjf2000/ants/v2 v2.8.1
)

require (
Expand Down
13 changes: 12 additions & 1 deletion go.sum
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
github.com/alitto/pond v1.8.3 h1:ydIqygCLVPqIX/USe5EaV/aSRXTRXDEI9JwuDdu+/xs=
github.com/alitto/pond v1.8.3/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
Expand All @@ -20,14 +22,23 @@ github.com/onsi/ginkgo/v2 v2.11.0 h1:WgqUCUt/lT6yXoQ8Wef0fsNn5cAuMK7+KT9UFRz2tcU
github.com/onsi/ginkgo/v2 v2.11.0/go.mod h1:ZhrRA5XmEE3x3rhlzamx/JJvujdZoJ2uvgI7kR0iZvM=
github.com/onsi/gomega v1.27.10 h1:naR28SdDFlqrG6kScpT8VWpu1xWY5nJRCF3XaYyBjhI=
github.com/onsi/gomega v1.27.10/go.mod h1:RsS8tutOdbdgzbPtzzATp12yT7kM5I5aElG3evPbQ0M=
github.com/panjf2000/ants/v2 v2.8.1 h1:C+n/f++aiW8kHCExKlpX6X+okmxKXP7DWLutxuAPuwQ=
github.com/panjf2000/ants/v2 v2.8.1/go.mod h1:KIBmYG9QQX5U2qzFP/yQJaq/nSb6rahS9iEHkrCMgM8=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
golang.org/x/mod v0.10.0 h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk=
golang.org/x/net v0.12.0 h1:cfawfvKITfUsFCeJIHJrbSxpeu/E81khclypR0GVT50=
golang.org/x/net v0.12.0/go.mod h1:zEVYFnQC7m/vmpQFELhcD1EWkZlX69l4oqgmer6hfKA=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.10.0 h1:SqMFp9UcQJZa+pmYuAKjd9xq1f0j5rLcDIk0mj4qAsA=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
Expand Down
53 changes: 48 additions & 5 deletions gopool_benchmark_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,19 @@ import (
"time"

"github.com/daniel-hutao/spinlock"
"github.com/alitto/pond"
"github.com/panjf2000/ants/v2"
)

const (
PoolSize = 1e4
TaskNum = 1e6
)

func BenchmarkGoPoolWithMutex(b *testing.B) {
var wg sync.WaitGroup
var taskNum = int(1e6)
pool := NewGoPool(1e4, WithLock(new(sync.Mutex)))
var taskNum = int(TaskNum)
pool := NewGoPool(PoolSize, WithLock(new(sync.Mutex)))
defer pool.Release()

b.ResetTimer()
Expand All @@ -31,8 +38,8 @@ func BenchmarkGoPoolWithMutex(b *testing.B) {

func BenchmarkGoPoolWithSpinLock(b *testing.B) {
var wg sync.WaitGroup
var taskNum = int(1e6)
pool := NewGoPool(1e4, WithLock(new(spinlock.SpinLock)))
var taskNum = int(TaskNum)
pool := NewGoPool(PoolSize, WithLock(new(spinlock.SpinLock)))
defer pool.Release()

b.ResetTimer()
Expand All @@ -52,7 +59,7 @@ func BenchmarkGoPoolWithSpinLock(b *testing.B) {

func BenchmarkGoroutines(b *testing.B) {
var wg sync.WaitGroup
var taskNum = int(1e6)
var taskNum = int(TaskNum)

for i := 0; i < b.N; i++ {
wg.Add(taskNum)
Expand All @@ -66,3 +73,39 @@ func BenchmarkGoroutines(b *testing.B) {
wg.Wait()
}
}

func BenchmarkPond(b *testing.B) {
pool := pond.New(PoolSize, 0, pond.MinWorkers(PoolSize))

taskFunc := func() {
time.Sleep(10 * time.Millisecond)
}

b.ResetTimer()
for i := 0; i < b.N; i++ {
for i := 0; i < TaskNum; i++ {
pool.Submit(taskFunc)
}
pool.StopAndWait()
}
}

func BenchmarkAnts(b *testing.B) {
var wg sync.WaitGroup
p, _ := ants.NewPool(PoolSize)
defer p.Release()

taskFunc := func() {
time.Sleep(10 * time.Millisecond)
wg.Done()
}

b.ResetTimer()
for i := 0; i < b.N; i++ {
for i := 0; i < TaskNum; i++ {
wg.Add(1)
_ = p.Submit(taskFunc)
}
wg.Wait()
}
}

0 comments on commit 9882ff4

Please sign in to comment.