Skip to content

Commit

Permalink
Reasonable memory restrictions when running local minio s3 plugin test
Browse files Browse the repository at this point in the history
Set 2g memory limit to local Minio docker.
Set s3 configuration chunksize and concurrency.

S3 plugin upload and download chunksize defaults to 500MB and concurrent
requests defaults to 6. This means that each s3 plugin upload call will
reserve a buffer size of 500MB * 6 = 3G.

This is observable during testing in the s3 plugin logs.
```
Uploading file [backup_file] with chunksize 524288000 and concurrency 6
```

On a demo cluster of 3 segments
this means that during backup or restore our plugin will try to reserve
a total of 3G * 3 segments = 9G. This way out of line for local minio
testing which only tries to upload a few bytes. If a system doesn't have
this kind of free memory on hand it will error out with the following
panic.

```
fatal error: runtime: out of memory

runtime stack:
...

goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:474 +0x8 fp=0xc00013f158 sp=0xc00013f148 pc=0x46a788
runtime.(*mheap).alloc(0x0?, 0x3?, 0xb8?)
	/usr/local/go/src/runtime/mheap.go:962 +0x5b fp=0xc00013f1a0 sp=0xc00013f158 pc=0x42cb3b
runtime.(*mcache).allocLarge(0xc000024540?, 0xbb800000, 0x58?)
	/usr/local/go/src/runtime/mcache.go:234 +0x85 fp=0xc00013f1e8 sp=0xc00013f1a0 pc=0x41ab85
runtime.mallocgc(0xbb800000, 0xa39600, 0x1)
	/usr/local/go/src/runtime/malloc.go:1123 +0x4f6 fp=0xc00013f250 sp=0xc00013f1e8 pc=0x411d96
runtime.makeslice(0xb5aa93?, 0x36?, 0xc00013f3a0?)
	/usr/local/go/src/runtime/slice.go:103 +0x49 fp=0xc00013f278 sp=0xc00013f250 pc=0x451d69
bufio.NewReaderSize(...)
	/usr/local/go/src/bufio/bufio.go:57
github.com/greenplum-db/gpbackup-s3-plugin/s3plugin.uploadFile(0xc0000305d0?, 0xc0002f1c20, {0xc0002e6460, 0x4a}, 0xc00003c148)
...
```
  • Loading branch information
kyeap-vmware committed Feb 28, 2024
1 parent 84c2a9f commit 7ee5d93
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ info-report:
test-s3-local: build install
${PWD}/plugins/generate_minio_config.sh
mkdir -p /tmp/minio/gpbackup-s3-test
docker run -d --name s3-minio -p 9000:9000 -p 9001:9001 -v /tmp/minio:/data/minio quay.io/minio/minio server /data/minio --console-address ":9001"
docker run -d --name s3-minio --memory="2g" -p 9000:9000 -p 9001:9001 -v /tmp/minio:/data/minio quay.io/minio/minio server /data/minio --console-address ":9001"
sleep 2 # Wait for minio server to start up
${PWD}/plugins/plugin_test.sh $(BIN_DIR)/gpbackup_s3_plugin /tmp/minio_config.yaml
docker stop s3-minio
Expand Down
4 changes: 4 additions & 0 deletions plugins/generate_minio_config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,8 @@ options:
aws_secret_access_key: minioadmin
bucket: gpbackup-s3-test
folder: test/backup
backup_max_concurrent_requests: 2
backup_multipart_chunksize: 5MB
restore_max_concurrent_requests: 2
restore_multipart_chunksize: 5MB
MINIO_CONFIG

0 comments on commit 7ee5d93

Please sign in to comment.