Commit graph

9967 commits

Author SHA1 Message Date
chrislu 76f26c4b99 mount: delete target entry first during move 2022-06-20 13:15:02 -07:00
Chris Lu 0d22877df9
Merge pull request #3209 from chrislusf/dependabot/go_modules/github.com/google/btree-1.1.2
Bump github.com/google/btree from 1.0.1 to 1.1.2
2022-06-20 10:40:00 -07:00
Chris Lu 758df5ade6
Merge pull request #3208 from chrislusf/dependabot/go_modules/github.com/stretchr/testify-1.7.3
Bump github.com/stretchr/testify from 1.7.2 to 1.7.3
2022-06-20 10:39:53 -07:00
Chris Lu 497ac1da27
Merge pull request #3207 from chrislusf/dependabot/go_modules/github.com/ydb-platform/ydb-go-sdk/v3-3.27.0
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.26.10 to 3.27.0
2022-06-20 10:39:45 -07:00
Chris Lu a608d71523
Merge pull request #3206 from chrislusf/dependabot/go_modules/github.com/aws/aws-sdk-go-1.44.37
Bump github.com/aws/aws-sdk-go from 1.44.32 to 1.44.37
2022-06-20 10:39:39 -07:00
Chris Lu 6eed6ceaa3
Merge pull request #3205 from chrislusf/dependabot/go_modules/cloud.google.com/go/pubsub-1.22.2
Bump cloud.google.com/go/pubsub from 1.21.1 to 1.22.2
2022-06-20 10:39:33 -07:00
Chris Lu 2064e49842
Merge pull request #3204 from chrislusf/dependabot/github_actions/actions/dependency-review-action-2.0.2
Bump actions/dependency-review-action from 1.0.2 to 2.0.2
2022-06-20 10:39:20 -07:00
dependabot[bot] 50d564216b
Bump github.com/google/btree from 1.0.1 to 1.1.2
Bumps [github.com/google/btree](https://github.com/google/btree) from 1.0.1 to 1.1.2.
- [Release notes](https://github.com/google/btree/releases)
- [Commits](https://github.com/google/btree/compare/v1.0.1...v1.1.2)

---
updated-dependencies:
- dependency-name: github.com/google/btree
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-20 15:24:50 +00:00
dependabot[bot] 43f5df522f
Bump github.com/stretchr/testify from 1.7.2 to 1.7.3
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.2 to 1.7.3.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.7.2...v1.7.3)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-20 15:24:44 +00:00
dependabot[bot] ddd261845b
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.26.10 to 3.27.0
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.26.10 to 3.27.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.26.10...v3.27.0)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-20 15:24:40 +00:00
dependabot[bot] f800404516
Bump github.com/aws/aws-sdk-go from 1.44.32 to 1.44.37
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.32 to 1.44.37.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.32...v1.44.37)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-20 15:24:27 +00:00
dependabot[bot] ff05008083
Bump cloud.google.com/go/pubsub from 1.21.1 to 1.22.2
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.21.1 to 1.22.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.21.1...pubsub/v1.22.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-20 15:24:14 +00:00
dependabot[bot] 2d75dda243
Bump actions/dependency-review-action from 1.0.2 to 2.0.2
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 1.0.2 to 2.0.2.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](a9c83d3af6...1c59cdf2a9)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-20 15:23:33 +00:00
Chris Lu 460c7cf68e
Merge pull request #3203 from shichanglin5/volume_copy_preallocate 2022-06-20 07:41:45 -07:00
Chris Lu 1166d9dcf1
Merge pull request #3202 from shichanglin5/fix_circuit_breaker 2022-06-20 07:35:40 -07:00
石昌林 81f7f08708 Determine whether to preallocate according to the master configuration before executing copy volume 2022-06-20 21:12:44 +08:00
石昌林 be5c901bea Fix global counter key is incorrect & Add read lock when reading counter map to avoid memory problems 2022-06-20 19:16:53 +08:00
Lapshin Vitaliy 5f5fd0bc48 validate tags on copy object and add regex for validating tags 2022-06-20 11:32:58 +03:00
Chris Lu 9138e68208
Merge pull request #3199 from shichanglin5/update_go.sum
update go.sum
2022-06-19 23:41:19 -07:00
石昌林 bd63b305dd update go.sum
go.mod has been changed but go.sum has not been updated accordingly, causing the program to fail to start:
../../pkg/mod/google.golang.org/api@v0.83.0/option/option.go:12:2: missing go.sum entry for module providing package golang.org/x/oauth2 (imported by google.golang.org/api/option); to add:
	go get google.golang.org/api/option@v0.83.0
../../pkg/mod/google.golang.org/api@v0.83.0/option/option.go:13:2: missing go.sum entry for module providing package golang.org/x/oauth2/google (imported by cloud.google.com/go/storage); to add:
	go get cloud.google.com/go/storage@v1.22.1
../../pkg/mod/github.com/chrislusf/raft@v1.0.8/protobuf/seaweed_raft.pb.go:12:2: missing go.sum entry for module providing package google.golang.org/grpc (imported by github.com/chrislusf/seaweedfs/weed/command); to add:
	go get github.com/chrislusf/seaweedfs/weed/command
2022-06-20 14:35:04 +08:00
Chris Lu 9e2d6e897e
Merge pull request #3198 from guol-fnst/fix_statuscode
AbortMultipartUploadHandler should return 204 instead of 200
2022-06-19 22:57:13 -07:00
Chris Lu 6270047202
Merge pull request #3189 from shichanglin5/circuit_breaker
add s3 circuit breaker support for 'simultaneous request count' and '…
2022-06-19 22:56:40 -07:00
LHHDZ 8db9f13bc6
Merge branch 'master' into circuit_breaker 2022-06-20 13:42:18 +08:00
guol-fnst 1d77deccd0 AbortMultipartUploadHandler should return 204 instead of 200
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
2022-06-20 13:26:22 +08:00
石昌林 9e036df356 remove go.uber.org/atomic 2022-06-20 12:35:29 +08:00
Chris Lu 1baa7ead59
Merge pull request #3195 from geekboood/master 2022-06-19 10:43:52 -07:00
geekboood fdacd94af5 fix: invalid chunk data when failed to read manifests 2022-06-19 16:54:04 +08:00
石昌林 3dd60529c5 some code optimizations 2022-06-17 19:07:39 +08:00
石昌林 37df209195 add some unit tests and some code optimizes 2022-06-17 17:11:18 +08:00
Chris Lu d28f8bb88f
Merge pull request #3192 from guol-fnst/weed_update
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-16 22:58:29 -07:00
guol-fnst c7c20881f5 remove "weed" from UsageLine, or weed will not show usage of weed update 2022-06-17 12:47:04 +08:00
chrislu 625fd16a2e reduce upfront memory usage for low density volume 2022-06-16 15:39:29 -07:00
Chris Lu 5f8d6b5f13
Merge pull request #3184 from guol-fnst/weed_update
Weed update
2022-06-16 10:02:45 -07:00
guol-fnst cd078471c4 fix path bug on windows
fix usage
2022-06-16 17:24:56 +08:00
chrislu aaa9938353 Revert "adjust conditions"
This reverts commit e024586ff1.
2022-06-16 01:26:36 -07:00
guol-fnst 995e221e97 remove -output, add -dir,-name for more flexibility 2022-06-16 16:07:32 +08:00
chrislu e024586ff1 adjust conditions 2022-06-15 23:19:39 -07:00
Chris Lu 82f3bcc65e
Merge pull request #3081 from paochiang/volume_upload_limit_fix
exclude replication from the concurrentUploadLimitMB
2022-06-15 23:16:44 -07:00
Chris Lu 40055211bd
Merge pull request #3190 from famosss/master
add bucket label to s3 prometheus metrics
2022-06-15 23:15:41 -07:00
liubaojiang df0ce31a2e add condition when inFlightUploadDataLimitCond signal 2022-06-16 14:07:11 +08:00
zzq09494 6449114e5e format 2022-06-16 13:52:36 +08:00
zzq09494 0a613876ca add bucket label to the grafana dashboard 2022-06-16 13:50:16 +08:00
zzq09494 62f74f5e3c add bucket label to s3 prometheus metrics 2022-06-16 13:21:25 +08:00
guol-fnst 4bc8792643 support specific version of weed 2022-06-16 11:40:23 +08:00
guol-fnst 1dbb925d29 remove unused function
remove log when weed is up to date

add check whether weed is full version
2022-06-16 11:40:20 +08:00
liubaojiang 3076ac101e move vs.concurrentUploadLimit != 0 out of the lock 2022-06-16 09:58:44 +08:00
Chris Lu 097a12dfb4
Merge pull request #3187 from ningfdx/remote 2022-06-15 06:34:32 -07:00
Chris Lu 55b9f281d1
Merge pull request #3185 from kmlebedev/logging_err_process_range_req 2022-06-15 06:33:33 -07:00
石昌林 78b3728169 add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply

# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply

# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply

# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply

# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
2022-06-15 21:07:55 +08:00
Konstantin Lebedev a9f6c4b43d fix ip.bind cmd for docker compose 2022-06-15 16:49:53 +05:00