Commit graph

7982 commits

Author SHA1 Message Date
石昌林 3dd60529c5 some code optimizations 2022-06-17 19:07:39 +08:00
石昌林 37df209195 add some unit tests and some code optimizes 2022-06-17 17:11:18 +08:00
石昌林 78b3728169 add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply

# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply

# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply

# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply

# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
2022-06-15 21:07:55 +08:00
Chris Lu b22ca85fbb
Merge pull request #3169 from shichanglin5/fix_auth
fix: When there is no access permission configured before startup, th…
2022-06-12 23:40:42 -07:00
shichanglin5 f01dd27752 fix: When there is no access permission configured before startup, the authentication does not take effect after configuring the permission after startup 2022-06-13 13:23:26 +08:00
chrislu e437ec5a68 3.10 2022-06-12 19:30:27 -07:00
chrislu d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
chrislu 444ac21050 go fmt 2022-06-11 09:51:11 -07:00
chrislu a74e926063 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-11 09:51:02 -07:00
chrislu 61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
Chris Lu 0a8b3948db
Merge pull request #3165 from blacktear23/reload-bug-fix 2022-06-10 07:47:43 -07:00
yulai.li 83f8cdd5cb Bug Fix: force reload filer page when upload/delete/rename/create directory 2022-06-10 22:39:21 +08:00
Chris Lu 3b8306c594
Merge pull request #3164 from guol-fnst/fix_typo 2022-06-10 07:36:05 -07:00
guol-fnst eabf4d4e64 fix typo 2022-06-10 22:03:11 +08:00
Chris Lu 36757d28a8
Merge pull request #3163 from kmlebedev/metric_replica_placement_mismatch
stats master_replica_placement_mismatch
2022-06-10 06:44:16 -07:00
chrislu 9f8b72a54d Revert "Merge pull request #3159 from shichanglin5/_duplicateUUID"
This reverts commit 37da689319, reversing
changes made to 00d53c34c4.
2022-06-10 06:38:17 -07:00
Konstantin Lebedev fb57aa431c stats master_replica_placement_mismatch 2022-06-10 15:30:40 +05:00
Chris Lu 873830a7bf
Merge pull request #3160 from shichanglin5/shichanglin5
fix: fix command prompt information
2022-06-10 00:52:11 -07:00
shichanglin5 a63559ee6c fix: fix command prompt information 2022-06-10 12:46:10 +08:00
Chris Lu 37da689319
Merge pull request #3159 from shichanglin5/_duplicateUUID
perf: Optimized volume handling duplicateUUID logic to avoid quitting…
2022-06-09 18:01:02 -07:00
Chris Lu 00d53c34c4
Merge pull request #3155 from qzhello/master
feat(filer.sync): add parameter fromTsMs.
2022-06-09 14:37:14 -07:00
shichanglin5 f5b0c04b14 perf: Optimized volume handling duplicateUUID logic to avoid quitting when volume is actualy normal
Under normal circumstances, there will be no problems, but when the
master is debugged in the local environment, the volume client cannot
communicate with the master normally, so the sendHeartBeat logic is
restarted, and a new connection is created to report the heartbeat. If
the master has not cleared the uuid of the volume at this time, then The
master will respond to volume duplicateUUIDS, and the volume service
will exit, but in fact the uuid of the volume is not duplicated
2022-06-09 20:41:16 +08:00
zhihao.qu cd5cca36a4 feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges 2022-06-09 10:53:19 +08:00
Chris Lu 4a5135961f
Merge pull request #3154 from kmlebedev/s3_test_invalid_range 2022-06-08 01:45:40 -07:00
Konstantin Lebedev 49f058da85 fix s3 tests:
ranged_request_invalid_range
ranged_request_empty_object
2022-06-08 13:25:13 +05:00
chrislu 8a1166c021 auto remove previous leftover socket 2022-06-08 00:32:09 -07:00
chrislu 633a4aed18 filer: fix customizable local socket file name
fix https://github.com/chrislusf/seaweedfs/issues/3147
2022-06-08 00:15:28 -07:00
chrislu acbc0286e2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-07 22:33:38 -07:00
chrislu ebd6097e6f fix: sequence read rate is slow when use filer.ChunkReadAt.ReadAt #3074
related to https://github.com/chrislusf/seaweedfs/pull/3074
2022-06-07 22:33:35 -07:00
Chris Lu f43ec9f363
Merge pull request #3130 from kmlebedev/fix_rm_parent_dir_via_nextcloud 2022-06-07 08:56:55 -07:00
Konstantin Lebedev c07820178f fix s3 tests
bucket_list_delimiter_prefix
bucket_list_delimiter_prefix_underscore
bucket_list_delimiter_prefix_ends_with_delimiter
2022-06-07 14:43:10 +05:00
Chris Lu 666e620b63
Merge pull request #3144 from chrislusf/dependabot/go_modules/google.golang.org/api-0.82.0
Bump google.golang.org/api from 0.81.0 to 0.82.0
2022-06-06 11:15:08 -07:00
Chris Lu 7ce519d4af
Merge pull request #3145 from chrislusf/dependabot/go_modules/github.com/stretchr/testify-1.7.2
Bump github.com/stretchr/testify from 1.7.1 to 1.7.2
2022-06-06 11:14:52 -07:00
Chris Lu f37fd2f883
Merge branch 'master' into dependabot/go_modules/google.golang.org/api-0.82.0 2022-06-06 10:43:02 -07:00
Chris Lu fba6a19d0b
Merge branch 'master' into dependabot/go_modules/github.com/stretchr/testify-1.7.2 2022-06-06 10:41:32 -07:00
Chris Lu a99536d523
Merge pull request #3143 from chrislusf/dependabot/go_modules/github.com/google/btree-1.1.1
Bump github.com/google/btree from 1.0.1 to 1.1.1
2022-06-06 10:39:33 -07:00
Chris Lu ae3a1c0235
Merge pull request #3146 from chrislusf/dependabot/go_modules/github.com/linxGnu/grocksdb-1.7.2
Bump github.com/linxGnu/grocksdb from 1.7.1 to 1.7.2
2022-06-06 10:39:15 -07:00
Chris Lu 99d5c3fddd
Merge pull request #3148 from chrislusf/dependabot/go_modules/google.golang.org/grpc-1.47.0
Bump google.golang.org/grpc from 1.46.2 to 1.47.0
2022-06-06 10:39:01 -07:00
dependabot[bot] af18bf3a3a
Bump google.golang.org/grpc from 1.46.2 to 1.47.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.46.2 to 1.47.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.46.2...v1.47.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 17:14:00 +00:00
dependabot[bot] f71cd52e2b
Bump github.com/linxGnu/grocksdb from 1.7.1 to 1.7.2
Bumps [github.com/linxGnu/grocksdb](https://github.com/linxGnu/grocksdb) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/linxGnu/grocksdb/releases)
- [Commits](https://github.com/linxGnu/grocksdb/compare/v1.7.1...v1.7.2)

---
updated-dependencies:
- dependency-name: github.com/linxGnu/grocksdb
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 17:12:50 +00:00
dependabot[bot] 5506986bb5
Bump google.golang.org/api from 0.81.0 to 0.82.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.81.0 to 0.82.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.81.0...v0.82.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 17:11:57 +00:00
dependabot[bot] 05c5230689
Bump github.com/google/btree from 1.0.1 to 1.1.1
Bumps [github.com/google/btree](https://github.com/google/btree) from 1.0.1 to 1.1.1.
- [Release notes](https://github.com/google/btree/releases)
- [Commits](https://github.com/google/btree/compare/v1.0.1...v1.1.1)

---
updated-dependencies:
- dependency-name: github.com/google/btree
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 17:11:55 +00:00
dependabot[bot] bcb83a03be
Bump github.com/stretchr/testify from 1.7.1 to 1.7.2
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.7.1...v1.7.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 17:11:51 +00:00
chrislu 78be3b0f6c fix compilation 2022-06-06 10:09:21 -07:00
chrislu 4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata
these metadata can change and are not used
2022-06-06 00:39:35 -07:00
chrislu cbf46de5f4 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-06 00:15:40 -07:00
chrislu 8402af0882 filer: detect ttl based on fs configure setting 2022-06-06 00:15:37 -07:00
Chris Lu 1e8d8029ec
Merge pull request #3140 from ningfdx/disable_xattr
add disableXAttr in mount option
2022-06-05 23:41:33 -07:00
ningfd f32142f6f5 add disableXAttr in mount option 2022-06-06 14:09:01 +08:00
chrislu 4a046e4de7 3.09 2022-06-05 19:00:24 -07:00