Commit graph

392 commits

Author SHA1 Message Date
石昌林 be5c901bea Fix global counter key is incorrect & Add read lock when reading counter map to avoid memory problems 2022-06-20 19:16:53 +08:00
Chris Lu 9e2d6e897e
Merge pull request #3198 from guol-fnst/fix_statuscode
AbortMultipartUploadHandler should return 204 instead of 200
2022-06-19 22:57:13 -07:00
LHHDZ 8db9f13bc6
Merge branch 'master' into circuit_breaker 2022-06-20 13:42:18 +08:00
guol-fnst 1d77deccd0 AbortMultipartUploadHandler should return 204 instead of 200
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
2022-06-20 13:26:22 +08:00
石昌林 9e036df356 remove go.uber.org/atomic 2022-06-20 12:35:29 +08:00
石昌林 3dd60529c5 some code optimizations 2022-06-17 19:07:39 +08:00
石昌林 37df209195 add some unit tests and some code optimizes 2022-06-17 17:11:18 +08:00
zzq09494 62f74f5e3c add bucket label to s3 prometheus metrics 2022-06-16 13:21:25 +08:00
石昌林 78b3728169 add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply

# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply

# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply

# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply

# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
2022-06-15 21:07:55 +08:00
shichanglin5 f01dd27752 fix: When there is no access permission configured before startup, the authentication does not take effect after configuring the permission after startup 2022-06-13 13:23:26 +08:00
Konstantin Lebedev 49f058da85 fix s3 tests:
ranged_request_invalid_range
ranged_request_empty_object
2022-06-08 13:25:13 +05:00
Chris Lu f43ec9f363
Merge pull request #3130 from kmlebedev/fix_rm_parent_dir_via_nextcloud 2022-06-07 08:56:55 -07:00
Konstantin Lebedev c07820178f fix s3 tests
bucket_list_delimiter_prefix
bucket_list_delimiter_prefix_underscore
bucket_list_delimiter_prefix_ends_with_delimiter
2022-06-07 14:43:10 +05:00
creeew 02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
Konstantin Lebedev b86628f85d fix show empty dir
https://github.com/chrislusf/seaweedfs/issues/3086
2022-06-01 20:10:52 +05:00
chrislu 27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu f4a6da6cb2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 22:47:29 -07:00
chrislu 596c3860ca use final destination to resolve fs configuration
related to https://github.com/chrislusf/seaweedfs/issues/3075
2022-05-30 22:47:26 -07:00
Chris Lu ca01ce0524
Merge pull request #3122 from chrislusf/filer-sync-with-peers
Filer bootstrap from peers
2022-05-30 21:38:31 -07:00
chrislu a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
Chris Lu 730c9cf601
Merge pull request #3114 from kmlebedev/nextcloud
Create folders in s3 via nextcloud
2022-05-30 13:10:10 -07:00
Konstantin Lebedev af562e1a20 rm log info 2022-05-30 23:26:26 +05:00
Konstantin Lebedev 62e5e3822c list self dir
https://github.com/chrislusf/seaweedfs/issues/3086
2022-05-30 18:09:08 +05:00
Konstantin Lebedev 26f3646961 show fake dirs
https://github.com/chrislusf/seaweedfs/issues/3086
2022-05-30 15:03:27 +05:00
chrislu b8f3db0d46 s3: keep auth enabled in case identities are set to empty
fix https://github.com/chrislusf/seaweedfs/issues/3084
2022-05-29 19:00:03 -07:00
Konstantin Lebedev 54356211c2 list empty folder 2022-05-27 13:17:18 +05:00
chrislu 866fed1eab avoid grpc name conflict
fix https://github.com/chrislusf/seaweedfs/issues/3055
2022-05-15 21:31:29 -07:00
chrislu 4076d74813 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-15 00:43:39 -07:00
chrislu 29198720f2 s3: add grpc server to accept configuration changes 2022-05-15 00:43:37 -07:00
shichanglin5 688d55488c test(s3api_object_copy_handlers_test.go): some unit tests have been added to the processMetadata & processMetadataBytes methods of s3api_object_copy_handlers.go 2022-05-14 10:40:29 +08:00
shichanglin5 1166dead00 fix the problem of metadata and tagging loss when files are copied
by adding processing of metadata and tagging in s3 api CopyObject (judging whether to copy or overwrite according to the directive header)
2022-05-13 19:46:20 +08:00
chrislu f7366a9668 skip unix socket mode for windows
fix https://github.com/chrislusf/seaweedfs/issues/3013
2022-05-04 10:14:34 -07:00
chrislu f17cd0d5cd return false if not found
fix https://github.com/chrislusf/seaweedfs/issues/3011
2022-05-03 07:18:34 -07:00
chrislu 00c1dfec4f go fmt 2022-05-01 23:16:29 -07:00
Konstantin Lebedev 306cf70c4a avoid empty listMultipartUploads response 2022-04-27 19:27:44 +05:00
leyou240 89eb87c1d1
Merge branch 'master' into slices.SortFunc 2022-04-18 10:39:29 +08:00
justin 3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
guol-fnst 44d810d163 rename functions and remove uncessary check 2022-04-12 13:40:53 +08:00
guol-fnst 180aa88a92 check uploadid using object name hash string 2022-04-12 11:04:38 +08:00
guol-fnst 2232cfb5b7 Check object name and uploadID when processing multipart uploading 2022-04-11 19:53:44 +08:00
root f15a737a0a Add "Location:" in response when creating bucket
according to "https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html"
2022-04-06 13:56:27 +08:00
root b7e34016d4 fix error response in case of wrong expire time format 2022-04-02 11:01:33 +08:00
Konstantin Lebedev 691ed50957 use constant for ErrBucketNotEmpty 2022-03-31 08:44:48 +05:00
Konstantin Lebedev b6a2b43efe allowDeleteBucketNotEmpty 2022-03-30 22:46:13 +05:00
Konstantin Lebedev a8b15c676c s3 test bucket delete nonempty 2022-03-30 21:06:26 +05:00
Chris Lu ce8ef60c59
Merge pull request #2840 from kmlebedev/fix_multi_object_delete_key_limit
fix test_s3.test_multi_object_delete_key_limit and test_multi_objectv2_delete_key_limit
2022-03-30 01:27:13 -07:00
Konstantin Lebedev 6317f6a8a9 fix Description 2022-03-30 13:04:15 +05:00
Konstantin Lebedev 993283bb1d revert sort.Search 2022-03-29 19:53:12 +05:00
Konstantin Lebedev 5b90a39954 fix s3 tests:
multipart_upload_incorrect_etag
multipart_resend_first_finishes_last
2022-03-29 19:21:09 +05:00
Konstantin Lebedev 9231971c88 fix test_s3.test_multi_object_delete_key_limit and
test_multi_objectv2_delete_key_limit
2022-03-29 16:55:09 +05:00