Commit graph

476 commits

Author SHA1 Message Date
SmsS4 f61490966f
Add time to first byte metric for s3 (#4768)
* Add time to first byte metric for s3

* Change second to millisecond
2023-08-21 00:42:39 -07:00
chrislu 6792db21dc ErrInvalidPart when completed parts has extra parts
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
2023-08-17 00:51:42 -07:00
SmoothDenis 51bcc219ea
s3api should return 500 code from filer (#4699) 2023-07-22 07:22:38 -07:00
Henco Appel 89542db2d0
Fix S3 API multipart upload on Windows (#4666)
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.

Co-authored-by: Hendrik Appel <happel@europe.altair.com>
2023-07-11 09:05:14 -07:00
SmsS4 17e91d2917
Use filerGroup for s3 buckets collection prefix (#4465)
* Use filerGroup for s3 buckets collection prefix

* Fix templates

* Remove flags

* Remove s3CollectionPrefix
2023-05-16 09:39:43 -07:00
Konstantin Lebedev d75a7b7f62
allow deleting only older empty dir without recursion (#4430) 2023-04-25 08:31:14 -07:00
Konstantin Lebedev 5614ad0000
fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter (#4399)
* fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter

* fix list with delimiter and start token

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-12 16:53:49 -07:00
Konstantin Lebedev 88b2bbadd1
try fix s3test test_bucket_listv2_delimiter_prefix (#4396) 2023-04-12 08:47:20 -07:00
Konstantin Lebedev 095cdb0c7f
fix test prefix is a directory (#4393)
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-12 00:08:13 -07:00
Konstantin Lebedev 44ad07276c
s3 fix get list of dir object key with slash suffix (#4391)
* s3 fix get list of dir object key with slash suffix
https://github.com/seaweedfs/seaweedfs/issues/3086

* list only entry dir eq prefix

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-11 23:52:35 -07:00
Konstantin Lebedev 130bc3e668
s3 fix get fake dir object key (#4390) 2023-04-11 07:36:22 -07:00
chrislu 6bfb91b539 update entry Attributes.Mtime = time.Now().Unix()
fix https://github.com/seaweedfs/seaweedfs/issues/4337
2023-03-25 22:02:57 -07:00
chrislu 5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
zemul 0bf56298d5
fix chunk.ModifiedTsNs (#4264)
* fix

* fix mtime s > ns

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-03-02 08:24:36 -08:00
LHHDZ db5515eada
Fix s3api_object_list_handlers returning contents less than the specified limit when more data actually exists (#4240)
Fix when the stored data is actually enough but s3api_object_list_handlers returns less than the specified limit

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-02-21 23:08:52 -08:00
chrislu 79d68a81fd text change 2023-02-07 15:00:41 -08:00
chrislu 999a126a3f typo 2023-02-07 14:59:47 -08:00
chrislu 1c2364936a detect invalid bucket name
fix https://github.com/seaweedfs/seaweedfs/issues/4143
2023-02-07 14:57:29 -08:00
chrislu 5de93fe442 refactoring 2023-02-07 14:45:20 -08:00
chrislu 81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
Guo Lei 5f993a5f20
Fix ide warning (#4082)
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
2022-12-26 00:03:47 -08:00
LHHDZ f84c13dabd
fix response not allowed (#4057) 2022-12-13 05:48:50 -08:00
LHHDZ bb289d7d69
sort tags by key (#4047)
Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-12-11 22:49:57 -08:00
Konstantin Lebedev 2b910d1cf8
avoid recursive deleting newly created empty directories (#4016) 2022-11-25 08:45:47 -08:00
chrislu 70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
LHHDZ 39f0246deb
fix 'NoSuchUpload' when upload part (#3968) 2022-11-11 04:53:53 -08:00
famosss df0662038c
feat:add a uuid after uploadid (#3963) 2022-11-10 07:17:29 -08:00
pavelzhurov abe4a61659
Bug fix: empty key in DeleteMultipleObjects request caused bucket delete (#3939) 2022-11-02 05:51:13 -07:00
chrislu 8b9957d461 add back "/" prefix if it is missing in object
fix https://github.com/seaweedfs/seaweedfs/issues/3737
2022-10-29 17:54:30 -07:00
chrislu ea2637734a refactor filer proto chunk variable from mtime to modified_ts_ns 2022-10-28 12:53:19 -07:00
famosss 34132b2c9f
fix: isTruncated returns problematic (#3907) 2022-10-24 21:15:37 -07:00
famosss 874fd197b5
feat: simplify a bit (#3905) 2022-10-24 17:58:26 -07:00
famosss 25e012d30b
fix: set user metadata key to lowercase (#3894)
* fix: set user metadata key to lowercase

* feat: simplify a bit
2022-10-24 17:29:52 -07:00
Guo Lei 97edb40275
Fix errinfo (#3893)
* types packages is imported more than onece

* Fix error response when format of --expires is wrong.
It MUST be in RFC 1123 date format.
2022-10-24 09:25:48 -07:00
LHHDZ a5b867af69
don't skip empty value at AssembleEntryWithAcp (#3855)
* add acl helper functionalities

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* add tests

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* remove 0 when create map

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* delete when empty at `AssembleEntryWithAcp`

`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp`  shouldn't skip empty value

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-10-13 22:50:44 -07:00
LHHDZ bf5e45b66a
add acl helper functionalities (#3831) 2022-10-12 20:15:16 -07:00
LHHDZ d21e2f523d
split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… (#3824)
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-10-11 20:14:14 -07:00
LHHDZ 3550692afc
change s3_account.go package to avoid cycle dependency (#3813) 2022-10-10 06:44:29 -07:00
Konstantin Lebedev ec46a34f33
avoid increasing the coordinates of metric labels (#3785) 2022-10-04 01:20:01 -07:00
LHHDZ e9584d9661
add ownership rest apis (#3765) 2022-10-01 19:18:00 -07:00
LHHDZ 3de1e19780
s3: sync bucket info from filer (#3759) 2022-09-29 12:29:01 -07:00
LHHDZ aacdcc4cad
s3: add account (#3753)
associate `Account` and `Identity` by accountId
2022-09-28 12:45:18 -07:00
chrislu 31922b2bf2 s3 to watch specific directories 2022-09-20 09:30:05 -07:00
Ryan Russell 6f7ef8711a
docs(s3api): readability improvements (#3696)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 03:13:21 -07:00
Ryan Russell ad3a3c8782
refactor(s3api_object_handlers): deleteMultipleObjectsLimmit -> `de… (#3695)
refactor(s3api_object_handlers): `deleteMultipleObjectsLimmit` -> `deleteMultipleObjectsLimit`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 03:12:48 -07:00
ImFantuan b64411bda8
fix 0 size (#3683) 2022-09-15 01:27:02 -07:00
famosss bb880d6253
IsTruncated returned by the ListMultipartUploads method is set to false by default (#3682)
* simplify a bit

* fix: IsTruncated returned by the ListMultipartUploads method is set to false by default
2022-09-15 00:27:15 -07:00
chrislu 21c0587900 go fmt 2022-09-14 23:06:44 -07:00
Ryan Russell d54eb9966f
refactor: Directory readability (#3665) 2022-09-14 10:11:31 -07:00
Konstantin Lebedev 8c3040db81
avoid DATA RACE on S3Options.localFilerSocket (#3571)
* avoid DATA RACE on S3Options.localFilerSocket
https://github.com/seaweedfs/seaweedfs/issues/3552

* copy localSocket
2022-09-01 10:33:23 -07:00