gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
7523 commits 28 branches 278 tags 135 MiB
Commit graph

355 commits

Author SHA1 Message Date
Konstantin Lebedev 3ac48cd540 audit log 2021-12-07 12:13:21 +05:00
chrislu 5c3b783310 s3: copy object API needs to escape special characters 
fix https://github.com/chrislusf/seaweedfs/issues/2482
 2021-12-04 23:24:53 -08:00
Konstantin Lebedev 40abae7caa Avoid forbiddening for HeadBucketHandler if owners are specified 
https://github.com/chrislusf/seaweedfs/issues/2434
 2021-11-17 14:42:24 +05:00
Konstantin Lebedev 42f4aa824b force overwrite s3-is-admin 
https://github.com/chrislusf/seaweedfs/issues/2433
 2021-11-11 11:34:01 +05:00
Bl1tz23 c683409e92 s3: add RWMutex to iam, use RLock for concurrent reading 2021-11-09 18:11:06 +03:00
Bl1tz23 5018b22f36 s3: fix potencial iam identities data race 2021-11-09 12:19:50 +03:00
Chris Lu 59d1435d65 s3: avoid possible attacks by version 2021-11-08 17:47:56 -08:00
Chris Lu 244841f080 adjust logs 2021-11-07 13:27:57 -08:00
Chris Lu 6e999f1176 s3: fix ListObject if more than 10000 objects 2021-11-07 12:39:36 -08:00
Chris Lu 55973e8572 log unknown access key 2021-11-07 12:37:46 -08:00
Chris Lu 35c37562bc check auth only when enabled 2021-11-03 12:11:36 -07:00
Chris Lu e1ab8b01d0 s3: adjust permission for HEAD bucket operation 
fix https://github.com/chrislusf/seaweedfs/issues/2417#issuecomment-958391856
 2021-11-02 16:28:55 -07:00
Chris Lu b25661c6df s3: skip permission checking for creating bucket if the bucket already exists 
fix https://github.com/chrislusf/seaweedfs/issues/2417

Rclone was trying to create the bucket even though the bucket already exists.
 2021-11-02 13:13:36 -07:00
Chris Lu 9cf756b188 S3: support CORS 
related to https://github.com/chrislusf/seaweedfs/issues/1271
 2021-10-31 18:06:43 -07:00
Chris Lu f17fa400d5 refactoring 2021-10-31 18:05:34 -07:00
Chris Lu 006b4ec0f8 refactoring 2021-10-31 18:02:08 -07:00
Konstantin Lebedev dc3ae5e8a6 DeleteBucketLifecycleConfiguration return 204 2021-10-28 19:34:37 +05:00
Konstantin Lebedev 2afb5a13af fix DeleteBucketLifecycleConfiguration 2021-10-28 18:30:33 +05:00
Chris Lu 54b6e0f3fd adjust logs 2021-10-27 23:46:07 -07:00
Konstantin Lebedev 235329a92a fix Tagging add xmlns 2021-10-20 19:40:22 +05:00
Konstantin Lebedev dc7e525cb9 fix Tagging test 2021-10-20 19:12:00 +05:00
Konstantin Lebedev 0b834600d5 fix PutObjectTaggingHandler Unmarshal 2021-10-20 17:58:06 +05:00
Konstantin Lebedev c78220a7f2 fix object tagging https://github.com/chrislusf/seaweedfs/issues/2389 2021-10-20 16:01:06 +05:00
Chris Lu 309f46e3af S3: fix upload limit if the size is more than 8GB 2021-10-18 14:14:18 -07:00
Chris Lu a6bb509cb7 adjust mime detection 2021-10-18 10:47:39 -07:00
Konstantin Lebedev dd2cc1acaf fix https://github.com/chrislusf/seaweedfs/issues/2387 2021-10-18 16:27:57 +05:00
Chris Lu 004e56c1a6 save updated lastTsNs 2021-10-17 02:01:20 -07:00
Chris Lu 71a8c80f6f s3: continuously listen to file updates 
fix https://github.com/chrislusf/seaweedfs/issues/2342
 2021-10-17 01:04:19 -07:00
Konstantin Lebedev 8de7915950 s3 mime detect 2021-10-14 15:03:11 +05:00
Eng Zer Jun a23bcbb7ec
refactor: move from io/ioutil to io and os package 
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-14 12:27:58 +08:00
Chris Lu b693a8d6b9
Merge pull request #2380 from kmlebedev/bucket_policy 
Bucket policy
 2021-10-13 15:29:53 -07:00
Konstantin Lebedev 9d6ffa0ea1 GetBucketLifecycleConfigurationHandler 2021-10-14 01:35:33 +05:00
Konstantin Lebedev 458145425e S3 MultipartUpload pass contentType to meta 2021-10-12 17:14:54 +05:00
Konstantin Lebedev be4b3ed509 AclHandlers 2021-10-11 15:03:56 +05:00
Chris Lu 84fdda85e7 go fmt 2021-09-19 12:06:15 -07:00
Chris Lu 5abdc0be77 s3: avoid overwriting object with ACL/LegalHold/Retension/LockConfiguration requests 2021-09-19 03:24:47 -07:00
Chris Lu 59dd271734 more glog 2021-09-19 00:29:51 -07:00
Chris Lu 4cbba2b1c6 add more glog to s3 2021-09-19 00:28:22 -07:00
Chris Lu 71175461ef add glog for s3 handlers 2021-09-19 00:18:59 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu f62df7f695 fix wrong srcPath checking 2021-09-05 23:37:40 -07:00
Chris Lu 491f7636f8 s3: CopyObject return http Status 400 Bad Request for non-existing source 
fix https://github.com/chrislusf/seaweedfs/issues/2306
 2021-09-03 17:38:39 -07:00
Chris Lu 5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Bl1tz23 1c94b3d013 merge master, resolve conflicts 2021-08-10 13:45:24 +03:00
Bl1tz23 e6e57db530 Add liveness\readiness probe for s3 api handler on /status path 2021-08-10 13:42:46 +03:00
Chris Lu 6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu 7359193e97 go fmt 2021-07-21 14:38:12 -07:00
Chris Lu f0042f62dd readable logs 2021-07-15 12:17:48 -07:00
Chris Lu ecce300964 s3 config read via grpc 2021-07-09 02:48:03 -07:00
Chris Lu a024254ad7 logging 2021-07-03 14:51:01 -07:00
Chris Lu d39b2689a5 S3 authorization: StreamingSigned enforces access control 
fix https://github.com/chrislusf/seaweedfs/issues/2180
 2021-07-03 14:50:53 -07:00
lyg 1a4db87e19 fix s3 metadata error with multipart upload 2021-07-02 11:00:42 +08:00
Chris Lu c6d4c16079 S3: add metadata with multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/2173
 2021-07-01 19:12:11 -07:00
Chris Lu b71c3cfba4 avoid possible empty object 2021-06-10 22:17:53 -07:00
Chris Lu 310e31424e adjust the error output 
fix https://github.com/chrislusf/seaweedfs/issues/2123
 2021-06-10 21:55:13 -07:00
Chris Lu 8b382a8209 refactor 2021-06-10 21:50:21 -07:00
Chris Lu 33b87244ef refactoring 2021-06-10 16:54:36 -07:00
Chris Lu 2b60e2abb1 only disallow streaming signed when no auth enabled 
fix https://github.com/chrislusf/seaweedfs/issues/2101
 2021-05-31 01:03:04 -07:00
Konstantin Lebedev b612d5aebd s3 test get w/ If-Match: bogus ETag 2021-05-24 16:59:44 +05:00
Konstantin Lebedev 69c768870b - object write cache control 
- object write expires
 2021-05-24 15:43:55 +05:00
Konstantin Lebedev 84dce32a57
Merge branch 'master' into head_check_all_chunks 2021-05-24 12:28:19 +05:00
Chris Lu 431684798b s3: add errors if requests are signed by no authentication is setup 
fix https://github.com/chrislusf/seaweedfs/issues/2075
 2021-05-21 14:08:47 -07:00
Konstantin Lebedev 03d1199d5f Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader" 
This reverts commit ac71117e
 2021-05-20 11:45:21 +05:00
Chris Lu 26a55bbb5c Adjust error message when bucket name conflicts with existing collections 
fix https://github.com/chrislusf/seaweedfs/issues/2069
 2021-05-12 22:30:39 -07:00
Chris Lu ac71117ee6 revert PR #1903 avoid http error: superfluous response.WriteHeader 2021-05-05 15:11:39 -07:00
Chris Lu e87e6ef33c s3: return 404 if bucket does not exist 
fix https://github.com/chrislusf/seaweedfs/issues/2039
 2021-05-02 21:30:37 -07:00
Konstantin Lebedev a48785c7df auth use bucket wild cards 2021-04-27 21:45:40 +05:00
Chris Lu f5de42fae3
Merge pull request #1975 from kmlebedev/iam_handlers 
IAM handlers
 2021-04-12 12:07:45 -07:00
Merlin Gaillard 4d4acc715e s3api: handle 304 response code from filer 2021-04-09 12:13:19 +02:00
Konstantin Lebedev ba175f81b5 add auth aws signV4 2021-04-08 17:40:47 +05:00
Chris Lu b1a86cf808 s3: copy object to itself 
fix https://github.com/chrislusf/seaweedfs/issues/1922
 2021-03-19 01:31:49 -07:00
Konstantin Lebedev bf94149920 add line 2021-03-17 23:41:34 +05:00
Konstantin Lebedev c5705e7a35 dir + slash key does not exist 
https://github.com/chrislusf/seaweedfs/issues/1917
 2021-03-17 23:40:42 +05:00
Konstantin Lebedev eb54993a4e Merge branch 'upstreamMaster' into check_chunkviews_mr 
# Conflicts:
#	weed/filer/filechunk_manifest.go
#	weed/filer/stream.go
#	weed/replication/repl_util/replication_util.go
#	weed/util/fasthttp_util.go
 2021-03-16 15:29:49 +05:00
wuh-fnst 050a7f9599 s3: count correction for the number of files in the dir 2021-03-16 09:22:03 +08:00
Konstantin Lebedev 3a3699867b Status PartialContent for Content-Range response 2021-03-15 23:30:22 +05:00
Chris Lu d084334ffd
Merge pull request #1883 from kmlebedev/passDelNonEmptyFolder 
s3 delete-objects pass OK for fail to delete non-empty folder
 2021-03-11 11:01:44 -08:00
Chris Lu 79280e1513 escape special characters 
fix https://github.com/chrislusf/seaweedfs/issues/1884
 2021-03-11 09:49:40 -08:00
wuheng 828fbf3fb7 s3: "isLast" returns true when the file does not exist 2021-03-11 15:20:50 +08:00
Chris Lu ca546f47e1 s3: escape object key if containing special characters 
fix https://github.com/chrislusf/seaweedfs/issues/1884
 2021-03-10 13:19:28 -08:00
Konstantin Lebedev bb3af2d70b no directoriesWithDeletion 2021-03-10 19:52:41 +05:00
Konstantin Lebedev 15b018da34 s3 delete-objects pass OK for fail to delete non-empty folder 2021-03-10 19:41:35 +05:00
Chris Lu 387c6f4218 Revert "delete the folder object in multi_object_delete" 
This reverts commit 2e89c8c9ae.
 2021-03-09 10:07:27 -08:00
Chris Lu 2e89c8c9ae delete the folder object in multi_object_delete 
fix https://github.com/chrislusf/seaweedfs/issues/1857
 2021-03-04 14:03:40 -08:00
Chris Lu 5511722420 s3: list permission is needed to list my buckets 
fix https://github.com/chrislusf/seaweedfs/issues/1837
 2021-03-01 12:41:55 -08:00
bingoohuang 352ac2f271 Merge remote-tracking branch 'origin/master' 2021-02-18 14:05:51 +08:00
bingoohuang eab6e31d34 use backticks instead of double quotes to avoid escaped additionally in regex 2021-02-18 14:05:28 +08:00
Chris Lu 8b169aa47c s3: Critical - S3 Delete Multiple Objects, delete the bucket 
fix https://github.com/chrislusf/seaweedfs/issues/1806
 2021-02-15 13:38:29 -08:00
Chris Lu 487e435679 adjust http max idle connections per host 
related to https://github.com/chrislusf/seaweedfs/issues/1802
 2021-02-12 03:47:15 -08:00
Chris Lu a0e84c4fbc go fmt 2021-02-10 23:41:05 -08:00
Chris Lu 5353e38469 S3: should return 204 on DELETE to nonexistent file 
fix https://github.com/chrislusf/seaweedfs/issues/1776
 2021-02-03 00:35:44 -08:00
Chris Lu 609daaf387 s3: DeleteMultipleObjectsHandler clean up leftover empty folders 
fix https://github.com/chrislusf/seaweedfs/issues/1772
 2021-02-01 10:49:17 -08:00
Chris Lu 990fa69bfe add back AdjustedUrl() related code 2021-01-28 14:36:29 -08:00
Chris Lu 9292796ec2 fmt 2021-01-28 14:29:14 -08:00
Chris Lu c2bf1a88ac delete from the deepest directory first when checking empty folders 2021-01-28 14:28:40 -08:00
Chris Lu e9d8201925 s3: batch purge empty folders 2021-01-28 13:20:06 -08:00
Chris Lu 00707ec00f mount: outsideContainerClusterMode proxy through filer 
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
 2021-01-24 19:01:58 -08:00
Chris Lu 6ca10725b8 Revert "mount: when outside cluster network, use filer as proxy to access volume servers" 
This reverts commit 096e088d7b.
 2021-01-24 03:15:19 -08:00
Chris Lu 096e088d7b mount: when outside cluster network, use filer as proxy to access volume servers 2021-01-24 01:41:38 -08:00
Chris Lu ccbdb38c89 s3: fix for listing objects if more than 1000 in the folder 
many existing tools have max key set to 1000
 2021-01-24 00:27:52 -08:00