Commit graph

88 commits

Author SHA1 Message Date
Chris Lu 9b3109a5d8 filer: processing all response headers, no pass through to volume server
* filer calculate MD5 etag
* filer handle response headers, instread of pass it to volume servers
2020-03-08 15:42:44 -07:00
Chris Lu afb20de14c breaks dependency loop 2020-03-07 17:01:39 -08:00
Chris Lu 8645283a7b fuse mount: avoid lookup nil entry
fix https://github.com/chrislusf/seaweedfs/issues/1221
2020-03-07 16:51:46 -08:00
Chris Lu f90c43635d refactoring 2020-03-04 00:39:47 -08:00
Chris Lu ed0acd1722 go fmt 2020-02-26 16:52:57 -08:00
Chris Lu 97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu 892e726eb9 avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu bc38b72a20 s3: implemented DeleteMultipleObjects 2020-02-25 14:38:36 -08:00
Chris Lu 35dde56711 refactoring 2020-02-25 12:58:45 -08:00
Chris Lu 986d63cd5e s3: increase list objects or parts limit from 1000 to 10000
10x of Amazon S3 limits
2020-02-25 11:45:40 -08:00
Chris Lu 7d10fdf737 fix directory lookup nil 2020-02-25 11:13:06 -08:00
Chris Lu 6ab7368ef2 filer: dynamically create bucket under /buckets folder 2020-02-24 22:28:45 -08:00
Chris Lu 4ed6b584e2 s3: access control limited by bucket 2020-02-22 21:34:18 -08:00
Chris Lu 48b7ad5fa8 s3: deny anonymous type 2020-02-22 14:01:04 -08:00
Chris Lu dbabdd418e add unhandled request auth type
fix

2020-02-18 11:43:57.396699 I | http: panic serving 172.28.0.43:50658: runtime error: invalid memory address or nil pointer dereference
goroutine 595 [running]:
net/http.(*conn).serve.func1(0xc0001fe3c0)
    /usr/lib/go/src/net/http/server.go:1767 +0x13b
panic(0x55c4e35f3820, 0x55c4e48b3c40)
    /usr/lib/go/src/runtime/panic.go:679 +0x1b6
github.com/chrislusf/seaweedfs/weed/s3api.(*IdentityAccessManagement).authRequest(0xc0004b84e0, 0xc000115900, 0xc0000bb650, 0x1, 0x1, 0x55c4e399d740)
    /go/src/github.com/chrislusf/seaweedfs/weed/s3api/auth_credentials.go:143 +0x11c
github.com/chrislusf/seaweedfs/weed/s3api.(*IdentityAccessManagement).Auth.func1(0x55c4e3994c40, 0xc0007808c0, 0xc000115900)
    /go/src/github.com/chrislusf/seaweedfs/weed/s3api/auth_credentials.go:111 +0x5e
net/http.HandlerFunc.ServeHTTP(0xc0004b87e0, 0x55c4e3994c40, 0xc0007808c0, 0xc000115900)
    /usr/lib/go/src/net/http/server.go:2007 +0x46
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ba000, 0x55c4e3994c40, 0xc0007808c0, 0xc000115700)
    /root/go/pkg/mod/github.com/gorilla/mux@v1.7.3/mux.go:212 +0xe4
net/http.serverHandler.ServeHTTP(0xc00011e0e0, 0x55c4e3994c40, 0xc0007808c0, 0xc000115700)
    /usr/lib/go/src/net/http/server.go:2802 +0xa6
net/http.(*conn).serve(0xc0001fe3c0, 0x55c4e399d680, 0xc000894180)
    /usr/lib/go/src/net/http/server.go:1890 +0x877
created by net/http.(*Server).Serve
    /usr/lib/go/src/net/http/server.go:2927 +0x390
2020-02-18 09:16:04 -08:00
Chris Lu 0a57db7026
Merge pull request #1200 from chrislusf/add_s3_acl
Add s3 ACL
2020-02-17 12:53:30 -08:00
Chris Lu 4cdde5f569 configuration stores the identity list 2020-02-17 12:31:59 -08:00
Chris Lu 96c1ae8471 refactoring the close http response 2020-02-14 09:46:36 -08:00
Chris Lu cf5064d702 properly close http response 2020-02-14 09:09:15 -08:00
Chris Lu b4abe3c081 unused 2020-02-09 18:02:17 -08:00
Chris Lu f3ce3166ad add streaming v4 2020-02-09 17:42:17 -08:00
Chris Lu b90ad6f452 add v2 support 2020-02-09 16:02:05 -08:00
Chris Lu 9ed364f053 support acl 2020-02-09 14:30:02 -08:00
Chris Lu b2743afaee remove println 2020-01-31 00:21:18 -08:00
Chris Lu a80ecbfe84 s3: add s3 copy
fix https://github.com/chrislusf/seaweedfs/issues/1190
2020-01-31 00:11:12 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu c48fc8b4de grpc send error via response instead of grpc error 2020-01-25 09:17:19 -08:00
Chris Lu bbb6ebc3c0 filer: DeleteFolderChildren for deleting large folders 2019-12-13 00:23:05 -08:00
Chris Lu 0fa1269bc7 filer: streaming file listing 2019-12-13 00:22:37 -08:00
Chris Lu ba73c053c3 change uuid package
fix https://github.com/chrislusf/seaweedfs/issues/1126
2019-11-18 19:24:37 -08:00
Lei Liu 2cb348a2c2 s3 api: fix listbucket common_prefixes issue
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-09-29 21:15:40 +08:00
Chris Lu 972e881d48 add more logs
add more logs
2019-09-16 09:48:30 -07:00
Chris Lu 20ae559437 add logs
add logs for https://github.com/chrislusf/seaweedfs/issues/1064
2019-09-16 08:19:10 -07:00
Chris Lu 862c2cb8e6 go fmt 2019-07-21 21:51:38 -07:00
Chris Lu 1e2127e261 s3: adding Location for multipart upload
fix https://github.com/chrislusf/seaweedfs/issues/1002
2019-07-08 12:37:45 -07:00
Chris Lu 62843991f2 s3: adjust object key 2019-07-08 12:37:20 -07:00
Chris Lu 3fa1f150d9 refactoring 2019-06-22 20:04:56 -07:00
Chris Lu c789b496d8 use cached grpc client 2019-04-05 20:31:58 -07:00
Chris Lu 20dcb44077 fix tests 2019-04-01 23:59:31 -07:00
Chris Lu 78b9db34d5 weed s3: generate time format acceptable by aws cli s3
fix https://github.com/chrislusf/seaweedfs/issues/911
2019-04-01 16:13:29 -07:00
Chris Lu c5a3ff1c35 weed s3: multipart upload goes to the right bucket
fix https://github.com/chrislusf/seaweedfs/issues/908
2019-03-31 11:10:19 -07:00
Chris Lu 55bab1b456 add context.Context 2019-03-15 17:20:24 -07:00
Chris Lu cece860bfd add context to all filer APIs 2019-03-15 15:55:34 -07:00
Chris Lu cb07d15254 add namespace for s3 2019-02-27 00:21:37 -08:00
Chris Lu 70ac2f6ea8 add namespace for ListAllMyBucketsResult 2019-02-26 23:23:04 -08:00
Chris Lu 77b9af531d adding grpc mutual tls 2019-02-18 12:11:52 -08:00
Chris Lu 3339325334 fix s3 ListAllMyBucketsResult to work with s3cmd 2019-01-02 11:36:29 -08:00
Chris Lu 43db7ac123 fix spelling 2019-01-02 02:38:32 -08:00
Chris Lu 39ea8484c1 adjust proto API 2018-12-11 07:17:10 -08:00
Chris Lu a4ceb051a7 use MD5 for ETag to be consistent with Amazon S3 2018-11-08 21:41:02 -08:00