Chris Lu
83012f543a
s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file
2020-08-26 08:50:24 -07:00
Chris Lu
81e5124faf
s3: list bucket permission change from admin to read
...
fix https://github.com/chrislusf/seaweedfs/issues/1430
2020-08-24 11:22:45 -07:00
Chris Lu
c45ba5d7d4
fix listObjectsV2 response format
...
fix https://github.com/chrislusf/seaweedfs/issues/1426 issue 1
2020-08-19 12:08:04 -07:00
Chris Lu
c647deace1
file size support set file length
...
use Attr.FileSize and TotalChunkSize to determine file size
2020-08-15 09:32:47 -07:00
Chris Lu
f86c7d911a
remove println
2020-08-09 14:42:25 -07:00
Chris Lu
9ecc1170a3
istObjects和listObjectsV2不能查询子目录 #1418
...
fix https://github.com/chrislusf/seaweedfs/issues/1418
2020-08-09 14:35:53 -07:00
Chris Lu
3f4aff5dde
s3: fix delimiter in list response
2020-08-09 09:09:35 -07:00
Chris Lu
828a5ae429
check signature only when auth is enabled
2020-08-08 09:11:40 -07:00
Chris Lu
2b74abf766
S3: configurable access for anonymous user
...
fix https://github.com/chrislusf/seaweedfs/issues/1413
2020-08-06 03:41:34 -07:00
limd
b41b7ea4d0
fix spark read s3 bug (sc.binaryFiles)
2020-08-01 01:08:30 +08:00
Chris Lu
a566bfc6e1
s3: use bucket in the domain
...
fix https://github.com/chrislusf/seaweedfs/issues/1405
2020-07-28 08:47:27 -07:00
limd
92c32f9d46
PutObject: fix create folder bug
2020-07-28 00:58:42 +08:00
Chris Lu
54cd582653
refactor
2020-07-26 12:58:58 -07:00
Chris Lu
c50ec1599c
auth for presigned put object part
2020-07-26 12:09:50 -07:00
Chris Lu
ea0180de4e
auth for presigned put operation
2020-07-26 12:06:45 -07:00
popstk
a09fcfcf20
fix s3api auth bug
2020-07-25 17:14:49 +08:00
popstk
feb52400be
fix s3api copy object handler SerializationError
2020-07-25 15:52:31 +08:00
Chris Lu
bfabb4159b
refactoring
2020-07-11 09:11:15 -07:00
Chris Lu
a4e3cffe0a
S3: DeleteObject API also support deleting a folder recursively
2020-06-22 10:01:00 -07:00
Rinat Shigapov
320d70d305
return last modified in UTC
2020-06-19 10:26:40 +03:00
Chris Lu
00f96d86a1
fix typo
2020-06-11 10:53:25 -07:00
Rinat Shigapov
283b749ff1
keep 204 response
2020-06-11 19:00:34 +03:00
Rinat Shigapov
fafc41a27f
return xml encoded NotFound status code for s3 delete
2020-06-11 17:53:15 +03:00
Rinat Shigapov
2d2c5dfa39
proxy status code of filer delete response
2020-06-11 15:00:47 +03:00
Chris Lu
a25a8d8822
s3: print out time in UTC format
...
fix https://github.com/chrislusf/seaweedfs/issues/1297
2020-04-29 21:35:24 -07:00
Chris Lu
9e72e9e4b8
able to subscribe any topic from any point of time
2020-04-29 17:40:08 -07:00
Chris Lu
ed3cf811f5
refactoring
2020-04-29 13:26:02 -07:00
Chris Lu
d57559d3f7
chunk rememober encryption or not
2020-04-29 02:41:47 -07:00
Chris Lu
ec2eb8bc48
add If-None-Match and If-Modified-Since
...
fix https://github.com/chrislusf/seaweedfs/issues/1269
2020-04-08 08:12:00 -07:00
Chris Lu
86398d1efe
S3 API: set deletion to recursive
...
fix https://github.com/chrislusf/seaweedfs/issues/1251
2020-03-26 11:05:54 -07:00
Chris Lu
dd5b582d05
go fmt
2020-03-26 00:09:01 -07:00
Chris Lu
c07bcd5065
refactoring
2020-03-23 00:30:02 -07:00
Chris Lu
654a69ff52
refactoring
2020-03-23 00:06:24 -07:00
Chris Lu
fbca6b29bd
refactoring
2020-03-22 23:52:55 -07:00
Chris Lu
c4bea45099
S3 API: fix DeleteMultipleObjectsHandler
...
fix https://github.com/chrislusf/seaweedfs/issues/1241
2020-03-20 14:17:31 -07:00
Chris Lu
9b3109a5d8
filer: processing all response headers, no pass through to volume server
...
* filer calculate MD5 etag
* filer handle response headers, instread of pass it to volume servers
2020-03-08 15:42:44 -07:00
Chris Lu
afb20de14c
breaks dependency loop
2020-03-07 17:01:39 -08:00
Chris Lu
8645283a7b
fuse mount: avoid lookup nil entry
...
fix https://github.com/chrislusf/seaweedfs/issues/1221
2020-03-07 16:51:46 -08:00
Chris Lu
f90c43635d
refactoring
2020-03-04 00:39:47 -08:00
Chris Lu
ed0acd1722
go fmt
2020-02-26 16:52:57 -08:00
Chris Lu
97ab8a1976
remove ctx if possible
2020-02-25 22:23:59 -08:00
Chris Lu
892e726eb9
avoid reusing context object
...
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
bc38b72a20
s3: implemented DeleteMultipleObjects
2020-02-25 14:38:36 -08:00
Chris Lu
35dde56711
refactoring
2020-02-25 12:58:45 -08:00
Chris Lu
986d63cd5e
s3: increase list objects or parts limit from 1000 to 10000
...
10x of Amazon S3 limits
2020-02-25 11:45:40 -08:00
Chris Lu
7d10fdf737
fix directory lookup nil
2020-02-25 11:13:06 -08:00
Chris Lu
6ab7368ef2
filer: dynamically create bucket under /buckets folder
2020-02-24 22:28:45 -08:00
Chris Lu
4ed6b584e2
s3: access control limited by bucket
2020-02-22 21:34:18 -08:00
Chris Lu
48b7ad5fa8
s3: deny anonymous type
2020-02-22 14:01:04 -08:00
Chris Lu
dbabdd418e
add unhandled request auth type
...
fix
2020-02-18 11:43:57.396699 I | http: panic serving 172.28.0.43:50658: runtime error: invalid memory address or nil pointer dereference
goroutine 595 [running]:
net/http.(*conn).serve.func1(0xc0001fe3c0)
/usr/lib/go/src/net/http/server.go:1767 +0x13b
panic(0x55c4e35f3820, 0x55c4e48b3c40)
/usr/lib/go/src/runtime/panic.go:679 +0x1b6
github.com/chrislusf/seaweedfs/weed/s3api.(*IdentityAccessManagement).authRequest(0xc0004b84e0, 0xc000115900, 0xc0000bb650, 0x1, 0x1, 0x55c4e399d740)
/go/src/github.com/chrislusf/seaweedfs/weed/s3api/auth_credentials.go:143 +0x11c
github.com/chrislusf/seaweedfs/weed/s3api.(*IdentityAccessManagement).Auth.func1(0x55c4e3994c40, 0xc0007808c0, 0xc000115900)
/go/src/github.com/chrislusf/seaweedfs/weed/s3api/auth_credentials.go:111 +0x5e
net/http.HandlerFunc.ServeHTTP(0xc0004b87e0, 0x55c4e3994c40, 0xc0007808c0, 0xc000115900)
/usr/lib/go/src/net/http/server.go:2007 +0x46
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ba000, 0x55c4e3994c40, 0xc0007808c0, 0xc000115700)
/root/go/pkg/mod/github.com/gorilla/mux@v1.7.3/mux.go:212 +0xe4
net/http.serverHandler.ServeHTTP(0xc00011e0e0, 0x55c4e3994c40, 0xc0007808c0, 0xc000115700)
/usr/lib/go/src/net/http/server.go:2802 +0xa6
net/http.(*conn).serve(0xc0001fe3c0, 0x55c4e399d680, 0xc000894180)
/usr/lib/go/src/net/http/server.go:1890 +0x877
created by net/http.(*Server).Serve
/usr/lib/go/src/net/http/server.go:2927 +0x390
2020-02-18 09:16:04 -08:00
Chris Lu
0a57db7026
Merge pull request #1200 from chrislusf/add_s3_acl
...
Add s3 ACL
2020-02-17 12:53:30 -08:00
Chris Lu
4cdde5f569
configuration stores the identity list
2020-02-17 12:31:59 -08:00
Chris Lu
96c1ae8471
refactoring the close http response
2020-02-14 09:46:36 -08:00
Chris Lu
cf5064d702
properly close http response
2020-02-14 09:09:15 -08:00
Chris Lu
b4abe3c081
unused
2020-02-09 18:02:17 -08:00
Chris Lu
f3ce3166ad
add streaming v4
2020-02-09 17:42:17 -08:00
Chris Lu
b90ad6f452
add v2 support
2020-02-09 16:02:05 -08:00
Chris Lu
9ed364f053
support acl
2020-02-09 14:30:02 -08:00
Chris Lu
b2743afaee
remove println
2020-01-31 00:21:18 -08:00
Chris Lu
a80ecbfe84
s3: add s3 copy
...
fix https://github.com/chrislusf/seaweedfs/issues/1190
2020-01-31 00:11:12 -08:00
Chris Lu
72a64a5cf8
use the same context object in order to retry
2020-01-26 14:42:11 -08:00
Chris Lu
c48fc8b4de
grpc send error via response instead of grpc error
2020-01-25 09:17:19 -08:00
Chris Lu
bbb6ebc3c0
filer: DeleteFolderChildren for deleting large folders
2019-12-13 00:23:05 -08:00
Chris Lu
0fa1269bc7
filer: streaming file listing
2019-12-13 00:22:37 -08:00
Chris Lu
ba73c053c3
change uuid package
...
fix https://github.com/chrislusf/seaweedfs/issues/1126
2019-11-18 19:24:37 -08:00
Lei Liu
2cb348a2c2
s3 api: fix listbucket common_prefixes issue
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-09-29 21:15:40 +08:00
Chris Lu
972e881d48
add more logs
...
add more logs
2019-09-16 09:48:30 -07:00
Chris Lu
20ae559437
add logs
...
add logs for https://github.com/chrislusf/seaweedfs/issues/1064
2019-09-16 08:19:10 -07:00
Chris Lu
862c2cb8e6
go fmt
2019-07-21 21:51:38 -07:00
Chris Lu
1e2127e261
s3: adding Location for multipart upload
...
fix https://github.com/chrislusf/seaweedfs/issues/1002
2019-07-08 12:37:45 -07:00
Chris Lu
62843991f2
s3: adjust object key
2019-07-08 12:37:20 -07:00
Chris Lu
3fa1f150d9
refactoring
2019-06-22 20:04:56 -07:00
Chris Lu
c789b496d8
use cached grpc client
2019-04-05 20:31:58 -07:00
Chris Lu
20dcb44077
fix tests
2019-04-01 23:59:31 -07:00
Chris Lu
78b9db34d5
weed s3: generate time format acceptable by aws cli s3
...
fix https://github.com/chrislusf/seaweedfs/issues/911
2019-04-01 16:13:29 -07:00
Chris Lu
c5a3ff1c35
weed s3: multipart upload goes to the right bucket
...
fix https://github.com/chrislusf/seaweedfs/issues/908
2019-03-31 11:10:19 -07:00
Chris Lu
55bab1b456
add context.Context
2019-03-15 17:20:24 -07:00
Chris Lu
cece860bfd
add context to all filer APIs
2019-03-15 15:55:34 -07:00
Chris Lu
cb07d15254
add namespace for s3
2019-02-27 00:21:37 -08:00
Chris Lu
70ac2f6ea8
add namespace for ListAllMyBucketsResult
2019-02-26 23:23:04 -08:00
Chris Lu
77b9af531d
adding grpc mutual tls
2019-02-18 12:11:52 -08:00
Chris Lu
3339325334
fix s3 ListAllMyBucketsResult to work with s3cmd
2019-01-02 11:36:29 -08:00
Chris Lu
43db7ac123
fix spelling
2019-01-02 02:38:32 -08:00
Chris Lu
39ea8484c1
adjust proto API
2018-12-11 07:17:10 -08:00
Chris Lu
a4ceb051a7
use MD5 for ETag to be consistent with Amazon S3
2018-11-08 21:41:02 -08:00
Chris Lu
fdc8a267de
remove unused function
2018-09-22 22:11:13 -07:00
Chris Lu
d9b32db8d7
adjust uploads to a hidden folder
2018-09-19 22:03:16 -07:00
Chris Lu
f5471bcebf
add bucket name in the redirection
2018-09-19 22:01:41 -07:00
Chris Lu
bea4f6ca14
go fmt
2018-09-12 01:00:57 -07:00
Chris Lu
f628953c33
cleanup tmp upload directories
2018-09-12 01:00:51 -07:00
Chris Lu
f6d8525d1d
working S3 multipart uploads
2018-09-12 00:46:12 -07:00
Chris Lu
439e88b5a6
fix format error
2018-09-11 13:01:51 -07:00
Chris Lu
296acc4a0a
go fmt
2018-09-09 16:26:11 -07:00
Chris Lu
164091c269
add s3 multipart upload
2018-09-09 16:25:43 -07:00
Chris Lu
dd37f0540f
add extended attributes
2018-09-07 13:12:52 -07:00
Chris Lu
d614716ff2
fix compilation error
2018-09-04 00:43:46 -07:00
Chris Lu
ce6a2140a2
preparing to support S3 multipart uploads
2018-09-04 00:42:44 -07:00
Chris Lu
7ad5a90127
go fmt
2018-09-03 16:47:00 -07:00
Chris Lu
2ed6d8cca6
refactoring filer rm
2018-09-03 13:16:26 -07:00
Chris Lu
03f852c799
refactoring
2018-09-03 13:03:16 -07:00
Chris Lu
0b0ece9649
add place holders for multiplarts upload
2018-09-03 11:38:10 -07:00
Chris Lu
88f1d32cc4
add s3ChunkedReader
...
fix https://github.com/chrislusf/seaweedfs/issues/718
2018-09-02 14:20:47 -07:00
Chris Lu
77fadab560
go fmt
2018-07-24 01:38:08 -07:00
Chris Lu
6039b25bc9
s3API working with listing objects
2018-07-23 01:55:26 -07:00
Chris Lu
3c7ffa2a2c
go fmt
2018-07-23 01:16:17 -07:00
Chris Lu
ebad3a44ab
s3api do not proxy directory requests
2018-07-23 01:15:59 -07:00
Chris Lu
c133e72f62
print out unsupported methods
2018-07-22 19:55:53 -07:00
Chris Lu
19c537b687
releasing 0.91
2018-07-22 02:12:02 -07:00
Chris Lu
20853972b2
s3 API add ListObjectsV2Handler
2018-07-22 02:04:07 -07:00
Chris Lu
0655151b8c
go fmt
2018-07-22 01:15:11 -07:00
Chris Lu
6319d84f42
s3 API add ListObjectsV1
2018-07-22 01:14:36 -07:00
Chris Lu
49375d6031
adjust DELETE response to S3 format
2018-07-21 19:12:44 -07:00
Chris Lu
5068b6ae7d
fix compilation
2018-07-21 19:06:09 -07:00
Chris Lu
12825938d7
unused method
2018-07-21 18:50:56 -07:00
Chris Lu
8f240276db
s3 add HEAD DELETE
2018-07-21 18:49:47 -07:00
Chris Lu
7e2031b18f
go fmt
2018-07-21 17:39:10 -07:00
Chris Lu
8480008a9a
add s3 upload, and removing mono and multi part upload analyzer
...
removing mono and multi part upload analyzer, which were used just to determine the file name
2018-07-21 10:39:02 -07:00
Chris Lu
dad733086a
fix string printing
2018-07-20 00:58:10 -07:00
Chris Lu
568132c0c9
delete bucket also removes collection data
2018-07-20 00:10:01 -07:00
Chris Lu
feea33d5e0
add HeadBucket
2018-07-19 01:43:27 -07:00
Chris Lu
c7228fefa7
add bucket creation and deletion
...
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
2018-07-19 01:21:44 -07:00
Chris Lu
702fbdf731
refactoring
2018-07-18 02:39:12 -07:00
Chris Lu
834a25f084
add list all my buckets
2018-07-18 02:37:09 -07:00