Chris Lu
c057dd1f64
add back upload retries
2020-10-30 02:16:34 -07:00
Chris Lu
19772d70d7
print for debugging
2020-10-24 00:12:02 -07:00
Chris Lu
ef9c32ea0d
adjust compression threshold
2020-10-23 23:05:55 -07:00
Chris Lu
29c9fa2ef2
byte buffer for uploading
2020-10-23 11:29:51 -07:00
Chris Lu
2579edbc60
fix bug found by tests
2020-10-23 10:05:40 -07:00
Chris Lu
e0c8507d93
memory pool to reduce memory allocation
2020-10-23 00:41:40 -07:00
Chris Lu
c31b254248
mount: shortcut when there is only one chunk
2020-10-20 23:48:29 -07:00
Chris Lu
f64252023e
Revert "a better byte buffer pool"
...
This reverts commit 59c6422777
.
2020-10-20 23:21:54 -07:00
Chris Lu
59c6422777
a better byte buffer pool
2020-10-20 22:53:58 -07:00
Chris Lu
35f6518c36
weed upload: usePublicUrl was not being used on big files
...
fix https://github.com/chrislusf/seaweedfs/issues/1492
2020-09-25 02:41:38 -07:00
Chris Lu
daf0a449f7
properly cancel context for streaming grpc
2020-09-09 12:07:15 -07:00
Chris Lu
387ab6796f
filer: cross cluster synchronization
2020-09-09 11:21:23 -07:00
Chris Lu
50f7e2e89f
add log messages
2020-08-26 09:51:20 -07:00
Chris Lu
5f55a87101
close http response
2020-08-24 00:32:44 -07:00
Chris Lu
0b301b0b47
consume all response body in order to release requests
2020-08-23 21:32:29 -07:00
Chris Lu
003d48da21
adjust logs
2020-08-15 19:55:28 -07:00
Chris Lu
83cad3da79
add retry file upload
2020-08-11 20:30:11 -07:00
Chris Lu
20e2ac1add
filer: store md5 metadata for files uploaded by filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00
Chris Lu
4ecfa9879d
volume: report Content-MD5 in response header
2020-08-06 05:22:53 -07:00
Chris Lu
1b3a80dd3d
non-fatal error
2020-08-01 13:46:52 -07:00
Chris Lu
bbc8ae110f
report errors
2020-07-14 11:27:00 -07:00
Chris Lu
ddec7b2bb9
go fmt
2020-07-14 07:34:16 -07:00
Chris Lu
e4ef3814e7
fix compilation
2020-07-10 10:02:59 -07:00
Chris Lu
2ff37ccdbd
testing parsing the upload
...
The mime type is always the value passed in.
Compress or not depends on the content detection, file name extension, and compression ratio.
If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu
fe60db404a
prepare for zstd
2020-06-23 09:12:02 -07:00
Chris Lu
3080c197e3
rename UnCompressData to DecompressData
2020-06-20 08:16:16 -07:00
Chris Lu
97239ce6f1
rename filechunk is_gzipped to is_compressed
2020-06-20 08:15:49 -07:00
Chris Lu
e912fd15e3
renaming
2020-06-19 22:45:27 -07:00
Chris Lu
16fe132a20
detect mime type before replicating to other volume servers
2020-06-19 22:11:36 -07:00
Chris Lu
39e72fb23c
go fmt
2020-05-10 03:50:30 -07:00
Chris Lu
78422c2f69
Merge pull request #1298 from song-zhang/master
...
add fsync option for benchmark
2020-04-30 20:16:42 -07:00
Chris Lu
871efa4fc1
refactoring
...
some previous chunk etag was using md5, which should be wrong.
2020-04-30 17:20:47 -07:00
zhangsong
d819865fe7
add fsync option for benchmark
2020-04-30 23:31:08 +08:00
Chris Lu
9cacaf5eca
better detect file mime type
2020-04-14 11:32:31 -07:00
Chris Lu
3f1d79512f
only detect when not sure about the types
2020-04-14 11:02:05 -07:00
Chris Lu
d2d1597850
volume: detect gzip effectiveness only when content type is empty
...
revert part of 2286eda575
2020-04-14 10:52:53 -07:00
Chris Lu
8a73276602
refactor a bit
2020-04-08 22:33:36 -07:00
Chris Lu
bf270d9e8c
filer: able to tail meta data changes
2020-04-05 00:51:16 -07:00
Chris Lu
eedd33dda3
refactoring
2020-03-28 13:41:58 -07:00
Chris Lu
e1911760a7
refactoring
2020-03-27 00:30:55 -07:00
Chris Lu
2286eda575
sampling whether the data can be gzipped
2020-03-26 23:50:48 -07:00
Chris Lu
a5ce359999
refactoring
2020-03-26 22:46:49 -07:00
Chris Lu
6d87a0c5f5
batch delete orphan data by file keys
2020-03-25 02:21:15 -07:00
Chris Lu
6ff9e2835e
filer: skip resizing image if width or height larger than original image
...
fix https://github.com/chrislusf/seaweedfs/issues/1239#issuecomment-602140779
2020-03-21 22:16:00 -07:00
Chris Lu
308688c8d0
filer: add back image resizing capability
2020-03-20 20:31:11 -07:00
Chris Lu
f251d03673
refactoring
2020-03-20 15:46:16 -07:00
Chris Lu
11372dd548
refactoring
2020-03-20 15:38:04 -07:00
Chris Lu
d848d08944
use public url also for deletions
2020-03-18 11:16:45 -07:00
Chris Lu
6566c8e114
weed upload: add usePublicUrl option
...
fix https://github.com/chrislusf/seaweedfs/issues/1236
2020-03-18 10:50:53 -07:00
Chris Lu
f9b3750ad1
volume: handle repeated writes response
2020-03-15 04:16:00 -07:00
Chris Lu
ee635dcc36
refactoring
2020-03-14 14:08:00 -07:00
Chris Lu
18cda6adbb
cleaner fix
2020-03-14 13:55:32 -07:00
Chris Lu
17efa361f9
fix possible nil
2020-03-14 13:53:03 -07:00
Chris Lu
3934c69757
clean up
2020-03-08 21:54:47 -07:00
Chris Lu
2e3f6ad3a9
filer: remember content is gzipped or not
2020-03-08 21:39:33 -07:00
Chris Lu
5ac6297c68
adjust parameter names
2020-03-08 17:03:37 -07:00
Chris Lu
9228ff192c
encryption works
2020-03-07 11:08:57 -08:00
Chris Lu
0375ce2c2e
filer: set mime type from volume server
2020-03-07 07:25:15 -08:00
Chris Lu
ea1169dc80
filer cipher: single chunk http POST and PUT and read
2020-03-07 06:08:08 -08:00
Chris Lu
13e215ee5c
filer: option to encrypt data on volume server
2020-03-06 00:49:47 -08:00
Chris Lu
f90c43635d
refactoring
2020-03-04 00:39:47 -08:00
Chris Lu
892e726eb9
avoid reusing context object
...
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
6ab7368ef2
filer: dynamically create bucket under /buckets folder
2020-02-24 22:28:45 -08:00
Chris Lu
33b3bd467c
Revert "HEAD operation changes to fasthttp"
...
This reverts commit 58f126fd27
.
2020-02-10 13:43:53 -08:00
Chris Lu
58f126fd27
HEAD operation changes to fasthttp
2020-02-10 09:13:29 -08:00
Chris Lu
72a64a5cf8
use the same context object in order to retry
2020-01-26 14:42:11 -08:00
Chris Lu
8dd260623e
protect against nil deletion results
...
fix https://github.com/chrislusf/seaweedfs/issues/1180
2020-01-08 21:50:59 -08:00
Chris Lu
392678f8f3
upload skipping mimetype if not needed
2019-12-15 20:57:08 -08:00
Chris Lu
d0b423bbc0
filer: increase directory listing pagination size
2019-12-12 09:11:10 -08:00
Lei Liu
46755ea1e1
fix master maintenance logic
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-11-12 14:46:10 +08:00
zhangsong
61fa485700
add volume number param in assign operation
2019-11-10 20:11:03 +08:00
Chris Lu
327336ecf3
filer: avoid concurrent modification to result slice
...
fix https://github.com/chrislusf/seaweedfs/issues/972
2019-06-28 01:12:41 -07:00
Chris Lu
713596e781
caching ec shard locations
2019-05-27 22:54:58 -07:00
Chris Lu
2ffe98443b
refactoring
2019-04-20 12:05:28 -07:00
Chris Lu
6fc1f53018
shell: add command volume.move
2019-04-20 11:35:23 -07:00
Chris Lu
e5506152c0
refactoring
2019-04-18 21:43:36 -07:00
Chris Lu
a2d34d4802
filer.copy: add adjustable compression level
2019-04-18 20:21:28 -07:00
Chris Lu
79c2cca9c1
better error message
2019-04-16 01:15:30 -07:00
Chris Lu
e85048bcdc
http exhaust and close response body
2019-04-14 23:28:24 -07:00
Chris Lu
a32abda1a3
adjust compression to optimize for speed
2019-04-07 11:01:28 -07:00
Chris Lu
174bf1e8b2
more efficient client side gzip compression
2019-04-06 14:14:28 -07:00
Chris Lu
19728fe3f6
remove deprecated code
2019-03-25 23:12:14 -07:00
Chris Lu
95e0520182
weed volume: add grpc operation to relicate a volume to local
2019-03-23 11:33:34 -07:00
Chris Lu
da871896c3
weed filer: set grpc port to port + 10000
2019-03-19 05:47:41 -07:00
Chris Lu
55bab1b456
add context.Context
2019-03-15 17:20:24 -07:00
chenwanli
c668e55e07
Revert "fix https://github.com/chrislusf/seaweedfs/issues/780 "
...
This reverts commit 31038acccf
.
2019-03-11 11:48:11 +08:00
Chris Lu
e108688990
avoid grpc 5 seconds timeout
...
some operations may take longer than 5 seconds.
only keep the timeout for raft operations
2019-02-20 01:01:01 -08:00
Chris Lu
72c2e6bb8f
remove unused file
2019-02-19 14:16:19 -08:00
Chris Lu
448645203a
remove unused variables
2019-02-18 20:05:55 -08:00
Chris Lu
77b9af531d
adding grpc mutual tls
2019-02-18 12:11:52 -08:00
Chris Lu
7103c1ab7e
go fmt
2019-02-15 00:09:48 -08:00
Chris Lu
74fb237727
benchmark can work in secure mode
2019-02-15 00:09:19 -08:00
Chris Lu
4ff4a147b2
cleanup security.Secret
2019-02-09 21:56:32 -08:00
Sergey
aa5ccff6d2
fixing of typos
2019-02-06 18:59:15 +05:00
Chris Lu
67e2ea72be
master add separate grpc port
...
due to https://github.com/soheilhy/cmux/issues/64
fix https://github.com/chrislusf/seaweedfs/issues/820
fix https://github.com/chrislusf/seaweedfs/issues/840
fix https://github.com/chrislusf/seaweedfs/issues/841
2019-01-18 14:14:47 -08:00
bingoohuang
ab6be025d7
go fmt and fix some typo
2019-01-17 09:17:19 +08:00
chenwanli
39c7455881
Set http timeout to 5s
2019-01-10 20:21:08 +08:00
chenwanli
0a3e83a36a
Set timeout for master and volume non-streaming rpc
2019-01-10 19:41:03 +08:00
Chris Lu
6b5d6bb5a6
adjust error names
2019-01-05 19:52:38 -08:00
Chris Lu
e49a38842a
set etag in response
2019-01-02 12:57:54 -08:00