chrislu
fec8428fd8
POSIX: different inode for same named different file types
2022-01-12 11:51:13 -08:00
chrislu
6a12520a96
fix logging
2022-01-10 01:00:11 -08:00
chrislu
5c87fcc6d2
add client id for all metadata listening clients
2021-12-30 00:23:57 -08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
banjiaojuhao
083bf3a137
filer server: add "datacenter, rack and datanode" for path specific configuration
2021-12-23 23:25:05 +08:00
banjiaojuhao
08336be92e
filer server: allow upload file to specific dataNode
2021-12-22 21:57:26 +08:00
chrislu
a2d3f89c7b
add lock messages
2021-12-10 13:24:38 -08:00
chrislu
b70cb3e0b2
upgrade protoc to 3.17.3
...
$brew install protobuf
$ protoc --version
libprotoc 3.17.3
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
2021-12-05 00:42:25 -08:00
Chris Lu
e0fc2898e9
auto updated filer peer list
2021-11-06 14:23:35 -07:00
Chris Lu
4b9c42996a
refactor grpc API
2021-11-05 18:11:40 -07:00
Chris Lu
5ea86ef1da
Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()"
...
This reverts commit af71ae11aa
.
2021-11-05 17:52:15 -07:00
Chris Lu
af71ae11aa
master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()
2021-11-03 01:09:48 -07:00
Chris Lu
5160eb08f7
shell: optionally read filer address from master
2021-11-02 23:38:45 -07:00
Chris Lu
24858507cc
rename API to avoid confusion
2021-10-30 19:27:25 -07:00
Chris Lu
5435027ff0
volume copy: stream out copying progress and avoid grpc request timeout
...
fix https://github.com/chrislusf/seaweedfs/issues/2386
2021-10-24 02:52:56 -07:00
Chris Lu
3be3c17f59
volume vacuum: avoid timeout with streaming progress report
...
fix https://github.com/chrislusf/seaweedfs/issues/2396
2021-10-24 01:55:34 -07:00
Chris Lu
cca62fdb30
mount: streaming renaming folders
2021-10-17 04:22:42 -07:00
Chris Lu
004e56c1a6
save updated lastTsNs
2021-10-17 02:01:20 -07:00
Chris Lu
0a856241fe
avoid int bigger than math.MaxInt32
...
fix https://github.com/chrislusf/seaweedfs/issues/2363
2021-10-07 21:12:57 -07:00
Chris Lu
225b019fe0
stream read multiple volumes in a volume server
2021-09-27 02:51:31 -07:00
Chris Lu
c4d7ee6c5c
volume server: read all files in a volume
2021-09-27 01:45:32 -07:00
Chris Lu
52fe86df45
use default 10000 for grpc port
2021-09-20 14:05:59 -07:00
Chris Lu
2789d10342
go fmt
2021-09-14 10:37:06 -07:00
Chris Lu
e5fc35ed0c
change server address from string to a type
2021-09-12 22:47:52 -07:00
Chris Lu
0128239c0f
handle ipv6 addresses
2021-09-07 16:43:54 -07:00
Chris Lu
889a39a24f
ipv6
2021-09-07 02:30:25 -07:00
Chris Lu
2b1feb732c
remote.cache supports replication
2021-09-06 18:30:44 -07:00
Chris Lu
c218ef20c7
filer.remote.sync: automatically detect the primary remote storage
2021-09-06 15:10:55 -07:00
Chris Lu
0cda61d539
return error if not found
2021-09-05 23:43:44 -07:00
Chris Lu
e93d4935e3
add other replica locations when assigning volumes
2021-09-05 23:32:25 -07:00
Chris Lu
0652805236
cloud drive: add createBucket() deleteBucket()
2021-09-03 22:30:55 -07:00
Chris Lu
d1a4e19a3f
volume: copy file also copies modification time
...
to ensure ttl can work well
2021-09-01 02:42:57 -07:00
Chris Lu
8e125339d5
line wrap
2021-08-31 23:30:28 -07:00
Chris Lu
68ea99190f
add additionalPathPrefixes
2021-08-31 23:26:26 -07:00
Chris Lu
43fd11278e
support follow additional path prefixes
2021-08-31 23:23:08 -07:00
Chris Lu
1dfcdc53b7
cloud drive: add storj, filebase
2021-08-30 17:43:14 -07:00
Chris Lu
701cc133f3
cloud drive: s3 add option for v4 signature
2021-08-30 17:28:33 -07:00
Chris Lu
05f32376eb
add cluster id in filer configuration response
2021-08-29 21:02:10 -07:00
Chris Lu
001a472057
cloud mount: remote storage support hdfs
2021-08-29 18:41:29 -07:00
Chris Lu
a22f37b01c
remove tencent region
2021-08-28 02:23:03 -07:00
Chris Lu
05a648bb96
refactor: separating out remote.proto
2021-08-26 15:18:34 -07:00
Chris Lu
c08ac536ed
cloud drive: add support for Wasabi
...
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu
a7a914f120
cloud drive: add support for Baidu BOS
2021-08-24 23:46:33 -07:00
Chris Lu
a19c728034
cloud drive: add support for Tencent COS
2021-08-24 23:19:45 -07:00
Chris Lu
19a81d25af
cloud drive: add support for Aliyun OSS
2021-08-24 23:14:24 -07:00
Chris Lu
47d775cf68
cloud drive: add support for BackBlaze
2021-08-24 22:30:06 -07:00
Chris Lu
e9ebe24f2e
cloud drive: add support for Azure
2021-08-24 01:18:30 -07:00
Chris Lu
00c4e06caa
cloud drive: s3 configurable force path style
2021-08-23 03:30:41 -07:00
Chris Lu
2836a58d87
cloud drive: S3 supports storage class
2021-08-23 02:18:59 -07:00
Chris Lu
258063de26
cloud drive: add google cloud storage
2021-08-23 00:29:27 -07:00
Chris Lu
3ada61c875
rename
2021-08-14 21:50:35 -07:00
Chris Lu
c34747c79d
rename, fix wrong logic.
2021-08-14 21:46:34 -07:00
Chris Lu
08258a819d
fix mistake
2021-08-14 05:10:30 -07:00
Chris Lu
4909bd9684
gRpc connection error on filer when no volume left #2243
...
fix https://github.com/chrislusf/seaweedfs/issues/2243
grpc do not cache connections only when connection problem happens.
Normal error results should not close the shared grpc connection.
2021-08-14 05:06:44 -07:00
Chris Lu
5a0f92423e
use grpc and jwt
2021-08-12 21:40:33 -07:00
Chris Lu
5571f4f70a
master: add master.follower to handle read file id lookup requests
2021-08-12 18:10:59 -07:00
Chris Lu
69655ba8e5
mount: cache on reading remote storage
2021-08-09 22:11:57 -07:00
Chris Lu
713c035a6e
shell: remote.cache remote.uncache
2021-08-09 14:35:18 -07:00
Chris Lu
c0b12da4ef
shell: add filer.remote.unmount
2021-08-08 22:26:37 -07:00
Chris Lu
eed26af266
Merge branch 'master' into add_remote_storage
2021-08-08 15:48:04 -07:00
Chris Lu
4370a4db63
use int64 for volume count in case of negative overflow
2021-08-08 15:19:39 -07:00
Chris Lu
270770d7d7
refactor
2021-08-07 14:18:53 -07:00
Chris Lu
6b743dbbf9
refactor client subscribe metadata
2021-08-04 16:25:46 -07:00
Chris Lu
9df7d16791
read <- remote_storage
2021-07-31 22:39:38 -07:00
Chris Lu
899963ac20
remote storage location changed to struct
2021-07-29 02:08:55 -07:00
Chris Lu
1752eeb538
remote.mount saves the mapping
2021-07-27 03:26:35 -07:00
Chris Lu
4b94b03d90
directory to remote storage mapping
2021-07-27 01:16:28 -07:00
Chris Lu
99b599aa8a
remote.mount
2021-07-26 22:53:44 -07:00
Chris Lu
70effac0d3
configure and store remote configurations
2021-07-21 02:24:34 -07:00
Chris Lu
450222dd64
add remote to filer.Entry and filer_pb entry, add RemoteConf
2021-07-19 02:47:27 -07:00
Chris Lu
8dc5def435
proto: add remote
2021-07-19 01:12:31 -07:00
Chris Lu
b194f91f47
add version to filer configuration response
2021-07-12 01:23:20 -07:00
lyg
1a4db87e19
fix s3 metadata error with multipart upload
2021-07-02 11:00:42 +08:00
Chris Lu
215b169562
mount: recursively rename locally
2021-07-01 01:19:31 -07:00
Chris Lu
ab606dec2a
filer: add path-specific option to enforce readonly
2021-06-04 01:03:41 -07:00
Chris Lu
c5de97ddbe
keep alive for streaming connections
...
fix https://github.com/chrislusf/seaweedfs/issues/2096
related to https://github.com/chrislusf/seaweedfs/issues/1277
2021-05-31 01:26:24 -07:00
Chris Lu
1456616a77
recreate grpc connections if too many errors
...
address https://github.com/chrislusf/seaweedfs/issues/2098
2021-05-30 00:07:43 -07:00
Chris Lu
3d3fa43542
filer: re-create grpc connections if having transport error
...
fix https://github.com/chrislusf/seaweedfs/issues/2070
2021-05-22 17:46:53 -07:00
Chris Lu
dc1309f084
FUSE mount: support multiple filers
...
fix https://github.com/chrislusf/seaweedfs/issues/2015
fix https://github.com/chrislusf/seaweedfs/issues/1531
2021-05-21 01:28:00 -07:00
Chris Lu
f0ad172e80
shell: show which server holds the lock
...
fix https://github.com/chrislusf/seaweedfs/issues/1983
2021-04-22 23:56:35 -07:00
Chris Lu
b465095db1
shell: add volume.check.disk to fix inconsistency for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu
b1a86cf808
s3: copy object to itself
...
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Chris Lu
35d939a1d2
go fmt
2021-03-14 13:21:02 -07:00
wuh-fnst
4c1d945e46
make List correctly judge whether it is the last file
2021-03-12 11:22:26 +08:00
Chris Lu
828f6e9f4d
volume: auto add missing vif files
...
fix https://github.com/chrislusf/seaweedfs/issues/1878
2021-03-09 12:09:32 -08:00
Chris Lu
9d402ebe9f
refactoring
2021-03-06 14:26:27 -08:00
Chris Lu
bd727b7b53
go fmt
2021-03-02 20:59:56 -08:00
Chris Lu
e982b06bcd
support IPv6
2021-03-02 20:59:39 -08:00
Chris Lu
f465d63b5d
grpc should fail when heart beating to master
...
fix https://github.com/chrislusf/seaweedfs/issues/1820
2021-02-19 03:37:29 -08:00
Chris Lu
f8446b42ab
this can compile now!!!
2021-02-16 02:47:02 -08:00
Chris Lu
487e435679
adjust http max idle connections per host
...
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
Chris Lu
770393a48c
volume: add capability to change disk type when moving a volume
2021-02-09 23:58:08 -08:00
Chris Lu
821c46edf1
Merge branch 'master' into support_ssd_volume
2021-02-09 11:37:07 -08:00
Chris Lu
15c60cbb26
close the grpc connection after 10 hours
...
related to https://github.com/chrislusf/seaweedfs/issues/1782
2021-02-07 03:50:01 -08:00
Chris Lu
990fa69bfe
add back AdjustedUrl() related code
2021-01-28 14:36:29 -08:00
Chris Lu
00707ec00f
mount: outsideContainerClusterMode proxy through filer
...
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu
6ca10725b8
Revert "mount: when outside cluster network, use filer as proxy to access volume servers"
...
This reverts commit 096e088d7b
.
2021-01-24 03:15:19 -08:00
Chris Lu
096e088d7b
mount: when outside cluster network, use filer as proxy to access volume servers
2021-01-24 01:41:38 -08:00
Chris Lu
80b8692688
filer.sync: replicate outside of either cluster, only need to see filers
2021-01-24 00:01:44 -08:00
Chris Lu
2e8dba571b
adjust volume server UI
2020-12-14 00:51:57 -08:00