gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8904 commits 28 branches 278 tags 135 MiB
Commit graph

446 commits

Author SHA1 Message Date
Chris Lu 8efe55f981
Merge pull request #3059 from guol-fnst/avoid_dup_vol 
avoid loading duplicated volume directory
 2022-05-17 23:48:57 -07:00
guol-fnst 8e2426a879 revert changes of pb file 
orignal protoc-gen-go-grpc verion is 1.1.0
 2022-05-18 11:52:14 +08:00
guol-fnst b12944f9c6 fix naming convention 
notify volume server of duplicate directoris
improve searching efficiency
 2022-05-17 15:41:49 +08:00
guol-fnst de6aa9cce8 avoid duplicated volume directory 2022-05-16 19:33:51 +08:00
chrislu 866fed1eab avoid grpc name conflict 
fix https://github.com/chrislusf/seaweedfs/issues/3055
 2022-05-15 21:31:29 -07:00
chrislu 29198720f2 s3: add grpc server to accept configuration changes 2022-05-15 00:43:37 -07:00
chrislu 8bc16f1149 rename 2022-05-06 23:38:22 -07:00
chrislu 94635e9b5c filer: add filer group 2022-05-01 21:59:16 -07:00
chrislu 4bd6bea429 fix test 2022-05-01 09:02:01 -07:00
Konstantin Lebedev 1e35b4929f shell vacuum volume by collection and volume id 2022-04-18 18:40:58 +05:00
chrislu b4be56bb3b add timing info during ping operation 2022-04-16 12:45:49 -07:00
Konstantin Lebedev 35bc67f030 fix conflicts 2022-04-07 18:52:13 +05:00
Konstantin Lebedev f5246b748d Merge branch 'new_master' into hashicorp_raft 
# Conflicts:
#	weed/pb/master_pb/master.pb.go
 2022-04-07 18:50:27 +05:00
Konstantin Lebedev 85d80fd36d fix removing old raft server 2022-04-07 15:31:37 +05:00
Konstantin Lebedev 357aa818fe add raft shell cmds 2022-04-06 15:23:53 +05:00
chrislu bc888226fc erasure coding: tracking encoded/decoded volumes 
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
 2022-04-05 19:03:02 -07:00
chrislu 958f880b70 mount: add grpc method to adjust quota 2022-04-02 15:14:37 -07:00
chrislu daca2d22a5 use original server address string as map key 2022-04-01 17:34:42 -07:00
chrislu bbbbbd70a4 master supports grpc ping 2022-04-01 16:50:58 -07:00
chrislu 743ad690b6 filer supports grpc ping 2022-04-01 16:44:58 -07:00
chrislu 800cbc004c volume server adds ping function 2022-04-01 16:37:06 -07:00
chrislu 21e0898631 refactor: change masters from a slice to a map 2022-03-26 13:33:17 -07:00
chrislu a0bad1c997 remove any go mod changes 
This reverts commit 6c7f7d6887.
 2022-03-21 23:04:00 -07:00
chrislu 6c7f7d6887 Revert "Merge pull request #2782 from SadmiB/upstream" 
This reverts commit a644b7236a, reversing
changes made to 349257f822.
 2022-03-21 23:00:50 -07:00
Chris Lu 0d692a6254
Merge branch 'master' into upstream 2022-03-21 11:02:04 -07:00
SadmiB d12540c9f2 Add contabo api client 2022-03-21 17:16:49 +01:00
chrislu 4042fdf3bb rename to skipCheckParentDir 
related to https://github.com/chrislusf/seaweedfs/pull/2761

It's better to default to false.
 2022-03-16 23:55:31 -07:00
chrislu 202a29d014 refactoring 2022-02-25 01:17:26 -08:00
chrislu 91d6785cf3 define metadata action types 2022-02-25 00:54:16 -08:00
chrislu be3fc77391 mount2: use consistent inode 2022-02-25 00:53:27 -08:00
chrislu ceaf993a27 mount2: add rdev 2022-02-24 14:51:25 -08:00
chrislu b9b684194f remove max connection age 
following https://github.com/grpc/grpc-go/issues/3170#issuecomment-552517779
 2022-01-27 02:28:22 -08:00
chrislu b1063162b6 display bucket quota 2022-01-21 00:55:04 -08:00
chrislu fec8428fd8 POSIX: different inode for same named different file types 2022-01-12 11:51:13 -08:00
chrislu 6a12520a96 fix logging 2022-01-10 01:00:11 -08:00
chrislu 5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls 
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
 2021-12-26 00:15:03 -08:00
banjiaojuhao 083bf3a137 filer server: add "datacenter, rack and datanode" for path specific configuration 2021-12-23 23:25:05 +08:00
banjiaojuhao 08336be92e filer server: allow upload file to specific dataNode 2021-12-22 21:57:26 +08:00
chrislu a2d3f89c7b add lock messages 2021-12-10 13:24:38 -08:00
chrislu b70cb3e0b2 upgrade protoc to 3.17.3 
$brew install protobuf
$ protoc --version
libprotoc 3.17.3
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
 2021-12-05 00:42:25 -08:00
Chris Lu e0fc2898e9 auto updated filer peer list 2021-11-06 14:23:35 -07:00
Chris Lu 4b9c42996a refactor grpc API 2021-11-05 18:11:40 -07:00
Chris Lu 5ea86ef1da Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()" 
This reverts commit af71ae11aa.
 2021-11-05 17:52:15 -07:00
Chris Lu af71ae11aa master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates() 2021-11-03 01:09:48 -07:00
Chris Lu 5160eb08f7 shell: optionally read filer address from master 2021-11-02 23:38:45 -07:00
Chris Lu 24858507cc rename API to avoid confusion 2021-10-30 19:27:25 -07:00
Chris Lu 5435027ff0 volume copy: stream out copying progress and avoid grpc request timeout 
fix https://github.com/chrislusf/seaweedfs/issues/2386
 2021-10-24 02:52:56 -07:00
Chris Lu 3be3c17f59 volume vacuum: avoid timeout with streaming progress report 
fix https://github.com/chrislusf/seaweedfs/issues/2396
 2021-10-24 01:55:34 -07:00
Chris Lu cca62fdb30 mount: streaming renaming folders 2021-10-17 04:22:42 -07:00
Chris Lu 004e56c1a6 save updated lastTsNs 2021-10-17 02:01:20 -07:00
Chris Lu 0a856241fe avoid int bigger than math.MaxInt32 
fix https://github.com/chrislusf/seaweedfs/issues/2363
 2021-10-07 21:12:57 -07:00
Chris Lu 225b019fe0 stream read multiple volumes in a volume server 2021-09-27 02:51:31 -07:00
Chris Lu c4d7ee6c5c volume server: read all files in a volume 2021-09-27 01:45:32 -07:00
Chris Lu 52fe86df45 use default 10000 for grpc port 2021-09-20 14:05:59 -07:00
Chris Lu 2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu 0128239c0f handle ipv6 addresses 2021-09-07 16:43:54 -07:00
Chris Lu 889a39a24f ipv6 2021-09-07 02:30:25 -07:00
Chris Lu 2b1feb732c remote.cache supports replication 2021-09-06 18:30:44 -07:00
Chris Lu c218ef20c7 filer.remote.sync: automatically detect the primary remote storage 2021-09-06 15:10:55 -07:00
Chris Lu 0cda61d539 return error if not found 2021-09-05 23:43:44 -07:00
Chris Lu e93d4935e3 add other replica locations when assigning volumes 2021-09-05 23:32:25 -07:00
Chris Lu 0652805236 cloud drive: add createBucket() deleteBucket() 2021-09-03 22:30:55 -07:00
Chris Lu d1a4e19a3f volume: copy file also copies modification time 
to ensure ttl can work well
 2021-09-01 02:42:57 -07:00
Chris Lu 8e125339d5 line wrap 2021-08-31 23:30:28 -07:00
Chris Lu 68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Chris Lu 43fd11278e support follow additional path prefixes 2021-08-31 23:23:08 -07:00
Chris Lu 1dfcdc53b7 cloud drive: add storj, filebase 2021-08-30 17:43:14 -07:00
Chris Lu 701cc133f3 cloud drive: s3 add option for v4 signature 2021-08-30 17:28:33 -07:00
Chris Lu 05f32376eb add cluster id in filer configuration response 2021-08-29 21:02:10 -07:00
Chris Lu 001a472057 cloud mount: remote storage support hdfs 2021-08-29 18:41:29 -07:00
Chris Lu a22f37b01c remove tencent region 2021-08-28 02:23:03 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
Chris Lu c08ac536ed cloud drive: add support for Wasabi 
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
 2021-08-25 17:34:29 -07:00
Chris Lu a7a914f120 cloud drive: add support for Baidu BOS 2021-08-24 23:46:33 -07:00
Chris Lu a19c728034 cloud drive: add support for Tencent COS 2021-08-24 23:19:45 -07:00
Chris Lu 19a81d25af cloud drive: add support for Aliyun OSS 2021-08-24 23:14:24 -07:00
Chris Lu 47d775cf68 cloud drive: add support for BackBlaze 2021-08-24 22:30:06 -07:00
Chris Lu e9ebe24f2e cloud drive: add support for Azure 2021-08-24 01:18:30 -07:00
Chris Lu 00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00
Chris Lu 2836a58d87 cloud drive: S3 supports storage class 2021-08-23 02:18:59 -07:00
Chris Lu 258063de26 cloud drive: add google cloud storage 2021-08-23 00:29:27 -07:00
Chris Lu 3ada61c875 rename 2021-08-14 21:50:35 -07:00
Chris Lu c34747c79d rename, fix wrong logic. 2021-08-14 21:46:34 -07:00
Chris Lu 08258a819d fix mistake 2021-08-14 05:10:30 -07:00
Chris Lu 4909bd9684 gRpc connection error on filer when no volume left #2243 
fix https://github.com/chrislusf/seaweedfs/issues/2243

grpc do not cache connections only when connection problem happens.
Normal error results should not close the shared grpc connection.
 2021-08-14 05:06:44 -07:00
Chris Lu 5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu 5571f4f70a master: add master.follower to handle read file id lookup requests 2021-08-12 18:10:59 -07:00
Chris Lu 69655ba8e5 mount: cache on reading remote storage 2021-08-09 22:11:57 -07:00
Chris Lu 713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00
Chris Lu c0b12da4ef shell: add filer.remote.unmount 2021-08-08 22:26:37 -07:00
Chris Lu eed26af266 Merge branch 'master' into add_remote_storage 2021-08-08 15:48:04 -07:00
Chris Lu 4370a4db63 use int64 for volume count in case of negative overflow 2021-08-08 15:19:39 -07:00
Chris Lu 270770d7d7 refactor 2021-08-07 14:18:53 -07:00
Chris Lu 6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu 9df7d16791 read <- remote_storage 2021-07-31 22:39:38 -07:00
Chris Lu 899963ac20 remote storage location changed to struct 2021-07-29 02:08:55 -07:00
Chris Lu 1752eeb538 remote.mount saves the mapping 2021-07-27 03:26:35 -07:00
Chris Lu 4b94b03d90 directory to remote storage mapping 2021-07-27 01:16:28 -07:00