gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8787 commits 28 branches 278 tags 135 MiB
Commit graph

1390 commits

Author SHA1 Message Date
Konstantin Lebedev 4d4cd0948d
avoid infinite loop WaitUntilConnected() (#3431) 
https://github.com/seaweedfs/seaweedfs/issues/3421
 2022-08-11 15:03:26 -07:00
Eric Yang 986daecbfe
add upload response (#3428) 
* add upload response

* fixing comments

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
 2022-08-10 22:43:09 -07:00
chrislu ae93c966d9 ensure memory is aligned 
fix https://github.com/seaweedfs/seaweedfs/issues/3427
 2022-08-10 22:27:13 -07:00
LHHDZ 84ec68e11a
Add download speed limit support (#3408) 2022-08-05 01:16:42 -07:00
Konstantin Lebedev 4d08393b7c
filer prefer volume server in same data center (#3405) 
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404

* GetDataCenter

* prefer same data center for ReplicationSource

* GetDataCenterId

* remove glog
 2022-08-04 17:35:00 -07:00
Konstantin Lebedev 22181dd018
refactor FilerRequest metrics (#3402) 
* refactor FilerRequest metrics

* avoid double count proxy

* defer to
 2022-08-04 01:44:54 -07:00
Chris Lu fc8035d672
Merge pull request #3394 from kmlebedev/metricsReplicatedWrite 
Detailed metrics VolumeServerRequestHistogram for writing to disk and replication
 2022-08-01 13:31:01 -07:00
Konstantin Lebedev df9cc31a0a refactor 2022-08-02 01:18:26 +05:00
Konstantin Lebedev fc65122766 rename to LoadAvg_1M 2022-08-01 21:32:21 +05:00
Konstantin Lebedev 2f85e31f5c Merge branch 'master' into gentle_vacuum 2022-08-01 20:51:58 +05:00
Konstantin Lebedev 5209ebbeef remove percent 2022-08-01 20:40:38 +05:00
Chris Lu b59bc607bf
Merge pull request #3338 from kmlebedev/issues/3083 
rollback over onPeerUpdate implementation of automatic clean-up of failed servers in favor of synchronous ping
 2022-08-01 08:23:10 -07:00
Konstantin Lebedev cd5c7ad052 move to github.com/seaweedfs/seaweedfs 2022-08-01 16:36:32 +05:00
Konstantin Lebedev 3c75479e2b Merge branch 'master' into gentle_vacuum 
# Conflicts:
#	weed/pb/messaging_pb/messaging.pb.go
#	weed/pb/messaging_pb/messaging_grpc.pb.go
#	weed/pb/s3_pb/s3.pb.go
#	weed/pb/volume_server_pb/volume_server.pb.go
#	weed/server/volume_grpc_vacuum.go
 2022-08-01 14:45:22 +05:00
Konstantin Lebedev 1d29f67c02 revert disk stats 2022-08-01 14:29:41 +05:00
Konstantin Lebedev a98f6d66a3 rollback over onPeerupdate implementation of automatic clean-up of failed servers in favor of synchronous ping 2022-08-01 12:51:41 +05:00
Patrick Schmidt 1a4a36d510
Add healthy indicator for raft status 2022-07-30 19:34:26 +02:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev 2f0dda384d vacuum show LA 2022-07-29 11:59:33 +05:00
chrislu 9f479aab98 allocate brokers to serve segments 2022-07-28 23:24:38 -07:00
chrislu bb01b68fa0 refactor 2022-07-28 23:24:38 -07:00
chrislu 8060fdcac5 remove old code 2022-07-28 23:24:38 -07:00
chrislu f25e273e32 display data center and rack in cluster.ps 2022-07-28 23:22:52 -07:00
chrislu 68065128b8 add dc and rack 2022-07-28 23:22:51 -07:00
chrislu 3828b8ce87 "github.com/chrislusf/raft" => "github.com/seaweedfs/raft" 2022-07-27 12:12:40 -07:00
Konstantin Lebedev c88ea31f62 fix RUnlock of unlocked RWMutex 2022-07-26 12:57:07 +05:00
chrislu 64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
ningfd 58f2dd6740 change topology mapping definition, improve spelling 2022-07-22 17:18:56 +08:00
Konstantin Lebedev 3c42814b58 avoid deadlock 2022-07-21 17:15:10 +05:00
Konstantin Lebedev 93ca87b7cb use safe onPeerUpdateDoneCns 2022-07-21 15:51:14 +05:00
Konstantin Lebedev 7875470e74 onPeerUpdateGoroutineCount use int32 2022-07-20 18:40:50 +05:00
Konstantin Lebedev 6c390851e7 fix design 2022-07-20 18:08:12 +05:00
Konstantin Lebedev f6a966b4fc add waiting log message 2022-07-20 00:31:57 +05:00
Konstantin Lebedev 6cfbfb0849 check for ping before deleting raft server 
https://github.com/chrislusf/seaweedfs/issues/3083
 2022-07-20 00:04:12 +05:00
Chris Lu f0d560060f
Merge pull request #3336 from kmlebedev/raftSrvRemove 
raftServerRemoval typo and logs
 2022-07-19 11:32:17 -07:00
Konstantin Lebedev f419d5643a fix typo 
add remove logs
 2022-07-19 11:50:52 +05:00
chrislu aca20cd9f4 more logging related to filer metadata subscription 2022-07-15 00:42:13 -07:00
chrislu fbd8f868a1 filer may have trouble to re-connect 
clientId is used twice: one for local metadata subscription, one for combined metadata subscription.
 2022-07-14 12:15:31 -07:00
liubaojiang 4e83e92156 add subscriber clientId if it is the first time 2022-07-08 20:27:25 +08:00
xdadrm 3a42269da8
Fixes WebDAV 0-bytes files 
Fixes the issue where files created via WebDAV show as 0-bytes size when read via fuse.
 2022-06-27 20:19:21 +02:00
Konstantin Lebedev 6c20a3b622 avoid set currentMaster k8s svc.local discoveruy service domains 
https://github.com/chrislusf/seaweedfs/issues/2589
 2022-06-27 21:47:05 +05:00
chrislu 9dfea44aeb fix unit 2022-06-26 12:21:38 -07:00
chrislu 3142e77eca mount: stats report physical size instead of logical size 2022-06-26 12:14:34 -07:00
chrislu 0c13fc873c fix compilation 2022-06-23 00:45:11 -07:00
chrislu 96496d5286 master: broadcast new volume locations to clients to avoid possible racing condition 
fix https://github.com/chrislusf/seaweedfs/issues/3220
 2022-06-23 00:41:33 -07:00
Konstantin Lebedev c1c8dad677 avoid no such raft date directory 
https://github.com/chrislusf/seaweedfs/issues/3214
 2022-06-21 13:47:51 +05:00
石昌林 81f7f08708 Determine whether to preallocate according to the master configuration before executing copy volume 2022-06-20 21:12:44 +08:00
chrislu aaa9938353 Revert "adjust conditions" 
This reverts commit e024586ff1.
 2022-06-16 01:26:36 -07:00
chrislu e024586ff1 adjust conditions 2022-06-15 23:19:39 -07:00
Chris Lu 82f3bcc65e
Merge pull request #3081 from paochiang/volume_upload_limit_fix 
exclude replication from the concurrentUploadLimitMB
 2022-06-15 23:16:44 -07:00
liubaojiang df0ce31a2e add condition when inFlightUploadDataLimitCond signal 2022-06-16 14:07:11 +08:00
liubaojiang 3076ac101e move vs.concurrentUploadLimit != 0 out of the lock 2022-06-16 09:58:44 +08:00
Chris Lu 55b9f281d1
Merge pull request #3185 from kmlebedev/logging_err_process_range_req 2022-06-15 06:33:33 -07:00
Konstantin Lebedev 31a929f7f2 logging processRangeRequest errors 2022-06-15 15:36:43 +05:00
Chris Lu 4eeeb5f50d
Merge pull request #3180 from blacktear23/issue-3149 2022-06-15 00:55:20 -07:00
zhihao.qu 42d04c581b feat(filer.sync): add metricsServer in filer.sync. 
Metrics include:
(1) the offset of the filer.sync
(2) the last send timestamp of the filer subscription
 2022-06-15 11:33:18 +08:00
yulai.li 28cda854eb Add filer command line parameter to let Filer UI show/hide directory delete button 2022-06-15 00:30:49 +08:00
chrislu d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
chrislu 444ac21050 go fmt 2022-06-11 09:51:11 -07:00
Chris Lu 0a8b3948db
Merge pull request #3165 from blacktear23/reload-bug-fix 2022-06-10 07:47:43 -07:00
yulai.li 83f8cdd5cb Bug Fix: force reload filer page when upload/delete/rename/create directory 2022-06-10 22:39:21 +08:00
guol-fnst eabf4d4e64 fix typo 2022-06-10 22:03:11 +08:00
chrislu 9f8b72a54d Revert "Merge pull request #3159 from shichanglin5/_duplicateUUID" 
This reverts commit 37da689319, reversing
changes made to 00d53c34c4.
 2022-06-10 06:38:17 -07:00
Chris Lu 37da689319
Merge pull request #3159 from shichanglin5/_duplicateUUID 
perf: Optimized volume handling duplicateUUID logic to avoid quitting…
 2022-06-09 18:01:02 -07:00
shichanglin5 f5b0c04b14 perf: Optimized volume handling duplicateUUID logic to avoid quitting when volume is actualy normal 
Under normal circumstances, there will be no problems, but when the
master is debugged in the local environment, the volume client cannot
communicate with the master normally, so the sendHeartBeat logic is
restarted, and a new connection is created to report the heartbeat. If
the master has not cleared the uuid of the volume at this time, then The
master will respond to volume duplicateUUIDS, and the volume service
will exit, but in fact the uuid of the volume is not duplicated
 2022-06-09 20:41:16 +08:00
chrislu 4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata 
these metadata can change and are not used
 2022-06-06 00:39:35 -07:00
chrislu 8402af0882 filer: detect ttl based on fs configure setting 2022-06-06 00:15:37 -07:00
chrislu ecef844dfc stream read large files 2022-06-05 11:54:04 -07:00
chrislu 48cca4e54f remove unused logic 2022-06-04 19:11:06 -07:00
chrislu 376b005ad3 fix wrong logic 2022-05-31 11:55:58 -07:00
chrislu 27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu f4a6da6cb2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 22:47:29 -07:00
chrislu 596c3860ca use final destination to resolve fs configuration 
related to https://github.com/chrislusf/seaweedfs/issues/3075
 2022-05-30 22:47:26 -07:00
chrislu 6adc42147f fresh filer store bootstrap from the oldest peer 2022-05-30 21:27:48 -07:00
liubaojiang f0ee3e6f21 reduce the scope of inFlightUploadDataLimitCond lock 2022-05-31 09:49:07 +08:00
chrislu c59068d0f3 refactor 2022-05-30 16:28:36 -07:00
chrislu 682382648e collect cluster node start time 2022-05-30 16:23:52 -07:00
chrislu f214dfb1f5 stop when in memory log is done 2022-05-30 15:25:21 -07:00
chrislu aece35a64f stop when on disk log is done 2022-05-30 15:20:51 -07:00
chrislu a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu ca16fbf0ef shell: cluster.ps display filer group 2022-05-29 16:37:14 -07:00
chrislu 7e25a2d416 reduce busy waiting when reading metadata logs 2022-05-24 00:23:53 -07:00
liubaojiang 076e48a676 add inflight upload data wait timeout 2022-05-21 10:38:08 +08:00
liubaojiang 71b2e6223e add inFlightUploadDataLimitCond signal 2022-05-20 15:27:06 +08:00
liubaojiang 1a41691b4c exclude replication from the concurrentUploadLimitMB 2022-05-20 14:40:05 +08:00
Chris Lu cb64983c6b
Merge pull request #3079 from ztinpn/hotfix_fsync 2022-05-19 10:33:42 -07:00
tianzhang 66747ee9c9 hotfix_fsync fix fsync 2022-05-19 23:17:17 +08:00
guol-fnst b12944f9c6 fix naming convention 
notify volume server of duplicate directoris
improve searching efficiency
 2022-05-17 15:41:49 +08:00
guol-fnst 076595fbdd just exit in case of duplicated volume directories were loaded 2022-05-17 15:41:49 +08:00
guol-fnst 8fab39e775 rename UUID file 
fix typo
move locationUUID  into DiskLocation
 2022-05-17 15:41:47 +08:00
guol-fnst de6aa9cce8 avoid duplicated volume directory 2022-05-16 19:33:51 +08:00
Chris Lu a69bf53405
Merge pull request #2996 from kmlebedev/ydb 
ydb filer store
 2022-05-05 15:34:37 -07:00
chrislu b201edb9df fix wrong assignment 2022-05-03 07:12:33 -07:00
chrislu 9271866d1e fix segmentation violation 
fix https://github.com/chrislusf/seaweedfs/issues/3000
 2022-05-02 23:56:11 -07:00
Konstantin Lebedev d8925b4e83 Merge branch 'new_master' into ydb 
# Conflicts:
#	go.mod
#	go.sum
 2022-05-03 00:13:57 +05:00
chrislu 94635e9b5c filer: add filer group 2022-05-01 21:59:16 -07:00
Konstantin Lebedev 50c4f62ed4 ydb-go-sdk move to v3 2022-05-02 02:07:47 +05:00
chrislu 5d081d5c3c skip if already copied to remote 
fix https://github.com/chrislusf/seaweedfs/issues/2993
 2022-04-30 20:03:04 -07:00
Konstantin Lebedev 03aec9ce7e avoid panic if hashicorp raft not initialized 2022-04-30 13:56:35 +05:00
chrislu 94f824e1ce volume: sync to disk before copying volume files 
address https://github.com/chrislusf/seaweedfs/issues/2976
 2022-04-26 13:03:43 -07:00