gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8889 commits 28 branches 278 tags 135 MiB
Commit graph

674 commits

Author SHA1 Message Date
Chris Lu 4820b594a7
Merge pull request #3298 from kmlebedev/evacuateTargetServer 
volume server evacuate to target server
 2022-07-19 13:56:49 -07:00
Konstantin Lebedev d3f7c09c03 remove ticker 
update the topology before each file
 2022-07-20 00:54:23 +05:00
Konstantin Lebedev d422e7769c ticker.Stop 2022-07-18 16:38:19 +05:00
Konstantin Lebedev 73a0dea16b sync update topologyInfo 2022-07-18 16:27:02 +05:00
Konstantin Lebedev 2b4112e462 update otherNodes 2022-07-18 11:32:28 +05:00
Konstantin Lebedev 3c2774ec3d fix update topologyInfo 2022-07-18 01:46:31 +05:00
Konstantin Lebedev 4d5144e50d clouse background update 2022-07-15 13:51:08 +05:00
guosj ab1b9697e6 supplement check duplicate accesskey 2022-07-13 17:28:20 +08:00
石昌林 392da4e038 Make the prompt information clearer 2022-07-13 11:39:45 +08:00
石昌林 ad5e8f68ec Check whether there is a duplicate accessKey when modifying iam 2022-07-13 11:36:11 +08:00
Konstantin Lebedev 6622240df7 fix TestVolumeServerEvacuate 2022-07-12 14:56:34 +05:00
Konstantin Lebedev 8372721a62 update topologyInfo 2022-07-12 13:47:21 +05:00
Konstantin Lebedev ee95d23a22 help rack 2022-07-12 11:56:58 +05:00
Konstantin Lebedev 087fa1347f volume server evacuate from rack 2022-07-12 11:33:08 +05:00
Konstantin Lebedev 4236c36599 volume server evacuate to target server 2022-07-11 16:58:15 +05:00
chrislu 57975b93ff adjust used size reporting 2022-07-04 13:47:42 -07:00
chrislu a79f5bd0d5 adjust used size reporting 2022-07-04 13:46:32 -07:00
guol-fnst 5bc895673e add getting bucket quota 2022-07-04 15:36:44 +08:00
guol-fnst dd4664e610 fix usage 2022-07-04 11:31:39 +08:00
chrislu 141f662734 edge case checking when volume server does not have capacity to balance 
fix https://github.com/chrislusf/seaweedfs/issues/3257
 2022-07-01 12:16:18 -07:00
chrislu bee3a7c798 fix tests 2022-07-01 12:09:42 -07:00
Konstantin Lebedev 5ed8165161 fix logic 
add option targetServer
https://github.com/chrislusf/seaweedfs/issues/3255
 2022-06-30 17:58:24 +05:00
Konstantin Lebedev bcbdc4cb37 use const multipart uploads folder 
avoid error bucket NotEmpty if multipart uploads folder exist
 2022-06-29 16:21:16 +05:00
chrislu 1d0c53ea56 remote storage: stop supporting hdfs as a remote storage 2022-06-20 14:15:59 -07:00
石昌林 9e036df356 remove go.uber.org/atomic 2022-06-20 12:35:29 +08:00
石昌林 3dd60529c5 some code optimizations 2022-06-17 19:07:39 +08:00
石昌林 37df209195 add some unit tests and some code optimizes 2022-06-17 17:11:18 +08:00
石昌林 78b3728169 add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations 
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply

# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply

# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply

# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply

# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
 2022-06-15 21:07:55 +08:00
chrislu d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
shichanglin5 a63559ee6c fix: fix command prompt information 2022-06-10 12:46:10 +08:00
chrislu 4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata 
these metadata can change and are not used
 2022-06-06 00:39:35 -07:00
chrislu f97acdd489 volume.fix.replication fix retry logic 
fix https://github.com/chrislusf/seaweedfs/issues/3136
 2022-06-03 08:45:29 -07:00
chrislu 6793bc853c help message when in simulation mode 2022-05-31 14:48:46 -07:00
chrislu 89948a373b fix error reporting for "Need to a hdd disk type" 
related to https://github.com/chrislusf/seaweedfs/issues/3128
 2022-05-31 12:43:55 -07:00
chrislu 5b9347c938 typo 2022-05-30 12:37:41 -07:00
chrislu ca16fbf0ef shell: cluster.ps display filer group 2022-05-29 16:37:14 -07:00
Konstantin Lebedev 44f53ceda6 fix collectionIsMismatch charset 2022-05-16 13:23:23 +05:00
Konstantin Lebedev 10d435f2c2 fix skip loop 2022-05-16 13:16:27 +05:00
Konstantin Lebedev 279053572c avoid delete volume replica if collection mismatch 2022-05-16 13:07:05 +05:00
chrislu 271b5aed96 shell: volume.fsck add a note for -reallyDeleteFromVolume option 2022-05-15 11:07:04 -07:00
chrislu 94635e9b5c filer: add filer group 2022-05-01 21:59:16 -07:00
guosj 1a5de36257 fix return value 2022-04-29 11:19:36 +08:00
Chris Lu b557faf175
Merge pull request #2967 from kmlebedev/vl_chk_disk_vlid 
volume.check.disk add param volumeId
 2022-04-26 22:09:06 -07:00
Konstantin Lebedev d4343ab7da forcePurging desc 2022-04-25 23:11:56 +05:00
Konstantin Lebedev ae56b2c00f change forcePurging to a pointer 2022-04-25 23:10:01 +05:00
Konstantin Lebedev 6d2fda27d2 delete missing data from volumes in one replica 2022-04-25 22:59:46 +05:00
Konstantin Lebedev 67476d830a volume.check.disk add param volumeId 2022-04-25 18:27:14 +05:00
Chris Lu 88a669dd19
Merge pull request #2858 from kmlebedev/fsck_replicas 
fsck replicas
 2022-04-19 00:17:43 -07:00
Konstantin Lebedev 2364fab927 volume.list show only readonly, collectionPattern and volumeId 2022-04-19 11:44:41 +05:00
Konstantin Lebedev 36c5a59ed8 add help 2022-04-18 19:36:14 +05:00
Konstantin Lebedev 1e35b4929f shell vacuum volume by collection and volume id 2022-04-18 18:40:58 +05:00
leyou240 89eb87c1d1
Merge branch 'master' into slices.SortFunc 2022-04-18 10:39:29 +08:00
justin 3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
chrislu 460d56d283 shell: cluster.check prints out clock delta and network latency 2022-04-16 13:24:17 -07:00
chrislu 1f03fcccb1 fix nil in cluster_check shell command 
fix https://github.com/chrislusf/seaweedfs/issues/2905
 2022-04-12 08:47:27 -07:00
Konstantin Lebedev 76b1c5ce67 cleanupUploads use jwt token 2022-04-12 13:46:11 +05:00
Konstantin Lebedev 2dfbe210e0 avoid breaking loop in cleanupUploads if error is empty 2022-04-11 11:16:20 +05:00
Konstantin Lebedev 931cb9e581 use "cluster.raft.{ps,add,remove}" 2022-04-11 10:50:01 +05:00
Konstantin Lebedev f5246b748d Merge branch 'new_master' into hashicorp_raft 
# Conflicts:
#	weed/pb/master_pb/master.pb.go
 2022-04-07 18:50:27 +05:00
Konstantin Lebedev 85d80fd36d fix removing old raft server 2022-04-07 15:31:37 +05:00
Konstantin Lebedev 357aa818fe add raft shell cmds 2022-04-06 15:23:53 +05:00
chrislu bc888226fc erasure coding: tracking encoded/decoded volumes 
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
 2022-04-05 19:03:02 -07:00
chrislu 4aae87f405 check missing hdd disk type 2022-04-04 14:48:00 -07:00
chrislu 6a2bcd03aa configure mount quota 2022-04-02 21:34:26 -07:00
chrislu 105578a2f2 skip pinging self for master and volume server 2022-04-01 20:25:35 -07:00
chrislu 4ecba915f3 add check between peers 2022-04-01 17:40:25 -07:00
chrislu 4b5c0e3fa9 check cluster connectivities 2022-04-01 17:27:49 -07:00
Konstantin Lebedev 7f1383a41e findExtraChunksInVolumeServers in consideration of replication 2022-04-01 14:45:41 +05:00
Konstantin Lebedev 3817e05dd0 fix collect filer files 2022-04-01 10:17:09 +05:00
Konstantin Lebedev 3cedb21bb7 skip new entities 2022-03-31 21:36:10 +05:00
Konstantin Lebedev 1d9e30d8c0 fsck replicas 2022-03-31 19:10:06 +05:00
Konstantin Lebedev 4f5b018a02 collecting only bucket volumes 2022-03-31 13:35:58 +05:00
chrislu 21e0898631 refactor: change masters from a slice to a map 2022-03-26 13:33:17 -07:00
banjiaojuhao f7f2a597dd minor 2022-03-08 16:22:55 +08:00
chrislu ede6ce44c6 fix test 2022-03-06 17:09:55 -08:00
chrislu 6e49e75a5b use logical number of files and sizes for statistics and quota 2022-03-04 18:47:44 -08:00
chrislu 95717d1006 simpler output 2022-02-24 13:50:08 -08:00
Chris Lu 6834df77a0
Merge pull request #2701 from guo-sj/fix_bugs_in_return_value 2022-02-23 19:23:37 -08:00
guosj 26f3ab8d4b fix a return bug in func (c *commandVacuum) Do 2022-02-24 09:04:38 +08:00
Chris Lu 38dcaaa76e
Merge pull request #2683 from guo-sj/fix_fsconfigure_bug 
fix minor bug in commandFsConfigure.Do
 2022-02-16 18:16:47 -08:00
guosj ca121ecd66 fix minor bug in commandFsConfigure.Do 2022-02-17 09:57:52 +08:00
Chris Lu 64afbc5235
Merge pull request #2640 from Radtoo/fsck_with_delete 
Added basic deletion capabilities to fsck.
 2022-02-14 23:00:01 -08:00
Radtoo 724ce04b1c We picked the second flag's name. 2022-02-13 00:53:35 +01:00
Radtoo fbe7ed7927 Revert the the flag to "reallyDeleteFromVolume". 2022-02-10 20:01:48 +01:00
chrislu 21aaa4c1f1 ec.encode: calculate free ec slots based on (maxVolumeCount-volumeCount) 
fix https://github.com/chrislusf/seaweedfs/issues/2642
 2022-02-08 01:51:13 -08:00
chrislu 13f6ec1c4e test checking ec distribution 2022-02-08 01:50:05 -08:00
chrislu f34c2ff7c5 use embed txt 2022-02-08 01:14:27 -08:00
chrislu c8c7c10c3f volume.tier.move: avoid double counting 
related to https://github.com/chrislusf/seaweedfs/issues/2637
 2022-02-08 00:57:35 -08:00
chrislu f18803424a volume.balance: add delay during tight loop 
fix https://github.com/chrislusf/seaweedfs/issues/2637
 2022-02-08 00:53:55 -08:00
chrislu 85c1615b43 filer read empty file may cause OOM in some cases 
fix https://github.com/chrislusf/seaweedfs/issues/2641
 2022-02-07 23:08:54 -08:00
Radtoo fbb14e0ea8 Adding separate toggle to purge absent vols 
While this toggle is basically required to clean out entries for deleted volumes, having a separate description + toggling this separately seems like a good idea so people get a chance to check if their volumes are all mounted/connected as expected.

Also renamed forcePurge to just purge.
 2022-02-06 23:46:52 +01:00
Radtoo 6b17f45da2 Also delete paths for which a volume is entirely absent. 2022-02-06 23:22:04 +01:00
Radtoo fa0cfdfb7f Added basic deletion capabilities to fsck. 2022-02-06 17:05:58 +01:00
chrislu 6bee1e9714 [volume.check.disk] fix wrong logic to compare 
fix https://github.com/chrislusf/seaweedfs/issues/2626

minuend - subtrahend
 2022-02-01 07:48:28 -08:00
chrislu b8490fe427 adjust volume count even when not applying the changes 2022-01-28 19:11:46 -08:00
divanikus 67e3fe996a async volumeTierMove 2022-01-26 18:22:31 +03:00
chrislu 9d0f58c329 skip printing fs configuration 2022-01-21 13:29:47 -08:00
chrislu ce2049cdb6 refactoring, move genFn before saveFn 2022-01-21 12:08:58 -08:00
chrislu e47f63d159 enforce bucket quota 2022-01-21 02:34:42 -08:00
chrislu 6e57d8d0de s3: check bucket usage and adjust read only according to quota 2022-01-21 02:15:27 -08:00