gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8905 commits 28 branches 278 tags 135 MiB
Commit graph

111 commits

Author SHA1 Message Date
Ryan Russell 277976bd76
refactor(storage): readability improvements (#3703) 
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-09-16 02:43:17 -07:00
Guo Lei c57c79a0ab
optimiz commitig compact (#3388) 
* optimiz vacuuming volume

* fix bugx

* rename parameters

* fix conflict

* change copyDataBasedOnIndexFile to an instance method

* close needlemap

* optimiz commiting Vacuum volume for  leveldb index

* fix bugs

* fix leveldb loading bugs

* refactor

* fix leveldb loading bug

* add leveldb recovery

* add test case for levelDB

* modify test case to cover all the new branches

* use one tmpNm instead of two instances

* refactor

* refactor

* move setWatermark to the end

* add test for watermark and updating leveldb

* fix error logic

* refactor, add test

* check nil before close needlemapeer
add test case
fix metric bug

* add tests, fix bugs

* adjust log level
remove wrong test case
refactor

* avoid duplicate  updating metric for leveldb index
 2022-08-23 23:53:35 -07:00
Konstantin Lebedev 3c75479e2b Merge branch 'master' into gentle_vacuum 
# Conflicts:
#	weed/pb/messaging_pb/messaging.pb.go
#	weed/pb/messaging_pb/messaging_grpc.pb.go
#	weed/pb/s3_pb/s3.pb.go
#	weed/pb/volume_server_pb/volume_server.pb.go
#	weed/server/volume_grpc_vacuum.go
 2022-08-01 14:45:22 +05:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev 2f0dda384d vacuum show LA 2022-07-29 11:59:33 +05:00
guosj cc7a4b0a6e correct comment 2022-07-25 11:46:41 +08:00
chrislu 06a8b174b5 also remove Sync() for idx file 2022-06-30 13:50:53 -07:00
chrislu 81d6159290 volume: report error if a volume has nil data backend 
fix https://github.com/chrislusf/seaweedfs/issues/3105
 2022-05-29 16:59:30 -07:00
chrislu 37ab8909b0 use two flags: v.isCompacting and v.isCommitCompacting 2022-04-26 23:28:34 -07:00
Konstantin Lebedev 1e35b4929f shell vacuum volume by collection and volume id 2022-04-18 18:40:58 +05:00
Chris Lu 97a4b66df7
Merge pull request #2704 from guo-sj/fix_bugs_in_return_value 
fix return value in storage/volume_vacuum.go:444
 2022-02-24 00:49:29 -08:00
guosj 3e7aa1caf5 fix return value in storage/volume_vacuum.go:444 2022-02-24 15:54:36 +08:00
chrislu c29bc9a367 fix error handling 2022-02-23 15:34:25 -08:00
guosj d68c27f82d fix another return value bug 2022-02-23 16:21:25 +08:00
guosj 8f9aa0cddd fix bugs in return value 2022-02-23 16:17:48 +08:00
Konstantin Lebedev ef541972f8 updated needle with fsync 2022-02-08 00:10:53 +05:00
Chris Lu 3be3c17f59 volume vacuum: avoid timeout with streaming progress report 
fix https://github.com/chrislusf/seaweedfs/issues/2396
 2021-10-24 01:55:34 -07:00
Chris Lu 1aa7e99a89 skip file not found error when deleting 2021-05-15 09:37:39 -07:00
Chris Lu 400de380f4 volume server: support tcp direct put/get/delete 2021-03-05 02:29:38 -08:00
Chris Lu 7635f6b9fa disk file avoid file.Stat() 2021-02-20 20:06:06 -08:00
bingoohuang 7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
Chris Lu f3bb645018 file open error 2020-12-01 23:37:49 -08:00
Chris Lu dc0bc48257 return file open error 2020-12-01 23:36:49 -08:00
Chris Lu 6d30b21b10 volume: add "-dir.idx" option for separate index storage 
fix https://github.com/chrislusf/seaweedfs/issues/1265
 2020-11-27 03:17:10 -08:00
Chris Lu f2723c1bc8 do not idx file format 
revert c9ab8d05fa
 2020-09-12 12:42:36 -07:00
Chris Lu c9ab8d05fa fixes for reading deleted fid 2020-09-10 14:42:52 -07:00
Chris Lu 6ccd7f0a4d refactoring 2020-08-18 18:01:37 -07:00
Chris Lu ee11d98650 refactoring 2020-08-18 17:35:19 -07:00
Chris Lu 6a92f0bc7a refactoring to typed Size 
Go is amazing with refactoring!
 2020-08-18 17:04:28 -07:00
Chris Lu 6190fd665d printout error 2020-04-27 12:41:31 -07:00
Chris Lu c8ca234773 refactoring 2020-04-11 14:27:25 -07:00
Chris Lu df9d538044 rename function 2020-04-11 14:19:44 -07:00
Chris Lu cbfe31a9a8 idx file sync before compaction 2020-03-20 23:38:46 -07:00
Chris Lu 81797a059a volume: sync volume file right before compaction 
fix https://github.com/chrislusf/seaweedfs/issues/1237
 2020-03-19 23:54:52 -07:00
Chris Lu d439d83772 volume: follow compactionBytePerSecond 
related to https://github.com/chrislusf/seaweedfs/issues/1108
 2020-03-11 10:32:17 -07:00
Chris Lu d3e1ef64b3 ensure error checking on windows 2020-03-10 13:33:26 -07:00
Chris Lu 88a110e67e release file handle 2020-03-10 13:32:22 -07:00
Chris Lu 43e6261579 volume: fix memory leak 2020-03-10 13:31:23 -07:00
Chris Lu f87f2045b3 delete old file first on windows 
fix https://github.com/chrislusf/seaweedfs/issues/1210
 2020-02-27 00:59:35 -08:00
Chris Lu 757c7d67ad avoid nil DataBackend 2020-02-27 00:07:25 -08:00
Chris Lu 430b5a49e2 unused code 2020-02-26 19:35:00 -08:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu c8b2dac6c1 volume: avoid sharing volume dat file handle 
possibly help on https://github.com/chrislusf/seaweedfs/issues/1184
 2020-01-21 21:18:01 -08:00
Chris Lu acf7ca7b93 volume: fix compaction 2020-01-08 09:45:03 -08:00
Chris Lu efd2f50ede compaction changed to .idx based deletion 2019-12-24 14:55:50 -08:00
Chris Lu abffe857a1 change btree map to in memory level db 2019-12-24 10:18:56 -08:00
Chris Lu 72a561ab7c refactoring 2019-12-24 08:13:01 -08:00
Chris Lu 09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00
Chris Lu f60154f330 master load backend storage config from master.toml 2019-11-28 18:33:18 -08:00
Chris Lu 85f8649320 refactor memory mapped file into backend storage 2019-11-09 00:10:59 -08:00