gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9932 commits 28 branches 278 tags 135 MiB
Commit graph

116 commits

Author SHA1 Message Date
Chris Lu abffe857a1 change btree map to in memory level db 2019-12-24 10:18:56 -08:00
Chris Lu 72a561ab7c refactoring 2019-12-24 08:13:01 -08:00
Chris Lu 09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00
Chris Lu f60154f330 master load backend storage config from master.toml 2019-11-28 18:33:18 -08:00
Chris Lu 85f8649320 refactor memory mapped file into backend storage 2019-11-09 00:10:59 -08:00
Chris Lu db30a46050 skip memory mapped file when compacting 2019-10-29 23:34:38 -07:00
Chris Lu 5b950c735e Merge branch 'master' into refactoring_dat_backend 2019-10-29 23:18:41 -07:00
Chris Lu 57e441d67b fix compaction logic 2019-10-29 23:18:01 -07:00
Chris Lu 4b5ba4927f adjust NewDiskFile() paramaters 2019-10-29 22:37:36 -07:00
Chris Lu 19b6a16003 changed from os.file to backend.DataStorageBackend 2019-10-29 00:35:16 -07:00
Chris Lu fc412e428b refactor ScanVolumeFileFrom() 2019-10-22 00:50:30 -07:00
Chris Lu faec9076a4 adjust parameter names 2019-10-21 22:57:01 -07:00
j.laycock 6fc6322c90 Change joeslay paths to chrislusf paths 2019-09-12 14:18:21 +01:00
j.laycock 5885ab67b3 rename volume property MemoryMap to MemoryMapMaxSizeMB 2019-09-05 11:56:48 +01:00
Tom Maxwell 4a878c0006 Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve 2019-09-04 15:27:14 +01:00
j.laycock 1f01eb78e8 Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct 2019-09-03 17:00:59 +01:00
Tom Maxwell d637d86d22 Changes to try and pass the URL parameters through - in memory flag not working still 2019-09-03 15:41:28 +01:00
j.laycock 9a459d984b Do not vacuum memory mapped files 2019-09-03 15:05:43 +01:00
j.laycock 595a1beff0 Swap imports to use joeslay 2019-09-02 11:28:40 +01:00
Chris Lu d829df4f59 volume: protect against nil needle map 
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
 2019-08-14 01:08:01 -07:00
Chris Lu e40634e6b4 volume: fail the volume deletion if compaction is in progress 
fix https://github.com/chrislusf/seaweedfs/issues/1035
 2019-08-12 00:53:50 -07:00
Chris Lu f3b99cbfe5 fix wrong alignment fix 2019-07-21 13:50:42 -07:00
Chris Lu 644c5c27b4 refactoring 2019-07-21 13:50:24 -07:00
Chris Lu 33b3366043 volume: avoid compacting wait group 2019-07-21 13:49:59 -07:00
Chris Lu a7b1b23c58 fix wrong volume count 
fix https://github.com/chrislusf/seaweedfs/issues/1013
 2019-07-17 00:03:15 -07:00
Chris Lu 115558e5f5 adjust counters 2019-06-17 21:02:50 -07:00
Chris Lu 289fd7eb39 count number of volumes and ec shards 2019-06-16 02:24:15 -07:00
Chris Lu 17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00
Chris Lu cf58fc0e63 refactor: extract out the write throttler 2019-05-06 13:56:08 -07:00
Chris Lu b335f81a4f volume: add option to limit compaction speed 2019-05-03 17:22:39 -07:00
Chris Lu 3b3651dea3 volume: atomic copying file, adds version and stopOffset 2019-04-19 12:29:49 -07:00
Chris Lu ac2727853f fix needle map entry size 2019-04-19 00:39:34 -07:00
Chris Lu e5506152c0 refactoring 2019-04-18 21:43:36 -07:00
Chris Lu 000ee725fc refactor Offset into a struct of bytes 2019-04-08 19:40:56 -07:00
Chris Lu df95ce0b6c weed backup: efficient delta backup 
fix https://github.com/chrislusf/seaweedfs/issues/399
 2019-03-25 23:01:53 -07:00
Chris Lu 70815e9124 WIP 2019-03-25 09:16:12 -07:00
Chris Lu da871896c3 weed filer: set grpc port to port + 10000 2019-03-19 05:47:41 -07:00
Chris Lu 916b809c08 add error checking 2019-03-19 05:34:43 -07:00
bingoohuang ab6be025d7 go fmt and fix some typo 2019-01-17 09:17:19 +08:00
bingoohuang 6ddfaf33cb extract VolumeFileScanner for ScanVolumeFile 2019-01-16 17:48:59 +08:00
Chris Lu 9383c91eb1 wait to read again if the volume is compacting 2019-01-05 19:52:17 -08:00
Chris Lu 1478d7ea21 reduce file seek when writing 2018-12-31 15:08:32 -08:00
Chris Lu f050b22d6c close file in ScanVolumeFile() 
fix https://github.com/chrislusf/seaweedfs/issues/761
 2018-11-04 00:28:24 -07:00
Chris Lu 496a7398d1 adjust logging 2018-10-18 20:47:30 -07:00
Chris Lu 333709657c adjust log level 2018-10-15 00:40:12 -07:00
Chris Lu a362261972 avoid possible racing issue if leveldb or boltdb is used. 
remove the to-be-generated files during commit

fix https://github.com/chrislusf/seaweedfs/issues/738
 2018-09-26 18:45:51 -07:00
Chris Lu ea8ecf35f4 set needle AppendAtNs value during writes 2018-07-24 02:44:33 -07:00
Chris Lu 4f317c7e3d working version3 2018-07-24 01:36:04 -07:00
Chris Lu 7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu d4d7ced922 refactoring: add type for needle id, offset 
later the type size can possibly be adjusted
 2018-07-08 02:28:04 -07:00
Chris Lu defcd4edc5 adjust log level 2018-07-07 00:48:58 -07:00
Chris Lu 9bcaa65574 avoid divided by 0 
fix https://github.com/chrislusf/seaweedfs/issues/650
 2018-06-25 01:20:15 -07:00
Chris Lu 9ba335a7c4 prepare for flexible super block 2018-06-24 11:37:08 -07:00
Chris Lu 7f7e4e9885 fix error for deleted files during compaction 
deletion during commit may cause trouble when make up the difference during commitCompact()
 2018-06-23 18:24:59 -07:00
Chris Lu 58344980e4 preallocate disk space during compaction also, add cleanup for failed compaction 2017-08-29 23:59:53 -07:00
Chris Lu 5047bdb4a2 skip bytes cache 2017-06-03 11:44:24 -07:00
Chris Lu 82c3ccc8dd add btree for volume index 2017-05-26 22:51:25 -07:00
brstgt 4fb5bb09b6 Remove obsolete property Volume.dataFileSize 2017-01-20 16:31:11 +01:00
Chris Lu ed44f12f6d support Fallocate on linux 2017-01-08 11:01:46 -08:00
Chris Lu 13e7069eb9 keep track of total data file size 2017-01-06 10:22:20 -08:00
霍晓栋 ce1f7ab662 makediff func with UT case 2016-10-07 16:22:24 +08:00
霍晓栋 ed848425c7 supplemental data between compacting and commit compacting 2016-09-29 13:57:23 +08:00
Chris Lu dffad65f2f fix compilation 2016-09-26 22:30:44 -07:00
Chris Lu 7e29218327 add a template for makeupDiff 2016-09-26 22:26:41 -07:00
Chris Lu 1bc041b46d add a new way to manually compact corrupted volume 
fix https://github.com/chrislusf/seaweedfs/issues/371
 2016-09-22 20:31:20 -07:00
Chris Lu 5ce6bbf076 directory structure change to work with glide 
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
 2016-06-02 18:09:14 -07:00
Renamed from go/storage/volume_vacuum.go (Browse further)