gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
1012 commits 28 branches 278 tags 135 MiB
Commit graph

159 commits

Author SHA1 Message Date
Chris Lu 766512c475 close current needle map when switching 2016-05-09 10:04:21 -07:00
Chris Lu 1d86ac8761 just refactoring 2016-04-26 20:45:35 -07:00
Chris Lu c9e33d3709 separate into 2 files, no logic change 2016-04-26 20:10:26 -07:00
chrislusf adcfaa5735 correct cache: fix racing condition 2016-04-17 12:03:45 -07:00
chrislusf 6dd257a81f adjust to real case cache size 2016-04-15 12:06:43 -07:00
chrislusf b03e7b26b5 add []byte caching and pooling 
fixes https://github.com/chrislusf/seaweedfs/issues/211
 2016-04-15 11:56:53 -07:00
chrislusf 0649d778a7 pooling []byte 
reduce the number of requests to make([]byte)
 2016-04-14 01:30:26 -07:00
chrislusf 112e2c6922 clearer error message 2016-04-10 01:50:58 -07:00
Jianfei Wang eec72b5136 storage: faster ParseKeyHash 2016-04-10 15:54:40 +08:00
Jianfei Wang 5364b3d8f5 storage: add test for ParseKeyHash 2016-04-10 15:52:18 +08:00
chrislusf b39c384d6d Revert "Merge pull request #284 from thinxer/binary" 
This reverts commit 3523ad5239, reversing
changes made to 5d100994b1.
 2016-04-10 00:24:22 -07:00
Jianfei Wang a192373c25 storage: do not copy sync.Mutex by value 2016-04-10 13:47:03 +08:00
Jianfei Wang 18254d9aae replace util/bytes.go with binary.BigEndian 2016-04-09 14:38:01 +08:00
Jianfei Wang f0cc264757 storage: add test for ParseKeyHash 2016-04-09 11:33:06 +08:00
chrislusf 5d100994b1 Revert "Merge pull request #281 from thinxer/binary" 
This reverts commit a87fe8ffce, reversing
changes made to 6876bfa685.
 2016-04-08 15:52:03 -07:00
Jianfei Wang 7bc10d46e3 replace util/bytes.go with binary.BigEndian 2016-04-06 17:08:52 +08:00
Alexandr Razumov ca2a6cc36b use klauspost/crc32 instead of hash/crc32 2016-04-01 22:18:58 +03:00
chrislusf e921cb1a9d format changes 2015-12-14 22:38:58 -08:00
tnextday 662915e691 Delete all chunks when delete a ChunkManifest 
LoadChunkManifest can uncompress buffer
move compress.go from storage to operation because of import cycle
MakeFile add cross complete command
 2015-12-02 21:27:29 +08:00
tnextday 2c0a7fe75e complete submit chunked file 2015-12-02 15:00:46 +08:00
tnextday de5e07ce3e rename FlagChunkedFile to FlagChunkManifest 2015-12-01 20:53:44 +08:00
tnextday 6b0894d806 update ChunkedFile to seekable reader, so we can use io.* to read data 2015-12-01 20:23:50 +08:00
tnextday 1817864a45 add chunk file helper to support large file 2015-11-29 23:49:41 +08:00
Timur Nurutdinov 4b0904e9c4 fix extract collection name from .dat file 2015-08-18 19:58:45 +03:00
chrislusf 320e946d50 fix ttl change detection 
https://github.com/chrislusf/seaweedfs/issues/166
 2015-07-17 19:30:25 -07:00
chrislusf 430f371a97 fix wrong logic 2015-07-11 12:20:39 -07:00
chrislusf 9b8f8f4bb9 add logging 2015-07-11 12:20:09 -07:00
chrislusf 2c595d2d16 skip isFileUnchanged checking since ttl always change 
skip checking since ttl always change. Fixing
https://github.com/chrislusf/seaweedfs/issues/166
 2015-07-10 09:43:49 -07:00
chrislusf a4d3f2da3d Avoid nil case when fixing data 2015-07-08 23:21:56 -07:00
chrislusf 418878edce In case lastNode==0, need to set it to -1 
Fix https://github.com/chrislusf/seaweedfs/issues/156
 2015-06-23 22:49:09 -07:00
chrislusf 86cd40fba8 Add "weed backup" command. 
This is a pre-cursor for asynchronous replication.
 2015-05-26 00:58:41 -07:00
chrislusf 36a31771f1 Add Key String() function 2015-05-25 23:50:19 -07:00
chrislusf 8f88d382a5 Rename variables 2015-05-23 10:16:01 -07:00
chrislusf 85bfab612c Add comments 2015-05-23 10:13:26 -07:00
chrislusf c7cf4fa46c Fix looking for master nodes. 2015-05-22 08:12:51 -07:00
chrislusf 3188382ea7 More debug information for connecting to master 2015-05-14 21:46:59 -07:00
chrislusf b8314fb054 Textual changes. 2015-05-08 23:44:11 -07:00
chrislusf 66317b8438 Increase compact revision during each compaction 2015-05-08 23:43:59 -07:00
chrislusf dac3b592ed Add compact revision in volume super block 2015-05-08 23:34:14 -07:00
yanyiwu dc47db5512 unittest: TestSortVolumeInfos 2015-04-21 01:22:13 +08:00
yanyiwu b7a18580b8 Sort VolumeInfos by VolumeId in Store.Status(); 
Ordered VolumeInfos is more Human-readable,
especially when there is a lot of volumes.
 2015-04-21 01:21:21 +08:00
chrislusf 5c81fa1075 Prepend prefix "0" for odd number of hex characters. 2015-04-19 18:54:05 -07:00
Stuart P. Bentley f0c2a2dcb3 Change all chrislusf/weed-fs links to point to chrislu/seaweedfs 2015-04-16 19:18:06 +00:00
chrislusf 49d639ecab Add error checking for file reads. 2015-04-14 23:05:33 -07:00
chrislusf b96911c517 remove verbose logging. 2015-03-29 11:44:16 -07:00
chrislusf 1b6ab2f6af Add boltdb for volume needle map 
boltdb is fairly slow to write, about 6 minutes for recreating index
for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from
disk, and generate the boltdb as large as 134,217,728 bytes in 6
minutes.

To compare, for leveldb, it recreates index in leveldb as large as
27,188,148 bytes in 8 seconds.
For in memory version, it loads the index in

To test the memory consumption, the leveldb or boltdb index are
created. And the server is restarted. Using the benchmark tool to read
lots of files. There are 7 volumes in benchmark collection, each with
about 1553K files.
For leveldb, the memory starts at 142,884KB, and stays at 179,340KB.
For boltdb, the memory starts at 73,756KB, and stays at 144,564KB.
For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.
 2015-03-29 11:04:32 -07:00
chrislusf 0d23f49f74 removing unused files 2015-03-27 16:35:42 -07:00
chrislusf 020ba6c9a8 add leveldb support for needle map 
This supposedly should reduce memory consumption. However, for tests
with millions of, this shows consuming more memories. Need to see
whether this will work out. If not, later boltdb will be tested.
 2015-03-27 16:34:58 -07:00
chrislusf 78bc7b393f tweaking ui 2015-03-22 12:50:04 -07:00
chrislusf b07d81fb08 follow golint suggestions 2015-03-10 00:20:34 -07:00