Commit graph

8 commits

Author SHA1 Message Date
chrislusf b39c384d6d Revert "Merge pull request #284 from thinxer/binary"
This reverts commit 3523ad5239, reversing
changes made to 5d100994b1.
2016-04-10 00:24:22 -07:00
Jianfei Wang 18254d9aae replace util/bytes.go with binary.BigEndian 2016-04-09 14:38:01 +08:00
chrislusf 5d100994b1 Revert "Merge pull request #281 from thinxer/binary"
This reverts commit a87fe8ffce, reversing
changes made to 6876bfa685.
2016-04-08 15:52:03 -07:00
Jianfei Wang 7bc10d46e3 replace util/bytes.go with binary.BigEndian 2016-04-06 17:08:52 +08:00
chrislusf 86cd40fba8 Add "weed backup" command.
This is a pre-cursor for asynchronous replication.
2015-05-26 00:58:41 -07:00
Stuart P. Bentley f0c2a2dcb3 Change all chrislusf/weed-fs links to point to chrislu/seaweedfs 2015-04-16 19:18:06 +00:00
chrislusf b96911c517 remove verbose logging. 2015-03-29 11:44:16 -07:00
chrislusf 1b6ab2f6af Add boltdb for volume needle map
boltdb is fairly slow to write, about 6 minutes for recreating index
for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from
disk, and generate the boltdb as large as 134,217,728 bytes in 6
minutes.

To compare, for leveldb, it recreates index in leveldb as large as
27,188,148 bytes in 8 seconds.
For in memory version, it loads the index in

To test the memory consumption, the leveldb or boltdb index are
created. And the server is restarted. Using the benchmark tool to read
lots of files. There are 7 volumes in benchmark collection, each with
about 1553K files.
For leveldb, the memory starts at 142,884KB, and stays at 179,340KB.
For boltdb, the memory starts at 73,756KB, and stays at 144,564KB.
For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.
2015-03-29 11:04:32 -07:00