gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
853 commits 28 branches 278 tags 135 MiB
Commit graph

17 commits

Author SHA1 Message Date
Stuart P. Bentley 1a194a578c More consistency with SeaweedFS name 
Among the changes, this replaces a couple instances of "Seaweed File System"
with "SeaweedFS", for the same reason that nobody says "Mongo Data Base".
 2015-04-16 21:11:25 +00:00
Stuart P. Bentley f0c2a2dcb3 Change all chrislusf/weed-fs links to point to chrislu/seaweedfs 2015-04-16 19:18:06 +00:00
chrislusf 1b6ab2f6af Add boltdb for volume needle map 
boltdb is fairly slow to write, about 6 minutes for recreating index
for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from
disk, and generate the boltdb as large as 134,217,728 bytes in 6
minutes.

To compare, for leveldb, it recreates index in leveldb as large as
27,188,148 bytes in 8 seconds.
For in memory version, it loads the index in

To test the memory consumption, the leveldb or boltdb index are
created. And the server is restarted. Using the benchmark tool to read
lots of files. There are 7 volumes in benchmark collection, each with
about 1553K files.
For leveldb, the memory starts at 142,884KB, and stays at 179,340KB.
For boltdb, the memory starts at 73,756KB, and stays at 144,564KB.
For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.
 2015-03-29 11:04:32 -07:00
chrislusf 020ba6c9a8 add leveldb support for needle map 
This supposedly should reduce memory consumption. However, for tests
with millions of, this shows consuming more memories. Need to see
whether this will work out. If not, later boltdb will be tested.
 2015-03-27 16:34:58 -07:00
chrislusf 9d8a6d2562 adjust log printing 2015-03-09 01:10:04 -07:00
Chris Lu ca67ed69a1 Change name to Seaweed. 2014-11-28 17:02:10 -08:00
Chris Lu 179d36ba0e formatting code by: goimports -w=true . 2014-10-26 11:34:55 -07:00
wyy 4126280d55 use github.com/chrislusf instead of github.com/aszxqw 2014-09-25 16:57:22 +08:00
wyy 1cd19447e3 use github.com/aszxqw instead of code.google.com/p 2014-09-25 00:47:09 +08:00
Chris Lu 4b7b439be9 Reduce memory usage for "weed fix" 2014-05-31 17:10:51 -07:00
Chris Lu e6e85a6b2c truncate file content during creating 2014-03-09 18:50:09 -07:00
Chris Lu 3b68711139 support for collections! 2013-11-12 02:21:22 -08:00
Chris Lu 69ac6b6bf6 Issue 45 in weed-fs: [Compact issue] Offset overflow 
New issue 45 by hieu.hcmus@gmail.com: [Compact issue] Offset overflow
http://code.google.com/p/weed-fs/issues/detail?id=45

You are using uint32(Maximum 4Gb) to store needle offset(Maximum 32Gb)
when compacting.
Currently It is ok if the volume size is < 4gb
Change variable "offset" in ScanVolumeFile function to uint64 to fix the
issue.
 2013-09-19 11:06:14 -07:00
Chris Lu 82b74c7940 issue 43 "go fmt" chagnes from "Ryan S. Brown" <sb@ryansb.com> 
some basic changes to parse upload url
 2013-09-01 23:58:21 -07:00
Chris Lu ed154053c8 switching to temporarily use glog library 2013-08-08 23:57:22 -07:00
Chris Lu db8e27be6e add lots of error checking by GThomas 2013-02-26 22:54:22 -08:00
Chris Lu d4e5a22e53 rename from cmd to weed for easier "go build" 2013-02-10 04:05:28 -08:00
Renamed from go/cmd/fix.go (Browse further)