Commit graph

263 commits

Author SHA1 Message Date
Chris Lu a4cef2fbd4 only cares about error other than EOF 2013-02-11 23:54:21 -08:00
Chris Lu 91492b4947 avoid EOF error on windows. This is a "try to load" function anyway. 2013-02-11 23:38:48 -08:00
Chris Lu b395a5289b do not truncate file if any error happens. Truncating will lose all
files after this file entry.
2013-02-11 22:54:46 -08:00
Chris Lu 2ca983b6d8 print out error message 2013-02-11 22:37:13 -08:00
Chris Lu aecea22666 Issue 15: Stress test corrupts volume 2013-02-10 14:00:06 -08:00
Chris Lu 38f4425a65 checking error 2013-02-10 13:41:25 -08:00
Chris Lu 79d11ac951 go vet 2013-02-10 09:44:44 -08:00
Chris Lu d4e5a22e53 rename from cmd to weed for easier "go build" 2013-02-10 04:05:28 -08:00
Chris Lu 37a1a70cc3 remove unused files 2013-02-10 03:53:52 -08:00
Chris Lu 7743ddd7db Merge remote-tracking branch 'origin/master' 2013-02-10 03:51:04 -08:00
Chris Lu 5071f528f6 testing compilation with remove package 2013-02-10 03:49:51 -08:00
Chris Lu 7e0951ad9c testing compilation with remove package 2013-02-10 03:40:37 -08:00
Chris Lu 55f2627fcf testing new directory structure 2013-02-10 03:31:40 -08:00
Chris Lu ab6fb13ad7 avoid the "src" folder 2013-02-10 03:25:35 -08:00
Chris Lu 1b6f53cdac remove binary accidentally checked in 2013-02-10 03:12:58 -08:00
Chris Lu cb4e8ec16b re-organize code directory structure 2013-02-10 03:09:26 -08:00
Chris Lu d3b267bac2 Issue 15: Stress test corrupts volume
Checked in for GThomas(tgulacsi78)

I've also met with partial writes (pipe closed on localhost), and also
met with real corruption: after some partial write, the data size in
needle header was read as some huuuuge number, which resulted in memory
panic.

Please consider the attached patch for ensuring full writes (seeks back
to the beginning on needle append error). Hope this helps.
Tested with a small tmpfs, lot of "no space left on device" :)
(sudo umount -lf /tmp/weed; mkdir -p /tmp/weed && sudo mount -o
size=128M,mode=4777 -t tmpfs tmpfs /tmp/weed && bin/weed master
-mdir=/tmp/weed -debug=true & bin/weed volume -dir=/tmp/weed
-debug=true)

GThomas
2013-02-06 14:30:44 -08:00
Chris Lu 0b7a235c17 ensure file name does not contain any "/" 2013-01-22 15:07:51 -08:00
Chris Lu f2af477324 added file name format when exporting. patch from Tamás Gulácsi 2013-01-22 15:07:19 -08:00
Chris Lu 84e1a59df7 add option to output to stdout - patch from Tamás Gulácsi
report file size correctly
2013-01-21 17:50:10 -08:00
Chris Lu 030dde227a promptive error message 2013-01-20 20:01:59 -08:00
Chris Lu 06444b7c52 up the version to 0.28 beta 2013-01-20 19:57:45 -08:00
Chris Lu c3ef8b433c adjust text 2013-01-20 19:51:27 -08:00
Chris Lu 97556f6023 formatting 2013-01-20 19:47:04 -08:00
Chris Lu 81b9175c7a re-factoring volume file scanning code
add export function
fix bugs on deleted byte counting
2013-01-20 19:44:23 -08:00
Chris Lu f35b958d90 adding visit function for the NeedleMap 2013-01-20 11:27:32 -08:00
Chris Lu 866d15023d better error handling, check in for Gulácsi Tamás 2013-01-20 03:40:04 -08:00
Chris Lu 768eaff4b8 fix debug message error 2013-01-19 19:58:31 -08:00
Chris Lu ec7dfeeaf8 ignore compiled binary file 2013-01-19 19:50:26 -08:00
Chris Lu 921f1c626a fix debug message displaying 2013-01-19 19:49:57 -08:00
Chris Lu b0c7df0c3b go fmt for all source codes 2013-01-17 00:56:56 -08:00
Chris Lu ca9056d673 cosmetics change 2013-01-17 00:22:18 -08:00
Chris Lu 84c9ac66ac add git ignore 2013-01-17 00:17:00 -08:00
Chris Lu 50fafa3a2c Revert "sort writables to be able to skew write probability to lesser ids (older volumes)"
This reverts commit ce7394f3bf.
2013-01-17 00:15:13 -08:00
Chris Lu a502b78c3f Revert "add error return for (Un)GzipData"
This reverts commit 8243710351.
2013-01-17 00:15:09 -08:00
Chris Lu bf9c4ed033 Revert "add cmd/dump - a dumper"
This reverts commit 5d2a1e8d48.
2013-01-17 00:15:05 -08:00
Chris Lu d1494ea786 Revert "add CdbMap"
This reverts commit bf0ccf3461.
2013-01-17 00:15:01 -08:00
Chris Lu 5b49065a57 Revert "add Frozen attribute to VolumeInfo"
This reverts commit dd685fdd8d.
2013-01-17 00:14:58 -08:00
Chris Lu 3b21317863 Revert "add "freeze" subcommand to volume"
This reverts commit f262fed197.
2013-01-17 00:14:52 -08:00
Chris Lu ee08e38126 Revert "fix errors with frozen volume loading"
This reverts commit 92ffba2ab9.
2013-01-17 00:14:45 -08:00
Chris Lu be83a56bb9 Merge remote-tracking branch 'choose_remote_name/cdb' 2013-01-15 18:08:19 -08:00
Chris Lu 5f278c6bd7 move to 0.27 2013-01-14 14:23:43 -08:00
Tamás Gulácsi 92ffba2ab9 fix errors with frozen volume loading 2013-01-14 22:18:00 +01:00
Tamás Gulácsi f262fed197 add "freeze" subcommand to volume 2013-01-14 21:42:35 +01:00
Tamás Gulácsi dd685fdd8d add Frozen attribute to VolumeInfo 2013-01-13 17:07:38 +01:00
Tamás Gulácsi bf0ccf3461 add CdbMap 2013-01-12 19:56:47 +01:00
Tamás Gulácsi 5d2a1e8d48 add cmd/dump - a dumper
Walk needed to be added to NeedleMap and CompactMap, to be able to add WalkKeys and WalkValues to volume. This is needed for iterating through all the stored needles in a volume - this was dump's purpose.
2013-01-05 23:06:44 +01:00
Tamás Gulácsi 8243710351 add error return for (Un)GzipData 2013-01-04 23:33:07 +01:00
Tamás Gulácsi ce7394f3bf sort writables to be able to skew write probability to lesser ids (older volumes) 2013-01-04 22:49:40 +01:00
Chris Lu 8fb8ebdf14 master can redirect GET/DELETE/POST requests to volume servers. So
clients does not always need to remember the volume server locations.
This is good for low~medium sized web site traffic.
2013-01-04 01:02:52 -08:00