Commit graph

128 commits

Author SHA1 Message Date
Chris Lu da9abc2a93 a start for distributed master support, not working yet 2013-12-09 13:34:05 -08:00
Chris Lu 11e91bd549 mux router cannot handle "/" correctly. switching it off for volume
servers
2013-12-09 13:27:09 -08:00
Chris Lu 2e2f426fe2 clean up etcd backing for sequence persistent storage. It uses some OS
specific calls and is not OS-agnostic.
2013-12-03 23:30:55 -08:00
Chris Lu 5fdb1d89ce adding "server" command to start both volume server and an embedded
master server
2013-12-03 23:22:26 -08:00
Chris Lu eeeccf2f78 ssue 59: Multi Ip Enviromment support + publicUrl is disregarded by
master
2013-12-02 22:58:27 -08:00
Chris Lu e449a4a517 use glog 2013-12-02 01:59:04 -08:00
Chris Lu a234789c59 switch to glog debugging 2013-12-02 01:54:05 -08:00
Chris Lu c38eee73ca refactoring to separate master and volume server, so that these servers
can be embedded into other applications
2013-12-02 01:37:36 -08:00
Chris Lu bc2f3b26e7 refactoring, start to use gorilla/mux 2013-12-01 14:41:47 -08:00
Chris Lu bd0c7a3d28 comment out etcd support due to its os specific system calls 2013-11-19 01:43:16 -08:00
Chris Lu fa1bb3cee3 version 0.45! 2013-11-18 23:54:01 -08:00
Chris Lu ac66eee045 added download command to download file by fileid 2013-11-18 23:41:00 -08:00
Chris Lu 47aea42d66 fix filename not set error 2013-11-18 23:16:21 -08:00
Chris Lu 8111f7663d formatting 2013-11-18 23:04:33 -08:00
Chris Lu 0e5e0a3754 add option to split large files into parts and then upload 2013-11-18 21:47:31 -08:00
Chris Lu 3b68711139 support for collections! 2013-11-12 02:21:22 -08:00
Chris Lu 1888d01fa0 adding etcd storage support for cluster meta data. Currently just
sequence. More to come...
2013-11-10 01:31:50 -08:00
Chris Lu b579451db9 this is a fix important enough for a version bump 2013-10-31 12:57:32 -07:00
Chris Lu cca1df83da debug message 2013-10-31 12:56:05 -07:00
Chris Lu 54723c3713 Issue 51: Assign on empty cluster sometime fails under high concurrency
load

Contributed by philoops
2013-10-29 12:48:31 -07:00
Chris Lu c43dcfa3e2 prepare for v0.43 release 2013-10-16 08:39:36 -07:00
Chris Lu 3f5f8657d2 add a command to force compaction of a volume, removing deleted files 2013-09-28 22:18:52 -07:00
Chris Lu 738e528329 v0.42
Changes: 
* fixed issue 45
2013-09-19 11:38:52 -07:00
Chris Lu 69ac6b6bf6 Issue 45 in weed-fs: [Compact issue] Offset overflow
New issue 45 by hieu.hcmus@gmail.com: [Compact issue] Offset overflow
http://code.google.com/p/weed-fs/issues/detail?id=45

You are using uint32(Maximum 4Gb) to store needle offset(Maximum 32Gb)
when compacting.
Currently It is ok if the volume size is < 4gb
Change variable "offset" in ScanVolumeFile function to uint64 to fix the
issue.
2013-09-19 11:06:14 -07:00
Chris Lu 82b74c7940 issue 43 "go fmt" chagnes from "Ryan S. Brown" <sb@ryansb.com>
some basic changes to parse upload url
2013-09-01 23:58:21 -07:00
Chris Lu 2e70cc8be7 default to use os.TempDir() instead of hard coded "/tmp" 2013-08-28 10:39:15 -07:00
Chris Lu 8e1ac16d16 refactoring submit operation 2013-08-14 10:07:42 -07:00
Chris Lu d5e7c1de0a refactoring code
reusable code by go clients
2013-08-13 23:26:51 -07:00
Chris Lu e45c6b5e21 add white list to both master and volume servers
prepare for v0.41
2013-08-13 09:31:19 -07:00
Chris Lu 3572e1140e adjusting parameter names 2013-08-13 09:22:06 -07:00
Chris Lu 078118ecba v0.40 2013-08-12 23:48:10 -07:00
Chris Lu 11b4e0c77d fix syntax error 2013-08-12 21:43:13 -07:00
Chris Lu e0951dd44f added write whiteList, to make it a bit easier to secure volume servers
on hosting environments.
2013-08-12 21:27:47 -07:00
Chris Lu 0deda9b638 prepare to release 0.38 2013-08-11 11:50:18 -07:00
Chris Lu 7cef280bdc handle cases when .idx files are also readonly
adjusting log level
2013-08-11 11:38:55 -07:00
Chris Lu 27f04a382a avoid changing max size 2013-08-09 00:17:07 -07:00
Chris Lu ed154053c8 switching to temporarily use glog library 2013-08-08 23:57:22 -07:00
Chris Lu f457eef144 add /submit handler for both master and volume server 2013-08-06 13:23:10 -07:00
Chris Lu d51c8d37bd Issue 6: upload file without id specified 2013-08-06 12:13:10 -07:00
Chris Lu 54906c48f3 report errors when upload timeouts 2013-08-05 13:37:41 -07:00
Chris Lu e94d52c1af replicate mime type 2013-07-29 10:09:36 -07:00
Chris Lu fb53ec30f7 release 0.37 2013-07-28 22:54:06 -07:00
Chris Lu 63ebafa113 fix when volume growth are partial successful in automatic mode 2013-07-24 10:31:51 -07:00
Chris Lu 3ae2eabf09 add more description to upload command 2013-07-20 15:14:12 -07:00
Chris Lu 1b3f204220 release 0.36 2013-07-20 14:53:20 -07:00
Chris Lu b409ccc5ab upload respect master default replication setting 2013-07-15 21:34:04 -07:00
Chris Lu 5e3b6e968a escaping upload file name special characters
if already gzipped, avoid gzipping again
2013-07-15 17:26:00 -07:00
Chris Lu dd66193bfb remove logging 2013-07-15 16:44:24 -07:00
Chris Lu 524a44f027 bump up the versoin 2013-07-15 16:27:11 -07:00
Chris Lu 70fe7e6b5d support gzip file upload, fix problem during replication of gzipped data 2013-07-15 11:04:43 -07:00
Chris Lu c6bd4e656e add support for http://localhost:8080/vid/key_cookie
or
http://localhost:8080/vid/key_cookie.txt
2013-07-13 23:46:12 -07:00
Chris Lu de368ef3c3 adjust usage 2013-07-13 23:19:21 -07:00
Chris Lu d9c25ccbb0 upload files recursively matching a pattern 2013-07-13 23:17:41 -07:00
Chris Lu ddf4f27a56 better error message if directory is not found 2013-07-13 22:01:48 -07:00
Chris Lu 76ba0021e5 prepare for 0.34 release 2013-07-13 19:46:26 -07:00
Chris Lu ae3245f1dc Fix a file name parsing error 2013-07-13 15:57:44 -07:00
Chris Lu 72490bc00f Consistent result format when error happens 2013-07-13 14:35:29 -07:00
Chris Lu 208ed2a991 Add support to upload a directory recursively. 2013-07-13 14:14:16 -07:00
Chris Lu d4105f9b46 add support for multiple folders and multiple max limit: eg
-dir=folder1,folder2,folder3 -max=7,8,9
2013-07-13 11:38:01 -07:00
Chris Lu 175456870a bump up to 0.33 2013-07-12 14:35:29 -07:00
Chris Lu 49cc81fdab added support for /vid/keycookie/filename.txt format for HTTP GET 2013-07-12 14:19:11 -07:00
Chris Lu b0e5112702 "weed upload" return message changes 2013-07-11 22:45:29 -07:00
Chris Lu 4c200acd7d 1. ensure replicated file has the same timestamp
2. upload can specify modified time by &ts=...
3. correctly return code 304
2013-07-10 00:25:14 -07:00
Chris Lu 53b6831f73 bump up version number to 0.32 2013-07-09 10:50:48 -07:00
Chris Lu cbd9d14cc4 Issue 27: feature request - Last-Modified header 2013-07-08 23:38:38 -07:00
Chris Lu 53ae13a012 change default number of volumes to 7, so that the first volume growth
would not fail.
2013-07-08 01:58:28 -07:00
Chris Lu 50269b74ce add dataCenter option when assign file keys
add dataCenter option when starting volume servers
some work related to freeze a volume. Not tested yet.
2013-06-19 18:10:38 -07:00
Chris Lu 915b16f97a refactoring, same logic, but the store replication logic is moved to a
stand-alone file, for later easier improvements
2013-04-16 00:10:21 -07:00
Chris Lu 4d8ce2fe26 adding HEAD support 2013-03-20 22:57:41 -07:00
Chris Lu 437532d559 0.31 is next release 2013-03-20 22:08:54 -07:00
Chris Lu 0c75e0438e bump to 0.30 since it fixed an important bug 2013-03-20 01:39:20 -07:00
Chris Lu 2d0a555270 bump up the version 2013-03-19 11:28:11 -07:00
Chris Lu a3eddde525 fix documentation error 2013-03-18 21:29:25 -07:00
Chris Lu 9bbbbb7c2c releasing version 0.28 2013-03-08 16:50:40 -08:00
Chris Lu db8e27be6e add lots of error checking by GThomas 2013-02-26 22:54:22 -08:00
Chris Lu 38f4425a65 checking error 2013-02-10 13:41:25 -08:00
Chris Lu 79d11ac951 go vet 2013-02-10 09:44:44 -08:00
Chris Lu d4e5a22e53 rename from cmd to weed for easier "go build" 2013-02-10 04:05:28 -08:00