gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6268 commits 28 branches 278 tags 135 MiB
Commit graph

474 commits

Author SHA1 Message Date
Chris Lu 5e8bb86995 close the connection with better accouting 2020-04-20 02:53:50 -07:00
Chris Lu 2955b96ef1 refactor 2020-04-19 23:54:32 -07:00
Chris Lu ce3cb25cfb working for in memory single log buffer 2020-04-19 23:37:50 -07:00
Chris Lu e0f5996560 fix "call of Unmarshal passes non-pointer as second argument" 2020-04-17 10:04:41 -07:00
Chris Lu f5a748d33c refactoring 2020-04-16 02:55:09 -07:00
Chris Lu 23715af402
Merge pull request #1273 from wyxloading/config_race 
Fix data race about config
 2020-04-13 22:24:51 -07:00
Chris Lu 2b5c4fbbf3 tiered caching 
1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files
 2020-04-13 22:19:27 -07:00
Chris Lu f282ed444b refactoring 2020-04-13 21:58:10 -07:00
wuyuxiang b556d3d035 Fix data race about config 2020-04-14 10:22:40 +08:00
Chris Lu 5c9e91393f 1.74 2020-04-13 10:25:48 -07:00
Chris Lu 7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu 94e35cdb35 mount: fix 
fix bug found by git bisect, but I do not understand why it can cause error!
 2020-04-12 03:34:36 -07:00
Chris Lu 2a1f396df5 avoid duplicated setting chunks into cache 2020-04-12 01:06:50 -07:00
Chris Lu b9b7da905e handle nil chunk cache 2020-04-12 01:00:12 -07:00
Chris Lu df97da25f9 mount: add on disk caching 2020-04-11 21:12:41 -07:00
Chris Lu d7f3acb2c0 refactor 2020-04-11 12:45:24 -07:00
Chris Lu ed54682598 refactoring 2020-04-11 12:43:17 -07:00
Chris Lu e909b55633 refactoring 2020-04-11 12:39:46 -07:00
Chris Lu 43c62a8a95 1.73 2020-04-09 21:44:30 -07:00
Chris Lu 59f40e2027 volume: best effort to detect ip address 
fix https://github.com/chrislusf/seaweedfs/issues/1264
 2020-04-09 00:26:24 -07:00
Chris Lu f6a7e79dc3 weed shell: simplify CLI option for filer 2020-04-08 23:57:15 -07:00
Chris Lu 8764bdb9df 1.72 2020-04-08 22:35:14 -07:00
Chris Lu 00b993a234 add util for md5 2020-04-08 22:30:38 -07:00
Chris Lu b51d8f03bb 1.71 2020-04-05 22:04:16 -07:00
Chris Lu 91da7057b1 refactoring 2020-04-05 13:11:43 -07:00
Chris Lu 7bc3c93512 add util.PathJoin 2020-04-05 12:40:46 -07:00
Chris Lu 50a5018b7f writing meta logs is working 2020-03-30 01:19:33 -07:00
Chris Lu be6c2f8511 1.70 2020-03-29 13:40:55 -07:00
Chris Lu 057722bbf4 return part of the chunk if chunkview is not the full chunk 2020-03-29 00:54:39 -07:00
Chris Lu 005b4ab3fe 1.69 2020-03-27 04:51:31 -07:00
Chris Lu 5463560912 small test 2020-03-26 23:50:07 -07:00
Chris Lu 0a05052b5c 1.68 2020-03-26 22:22:36 -07:00
Chris Lu dd5b582d05 go fmt 2020-03-26 00:09:01 -07:00
Chris Lu 2e4fadd10a fix fscache move 2020-03-25 22:19:19 -07:00
Chris Lu c0f0fdb3ba refactoring 2020-03-23 00:01:34 -07:00
Chris Lu d6412f27a0 logging 2020-03-22 18:32:56 -07:00
Chris Lu 3b3e063f25 1.67 2020-03-20 23:42:49 -07:00
Chris Lu d3e4a31058 filer: fix where deletion can miss under high concurrency 2020-03-20 23:39:32 -07:00
Chris Lu 4bc19f1e52 master: fix connections count reporting 2020-03-20 22:41:37 -07:00
Chris Lu e1b5ac5c04 1.66 2020-03-20 20:39:48 -07:00
Chris Lu 165b0d22a4 1.65 2020-03-20 00:05:42 -07:00
Chris Lu db9854fa0a 1.64 2020-03-17 10:58:35 -07:00
Chris Lu cc52e8ca87 1.63 2020-03-14 14:25:07 -07:00
Chris Lu 91e0a987e4 1.62 2020-03-14 01:03:59 -07:00
Chris Lu e2e691d9c2 clean up, add test 2020-03-13 23:53:15 -07:00
Chris Lu 60f5f05c78 1.61 2020-03-10 13:52:49 -07:00
Chris Lu e6de42f888 1.60 2020-03-09 22:32:40 -07:00
Chris Lu c32f95c380 1.59 2020-03-08 22:02:34 -07:00
Chris Lu 2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu ea1169dc80 filer cipher: single chunk http POST and PUT and read 2020-03-07 06:08:08 -08:00
Chris Lu 13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu f90c43635d refactoring 2020-03-04 00:39:47 -08:00
Chris Lu 4532640ffd 1.58 2020-03-01 17:59:12 -08:00
Chris Lu ed0acd1722 go fmt 2020-02-26 16:52:57 -08:00
Chris Lu 37a3628b2e grpc add PermitWithoutStream to client and server options 2020-02-26 16:51:38 -08:00
Chris Lu 892e726eb9 avoid reusing context object 
fix https://github.com/chrislusf/seaweedfs/issues/1182
 2020-02-25 21:50:12 -08:00
Chris Lu 6ea394b802 1.57 2020-02-22 21:34:49 -08:00
Chris Lu 621cdbdf58 filer: avoid possible timeouts for updates and deletions 2020-02-20 15:44:17 -08:00
Chris Lu 443a68484a 1.56 2020-02-17 12:54:45 -08:00
Chris Lu 9232d3ac68 ignore error when counting closed connections 2020-02-14 10:28:02 -08:00
Chris Lu 96c1ae8471 refactoring the close http response 2020-02-14 09:46:36 -08:00
Chris Lu cf5064d702 properly close http response 2020-02-14 09:09:15 -08:00
Chris Lu 62c34454d8 1.55 2020-02-11 23:00:31 -08:00
Chris Lu 33b3bd467c Revert "HEAD operation changes to fasthttp" 
This reverts commit 58f126fd27.
 2020-02-10 13:43:53 -08:00
Chris Lu 58f126fd27 HEAD operation changes to fasthttp 2020-02-10 09:13:29 -08:00
Chris Lu 29945fad51 1.54 2020-02-09 18:07:17 -08:00
Chris Lu beb0a1599e 1.53 2020-01-31 00:32:09 -08:00
Chris Lu a80ecbfe84 s3: add s3 copy 
fix https://github.com/chrislusf/seaweedfs/issues/1190
 2020-01-31 00:11:12 -08:00
Chris Lu 3b043ead49
Merge pull request #1189 from eryx67/compression-wav 
Support wav files compression
 2020-01-29 22:37:35 -08:00
Chris Lu abf90ad7b7 remove testing for viper 2020-01-29 21:38:53 -08:00
Chris Lu 20b1fb9ab7 fix test 2020-01-29 21:24:06 -08:00
eryx67 2a87087590 support wav files compression 2020-01-30 10:11:47 +05:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu 27b94cb65b fix wrong url 
fix https://github.com/chrislusf/seaweedfs/issues/1187
 2020-01-28 00:49:47 -08:00
Chris Lu 234f69452b 1.52 2020-01-26 21:04:40 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu 107e8a56ea retry context canceled request 2020-01-24 01:40:51 -08:00
Chris Lu 6a5c037099 fix http range requests 2020-01-22 22:59:12 -08:00
Chris Lu bb1be61602 1.51 2020-01-20 20:30:03 -08:00
Chris Lu 39edcfde4c filer: set inode 
avoid "getcwd: No such file or directory"
 2020-01-19 12:06:19 -08:00
Chris Lu 30c7148020 1.50 2020-01-08 22:08:35 -08:00
Chris Lu 89e16bd2e8 skip error when draining reader 
fix https://github.com/chrislusf/seaweedfs/issues/1179
 2020-01-08 18:07:07 -08:00
Chris Lu 66ab09aa4a 1.49 2020-01-08 09:51:35 -08:00
Chris Lu aa1807e082 1.48 2020-01-01 09:27:02 -08:00
Chris Lu c06f7eb48a load volume info from .vif file, use superblock as a backup 2019-12-28 12:28:58 -08:00
Chris Lu 1fd8926ac7 ignore draining error 2019-12-25 21:39:33 -08:00
divinerapier 4cbb6fa199
feat: drains http body if buffer is too small 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-12-26 11:28:43 +08:00
divinerapier 84640d07b7
fix: handle errors for ReadUrl 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-12-26 10:07:07 +08:00
Chris Lu 573b74f24f add SetDefault function for Configuration 2019-12-20 20:56:14 -08:00
Chris Lu 54369eccea 1.47 2019-12-17 15:11:01 -08:00
Chris Lu 6808bfec49 1.47 2019-12-17 15:08:45 -08:00
Chris Lu 392678f8f3 upload skipping mimetype if not needed 2019-12-15 20:57:08 -08:00
Chris Lu a631b2a9e9 1.46 2019-12-14 11:28:48 -08:00
Chris Lu 10bd3c6b4b refactoring 2019-12-08 19:44:16 -08:00
Chris Lu f81d43442b filer: speed up filer.meta.save by parallelizing 2019-12-06 22:20:59 -08:00
Chris Lu 0da7b894cc pass backend config from master to volume servers 2019-11-29 01:05:09 -08:00
Chris Lu a9dbd52e06
Merge pull request #1145 from stlpmo-jn/volume_graceful_stop 
let volume server graceful stop
 2019-11-29 00:11:57 -08:00
Chris Lu 249b9d8d5d 1.45 2019-11-22 14:51:16 -08:00
stlpmo 753138a3f7 test passed 2019-11-17 11:40:36 +08:00
Chris Lu de0c983808 refactoring scaffolding for master.toml 2019-11-14 22:20:38 -08:00
Lei Liu 46755ea1e1 fix master maintenance logic 
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
 2019-11-12 14:46:10 +08:00
stlpmo d07701fa75 delete unused function 2019-11-11 18:08:48 +08:00
stlpmo 62d393d6c9 ut pass 2019-11-11 10:52:21 +08:00
stlpmo 802a0eb3fe move from cmd-line to scaffold 2019-11-11 09:15:17 +08:00
Chris Lu 50e885da45 1.44 2019-10-11 23:56:58 -07:00
Chris Lu 20d90dea5a filer: avoid hard-coded upload timeout 2019-09-05 01:26:59 -07:00
Chris Lu 89a0cb2e11 1.43 2019-09-03 01:31:58 -07:00
Chris Lu 35d471a818
Revert "use std encoding lib" 2019-08-23 07:59:09 -07:00
divinerapier e798e4e9c5
use std encoding lib 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-08-19 18:07:54 +08:00
Chris Lu 3f851feb59 1.42 2019-07-21 21:52:48 -07:00
Chris Lu cf2804eebd 1.41 2019-07-07 20:40:20 -07:00
Chris Lu 23d36dfbaf 1.40 
enable azure replication
 2019-07-02 21:27:05 -07:00
Chris Lu 36022f9416 1.39 2019-06-30 00:50:17 -07:00
Chris Lu fdff927b37 1.38 2019-06-24 00:36:46 -07:00
Chris Lu 95ad56d99d 1.37 2019-06-21 20:56:34 -07:00
Chris Lu fe4eb6bffa 1.36 2019-06-21 13:11:40 -07:00
Chris Lu be3f1f84a2 1.35 2019-06-10 21:34:06 -07:00
Chris Lu ede876cfdb periodic scripts exeuction from leader master 2019-06-05 01:30:24 -07:00
Chris Lu 7e80b2b882 fix multiple bugs 2019-06-03 02:26:31 -07:00
Chris Lu f2c4c888f6 1.34 2019-05-17 22:53:30 -07:00
Chris Lu cf58fc0e63 refactor: extract out the write throttler 2019-05-06 13:56:08 -07:00
Chris Lu 8c982db25d 1.33 2019-05-03 17:27:50 -07:00
Wine93 32f93fb09e style: go fmt for all 2019-04-30 03:22:19 +00:00
Chris Lu 59d532a8c3 1.31 2019-04-21 14:09:50 -07:00
Chris Lu e5506152c0 refactoring 2019-04-18 21:43:36 -07:00
Chris Lu 79c2cca9c1 better error message 2019-04-16 01:15:30 -07:00
Chris Lu 967e108b9a increase max idle conn per hosts 2019-04-15 22:00:50 -07:00
Chris Lu 59a9e15733 1.30 2019-04-11 19:40:36 -07:00
Chris Lu e11e127d91 close grpc connections during errors 2019-04-11 09:27:08 -07:00
Chris Lu 6d4c44a85c 1.29 2019-04-08 22:41:59 -07:00
Chris Lu 88ba08cb9e add large disk support 2019-04-08 22:32:42 -07:00
Chris Lu d14b614407 weed filer.copy: use existing file owner and gropu id 2019-04-07 11:31:50 -07:00
Chris Lu c94823f52c set default http idle connection per host 
possible fix https://github.com/chrislusf/seaweedfs/issues/915
 2019-04-04 20:28:40 -07:00
Chris Lu 8c823abe1f 1.28 2019-03-30 23:22:41 -07:00
Chris Lu eaa42c3865 1.27 2019-03-23 14:14:40 -07:00
Chris Lu da871896c3 weed filer: set grpc port to port + 10000 2019-03-19 05:47:41 -07:00
Chris Lu 55bab1b456 add context.Context 2019-03-15 17:20:24 -07:00
Chris Lu 2ce4dae795 move to 1.26 2019-03-11 00:58:12 -07:00
Chris Lu 344caf3cd7 1.25 2019-02-25 00:44:48 -08:00
Chris Lu 07af52cb6f raft change from http to grpc 
master grpc port is fixed to http port + 10000
 2019-02-18 22:38:14 -08:00
Chris Lu 77b9af531d adding grpc mutual tls 2019-02-18 12:11:52 -08:00
Chris Lu 55761ae806 skip nil options 2019-02-16 12:49:58 -08:00
Chris Lu 98a03b38e5 avoid util package depends on security package 2019-02-16 12:23:35 -08:00
Chris Lu 1334507595 Revert "randomize based on self address" 
This reverts commit 6230eb28a6.
 2019-01-28 12:12:51 -08:00
Chris Lu 6230eb28a6 randomize based on self address 
fix #851
 2019-01-28 11:55:33 -08:00
Chris Lu adcfe66034 1.24 2019-01-26 00:15:50 -06:00
Chris Lu 8ec57e18c7 1.23 2019-01-18 14:27:34 -08:00
Chris Lu 67e2ea72be master add separate grpc port 
due to https://github.com/soheilhy/cmux/issues/64

fix https://github.com/chrislusf/seaweedfs/issues/820
fix https://github.com/chrislusf/seaweedfs/issues/840
fix https://github.com/chrislusf/seaweedfs/issues/841
 2019-01-18 14:14:47 -08:00
Chris Lu dfae0f4e9d 1.22 2019-01-18 00:25:20 -08:00
Chris Lu e84dcb0bfe fix timeout issue, but can not reproduce with local built binaries 
fix https://github.com/chrislusf/seaweedfs/issues/841
 2019-01-18 00:24:40 -08:00
Chris Lu b881e11a16 1.21 2019-01-16 20:44:38 -08:00
chenwanli 80cf99db1b Set rpc dial timeout to 5s 2019-01-11 18:23:31 +08:00
chenwanli 39c7455881 Set http timeout to 5s 2019-01-10 20:21:08 +08:00
Chris Lu e768e64fbc 1.20 2019-01-06 10:52:26 -08:00
Chris Lu ceafb7d145 1.19 2019-01-02 12:59:20 -08:00
Chris Lu 81904ad336 debugging: track all memory allocations 2018-12-31 14:54:13 -08:00
Chris Lu 16e8610426 1.18 2018-12-30 00:52:20 -08:00
Chris Lu d8c7844d2e 1.17 2018-12-29 13:16:38 -08:00
Chris Lu ba9b7b62d7 1.16 2018-12-28 15:10:49 -08:00
Chris Lu 332668a3fd 1.15 2018-12-23 12:54:41 -08:00
Chris Lu 16634235c1 1.14 and HCFS 1.0.3 2018-12-17 10:03:03 -08:00
Chris Lu 66d04cb83f 1.12 2018-12-15 08:17:24 -08:00
Chris Lu 36d1e739e6 1.11 2018-12-08 00:32:34 -08:00
Chris Lu 1bfb96f34d optimization for reading whole chunk with gzip encoding 2018-12-07 01:57:55 -08:00
Chris Lu 29f1673d97 refactoring 2018-12-07 01:25:01 -08:00
Chris Lu a3def2bbd9 for 1.10 2018-11-29 09:15:04 -08:00
Chris Lu 19d546938a forgot to upgrade version number 2018-11-21 23:30:59 -08:00
Chris Lu 06474e5990 1.08 2018-11-18 22:43:05 -08:00
Chris Lu f8eb988347 1.07 2018-11-18 08:10:15 -08:00
Chris Lu 2929563294 1.06 2018-11-14 23:51:15 -08:00
Chris Lu f224b9fe28 1.05 2018-11-13 23:54:17 -08:00
Chris Lu 0d095f72e4 update to 1.04 2018-11-08 21:42:45 -08:00
Chris Lu 6e53c38c2f update to 1.03 2018-11-08 07:37:44 -08:00
Chris Lu 6e11923551 upgrade to 1.02 2018-11-07 11:37:32 -08:00
Chris Lu 60db731e36 update to 1.01 2018-11-05 08:54:40 -08:00
Chris Lu 1a64a817ac update to 1.00 2018-11-04 22:13:16 -08:00
Chris Lu 249eeccf32 0.99 release 2018-10-14 00:19:04 -07:00
Chris Lu b7763c1cb6 releasing 0.98 2018-10-10 21:44:10 -07:00
Alex Wang 4db68669b6 [bugfix] Fix interrupt hook overwritten bug 2018-10-09 15:43:32 +08:00
Chris Lu 746017ea4f releasing 0.97 2018-09-27 21:51:41 -07:00
Chris Lu a6cfaba018 able to sync the changes 2018-09-21 01:54:29 -07:00
Chris Lu 865a017936 fix when if buffer is not aligned 2018-09-17 00:27:40 -07:00
Chris Lu 83ca595472 adjust version number 2018-09-07 14:19:48 -07:00
Chris Lu 7f760f16b0 prepare for 0.95 2018-09-03 20:52:52 -07:00
Chris Lu 35bafd9861 remove unused function 2018-09-03 16:50:32 -07:00
Chris Lu 06b81c063a fix possible connection counting error 2018-09-03 00:28:58 -07:00
Chris Lu 37e4e022fd prepare for 0.94 release 2018-08-23 00:02:13 -07:00
Chris Lu 53d171cf5b releasing 0.93 2018-08-19 18:45:26 -07:00
Chris Lu f827ada811 merge notification config with filer.toml 2018-08-19 15:17:55 -07:00
Chris Lu 0d98949199 tmp commit 2018-07-27 01:54:45 -07:00
Chris Lu 69d1b88b9e starting on 0.93 beta 2018-07-24 02:18:39 -07:00
Chris Lu e8a5ee4f48 releasing 0.92 
ensure the nil problem is resolved.
 2018-07-24 01:54:15 -07:00
Chris Lu a9069520ae start 0.92 beta2 2018-07-23 01:59:23 -07:00
Chris Lu 04356ad141 re-releasing 0.91 2018-07-23 01:58:13 -07:00
Chris Lu 5e55f0da4d add WIP 0.92 beta 2018-07-22 02:15:24 -07:00
Chris Lu 19c537b687 releasing 0.91 2018-07-22 02:12:02 -07:00
Chris Lu 77fc8c5914 keep alive for gRpc calls 2018-07-03 19:07:55 -07:00
Chris Lu c0122eeef3 start 0.91 beta 2018-06-24 09:29:34 -07:00
Chris Lu cf76711789 releasing 0.90 version 2018-06-23 18:36:27 -07:00
Chris Lu 4d1eedfa28 prepare to merge with master 2018-05-30 22:29:18 -07:00
Chris Lu 07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu 8b0718ac92 go vet 2018-05-27 11:56:49 -07:00
Chris Lu c11d84f314 fix reading from a url 2018-05-24 23:20:56 -07:00
Chris Lu d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu 67401f1908 fix gRpc nil response 2018-05-14 00:17:22 -07:00
Chris Lu 2dd5b777f7 prepare for 0.77 release 
right before changing to filer2
 2018-05-13 00:26:23 -07:00
kelgon 3bf883327e (fix #543)added body to error when Post encounter 4xx response 2017-07-28 16:11:23 +08:00
Chris Lu 1fbb8723dc add cpu and mem pprof 2017-06-22 01:33:58 -07:00
Chris Lu 5047bdb4a2 skip bytes cache 2017-06-03 11:44:24 -07:00
Chris Lu 021ef0d14b adjust version to 0.76 2017-05-26 23:04:38 -07:00
Chris Lu e3fc676260 add doc for 0.75 2017-05-26 23:01:40 -07:00
sparklxb c46e91d229 complement weed mount: add read and delete 2017-03-17 15:02:27 +08:00
Chris Lu 8de0027df5 Release 0.74 2017-01-18 09:39:47 -08:00
Chris Lu 76da296799 upgrade to 0.73 2017-01-15 12:29:53 -08:00
Chris Lu e767c3ea4f disable master connection timeout 
temporarily disable master connection timeout due to heartbeat
connection timeout
 2017-01-10 01:30:00 -08:00
wangjie 90a6f43c56 fix the bug than we can't get filename when download file. 2017-01-09 14:55:21 +08:00
Chris Lu dc42d15352 release 0.72 2017-01-03 21:32:59 -08:00
Chris Lu 8fb29e5230 change to 0.71 version 2017-01-03 21:17:54 -08:00
Chris Lu a57162e8bf delete operation does not need this checking 2016-07-21 00:40:13 -07:00
Chris Lu cdae9fc680 add "weed copy" command to copy files to filer 2016-07-20 23:45:55 -07:00
霍晓栋 dda13def2a refactor data integrity checking code 2016-07-03 12:53:03 +08:00
Chris Lu 5ce6bbf076 directory structure change to work with glide 
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
 2016-06-02 18:09:14 -07:00
Chris Lu 5071f528f6 testing compilation with remove package 2013-02-10 03:49:51 -08:00
Chris Lu ab6fb13ad7 avoid the "src" folder 2013-02-10 03:25:35 -08:00