gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
5891 commits 28 branches 278 tags 135 MiB
Commit graph

455 commits

Author SHA1 Message Date
Chris Lu 19537c9d21 1.96 2020-09-10 19:52:07 -07:00
Chris Lu 660d7c0edd 1.95 2020-09-10 01:32:05 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu 4fc0bd1a81 return http response directly 2020-09-09 03:53:09 -07:00
Chris Lu c18ea21f7a 1.94 2020-09-07 13:13:53 -07:00
Chris Lu 432fe3d926 1.93 2020-09-06 12:12:09 -07:00
Chris Lu b8f32bcab9 filer: compress stored metadata 2020-09-03 11:00:20 -07:00
Chris Lu dd52037c27 1.92 2020-08-30 22:56:35 -07:00
Chris Lu a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu b69cb74c03 read meta logs by timestamp 
pass in event ts when moving logs
meta aggregator reads in memory logs only
 2020-08-29 17:37:19 -07:00
Chris Lu ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu 0be6863c87 rename 2020-08-26 22:40:15 -07:00
Chris Lu 5f55a87101 close http response 2020-08-24 00:32:44 -07:00
Chris Lu 81a8aa6581 1.91 2020-08-23 17:28:40 -07:00
Chris Lu aee27ccbe1 multiple fixes 
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
 2020-08-23 15:48:02 -07:00
Chris Lu 99d05f758c adjust logs 2020-08-18 23:39:18 -07:00
Chris Lu 6a92f0bc7a refactoring to typed Size 
Go is amazing with refactoring!
 2020-08-18 17:04:28 -07:00
Chris Lu 09e126bae5 refactoring: use interface 2020-08-17 20:20:08 -07:00
Chris Lu be4d42b8e2 rename 2020-08-17 20:15:53 -07:00
Chris Lu 97e54a80d4 rename variables 2020-08-17 16:05:13 -07:00
Chris Lu ff200398bb 1.90 2020-08-16 21:18:50 -07:00
Chris Lu b71df82292 1.89 2020-08-16 00:57:28 -07:00
Chris Lu 003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu e74dc4e4bc add back fs node cache for renaming 2020-08-09 21:56:09 -07:00
Chris Lu b056707770 1.88 2020-08-08 22:55:12 -07:00
Chris Lu 20e2ac1add filer: store md5 metadata for files uploaded by filer 
fix https://github.com/chrislusf/seaweedfs/issues/1412
 2020-08-06 10:04:17 -07:00
Chris Lu 1b3a80dd3d non-fatal error 2020-08-01 13:46:52 -07:00
Chris Lu bbbea8159c http request use gzip if possible 2020-08-01 13:20:52 -07:00
Chris Lu 2f03481cb2 in case when content is not compressed 2020-08-01 13:08:10 -07:00
Chris Lu a6b59d50f7 1.87 2020-07-30 13:52:04 -07:00
Chris Lu 7a7e70b6cc 1.86 2020-07-20 23:39:58 -07:00
Chris Lu befb396892 mount: resolve home directory 
fix https://github.com/chrislusf/seaweedfs/issues/1391
 2020-07-16 22:38:55 -07:00
Chris Lu ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu 5a4f258a8c 1.85 2020-07-12 17:34:09 -07:00
Chris Lu 224103e13b aggregated logs has empty flushFn 2020-07-11 09:12:03 -07:00
Chris Lu b813fac4a3 1.84 2020-06-28 07:28:21 -07:00
Chris Lu f0f884edd5 1.83 2020-06-27 15:46:59 -07:00
Chris Lu bef356ce4c since we already know the chunk size, no need to iterate 2020-06-27 12:51:04 -07:00
Chris Lu a808b3b5df incase the memory data is too small 2020-06-27 11:59:15 -07:00
Chris Lu 3dbd51c3c2 a little bit more efficient 2020-06-26 10:02:37 -07:00
Chris Lu 212b6e7d42 error logging 2020-06-26 10:01:55 -07:00
Chris Lu 48b23f2fdd FUSE mount: prevent concurrent modification 2020-06-26 10:00:48 -07:00
Chris Lu 3cec4b3c49 FUSE mount: fix file id written twice? 
fix https://github.com/chrislusf/seaweedfs/issues/1373
 2020-06-25 20:11:49 -07:00
Chris Lu 5be12eea37 zstd fix 2020-06-24 11:39:12 -07:00
Chris Lu fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Chris Lu 3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu a735fd767c adjust uncompress data 2020-06-20 08:01:00 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu c48b407053 1.82 2020-06-19 13:47:27 -07:00
Chris Lu c0283eee1a more test cases 2020-06-19 09:45:42 -07:00
Chris Lu 55b6efb755 fix checking visited nodes 2020-06-19 09:45:42 -07:00
Chris Lu 546709c1de check has visited by the boundary 2020-06-18 22:28:17 -07:00
Chris Lu 0e3b970c0c added bounded tree to track exploration boundary 2020-06-18 21:09:58 -07:00
Chris Lu 95a9686eaf 1.81 2020-06-10 13:00:42 -07:00
Chris Lu bb3e86fbd5 auto create non-existing cache directory 
fix https://github.com/chrislusf/seaweedfs/issues/1350
 2020-06-08 08:49:57 -07:00
Chris Lu b8e346cc86 1.80 2020-06-05 15:29:25 -07:00
Chris Lu 45a9d852d3 go fmt 2020-06-05 15:27:10 -07:00
Chris Lu bc2ec6774d inject git version into build 2020-06-02 00:10:38 -07:00
bingoohuang 1f8782a1ed try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
bingoohuang 1a642b9876 add Volume Ids column only for max 100 volumes for convenience in the master ui. 2020-05-29 15:37:58 +08:00
bingoohuang accb4964b7 util.BytesToHumanReadable 2020-05-29 10:00:07 +08:00
Chris Lu ed9e4dc749 1.79 
fix compilation problem
 2020-05-19 08:21:51 -07:00
Chris Lu 85aeead3d2 1.78 2020-05-17 20:28:04 -07:00
Chris Lu 759cda0fe2 log 2020-05-16 18:47:41 -07:00
Chris Lu 4b7fa31468 ensure montonically increasing tsNs 2020-05-11 01:53:54 -07:00
Chris Lu 39e72fb23c go fmt 2020-05-10 03:50:30 -07:00
Chris Lu 6bf3eb69cb async chan write read, no write for closed chan 2020-05-10 03:48:35 -07:00
Chris Lu dfccc3c263 able to read chan and write chan 2020-05-08 02:47:22 -07:00
Chris Lu f9b6178b8f log messages 2020-04-30 03:05:34 -07:00
Chris Lu 4e16a90454 for debugging 2020-04-30 02:19:32 -07:00
Chris Lu 5a0986dca0 reset on getting the buffer 2020-04-29 02:41:01 -07:00
wuyuxiang 6850d28d6b refacotr: move signal handling and pprof to grace package 2020-04-28 14:10:23 +08:00
Chris Lu f4fcde05b6 1.77 2020-04-26 05:54:47 -07:00
Chris Lu 369aa8a10a 1.76 2020-04-22 23:31:59 -07:00
Chris Lu 72c5dba4d4 1.75 2020-04-21 22:03:07 -07:00
Chris Lu cb3985be70 go fmt 2020-04-20 17:48:06 -07:00
Chris Lu 5eb83dfbd3 add memory buffer size limit 2020-04-20 17:43:50 -07:00
Chris Lu 258fba8a0f continue for reading from sealed memory buffer 2020-04-20 17:28:18 -07:00
Chris Lu 4bf959edf0 message broker: read also from sealed memory buffer 2020-04-20 17:26:38 -07:00
Chris Lu bd43c62fbd lower the test criteria 2020-04-20 03:08:10 -07:00
Chris Lu c3f9fdf48f adjust test 2020-04-20 02:54:21 -07:00
Chris Lu 5e8bb86995 close the connection with better accouting 2020-04-20 02:53:50 -07:00
Chris Lu 2955b96ef1 refactor 2020-04-19 23:54:32 -07:00
Chris Lu ce3cb25cfb working for in memory single log buffer 2020-04-19 23:37:50 -07:00
Chris Lu e0f5996560 fix "call of Unmarshal passes non-pointer as second argument" 2020-04-17 10:04:41 -07:00
Chris Lu f5a748d33c refactoring 2020-04-16 02:55:09 -07:00
Chris Lu 23715af402
Merge pull request #1273 from wyxloading/config_race 
Fix data race about config
 2020-04-13 22:24:51 -07:00
Chris Lu 2b5c4fbbf3 tiered caching 
1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files
 2020-04-13 22:19:27 -07:00
Chris Lu f282ed444b refactoring 2020-04-13 21:58:10 -07:00
wuyuxiang b556d3d035 Fix data race about config 2020-04-14 10:22:40 +08:00
Chris Lu 5c9e91393f 1.74 2020-04-13 10:25:48 -07:00
Chris Lu 7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu 94e35cdb35 mount: fix 
fix bug found by git bisect, but I do not understand why it can cause error!
 2020-04-12 03:34:36 -07:00
Chris Lu 2a1f396df5 avoid duplicated setting chunks into cache 2020-04-12 01:06:50 -07:00
Chris Lu b9b7da905e handle nil chunk cache 2020-04-12 01:00:12 -07:00
Chris Lu df97da25f9 mount: add on disk caching 2020-04-11 21:12:41 -07:00
Chris Lu d7f3acb2c0 refactor 2020-04-11 12:45:24 -07:00
Chris Lu ed54682598 refactoring 2020-04-11 12:43:17 -07:00
Chris Lu e909b55633 refactoring 2020-04-11 12:39:46 -07:00
Chris Lu 43c62a8a95 1.73 2020-04-09 21:44:30 -07:00
Chris Lu 59f40e2027 volume: best effort to detect ip address 
fix https://github.com/chrislusf/seaweedfs/issues/1264
 2020-04-09 00:26:24 -07:00
Chris Lu f6a7e79dc3 weed shell: simplify CLI option for filer 2020-04-08 23:57:15 -07:00
Chris Lu 8764bdb9df 1.72 2020-04-08 22:35:14 -07:00
Chris Lu 00b993a234 add util for md5 2020-04-08 22:30:38 -07:00
Chris Lu b51d8f03bb 1.71 2020-04-05 22:04:16 -07:00
Chris Lu 91da7057b1 refactoring 2020-04-05 13:11:43 -07:00
Chris Lu 7bc3c93512 add util.PathJoin 2020-04-05 12:40:46 -07:00
Chris Lu 50a5018b7f writing meta logs is working 2020-03-30 01:19:33 -07:00
Chris Lu be6c2f8511 1.70 2020-03-29 13:40:55 -07:00
Chris Lu 057722bbf4 return part of the chunk if chunkview is not the full chunk 2020-03-29 00:54:39 -07:00
Chris Lu 005b4ab3fe 1.69 2020-03-27 04:51:31 -07:00
Chris Lu 5463560912 small test 2020-03-26 23:50:07 -07:00
Chris Lu 0a05052b5c 1.68 2020-03-26 22:22:36 -07:00
Chris Lu dd5b582d05 go fmt 2020-03-26 00:09:01 -07:00
Chris Lu 2e4fadd10a fix fscache move 2020-03-25 22:19:19 -07:00
Chris Lu c0f0fdb3ba refactoring 2020-03-23 00:01:34 -07:00
Chris Lu d6412f27a0 logging 2020-03-22 18:32:56 -07:00
Chris Lu 3b3e063f25 1.67 2020-03-20 23:42:49 -07:00
Chris Lu d3e4a31058 filer: fix where deletion can miss under high concurrency 2020-03-20 23:39:32 -07:00
Chris Lu 4bc19f1e52 master: fix connections count reporting 2020-03-20 22:41:37 -07:00
Chris Lu e1b5ac5c04 1.66 2020-03-20 20:39:48 -07:00
Chris Lu 165b0d22a4 1.65 2020-03-20 00:05:42 -07:00
Chris Lu db9854fa0a 1.64 2020-03-17 10:58:35 -07:00
Chris Lu cc52e8ca87 1.63 2020-03-14 14:25:07 -07:00
Chris Lu 91e0a987e4 1.62 2020-03-14 01:03:59 -07:00
Chris Lu e2e691d9c2 clean up, add test 2020-03-13 23:53:15 -07:00
Chris Lu 60f5f05c78 1.61 2020-03-10 13:52:49 -07:00
Chris Lu e6de42f888 1.60 2020-03-09 22:32:40 -07:00
Chris Lu c32f95c380 1.59 2020-03-08 22:02:34 -07:00
Chris Lu 2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu ea1169dc80 filer cipher: single chunk http POST and PUT and read 2020-03-07 06:08:08 -08:00
Chris Lu 13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu f90c43635d refactoring 2020-03-04 00:39:47 -08:00
Chris Lu 4532640ffd 1.58 2020-03-01 17:59:12 -08:00
Chris Lu ed0acd1722 go fmt 2020-02-26 16:52:57 -08:00
Chris Lu 37a3628b2e grpc add PermitWithoutStream to client and server options 2020-02-26 16:51:38 -08:00
Chris Lu 892e726eb9 avoid reusing context object 
fix https://github.com/chrislusf/seaweedfs/issues/1182
 2020-02-25 21:50:12 -08:00
Chris Lu 6ea394b802 1.57 2020-02-22 21:34:49 -08:00
Chris Lu 621cdbdf58 filer: avoid possible timeouts for updates and deletions 2020-02-20 15:44:17 -08:00
Chris Lu 443a68484a 1.56 2020-02-17 12:54:45 -08:00
Chris Lu 9232d3ac68 ignore error when counting closed connections 2020-02-14 10:28:02 -08:00
Chris Lu 96c1ae8471 refactoring the close http response 2020-02-14 09:46:36 -08:00
Chris Lu cf5064d702 properly close http response 2020-02-14 09:09:15 -08:00
Chris Lu 62c34454d8 1.55 2020-02-11 23:00:31 -08:00
Chris Lu 33b3bd467c Revert "HEAD operation changes to fasthttp" 
This reverts commit 58f126fd27.
 2020-02-10 13:43:53 -08:00
Chris Lu 58f126fd27 HEAD operation changes to fasthttp 2020-02-10 09:13:29 -08:00
Chris Lu 29945fad51 1.54 2020-02-09 18:07:17 -08:00
Chris Lu beb0a1599e 1.53 2020-01-31 00:32:09 -08:00
Chris Lu a80ecbfe84 s3: add s3 copy 
fix https://github.com/chrislusf/seaweedfs/issues/1190
 2020-01-31 00:11:12 -08:00
Chris Lu 3b043ead49
Merge pull request #1189 from eryx67/compression-wav 
Support wav files compression
 2020-01-29 22:37:35 -08:00
Chris Lu abf90ad7b7 remove testing for viper 2020-01-29 21:38:53 -08:00
Chris Lu 20b1fb9ab7 fix test 2020-01-29 21:24:06 -08:00
eryx67 2a87087590 support wav files compression 2020-01-30 10:11:47 +05:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu 27b94cb65b fix wrong url 
fix https://github.com/chrislusf/seaweedfs/issues/1187
 2020-01-28 00:49:47 -08:00
Chris Lu 234f69452b 1.52 2020-01-26 21:04:40 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu 107e8a56ea retry context canceled request 2020-01-24 01:40:51 -08:00
Chris Lu 6a5c037099 fix http range requests 2020-01-22 22:59:12 -08:00
Chris Lu bb1be61602 1.51 2020-01-20 20:30:03 -08:00
Chris Lu 39edcfde4c filer: set inode 
avoid "getcwd: No such file or directory"
 2020-01-19 12:06:19 -08:00
Chris Lu 30c7148020 1.50 2020-01-08 22:08:35 -08:00
Chris Lu 89e16bd2e8 skip error when draining reader 
fix https://github.com/chrislusf/seaweedfs/issues/1179
 2020-01-08 18:07:07 -08:00
Chris Lu 66ab09aa4a 1.49 2020-01-08 09:51:35 -08:00
Chris Lu aa1807e082 1.48 2020-01-01 09:27:02 -08:00
Chris Lu c06f7eb48a load volume info from .vif file, use superblock as a backup 2019-12-28 12:28:58 -08:00
Chris Lu 1fd8926ac7 ignore draining error 2019-12-25 21:39:33 -08:00
divinerapier 4cbb6fa199
feat: drains http body if buffer is too small 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-12-26 11:28:43 +08:00
divinerapier 84640d07b7
fix: handle errors for ReadUrl 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-12-26 10:07:07 +08:00
Chris Lu 573b74f24f add SetDefault function for Configuration 2019-12-20 20:56:14 -08:00
Chris Lu 54369eccea 1.47 2019-12-17 15:11:01 -08:00
Chris Lu 6808bfec49 1.47 2019-12-17 15:08:45 -08:00
Chris Lu 392678f8f3 upload skipping mimetype if not needed 2019-12-15 20:57:08 -08:00
Chris Lu a631b2a9e9 1.46 2019-12-14 11:28:48 -08:00
Chris Lu 10bd3c6b4b refactoring 2019-12-08 19:44:16 -08:00
Chris Lu f81d43442b filer: speed up filer.meta.save by parallelizing 2019-12-06 22:20:59 -08:00
Chris Lu 0da7b894cc pass backend config from master to volume servers 2019-11-29 01:05:09 -08:00
Chris Lu a9dbd52e06
Merge pull request #1145 from stlpmo-jn/volume_graceful_stop 
let volume server graceful stop
 2019-11-29 00:11:57 -08:00
Chris Lu 249b9d8d5d 1.45 2019-11-22 14:51:16 -08:00
stlpmo 753138a3f7 test passed 2019-11-17 11:40:36 +08:00
Chris Lu de0c983808 refactoring scaffolding for master.toml 2019-11-14 22:20:38 -08:00
Lei Liu 46755ea1e1 fix master maintenance logic 
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
 2019-11-12 14:46:10 +08:00
stlpmo d07701fa75 delete unused function 2019-11-11 18:08:48 +08:00
stlpmo 62d393d6c9 ut pass 2019-11-11 10:52:21 +08:00
stlpmo 802a0eb3fe move from cmd-line to scaffold 2019-11-11 09:15:17 +08:00
Chris Lu 50e885da45 1.44 2019-10-11 23:56:58 -07:00
Chris Lu 20d90dea5a filer: avoid hard-coded upload timeout 2019-09-05 01:26:59 -07:00
Chris Lu 89a0cb2e11 1.43 2019-09-03 01:31:58 -07:00
Chris Lu 35d471a818
Revert "use std encoding lib" 2019-08-23 07:59:09 -07:00
divinerapier e798e4e9c5
use std encoding lib 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-08-19 18:07:54 +08:00
Chris Lu 3f851feb59 1.42 2019-07-21 21:52:48 -07:00
Chris Lu cf2804eebd 1.41 2019-07-07 20:40:20 -07:00
Chris Lu 23d36dfbaf 1.40 
enable azure replication
 2019-07-02 21:27:05 -07:00
Chris Lu 36022f9416 1.39 2019-06-30 00:50:17 -07:00
Chris Lu fdff927b37 1.38 2019-06-24 00:36:46 -07:00
Chris Lu 95ad56d99d 1.37 2019-06-21 20:56:34 -07:00
Chris Lu fe4eb6bffa 1.36 2019-06-21 13:11:40 -07:00
Chris Lu be3f1f84a2 1.35 2019-06-10 21:34:06 -07:00
Chris Lu ede876cfdb periodic scripts exeuction from leader master 2019-06-05 01:30:24 -07:00
Chris Lu 7e80b2b882 fix multiple bugs 2019-06-03 02:26:31 -07:00