gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4193 commits 28 branches 278 tags 135 MiB
Commit graph

756 commits

Author SHA1 Message Date
Chris Lu c647deace1 file size support set file length 
use Attr.FileSize and TotalChunkSize to determine file size
 2020-08-15 09:32:47 -07:00
Chris Lu 3b1a95ac26 filer refactoring: same auto chunking logic for POST and PUT, no size limit 2020-08-08 12:02:06 -07:00
Chris Lu 67348e7b15 less noisy heartbeat logs 2020-08-08 10:53:35 -07:00
Chris Lu bee0d7e5eb lower log priority for noisy heartbeat 2020-08-08 10:52:13 -07:00
Chris Lu ae00cce4bd support POST and PUT auto chunking 2020-08-08 10:45:37 -07:00
Chris Lu bd8bfdae07 refactoring 2020-08-08 10:18:43 -07:00
Chris Lu 20e2ac1add filer: store md5 metadata for files uploaded by filer 
fix https://github.com/chrislusf/seaweedfs/issues/1412
 2020-08-06 10:04:17 -07:00
Chris Lu 93ea0801ea volume: the variable for the master node may be stale? 
related to https://github.com/chrislusf/seaweedfs/issues/1414
 2020-08-06 09:48:54 -07:00
Chris Lu 4ecfa9879d volume: report Content-MD5 in response header 2020-08-06 05:22:53 -07:00
Устюжанин Антон Александрович 2d3b355fb6 test ListDirectoryPrefixedEntries 2020-08-05 23:56:06 +05:00
Устюжанин Антон Александрович 33a9e5e2d1 test ListDirectoryPrefixedEntries 2020-08-05 22:19:16 +05:00
Chris Lu 49199c69d4 volume: avoid special logic for .gz files 2020-08-01 11:19:57 -07:00
Chris Lu 3faa0aa448 rename 2020-07-25 20:09:19 -07:00
Chris Lu 023a1efdf2 set filename in Content-Disposition header 2020-07-25 20:06:40 -07:00
James Hartig 229f11c660 Added VolumeNeedleStatus volume server grpc method 
This is needed for the diffing tool to get the cookie for a needle
 2020-07-22 15:02:21 -04:00
Chris Lu 97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
李远军 4162766c46 Fix the bug of "http://xx/xx,xxx.PNG?width=100" 
If you want to do a resize a UPPER ext pic file, there is a bug caused by commit 308688c8d0 (also see https://github.com/chrislusf/seaweedfs/blob/master/weed/images/resizing.go#L43)
 2020-07-17 19:57:35 +08:00
Chris Lu ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu 4f6096c7f0 add reading from persisted logs for local filer store 2020-07-13 22:55:28 -07:00
Chris Lu 1dd2c76532 refactoring 2020-07-12 17:31:24 -07:00
Chris Lu e7c04af1d0 skip http.StatusPartialContent in case of error and superfluous response.WriteHeader 2020-07-10 08:56:23 -07:00
李远军 6608cb5f43 Revert "remove fixJpgOrientation" 
This reverts commit de5ca9b2
 2020-07-10 10:08:36 +08:00
Chris Lu a2eb680f34 filer: default filer store directory 
1. set default filer store directory
2. set peers, avoiding empty string counted as 1.
 2020-07-07 23:06:48 -07:00
Chris Lu 3c269da37f fix compilation 2020-07-06 09:46:24 -07:00
Chris Lu d198e8c6d6 filer: support cross filer meta data sync if sharing the same store 2020-07-06 09:24:54 -07:00
Chris Lu 0b8cc9b626 aggregate multiple filer metadata chagne events 2020-07-05 23:05:02 -07:00
Chris Lu 49929e0869 rename 2020-07-05 15:52:36 -07:00
Chris Lu 70d8a3a1d3 add SubscribeLocalMetadata without checking persisted meta logs 2020-07-05 15:50:07 -07:00
Chris Lu 55e40b08fc refactoring 2020-07-05 15:43:06 -07:00
Chris Lu 881e0fde2e rename 2020-07-05 08:55:09 -07:00
Chris Lu 4d1484628a refactoring 2020-07-03 16:41:30 -07:00
Chris Lu 31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu df75b5c98d passing ctx for metadata logging 2020-06-28 14:34:51 -07:00
Chris Lu a4dc25cd22 volume: generate ec shard first, before generating the ec index file. 
possible fix https://github.com/chrislusf/seaweedfs/issues/1364

the ec index file is used to determine whether the ec files are ready or not
 2020-06-25 11:06:58 -07:00
Chris Lu 3b638d3994 add more ec encoding logging 2020-06-25 09:43:38 -07:00
Chris Lu fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Chris Lu a4e3cffe0a S3: DeleteObject API also support deleting a folder recursively 2020-06-22 10:01:00 -07:00
Chris Lu de5ca9b258 remove fixJpgOrientation 2020-06-20 12:50:40 -07:00
Chris Lu 3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu ca3516ac6d adjust protoc 2020-06-20 08:00:25 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu 16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu 044841c885 master: always clear previous master meta data directory 2020-06-19 20:42:16 -07:00
Chris Lu ae1994cbc1 erasure coding: fix cases where there are no .ecj files 2020-06-18 09:52:35 -07:00
Chris Lu c66f357715 report error correctly 2020-06-10 10:59:25 -07:00
Chris Lu c38066be89 also add it for webdav 2020-06-09 18:07:03 -07:00
Evgenii Kozlov 2410ffbd79 Name fix 2020-06-05 19:00:38 +03:00
Evgenii Kozlov 0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
Chris Lu 393ab07c7c disable pulse seconds 
there are a few other hard coded timing settings, which will cause disconnected volume servers.
 2020-06-04 10:52:01 -07:00
Chris Lu bc2ec6774d inject git version into build 2020-06-02 00:10:38 -07:00
bingoohuang 1a642b9876 add Volume Ids column only for max 100 volumes for convenience in the master ui. 2020-05-29 15:37:58 +08:00
bingoohuang accb4964b7 util.BytesToHumanReadable 2020-05-29 10:00:07 +08:00
bingoohuang 21d0a013d5 rename 'needle' which collides with imported package name 2020-05-28 17:00:00 +08:00
bingoohuang 41cab5d69f log JSON response if httpStatus >= 400 2020-05-28 08:56:55 +08:00
Chris Lu d4235afe4d support multiple locks 2020-05-25 18:39:53 -07:00
bingoohuang ecdeef8c66 simplify func(w http.ResponseWriter, r *http.Request) to http.HandlerFunc 2020-05-25 21:00:12 +08:00
Chris Lu 91b91d6cb7 add error to avoid copying not found volume 
fix https://github.com/chrislusf/seaweedfs/issues/1317
 2020-05-17 20:20:12 -07:00
Chris Lu 3fbf635308 Merge branch 'master' into msg_channel 2020-05-13 03:54:15 -07:00
Chris Lu 3a5e4769cf prevent empty locations returned 
fix https://github.com/chrislusf/seaweedfs/issues/1313
 2020-05-13 03:46:38 -07:00
Chris Lu dfccc3c263 able to read chan and write chan 2020-05-08 02:47:22 -07:00
Chris Lu 1e3e4b3072 add broker connects to filer 2020-05-05 02:05:28 -07:00
Chris Lu 871efa4fc1 refactoring 
some previous chunk etag was using md5, which should be wrong.
 2020-04-30 17:20:47 -07:00
Chris Lu 9e72e9e4b8 able to subscribe any topic from any point of time 2020-04-29 17:40:08 -07:00
Chris Lu ed3cf811f5 refactoring 2020-04-29 13:26:02 -07:00
Chris Lu 8d42a1263b meta subscription: update the last read time 2020-04-28 17:30:04 -07:00
Chris Lu 31583b5d6c master and volume server: avoid race condition 
The volume server may disconnect and reconnect to the same master.

The master's unregistration may happen after the reconnection. Thus the volume server will disappear.
 2020-04-28 17:29:10 -07:00
Chris Lu 98edec7c3e remove printouts 2020-04-28 16:45:48 -07:00
Chris Lu fb81f12686 filer: simplify image resize 2020-04-28 00:05:57 -07:00
Chris Lu 5c57297bd1 metadata log: read from any timestamp 2020-04-28 00:05:57 -07:00
wuyuxiang 6850d28d6b refacotr: move signal handling and pprof to grace package 2020-04-28 14:10:23 +08:00
Chris Lu b52b8ec685 Hadoop: fix entry not found for HCFS 
also fix cipher related changes.
 2020-04-26 05:21:54 -07:00
Chris Lu 662b5d0cf7 support semicolon seperated command lines 2020-04-23 14:01:46 -07:00
Chris Lu 73564e6a01 master: add cluster wide lock/unlock operation in weed shell 
fix https://github.com/chrislusf/seaweedfs/issues/1286
 2020-04-23 13:37:31 -07:00
Chris Lu bdc337a719 make lock/unlock optional 2020-04-23 03:50:05 -07:00
Chris Lu 842e3301ee make second lock instant 2020-04-23 03:32:18 -07:00
Chris Lu 30ee4f3291 add exclusive lock library on shell 2020-04-23 02:31:04 -07:00
Chris Lu 77873b832b add master side code for cluster wise exclusive lock 2020-04-23 01:55:44 -07:00
Chris Lu 69eda558ef reduce the connections 
fix https://github.com/chrislusf/seaweedfs/issues/1277

this avoids sending error to wdclients, and clients retry the connections
 2020-04-22 20:43:34 -07:00
Chris Lu e24b25de78 async meta caching: can stream updates now 2020-04-21 21:16:13 -07:00
Chris Lu 27128c7875 adjust logging 2020-04-21 16:13:13 -07:00
Chris Lu cb3985be70 go fmt 2020-04-20 17:48:06 -07:00
Chris Lu ebfab42a50 refactoring 2020-04-20 00:08:47 -07:00
Chris Lu 2955b96ef1 refactor 2020-04-19 23:54:32 -07:00
Chris Lu f373232227
Merge pull request #1280 from bukton123/filer_mongodb 
Add new filer support mongodb
 2020-04-19 20:35:19 -07:00
Chris Lu 076c8bd3bc filer master start up with default ip address instead of just localhost 2020-04-18 15:17:27 -07:00
bukton 290c6b7f01 Merge remote-tracking branch 'origin/master' into filer_mongodb 
# Conflicts:
#	go.mod
#	go.sum
#	weed/server/filer_server.go
 2020-04-19 00:21:45 +07:00
bukton cd9cccec55 add import mongodb to filer server 2020-04-18 22:30:13 +07:00
Chris Lu 826f96b6f3 fix unreachable code 2020-04-17 10:04:28 -07:00
Chris Lu bda82f61bc filer: able to append to a file 2020-04-17 02:28:09 -07:00
Chris Lu f5a748d33c refactoring 2020-04-16 02:55:09 -07:00
Chris Lu ce4b369be2 scaffolding messaging 2020-04-16 02:21:23 -07:00
Reed aada5b9aa7 making volume stats size info readable. 2020-04-16 10:08:08 +08:00
Chris Lu 2b5c4fbbf3 tiered caching 
1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files
 2020-04-13 22:19:27 -07:00
Chris Lu 7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu 8e23dc078b refactoring 2020-04-12 20:48:21 -07:00
Chris Lu 5361f999ed shell: only one shell is allowed to connect to the cluster 
fix https://github.com/chrislusf/seaweedfs/issues/1266
 2020-04-12 17:48:39 -07:00
Chris Lu d30483d642 re-enable system logs 2020-04-12 14:03:07 -07:00
Chris Lu 6f948e4887 remove configurable topics folder location 2020-04-12 13:07:59 -07:00
Chris Lu b5a713be68 filer store: add redis2 redis_cluster2 as default redis store 2020-04-12 02:50:41 -07:00
Chris Lu 1c65656fb4 s3: add option to fsync buckets 2020-04-11 23:37:10 -07:00