gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
3782 commits 28 branches 278 tags 135 MiB
Commit graph

2251 commits

Author SHA1 Message Date
Chris Lu a735fd767c adjust uncompress data 2020-06-20 08:01:00 -07:00
Chris Lu ca3516ac6d adjust protoc 2020-06-20 08:00:25 -07:00
Chris Lu e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu 16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu 044841c885 master: always clear previous master meta data directory 2020-06-19 20:42:16 -07:00
Chris Lu 2eeb7551f9 less noisy disk size checking 2020-06-19 15:06:33 -07:00
Chris Lu c48b407053 1.82 2020-06-19 13:47:27 -07:00
Chris Lu b78a2dc59b adjust help message 2020-06-19 12:55:09 -07:00
Chris Lu f7a45d448f FUSE mount: lazy loading meta cache 2020-06-19 09:45:42 -07:00
Chris Lu c0283eee1a more test cases 2020-06-19 09:45:42 -07:00
Chris Lu 55b6efb755 fix checking visited nodes 2020-06-19 09:45:42 -07:00
Chris Lu 0e7c1a300b
Merge pull request #1363 from DXist/fix_utc 
Return last modified in UTC
 2020-06-19 08:49:00 -07:00
Rinat Shigapov 320d70d305 return last modified in UTC 2020-06-19 10:26:40 +03:00
Chris Lu 546709c1de check has visited by the boundary 2020-06-18 22:28:17 -07:00
Chris Lu 0e3b970c0c added bounded tree to track exploration boundary 2020-06-18 21:09:58 -07:00
Chris Lu ae1994cbc1 erasure coding: fix cases where there are no .ecj files 2020-06-18 09:52:35 -07:00
Chris Lu b74eced7a3 mount: a fix to prevent possible repeated calls 
related to https://github.com/bazil/fuse/issues/130
 2020-06-17 13:55:16 -07:00
Chris Lu 7ccc31853c hard limit volume file size 2020-06-16 00:27:48 -07:00
Chris Lu 48880edc2c volume: fix typo in minFreeSpacePercent option 2020-06-14 20:37:05 -07:00
Chris Lu a0464fc267 fix help 
fix https://github.com/chrislusf/seaweedfs/issues/1359
 2020-06-12 22:06:21 -07:00
Chris Lu 0d1d83a5a5 mount: add option to auto create dir to mount 2020-06-12 11:46:58 -07:00
Chris Lu 00f96d86a1 fix typo 2020-06-11 10:53:25 -07:00
Chris Lu 63a1db64c5
Merge pull request #1358 from DXist/s3fixes 
Return NotSuchKey error code for bucket S3 DeleteObject method
 2020-06-11 10:37:19 -07:00
Rinat Shigapov 283b749ff1 keep 204 response 2020-06-11 19:00:34 +03:00
Rinat Shigapov fafc41a27f return xml encoded NotFound status code for s3 delete 2020-06-11 17:53:15 +03:00
Rinat Shigapov 2d2c5dfa39 proxy status code of filer delete response 2020-06-11 15:00:47 +03:00
Chris Lu 628b27ef3b purge old cache implementation 2020-06-11 01:50:00 -07:00
Chris Lu b9365de47b fix volume server pprof 2020-06-10 13:10:10 -07:00
Chris Lu 718052656a
Merge pull request #1355 from Kimbsen/add_http_pprof 
Add --pprof to volume servers to enable http pprof handlers
 2020-06-10 13:05:04 -07:00
Chris Lu 95a9686eaf 1.81 2020-06-10 13:00:42 -07:00
Chris Lu c66f357715 report error correctly 2020-06-10 10:59:25 -07:00
Kimbsen 03e8f976d9 added --pprof to volume servers 2020-06-10 11:52:24 +02:00
Chris Lu c38066be89 also add it for webdav 2020-06-09 18:07:03 -07:00
Chris Lu 4598473251 fix possible EOF problem 2020-06-09 18:04:40 -07:00
Chris Lu bb3e86fbd5 auto create non-existing cache directory 
fix https://github.com/chrislusf/seaweedfs/issues/1350
 2020-06-08 08:49:57 -07:00
Chris Lu bdbe5ddbe9 volume server: add a fix for dead lock 
https://github.com/chrislusf/seaweedfs/issues/1353
 2020-06-08 08:12:59 -07:00
Chris Lu b8e346cc86 1.80 2020-06-05 15:29:25 -07:00
Chris Lu 45a9d852d3 go fmt 2020-06-05 15:27:10 -07:00
Chris Lu 03c2f03509
Merge pull request #1351 from ekozlov-search/master 
Set volumes ReadOnly if low free disk space
 2020-06-05 09:56:46 -07:00
Chris Lu ebd1759741 fix ReadAt 
fix https://github.com/chrislusf/seaweedfs/issues/1344
 2020-06-05 09:54:14 -07:00
Evgenii Kozlov 2410ffbd79 Name fix 2020-06-05 19:00:38 +03:00
Evgenii Kozlov 0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
Chris Lu 393ab07c7c disable pulse seconds 
there are a few other hard coded timing settings, which will cause disconnected volume servers.
 2020-06-04 10:52:01 -07:00
Evgenii Kozlov 48f9ff52cf Discs statistics on Windows platform. 2020-06-02 11:52:16 +03:00
Chris Lu bc2ec6774d inject git version into build 2020-06-02 00:10:38 -07:00
bingoohuang 1f8782a1ed try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
bingoohuang 1a642b9876 add Volume Ids column only for max 100 volumes for convenience in the master ui. 2020-05-29 15:37:58 +08:00
bingoohuang accb4964b7 util.BytesToHumanReadable 2020-05-29 10:00:07 +08:00
bingoohuang 21d0a013d5 rename 'needle' which collides with imported package name 2020-05-28 17:00:00 +08:00
bingoohuang 41cab5d69f log JSON response if httpStatus >= 400 2020-05-28 08:56:55 +08:00
Chris Lu ef2b3a0801 leveldb recover error handling 2020-05-26 00:03:44 -07:00
Chris Lu 5d3ec22975 refactoring 2020-05-26 00:03:31 -07:00
Chris Lu d4235afe4d support multiple locks 2020-05-25 18:39:53 -07:00
bingoohuang ecdeef8c66 simplify func(w http.ResponseWriter, r *http.Request) to http.HandlerFunc 2020-05-25 21:00:12 +08:00
Chris Lu e5a0787653 filer: leveldb store add auto recovery, report correct error 
fix https://github.com/chrislusf/seaweedfs/issues/1326
 2020-05-22 10:54:42 -07:00
Chris Lu ed9e4dc749 1.79 
fix compilation problem
 2020-05-19 08:21:51 -07:00
Chris Lu 7ff2b0121d specify constant data type to uint64 instead of default int 
fix https://github.com/chrislusf/seaweedfs/issues/1324
 2020-05-19 07:55:23 -07:00
Chris Lu 11b33c0760 fix compilation problem 2020-05-18 10:53:31 -07:00
Chris Lu ac48c899e8 add printouts for synchronizing metadata which can be slow for large folders 2020-05-18 10:31:12 -07:00
Chris Lu 0c856e8387 avoid possible wrong arguments 
fix https://github.com/chrislusf/seaweedfs/issues/1320
 2020-05-18 10:07:12 -07:00
Chris Lu 2bfd810912 combine msgBroker into weed server command 2020-05-17 22:44:20 -07:00
Chris Lu 85aeead3d2 1.78 2020-05-17 20:28:04 -07:00
Chris Lu 8005786d9a logs 2020-05-17 20:27:10 -07:00
Chris Lu 75179544c3 logging 2020-05-17 20:20:21 -07:00
Chris Lu 91b91d6cb7 add error to avoid copying not found volume 
fix https://github.com/chrislusf/seaweedfs/issues/1317
 2020-05-17 20:20:12 -07:00
Chris Lu d3925d086c add delete channel 2020-05-17 17:42:42 -07:00
Chris Lu e0e31e67a8
Merge pull request #1318 from chrislusf/msg_channel 
Add messaging, add channel
 2020-05-17 17:39:16 -07:00
Chris Lu 081ee6fe34 reduc logs 2020-05-17 17:38:31 -07:00
Chris Lu bff09fb74f fix looping 2020-05-17 17:38:21 -07:00
Chris Lu f5684839a2 add DeleteTopic 2020-05-17 17:34:10 -07:00
Chris Lu 3a57aef7a9 sync subscribe() 2020-05-17 17:33:53 -07:00
Chris Lu 95ca9dd8a2 subscribe support cancel 2020-05-17 11:10:45 -07:00
Chris Lu f11233cd49 simplify disconnected stream detection 2020-05-17 08:57:47 -07:00
Chris Lu 1ea9bc66d9 avoid leaking grpc.NewClientStream 2020-05-17 08:56:33 -07:00
Chris Lu 85b53ac510 detect disconnected subscribers 2020-05-16 18:53:54 -07:00
Chris Lu 759cda0fe2 log 2020-05-16 18:47:41 -07:00
Chris Lu b0de01ff3b able to detect disconnected subscribers 2020-05-16 08:57:29 -07:00
Chris Lu e02a8c67da revert to one subscriber one thread 2020-05-15 21:38:42 -07:00
Chris Lu b4e02ec525 fix prometheus problem 
metrics.go:123] could not push metrics to prometheus push gateway 192.168.xx:9091: unexpected status code 200 while pushing to http://192.168.xx:9091/metrics/job/filer/instance/m111p112%3A8888:
 2020-05-14 21:08:34 -07:00
Chris Lu 3fbf635308 Merge branch 'master' into msg_channel 2020-05-13 03:54:15 -07:00
Chris Lu 3a5e4769cf prevent empty locations returned 
fix https://github.com/chrislusf/seaweedfs/issues/1313
 2020-05-13 03:46:38 -07:00
Chris Lu ca4017dd87 rename 2020-05-12 22:55:55 -07:00
Chris Lu 25257acd51 rename 2020-05-12 21:26:49 -07:00
Chris Lu a7959c1c48 multiple subscriber with same subscriberId shares the topic manager 
rename topicControl to topicCursor
 2020-05-12 21:26:02 -07:00
Chris Lu 2f243f5b0b refactor 2020-05-12 08:48:00 -07:00
Chris Lu 4b7fa31468 ensure montonically increasing tsNs 2020-05-11 01:53:54 -07:00
Chris Lu d5a8297a1c able to connect publisher 2020-05-10 23:45:48 -07:00
Chris Lu d58371c5cd Merge branch 'master' into msg_channel 2020-05-10 23:44:59 -07:00
Chris Lu 015dd3a147 batch file id deletion 2020-05-10 23:25:39 -07:00
Chris Lu 39e72fb23c go fmt 2020-05-10 03:50:30 -07:00
Chris Lu 6bf3eb69cb async chan write read, no write for closed chan 2020-05-10 03:48:35 -07:00
Chris Lu 78afb8bf46 Merge branch 'master' into msg_channel 2020-05-09 00:43:58 -07:00
Chris Lu d693e77418 add pub sub md5 2020-05-09 00:43:53 -07:00
Chris Lu 07d7abe428 add deleteTopic, refactoring 2020-05-09 00:31:34 -07:00
Chris Lu dfccc3c263 able to read chan and write chan 2020-05-08 02:47:22 -07:00
zhangsong b161baa3f0 fix volume vacuum test unit 2020-05-07 19:58:06 +08:00
Chris Lu a8bc8eb351 Merge branch 'master' into msg_channel 2020-05-07 03:18:24 -07:00
Chris Lu 73087f59b1 filer store: mongodb deleting children 
fix https://github.com/chrislusf/seaweedfs/issues/1303
 2020-05-07 03:11:26 -07:00
Chris Lu be1d9f324f Merge branch 'master' into msg_channel 2020-05-07 01:43:34 -07:00
Chris Lu 0952eb2854
Merge pull request #1302 from song-zhang/master 
write request keep compatible with before
 2020-05-06 15:37:34 -07:00
Chris Lu 5568395edd Revert "Revert "Merge pull request #1299 from song-zhang/master"" 
This reverts commit afb6a1dbb4.
 2020-05-06 15:37:17 -07:00
zhangsong 3703c7e176 write request keep compatible with before 2020-05-06 21:35:32 +08:00
Chris Lu 1e3e4b3072 add broker connects to filer 2020-05-05 02:05:28 -07:00
Chris Lu afb6a1dbb4 Revert "Merge pull request #1299 from song-zhang/master" 
This reverts commit 9016fa19ba, reversing
changes made to 47234760f4.
 2020-05-04 20:34:26 -07:00
Chris Lu 9016fa19ba
Merge pull request #1299 from song-zhang/master 
use async write to persistent file to disk - part1
 2020-05-04 15:51:03 -07:00
zhangsong f9e8702bb4 use async write to persistent file to disk - part1 2020-05-04 17:39:44 +08:00
Chris Lu 47234760f4 log adjust format 2020-05-02 01:33:49 -07:00
Chris Lu 97a504b0bb fix possible directory deletion error 2020-05-02 01:11:53 -07:00
Chris Lu 78422c2f69
Merge pull request #1298 from song-zhang/master 
add fsync option for benchmark
 2020-04-30 20:16:42 -07:00
Chris Lu 871efa4fc1 refactoring 
some previous chunk etag was using md5, which should be wrong.
 2020-04-30 17:20:47 -07:00
zhangsong d819865fe7 add fsync option for benchmark 2020-04-30 23:31:08 +08:00
Chris Lu ea809a45ab add cipher key 2020-04-30 03:05:43 -07:00
Chris Lu f9b6178b8f log messages 2020-04-30 03:05:34 -07:00
Chris Lu 8c73410a51 subscribe from a timestamp 2020-04-30 02:19:51 -07:00
Chris Lu 4e16a90454 for debugging 2020-04-30 02:19:32 -07:00
Chris Lu a25a8d8822 s3: print out time in UTC format 
fix https://github.com/chrislusf/seaweedfs/issues/1297
 2020-04-29 21:35:24 -07:00
Chris Lu e93588ec78 FUSE mount: atomic local cache updates 2020-04-29 18:20:54 -07:00
Chris Lu 9e72e9e4b8 able to subscribe any topic from any point of time 2020-04-29 17:40:08 -07:00
Chris Lu ed3cf811f5 refactoring 2020-04-29 13:26:02 -07:00
Chris Lu 7c10602b49 read in case cross chunks 2020-04-29 02:42:58 -07:00
Chris Lu f9da859720 better error message 2020-04-29 02:42:06 -07:00
Chris Lu d57559d3f7 chunk rememober encryption or not 2020-04-29 02:41:47 -07:00
Chris Lu 5a0986dca0 reset on getting the buffer 2020-04-29 02:41:01 -07:00
Chris Lu 8d42a1263b meta subscription: update the last read time 2020-04-28 17:30:04 -07:00
Chris Lu 31583b5d6c master and volume server: avoid race condition 
The volume server may disconnect and reconnect to the same master.

The master's unregistration may happen after the reconnection. Thus the volume server will disappear.
 2020-04-28 17:29:10 -07:00
Chris Lu 98edec7c3e remove printouts 2020-04-28 16:45:48 -07:00
Chris Lu 8ed490164e refactoring 2020-04-28 02:05:44 -07:00
Chris Lu 66effaed9e fix compilation 2020-04-28 02:05:32 -07:00
Chris Lu eab4c9219c randomize benchmark content 
fix https://github.com/chrislusf/seaweedfs/issues/1294
 2020-04-28 01:50:56 -07:00
Chris Lu fb81f12686 filer: simplify image resize 2020-04-28 00:05:57 -07:00
Chris Lu 5c57297bd1 metadata log: read from any timestamp 2020-04-28 00:05:57 -07:00
wuyuxiang 6850d28d6b refacotr: move signal handling and pprof to grace package 2020-04-28 14:10:23 +08:00
Chris Lu 2a7957b4ca FUSE: subcribe meta events based on timestamp 2020-04-27 16:52:20 -07:00
Chris Lu 6190fd665d printout error 2020-04-27 12:41:31 -07:00
Chris Lu 725e47dcc3 filer: pass along user name and group name for HDFS 2020-04-27 05:48:48 -07:00
Chris Lu 508ec01ab0 tweaking 2020-04-26 16:11:06 -07:00
Chris Lu f4fcde05b6 1.77 2020-04-26 05:54:47 -07:00
Chris Lu b52b8ec685 Hadoop: fix entry not found for HCFS 
also fix cipher related changes.
 2020-04-26 05:21:54 -07:00
Chris Lu 0c2248f83a add lock/unlock to example admin scripts 2020-04-23 18:41:48 -07:00
Chris Lu 913aa0f14e mount: default to enable asyncMetaDataCaching 2020-04-23 14:14:28 -07:00
Chris Lu 662b5d0cf7 support semicolon seperated command lines 2020-04-23 14:01:46 -07:00
Chris Lu 73564e6a01 master: add cluster wide lock/unlock operation in weed shell 
fix https://github.com/chrislusf/seaweedfs/issues/1286
 2020-04-23 13:37:31 -07:00
Chris Lu bdc337a719 make lock/unlock optional 2020-04-23 03:50:05 -07:00
Chris Lu 57ec7f2ba9 fix builds 2020-04-23 03:43:45 -07:00
Chris Lu eab9701823 allow lock with an existing lock 2020-04-23 03:35:52 -07:00
Chris Lu 842e3301ee make second lock instant 2020-04-23 03:32:18 -07:00
Chris Lu 69f336e59f shell: add lock/unlock command 2020-04-23 03:11:07 -07:00
Chris Lu ff0a7c1d18 clean up 2020-04-23 03:10:47 -07:00
Chris Lu 30ee4f3291 add exclusive lock library on shell 2020-04-23 02:31:04 -07:00
Chris Lu 77873b832b add master side code for cluster wise exclusive lock 2020-04-23 01:55:44 -07:00