gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
3733 commits 28 branches 278 tags 135 MiB
Commit graph

2233 commits

Author SHA1 Message Date
Chris Lu 48880edc2c volume: fix typo in minFreeSpacePercent option 2020-06-14 20:37:05 -07:00
Chris Lu a0464fc267 fix help 
fix https://github.com/chrislusf/seaweedfs/issues/1359
 2020-06-12 22:06:21 -07:00
Chris Lu 0d1d83a5a5 mount: add option to auto create dir to mount 2020-06-12 11:46:58 -07:00
Chris Lu 00f96d86a1 fix typo 2020-06-11 10:53:25 -07:00
Chris Lu 63a1db64c5
Merge pull request #1358 from DXist/s3fixes 
Return NotSuchKey error code for bucket S3 DeleteObject method
 2020-06-11 10:37:19 -07:00
Rinat Shigapov 283b749ff1 keep 204 response 2020-06-11 19:00:34 +03:00
Rinat Shigapov fafc41a27f return xml encoded NotFound status code for s3 delete 2020-06-11 17:53:15 +03:00
Rinat Shigapov 2d2c5dfa39 proxy status code of filer delete response 2020-06-11 15:00:47 +03:00
Chris Lu 628b27ef3b purge old cache implementation 2020-06-11 01:50:00 -07:00
Chris Lu b9365de47b fix volume server pprof 2020-06-10 13:10:10 -07:00
Chris Lu 718052656a
Merge pull request #1355 from Kimbsen/add_http_pprof 
Add --pprof to volume servers to enable http pprof handlers
 2020-06-10 13:05:04 -07:00
Chris Lu 95a9686eaf 1.81 2020-06-10 13:00:42 -07:00
Chris Lu c66f357715 report error correctly 2020-06-10 10:59:25 -07:00
Kimbsen 03e8f976d9 added --pprof to volume servers 2020-06-10 11:52:24 +02:00
Chris Lu c38066be89 also add it for webdav 2020-06-09 18:07:03 -07:00
Chris Lu 4598473251 fix possible EOF problem 2020-06-09 18:04:40 -07:00
Chris Lu bb3e86fbd5 auto create non-existing cache directory 
fix https://github.com/chrislusf/seaweedfs/issues/1350
 2020-06-08 08:49:57 -07:00
Chris Lu bdbe5ddbe9 volume server: add a fix for dead lock 
https://github.com/chrislusf/seaweedfs/issues/1353
 2020-06-08 08:12:59 -07:00
Chris Lu b8e346cc86 1.80 2020-06-05 15:29:25 -07:00
Chris Lu 45a9d852d3 go fmt 2020-06-05 15:27:10 -07:00
Chris Lu 03c2f03509
Merge pull request #1351 from ekozlov-search/master 
Set volumes ReadOnly if low free disk space
 2020-06-05 09:56:46 -07:00
Chris Lu ebd1759741 fix ReadAt 
fix https://github.com/chrislusf/seaweedfs/issues/1344
 2020-06-05 09:54:14 -07:00
Evgenii Kozlov 2410ffbd79 Name fix 2020-06-05 19:00:38 +03:00
Evgenii Kozlov 0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
Chris Lu 393ab07c7c disable pulse seconds 
there are a few other hard coded timing settings, which will cause disconnected volume servers.
 2020-06-04 10:52:01 -07:00
Evgenii Kozlov 48f9ff52cf Discs statistics on Windows platform. 2020-06-02 11:52:16 +03:00
Chris Lu bc2ec6774d inject git version into build 2020-06-02 00:10:38 -07:00
bingoohuang 1f8782a1ed try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
bingoohuang 1a642b9876 add Volume Ids column only for max 100 volumes for convenience in the master ui. 2020-05-29 15:37:58 +08:00
bingoohuang accb4964b7 util.BytesToHumanReadable 2020-05-29 10:00:07 +08:00
bingoohuang 21d0a013d5 rename 'needle' which collides with imported package name 2020-05-28 17:00:00 +08:00
bingoohuang 41cab5d69f log JSON response if httpStatus >= 400 2020-05-28 08:56:55 +08:00
Chris Lu ef2b3a0801 leveldb recover error handling 2020-05-26 00:03:44 -07:00
Chris Lu 5d3ec22975 refactoring 2020-05-26 00:03:31 -07:00
Chris Lu d4235afe4d support multiple locks 2020-05-25 18:39:53 -07:00
bingoohuang ecdeef8c66 simplify func(w http.ResponseWriter, r *http.Request) to http.HandlerFunc 2020-05-25 21:00:12 +08:00
Chris Lu e5a0787653 filer: leveldb store add auto recovery, report correct error 
fix https://github.com/chrislusf/seaweedfs/issues/1326
 2020-05-22 10:54:42 -07:00
Chris Lu ed9e4dc749 1.79 
fix compilation problem
 2020-05-19 08:21:51 -07:00
Chris Lu 7ff2b0121d specify constant data type to uint64 instead of default int 
fix https://github.com/chrislusf/seaweedfs/issues/1324
 2020-05-19 07:55:23 -07:00
Chris Lu 11b33c0760 fix compilation problem 2020-05-18 10:53:31 -07:00
Chris Lu ac48c899e8 add printouts for synchronizing metadata which can be slow for large folders 2020-05-18 10:31:12 -07:00
Chris Lu 0c856e8387 avoid possible wrong arguments 
fix https://github.com/chrislusf/seaweedfs/issues/1320
 2020-05-18 10:07:12 -07:00
Chris Lu 2bfd810912 combine msgBroker into weed server command 2020-05-17 22:44:20 -07:00
Chris Lu 85aeead3d2 1.78 2020-05-17 20:28:04 -07:00
Chris Lu 8005786d9a logs 2020-05-17 20:27:10 -07:00
Chris Lu 75179544c3 logging 2020-05-17 20:20:21 -07:00
Chris Lu 91b91d6cb7 add error to avoid copying not found volume 
fix https://github.com/chrislusf/seaweedfs/issues/1317
 2020-05-17 20:20:12 -07:00
Chris Lu d3925d086c add delete channel 2020-05-17 17:42:42 -07:00
Chris Lu e0e31e67a8
Merge pull request #1318 from chrislusf/msg_channel 
Add messaging, add channel
 2020-05-17 17:39:16 -07:00
Chris Lu 081ee6fe34 reduc logs 2020-05-17 17:38:31 -07:00
Chris Lu bff09fb74f fix looping 2020-05-17 17:38:21 -07:00
Chris Lu f5684839a2 add DeleteTopic 2020-05-17 17:34:10 -07:00
Chris Lu 3a57aef7a9 sync subscribe() 2020-05-17 17:33:53 -07:00
Chris Lu 95ca9dd8a2 subscribe support cancel 2020-05-17 11:10:45 -07:00
Chris Lu f11233cd49 simplify disconnected stream detection 2020-05-17 08:57:47 -07:00
Chris Lu 1ea9bc66d9 avoid leaking grpc.NewClientStream 2020-05-17 08:56:33 -07:00
Chris Lu 85b53ac510 detect disconnected subscribers 2020-05-16 18:53:54 -07:00
Chris Lu 759cda0fe2 log 2020-05-16 18:47:41 -07:00
Chris Lu b0de01ff3b able to detect disconnected subscribers 2020-05-16 08:57:29 -07:00
Chris Lu e02a8c67da revert to one subscriber one thread 2020-05-15 21:38:42 -07:00
Chris Lu b4e02ec525 fix prometheus problem 
metrics.go:123] could not push metrics to prometheus push gateway 192.168.xx:9091: unexpected status code 200 while pushing to http://192.168.xx:9091/metrics/job/filer/instance/m111p112%3A8888:
 2020-05-14 21:08:34 -07:00
Chris Lu 3fbf635308 Merge branch 'master' into msg_channel 2020-05-13 03:54:15 -07:00
Chris Lu 3a5e4769cf prevent empty locations returned 
fix https://github.com/chrislusf/seaweedfs/issues/1313
 2020-05-13 03:46:38 -07:00
Chris Lu ca4017dd87 rename 2020-05-12 22:55:55 -07:00
Chris Lu 25257acd51 rename 2020-05-12 21:26:49 -07:00
Chris Lu a7959c1c48 multiple subscriber with same subscriberId shares the topic manager 
rename topicControl to topicCursor
 2020-05-12 21:26:02 -07:00
Chris Lu 2f243f5b0b refactor 2020-05-12 08:48:00 -07:00
Chris Lu 4b7fa31468 ensure montonically increasing tsNs 2020-05-11 01:53:54 -07:00
Chris Lu d5a8297a1c able to connect publisher 2020-05-10 23:45:48 -07:00
Chris Lu d58371c5cd Merge branch 'master' into msg_channel 2020-05-10 23:44:59 -07:00
Chris Lu 015dd3a147 batch file id deletion 2020-05-10 23:25:39 -07:00
Chris Lu 39e72fb23c go fmt 2020-05-10 03:50:30 -07:00
Chris Lu 6bf3eb69cb async chan write read, no write for closed chan 2020-05-10 03:48:35 -07:00
Chris Lu 78afb8bf46 Merge branch 'master' into msg_channel 2020-05-09 00:43:58 -07:00
Chris Lu d693e77418 add pub sub md5 2020-05-09 00:43:53 -07:00
Chris Lu 07d7abe428 add deleteTopic, refactoring 2020-05-09 00:31:34 -07:00
Chris Lu dfccc3c263 able to read chan and write chan 2020-05-08 02:47:22 -07:00
zhangsong b161baa3f0 fix volume vacuum test unit 2020-05-07 19:58:06 +08:00
Chris Lu a8bc8eb351 Merge branch 'master' into msg_channel 2020-05-07 03:18:24 -07:00
Chris Lu 73087f59b1 filer store: mongodb deleting children 
fix https://github.com/chrislusf/seaweedfs/issues/1303
 2020-05-07 03:11:26 -07:00
Chris Lu be1d9f324f Merge branch 'master' into msg_channel 2020-05-07 01:43:34 -07:00
Chris Lu 0952eb2854
Merge pull request #1302 from song-zhang/master 
write request keep compatible with before
 2020-05-06 15:37:34 -07:00
Chris Lu 5568395edd Revert "Revert "Merge pull request #1299 from song-zhang/master"" 
This reverts commit afb6a1dbb4.
 2020-05-06 15:37:17 -07:00
zhangsong 3703c7e176 write request keep compatible with before 2020-05-06 21:35:32 +08:00
Chris Lu 1e3e4b3072 add broker connects to filer 2020-05-05 02:05:28 -07:00
Chris Lu afb6a1dbb4 Revert "Merge pull request #1299 from song-zhang/master" 
This reverts commit 9016fa19ba, reversing
changes made to 47234760f4.
 2020-05-04 20:34:26 -07:00
Chris Lu 9016fa19ba
Merge pull request #1299 from song-zhang/master 
use async write to persistent file to disk - part1
 2020-05-04 15:51:03 -07:00
zhangsong f9e8702bb4 use async write to persistent file to disk - part1 2020-05-04 17:39:44 +08:00
Chris Lu 47234760f4 log adjust format 2020-05-02 01:33:49 -07:00
Chris Lu 97a504b0bb fix possible directory deletion error 2020-05-02 01:11:53 -07:00
Chris Lu 78422c2f69
Merge pull request #1298 from song-zhang/master 
add fsync option for benchmark
 2020-04-30 20:16:42 -07:00
Chris Lu 871efa4fc1 refactoring 
some previous chunk etag was using md5, which should be wrong.
 2020-04-30 17:20:47 -07:00
zhangsong d819865fe7 add fsync option for benchmark 2020-04-30 23:31:08 +08:00
Chris Lu ea809a45ab add cipher key 2020-04-30 03:05:43 -07:00
Chris Lu f9b6178b8f log messages 2020-04-30 03:05:34 -07:00
Chris Lu 8c73410a51 subscribe from a timestamp 2020-04-30 02:19:51 -07:00
Chris Lu 4e16a90454 for debugging 2020-04-30 02:19:32 -07:00
Chris Lu a25a8d8822 s3: print out time in UTC format 
fix https://github.com/chrislusf/seaweedfs/issues/1297
 2020-04-29 21:35:24 -07:00
Chris Lu e93588ec78 FUSE mount: atomic local cache updates 2020-04-29 18:20:54 -07:00
Chris Lu 9e72e9e4b8 able to subscribe any topic from any point of time 2020-04-29 17:40:08 -07:00
Chris Lu ed3cf811f5 refactoring 2020-04-29 13:26:02 -07:00
Chris Lu 7c10602b49 read in case cross chunks 2020-04-29 02:42:58 -07:00
Chris Lu f9da859720 better error message 2020-04-29 02:42:06 -07:00
Chris Lu d57559d3f7 chunk rememober encryption or not 2020-04-29 02:41:47 -07:00
Chris Lu 5a0986dca0 reset on getting the buffer 2020-04-29 02:41:01 -07:00
Chris Lu 8d42a1263b meta subscription: update the last read time 2020-04-28 17:30:04 -07:00
Chris Lu 31583b5d6c master and volume server: avoid race condition 
The volume server may disconnect and reconnect to the same master.

The master's unregistration may happen after the reconnection. Thus the volume server will disappear.
 2020-04-28 17:29:10 -07:00
Chris Lu 98edec7c3e remove printouts 2020-04-28 16:45:48 -07:00
Chris Lu 8ed490164e refactoring 2020-04-28 02:05:44 -07:00
Chris Lu 66effaed9e fix compilation 2020-04-28 02:05:32 -07:00
Chris Lu eab4c9219c randomize benchmark content 
fix https://github.com/chrislusf/seaweedfs/issues/1294
 2020-04-28 01:50:56 -07:00
Chris Lu fb81f12686 filer: simplify image resize 2020-04-28 00:05:57 -07:00
Chris Lu 5c57297bd1 metadata log: read from any timestamp 2020-04-28 00:05:57 -07:00
wuyuxiang 6850d28d6b refacotr: move signal handling and pprof to grace package 2020-04-28 14:10:23 +08:00
Chris Lu 2a7957b4ca FUSE: subcribe meta events based on timestamp 2020-04-27 16:52:20 -07:00
Chris Lu 6190fd665d printout error 2020-04-27 12:41:31 -07:00
Chris Lu 725e47dcc3 filer: pass along user name and group name for HDFS 2020-04-27 05:48:48 -07:00
Chris Lu 508ec01ab0 tweaking 2020-04-26 16:11:06 -07:00
Chris Lu f4fcde05b6 1.77 2020-04-26 05:54:47 -07:00
Chris Lu b52b8ec685 Hadoop: fix entry not found for HCFS 
also fix cipher related changes.
 2020-04-26 05:21:54 -07:00
Chris Lu 0c2248f83a add lock/unlock to example admin scripts 2020-04-23 18:41:48 -07:00
Chris Lu 913aa0f14e mount: default to enable asyncMetaDataCaching 2020-04-23 14:14:28 -07:00
Chris Lu 662b5d0cf7 support semicolon seperated command lines 2020-04-23 14:01:46 -07:00
Chris Lu 73564e6a01 master: add cluster wide lock/unlock operation in weed shell 
fix https://github.com/chrislusf/seaweedfs/issues/1286
 2020-04-23 13:37:31 -07:00
Chris Lu bdc337a719 make lock/unlock optional 2020-04-23 03:50:05 -07:00
Chris Lu 57ec7f2ba9 fix builds 2020-04-23 03:43:45 -07:00
Chris Lu eab9701823 allow lock with an existing lock 2020-04-23 03:35:52 -07:00
Chris Lu 842e3301ee make second lock instant 2020-04-23 03:32:18 -07:00
Chris Lu 69f336e59f shell: add lock/unlock command 2020-04-23 03:11:07 -07:00
Chris Lu ff0a7c1d18 clean up 2020-04-23 03:10:47 -07:00
Chris Lu 30ee4f3291 add exclusive lock library on shell 2020-04-23 02:31:04 -07:00
Chris Lu 77873b832b add master side code for cluster wise exclusive lock 2020-04-23 01:55:44 -07:00
Chris Lu 369aa8a10a 1.76 2020-04-22 23:31:59 -07:00
Chris Lu 69eda558ef reduce the connections 
fix https://github.com/chrislusf/seaweedfs/issues/1277

this avoids sending error to wdclients, and clients retry the connections
 2020-04-22 20:43:34 -07:00
Chris Lu aebe39a803 avoid repeated grpc connection creation 
fix https://github.com/chrislusf/seaweedfs/issues/1277
 2020-04-22 18:26:24 -07:00
Chris Lu 5d0e1d8d74 also writes to local meta cache 
before waiting for subscribed meta events
 2020-04-22 15:40:47 -07:00
Chris Lu 72c5dba4d4 1.75 2020-04-21 22:03:07 -07:00
Chris Lu a207285af7 cache metadata on startup 2020-04-21 22:00:34 -07:00
Chris Lu 9f9826e95a refactor 2020-04-21 21:28:47 -07:00
Chris Lu e24b25de78 async meta caching: can stream updates now 2020-04-21 21:16:13 -07:00
Chris Lu 4f02f7121d read from meta cache 
meta cache is not initialized
 2020-04-21 18:50:30 -07:00
Chris Lu b8e4238ad2 add placeholder for cached meta data reading 2020-04-21 18:02:08 -07:00
Chris Lu 27128c7875 adjust logging 2020-04-21 16:13:13 -07:00
Chris Lu bafa95045b volume: deletion checks all disk locations 
fix https://github.com/chrislusf/seaweedfs/issues/1283
 2020-04-21 14:49:58 -07:00
Chris Lu 57df14f76f report error if copying failed 2020-04-21 14:28:46 -07:00
Chris Lu cf54d70311 filer: add -ip.bind option 2020-04-21 14:21:06 -07:00
Chris Lu 5c348087dc messaging: able to pub sub multiple partitions 2020-04-21 00:59:55 -07:00
Chris Lu cb3985be70 go fmt 2020-04-20 17:48:06 -07:00
Chris Lu 5eb83dfbd3 add memory buffer size limit 2020-04-20 17:43:50 -07:00
Chris Lu 258fba8a0f continue for reading from sealed memory buffer 2020-04-20 17:28:18 -07:00