gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
3956 commits 28 branches 278 tags 135 MiB
Commit graph

337 commits

Author SHA1 Message Date
Chris Lu ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu 2d1df97933 mount: fatal error if can not process meta data 2020-07-11 06:16:48 -07:00
Chris Lu e3e00d36c5 reset cache for each new release 2020-07-11 06:16:17 -07:00
Chris Lu 7ad0bce2f5 remove old code 2020-07-10 23:03:22 -07:00
Chris Lu 54d6b3c30f filer: support running multiple mount to different filers avoiding duplicated cache 
fix https://github.com/chrislusf/seaweedfs/issues/1382
 2020-07-07 23:18:44 -07:00
Chris Lu ce3630e7a4 refactoring 2020-07-03 22:16:59 -07:00
Chris Lu 31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu 42a338d7b0 FUSE mount: clean up file handles during renaming 
resolve dir rename when file is still open. Need to clean the file handles as soon as possible.

These can happen out of order:
file rename, then file release
file release, then file rename
 2020-06-28 13:41:00 -07:00
Chris Lu 95a1860d65 remove empty calls for list cache 2020-06-28 10:25:54 -07:00
Chris Lu 6498d0c86b remove list cache 2020-06-28 10:21:52 -07:00
Chris Lu 9033a7d369 removed async option 2020-06-28 10:18:32 -07:00
Chris Lu 1bb8cae65d reverting and working 2020-06-28 10:14:17 -07:00
Chris Lu 3cec4b3c49 FUSE mount: fix file id written twice? 
fix https://github.com/chrislusf/seaweedfs/issues/1373
 2020-06-25 20:11:49 -07:00
Chris Lu e8e0d629af ensure correct file mode 2020-06-24 23:26:54 -07:00
Chris Lu f7a45d448f FUSE mount: lazy loading meta cache 2020-06-19 09:45:42 -07:00
Chris Lu b74eced7a3 mount: a fix to prevent possible repeated calls 
related to https://github.com/bazil/fuse/issues/130
 2020-06-17 13:55:16 -07:00
Chris Lu 628b27ef3b purge old cache implementation 2020-06-11 01:50:00 -07:00
Chris Lu 4598473251 fix possible EOF problem 2020-06-09 18:04:40 -07:00
Chris Lu bb3e86fbd5 auto create non-existing cache directory 
fix https://github.com/chrislusf/seaweedfs/issues/1350
 2020-06-08 08:49:57 -07:00
Chris Lu ac48c899e8 add printouts for synchronizing metadata which can be slow for large folders 2020-05-18 10:31:12 -07:00
Chris Lu 47234760f4 log adjust format 2020-05-02 01:33:49 -07:00
Chris Lu 97a504b0bb fix possible directory deletion error 2020-05-02 01:11:53 -07:00
Chris Lu 871efa4fc1 refactoring 
some previous chunk etag was using md5, which should be wrong.
 2020-04-30 17:20:47 -07:00
Chris Lu e93588ec78 FUSE mount: atomic local cache updates 2020-04-29 18:20:54 -07:00
Chris Lu 9e72e9e4b8 able to subscribe any topic from any point of time 2020-04-29 17:40:08 -07:00
Chris Lu ed3cf811f5 refactoring 2020-04-29 13:26:02 -07:00
wuyuxiang 6850d28d6b refacotr: move signal handling and pprof to grace package 2020-04-28 14:10:23 +08:00
Chris Lu 2a7957b4ca FUSE: subcribe meta events based on timestamp 2020-04-27 16:52:20 -07:00
Chris Lu 5d0e1d8d74 also writes to local meta cache 
before waiting for subscribed meta events
 2020-04-22 15:40:47 -07:00
Chris Lu a207285af7 cache metadata on startup 2020-04-21 22:00:34 -07:00
Chris Lu e24b25de78 async meta caching: can stream updates now 2020-04-21 21:16:13 -07:00
Chris Lu 4f02f7121d read from meta cache 
meta cache is not initialized
 2020-04-21 18:50:30 -07:00
Chris Lu b8e4238ad2 add placeholder for cached meta data reading 2020-04-21 18:02:08 -07:00
Chris Lu 9cacaf5eca better detect file mime type 2020-04-14 11:32:31 -07:00
Chris Lu 2b5c4fbbf3 tiered caching 
1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files
 2020-04-13 22:19:27 -07:00
Chris Lu 7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu 211d87cf4c mount: option to disable caching 2020-04-12 00:52:54 -07:00
Chris Lu df97da25f9 mount: add on disk caching 2020-04-11 21:12:41 -07:00
Chris Lu d7f3acb2c0 refactor 2020-04-11 12:45:24 -07:00
Chris Lu e909b55633 refactoring 2020-04-11 12:39:46 -07:00
Chris Lu 13ab1fdaf0 mount: use file default permission 0666 2020-04-10 02:23:23 -07:00
Chris Lu eb39df2704 avoid dead loop 2020-04-09 21:43:26 -07:00
Chris Lu 56ec8c4513 fix to avoid nil file an dirty pages 2020-04-09 21:43:05 -07:00
Chris Lu bb78ab9915 wait for master to be aware of the failed volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1268
 2020-04-08 23:12:37 -07:00
Chris Lu 006f78b70b make a copy of request data 
to avoid concurrency issues
 2020-04-08 22:31:19 -07:00
Chris Lu 6630541399 ensure correct file size when opening existing files 2020-04-08 12:50:59 -07:00
Chris Lu b524a40375 add locking to fs cache 2020-04-08 12:50:34 -07:00
Chris Lu bd56172b82 simplify file handle management 2020-04-08 12:50:20 -07:00
Chris Lu 9dc0b1df8f refactoring to compile for windows OS 
windows os does not like to work with fuse
 2020-03-29 21:07:55 -07:00
Chris Lu 54768d0761 fix: delete a file and then create a directory with the same name 2020-03-29 01:39:48 -07:00
Chris Lu 057722bbf4 return part of the chunk if chunkview is not the full chunk 2020-03-29 00:54:39 -07:00
Chris Lu af6d9976ad adjust print out 2020-03-28 14:07:25 -07:00
Chris Lu a75d50bbb8 FUSE: add configurable in memory chunk cache size 2020-03-28 14:07:16 -07:00
Chris Lu 826bc0b7e3 FUSE: add chunk cache for recently accessed file chunks 2020-03-28 13:43:31 -07:00
Chris Lu eedd33dda3 refactoring 2020-03-28 13:41:58 -07:00
Chris Lu d1439c5bd3 fix FUSE read for large files 
FUSE expects ReadAt do not return partial filled buffer with a nil error.
 2020-03-27 04:50:51 -07:00
Chris Lu e1911760a7 refactoring 2020-03-27 00:30:55 -07:00
Chris Lu 2f5ed29c37 clean up 2020-03-27 00:30:39 -07:00
Chris Lu e7e8691720 add some tests 2020-03-26 22:21:52 -07:00
Chris Lu 1fef598347 set target Dir object when moving 2020-03-26 10:56:40 -07:00
Chris Lu 9e2e07b238 tweak 2020-03-26 10:56:18 -07:00
Chris Lu cbce793753 rename: set new name for renamed files 2020-03-26 03:30:23 -07:00
Chris Lu c763a492f5 refactor 2020-03-26 03:30:02 -07:00
Chris Lu dd5b582d05 go fmt 2020-03-26 00:09:01 -07:00
Chris Lu 7f0d87b206 tree structured fs cache 
FsCache for FsNode, wrapping fs.Node
 2020-03-26 00:08:14 -07:00
Chris Lu 2e4fadd10a fix fscache move 2020-03-25 22:19:19 -07:00
Chris Lu 45ee3736aa refactoring 2020-03-23 01:25:38 -07:00
Chris Lu c0f0fdb3ba refactoring 2020-03-23 00:01:34 -07:00
Chris Lu 0bf148f49d logging 2020-03-22 16:19:00 -07:00
Chris Lu bda13ed593 commented prints 2020-03-22 13:09:51 -07:00
Chris Lu 74e23e8d64 FUSE mount: rename correctly across folders 2020-03-22 13:09:33 -07:00
Chris Lu 64000f5c45 FUSE mount: rename under root 
fix https://github.com/chrislusf/seaweedfs/issues/1242
 2020-03-22 13:09:16 -07:00
Chris Lu ae2ee379c0 consistent 64bit size 2020-03-22 01:37:46 -07:00
Chris Lu 65d2ea9fb0 FUSE mount: stream read data with buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1244
 2020-03-22 01:00:36 -07:00
Chris Lu 2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu afb20de14c breaks dependency loop 2020-03-07 17:01:39 -08:00
Chris Lu 8645283a7b fuse mount: avoid lookup nil entry 
fix https://github.com/chrislusf/seaweedfs/issues/1221
 2020-03-07 16:51:46 -08:00
Chris Lu 13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu f90c43635d refactoring 2020-03-04 00:39:47 -08:00
Chris Lu 0156e2975a mount: add mode to run external to SeaweedFS container cluster 2020-02-26 16:46:01 -08:00
Chris Lu 36d6595658 remove ctx 2020-02-25 22:38:27 -08:00
Chris Lu fd9612d66e remove ctx 2020-02-25 22:37:54 -08:00
Chris Lu 97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu 892e726eb9 avoid reusing context object 
fix https://github.com/chrislusf/seaweedfs/issues/1182
 2020-02-25 21:50:12 -08:00
Chris Lu bd3254b53f adjust logging 2020-02-25 17:24:08 -08:00
Chris Lu 0841bedb15 move filer assign volume grpc errror to response 2020-02-25 17:15:09 -08:00
Chris Lu 7d10fdf737 fix directory lookup nil 2020-02-25 11:13:06 -08:00
Chris Lu e86da5a491 minor 2020-02-25 00:42:48 -08:00
Chris Lu 6ab7368ef2 filer: dynamically create bucket under /buckets folder 2020-02-24 22:28:45 -08:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu f724057051 reduce logs 2020-01-27 21:20:41 -08:00
Chris Lu c8d543ef51 avoid retry delay 2020-01-27 00:58:26 -08:00
Chris Lu 081bc1ea25 mount: able to write to any part of a file 2020-01-27 00:54:52 -08:00
Chris Lu 2f6bb57979 logging 2020-01-27 00:54:21 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu 08e4b56a8a mount: able to handle large git clone 2020-01-26 13:01:11 -08:00
Chris Lu c48fc8b4de grpc send error via response instead of grpc error 2020-01-25 09:17:19 -08:00
Chris Lu ccf3859152 log 2020-01-25 08:16:43 -08:00
Chris Lu 4433c7bdca logs 2020-01-25 07:34:09 -08:00
Chris Lu 8cccccce9f mount: reuse the entry object for the new directory 
avoid mkdir and then query for the same directory
reduces these "context canceled" issues

attr read directory:"/seaweedfs/other/java/s3copier/src/main/java/com" name:"seaweedfs" : rpc error: code = Canceled desc = context canceled
 2020-01-25 01:15:54 -08:00