gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6910 commits 28 branches 278 tags 135 MiB
Commit graph

632 commits

Author SHA1 Message Date
chrislu 5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu f7a6f6b4c0 if this is enabled, there are some "bus error" with git clone 2021-12-28 13:44:52 -08:00
chrislu 0da2dfd640 fuse: change to direct io mode 
before and after:

chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192
8192+0 records in
8192+0 records out
1073741824 bytes transferred in 4.534068 secs (236816430 bytes/sec)
dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192  0.01s user 3.86s system 84% cpu 4.561 total
chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192
8192+0 records in
8192+0 records out
1073741824 bytes transferred in 3.824072 secs (280784948 bytes/sec)
dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192  0.01s user 3.22s system 83% cpu 3.857 total
 2021-12-28 12:22:56 -08:00
chrislu 2422556456 monitor write pattern: avoid timing due to locking 2021-12-28 01:10:35 -08:00
chrislu 80db8b13d8 bug: cleanup function was called twice 2021-12-27 20:53:02 -08:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls 
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
 2021-12-26 00:15:03 -08:00
chrislu eb4ad2546f use proper chunk size limit option 2021-12-24 22:52:18 -08:00
chrislu 41bbf320bb use 2MB chunk size. cache size is the wrong option 2021-12-24 22:50:19 -08:00
chrislu 083d8e9ece add stream writer 
this should improve streaming write performance, which is common in many cases, e.g., copying large files.

This is additional to improved random read write operations: 3e69d19380...19084d8791
 2021-12-24 22:38:22 -08:00
chrislu 255a1c7dcd refactor type names 2021-12-23 18:23:18 -08:00
chrislu f77ca41769 refactor 2021-12-23 17:48:34 -08:00
chrislu 1d36884845 rename files 2021-12-23 17:47:58 -08:00
chrislu 2d1a1f5e03 rename variables and functions 2021-12-23 17:35:57 -08:00
chrislu 6de331b014 clean up 2021-12-23 17:23:21 -08:00
chrislu 032df784ed chunked file works now 2021-12-23 17:17:32 -08:00
chrislu c2aad1c7ff detect non streaming mode on first write request 2021-12-22 17:20:44 -08:00
chrislu 7b78fc72b0 add page chunk interval list 2021-12-22 02:53:33 -08:00
chrislu 3981d65b68 remove println 2021-12-21 22:24:38 -08:00
chrislu 9a73319b45 mount: different write strategy for streaming write and random write 2021-12-21 17:28:55 -08:00
chrislu b21a67bbe6 add writer pattern object for later use 2021-12-20 11:53:48 -08:00
chrislu 4fd29dad86 remove writeOnly flag 2021-12-20 01:11:43 -08:00
chrislu bc96682760 refactor, change file locations 2021-12-20 01:02:23 -08:00
chrislu 866c2657f0 avoid FUSE cache only for the first 512 bytes 2021-12-19 23:13:36 -08:00
chrislu a152f17937 mount: improve read performance on random reads 2021-12-19 22:43:14 -08:00
liubaojiang c7abc34dbb rename operation returns the correct old parent path 2021-12-10 00:02:57 +08:00
Chris Lu 0c8dea9de8 go fmt 2021-11-02 23:39:16 -07:00
Chris Lu 24858507cc rename API to avoid confusion 2021-10-30 19:27:25 -07:00
Chris Lu cca62fdb30 mount: streaming renaming folders 2021-10-17 04:22:42 -07:00
Chris Lu 004e56c1a6 save updated lastTsNs 2021-10-17 02:01:20 -07:00
Chris Lu 93bb7869b8 Revert "mount: fix renaming a deep directory with unvisited directories" 
This reverts commit 0ccdb937bb.
 2021-10-16 23:52:30 -07:00
Chris Lu 0ccdb937bb mount: fix renaming a deep directory with unvisited directories 2021-10-16 23:33:45 -07:00
Chris Lu 7504be58f9 Avoid xattr printed out as HTTP headers 
fix https://github.com/chrislusf/seaweedfs/issues/2336
 2021-09-13 04:00:57 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu 6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu 223c1b3571 adjust invalidation logic 2021-08-28 05:21:01 -07:00
nivekuil 8c523d50ad Invalidate mount meta cache more aggressively 2021-08-28 04:26:37 -07:00
Chris Lu 3b2c39f1be mount: set name when mount path equals mount folder 
fix https://github.com/chrislusf/seaweedfs/issues/2275#issuecomment-903255876
 2021-08-22 04:59:09 -07:00
Chris Lu c3ffd457ef fix compilation error 2021-08-15 12:40:22 -07:00
Chris Lu fda2fc47b1 add RetryForever 2021-08-15 12:37:35 -07:00
byunghwa.yun 775dfbae85 Synchronize number of open files 2021-08-11 23:14:56 +09:00
Chris Lu 69655ba8e5 mount: cache on reading remote storage 2021-08-09 22:11:57 -07:00
Chris Lu 6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu 84d91f143f mount: hide /etc folder also from mount 2021-07-22 00:40:16 -07:00
Chris Lu 7359193e97 go fmt 2021-07-21 14:38:12 -07:00
Chris Lu 7ab389e7ec optimization: improve random range query for large files 2021-07-19 23:07:22 -07:00
Chris Lu fa0dab6029 mount: rename also recursively move file handles 
related to https://github.com/chrislusf/seaweedfs/issues/2169
 2021-07-03 02:59:35 -07:00
Chris Lu b624090398 go fmt 2021-07-01 01:21:14 -07:00
Chris Lu 215b169562 mount: recursively rename locally 2021-07-01 01:19:31 -07:00
Chris Lu c795183887 return node itself as directory handler 2021-07-01 01:19:31 -07:00
Chris Lu 07f20155fd small optimization 2021-07-01 01:19:31 -07:00
Chris Lu 055374a50b FUSE: skip flushing if file is deleted 
related to https://github.com/chrislusf/seaweedfs/issues/2110
 2021-06-15 12:45:23 -07:00
Chris Lu e00443a940 mount: adjust starting order 
avoid possible nil wfs.Server
 2021-06-06 20:22:42 -07:00
Chris Lu e6ba2f9c37 verbose but cleaner logic to invalidate fuse cache 2021-06-05 13:09:37 -07:00
Chris Lu fadc1febdb FUSE: invalidate FUSE cached entries 
fix https://github.com/chrislusf/seaweedfs/issues/2108
 2021-06-05 02:23:07 -07:00
Patrick Schmidt 77100754e6 Return artificial . and .. directories 2021-06-02 21:28:02 +02:00
Chris Lu cc34475012 remove file handle locking for setattr 2021-06-02 11:44:12 -07:00
Chris Lu 3db1642392 mount: skip persisting metadata if file is open 2021-06-02 03:40:08 -07:00
Chris Lu 2f136a04a1 organize a bit better for temp file directory 2021-05-21 13:02:18 -07:00
Chris Lu f37a4cbd0f randomize initial filer 2021-05-21 01:41:34 -07:00
Chris Lu dc1309f084 FUSE mount: support multiple filers 
fix https://github.com/chrislusf/seaweedfs/issues/2015
fix https://github.com/chrislusf/seaweedfs/issues/1531
 2021-05-21 01:28:00 -07:00
Chris Lu 789465d99e set renamed item to new directory 
related to https://github.com/chrislusf/seaweedfs/issues/2064
 2021-05-13 22:52:19 -07:00
Chris Lu 83c037e093 fix logs 2021-05-13 22:41:19 -07:00
Chris Lu 5a29d284fa ensure proper directory name 
a fix related to https://github.com/chrislusf/seaweedfs/issues/2064
 2021-05-13 10:18:35 -07:00
Chris Lu 3ff307e842 reduce logs 2021-05-13 00:32:37 -07:00
Chris Lu a48ebd7c73 mount: read file when file is still being written 
a possible fix for https://github.com/chrislusf/seaweedfs/issues/2065
 2021-05-13 00:32:23 -07:00
Chris Lu ceb620a30a directory rename: change directory name after renaming 
fix https://github.com/chrislusf/seaweedfs/issues/2068
 2021-05-12 22:04:47 -07:00
Chris Lu 1737af480a adjust logs 2021-05-10 21:47:51 -07:00
Chris Lu d06ecc2649 working properly 2021-05-10 21:47:07 -07:00
Chris Lu 55d547afec switch back to in memory uploading for now 2021-05-10 13:50:10 -07:00
Chris Lu ba77833343 EOF handling 2021-05-09 23:04:24 -07:00
Chris Lu f4cf30b497 mount: write to tmp file before uploading 2021-05-09 22:56:10 -07:00
Chris Lu 280e7cf289 mount: in case the set attribute is called before persisting the file 2021-05-09 22:55:30 -07:00
Chris Lu 74052064b6 add one missing optimization 2021-05-09 17:25:42 -07:00
Chris Lu 59ace54925 refactor 2021-05-09 17:22:30 -07:00
Chris Lu 50be19d23e refactor 2021-05-09 15:33:01 -07:00
Chris Lu 55e060cf61 refactor 2021-05-09 15:28:54 -07:00
Chris Lu 735e65be17 refactor 2021-05-09 15:22:38 -07:00
Chris Lu 93e84a12f2 refactor 2021-05-09 15:15:18 -07:00
Chris Lu 8f8738867f add retry to assign volume 
fix https://github.com/chrislusf/seaweedfs/issues/2056
 2021-05-07 07:29:26 -07:00
Chris Lu 55a8f57381 go fmt 2021-05-06 03:37:51 -07:00
Chris Lu 5753749c90 remove verbose logs 2021-05-06 03:34:34 -07:00
Chris Lu 38f411219a mount: skip local chunk cache if opened write only 2021-05-06 03:31:40 -07:00
Chris Lu 2e56407c6b fix visited checking 2021-05-01 16:09:29 -07:00
Chris Lu 3a86d4dbfd mount: fix directory invalidation 
fix https://github.com/chrislusf/seaweedfs/issues/2038
 2021-04-30 22:51:06 -07:00
Chris Lu 11c405fc85 ensure file handles are released 2021-04-20 19:56:55 -07:00
Chris Lu 83cf94ad2d delay new file creation unless file is opened exclusively 2021-04-19 10:58:25 -07:00
Chris Lu d1c813c470 let the fuse library manage directory id 
otherwise, on mac, during large directory deletion, if some ReaDirAll happens, the lib seems confused about the directories, and some child directories are not deleted.
 2021-04-18 13:07:28 -07:00
Chris Lu 372872ebbf set root node inode number 2021-04-18 13:07:28 -07:00
Chris Lu e332da4837 set inode value 2021-04-18 13:07:28 -07:00
Chris Lu 6cbd786db9 correctly runs git clone 2021-04-18 13:07:28 -07:00
Chris Lu d9a2a7f1c4 WIP 
no memory issue

if some directory is removed, it may have this error

$ rm -Rf ~/tmp/m2/s1
rm: fts_read: Device not configured
 2021-04-18 13:06:38 -07:00
Chris Lu 54410ca955 cleaner way to set readonly 2021-04-18 10:02:02 -07:00
Chris Lu c83ab91e2e remove unused variable 2021-04-16 10:34:02 -07:00
Chris Lu 3074e9b428 ensure consistent inode value 2021-04-15 22:42:24 -07:00
Chris Lu 16c0304416 ensure to delete on filer also 2021-04-15 02:29:04 -07:00
Chris Lu 217e0f9066 mount: remove folder recursively 2021-04-15 01:51:10 -07:00
Chris Lu 3e669e6d7b mostly refactoring, add some error handling 2021-04-14 23:33:37 -07:00
Chris Lu 07f712c83f fix typo 2021-04-14 23:21:38 -07:00
Chris Lu 36c79de3f4 fuse mount: dir ReadDirAll avoid extra conversion to filer_pb.Entry 2021-04-14 23:21:24 -07:00
Chris Lu e41766feb6 fuse mount: dir lookup avoids extra conversion to filer_pb.Entry object 2021-04-14 22:38:34 -07:00