gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6029 commits 28 branches 278 tags 135 MiB
Commit graph

542 commits

Author SHA1 Message Date
Chris Lu d9a2a7f1c4 WIP 
no memory issue

if some directory is removed, it may have this error

$ rm -Rf ~/tmp/m2/s1
rm: fts_read: Device not configured
 2021-04-18 13:06:38 -07:00
Chris Lu 54410ca955 cleaner way to set readonly 2021-04-18 10:02:02 -07:00
Chris Lu c83ab91e2e remove unused variable 2021-04-16 10:34:02 -07:00
Chris Lu 3074e9b428 ensure consistent inode value 2021-04-15 22:42:24 -07:00
Chris Lu 16c0304416 ensure to delete on filer also 2021-04-15 02:29:04 -07:00
Chris Lu 217e0f9066 mount: remove folder recursively 2021-04-15 01:51:10 -07:00
Chris Lu 3e669e6d7b mostly refactoring, add some error handling 2021-04-14 23:33:37 -07:00
Chris Lu 07f712c83f fix typo 2021-04-14 23:21:38 -07:00
Chris Lu 36c79de3f4 fuse mount: dir ReadDirAll avoid extra conversion to filer_pb.Entry 2021-04-14 23:21:24 -07:00
Chris Lu e41766feb6 fuse mount: dir lookup avoids extra conversion to filer_pb.Entry object 2021-04-14 22:38:34 -07:00
Chris Lu 1adc8f86ea lighten up File object 
file.entry only exists when file.isOpen
 2021-04-14 20:49:15 -07:00
Chris Lu c04b7e106f mount: remove entry from Dir object 2021-04-14 20:26:13 -07:00
Chris Lu 3f3268cd1b go fmt 2021-04-14 00:30:16 -07:00
Chris Lu ca0f07a188 move file reader, entryViewCache to file handle 
reduce file object size
 2021-04-14 00:29:58 -07:00
Chris Lu f62c153274 go fmt 2021-04-10 23:48:18 -07:00
Chris Lu 93f4146ffa properly release the view cache 2021-04-09 12:36:39 -07:00
Chris Lu 6deb647a8f mount: fix possible memory leak 
if many files are read repeatedly, their metadata are accumulated in memory. This fix cleared the metadata after the file is read.
 2021-04-08 19:47:31 -07:00
Chris Lu 0f64f5b9c8 mount: add readOnly option 
fix https://github.com/chrislusf/seaweedfs/issues/1961
 2021-04-04 21:40:58 -07:00
Chris Lu bdf2ddddfd revert to same implementation as before 
This reverts commit 7e8edc3c4a.
 2021-04-02 02:21:38 -07:00
Chris Lu 7e8edc3c4a refactoring 2021-04-02 01:10:24 -07:00
Chris Lu 3cbc40fa48 avoid creating multiple reader 2021-03-22 22:32:47 -07:00
Chris Lu 288369cfc7 mount: release resources when Forget() is called 
address https://github.com/chrislusf/seaweedfs/issues/1929
 2021-03-22 16:22:19 -07:00
Chris Lu df461402cc ensure entry attribute is not nill 
fix https://github.com/chrislusf/seaweedfs/issues/1926
 2021-03-21 22:27:30 -07:00
Chris Lu 69694a17be reverting 7d57664c2d 2021-03-16 02:59:26 -07:00
Chris Lu c00dd5e62e report error if entry not found 2021-03-16 02:55:49 -07:00
Chris Lu 22a07a12d2 [mount] [regression] filer.path option results in empty mount in 2.32 
fix https://github.com/chrislusf/seaweedfs/issues/1906
 2021-03-15 20:46:25 -07:00
Chris Lu 35d939a1d2 go fmt 2021-03-14 13:21:02 -07:00
Chris Lu 5fbcaaf37a fullpath() test is not needed 2021-03-12 09:45:25 -08:00
Chris Lu 7d57664c2d mount: internals switch to filer.Entry instead of protobuf 2021-03-12 00:36:38 -08:00
Chris Lu aba47fd9e8 reduce repeated calls to dir.FullPath() 2021-03-11 14:08:20 -08:00
Chris Lu 5ba4b479f8 properly lock file.entry object 
fix https://github.com/chrislusf/seaweedfs/issues/1882
 2021-03-09 23:08:38 -08:00
Chris Lu bcf32591b7 reduce possibility of nil entry 2021-03-02 13:33:56 -08:00
Chris Lu be9c7c21ec reduce possibility of nil file entry 2021-03-02 11:17:17 -08:00
Chris Lu 02e146f3e1 return err 2021-03-02 08:54:18 -08:00
Chris Lu f8446b42ab this can compile now!!! 2021-02-16 02:47:02 -08:00
Chris Lu 712b3e9e53 mount: release need to avoid nil reader 
fix https://github.com/chrislusf/seaweedfs/issues/1803
 2021-02-13 13:39:37 -08:00
Chris Lu 0604afcda7 mount: remove unnecessary flush on release 2021-02-12 06:01:04 -08:00
Chris Lu a0e84c4fbc go fmt 2021-02-10 23:41:05 -08:00
Chris Lu 821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu d475c89fcc go fmt 2021-01-28 15:23:46 -08:00
Chris Lu 19295600f9 mount: change option name to volumeServerAccess, with publicUrl and filerProxy modes 2021-01-28 15:23:16 -08:00
Chris Lu 990fa69bfe add back AdjustedUrl() related code 2021-01-28 14:36:29 -08:00
Chris Lu cf252fc0cd mount: report error when Flush() 
The error in Release() is not working.

See https://github.com/jaderhs/libfuse/blob/master/FAQ

related to https://github.com/chrislusf/seaweedfs/issues/1765
 2021-01-28 04:46:37 -08:00
Chris Lu b81956bcb5 mount: invalidate kernel cache when mounted to a filer path 
fix https://github.com/chrislusf/seaweedfs/issues/1752#issuecomment-768178422
 2021-01-27 10:28:37 -08:00
Chris Lu 3a1d3d3413 mount: properly invalidate kernel node cache entry 
fix https://github.com/chrislusf/seaweedfs/issues/1752
 2021-01-26 02:50:53 -08:00
Chris Lu 62ebb917e0 mount: avoid the need to specify different cache directory for different mount 
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/11
 2021-01-25 09:10:20 -08:00
Chris Lu 17f7c1c43f Merge branch 'mount_from_outside_cluster' 2021-01-24 19:03:02 -08:00
Chris Lu 00707ec00f mount: outsideContainerClusterMode proxy through filer 
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
 2021-01-24 19:01:58 -08:00
Chris Lu 9bcb28a3ea avoid possible nil reader 
fix https://github.com/chrislusf/seaweedfs/issues/1754
 2021-01-24 17:19:35 -08:00
Chris Lu 6ca10725b8 Revert "mount: when outside cluster network, use filer as proxy to access volume servers" 
This reverts commit 096e088d7b.
 2021-01-24 03:15:19 -08:00
Chris Lu 096e088d7b mount: when outside cluster network, use filer as proxy to access volume servers 2021-01-24 01:41:38 -08:00
Chris Lu 20ef3bb8d4 mount: use direct_io to avoid OS page cache 
fix https://github.com/chrislusf/seaweedfs/issues/1752
 2021-01-22 22:39:46 -08:00
Chris Lu e439b65e38 fix test 2021-01-18 01:48:00 -08:00
Chris Lu 5a3386e39b Revert "Revert "mount: fake support for socket/block/character/fifo devices"" 
This reverts commit 3d5cb7eb86.
 2021-01-18 01:15:07 -08:00
Chris Lu 67faa56920 Revert "Revert "mount: add more entry type"" 
This reverts commit 4085b79d38.
 2021-01-18 01:14:58 -08:00
Chris Lu 7119e7b491 mount: ensure reading from dirty pages 2021-01-18 01:14:42 -08:00
Chris Lu 4085b79d38 Revert "mount: add more entry type" 
This reverts commit d7d907be27.
 2021-01-18 00:19:19 -08:00
Chris Lu 3d5cb7eb86 Revert "mount: fake support for socket/block/character/fifo devices" 
This reverts commit 61ef2d8658.
 2021-01-18 00:18:57 -08:00
Chris Lu 61ef2d8658 mount: fake support for socket/block/character/fifo devices 
to pass pjdfstest. A distributed file system can not really support these things anyway.
 2021-01-17 23:51:10 -08:00
Chris Lu d7d907be27 mount: add more entry type 2021-01-17 23:46:39 -08:00
Chris Lu a4063a5437 add stream list directory entries 2021-01-15 23:56:24 -08:00
Chris Lu f002e668de change limit to int64 in case of overflow 2021-01-14 23:10:37 -08:00
Chris Lu 9a50dbcda0 chagned api 2021-01-14 22:18:38 -08:00
Chris Lu d0826c2f62 mount: avoid showing "topics" folder 
fix https://github.com/chrislusf/seaweedfs/issues/1325
 2021-01-14 19:03:15 -08:00
Chris Lu 38d516251e Revert "mount: avoid fatal error" 
This reverts commit a4f0bac9b7.
 2021-01-11 13:50:15 -08:00
Chris Lu a4f0bac9b7 mount: avoid fatal error 2021-01-11 13:26:59 -08:00
Chris Lu 1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu 0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu 3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu 4211601eab set file handle reader to nil 2020-12-08 22:48:18 -08:00
Chris Lu 8e78187a97 add back last read chunk cache to reader and properly close the reader 2020-12-08 22:26:46 -08:00
Chris Lu f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu 1ae108efca add debug message 2020-11-26 10:41:10 -08:00
Chris Lu 0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu 06ee199ef3 ensure entryViewCache is in sync with the chunks 2020-11-03 12:22:19 -08:00
Chris Lu b132cb893f in case some chunks modified earliar are added later 2020-11-03 12:22:19 -08:00
Chris Lu 0ae4d7bfdf lock "addChunks()" 2020-11-03 12:22:19 -08:00
Chris Lu 7d5db73d98 go fmt 2020-11-02 14:21:07 -08:00
Chris Lu a9662d44ac avoid closing chunkSaveErrChan too early 2020-11-02 14:20:38 -08:00
Chris Lu e1423a83dd mount: clear possible leftover entry data in memory 2020-11-01 23:28:56 -08:00
Chris Lu 8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu 4609644658 mount: remove files that may still are open 
related to https://github.com/chrislusf/seaweedfs/issues/1581
 2020-11-01 01:33:26 -08:00
Chris Lu df8d976bb0 refactoring 2020-11-01 01:58:48 -07:00
Chris Lu f7b21973ac mount: add retry for all operations with filer 
fix https://github.com/chrislusf/seaweedfs/issues/1589
 2020-11-01 01:11:08 -07:00
Chris Lu 10f9f1cc26 handle mknod requests 2020-10-30 23:51:32 -07:00
Chris Lu 8826601be1 mount: optional limit for the number of concurrent writers 2020-10-30 21:22:20 -07:00
Chris Lu 19098f2c2d make a copy of the filer.entry for most read operations 2020-10-30 01:23:26 -07:00
Chris Lu 6135cbaa02 clear file.entry 2020-10-30 01:23:26 -07:00
Chris Lu 02dc51b1ed properly clear out the file.entry 2020-10-29 23:31:06 -07:00
Chris Lu 0b68b68ec4 reload entry only when it is a hard link 
fix https://github.com/chrislusf/seaweedfs/issues/1581
 2020-10-29 16:32:45 -07:00
Chris Lu e71463a9eb mount: invalide file cache when metadata is changed 2020-10-25 19:24:15 -07:00
Chris Lu e219c57849 passing full path when assign volume locations 2020-10-25 15:46:29 -07:00
Chris Lu f375b93aef renaming 2020-10-25 15:32:43 -07:00
Chris Lu 91fd311f7a fix logic to read entry or not 2020-10-25 10:33:51 -07:00
Chris Lu 10a4a628e9 refresh cached file entry from sync metadata updates 2020-10-25 09:42:50 -07:00
Chris Lu da36abf033 go fmt 2020-10-24 20:12:04 -07:00
Chris Lu 79690b9cd8 revert changes causing wrong data 2020-10-24 20:07:19 -07:00
Chris Lu 6c4fb243a7 remove limits to concurrent writers 2020-10-24 02:05:05 -07:00
Chris Lu ef9c32ea0d adjust compression threshold 2020-10-23 23:05:55 -07:00