gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4836 commits 28 branches 278 tags 135 MiB
Commit graph

426 commits

Author SHA1 Message Date
Chris Lu e714c28a02 mount: async write file chunk 2020-10-14 23:28:03 -07:00
Chris Lu 58fa506491 minor 2020-10-13 19:50:22 -07:00
Chris Lu 28d4e1a51b mount: retry for directory listing with filer 
related to https://github.com/chrislusf/seaweedfs/issues/1530
 2020-10-13 19:49:52 -07:00
Chris Lu 9b4f7fed14 mount: report filer IO error 
related to https://github.com/chrislusf/seaweedfs/issues/1530
 2020-10-13 11:21:13 -07:00
Chris Lu b18f21cce1 mount: fix bound tree with filer.path 
fix https://github.com/chrislusf/seaweedfs/issues/1528
 2020-10-12 21:58:37 -07:00
Chris Lu 7704469d29 mount: outsideContainerClusterMode changed to use volume server publicUrl 2020-10-11 20:42:15 -07:00
Chris Lu 723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu cff8bb6554 return proper error 2020-10-10 15:43:22 -07:00
Chris Lu b067853162 mount: ignore recursion error 
fix https://github.com/chrislusf/seaweedfs/issues/1514

the host OS should manage recursive deletion.
 2020-10-08 21:55:55 -07:00
Chris Lu d77e3c96e9 FUSE: use the mount directory permissions 2020-10-03 13:37:35 -07:00
Chris Lu 474e2b6ac3 add a hard link marker to 16byte + maker, for future extensions 2020-09-27 22:38:30 -07:00
Chris Lu 31fc7bb2e1 refactor 
adjust for faster test
 2020-09-27 10:41:29 -07:00
Chris Lu 043b063136
Merge pull request #1482 from hilimd/master 
Fix: s3 delete object
 2020-09-24 18:21:34 -07:00
Chris Lu 7726965a47 minor 2020-09-24 11:20:12 -07:00
Chris Lu 1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu 2e7c361a0d hardlink deletion factors in hardlink counter 2020-09-24 09:43:52 -07:00
Chris Lu 1295347958 adjust hardlink update 
simplify logic, pass entity content directly to hard link. The "weed mount" handles the logic to calculate hard link counter.
 2020-09-24 09:43:00 -07:00
limd cbca14edc5 mount: fix k8s pvc and os mount directory permission bug 2020-09-24 18:07:16 +08:00
Chris Lu 5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
Chris Lu f0e325b6fc mount: auto created directory follow umask 2020-09-23 02:27:57 -07:00
Chris Lu 0adbb56cc1 rename 2020-09-22 16:24:13 -07:00
Chris Lu 4a1fe4b8e2 add error logs 2020-09-22 09:16:10 -07:00
Chris Lu daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu 1d56ea24ef fix 2020-09-03 00:08:37 -07:00
Chris Lu 7e1aad0b54 mount: map uid/gid between local and filer 2020-09-03 00:07:22 -07:00
Chris Lu eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu 408e339c53 also delete the manifest chunk itself 2020-08-31 00:16:03 -07:00
Chris Lu 33d8c6c617 change log to warning 2020-08-30 21:01:44 -07:00
Chris Lu e62d7f221f a bit more logs 2020-08-30 21:00:24 -07:00
Chris Lu 22fe4ae573 mount: fix for UrBackup 2020-08-30 21:00:09 -07:00
Chris Lu ae84a9ee2e add logs 2020-08-30 20:59:04 -07:00
Chris Lu a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu f2a8574448 filer and mount deletion resolves manifest chunks also 2020-08-30 02:07:14 -07:00
Chris Lu 99ecf63276 go fmt 2020-08-29 22:28:33 -07:00
Chris Lu 063c9ddac5 adjust logs 2020-08-29 11:56:22 -07:00
Chris Lu ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu 51346a5930 always keep the manifest list of chunks 2020-08-23 17:15:12 -07:00
Chris Lu c1d1677a28 keep manifest chunks forever 2020-08-23 16:59:01 -07:00
Chris Lu 98175548c2 adjust deletion ordering 2020-08-23 16:27:12 -07:00
Chris Lu aee27ccbe1 multiple fixes 
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
 2020-08-23 15:48:02 -07:00
Chris Lu 77393d3d30 add file handle locking when changing file entry 2020-08-23 00:35:50 -07:00
Chris Lu d60bcbf08a sorting chunks 2020-08-23 00:00:36 -07:00
Chris Lu 5e6b714836 add random test 2020-08-22 16:33:00 -07:00
Chris Lu f48567c5c6 remove unused function 2020-08-19 22:53:49 -07:00
Chris Lu 839634097f also do flush on release 2020-08-19 01:27:10 -07:00
Chris Lu a78772d5ea avoid shutdown in the middle of running 2020-08-19 00:42:02 -07:00
Chris Lu ae9bc4a508 logs 2020-08-18 23:42:26 -07:00
Chris Lu ed4b43b419 adjust logs 2020-08-18 23:42:09 -07:00
Chris Lu 9f1e0aeef5 delete chunks in the last step 2020-08-18 23:40:53 -07:00
Chris Lu 618b2f6829 release resources only when needed to 2020-08-18 12:53:08 -07:00
Chris Lu 208849702d logs 2020-08-18 12:52:54 -07:00
Chris Lu 1fcd083db3 printout data size 2020-08-18 09:09:29 -07:00
Chris Lu be4d42b8e2 rename 2020-08-17 20:15:53 -07:00
Chris Lu abdaf9958d possibly read more 2020-08-17 16:04:56 -07:00
Chris Lu 9d46c7bc78 rename 2020-08-17 11:12:10 -07:00
Chris Lu 4ccfdaeb4d prevent nil 2020-08-17 10:07:34 -07:00
Chris Lu 24c8e6bcb4 minor optimization 2020-08-17 10:03:34 -07:00
Chris Lu f5837b7000 report error first 2020-08-16 23:49:10 -07:00
Chris Lu 2ac27616bc fix possible out of range bytes 
avoid buff out of range resp.Data = buff[:totalRead]
 2020-08-16 23:47:34 -07:00
Chris Lu 22e5132b3a adjust log level 2020-08-16 16:25:11 -07:00
Chris Lu ee0f92a6be reduce memory allocation 2020-08-16 16:24:40 -07:00
Chris Lu 4a77f0820a clean up logs 2020-08-16 01:37:50 -07:00
Chris Lu aec7f32b02 fix reader_at 2020-08-16 00:49:08 -07:00
Chris Lu 5d80fc2ec7 adjust logs 2020-08-15 21:09:31 -07:00
Chris Lu e0bfd3161a update metadata only if changed 2020-08-15 19:56:24 -07:00
Chris Lu 003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu 6ee8d952d2 adjust log level 2020-08-15 18:24:35 -07:00
Chris Lu a22ee30596 fix nil 2020-08-15 17:01:42 -07:00
Chris Lu 0d60e67816 ensure meta data changes are updated 2020-08-15 14:15:07 -07:00
Chris Lu 5b43bddf20 proper deletion ordering 
delete central file store first, then delete local cache
 2020-08-15 09:33:41 -07:00
Chris Lu c647deace1 file size support set file length 
use Attr.FileSize and TotalChunkSize to determine file size
 2020-08-15 09:32:47 -07:00
Chris Lu a7f669044e rename also applies to open file handle 2020-08-14 00:22:49 -07:00
Chris Lu edfa73782f adjust log level 2020-08-14 00:22:21 -07:00
Chris Lu 090612492c fix compilation 2020-08-13 00:07:56 -07:00
Chris Lu d43129d27f add back handles lock 2020-08-12 23:52:13 -07:00
Chris Lu e74dc4e4bc add back fs node cache for renaming 2020-08-09 21:56:09 -07:00
Chris Lu 9832653e1d FUSE mount: proper error with deleting non empty folder 2020-08-08 21:37:36 -07:00
Chris Lu 20e2ac1add filer: store md5 metadata for files uploaded by filer 
fix https://github.com/chrislusf/seaweedfs/issues/1412
 2020-08-06 10:04:17 -07:00
Chris Lu eed525b717 FUSE mount: remove DirListCacheLimit 
outdated parameter
 2020-07-28 09:24:39 -07:00
Chris Lu d40de39e75 FUSE: do not change crtime, uid, gid on save 2020-07-24 10:13:45 -07:00
Chris Lu 8dfeba8023 FUSE: skip changing to empty uid and gid during flush 
Skip uid and gid changes if request uid and gid are zero

mongodb lock file, and many interim files, has empty uid and gid

I0724 10:05:51 93643 filehandle.go:173] /db/diagnostic.data/metrics.interim fh 1333342842031408359 flush Flush [ID=0x3 Node=0x50 Uid=0 Gid=0 Pid=178] 0x10 fl=0x0 lk=0x0
 2020-07-24 10:06:43 -07:00
Chris Lu dd29b8c81d fix compilation 2020-07-24 09:36:05 -07:00
Chris Lu 5788f1f6bf remove fsnode cache 
fix https://github.com/chrislusf/seaweedfs/issues/1400
 2020-07-24 09:25:54 -07:00
Chris Lu 6962ca48d3 fix compilation 2020-07-23 21:31:23 -07:00
Chris Lu df7eee523b FUSE: implement dir fsync 
fix https://github.com/chrislusf/seaweedfs/issues/1398
 2020-07-23 21:09:40 -07:00
Chris Lu 3bbac0820b FUSE: faster rename implementation 2020-07-23 21:08:42 -07:00
Chris Lu 60d14a9800 mount: fix difference with storage format in local cache 2020-07-20 00:17:50 -07:00
Chris Lu 97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
Chris Lu ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu 2d1df97933 mount: fatal error if can not process meta data 2020-07-11 06:16:48 -07:00
Chris Lu e3e00d36c5 reset cache for each new release 2020-07-11 06:16:17 -07:00
Chris Lu 7ad0bce2f5 remove old code 2020-07-10 23:03:22 -07:00
Chris Lu 54d6b3c30f filer: support running multiple mount to different filers avoiding duplicated cache 
fix https://github.com/chrislusf/seaweedfs/issues/1382
 2020-07-07 23:18:44 -07:00
Chris Lu ce3630e7a4 refactoring 2020-07-03 22:16:59 -07:00
Chris Lu 31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu 42a338d7b0 FUSE mount: clean up file handles during renaming 
resolve dir rename when file is still open. Need to clean the file handles as soon as possible.

These can happen out of order:
file rename, then file release
file release, then file rename
 2020-06-28 13:41:00 -07:00
Chris Lu 95a1860d65 remove empty calls for list cache 2020-06-28 10:25:54 -07:00
Chris Lu 6498d0c86b remove list cache 2020-06-28 10:21:52 -07:00
Chris Lu 9033a7d369 removed async option 2020-06-28 10:18:32 -07:00