gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
5421 commits 28 branches 278 tags 135 MiB
Commit graph

470 commits

Author SHA1 Message Date
Chris Lu f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu 1ae108efca add debug message 2020-11-26 10:41:10 -08:00
Chris Lu 0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu 06ee199ef3 ensure entryViewCache is in sync with the chunks 2020-11-03 12:22:19 -08:00
Chris Lu b132cb893f in case some chunks modified earliar are added later 2020-11-03 12:22:19 -08:00
Chris Lu 0ae4d7bfdf lock "addChunks()" 2020-11-03 12:22:19 -08:00
Chris Lu 7d5db73d98 go fmt 2020-11-02 14:21:07 -08:00
Chris Lu a9662d44ac avoid closing chunkSaveErrChan too early 2020-11-02 14:20:38 -08:00
Chris Lu e1423a83dd mount: clear possible leftover entry data in memory 2020-11-01 23:28:56 -08:00
Chris Lu 8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu 4609644658 mount: remove files that may still are open 
related to https://github.com/chrislusf/seaweedfs/issues/1581
 2020-11-01 01:33:26 -08:00
Chris Lu df8d976bb0 refactoring 2020-11-01 01:58:48 -07:00
Chris Lu f7b21973ac mount: add retry for all operations with filer 
fix https://github.com/chrislusf/seaweedfs/issues/1589
 2020-11-01 01:11:08 -07:00
Chris Lu 10f9f1cc26 handle mknod requests 2020-10-30 23:51:32 -07:00
Chris Lu 8826601be1 mount: optional limit for the number of concurrent writers 2020-10-30 21:22:20 -07:00
Chris Lu 19098f2c2d make a copy of the filer.entry for most read operations 2020-10-30 01:23:26 -07:00
Chris Lu 6135cbaa02 clear file.entry 2020-10-30 01:23:26 -07:00
Chris Lu 02dc51b1ed properly clear out the file.entry 2020-10-29 23:31:06 -07:00
Chris Lu 0b68b68ec4 reload entry only when it is a hard link 
fix https://github.com/chrislusf/seaweedfs/issues/1581
 2020-10-29 16:32:45 -07:00
Chris Lu e71463a9eb mount: invalide file cache when metadata is changed 2020-10-25 19:24:15 -07:00
Chris Lu e219c57849 passing full path when assign volume locations 2020-10-25 15:46:29 -07:00
Chris Lu f375b93aef renaming 2020-10-25 15:32:43 -07:00
Chris Lu 91fd311f7a fix logic to read entry or not 2020-10-25 10:33:51 -07:00
Chris Lu 10a4a628e9 refresh cached file entry from sync metadata updates 2020-10-25 09:42:50 -07:00
Chris Lu da36abf033 go fmt 2020-10-24 20:12:04 -07:00
Chris Lu 79690b9cd8 revert changes causing wrong data 2020-10-24 20:07:19 -07:00
Chris Lu 6c4fb243a7 remove limits to concurrent writers 2020-10-24 02:05:05 -07:00
Chris Lu ef9c32ea0d adjust compression threshold 2020-10-23 23:05:55 -07:00
Chris Lu 71624858af avoid reusing cached []byte 
this logic is just to ensure 100% correct.

fuse message, data+message hearder > 512 will not be cached.
 2020-10-23 11:31:57 -07:00
Chris Lu f5ee19e0db shortcut for appending operations 2020-10-21 20:44:01 -07:00
Chris Lu b5917bf846 clean up 2020-10-21 19:31:26 -07:00
Chris Lu 9884dfc369 use bytebuffer for interval list 2020-10-21 19:29:51 -07:00
Chris Lu 81cf8d04df minor 2020-10-21 02:42:23 -07:00
Chris Lu d90e7ee3cc adjust log level 2020-10-21 02:17:40 -07:00
Chris Lu 3bf0116de1 mount: less channel waiting 2020-10-21 02:16:21 -07:00
Chris Lu c31b254248 mount: shortcut when there is only one chunk 2020-10-20 23:48:29 -07:00
Chris Lu ad652ffb97 avoid close closed channel 2020-10-20 22:54:34 -07:00
Chris Lu 20442dc1bc avoid extra data copying 2020-10-20 22:54:21 -07:00
Chris Lu 69f0da5d73 mount: pass along replication and collection parameters 2020-10-19 00:12:42 -07:00
Chris Lu c0ab458671 report nil instead of EOF for empty files 
related to https://github.com/chrislusf/seaweedfs/issues/1541
 2020-10-17 11:03:46 -07:00
Chris Lu 09bab17aff mount: avoid "send on closed channel" 2020-10-16 12:54:23 -07:00
Chris Lu 06e6341097 set initial write time to avoid race condition with delayed chunks 2020-10-15 11:08:45 -07:00
Chris Lu d598c47136 log errors 2020-10-15 11:01:43 -07:00
Chris Lu 5039aff310 release the goroutine to read errors 2020-10-15 10:59:18 -07:00
Chris Lu e714c28a02 mount: async write file chunk 2020-10-14 23:28:03 -07:00
Chris Lu 58fa506491 minor 2020-10-13 19:50:22 -07:00
Chris Lu 28d4e1a51b mount: retry for directory listing with filer 
related to https://github.com/chrislusf/seaweedfs/issues/1530
 2020-10-13 19:49:52 -07:00
Chris Lu 9b4f7fed14 mount: report filer IO error 
related to https://github.com/chrislusf/seaweedfs/issues/1530
 2020-10-13 11:21:13 -07:00
Chris Lu b18f21cce1 mount: fix bound tree with filer.path 
fix https://github.com/chrislusf/seaweedfs/issues/1528
 2020-10-12 21:58:37 -07:00
Chris Lu 7704469d29 mount: outsideContainerClusterMode changed to use volume server publicUrl 2020-10-11 20:42:15 -07:00
Chris Lu 723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu cff8bb6554 return proper error 2020-10-10 15:43:22 -07:00
Chris Lu b067853162 mount: ignore recursion error 
fix https://github.com/chrislusf/seaweedfs/issues/1514

the host OS should manage recursive deletion.
 2020-10-08 21:55:55 -07:00
Chris Lu d77e3c96e9 FUSE: use the mount directory permissions 2020-10-03 13:37:35 -07:00
Chris Lu 474e2b6ac3 add a hard link marker to 16byte + maker, for future extensions 2020-09-27 22:38:30 -07:00
Chris Lu 31fc7bb2e1 refactor 
adjust for faster test
 2020-09-27 10:41:29 -07:00
Chris Lu 043b063136
Merge pull request #1482 from hilimd/master 
Fix: s3 delete object
 2020-09-24 18:21:34 -07:00
Chris Lu 7726965a47 minor 2020-09-24 11:20:12 -07:00
Chris Lu 1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu 2e7c361a0d hardlink deletion factors in hardlink counter 2020-09-24 09:43:52 -07:00
Chris Lu 1295347958 adjust hardlink update 
simplify logic, pass entity content directly to hard link. The "weed mount" handles the logic to calculate hard link counter.
 2020-09-24 09:43:00 -07:00
limd cbca14edc5 mount: fix k8s pvc and os mount directory permission bug 2020-09-24 18:07:16 +08:00
Chris Lu 5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
Chris Lu f0e325b6fc mount: auto created directory follow umask 2020-09-23 02:27:57 -07:00
Chris Lu 0adbb56cc1 rename 2020-09-22 16:24:13 -07:00
Chris Lu 4a1fe4b8e2 add error logs 2020-09-22 09:16:10 -07:00
Chris Lu daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu 1d56ea24ef fix 2020-09-03 00:08:37 -07:00
Chris Lu 7e1aad0b54 mount: map uid/gid between local and filer 2020-09-03 00:07:22 -07:00
Chris Lu eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu 408e339c53 also delete the manifest chunk itself 2020-08-31 00:16:03 -07:00
Chris Lu 33d8c6c617 change log to warning 2020-08-30 21:01:44 -07:00
Chris Lu e62d7f221f a bit more logs 2020-08-30 21:00:24 -07:00
Chris Lu 22fe4ae573 mount: fix for UrBackup 2020-08-30 21:00:09 -07:00
Chris Lu ae84a9ee2e add logs 2020-08-30 20:59:04 -07:00
Chris Lu a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu f2a8574448 filer and mount deletion resolves manifest chunks also 2020-08-30 02:07:14 -07:00
Chris Lu 99ecf63276 go fmt 2020-08-29 22:28:33 -07:00
Chris Lu 063c9ddac5 adjust logs 2020-08-29 11:56:22 -07:00
Chris Lu ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu 51346a5930 always keep the manifest list of chunks 2020-08-23 17:15:12 -07:00
Chris Lu c1d1677a28 keep manifest chunks forever 2020-08-23 16:59:01 -07:00
Chris Lu 98175548c2 adjust deletion ordering 2020-08-23 16:27:12 -07:00
Chris Lu aee27ccbe1 multiple fixes 
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
 2020-08-23 15:48:02 -07:00
Chris Lu 77393d3d30 add file handle locking when changing file entry 2020-08-23 00:35:50 -07:00
Chris Lu d60bcbf08a sorting chunks 2020-08-23 00:00:36 -07:00
Chris Lu 5e6b714836 add random test 2020-08-22 16:33:00 -07:00
Chris Lu f48567c5c6 remove unused function 2020-08-19 22:53:49 -07:00
Chris Lu 839634097f also do flush on release 2020-08-19 01:27:10 -07:00
Chris Lu a78772d5ea avoid shutdown in the middle of running 2020-08-19 00:42:02 -07:00
Chris Lu ae9bc4a508 logs 2020-08-18 23:42:26 -07:00
Chris Lu ed4b43b419 adjust logs 2020-08-18 23:42:09 -07:00
Chris Lu 9f1e0aeef5 delete chunks in the last step 2020-08-18 23:40:53 -07:00
Chris Lu 618b2f6829 release resources only when needed to 2020-08-18 12:53:08 -07:00
Chris Lu 208849702d logs 2020-08-18 12:52:54 -07:00
Chris Lu 1fcd083db3 printout data size 2020-08-18 09:09:29 -07:00
Chris Lu be4d42b8e2 rename 2020-08-17 20:15:53 -07:00
Chris Lu abdaf9958d possibly read more 2020-08-17 16:04:56 -07:00
Chris Lu 9d46c7bc78 rename 2020-08-17 11:12:10 -07:00