gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9903 commits 28 branches 278 tags 135 MiB
Commit graph

222 commits

Author SHA1 Message Date
Ryan Russell 72d8a9f9a8
refactor(exclusive_locker): Interval readability batch of updates (#3668) 
* refactor(filechunk_manifest): `localProcesed` -> `localProcessed`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `saveChunkedFileIntevalToStorage` -> `saveChunkedFileIntervalToStorage`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `SafeRenewInteval` -> `SafeRenewInterval`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `InitLockInteval` -> `InitLockInterval`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `RenewInteval` -> `RenewInterval`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-09-14 10:29:55 -07:00
chrislu 22064c3425 mount: ensure ordered file handle lock and unlock 2022-09-11 19:44:34 -07:00
VictorSFF 10d545060f
mount: fix new inode allocation (#3633) 2022-09-09 06:32:19 -07:00
Patrick Schmidt a73e177ecf
Add an End-to-End workflow for FUSE mount (#3562) 
* Add an e2e workflow to test FUSE mount

* Fix deadlocks during concurrent r/w
 2022-08-31 09:27:53 -07:00
Patrick Schmidt 00041f6e08
Fix corrupted mount reads after changes on fh (#3536) 2022-08-29 08:27:13 -07:00
chrislu 301b49b63f atomic operation 2022-08-26 17:04:11 -07:00
Patrick Schmidt 5df105b1f9
Fix a few data races when reading files in mount (#3527) 2022-08-26 16:41:37 -07:00
askeipx 2e78a522ab
remove old raft servers if they don't answer to pings for too long (#3398) 
* remove old raft servers if they don't answer to pings for too long

add ping durations as options

rename ping fields

fix some todos

get masters through masterclient

raft remove server from leader

use raft servers to ping them

CheckMastersAlive for hashicorp raft only

* prepare blocking ping

* pass waitForReady as param

* pass waitForReady through all functions

* waitForReady works

* refactor

* remove unneeded params

* rollback unneeded changes

* fix
 2022-08-23 23:18:21 -07:00
Patrick Schmidt cda8cc22bc
Implement lseek syscall in FUSE (#3491) 
See the man page of lseek:
https://man7.org/linux/man-pages/man2/lseek.2.html
 2022-08-23 01:47:18 -07:00
Patrick Schmidt f875031f06
Reuse readDataByFileHandle in Read call (#3482) 2022-08-22 08:24:06 -07:00
chrislu 8bac903d29 mount: add locking to prevent nil pointer 
fix https://github.com/seaweedfs/seaweedfs/issues/3479
 2022-08-21 23:26:06 -07:00
chrislu 7807f6641a minor 2022-08-21 22:54:56 -07:00
chrislu 63fbf281c7 remove same file copying rage limitation 2022-08-21 17:16:21 -07:00
Patrick Schmidt 3a75d7f7aa
Implement copy_file_range syscall in FUSE (#3475) 
See the man page of copy_file_range:
https://man7.org/linux/man-pages/man2/copy_file_range.2.html
 2022-08-21 17:11:43 -07:00
chrislu 8b3429858d align memory for atomic read/write 2022-08-21 12:20:27 -07:00
Patrick Schmidt 2ef6ab998c
Avoid race conditions with current filer address (#3474) 
When multiple filer requests are in-flight and the current filer
disappears and a new one is selected by the first goroutine, then
there can be a lot of race conditions while retrieving the current
filer.
Therefore, load/save the current filer index atomically.
 2022-08-21 12:18:13 -07:00
chrislu 409f39390d mount: retryable data chunk uploading 2022-08-20 19:04:36 -07:00
chrislu 689b4ecdcc remove unused collection and replication from upload result 2022-08-20 18:14:57 -07:00
chrislu ae93c966d9 ensure memory is aligned 
fix https://github.com/seaweedfs/seaweedfs/issues/3427
 2022-08-10 22:27:13 -07:00
chrislu 662ec97602 mount: adjust on du stats reporting 
fix https://github.com/seaweedfs/seaweedfs/issues/3424
 2022-08-09 20:21:01 -07:00
chrislu d8e69df295 adjust comments 2022-08-07 11:35:33 -07:00
chrislu 0aeec04c31 quicker to adapt to pattern change 2022-08-07 10:14:01 -07:00
chrislu 8a880a139d mount: fix truncate operation 
fix https://github.com/seaweedfs/seaweedfs/issues/2609
 2022-08-06 13:58:45 -07:00
chrislu b278bb24d3 mount: adjust df stats reporting when close to the limit 
fix https://github.com/seaweedfs/seaweedfs/issues/3407
 2022-08-05 00:31:41 -07:00
Konstantin Lebedev 4d08393b7c
filer prefer volume server in same data center (#3405) 
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404

* GetDataCenter

* prefer same data center for ReplicationSource

* GetDataCenterId

* remove glog
 2022-08-04 17:35:00 -07:00
chrislu bd13a7968f remove unused symlink resolving 2022-08-04 01:35:18 -07:00
chrislu 88945d9954 mount: do not follow soft link with xattr 2022-08-04 01:21:15 -07:00
chrislu 303bd067b5 Revert "rename: delete source entry metadata only, skipping hard links" 
This reverts commit 03466f955e.

fix https://github.com/seaweedfs/seaweedfs/issues/3386
 2022-07-31 22:51:41 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 89e276de64 fix loop 2022-07-28 18:06:49 -07:00
chrislu e0eda52c54 mount: ensure symlink parent directory is tracked 
fix https://github.com/chrislusf/seaweedfs/issues/3373
 2022-07-28 16:32:00 -07:00
chrislu f401b996eb mount: fix symlink size reporting 2022-07-26 23:12:25 -07:00
chrislu b98c0b4bf4 disable symlink translation for now 2022-07-24 18:46:32 -07:00
chrislu 2dc5d4adac disable symlink translation for now 2022-07-24 18:24:34 -07:00
chrislu 2b64916d3e set status to fuse.OK 2022-07-24 15:30:55 -07:00
chrislu 7386cf6a06 mount: resolve symlink 2022-07-24 15:11:24 -07:00
chrislu 40089b6753 maintain node lookup counter 2022-07-24 13:43:11 -07:00
chrislu d24db396cc fix removing path from inode2path 2022-07-23 18:20:29 -07:00
chrislu 41eeb4deef do not add new inode during link 2022-07-23 13:37:07 -07:00
chrislu 02c2d81cde fix removePathFromInode2Path 2022-07-23 13:21:42 -07:00
chrislu 4d5dc55766 mount: better support hard links 2022-07-23 12:57:00 -07:00
chrislu ebda0689ab use linux default max name length 2022-07-23 10:51:06 -07:00
chrislu 64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
chrislu 2c8818351f mount: comment out stuck fuse inode notify during renaming 2022-07-22 18:17:34 -07:00
chrislu a5c94acd4f mount: check possible nil entry case 2022-07-22 15:33:37 -07:00
chrislu 6147b61b0a mount: fix concurrent map read and map write 
fix https://github.com/chrislusf/seaweedfs/issues/3344
 2022-07-21 01:38:26 -07:00
chrislu 289402a741 writer pattern: similar changes to reader pattern 2022-07-13 02:30:44 -07:00
chrislu 578c8aea0e typo 2022-07-13 02:18:31 -07:00
chrislu 3d0defa663 fix compilation 2022-07-08 19:16:58 -07:00
chrislu 6f00c7f061 comments 2022-07-08 19:07:31 -07:00
chrislu d685b94102 mount: rename also invalidate source inode 2022-07-08 19:07:03 -07:00
chrislu e2ecf137a2 fix resetting fh.reader 2022-07-08 01:04:15 -07:00
chrislu 05c53820b9 mount: file handler release memory 2022-07-07 11:49:40 -07:00
chrislu 6b3beb709d fix format 2022-06-21 12:21:02 -07:00
chrislu 69a2e503ab non-recursive directory deletion 
fix a problem during git checkout a different version, where updated directories are deleted even though the directory is not empty.
 2022-06-20 13:18:08 -07:00
chrislu 76f26c4b99 mount: delete target entry first during move 2022-06-20 13:15:02 -07:00
chrislu 4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata 
these metadata can change and are not used
 2022-06-06 00:39:35 -07:00
ningfd f32142f6f5 add disableXAttr in mount option 2022-06-06 14:09:01 +08:00
chrislu d65bb2c6df mount: file handle locks entry better 
related to https://github.com/chrislusf/seaweedfs/issues/2952
 2022-06-05 18:15:06 -07:00
chrislu 746092a60b add a comment 2022-06-05 16:44:07 -07:00
creeew 02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
chrislu a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu 5b8b022985 remove unused parameter 2022-05-24 18:52:04 -07:00
Robert Coelho 0e6e72d462 mount: ReadDir return EIO on EnsureVisited err 2022-05-24 13:02:59 -04:00
Robert Coelho 1fabbe8a25 mount: cleanup ReadDir rewrite's branches to not assume offsets 2022-05-24 12:48:32 -04:00
Robert Coelho cb422d96f7 mount: rewrite ReadDir to respect input.Offset to fix partial results 2022-05-24 12:16:58 -04:00
Robert Coelho 05602167f4 mount: do not mark meta_cache when there is an err 2022-05-24 12:13:15 -04:00
leyou240 89eb87c1d1
Merge branch 'master' into slices.SortFunc 2022-04-18 10:39:29 +08:00
justin 3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
chrislu b597baf488 mount: set file write time correct to work well with rsync 
fix https://github.com/chrislusf/seaweedfs/issues/2867
 2022-04-16 22:56:45 -07:00
chrislu 261ba0208e lock-protected setting entry 2022-04-09 22:52:59 -07:00
chrislu 3ab2c0e5c0 avoid possible nil attributes 2022-04-06 10:11:11 -07:00
chrislu 9d44b89c21 Revert "avoid possible nil attributes" 
This reverts commit 0a6703c7f7.
 2022-04-06 10:01:45 -07:00
chrislu 0a6703c7f7 avoid possible nil attributes 2022-04-06 09:58:48 -07:00
chrislu 8b3d76b24d prevent concurrent access during shutdown 2022-04-05 10:53:36 -07:00
chrislu e8d7bb42e2 avoid concurrent access to map object 
fix https://github.com/chrislusf/seaweedfs/issues/2866
 2022-04-03 01:08:25 -07:00
chrislu 6a2bcd03aa configure mount quota 2022-04-02 21:34:26 -07:00
chrislu 958f880b70 mount: add grpc method to adjust quota 2022-04-02 15:14:37 -07:00
chrislu 7542fd7f4d mount: optimize a bit when writing files 2022-03-17 00:02:38 -07:00
chrislu bd5c5586b5 generate inode via path and time 2022-03-14 00:03:29 -07:00
chrislu 5cba8e51c5 refactor 2022-03-13 18:34:57 -07:00
chrislu f2f68f675e write to disk during random writes, limiting total disk spaces used 2022-03-13 18:17:35 -07:00
chrislu f70c1e449b add useful doc link 2022-03-13 00:14:50 -08:00
chrislu b20ddc57a7 mount: return open status 2022-03-12 22:38:14 -08:00
chrislu b40d252761 mount: chmod for root 2022-03-12 12:10:56 -08:00
chrislu 011a41b561 add back writes to swap file when too many in memory chunks are used. 2022-03-09 22:26:51 -08:00
chrislu dc204dd137 fix nil entry 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x1d340b4]

goroutine 130523 [running]:
github.com/chrislusf/seaweedfs/weed/filer.FileSize(...)
        /code/seaweedfs/weed/filer/filechunks.go:26
github.com/chrislusf/seaweedfs/weed/mount.(*WFS).Lookup(0xc000866d80, 0x1, 0xc002897f40, {0xc004b00980, 0x39}, 0x1ec19e0)
        /code/seaweedfs/weed/mount/weedfs_dir_lookup.go:59 +0x654
github.com/hanwen/go-fuse/v2/fuse.doLookup(0xc00033c000, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/opcode.go:333 +0x6b
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000ab2420, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:483 +0x1f3
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000ab2420, 0x0)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:456 +0x110
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:323 +0x534
 2022-03-08 23:02:30 -08:00
chrislu 3aeee3d748 ensure releasing file handle 2022-03-07 14:01:24 -08:00
chrislu 8136384473 remove debug message 2022-03-07 11:22:26 -08:00
chrislu 46a28b8819 mount: adjust disk space based on quota 2022-03-06 17:22:49 -08:00
chrislu f3442e36e6 mount: quota adjust error type to be syscall.ENOSPC 2022-03-06 17:04:21 -08:00
chrislu c7e8ac18f0 mount: quota for one mounted collection 
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
 2022-03-06 02:44:40 -08:00
chrislu b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu 28b8974a3a mount: fix directory pagination when using midnight commander 2022-03-03 02:59:31 -08:00
chrislu fcf3714443 mount: add back support for filer.path 2022-02-28 12:16:53 -08:00
chrislu aad62ee148 mount: mark directory uncached if forgotten 2022-02-28 02:08:24 -08:00
chrislu 554e239097 release file handle 2022-02-28 01:23:14 -08:00
chrislu 3639fedd01 mount: fix fsync opened and renamed files 2022-02-28 00:34:17 -08:00
chrislu 63a9d8f01d ensure inodes are not duplicating unless hardlinked 2022-02-27 23:13:49 -08:00
chrislu de77d00c81 correctly clean up for a file 2022-02-27 23:12:28 -08:00