gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
7326 commits 28 branches 278 tags 135 MiB
Commit graph

4887 commits

Author SHA1 Message Date
chrislu 011a41b561 add back writes to swap file when too many in memory chunks are used. 2022-03-09 22:26:51 -08:00
zzq09494 2ea18fca48 fix:support some databases which not support 'IF NOT EXISTS' keyword of 'create table command' 2022-03-10 13:37:34 +08:00
banjiaojuhao f28dbbe5c5 [bugfix] filer: 1. Delete uploaded chunks when upload failed. 2. Report error when upload is interrupted by user. 2022-03-10 11:40:39 +08:00
chrislu e2b07737da Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-08 23:02:33 -08:00
chrislu dc204dd137 fix nil entry 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x1d340b4]

goroutine 130523 [running]:
github.com/chrislusf/seaweedfs/weed/filer.FileSize(...)
        /code/seaweedfs/weed/filer/filechunks.go:26
github.com/chrislusf/seaweedfs/weed/mount.(*WFS).Lookup(0xc000866d80, 0x1, 0xc002897f40, {0xc004b00980, 0x39}, 0x1ec19e0)
        /code/seaweedfs/weed/mount/weedfs_dir_lookup.go:59 +0x654
github.com/hanwen/go-fuse/v2/fuse.doLookup(0xc00033c000, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/opcode.go:333 +0x6b
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000ab2420, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:483 +0x1f3
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000ab2420, 0x0)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:456 +0x110
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:323 +0x534
 2022-03-08 23:02:30 -08:00
banjiaojuhao f7f2a597dd minor 2022-03-08 16:22:55 +08:00
banjiaojuhao d61bea9038 [bugfix] filer: In file modification, old chunks will be mis-deleted when they are merged(Manifestized). 2022-03-08 16:22:55 +08:00
chrislu 3aeee3d748 ensure releasing file handle 2022-03-07 14:01:24 -08:00
chrislu 8136384473 remove debug message 2022-03-07 11:22:26 -08:00
banjiaojuhao b9ff7723dd [bugfix] filer: nil pointer dereference 2022-03-07 23:26:25 +08:00
chrislu da3d330616 s3 and filer transport using unix domain socket instead of tcp 2022-03-07 02:00:14 -08:00
chrislu 0cb17b45b1 refactoring 2022-03-07 01:59:01 -08:00
chrislu 6d3db4445b buffer for all range requests 2022-03-07 01:56:47 -08:00
chrislu f3bcbeb60a a little optimization 2022-03-07 00:24:59 -08:00
chrislu f7ee60996c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-07 00:08:09 -08:00
chrislu 6f3ec989a7 fix manifest batch size 2022-03-07 00:07:53 -08:00
Chris Lu 1e7fcef581
Merge pull request #2729 from banjiaojuhao/filer_metadata-resolve-manifest 
filer: support get metadata with resolved manifest chunk
 2022-03-07 00:05:46 -08:00
chrislu bb0b784544 minor 2022-03-07 00:04:59 -08:00
banjiaojuhao bfcc9ca808 filer: support metadata with resolved manifest chunk 2022-03-07 15:47:51 +08:00
banjiaojuhao 71f3046841 filer: add back isAppend function 2022-03-07 15:41:07 +08:00
chrislu 0ba4e4cd23 2.93 2022-03-06 18:54:12 -08:00
chrislu 46a28b8819 mount: adjust disk space based on quota 2022-03-06 17:22:49 -08:00
chrislu ede6ce44c6 fix test 2022-03-06 17:09:55 -08:00
chrislu 21ef152423 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-06 17:04:24 -08:00
chrislu f3442e36e6 mount: quota adjust error type to be syscall.ENOSPC 2022-03-06 17:04:21 -08:00
Chris Lu 89cd0c27a8
Merge pull request #2728 from kmlebedev/set_default_leveldb2 2022-03-06 07:26:30 -08:00
Konstantin Lebedev cf444ebd07 Set default leveldb2 enabled 
avoid Filer store is enabled for both leveldb2 and mysql
 2022-03-06 18:27:25 +05:00
banjiaojuhao a07c93f4dd filer: reset bytesBuffer before use 2022-03-06 21:07:36 +08:00
chrislu c7e8ac18f0 mount: quota for one mounted collection 
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
 2022-03-06 02:44:40 -08:00
chrislu b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu f1713c96ae avoid possible runtime error: index out of range [0] with length 0 2022-03-05 21:14:31 -08:00
chrislu da76af187f mount: avoid possible index out of bounds error 2022-03-04 22:36:01 -08:00
chrislu 6e49e75a5b use logical number of files and sizes for statistics and quota 2022-03-04 18:47:44 -08:00
chrislu f51e20028a mount: avoid comma in mount options 
fix https://github.com/chrislusf/seaweedfs/issues/2719
 2022-03-03 03:42:29 -08:00
chrislu dc0f48682c unused 2022-03-03 03:41:35 -08:00
chrislu 28b8974a3a mount: fix directory pagination when using midnight commander 2022-03-03 02:59:31 -08:00
chrislu a96d4254e9 filer, s3, volume server: a bit memory optimization 2022-03-02 20:15:28 -08:00
chrislu 6fbbc78574 stream reading a whole chunk 2022-03-02 13:50:46 -08:00
chrislu 784583afc6 avoid pool memory allocation if too large 2022-03-02 13:50:28 -08:00
chrislu ba14307319 2.92 2022-02-28 15:22:19 -08:00
chrislu fcf3714443 mount: add back support for filer.path 2022-02-28 12:16:53 -08:00
chrislu 80c017907b filer.backup: fix backing up encrypted chunks 
I have done filer.backup test:
replication.toml:
[sink.local]
enabled = true
directory = "/srv/test"
___
system@dat1:/srv/test$ weed filer.backup -filer=app1:8888 -filerProxy
I0228 12:39:28 19571 filer_replication.go:129] Configure sink to local
I0228 12:39:28 19571 filer_backup.go:98] resuming from 2022-02-28 12:04:20.210984693 +0100 CET
I0228 12:39:29 19571 filer_backup.go:113] backup app1:8888 progressed to 2022-02-28 12:04:20.211726749 +0100 CET 0.33/sec

system@dat1:/srv/test$ ls -l
total 16
drwxr-xr-x 2 system system 4096 Feb 28 12:39 a
-rw-r--r-- 1 system system   48 Feb 28 12:39 fu.txt
-rw-r--r-- 1 system system   32 Feb 28 12:39 _index.html
-rw-r--r-- 1 system system   68 Feb 28 12:39 index.php
system@dat1:/srv/test$ cat fu.txt
?	?=?^??`?f^};?{4?Z%?X0=??rV????|"?1??踪~??
system@dat1:/srv/test$
On the active mount on the target server it's:
system@app1:/srv/app$ ls -l
total 2
drwxrwxr-x 1 system system  0 Feb 28 12:04 a
-rw-r--r-- 1 system system 20 Feb 28 12:04 fu.txt
-rw-r--r-- 1 system system  4 Feb 28 12:04 _index.html
-rw-r--r-- 1 system system 40 Feb 28 12:04 index.php
system@app1:/srv/app$ cat fu.txt
This is static boy!
Filer was started with: weed filer master="app1:9333,app2:9333,app3:9333" -encryptVolumeData
It seems like it's still encrypted?
 2022-02-28 10:07:06 -08:00
chrislu aad62ee148 mount: mark directory uncached if forgotten 2022-02-28 02:08:24 -08:00
chrislu 554e239097 release file handle 2022-02-28 01:23:14 -08:00
chrislu 3639fedd01 mount: fix fsync opened and renamed files 2022-02-28 00:34:17 -08:00
chrislu 63a9d8f01d ensure inodes are not duplicating unless hardlinked 2022-02-27 23:13:49 -08:00
chrislu de77d00c81 correctly clean up for a file 2022-02-27 23:12:28 -08:00
chrislu 18543c6e8b minor 2022-02-27 23:11:09 -08:00
chrislu 09cd00f356 2.91 2022-02-27 04:03:39 -08:00
chrislu 941ced60a4 download 2 chunks if at the beginning of a file 2022-02-27 03:57:24 -08:00
chrislu f9d9eed0c9 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-27 03:41:35 -08:00
chrislu d602d68fd1 remove dead code 2022-02-27 03:41:32 -08:00
Chris Lu 6c07af6014
Merge pull request #2715 from gfx-labs/acl-pr 2022-02-27 03:27:32 -08:00
chrislu c3792c8352 remove dead code 2022-02-27 03:03:19 -08:00
chrislu aa9eef81e6 retire mount v1 2022-02-27 02:57:27 -08:00
eddy-gfx fb940dd807
Merge branch 'chrislusf:master' into acl-pr 2022-02-27 04:52:08 -06:00
elee 881a0fe806 ensure compatibility 2022-02-27 04:50:59 -06:00
elee 954ad98e0d set canned acl on replication create 2022-02-27 04:49:31 -06:00
chrislu 6e1ab97988 use debug option to see operations 2022-02-27 02:02:30 -08:00
chrislu 2112d99140 mount2: add back readonly mode 2022-02-27 01:13:32 -08:00
chrislu 9ef5bb20f6 mount2: invalidate fuse cache for replaced inode 2022-02-27 00:00:23 -08:00
chrislu 4ddcbaab57 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-26 23:20:47 -08:00
chrislu 551d00d51a prefetch other chunks when stream reading 2022-02-26 23:20:45 -08:00
Chris Lu e1d3dd5e18
Merge pull request #2712 from guo-sj/correct_comments 2022-02-26 09:16:12 -08:00
guosj 82cad5e330
correct comments 2022-02-26 22:44:26 +08:00
chrislu 7b1a713d2a remove dead code 2022-02-26 03:23:15 -08:00
chrislu 3a58b7bac6 a little safer 2022-02-26 03:23:06 -08:00
chrislu 708e14fcfa avoid possible too big memory allocation 2022-02-26 03:22:41 -08:00
chrislu 3345a50d9b prefetch 2 chunks 2022-02-26 03:06:17 -08:00
chrislu 86ce69f709 remove logs 2022-02-26 03:00:20 -08:00
chrislu b2a148cb4c use file size as max range 2022-02-26 03:00:08 -08:00
chrislu 2ab0ad24a3 use memory pool 2022-02-26 02:59:19 -08:00
chrislu 28b395bef4 better control for reader caching 2022-02-26 02:16:47 -08:00
chrislu 3ad5fa6f6f chunk cache adds function ReadChunkAt 2022-02-25 21:55:04 -08:00
chrislu fc7a4957ea fix mount2 options 2022-02-25 21:22:44 -08:00
chrislu 72c0233938 less logs 2022-02-25 15:34:24 -08:00
chrislu 101e6d80b0 mount2: listXattr return ok if xattr is empty 2022-02-25 14:38:56 -08:00
chrislu 03466f955e rename: delete source entry metadata only, skipping hard links 2022-02-25 02:57:54 -08:00
chrislu 8080fe4cc1 logs 2022-02-25 02:56:23 -08:00
chrislu e423548673 rename: pass along entry metadata 2022-02-25 02:53:37 -08:00
chrislu e8110bb54c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-25 01:17:28 -08:00
chrislu 202a29d014 refactoring 2022-02-25 01:17:26 -08:00
chrislu 91d6785cf3 define metadata action types 2022-02-25 00:54:16 -08:00
chrislu be3fc77391 mount2: use consistent inode 2022-02-25 00:53:27 -08:00
Chris Lu 9873bae115
Merge pull request #2706 from guo-sj/fix_log_info 2022-02-24 22:48:19 -08:00
guosj 121b31f750 fix incorrect log information 2022-02-25 13:41:20 +08:00
chrislu ceaf993a27 mount2: add rdev 2022-02-24 14:51:25 -08:00
chrislu 95717d1006 simpler output 2022-02-24 13:50:08 -08:00
chrislu b05962b90e rename: handle hard links 2022-02-24 02:59:00 -08:00
chrislu e31ec04f4f mount2: POSIX deleted opened file nlink should be 0 2022-02-24 01:59:37 -08:00
chrislu 419e355e9e Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-24 01:31:07 -08:00
chrislu 91f0481f4e mount2: SetAttr set mode correctly 2022-02-24 01:31:04 -08:00
Chris Lu 97a4b66df7
Merge pull request #2704 from guo-sj/fix_bugs_in_return_value 
fix return value in storage/volume_vacuum.go:444
 2022-02-24 00:49:29 -08:00
chrislu abe7214c1f Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-24 00:37:59 -08:00
chrislu b93d57da31 mount2: dir read opened file 2022-02-24 00:37:57 -08:00
guosj 3e7aa1caf5 fix return value in storage/volume_vacuum.go:444 2022-02-24 15:54:36 +08:00
Chris Lu 6834df77a0
Merge pull request #2701 from guo-sj/fix_bugs_in_return_value 2022-02-23 19:23:37 -08:00
guosj 26f3ab8d4b fix a return bug in func (c *commandVacuum) Do 2022-02-24 09:04:38 +08:00
chrislu 320637dc7a use "mv.from" for moving files 2022-02-23 15:34:42 -08:00
chrislu c29bc9a367 fix error handling 2022-02-23 15:34:25 -08:00