gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
7411 commits 28 branches 278 tags 135 MiB
Commit graph

4920 commits

Author SHA1 Message Date
chrislu 3da2b83b38 Added a "-conf_dir" option to customize *.toml configuration file directory. 
fix https://github.com/chrislusf/seaweedfs/issues/2753
 2022-03-19 00:22:47 -07:00
zzq09494 9f16df736e fix-filer: calculation error of the method skipCheckParentDirEntry 2022-03-19 09:15:42 +08:00
chrislu 2824940ecf Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-17 16:54:30 -07:00
chrislu f247cab5cd skip localhost if bound to all interfaces already 0.0.0.0 or 127.0.0.1 2022-03-17 16:54:29 -07:00
Berck Nash 7ee38fa3a4 The fixes for https://github.com/chrislusf/seaweedfs/issues/1937 had a few problems: 
(1) The help file says that in the absence of a ipBind being specified, that it will bind to the "ip" specified.  Instead, it bound to localhost which broke the default configuration.  This change implements the documented behavior instead.

(2) The new IAM filer ip address has no default. This instantiates it to the same as the filer IP.  I'm not sure if there should be a corresponding iam.ip or iam.ipBind option added to the filer command?
 2022-03-17 15:30:23 -06:00
chrislu 6886167d5f minor optimization 2022-03-17 00:03:28 -07:00
chrislu 7542fd7f4d mount: optimize a bit when writing files 2022-03-17 00:02:38 -07:00
chrislu 4042fdf3bb rename to skipCheckParentDir 
related to https://github.com/chrislusf/seaweedfs/pull/2761

It's better to default to false.
 2022-03-16 23:55:31 -07:00
zzq09494 40b0033fa7 go fmt 2022-03-17 14:19:48 +08:00
zzq09494 ee5417a5b3 fix: ut test error 2022-03-17 11:16:46 +08:00
zzq09494 81cce4b4c3 filer: support uploading file without needEnsureParentDir 2022-03-17 10:53:47 +08:00
zzq09494 a6a8892255 Revert "filer: support uploading file without needEnsureParentDir" 
This reverts commit a93c4947ba.
 2022-03-17 10:27:17 +08:00
zzq09494 a93c4947ba filer: support uploading file without needEnsureParentDir 2022-03-17 10:18:23 +08:00
Berck Nash 9b14f0c81a Add mTLS support for both master and volume http server. 2022-03-16 09:52:17 -06:00
chrislu ed818c90a9 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-15 22:28:21 -07:00
chrislu 3639cad69c master, filer, s3: also listen to "localhost" in addition to specific ip address 
related to https://github.com/chrislusf/seaweedfs/issues/1937
 2022-03-15 22:28:18 -07:00
Konstantin Lebedev f43c6daeda Need to exit waiting if request is was canceled 2022-03-15 19:55:22 +05:00
Konstantin Lebedev c2e7d663bc reduced the timeout for creating a connection from 30 to 10 seconds 2022-03-15 13:32:05 +05:00
chrislu fbc9f0eb64 minor 2022-03-14 03:19:16 -07:00
chrislu 2eda3a686f 2.94 2022-03-14 00:55:01 -07:00
chrislu bd5c5586b5 generate inode via path and time 2022-03-14 00:03:29 -07:00
chrislu 5cba8e51c5 refactor 2022-03-13 18:34:57 -07:00
chrislu f2f68f675e write to disk during random writes, limiting total disk spaces used 2022-03-13 18:17:35 -07:00
chrislu 2f4cd20f90 tests skip reader pattern monitoring 2022-03-13 18:15:53 -07:00
chrislu 53513475bf mount: add back random read support 
avoid too much memory used also
 2022-03-13 01:38:52 -08:00
chrislu f70c1e449b add useful doc link 2022-03-13 00:14:50 -08:00
chrislu b20ddc57a7 mount: return open status 2022-03-12 22:38:14 -08:00
chrislu b40d252761 mount: chmod for root 2022-03-12 12:10:56 -08:00
chrislu 3a6eb8ca5f default bind to one ip address 
fix https://github.com/chrislusf/seaweedfs/issues/1937
 2022-03-11 14:02:39 -08:00
Konstantin Lebedev f53cff045f checks disk file exist 2022-03-10 18:58:56 +05:00
Konstantin Lebedev 834210a9dc avoid connect to the old filler address 
https://github.com/chrislusf/seaweedfs/issues/2545
 2022-03-10 15:24:45 +05:00
chrislu 197ade6aeb Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-09 22:26:55 -08:00
chrislu 011a41b561 add back writes to swap file when too many in memory chunks are used. 2022-03-09 22:26:51 -08:00
zzq09494 2ea18fca48 fix:support some databases which not support 'IF NOT EXISTS' keyword of 'create table command' 2022-03-10 13:37:34 +08:00
banjiaojuhao f28dbbe5c5 [bugfix] filer: 1. Delete uploaded chunks when upload failed. 2. Report error when upload is interrupted by user. 2022-03-10 11:40:39 +08:00
chrislu e2b07737da Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-08 23:02:33 -08:00
chrislu dc204dd137 fix nil entry 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x1d340b4]

goroutine 130523 [running]:
github.com/chrislusf/seaweedfs/weed/filer.FileSize(...)
        /code/seaweedfs/weed/filer/filechunks.go:26
github.com/chrislusf/seaweedfs/weed/mount.(*WFS).Lookup(0xc000866d80, 0x1, 0xc002897f40, {0xc004b00980, 0x39}, 0x1ec19e0)
        /code/seaweedfs/weed/mount/weedfs_dir_lookup.go:59 +0x654
github.com/hanwen/go-fuse/v2/fuse.doLookup(0xc00033c000, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/opcode.go:333 +0x6b
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000ab2420, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:483 +0x1f3
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000ab2420, 0x0)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:456 +0x110
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:323 +0x534
 2022-03-08 23:02:30 -08:00
banjiaojuhao f7f2a597dd minor 2022-03-08 16:22:55 +08:00
banjiaojuhao d61bea9038 [bugfix] filer: In file modification, old chunks will be mis-deleted when they are merged(Manifestized). 2022-03-08 16:22:55 +08:00
chrislu 3aeee3d748 ensure releasing file handle 2022-03-07 14:01:24 -08:00
chrislu 8136384473 remove debug message 2022-03-07 11:22:26 -08:00
banjiaojuhao b9ff7723dd [bugfix] filer: nil pointer dereference 2022-03-07 23:26:25 +08:00
chrislu da3d330616 s3 and filer transport using unix domain socket instead of tcp 2022-03-07 02:00:14 -08:00
chrislu 0cb17b45b1 refactoring 2022-03-07 01:59:01 -08:00
chrislu 6d3db4445b buffer for all range requests 2022-03-07 01:56:47 -08:00
chrislu f3bcbeb60a a little optimization 2022-03-07 00:24:59 -08:00
chrislu f7ee60996c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-07 00:08:09 -08:00
chrislu 6f3ec989a7 fix manifest batch size 2022-03-07 00:07:53 -08:00
Chris Lu 1e7fcef581
Merge pull request #2729 from banjiaojuhao/filer_metadata-resolve-manifest 
filer: support get metadata with resolved manifest chunk
 2022-03-07 00:05:46 -08:00
chrislu bb0b784544 minor 2022-03-07 00:04:59 -08:00
banjiaojuhao bfcc9ca808 filer: support metadata with resolved manifest chunk 2022-03-07 15:47:51 +08:00
banjiaojuhao 71f3046841 filer: add back isAppend function 2022-03-07 15:41:07 +08:00
chrislu 0ba4e4cd23 2.93 2022-03-06 18:54:12 -08:00
chrislu 46a28b8819 mount: adjust disk space based on quota 2022-03-06 17:22:49 -08:00
chrislu ede6ce44c6 fix test 2022-03-06 17:09:55 -08:00
chrislu 21ef152423 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-06 17:04:24 -08:00
chrislu f3442e36e6 mount: quota adjust error type to be syscall.ENOSPC 2022-03-06 17:04:21 -08:00
Chris Lu 89cd0c27a8
Merge pull request #2728 from kmlebedev/set_default_leveldb2 2022-03-06 07:26:30 -08:00
Konstantin Lebedev cf444ebd07 Set default leveldb2 enabled 
avoid Filer store is enabled for both leveldb2 and mysql
 2022-03-06 18:27:25 +05:00
banjiaojuhao a07c93f4dd filer: reset bytesBuffer before use 2022-03-06 21:07:36 +08:00
chrislu c7e8ac18f0 mount: quota for one mounted collection 
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
 2022-03-06 02:44:40 -08:00
chrislu b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu f1713c96ae avoid possible runtime error: index out of range [0] with length 0 2022-03-05 21:14:31 -08:00
chrislu da76af187f mount: avoid possible index out of bounds error 2022-03-04 22:36:01 -08:00
chrislu 6e49e75a5b use logical number of files and sizes for statistics and quota 2022-03-04 18:47:44 -08:00
chrislu f51e20028a mount: avoid comma in mount options 
fix https://github.com/chrislusf/seaweedfs/issues/2719
 2022-03-03 03:42:29 -08:00
chrislu dc0f48682c unused 2022-03-03 03:41:35 -08:00
chrislu 28b8974a3a mount: fix directory pagination when using midnight commander 2022-03-03 02:59:31 -08:00
chrislu a96d4254e9 filer, s3, volume server: a bit memory optimization 2022-03-02 20:15:28 -08:00
chrislu 6fbbc78574 stream reading a whole chunk 2022-03-02 13:50:46 -08:00
chrislu 784583afc6 avoid pool memory allocation if too large 2022-03-02 13:50:28 -08:00
chrislu ba14307319 2.92 2022-02-28 15:22:19 -08:00
chrislu fcf3714443 mount: add back support for filer.path 2022-02-28 12:16:53 -08:00
chrislu 80c017907b filer.backup: fix backing up encrypted chunks 
I have done filer.backup test:
replication.toml:
[sink.local]
enabled = true
directory = "/srv/test"
___
system@dat1:/srv/test$ weed filer.backup -filer=app1:8888 -filerProxy
I0228 12:39:28 19571 filer_replication.go:129] Configure sink to local
I0228 12:39:28 19571 filer_backup.go:98] resuming from 2022-02-28 12:04:20.210984693 +0100 CET
I0228 12:39:29 19571 filer_backup.go:113] backup app1:8888 progressed to 2022-02-28 12:04:20.211726749 +0100 CET 0.33/sec

system@dat1:/srv/test$ ls -l
total 16
drwxr-xr-x 2 system system 4096 Feb 28 12:39 a
-rw-r--r-- 1 system system   48 Feb 28 12:39 fu.txt
-rw-r--r-- 1 system system   32 Feb 28 12:39 _index.html
-rw-r--r-- 1 system system   68 Feb 28 12:39 index.php
system@dat1:/srv/test$ cat fu.txt
?	?=?^??`?f^};?{4?Z%?X0=??rV????|"?1??踪~??
system@dat1:/srv/test$
On the active mount on the target server it's:
system@app1:/srv/app$ ls -l
total 2
drwxrwxr-x 1 system system  0 Feb 28 12:04 a
-rw-r--r-- 1 system system 20 Feb 28 12:04 fu.txt
-rw-r--r-- 1 system system  4 Feb 28 12:04 _index.html
-rw-r--r-- 1 system system 40 Feb 28 12:04 index.php
system@app1:/srv/app$ cat fu.txt
This is static boy!
Filer was started with: weed filer master="app1:9333,app2:9333,app3:9333" -encryptVolumeData
It seems like it's still encrypted?
 2022-02-28 10:07:06 -08:00
chrislu aad62ee148 mount: mark directory uncached if forgotten 2022-02-28 02:08:24 -08:00
chrislu 554e239097 release file handle 2022-02-28 01:23:14 -08:00
chrislu 3639fedd01 mount: fix fsync opened and renamed files 2022-02-28 00:34:17 -08:00
chrislu 63a9d8f01d ensure inodes are not duplicating unless hardlinked 2022-02-27 23:13:49 -08:00
chrislu de77d00c81 correctly clean up for a file 2022-02-27 23:12:28 -08:00
chrislu 18543c6e8b minor 2022-02-27 23:11:09 -08:00
chrislu 09cd00f356 2.91 2022-02-27 04:03:39 -08:00
chrislu 941ced60a4 download 2 chunks if at the beginning of a file 2022-02-27 03:57:24 -08:00
chrislu f9d9eed0c9 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-27 03:41:35 -08:00
chrislu d602d68fd1 remove dead code 2022-02-27 03:41:32 -08:00
Chris Lu 6c07af6014
Merge pull request #2715 from gfx-labs/acl-pr 2022-02-27 03:27:32 -08:00
chrislu c3792c8352 remove dead code 2022-02-27 03:03:19 -08:00
chrislu aa9eef81e6 retire mount v1 2022-02-27 02:57:27 -08:00
eddy-gfx fb940dd807
Merge branch 'chrislusf:master' into acl-pr 2022-02-27 04:52:08 -06:00
elee 881a0fe806 ensure compatibility 2022-02-27 04:50:59 -06:00
elee 954ad98e0d set canned acl on replication create 2022-02-27 04:49:31 -06:00
chrislu 6e1ab97988 use debug option to see operations 2022-02-27 02:02:30 -08:00
chrislu 2112d99140 mount2: add back readonly mode 2022-02-27 01:13:32 -08:00
chrislu 9ef5bb20f6 mount2: invalidate fuse cache for replaced inode 2022-02-27 00:00:23 -08:00
chrislu 4ddcbaab57 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-26 23:20:47 -08:00
chrislu 551d00d51a prefetch other chunks when stream reading 2022-02-26 23:20:45 -08:00
Chris Lu e1d3dd5e18
Merge pull request #2712 from guo-sj/correct_comments 2022-02-26 09:16:12 -08:00
guosj 82cad5e330
correct comments 2022-02-26 22:44:26 +08:00
chrislu 7b1a713d2a remove dead code 2022-02-26 03:23:15 -08:00
chrislu 3a58b7bac6 a little safer 2022-02-26 03:23:06 -08:00
chrislu 708e14fcfa avoid possible too big memory allocation 2022-02-26 03:22:41 -08:00