gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4836 commits 28 branches 278 tags 135 MiB
Commit graph

3039 commits

Author SHA1 Message Date
Chris Lu b7e3ca9172
Merge pull request #1650 from kmlebedev/masterServerEnableUI 
security master use access.ui
 2020-12-14 00:31:51 -08:00
Chris Lu 021358749b better detect IP address 
following advice from https://github.com/chrislusf/seaweedfs/issues/1671
 2020-12-14 00:30:20 -08:00
Konstantin Lebedev c4459249ef ui interface is always on 2020-12-14 13:24:41 +05:00
Chris Lu 5cffddebb9 add logging 2020-12-14 00:11:52 -08:00
Chris Lu 933f2f4cfd find location with matching disk type 2020-12-13 23:39:00 -08:00
Chris Lu 94525aa0fd allocate volume by disk type 2020-12-13 23:08:21 -08:00
Chris Lu f6a419c26c disk type configurable for each folder 2020-12-13 22:49:56 -08:00
Chris Lu 7c52a35942 refactor 2020-12-13 22:29:52 -08:00
Chris Lu c45d6a865b Merge branch 'master' into support_ssd_volume 2020-12-13 22:14:18 -08:00
Chris Lu 38fc89d041 filer store: sql update if any insert error happens 
fix https://github.com/chrislusf/seaweedfs/issues/1673
 2020-12-13 20:49:44 -08:00
Chris Lu 23280257df sql put kv: avoid unnecessary update 2020-12-13 19:45:47 -08:00
Chris Lu a9db24cd05 master allocate volumes if ssd type runs out 2020-12-13 19:44:57 -08:00
Chris Lu ac4d4a65af Update Makefile 2020-12-13 12:34:58 -08:00
Chris Lu 8baba93fce rename parameter name to "disk" 2020-12-13 12:06:21 -08:00
Chris Lu 51eadaf2b6 rename parameter name to "disk" 2020-12-13 12:05:31 -08:00
Chris Lu 0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu 715b199eeb fix tests 2020-12-13 04:14:50 -08:00
Chris Lu 66cd219b15 Merge branch 'master' into support_ssd_volume 2020-12-13 04:08:36 -08:00
Chris Lu 813453f3e1 2.15 2020-12-13 03:45:35 -08:00
Chris Lu 17372ac3da balance ssd volumes 2020-12-13 03:40:33 -08:00
Chris Lu d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu 16cd6fb278 fix variable name 2020-12-12 21:19:40 -08:00
Chris Lu 5c465293e9 correctly determine whether a folder is empty 
avoid edge cases that deleting the folder if previous 32 directories are all empty

early terminate if one file is found
 2020-12-12 16:19:29 -08:00
Chris Lu 14910d035c the pagination size was too big for recursive deletion 2020-12-12 13:26:10 -08:00
Chris Lu f930c713fc more efficient recursion 2020-12-12 13:25:19 -08:00
Chris Lu 316d1b4e69 refactor APIs 2020-12-12 12:42:53 -08:00
Chris Lu 03637d6f57 s3: move "delete-directory-if-empty" to read time 
move "delete-directory-if-empty" to read time instead of entry deletion time

the listing speed for a s3 bucket folder will slow down if it has many sub folders

related to 0d345ac97d

fix https://github.com/chrislusf/seaweedfs/issues/1647

fix https://github.com/chrislusf/seaweedfs/issues/1670
 2020-12-12 03:38:34 -08:00
Chris Lu 37075a414d adjust logs 2020-12-12 03:33:57 -08:00
Chris Lu ae655033ac adjust logging 2020-12-11 16:57:53 -08:00
Chris Lu d986c7196d use append time instead of filer's own modification time 
fix https://github.com/chrislusf/seaweedfs/issues/1669
 2020-12-11 16:55:18 -08:00
Chris Lu 3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu c2f18a10cb minor 2020-12-10 22:23:22 -08:00
Chris Lu 4b0c2a846b add logs 2020-12-10 19:55:28 -08:00
Chris Lu 83078ac6ce filer: change to /etc/seaweedfs folder on filer 
fix https://github.com/chrislusf/seaweedfs/issues/1666
 2020-12-10 11:11:02 -08:00
Chris Lu 26731694f8 s3: use static configuration by default 
So that users can still use the previous configuration files.

If leave it empty, s3 will try to use the version from filer
 2020-12-10 00:59:04 -08:00
Chris Lu 765b3ef1be save /etc/iam/identity.json inside filer store 2020-12-10 00:15:22 -08:00
Chris Lu 97c942b5ee paginate through large directories 2020-12-09 23:23:38 -08:00
Chris Lu 02a160c3fd s3: recursively iterate all sub folders 
fix https://github.com/chrislusf/seaweedfs/issues/1656
 2020-12-09 16:47:34 -08:00
Chris Lu a9c619c270 added error handling for saving 2020-12-09 01:34:24 -08:00
Chris Lu 4211601eab set file handle reader to nil 2020-12-08 22:48:18 -08:00
Chris Lu b52ae9cef8 add s3 debug 2020-12-08 22:27:04 -08:00
Chris Lu 8e78187a97 add back last read chunk cache to reader and properly close the reader 2020-12-08 22:26:46 -08:00
Chris Lu 900d22c6ec mount: avoid memory leaking read buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1654

the reader goes together with the file handle, which may stay for a long time.
 2020-12-08 02:38:53 -08:00
Chris Lu 06bb7bf6c0 adding /usr/local/etc/seaweedfs for bsd style config search path 
related to https://reviews.freebsd.org/D27391
 2020-12-07 16:46:48 -08:00
Chris Lu fce8803087 break import cycle 2020-12-07 00:29:17 -08:00
Chris Lu 606051cc79 filer: cache "/etc" filder small files to filer store 2020-12-07 00:11:35 -08:00
Chris Lu ac22f1dd26 filer subscribe: handle rename subscription 2020-12-07 00:10:49 -08:00
Chris Lu eed87791b7 s3: subscribe to s3.configure changes 2020-12-07 00:10:29 -08:00
Chris Lu 01e2da5782 refactoring 2020-12-06 23:16:20 -08:00
Chris Lu 77286f8bea fix test 2020-12-06 21:56:13 -08:00
Chris Lu 57578a6cd1 change parameter help message 2020-12-06 21:54:55 -08:00
Chris Lu 78f3ab439c refactoring 2020-12-06 21:54:34 -08:00
Chris Lu 9acda432fe fix import cycle 2020-12-06 20:12:52 -08:00
Chris Lu ae5eb85a06 refactoring 2020-12-06 20:05:06 -08:00
Chris Lu 5931a2f53f filer conf: support json conf only 2020-12-06 19:47:06 -08:00
Chris Lu 4c72482536 shell: fix fs.configure 2020-12-06 19:43:31 -08:00
Chris Lu 46b91228af 2.14 2020-12-06 18:56:58 -08:00
Chris Lu 263eb29e9f filer: add option to cache small files to filer store 2020-12-04 22:39:43 -08:00
Chris Lu d171d9f988 volume: ensure the volume dat and idx files are always closed 
fix https://github.com/chrislusf/seaweedfs/issues/1646
 2020-12-04 21:50:26 -08:00
Konstantin Lebedev eadbba5c95 security master use access.ui 2020-12-04 12:57:59 +05:00
Chris Lu cc839f935d better unit detection 2020-12-03 19:23:59 -08:00
Chris Lu b219ccfe68 Update volume_ttl_test.go 2020-12-03 12:55:44 -08:00
Chris Lu a83b8e7b01 ttl fix: 3y was converted into 96m 2020-12-03 10:53:29 -08:00
Chris Lu 5fb60b7135
Merge pull request #1596 from kmlebedev/store_s3cred 
S3 credentials store in filer
 2020-12-03 00:40:20 -08:00
Konstantin Lebedev 14699dfcef use content field of entry 2020-12-02 17:19:05 +05:00
Konstantin Lebedev a3d4b50a49 use entry content filed 2020-12-02 16:12:13 +05:00
Chris Lu 003b6245e7 fix nil 2020-12-02 00:09:19 -08:00
Chris Lu f3bb645018 file open error 2020-12-01 23:37:49 -08:00
Chris Lu dc0bc48257 return file open error 2020-12-01 23:36:49 -08:00
Chris Lu 45dba088cd fix tests 2020-12-01 19:37:21 -08:00
Chris Lu 04062c56c7 webdav: improve webdav upload speed 2020-12-01 15:32:27 -08:00
Konstantin Lebedev 03620776ec Merge branch 'upstream_master' into store_s3cred 2020-12-01 16:03:34 +05:00
Chris Lu 005a6123e9
Merge pull request #1643 from hilimd/master 
fix tls grpc ca path
 2020-11-30 17:16:46 -08:00
Chris Lu f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu a9c6be5fc3 set creation time 2020-11-30 03:11:52 -08:00
Chris Lu 141ce67c09 close http request body 2020-11-30 02:45:00 -08:00
Chris Lu 0e99531dbf 2.13 2020-11-29 17:00:03 -08:00
Chris Lu 965413c21b shell: add volume.vacuum command 2020-11-28 23:18:02 -08:00
Chris Lu 96c48bc8a8 fix test 2020-11-28 03:03:43 -08:00
Chris Lu c57a7839ec adjust comments 2020-11-28 03:01:03 -08:00
Chris Lu 8438fb0cc3 fix ec shard spreading error 2020-11-28 02:21:16 -08:00
Chris Lu 536202e4e0 minor 2020-11-28 02:16:06 -08:00
Chris Lu 99c4e50d3d minor 2020-11-28 00:14:11 -08:00
Chris Lu 2c913dde04 volume: detect and drop volumes with disk IO error 
from Jethro in slack:
is it possible to make the assign request a bit smarter? Currently I’m in the state that a disk failed but all assign request are being send to this volume. It would be cool if the master sees this and stopped using this volume.
e=HTTP(http://x:8089/913,045a782b63176edf) not 200 but 500 Internal Server Error
Body={"size":740167,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"ee4381e202212ff3aee647704c036689"}
e=HTTP(http://x:8089/913,045a782c90240077) not 200 but 500 Internal Server Error
Body={"size":792779,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"c43463ccc11eb6eb2fc306f407a6a953"}
e=HTTP(http://x:8089/913,045a782e6b7901ea) not 200 but 500 Internal Server Error
Body={"size":3962392,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"04c91198e9b276c81f11dbf189af5d28"}
 2020-11-28 00:09:29 -08:00
Chris Lu 9ac4935f22 read from volume index file directly instead of open a separate file 
fix https://github.com/chrislusf/seaweedfs/issues/1640

read from volume index file directly instead of open a separate file,
to ensure reading latest index entries.
 2020-11-27 16:18:48 -08:00
Chris Lu 85554bea38 filer: readonly handle static resources 
fix https://github.com/chrislusf/seaweedfs/issues/1641
 2020-11-27 13:42:14 -08:00
Chris Lu 6d30b21b10 volume: add "-dir.idx" option for separate index storage 
fix https://github.com/chrislusf/seaweedfs/issues/1265
 2020-11-27 03:17:10 -08:00
Chris Lu 3c229eb677 refactor 2020-11-26 17:21:55 -08:00
Chris Lu c0fb4a3766 add comments 2020-11-26 15:22:42 -08:00
Chris Lu 6783ae5f8f rename 2020-11-26 15:19:43 -08:00
Chris Lu 983fd90010 marked as WIP 2020-11-26 15:08:52 -08:00
Chris Lu 97f706f35b just refactoring 2020-11-26 15:08:05 -08:00
Chris Lu b11449f955 pass in dir.idx parameter to DiskLocation 2020-11-26 14:59:03 -08:00
Chris Lu 0d345ac97d s3: remove empty parent folder on delete 
fix https://github.com/chrislusf/seaweedfs/issues/1637
 2020-11-26 12:21:58 -08:00
Chris Lu cc2bd97ad9 refactor 2020-11-26 11:25:56 -08:00
Chris Lu 0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu effa00ed08 refactor 2020-11-26 11:21:58 -08:00
Chris Lu 513bcd6e0d filer: avoid duplicated FindEntry for deletion 2020-11-26 11:14:56 -08:00
Chris Lu 1ae108efca add debug message 2020-11-26 10:41:10 -08:00