gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4394 commits 28 branches 278 tags 135 MiB
Commit graph

2730 commits

Author SHA1 Message Date
Chris Lu 58fa506491 minor 2020-10-13 19:50:22 -07:00
Chris Lu 28d4e1a51b mount: retry for directory listing with filer 
related to https://github.com/chrislusf/seaweedfs/issues/1530
 2020-10-13 19:49:52 -07:00
Chris Lu c127da1219 filer: linearize timeout for large chunk of data 2020-10-13 14:04:46 -07:00
Chris Lu aac4cb1f0c adds errror on read and write 2020-10-13 13:53:34 -07:00
Chris Lu 9b4f7fed14 mount: report filer IO error 
related to https://github.com/chrislusf/seaweedfs/issues/1530
 2020-10-13 11:21:13 -07:00
Chris Lu 3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu b18f21cce1 mount: fix bound tree with filer.path 
fix https://github.com/chrislusf/seaweedfs/issues/1528
 2020-10-12 21:58:37 -07:00
Chris Lu f022aff289 add back http.StatusPartialContent 
revert e7c04af1d0
 2020-10-12 12:26:25 -07:00
Chris Lu bbd0afd37e 2.04 2020-10-11 21:25:30 -07:00
Chris Lu 7704469d29 mount: outsideContainerClusterMode changed to use volume server publicUrl 2020-10-11 20:42:15 -07:00
Chris Lu 723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu d155f907c2 mount: configurable read wait time 2020-10-10 20:09:43 -07:00
Chris Lu 9b0e8ef026 filer: added QR code to transfer files with mobile devices 2020-10-10 18:00:21 -07:00
Chris Lu 8a52379ecb add retry if volume can not be found 2020-10-10 16:02:39 -07:00
Chris Lu 70af0ec24c filer: fix hanging on read 2020-10-10 16:02:10 -07:00
Chris Lu cff8bb6554 return proper error 2020-10-10 15:43:22 -07:00
Chris Lu 6da8eef54f fix logic error 2020-10-09 00:01:47 -07:00
Chris Lu 5d01dd28a6 add logging 2020-10-08 23:38:00 -07:00
Chris Lu 8d34eb0050 mount:exponentially backoff if read error for about 10 minutes 2020-10-08 23:31:26 -07:00
Chris Lu 6e1f936efd refactoring 2020-10-08 23:19:42 -07:00
Chris Lu b2ee5873fb fix error not being returned 2020-10-08 23:19:20 -07:00
Chris Lu b067853162 mount: ignore recursion error 
fix https://github.com/chrislusf/seaweedfs/issues/1514

the host OS should manage recursive deletion.
 2020-10-08 21:55:55 -07:00
Chris Lu e1a8e3f900 fix logging 2020-10-08 18:40:38 -07:00
Chris Lu 8676db0f6a adjust logging 2020-10-08 18:33:37 -07:00
Chris Lu bd8b9b0c2e some changes to prepare fix after failed reads 2020-10-08 18:33:06 -07:00
Chris Lu 78859e35fc s3: adjust config examples 2020-10-08 10:12:09 -07:00
Chris Lu 9e7a2772b1 s3: only admin can list all buckets 2020-10-08 10:12:09 -07:00
Chris Lu eed492b73b randomize file locations 2020-10-07 23:58:32 -07:00
Chris Lu ec08a1670b volume: avoid hanging when stopping a volume server 2020-10-07 23:48:24 -07:00
Chris Lu 62d3d3aea0 go fmt 2020-10-07 23:30:54 -07:00
Chris Lu e91b9c85a2 s3: Added support for "List" action in weed s3 -config=... in the config file. 
fix https://github.com/chrislusf/seaweedfs/issues/1511
 2020-10-07 23:22:35 -07:00
Chris Lu a8624c2e4f read from alternative replica 
related to https://github.com/chrislusf/seaweedfs/issues/1512
 2020-10-07 22:49:04 -07:00
Chris Lu da4edf3651 master: check peers for existing leader before starting a leader election 
fix https://github.com/chrislusf/seaweedfs/issues/1509
 2020-10-07 01:25:39 -07:00
Chris Lu c543762e23 volume: auto expands -max setting 
I have tried to run weed volume but got errors...
cmd:
weed volume -max=0 -mserver="127.0.0.1:9333" -port=8080 -dir="/var/www/d1/sfs,/var/www/d2/sfs,/var/www/d3/sfs"
log:
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d1/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d2/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d3/sfs Permission: -rwxr-xr-x
F1006 15:09:26 17040 volume.go:142] 3 directories by -dir, but only 1 max is set by -max
What am I doing wrong? Thx.
 2020-10-06 09:05:30 -07:00
Chris Lu 36492c47ec adjust 2020-10-05 14:06:18 -07:00
Chris Lu 8f8e9ddbde adjust API 2020-10-05 09:47:07 -07:00
Konstantin Lebedev e4f2d9eb4a We return etag using the same algorithm as aws s3 
https://teppen.io/2018/06/23/aws_s3_etags/
 2020-10-05 14:43:32 +05:00
Chris Lu 4fc673341f 2.03 2020-10-04 21:58:22 -07:00
Chris Lu 6b591b02af adjust log level 2020-10-04 16:21:43 -07:00
Устюжанин Антон Александрович 702b1cb876 fix: remove deleted peers if resumeState = true 2020-10-04 21:56:17 +05:00
Chris Lu d522df0ecd adjust log level 2020-10-04 01:35:32 -07:00
Chris Lu 666859f7cd mount: read prefetching 2020-10-04 01:31:04 -07:00
Chris Lu bc47835997 mount: avoid duplicated reads 2020-10-03 20:16:42 -07:00
Chris Lu 8d65ad1444 Revert "mount: adds read prefetching" 
This reverts commit 692f0614d1.
 2020-10-03 19:40:56 -07:00
Chris Lu 692f0614d1 mount: adds read prefetching 2020-10-03 16:50:09 -07:00
Chris Lu 707936f482 re-enable caching larger than 16MB 
revert 62ce85610e
 2020-10-03 14:12:38 -07:00
Chris Lu 2c20ef72ae fix starting problem 2020-10-03 13:56:38 -07:00
Chris Lu d77e3c96e9 FUSE: use the mount directory permissions 2020-10-03 13:37:35 -07:00
Chris Lu 95ffed87c0
Merge pull request #1505 from ustuzhanin/remove_raft_state 
Resume raft state
 2020-10-03 10:38:47 -07:00
Устюжанин Антон Александрович 6fbfe73683 fix: resumeState 2020-10-03 22:16:47 +05:00
Устюжанин Антон Александрович dc31b19469 fix: restore raft state 2020-10-03 14:03:41 +05:00
Chris Lu f781cce500 s3: support object tagging 
* GetObjectTagging
* PutObjectTagging
* DeleteObjectTagging
 2020-10-02 22:21:51 -07:00
Устюжанин Антон Александрович 8c82fb7e5f fix: restore raft state 2020-10-02 23:01:20 +05:00
Chris Lu 9ab98fa912 s3 metrics adjust the label 2020-10-01 07:17:47 -07:00
Chris Lu f1e879fe7a 2.02 2020-10-01 07:15:49 -07:00
Chris Lu a1c01d716b volume: avoid deadlock when deleting volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1501
 2020-10-01 07:10:03 -07:00
Konstantin Lebedev 86329bbf2b label name is statusCode 2020-10-01 01:22:38 +05:00
Konstantin Lebedev 68463e92c1 add status code in S3RequestCounter 2020-10-01 00:59:39 +05:00
Chris Lu 79ab10e300 adjust help message 2020-09-30 09:32:00 -07:00
Chris Lu 55cb68c09c filer: replication follows master config if not specified 2020-09-30 09:15:55 -07:00
Chris Lu b9887504e8 fix test 2020-09-27 23:19:50 -07:00
Chris Lu f46eae284e adjust for test 2020-09-27 23:08:11 -07:00
Chris Lu 75aca5e13d 2.01 2020-09-27 23:02:41 -07:00
Chris Lu e6552b5e1e filer: able to start s3 together 2020-09-27 23:00:43 -07:00
Chris Lu 474e2b6ac3 add a hard link marker to 16byte + maker, for future extensions 2020-09-27 22:38:30 -07:00
Chris Lu c49e2bb9a3 adjust 2020-09-27 12:07:45 -07:00
Chris Lu 62ce85610e skip caching too large chunks 2020-09-27 11:58:48 -07:00
Chris Lu 9ad2dcca2b more tests 2020-09-27 11:42:51 -07:00
Chris Lu e43d86c796 fix pre allocated volume size 2020-09-27 10:58:19 -07:00
Chris Lu 31fc7bb2e1 refactor 
adjust for faster test
 2020-09-27 10:41:29 -07:00
Chris Lu a37535cd9f avoid non utf-8 in filename 
fix https://github.com/chrislusf/seaweedfs/issues/1493
 2020-09-26 00:13:39 -07:00
Chris Lu 41be7a4c8f filer: upload also set file size 
ensure works same as fuse mount
 2020-09-25 12:02:06 -07:00
Chris Lu 35f6518c36 weed upload: usePublicUrl was not being used on big files 
fix https://github.com/chrislusf/seaweedfs/issues/1492
 2020-09-25 02:41:38 -07:00
Chris Lu 22049dc2e6
Merge pull request #1491 from hilimd/master 
s3:  fix get object metadata problem
 2020-09-25 02:35:00 -07:00
limd 402aef8f30 s3: 
1.fix spark reading S3 directory wildcard problem
2.fix the problem of the spark history service writing S3 directory

reference git revsion number: b41b7ea4d0
 2020-09-25 14:37:02 +08:00
Chris Lu 8516517c48 filer store: Cassandra supports username/password 2020-09-24 21:31:06 -07:00
limd ed7816681a Add cassandra authenticator mode 2020-09-25 11:22:47 +08:00
limd 370a98cf6b code style 2020-09-25 11:19:42 +08:00
limd aee3fd08b9 code style 2020-09-25 11:04:32 +08:00
Chris Lu 043b063136
Merge pull request #1482 from hilimd/master 
Fix: s3 delete object
 2020-09-24 18:21:34 -07:00
limd 48c578410f mount: rollback default value 2020-09-25 09:18:52 +08:00
Chris Lu dbf5327b98 s3: handle response-content-encoding 
fix https://github.com/chrislusf/seaweedfs/issues/1487
 2020-09-24 18:09:52 -07:00
Chris Lu 7726965a47 minor 2020-09-24 11:20:12 -07:00
Chris Lu 0790c6d605 fix empty hard link id 2020-09-24 11:16:43 -07:00
Chris Lu 1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu 4856bce0ee adjust for metrics port 2020-09-24 10:21:23 -07:00
Chris Lu d2d3aec3e1 consolidate to one metricsPort in "weed server" mode 2020-09-24 09:55:02 -07:00
Chris Lu 2e9099369e
Merge pull request #1489 from kmlebedev/promhttp 
Promhttp
 2020-09-24 09:50:48 -07:00
Chris Lu 2e7c361a0d hardlink deletion factors in hardlink counter 2020-09-24 09:43:52 -07:00
Chris Lu 1295347958 adjust hardlink update 
simplify logic, pass entity content directly to hard link. The "weed mount" handles the logic to calculate hard link counter.
 2020-09-24 09:43:00 -07:00
Konstantin Lebedev 98e9de6e11 fix style 2020-09-24 17:48:39 +05:00
Konstantin Lebedev 324e44d4b3 add start metrics server 2020-09-24 17:45:39 +05:00
limd cbca14edc5 mount: fix k8s pvc and os mount directory permission bug 2020-09-24 18:07:16 +08:00
Chris Lu 5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
limd 59e91e9c7e mount: fix k8s pvc mount directory permission 2020-09-24 10:25:45 +08:00
Chris Lu c7d7b1a0f6
Merge pull request #1485 from LIBA-S/fix_oversized 
Correct the oversized state of volume after compaction
 2020-09-23 19:24:30 -07:00
Chris Lu b80a2b3cc9
Merge pull request #1486 from LIBA-S/fix_race_condition 
Fix a race condition when handle VolumeLocationList
 2020-09-23 19:24:16 -07:00
LIBA-S eecd6b5d35 Fix a race condition when handle VolumeLocationList 2020-09-23 20:56:51 +08:00
LIBA-S 0157798ebf Correct the oversized state of volume after compaction 2020-09-23 20:27:42 +08:00
limd 9b21ec27e4 mount: auto created dir set corrct umask fix bug 2020-09-23 18:35:37 +08:00