gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4356 commits 28 branches 278 tags 135 MiB
Commit graph

2697 commits

Author SHA1 Message Date
Chris Lu c543762e23 volume: auto expands -max setting 
I have tried to run weed volume but got errors...
cmd:
weed volume -max=0 -mserver="127.0.0.1:9333" -port=8080 -dir="/var/www/d1/sfs,/var/www/d2/sfs,/var/www/d3/sfs"
log:
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d1/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d2/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d3/sfs Permission: -rwxr-xr-x
F1006 15:09:26 17040 volume.go:142] 3 directories by -dir, but only 1 max is set by -max
What am I doing wrong? Thx.
 2020-10-06 09:05:30 -07:00
Chris Lu 36492c47ec adjust 2020-10-05 14:06:18 -07:00
Chris Lu 8f8e9ddbde adjust API 2020-10-05 09:47:07 -07:00
Konstantin Lebedev e4f2d9eb4a We return etag using the same algorithm as aws s3 
https://teppen.io/2018/06/23/aws_s3_etags/
 2020-10-05 14:43:32 +05:00
Chris Lu 4fc673341f 2.03 2020-10-04 21:58:22 -07:00
Chris Lu 6b591b02af adjust log level 2020-10-04 16:21:43 -07:00
Устюжанин Антон Александрович 702b1cb876 fix: remove deleted peers if resumeState = true 2020-10-04 21:56:17 +05:00
Chris Lu d522df0ecd adjust log level 2020-10-04 01:35:32 -07:00
Chris Lu 666859f7cd mount: read prefetching 2020-10-04 01:31:04 -07:00
Chris Lu bc47835997 mount: avoid duplicated reads 2020-10-03 20:16:42 -07:00
Chris Lu 8d65ad1444 Revert "mount: adds read prefetching" 
This reverts commit 692f0614d1.
 2020-10-03 19:40:56 -07:00
Chris Lu 692f0614d1 mount: adds read prefetching 2020-10-03 16:50:09 -07:00
Chris Lu 707936f482 re-enable caching larger than 16MB 
revert 62ce85610e
 2020-10-03 14:12:38 -07:00
Chris Lu 2c20ef72ae fix starting problem 2020-10-03 13:56:38 -07:00
Chris Lu d77e3c96e9 FUSE: use the mount directory permissions 2020-10-03 13:37:35 -07:00
Chris Lu 95ffed87c0
Merge pull request #1505 from ustuzhanin/remove_raft_state 
Resume raft state
 2020-10-03 10:38:47 -07:00
Устюжанин Антон Александрович 6fbfe73683 fix: resumeState 2020-10-03 22:16:47 +05:00
Устюжанин Антон Александрович dc31b19469 fix: restore raft state 2020-10-03 14:03:41 +05:00
Chris Lu f781cce500 s3: support object tagging 
* GetObjectTagging
* PutObjectTagging
* DeleteObjectTagging
 2020-10-02 22:21:51 -07:00
Устюжанин Антон Александрович 8c82fb7e5f fix: restore raft state 2020-10-02 23:01:20 +05:00
Chris Lu 9ab98fa912 s3 metrics adjust the label 2020-10-01 07:17:47 -07:00
Chris Lu f1e879fe7a 2.02 2020-10-01 07:15:49 -07:00
Chris Lu a1c01d716b volume: avoid deadlock when deleting volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1501
 2020-10-01 07:10:03 -07:00
Konstantin Lebedev 86329bbf2b label name is statusCode 2020-10-01 01:22:38 +05:00
Konstantin Lebedev 68463e92c1 add status code in S3RequestCounter 2020-10-01 00:59:39 +05:00
Chris Lu 79ab10e300 adjust help message 2020-09-30 09:32:00 -07:00
Chris Lu 55cb68c09c filer: replication follows master config if not specified 2020-09-30 09:15:55 -07:00
Chris Lu b9887504e8 fix test 2020-09-27 23:19:50 -07:00
Chris Lu f46eae284e adjust for test 2020-09-27 23:08:11 -07:00
Chris Lu 75aca5e13d 2.01 2020-09-27 23:02:41 -07:00
Chris Lu e6552b5e1e filer: able to start s3 together 2020-09-27 23:00:43 -07:00
Chris Lu 474e2b6ac3 add a hard link marker to 16byte + maker, for future extensions 2020-09-27 22:38:30 -07:00
Chris Lu c49e2bb9a3 adjust 2020-09-27 12:07:45 -07:00
Chris Lu 62ce85610e skip caching too large chunks 2020-09-27 11:58:48 -07:00
Chris Lu 9ad2dcca2b more tests 2020-09-27 11:42:51 -07:00
Chris Lu e43d86c796 fix pre allocated volume size 2020-09-27 10:58:19 -07:00
Chris Lu 31fc7bb2e1 refactor 
adjust for faster test
 2020-09-27 10:41:29 -07:00
Chris Lu a37535cd9f avoid non utf-8 in filename 
fix https://github.com/chrislusf/seaweedfs/issues/1493
 2020-09-26 00:13:39 -07:00
Chris Lu 41be7a4c8f filer: upload also set file size 
ensure works same as fuse mount
 2020-09-25 12:02:06 -07:00
Chris Lu 35f6518c36 weed upload: usePublicUrl was not being used on big files 
fix https://github.com/chrislusf/seaweedfs/issues/1492
 2020-09-25 02:41:38 -07:00
Chris Lu 22049dc2e6
Merge pull request #1491 from hilimd/master 
s3:  fix get object metadata problem
 2020-09-25 02:35:00 -07:00
limd 402aef8f30 s3: 
1.fix spark reading S3 directory wildcard problem
2.fix the problem of the spark history service writing S3 directory

reference git revsion number: b41b7ea4d0
 2020-09-25 14:37:02 +08:00
Chris Lu 8516517c48 filer store: Cassandra supports username/password 2020-09-24 21:31:06 -07:00
limd ed7816681a Add cassandra authenticator mode 2020-09-25 11:22:47 +08:00
limd 370a98cf6b code style 2020-09-25 11:19:42 +08:00
limd aee3fd08b9 code style 2020-09-25 11:04:32 +08:00
Chris Lu 043b063136
Merge pull request #1482 from hilimd/master 
Fix: s3 delete object
 2020-09-24 18:21:34 -07:00
limd 48c578410f mount: rollback default value 2020-09-25 09:18:52 +08:00
Chris Lu dbf5327b98 s3: handle response-content-encoding 
fix https://github.com/chrislusf/seaweedfs/issues/1487
 2020-09-24 18:09:52 -07:00
Chris Lu 7726965a47 minor 2020-09-24 11:20:12 -07:00
Chris Lu 0790c6d605 fix empty hard link id 2020-09-24 11:16:43 -07:00
Chris Lu 1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu 4856bce0ee adjust for metrics port 2020-09-24 10:21:23 -07:00
Chris Lu d2d3aec3e1 consolidate to one metricsPort in "weed server" mode 2020-09-24 09:55:02 -07:00
Chris Lu 2e9099369e
Merge pull request #1489 from kmlebedev/promhttp 
Promhttp
 2020-09-24 09:50:48 -07:00
Chris Lu 2e7c361a0d hardlink deletion factors in hardlink counter 2020-09-24 09:43:52 -07:00
Chris Lu 1295347958 adjust hardlink update 
simplify logic, pass entity content directly to hard link. The "weed mount" handles the logic to calculate hard link counter.
 2020-09-24 09:43:00 -07:00
Konstantin Lebedev 98e9de6e11 fix style 2020-09-24 17:48:39 +05:00
Konstantin Lebedev 324e44d4b3 add start metrics server 2020-09-24 17:45:39 +05:00
limd cbca14edc5 mount: fix k8s pvc and os mount directory permission bug 2020-09-24 18:07:16 +08:00
Chris Lu 5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
limd 59e91e9c7e mount: fix k8s pvc mount directory permission 2020-09-24 10:25:45 +08:00
Chris Lu c7d7b1a0f6
Merge pull request #1485 from LIBA-S/fix_oversized 
Correct the oversized state of volume after compaction
 2020-09-23 19:24:30 -07:00
Chris Lu b80a2b3cc9
Merge pull request #1486 from LIBA-S/fix_race_condition 
Fix a race condition when handle VolumeLocationList
 2020-09-23 19:24:16 -07:00
LIBA-S eecd6b5d35 Fix a race condition when handle VolumeLocationList 2020-09-23 20:56:51 +08:00
LIBA-S 0157798ebf Correct the oversized state of volume after compaction 2020-09-23 20:27:42 +08:00
limd 9b21ec27e4 mount: auto created dir set corrct umask fix bug 2020-09-23 18:35:37 +08:00
hilimd 0db149fb5f
Merge pull request #18 from chrislusf/master 
sync
 2020-09-23 17:32:05 +08:00
Chris Lu f7a0ccb595 mount: auto created dir set corrct umask 2020-09-23 02:31:19 -07:00
Chris Lu f0e325b6fc mount: auto created directory follow umask 2020-09-23 02:27:57 -07:00
limd d506080c36 rollback 2020-09-23 14:29:53 +08:00
limd 8f9f29b773 fixed the problem of empty directory when S3 deleted the directory 2020-09-23 13:33:13 +08:00
limd 1892677b22 fixed the problem of empty directory when S3 deleted the directory 2020-09-23 13:15:06 +08:00
hilimd 85570f27a0
Merge pull request #17 from chrislusf/master 
sync
 2020-09-23 11:39:57 +08:00
limd a99f63cb4d 1.add S3 copy directory function 
2.fixed the problem of empty directory when S3 deleted the directory
 2020-09-23 11:26:01 +08:00
Chris Lu b61d33f251 expose only store wrapper to meta changes 2020-09-22 16:27:36 -07:00
Chris Lu 0adbb56cc1 rename 2020-09-22 16:24:13 -07:00
Chris Lu 3e52329cee Revert "Merge pull request #1479 from LIBA-S/fix_oversized" 
This reverts commit bd11f0b3e4, reversing
changes made to ec5b9f1e91.
 2020-09-22 15:05:37 -07:00
Chris Lu 4a1fe4b8e2 add error logs 2020-09-22 09:16:10 -07:00
LIBA-S 8573ac82c6 Fix: remove the oversized state after compaction 2020-09-22 21:48:39 +08:00
James Hartig 658fc2e5b6 Allow option to enable volume pprof on server 2020-09-21 22:43:10 -04:00
James Hartig 91e4eca1e9 Fix deadlock with KeepConnected and SendHeartbeat 
There's the potential where we're writing to a clientConn and it goes away
and we're stuck keeping a read lock on clientChansLock. This causes
KeepConnected to not be able to remove the client since it requires a write
lock on clientChansLock. This ends up backing up SendHeartbeat because it
can't get a read lock.
 2020-09-21 22:41:38 -04:00
Chris Lu 9cdbfc1a49 refactor 2020-09-21 17:34:38 -07:00
Chris Lu 63373a9f9f filer: add file extended properties 2020-09-21 11:08:34 -07:00
Chris Lu b9c1f3e9de s3: fixes for list multipart upload 2020-09-21 10:51:24 -07:00
Chris Lu 289e62a305 master: better locking of in memory volume data 
related to https://github.com/chrislusf/seaweedfs/issues/1436#issuecomment-695880135
 2020-09-20 23:07:55 -07:00
Chris Lu 9a3b564508 adjust printout 2020-09-20 16:01:56 -07:00
Chris Lu 62563a895a refactoring 2020-09-20 16:00:01 -07:00
Chris Lu c2faab23b6 refactor 2020-09-20 15:40:49 -07:00
Chris Lu d013d09a9b adjust logging 2020-09-20 15:38:59 -07:00
Chris Lu f498c71199 shell: move volume operations to use flag parsing arguments 2020-09-20 09:27:34 -07:00
Chris Lu fcbc520373 fix print out 2020-09-20 08:47:30 -07:00
Chris Lu 56094541c8 shell: adjust command option for collection.delete 2020-09-20 08:46:16 -07:00
Chris Lu 9cac9c4001 2.00 2020-09-20 00:47:04 -07:00
Chris Lu dc4d2145ff open up 2020-09-19 20:23:12 -07:00
Chris Lu 29abe980df s3: add support for PostPolicy 
fix https://github.com/chrislusf/seaweedfs/issues/1426
 2020-09-19 20:14:19 -07:00
Chris Lu 41d508edfd go fmt 2020-09-19 14:10:26 -07:00
Chris Lu 5b40a2690a refactoring 2020-09-19 14:09:58 -07:00
Chris Lu 2c21eb1971 volume: get metrics configuration from master 
fix https://github.com/chrislusf/seaweedfs/issues/1354
 2020-09-19 00:03:00 -07:00
Chris Lu 2cbd1cf121 fix compilation 2020-09-18 00:15:54 -07:00