gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6935 commits 28 branches 278 tags 135 MiB
Commit graph

4610 commits

Author SHA1 Message Date
Radtoo fbb14e0ea8 Adding separate toggle to purge absent vols 
While this toggle is basically required to clean out entries for deleted volumes, having a separate description + toggling this separately seems like a good idea so people get a chance to check if their volumes are all mounted/connected as expected.

Also renamed forcePurge to just purge.
 2022-02-06 23:46:52 +01:00
Radtoo 6b17f45da2 Also delete paths for which a volume is entirely absent. 2022-02-06 23:22:04 +01:00
Radtoo fa0cfdfb7f Added basic deletion capabilities to fsck. 2022-02-06 17:05:58 +01:00
chrislu 433fde4b18 move error to a separate file 
This file contains metric names for all errors
The naming convention is ErrorSomeThing = "error.some.thing"
 2022-02-04 22:57:51 -08:00
Chris Lu a23fcb9a7c
Merge pull request #2634 from kmlebedev/errorMetrics 
error metrics for filer and store
 2022-02-04 22:35:13 -08:00
Chris Lu 247bbabda5
Merge pull request #2632 from lapshin-vitaly/s3api_errors 
add s3api error for copy in file, not directory
 2022-02-04 22:34:34 -08:00
chrislu ced3b89395 add util package 2022-02-04 21:34:58 -08:00
chrislu affe3c2c12 change to util.WriteFile 2022-02-04 21:32:27 -08:00
chrislu 76e297d64f sync call to write file, avoid vif loading error 
fix https://github.com/chrislusf/seaweedfs/issues/2633
 2022-02-04 11:14:04 -08:00
zerospiel f3364fec99 weed/s3api: rearrange s3 methods handlers to ensure correct methods requesting 
Otherwise current calls for some methods (i.e. GetObjectAcl) ends up with wrong method selection (i.e. GetObject).

Added generic comment rule of traversing methods
 2022-02-04 15:14:48 +03:00
Konstantin Lebedev 9978f54acf fix metric names 2022-02-04 16:45:16 +05:00
Konstantin Lebedev 3f4e17aa24 error metrics for filer and store 2022-02-04 14:07:14 +05:00
Lapshinn Vitaly 6bdc274d4d add s3api error for copy in file, not directory 2022-02-04 03:28:37 +03:00
chrislu 6cf2e7d493 s3: listObjectParts return ErrNoSuchUpload if does not exist 
ubuntu@prod-master-1:~$ aws --endpoint http://10.244.15.66:8333 s3api abort-multipart-upload --bucket prod-cache --key multipart-test --upload-id 5347f936-6adc-43de-8e5c-1fd137c3b2bc
ubuntu@prod-master-1:~$ aws --endpoint http://10.244.15.66:8333 s3api list-parts --bucket prod-cache --key multipart-test --upload-id 5347f936-6adc-43de-8e5c-1fd137c3b2bc
{
    "Initiator": null,
    "Owner": null,
    "StorageClass": "STANDARD"
}

If we abort a multipart upload, it appears that records are left behind. We should get a 404 NoSuchKey error.
 2022-02-03 12:34:16 -08:00
zerospiel b54a65ba5a weed/s3api: added new bucket handlers for more compatibility with AWS S3 
Protocol

Otherwise any requests to the underlying handlers results in calls to
ListObjects (v1) that may intensively load gateway and volume servers.

Added the following handlers with default responses:
- GetBucketLocation
- GetBucketRequestPayment

Added the following handlers with NotFound and NotImplemented responses:
- PutBucketAcl
- GetBucketPolicy
- PutBucketPolicy
- DeleteBucketPolicy
- GetBucketCors
- PutBucketCors
- DeleteBucketCors
 2022-02-03 17:17:05 +03:00
chrislu 6bee1e9714 [volume.check.disk] fix wrong logic to compare 
fix https://github.com/chrislusf/seaweedfs/issues/2626

minuend - subtrahend
 2022-02-01 07:48:28 -08:00
chrislu 7270067289 2.88 2022-01-30 20:25:26 -08:00
chrislu 84c9bc4389 edge case: old entry was not replicated to remote storage 2022-01-30 20:23:24 -08:00
chrislu b8490fe427 adjust volume count even when not applying the changes 2022-01-28 19:11:46 -08:00
Chris Lu b3f0f170b4
Merge pull request #2618 from divanikus/master 2022-01-28 03:34:52 -08:00
chrislu b9b684194f remove max connection age 
following https://github.com/grpc/grpc-go/issues/3170#issuecomment-552517779
 2022-01-27 02:28:22 -08:00
divanikus 67e3fe996a async volumeTierMove 2022-01-26 18:22:31 +03:00
chrislu 62d815d1ca use limited in memory buffer instead of swap file 2022-01-26 00:36:19 -08:00
Chris Lu 9596fce562
Merge pull request #2613 from kmlebedev/masterPromMetrics 
Master prom metrics
 2022-01-25 08:13:14 -08:00
Konstantin Lebedev 5c9259fa3c fix metrics master name 2022-01-25 14:42:47 +05:00
chrislu 7328962009 revert "POSIX: should not delete if a directory is not empty" 
revert 0c75f15062

Reported:

Hi, about commit: 0c75f15062
POSIX: should not delete if a directory is not empty
It should still delete with a command like rm -rf "${path}"/  because it is a forced delete, but now it gets fail to delete non-empty folder: [...]
Can you enable the delete if it is forced?
 2022-01-24 14:21:50 -08:00
Konstantin Lebedev c9952759c4 metrics master is leader 2022-01-24 20:13:07 +05:00
Konstantin Lebedev 28efe31524 new master metrics 2022-01-24 19:09:43 +05:00
chrislu 3bba2124ef use a sliding window of in-memory writable chunks 2022-01-23 23:02:05 -08:00
chrislu 520591e6ea reset swap file chunk after uploading 2022-01-23 18:30:53 -08:00
chrislu e185d90d24 2.87 2022-01-23 16:18:55 -08:00
chrislu 4a311c7f5e dedup local metadata subscribers 
fix https://github.com/chrislusf/seaweedfs/discussions/2542
 2022-01-23 16:14:22 -08:00
chrislu bb6854b972 adjust log 2022-01-22 12:35:09 -08:00
chrislu 2bdd737971 rename file 2022-01-22 08:28:35 -08:00
chrislu 643bbbeb49 rename 2022-01-22 08:27:40 -08:00
chrislu 18ed06b420 use memory when under 16 chunks 2022-01-22 08:11:01 -08:00
chrislu 8e80f3cd65 move upload pipeline locking to a different file 2022-01-22 08:09:55 -08:00
chrislu dde34fa99d rename 2022-01-22 08:08:01 -08:00
chrislu a2aa542370 rename 2022-01-22 08:07:11 -08:00
chrislu c376ccc5a5 swap file based random write large file upload 2022-01-22 08:06:53 -08:00
chrislu b9ae16fbc5 fix memory allocation 2022-01-22 08:05:04 -08:00
chrislu 1ee828b768 refactor 
do not expose internal offset
 2022-01-22 06:34:29 -08:00
chrislu 02d0c12cdd rename 2022-01-22 06:00:10 -08:00
chrislu 5dea5c0449 refactor 2022-01-22 05:59:07 -08:00
chrislu 8aa6bf0bb9 refactoring 2022-01-22 05:40:10 -08:00
chrislu d97bd54e63 just refactoring 2022-01-22 04:18:54 -08:00
chrislu 482014f9da rename file 2022-01-22 03:50:18 -08:00
chrislu 3b4a9addaf rename 2022-01-22 01:46:10 -08:00
chrislu 4acfc098e9 re-order 2022-01-22 01:43:14 -08:00
chrislu e71dcfb3a6 add logging for memory allocation 2022-01-22 01:35:12 -08:00