gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9932 commits 28 branches 278 tags 135 MiB
Commit graph

6381 commits

Author SHA1 Message Date
chrislu 0fd7222d65 default to skip if less than 4 nodes 2023-10-05 11:13:48 -07:00
chrislu 7b3c63a8af handle wrapped errors 2023-10-05 08:15:11 -07:00
chrislu 4e493d596c minor 2023-10-05 08:13:53 -07:00
Lars Lehtonen 530bdedf76
weed/operation: fix dropped error (#4888) 2023-10-03 10:55:20 -07:00
chrislu 6ebe26a765 Revert "Revert "Revert "Add disk type to prometheus metrics" (#4777)"" 
This reverts commit 567d788928.
 2023-10-03 08:28:52 -07:00
chrislu 7540d43ee9 Revert "Revert "fix compilation"" 
This reverts commit f9abfd0b03.
 2023-10-03 08:28:02 -07:00
chrislu 249c0e06ef Revert "fix compilation" 
This reverts commit 451ec6504d.
 2023-10-03 08:27:50 -07:00
chrislu 451ec6504d fix compilation 2023-10-03 08:15:18 -07:00
chrislu f9abfd0b03 Revert "fix compilation" 
This reverts commit 0483ba3889.
 2023-10-03 07:58:11 -07:00
chrislu 0483ba3889 fix compilation 2023-10-02 11:55:50 -07:00
chrislu 567d788928 Revert "Revert "Add disk type to prometheus metrics" (#4777)" 
This reverts commit 9215ba24be.
 2023-10-02 11:49:54 -07:00
chrislu b5070d6280 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-10-02 10:52:40 -07:00
Konstantin Lebedev dffe00a822
fix: logger place msg (#4880) 2023-10-02 08:29:09 -07:00
chrislu bf13f3ced7 add subscriber coordinator 2023-10-02 01:02:27 -07:00
chrislu 734178093e refactor TopicPartition struct 2023-10-02 01:01:45 -07:00
chrislu 2a578b9033 refactor 2023-10-01 11:59:19 -07:00
chrislu 321fda552c comment 2023-10-01 11:36:27 -07:00
chrislu b02fdeabff RetryForever => RetryUntil 2023-10-01 11:33:56 -07:00
chrislu ebd6f96d35 adjust mq.proto 2023-09-30 21:47:57 -07:00
chrislu 02cb799481 add a test framework for grpc 2023-09-30 21:36:58 -07:00
chrislu c4e6545197 fix compilation 2023-09-30 21:26:09 -07:00
chrislu 358cba43ef update proto generated files 2023-09-30 13:19:25 -07:00
chrislu 597ab1e4bf add design for mq balancer 2023-09-30 13:19:05 -07:00
chrislu d7c211fe8c adjust mq proto 2023-09-30 13:18:49 -07:00
chrislu 40888b5612 align 64bit atomic operation 
fix https://github.com/seaweedfs/seaweedfs/issues/4878
 2023-09-28 07:58:43 -07:00
Konstantin Lebedev dd580190b4
fix: avoid deleting one replica without sync (#4875) 
* fix: avoid deleting one replica without sync
https://github.com/seaweedfs/seaweedfs/issues/4647

* Update weed/shell/command_volume_fix_replication.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* fix: revert this existing do option to positive

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-27 23:12:10 -07:00
Konstantin Lebedev 44906f1f3b
fix: avoid error file name too long when writing a file (#4876) 2023-09-27 05:40:51 -07:00
chrislu 9d589b48e6 rename function 2023-09-26 15:17:33 -07:00
Konstantin Lebedev df4ded758e
fix: avoid deleting more than one replica (#4873) 
https://github.com/seaweedfs/seaweedfs/issues/4647

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-26 00:20:48 -07:00
chrislu 19505c1cf4 describe a topic 2023-09-25 20:46:00 -07:00
Tobias Gurtzick 78dbac7702
fix(ec): volumes created by foreign collection due to bug in ec balance (#4864) 
* fix(ec): ignore 0 byte data files

refers to parts of #4861

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* fix(ec): ignore volumes not from the current collection during balance

fixes #4861

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

---------

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
 2023-09-25 10:35:43 -07:00
chrislu 645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"" 
This reverts commit 8cb42c39
 2023-09-25 09:35:16 -07:00
chrislu 3d07895518 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-25 09:16:35 -07:00
chrislu a6e35e4f68 fix compilation 2023-09-25 09:16:33 -07:00
Konstantin Lebedev f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-25 08:34:12 -07:00
chrislu c9177c92e5 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-25 08:20:16 -07:00
chrislu 81e57bd45e fix test 2023-09-25 08:20:03 -07:00
Lars Lehtonen 28a3a31b27
weed/replication/sub: fix dropped error (#4865) 2023-09-25 07:33:18 -07:00
chrislu 23f334dd13 comment 2023-09-24 23:36:43 -07:00
chrislu c7e05e4e71 ensure latest stats are reported 2023-09-24 23:34:31 -07:00
chrislu dff2ce5d2f read existing broker stats 2023-09-24 23:10:33 -07:00
chrislu 1492bf7552 fix listing topics 2023-09-24 23:05:41 -07:00
chrislu 3cf9b8d621 create topic and report topic 2023-09-24 22:00:43 -07:00
chrislu b3f94feede do create topic 2023-09-24 22:00:43 -07:00
chrislu d74348048a implement create topic 2023-09-24 22:00:43 -07:00
chrislu 0361c321b4 add CreateTopic API 2023-09-24 22:00:43 -07:00
chrislu 0f8168c0c9 3.57 2023-09-24 21:29:31 -07:00
chrislu 1df4223195 minor 2023-09-24 09:54:57 -07:00
chrislu 10ac25f857 avoid deleting the entry too early 2023-09-23 12:43:00 -07:00
chrislu c43238b30a fix waiting condition 2023-09-23 12:28:02 -07:00
chrislu 322af1fc04 fix compilation 2023-09-23 12:13:32 -07:00
chrislu 0414b68a4f if shared lock, still need to wait for exclusive lock to release 2023-09-23 12:08:23 -07:00
chrislu 9ba4639a27 adjust for better logs 2023-09-23 11:53:44 -07:00
chrislu 186b1f029b read source uses shared lock 2023-09-23 10:33:44 -07:00
chrislu 2df70ce95f adjust logs 2023-09-21 23:51:32 -07:00
chrislu 3e23421608 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-21 20:08:30 -07:00
chrislu 49a470bcdf reduce logs 2023-09-21 20:08:20 -07:00
Konstantin Lebedev 750a0ba1b2
[iam] Replace action read/write to readAcp/writeAcp for handlers with acl (#4858) 
Replace action read/write to readAcp/writeAcp for handlers with acl query
 https://github.com/seaweedfs/seaweedfs/issues/4519

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-21 14:07:04 -07:00
chrislu 8b2c39f2c0 reduce binary size 2023-09-21 13:22:08 -07:00
chrislu b388957340 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-21 11:08:30 -07:00
chrislu 31fc165715 mount switch to ordered lock requests 2023-09-21 11:08:26 -07:00
chrislu 7669852241 adjust logs 2023-09-21 11:04:12 -07:00
chrislu e3b1bacf3f add lock table to ensure ordered locks 2023-09-21 10:24:34 -07:00
Konstantin Lebedev a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856) 
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-21 08:20:05 -07:00
Konstantin Lebedev d8b424d123
[s3] optimization iam lookup for reducing algorithm complexity (#4857) 
optimization iam lookup for reducing algorithm complexity
https://github.com/seaweedfs/seaweedfs/issues/4519

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-21 08:19:11 -07:00
chrislu 411bdda08d fix compilation 2023-09-19 16:22:41 -07:00
chrislu d531b7cbbf merge some previous missing changes 2023-09-19 14:10:54 -07:00
chrislu b18112ae10 fix compilation bugs during merge 2023-09-19 14:08:17 -07:00
chrislu 6c6655de9d wip 2023-09-19 14:03:42 -07:00
chrislu cf28108ea4 fix grpd dial option 2023-09-19 14:02:51 -07:00
chrislu 27af11f1e8 Revert "Revert "Merge branch 'master' into sub"" 
This reverts commit 0bb97709d4.
 2023-09-18 18:47:34 -07:00
chrislu 0bb97709d4 Revert "Merge branch 'master' into sub" 
This reverts commit 4d414f54a2, reversing
changes made to 4827425146.
 2023-09-18 16:13:20 -07:00
chrislu 8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs" 
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
 2023-09-18 16:12:50 -07:00
chrislu 2e5aa06026 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-18 16:06:41 -07:00
chrislu 4d414f54a2 Merge branch 'master' into sub 2023-09-18 16:04:59 -07:00
dependabot[bot] a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850) 
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2023-09-18 14:43:05 -07:00
chrislu b64fc44e8d Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-18 09:01:17 -07:00
chrislu 340aea9b34 add broker leader 2023-09-18 09:01:14 -07:00
chrislu 0d390ec882 fix nil 2023-09-18 08:52:35 -07:00
Lars Lehtonen d83107efe5 fix: weed/storage: dropped error 2023-09-18 08:18:57 -07:00
zemul 91cbaef02b fix: mount proxyByFiler write error: wrong jwt 2023-09-18 07:44:50 -07:00
chrislu 4e17d399a6 refactor 2023-09-17 00:04:20 -07:00
chrislu 7a49d3541a avoid nil 2023-09-17 00:04:13 -07:00
chrislu a65088d739 broker report stats to balancer 2023-09-16 23:00:51 -07:00
chrislu a1cb49d078 adjust log 2023-09-16 23:00:28 -07:00
chrislu b590a68e75 revert to non-streaming mode to assign file id 
fix https://github.com/seaweedfs/seaweedfs/issues/4838#issuecomment-1722290685
 2023-09-16 16:13:08 -07:00
chrislu 4650c4c65f Revert "turn on streaming assign file id" 
This reverts commit 733db2bc88.
 2023-09-16 16:09:20 -07:00
chrislu fa62926347 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-16 15:06:22 -07:00
chrislu 89a1fd1751 Squashed commit of the following: 
commit 4827425146
Author: chrislu <chris.lu@gmail.com>
Date:   Sat Sep 16 15:05:38 2023 -0700

    balancer works

commit 3b50139f68
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 15 22:22:32 2023 -0700

    comments

commit 7f685ce7ba
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 15 22:20:05 2023 -0700

    adjust APIs

commit 436d99443b
Author: chrislu <chris.lu@gmail.com>
Date:   Thu Sep 14 23:49:05 2023 -0700

    receive broker stats

commit b771fefa37
Merge: 0a851ec00 890881037
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 13 00:03:47 2023 -0700

    Merge branch 'master' into sub

commit 0a851ec00b
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Sep 10 22:01:25 2023 -0700

    Create balancer.go

commit 39941edc0b
Author: chrislu <chris.lu@gmail.com>
Date:   Thu Sep 7 23:55:19 2023 -0700

    add publisher shutdown

commit 875f562779
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 23:16:41 2023 -0700

    server side send response at least once per second

commit 984b6c54cf
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 23:15:29 2023 -0700

    ack interval 128

commit 2492a45499
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 22:39:46 2023 -0700

    ack interval

commit ba67e6ca29
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Sep 4 21:43:50 2023 -0700

    api for sub

commit 9e4f985698
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Sep 4 21:43:30 2023 -0700

    publish, benchmark

commit cb470d44df
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 1 00:36:51 2023 -0700

    can pub and sub

commit 1eb2da46d5
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Aug 28 09:02:12 2023 -0700

    connect and publish

commit 504ae8383a
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Aug 28 09:01:25 2023 -0700

    protoc version

commit dbcba75271
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 18:59:04 2023 -0700

    rename to lookup

commit c9caf33119
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 18:33:46 2023 -0700

    move functions

commit 4d6c18d86f
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 17:50:59 2023 -0700

    pub sub initial tests

commit 4eb8e8624d
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 13:14:39 2023 -0700

    rename

commit 1990456670
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 13:13:14 2023 -0700

    sub

commit 905911853d
Author: chrislu <chris.lu@gmail.com>
Date:   Sat Aug 26 13:39:21 2023 -0700

    adjust proto
 2023-09-16 15:06:16 -07:00
chrislu 4827425146 balancer works 2023-09-16 15:05:38 -07:00
chrislu 3b50139f68 comments 2023-09-15 22:22:32 -07:00
chrislu 7f685ce7ba adjust APIs 2023-09-15 22:20:05 -07:00
chrislu 436d99443b receive broker stats 2023-09-14 23:49:05 -07:00
wang wusong 9e5b1eef9b avoid passe lock by value 2023-09-14 06:51:12 -07:00
chrislu b771fefa37 Merge branch 'master' into sub 2023-09-13 00:03:47 -07:00
annluda 8908810376 Fix s3 api object list with params max-keys and prefix 2023-09-11 21:35:15 -07:00
chrislu 5407b86766 3.56 2023-09-10 22:05:41 -07:00
chrislu 07c2d75ac7 ensure single threaded access 2023-09-10 22:03:44 -07:00
chrislu 0a851ec00b Create balancer.go 2023-09-10 22:01:25 -07:00
chrislu 39941edc0b add publisher shutdown 2023-09-07 23:55:19 -07:00