Commit graph

9652 commits

Author SHA1 Message Date
chrislu 340aea9b34 add broker leader 2023-09-18 09:01:14 -07:00
chrislu 0d390ec882 fix nil 2023-09-18 08:52:35 -07:00
chrislu 4e17d399a6 refactor 2023-09-17 00:04:20 -07:00
chrislu 7a49d3541a avoid nil 2023-09-17 00:04:13 -07:00
chrislu a65088d739 broker report stats to balancer 2023-09-16 23:00:51 -07:00
chrislu a1cb49d078 adjust log 2023-09-16 23:00:28 -07:00
chrislu b590a68e75 revert to non-streaming mode to assign file id
fix https://github.com/seaweedfs/seaweedfs/issues/4838#issuecomment-1722290685
2023-09-16 16:13:08 -07:00
chrislu 4650c4c65f Revert "turn on streaming assign file id"
This reverts commit 733db2bc88.
2023-09-16 16:09:20 -07:00
chrislu fa62926347 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-16 15:06:22 -07:00
chrislu 89a1fd1751 Squashed commit of the following:
commit 4827425146
Author: chrislu <chris.lu@gmail.com>
Date:   Sat Sep 16 15:05:38 2023 -0700

    balancer works

commit 3b50139f68
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 15 22:22:32 2023 -0700

    comments

commit 7f685ce7ba
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 15 22:20:05 2023 -0700

    adjust APIs

commit 436d99443b
Author: chrislu <chris.lu@gmail.com>
Date:   Thu Sep 14 23:49:05 2023 -0700

    receive broker stats

commit b771fefa37
Merge: 0a851ec00 890881037
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 13 00:03:47 2023 -0700

    Merge branch 'master' into sub

commit 0a851ec00b
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Sep 10 22:01:25 2023 -0700

    Create balancer.go

commit 39941edc0b
Author: chrislu <chris.lu@gmail.com>
Date:   Thu Sep 7 23:55:19 2023 -0700

    add publisher shutdown

commit 875f562779
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 23:16:41 2023 -0700

    server side send response at least once per second

commit 984b6c54cf
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 23:15:29 2023 -0700

    ack interval 128

commit 2492a45499
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 22:39:46 2023 -0700

    ack interval

commit ba67e6ca29
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Sep 4 21:43:50 2023 -0700

    api for sub

commit 9e4f985698
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Sep 4 21:43:30 2023 -0700

    publish, benchmark

commit cb470d44df
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 1 00:36:51 2023 -0700

    can pub and sub

commit 1eb2da46d5
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Aug 28 09:02:12 2023 -0700

    connect and publish

commit 504ae8383a
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Aug 28 09:01:25 2023 -0700

    protoc version

commit dbcba75271
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 18:59:04 2023 -0700

    rename to lookup

commit c9caf33119
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 18:33:46 2023 -0700

    move functions

commit 4d6c18d86f
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 17:50:59 2023 -0700

    pub sub initial tests

commit 4eb8e8624d
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 13:14:39 2023 -0700

    rename

commit 1990456670
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 13:13:14 2023 -0700

    sub

commit 905911853d
Author: chrislu <chris.lu@gmail.com>
Date:   Sat Aug 26 13:39:21 2023 -0700

    adjust proto
2023-09-16 15:06:16 -07:00
wang wusong 9e5b1eef9b avoid passe lock by value 2023-09-14 06:51:12 -07:00
annluda 8908810376 Fix s3 api object list with params max-keys and prefix 2023-09-11 21:35:15 -07:00
dependabot[bot] 9b666a53ed Bump github.com/aws/aws-sdk-go-v2/config from 1.18.33 to 1.18.39
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.33 to 1.18.39.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.33...config/v1.18.39)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 13:04:52 -07:00
dependabot[bot] 962461fc33 Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.35 to 1.13.37
Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.13.35 to 1.13.37.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.13.35...credentials/v1.13.37)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:33:00 -07:00
dependabot[bot] b28595d8db Bump google.golang.org/grpc from 1.57.0 to 1.58.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.57.0 to 1.58.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.57.0...v1.58.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:32:05 -07:00
dependabot[bot] d99d99a911 Bump github.com/google/uuid from 1.3.0 to 1.3.1
Bumps [github.com/google/uuid](https://github.com/google/uuid) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/google/uuid/releases)
- [Changelog](https://github.com/google/uuid/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/uuid/compare/v1.3.0...v1.3.1)

---
updated-dependencies:
- dependency-name: github.com/google/uuid
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:31:56 -07:00
dependabot[bot] 324f3fbb15 Bump modernc.org/mathutil from 1.5.0 to 1.6.0
Bumps [modernc.org/mathutil](https://gitlab.com/cznic/mathutil) from 1.5.0 to 1.6.0.
- [Commits](https://gitlab.com/cznic/mathutil/compare/v1.5.0...v1.6.0)

---
updated-dependencies:
- dependency-name: modernc.org/mathutil
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:31:39 -07:00
dependabot[bot] c097994f63 Bump docker/build-push-action from 4.1.1 to 4.2.1
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4.1.1 to 4.2.1.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](2eb1c1961a...0a97817b6a)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:31:31 -07:00
dependabot[bot] e3a7c503c3 Bump actions/dependency-review-action from 3.0.8 to 3.1.0
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 3.0.8 to 3.1.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](f6fff72a32...6c5ccdad46)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:31:13 -07:00
chrislu 5407b86766 3.56 2023-09-10 22:05:41 -07:00
chrislu 07c2d75ac7 ensure single threaded access 2023-09-10 22:03:44 -07:00
Lars Lehtonen f24c7e803f weed/storage/erasure_coding: Close() after error handling 2023-09-06 07:35:54 -07:00
Lars Lehtonen d762b12e11 weed/storage/erasure_coding: fix dropped test error 2023-09-06 07:35:54 -07:00
Lars Lehtonen a879e1bbb5 weed/storage/erasure_coding: remove unused err from encodeDatFile() signature 2023-09-06 07:35:54 -07:00
dependabot[bot] c3810520fc Bump golang.org/x/sys from 0.11.0 to 0.12.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/sys/compare/v0.11.0...v0.12.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 10:35:38 -07:00
dependabot[bot] f555bb0ca4 Bump github.com/fclairamb/ftpserverlib from 0.21.0 to 0.22.0
Bumps [github.com/fclairamb/ftpserverlib](https://github.com/fclairamb/ftpserverlib) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/fclairamb/ftpserverlib/releases)
- [Commits](https://github.com/fclairamb/ftpserverlib/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: github.com/fclairamb/ftpserverlib
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 10:35:32 -07:00
Patrick Schmidt 98dcec0ee2 Clean up old signature hash pools 2023-09-05 10:33:27 -07:00
Patrick Schmidt cdd817edf9 Improve S3 request signing performance
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
2023-09-05 10:33:27 -07:00
dependabot[bot] f07876cb23
Bump actions/checkout from 3 to 4 (#4812)
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...3df4ab11eba7bda6032a0b82a6bb43b11571feac)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 10:03:49 -07:00
dependabot[bot] f3dfeb3735
Bump github.com/tidwall/gjson from 1.14.4 to 1.16.0 (#4811)
Bumps [github.com/tidwall/gjson](https://github.com/tidwall/gjson) from 1.14.4 to 1.16.0.
- [Commits](https://github.com/tidwall/gjson/compare/v1.14.4...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/tidwall/gjson
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 10:03:41 -07:00
dependabot[bot] 41ebc9c3cb
Bump github.com/puzpuzpuz/xsync/v2 from 2.4.1 to 2.5.0 (#4810)
Bumps [github.com/puzpuzpuz/xsync/v2](https://github.com/puzpuzpuz/xsync) from 2.4.1 to 2.5.0.
- [Release notes](https://github.com/puzpuzpuz/xsync/releases)
- [Commits](https://github.com/puzpuzpuz/xsync/compare/v2.4.1...v2.5.0)

---
updated-dependencies:
- dependency-name: github.com/puzpuzpuz/xsync/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 10:03:33 -07:00
dependabot[bot] 5c613f18ad
Bump github.com/go-redsync/redsync/v4 from 4.8.1 to 4.8.2 (#4807)
Bumps [github.com/go-redsync/redsync/v4](https://github.com/go-redsync/redsync) from 4.8.1 to 4.8.2.
- [Release notes](https://github.com/go-redsync/redsync/releases)
- [Commits](https://github.com/go-redsync/redsync/compare/v4.8.1...v4.8.2)

---
updated-dependencies:
- dependency-name: github.com/go-redsync/redsync/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 10:03:03 -07:00
fibbanachi 7374ceecda
improve k8s probes templating in helm chart (#4805) 2023-09-02 13:26:27 -07:00
Lars Lehtonen e6385e5e8b
weed/util: fix dropped errors (#4803) 2023-09-01 13:10:45 -07:00
ginn13 f139a56cb9
Add master servers to grafana dashboard (#4801) 2023-08-31 06:42:07 -07:00
Dmitriy Pavlov de1dcf3b2f
[heml] fix default affinity and allow owerride ingress annotations (#4799) 2023-08-29 08:43:11 -07:00
Alexandre Viau 271c51f273
helm: allow setting master.toml config (#4797) 2023-08-28 20:44:09 -07:00
dependabot[bot] 03389bf844
Bump github.com/gocql/gocql from 1.5.2 to 1.6.0 (#4794)
Bumps [github.com/gocql/gocql](https://github.com/gocql/gocql) from 1.5.2 to 1.6.0.
- [Release notes](https://github.com/gocql/gocql/releases)
- [Changelog](https://github.com/gocql/gocql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gocql/gocql/compare/v1.5.2...v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/gocql/gocql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 09:18:44 -07:00
dependabot[bot] 1faf9afb91
Bump modernc.org/sqlite from 1.24.0 to 1.25.0 (#4793)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.24.0 to 1.25.0.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.24.0...v1.25.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 09:18:35 -07:00
dependabot[bot] cf44633491
Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.34 to 1.13.35 (#4792)
Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.13.34 to 1.13.35.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.13.34...credentials/v1.13.35)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 09:18:22 -07:00
Alexandre Viau db5d23defe
k8s: allow configuring serviceAccountName (#4790) 2023-08-28 09:06:06 -07:00
dependabot[bot] ac1b144526
Bump gocloud.dev from 0.33.0 to 0.34.0 (#4795)
Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.33.0 to 0.34.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.33.0...v0.34.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 09:04:59 -07:00
dependabot[bot] 67f1bb69c4
Bump github.com/jinzhu/copier from 0.3.5 to 0.4.0 (#4791)
Bumps [github.com/jinzhu/copier](https://github.com/jinzhu/copier) from 0.3.5 to 0.4.0.
- [Commits](https://github.com/jinzhu/copier/compare/v0.3.5...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/jinzhu/copier
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 09:04:10 -07:00
dependabot[bot] 47fae7d0eb
Bump docker/setup-buildx-action from 2.9.1 to 2.10.0 (#4789) 2023-08-28 08:33:51 -07:00
Dmitriy Pavlov 56aef67912
[heml] complete redesign of labels (#4788) 2023-08-28 07:37:05 -07:00
Dmitriy Pavlov 3d45ecd326
[helm] fix log to stderr and default host for ingress (#4787) 2023-08-27 13:53:28 -07:00
Nico D'Cotta 796b7508f3
Implement SRV lookups for filer (#4767) 2023-08-24 07:08:56 -07:00
tikhon-opsfleet 5251b4d50e
[helm] add possibility to use array for imagePullSecrets (#4782)
add possibility to use array for imagePullSecrets
2023-08-23 01:01:41 -07:00
chrislu 733db2bc88 turn on streaming assign file id 2023-08-23 00:34:21 -07:00
chrislu 99f037b958 streaming assign file ids 2023-08-23 00:31:33 -07:00