Commit graph

8924 commits

Author SHA1 Message Date
chrislu 0452ae6a6c filer.sync: limit concurrency when fetching file chunks
fix https://github.com/seaweedfs/seaweedfs/issues/3787
2022-10-04 11:35:07 -07:00
Konstantin Lebedev ec46a34f33
avoid increasing the coordinates of metric labels (#3785) 2022-10-04 01:20:01 -07:00
dependabot[bot] 18d92dbd45
build(deps): bump github.com/aws/aws-sdk-go from 1.44.105 to 1.44.110 (#3783)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.105 to 1.44.110.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.105...v1.44.110)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-03 12:49:06 -07:00
dependabot[bot] 93d15488e4
build(deps): bump gocloud.dev/pubsub/rabbitpubsub from 0.26.0 to 0.27.0 (#3776)
Bumps [gocloud.dev/pubsub/rabbitpubsub](https://github.com/google/go-cloud) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/rabbitpubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-03 12:37:08 -07:00
dependabot[bot] 595db62fe7
build(deps): bump google.golang.org/api from 0.97.0 to 0.98.0 (#3780)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.97.0 to 0.98.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.97.0...v0.98.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-03 12:36:54 -07:00
dependabot[bot] 8ef802c259
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.38.1 to 3.38.3 (#3775)
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.38.1 to 3.38.3.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.38.1...v3.38.3)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-03 10:25:42 -07:00
dependabot[bot] 5b106630aa
build(deps): bump gocloud.dev/pubsub/natspubsub from 0.26.0 to 0.27.0 (#3777)
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-03 10:25:30 -07:00
dependabot[bot] 1cdbc8beb5
build(deps): bump github.com/Shopify/sarama from 1.36.0 to 1.37.0 (#3778) 2022-10-03 08:53:03 -07:00
dependabot[bot] d3f89af079
build(deps): bump github.com/google/flatbuffers from 2.0.8+incompatible to 22.9.29+incompatible (#3779) 2022-10-03 08:52:43 -07:00
chrislu 707abd5b2d 3.30 java 2022-10-02 18:54:39 -07:00
chrislu 29632d5a34 3.30 2022-10-02 18:13:44 -07:00
LHHDZ e9584d9661
add ownership rest apis (#3765) 2022-10-01 19:18:00 -07:00
Konstantin Lebedev 6fa3d0cc46
fix delete key panic of remote sync dir (#3770) 2022-10-01 02:33:47 -07:00
chrislu c3487d3da7 better fix for option format 2022-09-30 08:49:38 -07:00
chrislu ba0db28dbb mount: fix extra option format 2022-09-30 08:44:41 -07:00
chrislu 8e81619d02 mount: accept all extra mount options
fix https://github.com/seaweedfs/seaweedfs/issues/3767
2022-09-30 08:40:37 -07:00
kvtb 620be2be16
client-java: provide Content-MD5 when uploading chunks (#3761)
* client-java: provide Content-MD5 when uploading chunks

* Update SeaweedWrite.java
2022-09-29 22:56:12 -07:00
Eric Yang 56c94cc08e
ADHOC: filter deleted files from idx file binary search (#3763)
* ADHOC: filter deleted files from idx file binary search

* remove unwanted check

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-09-29 12:48:36 -07:00
LHHDZ 3de1e19780
s3: sync bucket info from filer (#3759) 2022-09-29 12:29:01 -07:00
kvtb 5e9039d728
client-java: handle server HTTP errors (#3762)
* client-java: handle server HTTP errors

* Update SeaweedWrite.java
2022-09-29 12:20:14 -07:00
LHHDZ e00a12b099
associate Account and Identity by accountId (#3754) 2022-09-28 13:25:59 -07:00
LHHDZ aacdcc4cad
s3: add account (#3753)
associate `Account` and `Identity` by accountId
2022-09-28 12:45:18 -07:00
Konstantin Lebedev 301b678147
[volume] Add new volumes to HUP(reload) signal (#3755)
Add new volumes to HUP(reload) signal
2022-09-28 12:44:13 -07:00
chrislu 47fd5d3fa1 skip debug message 2022-09-27 13:12:31 -07:00
Konstantin Lebedev 7ae51d1ec6
fix set master options for single server (#3708)
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-09-27 12:06:35 -07:00
jerebear12 190afcc3fd
Allow CORS on Filer (#3750)
Adjusted filer response headers
2022-09-27 11:47:52 -07:00
Konstantin Lebedev faa6167b6b
fs.meta.load load any dirs with prefix "important" (#3747)
* fs.meta.load load any dirs with prefix "important"

* replace dirPattern to dirPrefix

* help dirPrefix
2022-09-27 09:58:46 -07:00
chrislu b6d7556dda skip truncation on error
fix https://github.com/seaweedfs/seaweedfs/issues/3746
2022-09-27 09:48:23 -07:00
dependabot[bot] ea15476b06
build(deps): bump github.com/fclairamb/ftpserverlib from 0.19.1 to 0.20.0 (#3742)
build(deps): bump github.com/fclairamb/ftpserverlib

Bumps [github.com/fclairamb/ftpserverlib](https://github.com/fclairamb/ftpserverlib) from 0.19.1 to 0.20.0.
- [Release notes](https://github.com/fclairamb/ftpserverlib/releases)
- [Commits](https://github.com/fclairamb/ftpserverlib/compare/v0.19.1...v0.20.0)

---
updated-dependencies:
- dependency-name: github.com/fclairamb/ftpserverlib
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-26 09:04:12 -07:00
dependabot[bot] 53c7161f27
build(deps): bump github.com/aws/aws-sdk-go from 1.44.100 to 1.44.105 (#3744) 2022-09-26 08:43:27 -07:00
dependabot[bot] 30200cb6af
build(deps): bump google.golang.org/api from 0.96.0 to 0.97.0 (#3743) 2022-09-26 08:42:06 -07:00
dependabot[bot] b5d99110d1
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.38.0 to 3.38.1 (#3741) 2022-09-26 08:41:19 -07:00
dependabot[bot] 6b93148f9c
build(deps): bump cloud.google.com/go/storage from 1.26.0 to 1.27.0 (#3740) 2022-09-26 08:40:47 -07:00
dependabot[bot] d1357c1a3a
build(deps): bump actions/dependency-review-action from 2.1.0 to 2.4.0 (#3739) 2022-09-26 08:35:23 -07:00
chrislu 1ffb1e696e 3.29 2022-09-25 19:20:45 -07:00
chrislu 640c53cda8 add future list 2022-09-25 17:43:42 -07:00
chrislu cc570a4477 assert results are sorted 2022-09-25 13:49:28 -07:00
chrislu 5c8f1467a1 ordered execution async wait 2022-09-25 13:45:55 -07:00
chrislu 973b8ff067 rename 2022-09-25 13:34:55 -07:00
chrislu e31fdbc89b rename file 2022-09-25 12:59:38 -07:00
chrislu 31922b2bf2 s3 to watch specific directories 2022-09-20 09:30:05 -07:00
chrislu b90d92fca9 grpc: watch metadata changes by directory 2022-09-20 09:25:18 -07:00
chrislu b463ca1a2f filer replication: compare content changes directly
Fix https://github.com/seaweedfs/seaweedfs/issues/3714

The destination chunks may be empty. For example, the file is updated and the volume is vacuumed. In this case, the sync would miss the old chunks. This is fine. However, the entry would have correct metadata but missing chunks.

For this case, the simple metadata comparison would be wrongly skipping data changes, and the file will stay empty unless file content md5 is changed.
2022-09-20 08:35:10 -07:00
dependabot[bot] 023155bea5
build(deps): bump github.com/aws/aws-sdk-go from 1.44.95 to 1.44.100 (#3728)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.95 to 1.44.100.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.95...v1.44.100)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 11:17:46 -07:00
dependabot[bot] cdadf2f43d
build(deps): bump golang.org/x/tools from 0.1.10 to 0.1.12 (#3727)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.1.10 to 0.1.12.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.1.10...v0.1.12)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 09:19:31 -07:00
dependabot[bot] d0045c643e
build(deps): bump google.golang.org/api from 0.95.0 to 0.96.0 (#3726)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.95.0 to 0.96.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.95.0...v0.96.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 09:17:49 -07:00
dependabot[bot] 73c6de7de0
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.37.7 to 3.38.0 (#3725)
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.37.7 to 3.38.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.37.7...v3.38.0)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 09:17:18 -07:00
dependabot[bot] 5fce582bb7
build(deps): bump go.etcd.io/etcd/client/v3 from 3.5.4 to 3.5.5 (#3724)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.4 to 3.5.5.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Changelog](https://github.com/etcd-io/etcd/blob/main/Dockerfile-release.amd64)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.4...v3.5.5)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 09:17:08 -07:00
dependabot[bot] a0114c0a72
build(deps): bump github.com/klauspost/reedsolomon from 1.10.0 to 1.11.0 (#3723)
Bumps [github.com/klauspost/reedsolomon](https://github.com/klauspost/reedsolomon) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/klauspost/reedsolomon/releases)
- [Commits](https://github.com/klauspost/reedsolomon/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: github.com/klauspost/reedsolomon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 09:17:01 -07:00
dependabot[bot] 960902735f
build(deps): bump modernc.org/sqlite from 1.18.2 to 1.19.1 (#3722)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.18.2 to 1.19.1.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.18.2...v1.19.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19 09:16:51 -07:00