Commit graph

8543 commits

Author SHA1 Message Date
chrislu 2b580a7566 also migrate jsonpb 2022-08-17 12:42:03 -07:00
chrislu eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu e259052a70 volume: fix metrics for volume request time 2022-08-17 04:29:31 -07:00
qzh 7fcfaf7bc9
fix(filer.sync): offset may be set to 0 (#3451)
* fix(filer.sync): initializing the offset is related to the path

* fix(filer.sync): the offset maybe to be set to 0.

Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 23:43:52 -07:00
qzh 400f0c3e5d
fix(filer.sync): initializing the offset is related to the path (#3450)
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 21:56:47 -07:00
chrislu fa4d0093e1 3.22 2022-08-15 16:48:23 -07:00
chrislu 93261f5199 Revert "Refactor for Sync method (#3426)"
This reverts commit 670cb759f8.

with the pr
weed/storage () - (master) > go test -count=1 ./...
ok  	github.com/seaweedfs/seaweedfs/weed/storage	18.486s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map	0.025s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding	0.864s
?   	github.com/seaweedfs/seaweedfs/weed/storage/idx	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle	0.110s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle_map	24.414s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/super_block	0.203s
?   	github.com/seaweedfs/seaweedfs/weed/storage/types	[no test files]
?   	github.com/seaweedfs/seaweedfs/weed/storage/volume_info	[no test files]
weed/storage () - (master) >
weed/storage () - (master) >

without the pr
weed/storage () - (master) >
weed/storage () - (master) > go test -count=1 ./...
ok  	github.com/seaweedfs/seaweedfs/weed/storage	1.617s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map	0.026s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding	0.906s
?   	github.com/seaweedfs/seaweedfs/weed/storage/idx	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle	0.202s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle_map	24.533s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/super_block	0.280s
?   	github.com/seaweedfs/seaweedfs/weed/storage/types	[no test files]
?   	github.com/seaweedfs/seaweedfs/weed/storage/volume_info	[no test files]
2022-08-15 15:35:31 -07:00
chrislu 9c588d4010 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-15 10:03:56 -07:00
chrislu 96caf21d09 less verbose log 2022-08-15 10:03:52 -07:00
dependabot[bot] 55f90302a7
Bump google.golang.org/api from 0.90.0 to 0.92.0 (#3449) 2022-08-15 08:40:17 -07:00
dependabot[bot] 21106002be
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.32.1 to 3.33.0 (#3448) 2022-08-15 08:39:49 -07:00
dependabot[bot] 8affcf070c
Bump github.com/linxGnu/grocksdb from 1.7.4 to 1.7.5 (#3447) 2022-08-15 08:39:24 -07:00
dependabot[bot] 3723eb09c0
Bump cloud.google.com/go/storage from 1.24.0 to 1.25.0 (#3446) 2022-08-15 08:38:55 -07:00
dependabot[bot] 4d8bbd559a
Bump github.com/tidwall/gjson from 1.14.1 to 1.14.2 (#3445) 2022-08-15 08:38:19 -07:00
dependabot[bot] 373736b984
Bump github.com/aws/aws-sdk-go from 1.44.66 to 1.44.76 (#3444) 2022-08-15 08:37:45 -07:00
Andrey Triumfov 31faa6d43d
Remove duplicate slashes in object path to prevent 500 errors (#3442) 2022-08-15 08:19:28 -07:00
chrislu cb476a53ff remove logs 2022-08-15 01:05:35 -07:00
chrislu 7c029b2183 3.21 2022-08-15 00:32:15 -07:00
Chris Lu e88392b50f
Fix s3 pagination (#3436)
* Revert previous changes

* s3: use cursor to track tree traversal

fix https://github.com/seaweedfs/seaweedfs/issues/3166

* special cases for empty prefix and empty directory

* use constants

* address empty folder

* undo local changes

* fix IsTruncated

* adjust counting directories

* fix cases when prefix is a directory

* s3: handle directory object

works for

aws --endpoint-url http://127.0.0.1:8333/ s3api list-objects-v2 --bucket test --prefix "fakedir"
2022-08-15 00:30:19 -07:00
chrislu 7457c746f0 s3: fix aws s3api head-object 2022-08-14 23:52:35 -07:00
chrislu 789d9dbfe1 webdav: fix nil
fix https://github.com/seaweedfs/seaweedfs/issues/3440
2022-08-14 23:33:30 -07:00
Konstantin Lebedev 6199d9b923
rand start garbageThreshold and reset metric ReplicaPlacementMismatch (#3396)
* rand start garbageThreshold
reset metrics MasterReplicaPlacementMismatch

* vacuum through sleep
2022-08-12 13:11:09 -07:00
Konstantin Lebedev d8c54b527e
avoid Dockerfile Entrypoint: "file not found" (#3430)
https://www.koskila.net/how-to-fix-exec-user-process-caused-no-such-file-or-directory-in-docker-on-windows/
2022-08-11 15:07:44 -07:00
Konstantin Lebedev 4d4cd0948d
avoid infinite loop WaitUntilConnected() (#3431)
https://github.com/seaweedfs/seaweedfs/issues/3421
2022-08-11 15:03:26 -07:00
dependabot[bot] 53b601616c
Bump hadoop-common from 3.2.3 to 3.2.4 in /other/java/hdfs-over-ftp (#3433)
Bumps hadoop-common from 3.2.3 to 3.2.4.

---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-11 14:40:19 -07:00
dependabot[bot] 710d2a6f16
Bump hadoop-common from 3.2.3 to 3.2.4 in /other/java/hdfs3 (#3432)
Bumps hadoop-common from 3.2.3 to 3.2.4.

---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-11 14:40:09 -07:00
Eric Yang 986daecbfe
add upload response (#3428)
* add upload response

* fixing comments

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-08-10 22:43:09 -07:00
chrislu ae93c966d9 ensure memory is aligned
fix https://github.com/seaweedfs/seaweedfs/issues/3427
2022-08-10 22:27:13 -07:00
Rain Li 670cb759f8
Refactor for Sync method (#3426) 2022-08-10 08:21:57 -07:00
chrislu 662ec97602 mount: adjust on du stats reporting
fix https://github.com/seaweedfs/seaweedfs/issues/3424
2022-08-09 20:21:01 -07:00
Chris Lu 9b006250b2
Update pull_request_template.md 2022-08-09 01:25:55 -07:00
Abirdcfly b0633716b7
delete minor unreachable code (#3423)
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-09 01:24:43 -07:00
Chris Lu 33a5225302
Update README.md 2022-08-08 15:58:36 -07:00
dependabot[bot] 1d8645b262
Bump github.com/prometheus/client_golang from 1.12.2 to 1.13.0 (#3420)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.12.2 to 1.13.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.12.2...v1.13.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 10:06:15 -07:00
dependabot[bot] 16709e8ed3
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.30.0 to 3.32.1 (#3419)
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.30.0 to 3.32.1.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.30.0...v3.32.1)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 10:00:06 -07:00
dependabot[bot] 02a7df86f3
Bump gocloud.dev/pubsub/natspubsub from 0.25.0 to 0.26.0 (#3418)
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 09:59:54 -07:00
dependabot[bot] 64af8c8e53
Bump go.mongodb.org/mongo-driver from 1.10.0 to 1.10.1 (#3416)
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.10.0 to 1.10.1.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.10.0...v1.10.1)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 09:59:47 -07:00
dependabot[bot] aec57b8d6e
Bump github.com/hashicorp/raft from 1.3.9 to 1.3.10 (#3417)
Bumps [github.com/hashicorp/raft](https://github.com/hashicorp/raft) from 1.3.9 to 1.3.10.
- [Release notes](https://github.com/hashicorp/raft/releases)
- [Changelog](https://github.com/hashicorp/raft/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/raft/compare/v1.3.9...v1.3.10)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/raft
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 09:59:33 -07:00
dependabot[bot] da167e3134
Bump docker/build-push-action from 3.1.0 to 3.1.1 (#3415)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](1cb9d22b93...c84f382811)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 09:59:19 -07:00
chrislu 0854171d22 3.20 2022-08-07 14:42:28 -07:00
chrislu d8e69df295 adjust comments 2022-08-07 11:35:33 -07:00
chrislu 0aeec04c31 quicker to adapt to pattern change 2022-08-07 10:14:01 -07:00
chrislu 928d29af9e fix wrong logic about reader isRandomMode() 2022-08-07 10:13:04 -07:00
chrislu d3d52b1818 shell: fs.meta.load add quieter mode
fix https://github.com/seaweedfs/seaweedfs/issues/3414
2022-08-07 01:42:15 -07:00
chrislu 358ccb5e0e Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-07 01:34:36 -07:00
chrislu 67814a5c79 refactor and fix strings.Split 2022-08-07 01:34:32 -07:00
John W Higgins 3afda0c89c
Allow postgresql to use standard environment variables for connection (#3413) 2022-08-07 00:58:53 -07:00
chrislu 1a4bf0dcb5 filer.sync: parallelize the filer.sync 2022-08-07 00:56:15 -07:00
chrislu 02b38f894b debug 2022-08-07 00:55:49 -07:00
chrislu 0e9478488d filer.sync: fix when excluded paths is empty 2022-08-07 00:55:34 -07:00