gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8948 commits 28 branches 278 tags 135 MiB
Commit graph

8948 commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu f2d9049e6a fix size variable 2022-10-23 13:05:37 -07:00
chrislu 184fbb6c50 volume server: remote tier volumes only soft delete in local index 
fix https://github.com/seaweedfs/seaweedfs/issues/3889
 2022-10-23 13:04:38 -07:00
liubaojiang 25471d579a
add md5 header when UploadData to replication in ReplicatedWrite (#3881) 2022-10-19 01:34:14 -07:00
dependabot[bot] 7e9c53bb5e
build(deps): bump docker/setup-buildx-action from 2.0.0 to 2.2.0 (#3876) 
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.0.0 to 2.2.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](dc7b9719a9...c74574e6c8)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 10:08:05 -07:00
dependabot[bot] 76b50a23ae
build(deps): bump google.golang.org/api from 0.98.0 to 0.99.0 (#3873) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.98.0 to 0.99.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.98.0...v0.99.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 10:07:32 -07:00
dependabot[bot] 9b4b2cb83b
build(deps): bump google.golang.org/grpc from 1.50.0 to 1.50.1 (#3874) 
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.50.0 to 1.50.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.50.0...v1.50.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:58:29 -07:00
dependabot[bot] 3911298b96
build(deps): bump modernc.org/sqlite from 1.19.1 to 1.19.2 (#3872) 
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.19.1 to 1.19.2.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.19.1...v1.19.2)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:58:09 -07:00
dependabot[bot] a87f147236
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.38.3 to 3.38.5 (#3871) 
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.38.3 to 3.38.5.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.38.3...v3.38.5)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:58:00 -07:00
dependabot[bot] 84b8f8f66d
build(deps): bump github.com/aws/aws-sdk-go from 1.44.114 to 1.44.116 (#3870) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.114 to 1.44.116.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.114...v1.44.116)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:51 -07:00
dependabot[bot] b0f80e0262
build(deps): bump docker/metadata-action from 4.0.1 to 4.1.0 (#3869) 
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](69f6fc9d46...12cce9efe0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:40 -07:00
dependabot[bot] 144ba3996a
build(deps): bump actions/dependency-review-action from 2.4.0 to 2.5.0 (#3868) 
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](375c537008...fd675ced9c)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:33 -07:00
dependabot[bot] 23d85745c6
build(deps): bump docker/build-push-action from 3.1.1 to 3.2.0 (#3867) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3.1.1 to 3.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](c84f382811...c56af95754)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:25 -07:00
dependabot[bot] 2fa43f6948
build(deps): bump docker/setup-qemu-action from 2.0.0 to 2.1.0 (#3866) 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](8b122486ce...e81a89b173)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:16 -07:00
dependabot[bot] b889f734ec
build(deps): bump docker/login-action from 2.0.0 to 2.1.0 (#3865) 
Bumps [docker/login-action](https://github.com/docker/login-action) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](49ed152c8e...f4ef78c080)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:06 -07:00
chrislu 758d70bdc0 3.32 2022-10-16 20:43:37 -07:00
chrislu de286fe662 shell: volume.move handles volume moved to cloud tier 
fix https://github.com/seaweedfs/seaweedfs/issues/3803
 2022-10-16 17:52:22 -07:00
chrislu 52e0a88a15 shell: volume.tier.upload clean up replicated copies 
fix https://github.com/seaweedfs/seaweedfs/issues/3804
 2022-10-16 17:02:56 -07:00
Ryan Russell 949607fe90
refactor(compose)!: upgrade to v2 closes #3699 (#3705) 
* docs(compose): Update readme for compose v2 upgrade

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade `Makefile` for compose v2

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade yml from version 2 -> 3.9

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-10-16 14:02:33 -07:00
Konstantin Lebedev e20f0dbd2d
avoid data race of TraverseBfs (#3856) 
* avoid data race of TraverseBfs

* close is enough
avoid panic
I1014 12:29:59.207120 volume_loading.go:131 loading sorted db /tmp/sw/test2_19.sdx error: unexpected file /tmp/sw/test2_19.idx size: 255
I1014 12:29:59.207125 volume_loading.go:119 open to write file /tmp/sw/test4_26.idx
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5260a4c]

goroutine 287 [running]:
github.com/seaweedfs/seaweedfs/weed/storage.(*SortedFileNeedleMap).Close(0x0)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/needle_map_sorted_file.go:97 +0x2c
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load.func1()
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:32 +0x8e
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load(0xc001b36280, 0x1, 0x1, 0x0, 0x69228c0?)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:205 +0x256c
github.com/seaweedfs/seaweedfs/weed/storage.NewVolume({0x7ffeefbff6e0, 0x7}, {0x7ffeefbff6e0, 0x7}, {0xc0009a9284, 0x5}, 0x13, 0x0, 0x0, 0x0, ...)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume.go:62 +0x30f
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).loadExistingVolume(0xc0006f40c0, {0x846c8d0, 0xc0009ce600}, 0x0?, 0x1)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:161 +0x4da
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes.func2()
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:201 +0xf9
created by github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:198 +0x150
 2022-10-16 11:30:41 -07:00
Konstantin Lebedev ee38ab8581
avoid data race of LogBuffer isStopping (#3859) 2022-10-16 11:28:56 -07:00
Konstantin Lebedev 7836f7574e
[volume.fsck] hotfix apply purging and add option verifyNeedle #3860 (#3861) 
* fix apply purging and add verifyNeedle

* common readSourceNeedleBlob

* use consts
 2022-10-15 20:38:46 -07:00
Konstantin Lebedev f476cf3403
fix volume_server.proto conflict (#3858) 
fix conflict

https://github.com/seaweedfs/seaweedfs/pull/3841
 2022-10-15 20:01:07 -07:00
chrislu 5c2eb602c9 Revert "mount: remove mount directory from generating cache directory id" 
This reverts commit 109dc7fdf6.
 2022-10-14 22:24:14 -07:00
Konstantin Lebedev 2f72103c83
avoid load volume file with BytesOffset mismatch (#3841) 
* avoid load volume file with BytesOffset mismatch

https://github.com/seaweedfs/seaweedfs/issues/2966

* set BytesOffset if has not VolumeInfoFile

* typos fail => failed

* exit if bytesOffset mismatch
 2022-10-14 00:18:09 -07:00
Konstantin Lebedev f19c9e3d9d
Volume fsck by volume (#3851) 
* refactor

* refactor args verbose and writer

* refactor readFilerFileIdFile

* fix filter by collectMtime

* skip system log collection
 2022-10-13 23:30:30 -07:00
chrislu 9c8678ded9 ensure monotonic n.AppendAtNs 
fix https://github.com/seaweedfs/seaweedfs/issues/3852
 2022-10-13 23:15:00 -07:00
chrislu e05637c42c Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-10-13 23:10:49 -07:00
chrislu dcd0743a35 remove unused ReadNeedleBlobRequest.needle_id 
fix https://github.com/seaweedfs/seaweedfs/issues/3853
 2022-10-13 23:10:46 -07:00
LHHDZ a5b867af69
don't skip empty value at AssembleEntryWithAcp (#3855) 
* add acl helper functionalities

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* add tests

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* remove 0 when create map

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* delete when empty at `AssembleEntryWithAcp`

`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp`  shouldn't skip empty value

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-13 22:50:44 -07:00
Konstantin Lebedev 1f7e52c63e
vacuum metrics and force sync dst files (#3832) 2022-10-13 00:51:20 -07:00
chrislu f5d4952d73 filer: redis store reduce from 2 redis operations to 1 for updates. 2022-10-12 23:50:09 -07:00
Guo Lei f95c25e113
types packages is imported more than onece (#3838) 2022-10-12 22:59:07 -07:00
chrislu 9a339a9cfb default hasSlowRead to true 2022-10-12 21:15:10 -07:00
famosss cacc3e883b
volume server:set the default value of "hasSlowRead" to true (#3710) 
* simplify a bit

* feat: volume: add "readBufSize" option to customize read optimization

* refactor : redbufSIze -> readBufferSize

* simplify a bit

* simplify a bit

* volume server:set the default value of "hasSlowRead" to true
 2022-10-12 21:13:26 -07:00
LHHDZ bf5e45b66a
add acl helper functionalities (#3831) 2022-10-12 20:15:16 -07:00
Konstantin Lebedev 401315f337
master fix interruption through ctrl+c (#3834) 2022-10-12 07:18:40 -07:00
zemul a05725aea6
filer: get directory metadata (#3833) 2022-10-12 07:14:49 -07:00
chrislu e55076c46f cloud tier: add retry when copying data file 
fix https://github.com/seaweedfs/seaweedfs/issues/3828
 2022-10-12 00:38:32 -07:00
chrislu cea73ac008 serialize sqlite operations 
fix https://github.com/seaweedfs/seaweedfs/issues/3827
 2022-10-12 00:03:27 -07:00
chrislu dff85e9c71 fix error handling 2022-10-12 00:03:14 -07:00
chrislu 8aec786a6d Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-10-11 21:58:19 -07:00
chrislu 0d817bc347 fix invalid memory address or nil pointer dereference on filer.sync 
fix https://github.com/seaweedfs/seaweedfs/issues/3826
 2022-10-11 21:58:17 -07:00
Guo Lei 84c401e693
Optimiz leveldb metric (#3830) 
* optimiz updating mapmetric for leveldb

* import loading leveldb

* add comments
 2022-10-11 21:13:25 -07:00
LHHDZ d21e2f523d
split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… (#3824) 
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-11 20:14:14 -07:00
livehl c34f04b35b
fix bug : read error when position after 2147483647 (#3829) 
Co-authored-by: huang.lin <hh@chaintool.ai>
 2022-10-11 17:50:12 -07:00
dependabot[bot] f6c8720ded
build(deps): bump google.golang.org/grpc from 1.49.0 to 1.50.0 (#3819) 
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.49.0 to 1.50.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.49.0...v1.50.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-10 12:15:51 -07:00
dependabot[bot] 50ed9456ff
build(deps): bump github.com/hashicorp/raft from 1.3.10 to 1.3.11 (#3821) 
Bumps [github.com/hashicorp/raft](https://github.com/hashicorp/raft) from 1.3.10 to 1.3.11.
- [Release notes](https://github.com/hashicorp/raft/releases)
- [Changelog](https://github.com/hashicorp/raft/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/raft/compare/v1.3.10...v1.3.11)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/raft
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-10 12:15:29 -07:00
dependabot[bot] 0aa947913a
build(deps): bump github.com/arangodb/go-driver from 1.3.3 to 1.4.0 (#3823) 
Bumps [github.com/arangodb/go-driver](https://github.com/arangodb/go-driver) from 1.3.3 to 1.4.0.
- [Release notes](https://github.com/arangodb/go-driver/releases)
- [Changelog](https://github.com/arangodb/go-driver/blob/master/CHANGELOG.md)
- [Commits](https://github.com/arangodb/go-driver/compare/v1.3.3...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/arangodb/go-driver
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-10 12:15:19 -07:00
dependabot[bot] 358d9fb109
build(deps): bump github.com/armon/go-metrics from 0.3.10 to 0.4.1 (#3822) 
Bumps [github.com/armon/go-metrics](https://github.com/armon/go-metrics) from 0.3.10 to 0.4.1.
- [Release notes](https://github.com/armon/go-metrics/releases)
- [Commits](https://github.com/armon/go-metrics/compare/v0.3.10...v0.4.1)

---
updated-dependencies:
- dependency-name: github.com/armon/go-metrics
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-10 12:01:49 -07:00
dependabot[bot] 3fbf68de12
build(deps): bump github.com/Shopify/sarama from 1.37.0 to 1.37.2 (#3818) 
Bumps [github.com/Shopify/sarama](https://github.com/Shopify/sarama) from 1.37.0 to 1.37.2.
- [Release notes](https://github.com/Shopify/sarama/releases)
- [Changelog](https://github.com/Shopify/sarama/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Shopify/sarama/compare/v1.37.0...v1.37.2)

---
updated-dependencies:
- dependency-name: github.com/Shopify/sarama
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-10 12:01:29 -07:00