gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9252 commits 28 branches 278 tags 135 MiB
Commit graph

9252 commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Etienne G 7b90696601
Corrected the dashboard to use the new series name. (#3887) 
Corrected the dashboard to use the new metrics

The metric seems to have changed from SeaweedFS_filer_ to SeaweedFS_filerStore_. This commit replaces the ref in the dashboard with the new series name.
 2022-10-23 18:22:15 -07:00
Konstantin Lebedev 452202f35b
ReadNeedleMeta recovery panic occurred (#3883) 
ReadNeedleMeta panic occurred
https://github.com/seaweedfs/seaweedfs/issues/3882
 2022-10-23 14:00:28 -07:00
chrislu e3ea544d40 skip reading needle if size is invalid 
related to https://github.com/seaweedfs/seaweedfs/issues/3882
 2022-10-23 14:00:15 -07:00
chrislu 5294b6dd4d remove volume server tcp client 2022-10-23 13:15:03 -07:00
chrislu f2d9049e6a fix size variable 2022-10-23 13:05:37 -07:00
chrislu 184fbb6c50 volume server: remote tier volumes only soft delete in local index 
fix https://github.com/seaweedfs/seaweedfs/issues/3889
 2022-10-23 13:04:38 -07:00
liubaojiang 25471d579a
add md5 header when UploadData to replication in ReplicatedWrite (#3881) 2022-10-19 01:34:14 -07:00
dependabot[bot] 7e9c53bb5e
build(deps): bump docker/setup-buildx-action from 2.0.0 to 2.2.0 (#3876) 
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.0.0 to 2.2.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](dc7b9719a9...c74574e6c8)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 10:08:05 -07:00
dependabot[bot] 76b50a23ae
build(deps): bump google.golang.org/api from 0.98.0 to 0.99.0 (#3873) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.98.0 to 0.99.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.98.0...v0.99.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 10:07:32 -07:00
dependabot[bot] 9b4b2cb83b
build(deps): bump google.golang.org/grpc from 1.50.0 to 1.50.1 (#3874) 
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.50.0 to 1.50.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.50.0...v1.50.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:58:29 -07:00
dependabot[bot] 3911298b96
build(deps): bump modernc.org/sqlite from 1.19.1 to 1.19.2 (#3872) 
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.19.1 to 1.19.2.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.19.1...v1.19.2)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:58:09 -07:00
dependabot[bot] a87f147236
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.38.3 to 3.38.5 (#3871) 
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.38.3 to 3.38.5.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.38.3...v3.38.5)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:58:00 -07:00
dependabot[bot] 84b8f8f66d
build(deps): bump github.com/aws/aws-sdk-go from 1.44.114 to 1.44.116 (#3870) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.114 to 1.44.116.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.114...v1.44.116)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:51 -07:00
dependabot[bot] b0f80e0262
build(deps): bump docker/metadata-action from 4.0.1 to 4.1.0 (#3869) 
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](69f6fc9d46...12cce9efe0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:40 -07:00
dependabot[bot] 144ba3996a
build(deps): bump actions/dependency-review-action from 2.4.0 to 2.5.0 (#3868) 
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](375c537008...fd675ced9c)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:33 -07:00
dependabot[bot] 23d85745c6
build(deps): bump docker/build-push-action from 3.1.1 to 3.2.0 (#3867) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3.1.1 to 3.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](c84f382811...c56af95754)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:25 -07:00
dependabot[bot] 2fa43f6948
build(deps): bump docker/setup-qemu-action from 2.0.0 to 2.1.0 (#3866) 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](8b122486ce...e81a89b173)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:16 -07:00
dependabot[bot] b889f734ec
build(deps): bump docker/login-action from 2.0.0 to 2.1.0 (#3865) 
Bumps [docker/login-action](https://github.com/docker/login-action) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](49ed152c8e...f4ef78c080)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-17 09:57:06 -07:00
chrislu 758d70bdc0 3.32 2022-10-16 20:43:37 -07:00
chrislu de286fe662 shell: volume.move handles volume moved to cloud tier 
fix https://github.com/seaweedfs/seaweedfs/issues/3803
 2022-10-16 17:52:22 -07:00
chrislu 52e0a88a15 shell: volume.tier.upload clean up replicated copies 
fix https://github.com/seaweedfs/seaweedfs/issues/3804
 2022-10-16 17:02:56 -07:00
Ryan Russell 949607fe90
refactor(compose)!: upgrade to v2 closes #3699 (#3705) 
* docs(compose): Update readme for compose v2 upgrade

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade `Makefile` for compose v2

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade yml from version 2 -> 3.9

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-10-16 14:02:33 -07:00
Konstantin Lebedev e20f0dbd2d
avoid data race of TraverseBfs (#3856) 
* avoid data race of TraverseBfs

* close is enough
avoid panic
I1014 12:29:59.207120 volume_loading.go:131 loading sorted db /tmp/sw/test2_19.sdx error: unexpected file /tmp/sw/test2_19.idx size: 255
I1014 12:29:59.207125 volume_loading.go:119 open to write file /tmp/sw/test4_26.idx
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5260a4c]

goroutine 287 [running]:
github.com/seaweedfs/seaweedfs/weed/storage.(*SortedFileNeedleMap).Close(0x0)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/needle_map_sorted_file.go:97 +0x2c
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load.func1()
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:32 +0x8e
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load(0xc001b36280, 0x1, 0x1, 0x0, 0x69228c0?)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:205 +0x256c
github.com/seaweedfs/seaweedfs/weed/storage.NewVolume({0x7ffeefbff6e0, 0x7}, {0x7ffeefbff6e0, 0x7}, {0xc0009a9284, 0x5}, 0x13, 0x0, 0x0, 0x0, ...)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume.go:62 +0x30f
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).loadExistingVolume(0xc0006f40c0, {0x846c8d0, 0xc0009ce600}, 0x0?, 0x1)
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:161 +0x4da
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes.func2()
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:201 +0xf9
created by github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes
        /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:198 +0x150
 2022-10-16 11:30:41 -07:00
Konstantin Lebedev ee38ab8581
avoid data race of LogBuffer isStopping (#3859) 2022-10-16 11:28:56 -07:00
Konstantin Lebedev 7836f7574e
[volume.fsck] hotfix apply purging and add option verifyNeedle #3860 (#3861) 
* fix apply purging and add verifyNeedle

* common readSourceNeedleBlob

* use consts
 2022-10-15 20:38:46 -07:00
Konstantin Lebedev f476cf3403
fix volume_server.proto conflict (#3858) 
fix conflict

https://github.com/seaweedfs/seaweedfs/pull/3841
 2022-10-15 20:01:07 -07:00
chrislu 5c2eb602c9 Revert "mount: remove mount directory from generating cache directory id" 
This reverts commit 109dc7fdf6.
 2022-10-14 22:24:14 -07:00
Konstantin Lebedev 2f72103c83
avoid load volume file with BytesOffset mismatch (#3841) 
* avoid load volume file with BytesOffset mismatch

https://github.com/seaweedfs/seaweedfs/issues/2966

* set BytesOffset if has not VolumeInfoFile

* typos fail => failed

* exit if bytesOffset mismatch
 2022-10-14 00:18:09 -07:00
Konstantin Lebedev f19c9e3d9d
Volume fsck by volume (#3851) 
* refactor

* refactor args verbose and writer

* refactor readFilerFileIdFile

* fix filter by collectMtime

* skip system log collection
 2022-10-13 23:30:30 -07:00
chrislu 9c8678ded9 ensure monotonic n.AppendAtNs 
fix https://github.com/seaweedfs/seaweedfs/issues/3852
 2022-10-13 23:15:00 -07:00
chrislu e05637c42c Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-10-13 23:10:49 -07:00
chrislu dcd0743a35 remove unused ReadNeedleBlobRequest.needle_id 
fix https://github.com/seaweedfs/seaweedfs/issues/3853
 2022-10-13 23:10:46 -07:00
LHHDZ a5b867af69
don't skip empty value at AssembleEntryWithAcp (#3855) 
* add acl helper functionalities

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* add tests

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* remove 0 when create map

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* delete when empty at `AssembleEntryWithAcp`

`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp`  shouldn't skip empty value

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-13 22:50:44 -07:00
Konstantin Lebedev 1f7e52c63e
vacuum metrics and force sync dst files (#3832) 2022-10-13 00:51:20 -07:00
chrislu f5d4952d73 filer: redis store reduce from 2 redis operations to 1 for updates. 2022-10-12 23:50:09 -07:00
Guo Lei f95c25e113
types packages is imported more than onece (#3838) 2022-10-12 22:59:07 -07:00
chrislu 9a339a9cfb default hasSlowRead to true 2022-10-12 21:15:10 -07:00
famosss cacc3e883b
volume server:set the default value of "hasSlowRead" to true (#3710) 
* simplify a bit

* feat: volume: add "readBufSize" option to customize read optimization

* refactor : redbufSIze -> readBufferSize

* simplify a bit

* simplify a bit

* volume server:set the default value of "hasSlowRead" to true
 2022-10-12 21:13:26 -07:00
LHHDZ bf5e45b66a
add acl helper functionalities (#3831) 2022-10-12 20:15:16 -07:00
Konstantin Lebedev 401315f337
master fix interruption through ctrl+c (#3834) 2022-10-12 07:18:40 -07:00
zemul a05725aea6
filer: get directory metadata (#3833) 2022-10-12 07:14:49 -07:00
chrislu e55076c46f cloud tier: add retry when copying data file 
fix https://github.com/seaweedfs/seaweedfs/issues/3828
 2022-10-12 00:38:32 -07:00
chrislu cea73ac008 serialize sqlite operations 
fix https://github.com/seaweedfs/seaweedfs/issues/3827
 2022-10-12 00:03:27 -07:00
chrislu dff85e9c71 fix error handling 2022-10-12 00:03:14 -07:00
chrislu 8aec786a6d Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-10-11 21:58:19 -07:00
chrislu 0d817bc347 fix invalid memory address or nil pointer dereference on filer.sync 
fix https://github.com/seaweedfs/seaweedfs/issues/3826
 2022-10-11 21:58:17 -07:00
Guo Lei 84c401e693
Optimiz leveldb metric (#3830) 
* optimiz updating mapmetric for leveldb

* import loading leveldb

* add comments
 2022-10-11 21:13:25 -07:00
LHHDZ d21e2f523d
split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… (#3824) 
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-11 20:14:14 -07:00
livehl c34f04b35b
fix bug : read error when position after 2147483647 (#3829) 
Co-authored-by: huang.lin <hh@chaintool.ai>
 2022-10-11 17:50:12 -07:00
dependabot[bot] f6c8720ded
build(deps): bump google.golang.org/grpc from 1.49.0 to 1.50.0 (#3819) 
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.49.0 to 1.50.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.49.0...v1.50.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-10 12:15:51 -07:00