Eric Yang
51d462f204
ADHOC: volume fsck using append at ns ( #3906 )
...
* ADHOC: volume fsck using append at ns
* nit
* nit
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-10-24 22:09:38 -07:00
famosss
34132b2c9f
fix: isTruncated returns problematic ( #3907 )
2022-10-24 21:15:37 -07:00
famosss
874fd197b5
feat: simplify a bit ( #3905 )
2022-10-24 17:58:26 -07:00
famosss
25e012d30b
fix: set user metadata key to lowercase ( #3894 )
...
* fix: set user metadata key to lowercase
* feat: simplify a bit
2022-10-24 17:29:52 -07:00
chrislu
091f999e8c
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-10-24 16:50:41 -07:00
chrislu
377870f4a9
keep system log data
2022-10-24 16:50:39 -07:00
dependabot[bot]
131c70a7b4
build(deps): bump google.golang.org/api from 0.99.0 to 0.100.0 ( #3902 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.99.0 to 0.100.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.99.0...v0.100.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 09:45:12 -07:00
dependabot[bot]
88f8926317
build(deps): bump github.com/aws/aws-sdk-go from 1.44.116 to 1.44.121 ( #3900 )
...
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go ) from 1.44.116 to 1.44.121.
- [Release notes](https://github.com/aws/aws-sdk-go/releases )
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.116...v1.44.121 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 09:39:08 -07:00
dependabot[bot]
f922347c69
build(deps): bump golang.org/x/tools from 0.1.12 to 0.2.0 ( #3903 )
...
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.1.12 to 0.2.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.1.12...v0.2.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 09:38:49 -07:00
dependabot[bot]
94db6fb715
build(deps): bump actions/setup-go from 3.3.0 to 3.3.1 ( #3896 )
...
Bumps [actions/setup-go](https://github.com/actions/setup-go ) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/actions/setup-go/releases )
- [Commits](268d8c0ca0...c4a742cab1
2022-10-24 09:38:30 -07:00
dependabot[bot]
70897ab538
build(deps): bump github.com/stretchr/testify from 1.8.0 to 1.8.1 ( #3901 )
...
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify ) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/stretchr/testify/releases )
- [Commits](https://github.com/stretchr/testify/compare/v1.8.0...v1.8.1 )
---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 09:38:21 -07:00
dependabot[bot]
ceb92a2fc0
build(deps): bump github.com/linxGnu/grocksdb from 1.7.7 to 1.7.10 ( #3899 )
...
Bumps [github.com/linxGnu/grocksdb](https://github.com/linxGnu/grocksdb ) from 1.7.7 to 1.7.10.
- [Release notes](https://github.com/linxGnu/grocksdb/releases )
- [Commits](https://github.com/linxGnu/grocksdb/compare/v1.7.7...v1.7.10 )
---
updated-dependencies:
- dependency-name: github.com/linxGnu/grocksdb
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 09:38:12 -07:00
dependabot[bot]
207ed12af9
build(deps): bump docker/setup-buildx-action from 2.2.0 to 2.2.1 ( #3898 )
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](c74574e6c8...8c0edbc76e
2022-10-24 09:38:04 -07:00
dependabot[bot]
8ed0d827a0
build(deps): bump docker/metadata-action from 4.1.0 to 4.1.1 ( #3897 )
...
Bumps [docker/metadata-action](https://github.com/docker/metadata-action ) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/docker/metadata-action/releases )
- [Commits](12cce9efe0...57396166ad
2022-10-24 09:37:58 -07:00
dependabot[bot]
fcf715817b
build(deps): bump actions/dependency-review-action from 2.5.0 to 2.5.1 ( #3895 )
...
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action ) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/actions/dependency-review-action/releases )
- [Commits](fd675ced9c...0efb1d1d84
2022-10-24 09:37:51 -07:00
Guo Lei
97edb40275
Fix errinfo ( #3893 )
...
* types packages is imported more than onece
* Fix error response when format of --expires is wrong.
It MUST be in RFC 1123 date format.
2022-10-24 09:25:48 -07:00
chrislu
d5364218b2
adjust help message
2022-10-24 09:25:08 -07:00
chrislu
d880fc2bb3
fix merge
2022-10-23 18:26:22 -07:00
Konstantin Lebedev
6253058d9d
ensure monotonic n.AppendAtNs in each place ( #3880 )
...
https://github.com/seaweedfs/seaweedfs/issues/3852
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-10-23 18:24:52 -07:00
Etienne G
7b90696601
Corrected the dashboard to use the new series name. ( #3887 )
...
Corrected the dashboard to use the new metrics
The metric seems to have changed from SeaweedFS_filer_ to SeaweedFS_filerStore_. This commit replaces the ref in the dashboard with the new series name.
2022-10-23 18:22:15 -07:00
Konstantin Lebedev
452202f35b
ReadNeedleMeta recovery panic occurred ( #3883 )
...
ReadNeedleMeta panic occurred
https://github.com/seaweedfs/seaweedfs/issues/3882
2022-10-23 14:00:28 -07:00
chrislu
e3ea544d40
skip reading needle if size is invalid
...
related to https://github.com/seaweedfs/seaweedfs/issues/3882
2022-10-23 14:00:15 -07:00
chrislu
5294b6dd4d
remove volume server tcp client
2022-10-23 13:15:03 -07:00
chrislu
f2d9049e6a
fix size variable
2022-10-23 13:05:37 -07:00
chrislu
184fbb6c50
volume server: remote tier volumes only soft delete in local index
...
fix https://github.com/seaweedfs/seaweedfs/issues/3889
2022-10-23 13:04:38 -07:00
liubaojiang
25471d579a
add md5 header when UploadData to replication in ReplicatedWrite ( #3881 )
2022-10-19 01:34:14 -07:00
dependabot[bot]
7e9c53bb5e
build(deps): bump docker/setup-buildx-action from 2.0.0 to 2.2.0 ( #3876 )
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 2.0.0 to 2.2.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](dc7b9719a9...c74574e6c8
2022-10-17 10:08:05 -07:00
dependabot[bot]
76b50a23ae
build(deps): bump google.golang.org/api from 0.98.0 to 0.99.0 ( #3873 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.98.0 to 0.99.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.98.0...v0.99.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 10:07:32 -07:00
dependabot[bot]
9b4b2cb83b
build(deps): bump google.golang.org/grpc from 1.50.0 to 1.50.1 ( #3874 )
...
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go ) from 1.50.0 to 1.50.1.
- [Release notes](https://github.com/grpc/grpc-go/releases )
- [Commits](https://github.com/grpc/grpc-go/compare/v1.50.0...v1.50.1 )
---
updated-dependencies:
- dependency-name: google.golang.org/grpc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 09:58:29 -07:00
dependabot[bot]
3911298b96
build(deps): bump modernc.org/sqlite from 1.19.1 to 1.19.2 ( #3872 )
...
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite ) from 1.19.1 to 1.19.2.
- [Release notes](https://gitlab.com/cznic/sqlite/tags )
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.19.1...v1.19.2 )
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 09:58:09 -07:00
dependabot[bot]
a87f147236
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.38.3 to 3.38.5 ( #3871 )
...
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.38.3 to 3.38.5.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.38.3...v3.38.5 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 09:58:00 -07:00
dependabot[bot]
84b8f8f66d
build(deps): bump github.com/aws/aws-sdk-go from 1.44.114 to 1.44.116 ( #3870 )
...
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go ) from 1.44.114 to 1.44.116.
- [Release notes](https://github.com/aws/aws-sdk-go/releases )
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.114...v1.44.116 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 09:57:51 -07:00
dependabot[bot]
b0f80e0262
build(deps): bump docker/metadata-action from 4.0.1 to 4.1.0 ( #3869 )
...
Bumps [docker/metadata-action](https://github.com/docker/metadata-action ) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/docker/metadata-action/releases )
- [Commits](69f6fc9d46...12cce9efe0
2022-10-17 09:57:40 -07:00
dependabot[bot]
144ba3996a
build(deps): bump actions/dependency-review-action from 2.4.0 to 2.5.0 ( #3868 )
...
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action ) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases )
- [Commits](375c537008...fd675ced9c
2022-10-17 09:57:33 -07:00
dependabot[bot]
23d85745c6
build(deps): bump docker/build-push-action from 3.1.1 to 3.2.0 ( #3867 )
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 3.1.1 to 3.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](c84f382811...c56af95754
2022-10-17 09:57:25 -07:00
dependabot[bot]
2fa43f6948
build(deps): bump docker/setup-qemu-action from 2.0.0 to 2.1.0 ( #3866 )
...
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action ) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases )
- [Commits](8b122486ce...e81a89b173
2022-10-17 09:57:16 -07:00
dependabot[bot]
b889f734ec
build(deps): bump docker/login-action from 2.0.0 to 2.1.0 ( #3865 )
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](49ed152c8e...f4ef78c080
2022-10-17 09:57:06 -07:00
chrislu
758d70bdc0
3.32
2022-10-16 20:43:37 -07:00
chrislu
de286fe662
shell: volume.move handles volume moved to cloud tier
...
fix https://github.com/seaweedfs/seaweedfs/issues/3803
2022-10-16 17:52:22 -07:00
chrislu
52e0a88a15
shell: volume.tier.upload clean up replicated copies
...
fix https://github.com/seaweedfs/seaweedfs/issues/3804
2022-10-16 17:02:56 -07:00
Ryan Russell
949607fe90
refactor(compose)!: upgrade to v2 closes #3699 ( #3705 )
...
* docs(compose): Update readme for compose v2 upgrade
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(compose): Upgrade `Makefile` for compose v2
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(compose): Upgrade yml from version 2 -> 3.9
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-10-16 14:02:33 -07:00
Konstantin Lebedev
e20f0dbd2d
avoid data race of TraverseBfs ( #3856 )
...
* avoid data race of TraverseBfs
* close is enough
avoid panic
I1014 12:29:59.207120 volume_loading.go:131 loading sorted db /tmp/sw/test2_19.sdx error: unexpected file /tmp/sw/test2_19.idx size: 255
I1014 12:29:59.207125 volume_loading.go:119 open to write file /tmp/sw/test4_26.idx
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5260a4c]
goroutine 287 [running]:
github.com/seaweedfs/seaweedfs/weed/storage.(*SortedFileNeedleMap).Close(0x0)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/needle_map_sorted_file.go:97 +0x2c
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load.func1()
/Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:32 +0x8e
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load(0xc001b36280, 0x1, 0x1, 0x0, 0x69228c0?)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:205 +0x256c
github.com/seaweedfs/seaweedfs/weed/storage.NewVolume({0x7ffeefbff6e0, 0x7}, {0x7ffeefbff6e0, 0x7}, {0xc0009a9284, 0x5}, 0x13, 0x0, 0x0, 0x0, ...)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/volume.go:62 +0x30f
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).loadExistingVolume(0xc0006f40c0, {0x846c8d0, 0xc0009ce600}, 0x0?, 0x1)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:161 +0x4da
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes.func2()
/Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:201 +0xf9
created by github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes
/Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:198 +0x150
2022-10-16 11:30:41 -07:00
Konstantin Lebedev
ee38ab8581
avoid data race of LogBuffer isStopping ( #3859 )
2022-10-16 11:28:56 -07:00
Konstantin Lebedev
7836f7574e
[volume.fsck] hotfix apply purging and add option verifyNeedle #3860 ( #3861 )
...
* fix apply purging and add verifyNeedle
* common readSourceNeedleBlob
* use consts
2022-10-15 20:38:46 -07:00
Konstantin Lebedev
f476cf3403
fix volume_server.proto conflict ( #3858 )
...
fix conflict
https://github.com/seaweedfs/seaweedfs/pull/3841
2022-10-15 20:01:07 -07:00
chrislu
5c2eb602c9
Revert "mount: remove mount directory from generating cache directory id"
...
This reverts commit 109dc7fdf6
2022-10-14 22:24:14 -07:00
Konstantin Lebedev
2f72103c83
avoid load volume file with BytesOffset mismatch ( #3841 )
...
* avoid load volume file with BytesOffset mismatch
https://github.com/seaweedfs/seaweedfs/issues/2966
* set BytesOffset if has not VolumeInfoFile
* typos fail => failed
* exit if bytesOffset mismatch
2022-10-14 00:18:09 -07:00
Konstantin Lebedev
f19c9e3d9d
Volume fsck by volume ( #3851 )
...
* refactor
* refactor args verbose and writer
* refactor readFilerFileIdFile
* fix filter by collectMtime
* skip system log collection
2022-10-13 23:30:30 -07:00
chrislu
9c8678ded9
ensure monotonic n.AppendAtNs
...
fix https://github.com/seaweedfs/seaweedfs/issues/3852
2022-10-13 23:15:00 -07:00
chrislu
e05637c42c
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-10-13 23:10:49 -07:00
