gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9118 commits 28 branches 278 tags 135 MiB
Commit graph

9118 commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot] 9b42942cca
build(deps): bump google.golang.org/api from 0.105.0 to 0.106.0 (#4122) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.105.0 to 0.106.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.105.0...v0.106.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-09 08:24:43 -08:00
Guo Lei 83e29faa55
use time.NewTicker instead of time.Tick. (#4119) 2023-01-09 06:47:08 -08:00
chrislu 51d4a4b28d 3.39 2023-01-08 18:20:22 -08:00
chrislu 0b52f08226 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-01-08 18:03:25 -08:00
chrislu 469c959e6a add missing options for webdav 2023-01-08 18:03:22 -08:00
monchickey 3e2c9ea73d
Add image cropping. (#4117) 2023-01-06 09:28:07 -08:00
chrislu 296fdc296c mount: faster add chunks 2023-01-06 01:03:29 -08:00
chrislu 49b84d5866 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-01-05 11:19:42 -08:00
chrislu 2abf817580 fix for stream reader 
fix https://github.com/seaweedfs/seaweedfs/issues/4112
 2023-01-05 11:19:21 -08:00
Tobias Gurtzick 9ffe1d6aec
add code of conduct (#4109) 
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
 2023-01-05 11:01:22 -08:00
chrislu e367444586 add notes for full version 2023-01-04 09:52:25 -08:00
chrislu 5423790b2c add back previous chunk upload selection algo 2023-01-03 22:50:39 -08:00
chrislu 7bdae5172e batch delete EC needles 
fix https://github.com/seaweedfs/seaweedfs/issues/4107
 2023-01-03 22:05:26 -08:00
chrislu 8aec430df7 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-01-03 12:13:54 -08:00
chrislu c87a3ffce3 avoid hard coded versions 2023-01-03 12:13:52 -08:00
zemul 6b4c033431
add mount log (#4101) 
* filer.backup use replication.source.filer

* add mount log

* Revert "filer.backup use replication.source.filer"

This reverts commit 07bf6f956c.

* fix

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2023-01-03 00:00:45 -08:00
chrislu 48f2edc065 3.38 2023-01-02 23:23:02 -08:00
Chris Lu d4566d4aaa
more solid weed mount (#4089) 
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
 2023-01-02 23:20:45 -08:00
dependabot[bot] 367353b936
build(deps): bump github.com/aws/aws-sdk-go from 1.44.167 to 1.44.171 (#4100) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.167 to 1.44.171.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.167...v1.44.171)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-02 12:24:46 -08:00
dependabot[bot] de787ce4ab
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.41.0 to 3.42.1 (#4099) 
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.41.0 to 3.42.1.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.41.0...v3.42.1)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-02 12:24:28 -08:00
dependabot[bot] 04bba1a6b2
build(deps): bump modernc.org/sqlite from 1.20.0 to 1.20.1 (#4098) 
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-02 12:24:15 -08:00
CommanderRoot c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 (#4094) 2023-01-01 05:07:53 -08:00
CommanderRoot c2280e94cf
filer.store.mysql: Replace deprecated upsert syntax (#4096) 2023-01-01 05:06:57 -08:00
CommanderRoot 265a56630b
filer.store.mysql: Escape table columns in SQL query (#4095) 2023-01-01 05:06:41 -08:00
Guo Lei ed5f3f073b
add more help message, in case of misunderstanding (#4092) 2022-12-29 07:05:05 -08:00
Guo Lei d8cfa1552b
support enable/disable vacuum (#4087) 
* stop vacuum

* suspend/resume vacuum

* remove unused code

* rename

* rename param
 2022-12-28 01:36:44 -08:00
dependabot[bot] 84492976c9
build(deps): bump github.com/aws/aws-sdk-go from 1.44.162 to 1.44.167 (#4085) 2022-12-26 11:48:35 -08:00
dependabot[bot] bd176359cb
build(deps): bump github.com/prometheus/procfs from 0.8.0 to 0.9.0 (#4084) 2022-12-26 11:48:17 -08:00
dependabot[bot] b2aa1a23ff
build(deps): bump github.com/arangodb/go-driver from 1.4.0 to 1.4.1 (#4083) 2022-12-26 11:47:52 -08:00
Guo Lei 5f993a5f20
Fix ide warning (#4082) 
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
 2022-12-26 00:03:47 -08:00
fibbanachi 3901f2da4d
Add logtostderr option in helm chart (#4080) 2022-12-23 07:36:15 -08:00
Konstantin Lebedev 569bc7b54d
fs.verify needles status of file (#4060) 
* fs.verify needles status of file

* fix parse args

* refactor logging

* set modifyTimeAgoAtSec
 2022-12-21 15:54:17 -08:00
chrislu 77b7c9f7cc lock and unlock chunks for read 2022-12-21 14:15:35 -08:00
chrislu d1797deccb remove duplicated logic of removing from writableChunks 2022-12-21 14:15:11 -08:00
chrislu 4878e0fd46 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-12-20 19:47:23 -08:00
chrislu 6ede19e825 add a simple file replication progress bar 2022-12-20 19:47:21 -08:00
Naden 27d187eed9
Fixes 4074: S3 Gateway fails to start when installed via Helm (#4075) 
Fixes #4074: S3 Gateway fails to start when installed via Helm

Co-authored-by: Naden <naden@harana.com>
 2022-12-20 17:24:54 -08:00
Naden 3b90eb1911
Fixes: 4076: Host path prefix should be configurable (#4077) 
Co-authored-by: Naden <naden@harana.com>
 2022-12-20 17:24:12 -08:00
chrislu f7beba8515 unlock before submitting the uploading jobs 2022-12-20 12:52:30 -08:00
chrislu 28d479e5c0 mount: adjust locking for upload pipeline 2022-12-19 15:07:22 -08:00
chrislu f9fb3cd1fc Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-12-19 11:32:00 -08:00
chrislu 6c7fe40305 filer sink retries reading file chunks, skipping missing chunks 
if the file chunk is not available during replication time, the file is skipped
 2022-12-19 11:31:58 -08:00
chrislu 3fc3e7083c metadata skip reading un-available logs 
this means the volumes for metadata histories have been lost.
 2022-12-19 11:30:50 -08:00
dependabot[bot] a1ffa03d75
build(deps): bump google.golang.org/api from 0.104.0 to 0.105.0 (#4067) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.104.0 to 0.105.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.104.0...v0.105.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 10:21:20 -08:00
dependabot[bot] 3886c31ed8
build(deps): bump github.com/aws/aws-sdk-go from 1.44.146 to 1.44.162 (#4069) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.146 to 1.44.162.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.146...v1.44.162)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 09:22:36 -08:00
dependabot[bot] 9fdf98b826
build(deps): bump cloud.google.com/go/pubsub from 1.27.1 to 1.28.0 (#4068) 
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.27.1...pubsub/v1.28.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 09:22:23 -08:00
dependabot[bot] c7a28b20b5
build(deps): bump actions/checkout from 3.1.0 to 3.2.0 (#4066) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](93ea575cb5...755da8c3cf)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 09:22:12 -08:00
dependabot[bot] 6c5d44a5c4
build(deps): bump actions/setup-go from 3.4.0 to 3.5.0 (#4065) 
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](d0a58c1c4d...6edd4406fa)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 09:21:49 -08:00
dependabot[bot] ae847c2780
build(deps): bump actions/dependency-review-action from 3.0.1 to 3.0.2 (#4064) 
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 3.0.1 to 3.0.2.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](11310527b4...0ff3da6f81)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 09:21:41 -08:00
dependabot[bot] 3b85dc5fbb
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.40.1 to 3.41.0 (#4070) 
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.40.1 to 3.41.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.40.1...v3.41.0)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 09:21:30 -08:00