Commit graph

8692 commits

Author SHA1 Message Date
chrislu 39340f7e42 cloud tier: s3 consume all read response body
fix https://github.com/seaweedfs/seaweedfs/issues/3584
2022-09-04 22:23:28 -07:00
chrislu 7c6324b114 adjust log level 2022-09-04 22:21:24 -07:00
chrislu 8c5759a06d minor optimization 2022-09-04 18:50:45 -07:00
chrislu 70777d9b9f Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-09-04 18:26:51 -07:00
chrislu cb6cf331ca filer.backup and filer.sync: include headers during backup and sync
fix https://github.com/seaweedfs/seaweedfs/issues/3532
2022-09-04 18:26:36 -07:00
famosss 7f56a40553
Turn down the 404 log level (#3585) 2022-09-04 18:17:01 -07:00
Brian a28b668647
Added ability to change replication settings upon volume.tier.move (#3583) 2022-09-04 16:47:21 -07:00
bernardx 228b133afa
new 'concurrency' parameter for filer.sync (#3579)
Co-authored-by: XIAOYQ <xiaoyq@eudic.net>
2022-09-02 23:03:23 -07:00
Konstantin Lebedev 853880bd83
[filer] DATA RACE on signal_handling fixed (#3575) 2022-09-02 06:20:19 -07:00
chrislu a112cbe44c filer: rocksdb store report not found correctly
fix https://github.com/seaweedfs/seaweedfs/discussions/3555
2022-09-01 15:42:59 -07:00
chrislu 5afc20dbd3 add tmux for dev 2022-09-01 14:47:45 -07:00
chrislu d1bb23645e Revert "avoid data race on grace.hooks (#3572)"
This reverts commit c37d6fc01a.
2022-09-01 13:08:34 -07:00
Konstantin Lebedev c37d6fc01a
avoid data race on grace.hooks (#3572)
https://github.com/seaweedfs/seaweedfs/issues/3564
2022-09-01 10:34:06 -07:00
Konstantin Lebedev 8c3040db81
avoid DATA RACE on S3Options.localFilerSocket (#3571)
* avoid DATA RACE on S3Options.localFilerSocket
https://github.com/seaweedfs/seaweedfs/issues/3552

* copy localSocket
2022-09-01 10:33:23 -07:00
Konstantin Lebedev 90d55cd179
avoid data race read fileChunksSize (#3570)
https://github.com/seaweedfs/seaweedfs/issues/3557
2022-09-01 10:31:57 -07:00
chrislu 86a5b6183d Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-09-01 09:44:53 -07:00
chrislu 04300dc2ac build local rocksdb 2022-09-01 09:44:48 -07:00
chrislu 466bbe4cdd dockerfile fix directory 2022-09-01 09:43:41 -07:00
Konstantin Lebedev 82c2ce74db
more error details in the log (#3568)
* more error details in the log
https://github.com/seaweedfs/seaweedfs/issues/3567

* format message
2022-09-01 09:04:15 -07:00
chrislu 10efdc7aab align memory 2022-09-01 09:02:44 -07:00
Konstantin Lebedev 803ca3c958
avoid data race on doSubscribeToOneFiler/ma.filer.UniqueFilerEpoch (#3566)
https://github.com/seaweedfs/seaweedfs/issues/3565
2022-09-01 09:01:47 -07:00
chrislu 9c76896e2d fix building rocksdb image 2022-09-01 00:20:23 -07:00
chrislu bcf35876d1 add more logs 2022-08-31 23:16:05 -07:00
dependabot[bot] 97d69d5336
Bump gocloud.dev/pubsub/rabbitpubsub from 0.25.0 to 0.26.0 (#3541)
* Bump gocloud.dev/pubsub/rabbitpubsub from 0.25.0 to 0.26.0

Bumps [gocloud.dev/pubsub/rabbitpubsub](https://github.com/google/go-cloud) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/rabbitpubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update code

* more code fix

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2022-08-31 10:16:49 -07:00
Patrick Schmidt a73e177ecf
Add an End-to-End workflow for FUSE mount (#3562)
* Add an e2e workflow to test FUSE mount

* Fix deadlocks during concurrent r/w
2022-08-31 09:27:53 -07:00
chrislu 4a4ef3cc3c upload_content: upload close response.Body 2022-08-31 00:25:28 -07:00
chrislu 3d8b8ffced refactor 2022-08-31 00:24:29 -07:00
chrislu 870a30665b close responses 2022-08-31 00:24:17 -07:00
chrislu ef78631a7c just a bit safer 2022-08-31 00:10:17 -07:00
chrislu 9c944377dc minor 2022-08-31 00:09:32 -07:00
chrislu a54f30b6c6 s3: close response body with copying object 2022-08-31 00:09:23 -07:00
chrislu 7eb121d5ae print process before do the work 2022-08-30 13:37:05 -07:00
Konstantin Lebedev 77098de69a
migrate from boltdb to bbolt (#3554) 2022-08-30 10:12:02 -07:00
chrislu de5263364b master client: use GetMaster() for latest master address 2022-08-30 09:52:06 -07:00
chrislu e583dbdd2a more detailed error 2022-08-30 09:43:17 -07:00
chrislu d81db3c703 s3: fix configuring IAM for the same user
hi, how can I add bucket permission to a user now?
Previously, if I needed to add permission to an existing credential, I simply repeated the s3.configure command with a different bucket name.
Now I am getting error:
duplicate accessKey[ХХХХ], already configured in user[YYYY]

s3.configure -access_key key -actions Read,Write,List -buckets bucket1 -secret_key secr -user user1
s3.configure -access_key key -actions Read,Write,List -buckets bucket2 -secret_key secr -user user1
2022-08-30 09:37:52 -07:00
Konstantin Lebedev ade94b0d0a
avoid race conditions access to SuperBlock.Version (#3539)
* avoid race conditions access to  SuperBlock.Version
https://github.com/seaweedfs/seaweedfs/issues/3515

* superBlockAccessLock replace to sync.Mutex
2022-08-30 00:08:00 -07:00
chrislu ae6292f9f0 rename variable 2022-08-30 00:07:15 -07:00
Konstantin Lebedev e16dda88e4
avoid race conditions access to MasterClient.currentMaster (#3538)
https://github.com/seaweedfs/seaweedfs/issues/3510
2022-08-30 00:05:38 -07:00
Konstantin Lebedev 105702ebe0
avoid data race access to uploadReaderToChunks.uploadErr (#3550)
avoid data race access to uploadErr
https://github.com/seaweedfs/seaweedfs/issues/3549
2022-08-30 00:03:01 -07:00
Konstantin Lebedev 903a06a160
avoid data race access to startDelay (#3548) 2022-08-30 00:02:09 -07:00
chrislu 570b488572 ensure memory alignment on different CPUs 2022-08-29 13:24:12 -07:00
Konstantin Lebedev 4966a3abc7
avoid race conditions access to growRequestCount (#3537)
https://github.com/seaweedfs/seaweedfs/issues/3511
2022-08-29 13:23:02 -07:00
dependabot[bot] 42b72e4006
Bump actions/setup-go from 3.2.1 to 3.3.0 (#3540) 2022-08-29 08:49:47 -07:00
dependabot[bot] 99b7f8fa4b
Bump cloud.google.com/go/pubsub from 1.24.0 to 1.25.1 (#3542) 2022-08-29 08:49:04 -07:00
dependabot[bot] 85fe01eefa
Bump github.com/linxGnu/grocksdb from 1.7.5 to 1.7.7 (#3543) 2022-08-29 08:48:51 -07:00
dependabot[bot] 75fe6e4ab2
Bump modernc.org/strutil from 1.1.2 to 1.1.3 (#3544) 2022-08-29 08:47:53 -07:00
dependabot[bot] ac939ccb89
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.35.1 to 3.37.1 (#3545) 2022-08-29 08:47:16 -07:00
Patrick Schmidt 00041f6e08
Fix corrupted mount reads after changes on fh (#3536) 2022-08-29 08:27:13 -07:00
chrislu b7a887fea1 3.24 2022-08-28 21:53:13 -07:00