gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8862 commits 28 branches 278 tags 135 MiB
Commit graph

8862 commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu a54f30b6c6 s3: close response body with copying object 2022-08-31 00:09:23 -07:00
chrislu 7eb121d5ae print process before do the work 2022-08-30 13:37:05 -07:00
Konstantin Lebedev 77098de69a
migrate from boltdb to bbolt (#3554) 2022-08-30 10:12:02 -07:00
chrislu de5263364b master client: use GetMaster() for latest master address 2022-08-30 09:52:06 -07:00
chrislu e583dbdd2a more detailed error 2022-08-30 09:43:17 -07:00
chrislu d81db3c703 s3: fix configuring IAM for the same user 
hi, how can I add bucket permission to a user now?
Previously, if I needed to add permission to an existing credential, I simply repeated the s3.configure command with a different bucket name.
Now I am getting error:
duplicate accessKey[ХХХХ], already configured in user[YYYY]

s3.configure -access_key key -actions Read,Write,List -buckets bucket1 -secret_key secr -user user1
s3.configure -access_key key -actions Read,Write,List -buckets bucket2 -secret_key secr -user user1
 2022-08-30 09:37:52 -07:00
Konstantin Lebedev ade94b0d0a
avoid race conditions access to SuperBlock.Version (#3539) 
* avoid race conditions access to  SuperBlock.Version
https://github.com/seaweedfs/seaweedfs/issues/3515

* superBlockAccessLock replace to sync.Mutex
 2022-08-30 00:08:00 -07:00
chrislu ae6292f9f0 rename variable 2022-08-30 00:07:15 -07:00
Konstantin Lebedev e16dda88e4
avoid race conditions access to MasterClient.currentMaster (#3538) 
https://github.com/seaweedfs/seaweedfs/issues/3510
 2022-08-30 00:05:38 -07:00
Konstantin Lebedev 105702ebe0
avoid data race access to uploadReaderToChunks.uploadErr (#3550) 
avoid data race access to uploadErr
https://github.com/seaweedfs/seaweedfs/issues/3549
 2022-08-30 00:03:01 -07:00
Konstantin Lebedev 903a06a160
avoid data race access to startDelay (#3548) 2022-08-30 00:02:09 -07:00
chrislu 570b488572 ensure memory alignment on different CPUs 2022-08-29 13:24:12 -07:00
Konstantin Lebedev 4966a3abc7
avoid race conditions access to growRequestCount (#3537) 
https://github.com/seaweedfs/seaweedfs/issues/3511
 2022-08-29 13:23:02 -07:00
dependabot[bot] 42b72e4006
Bump actions/setup-go from 3.2.1 to 3.3.0 (#3540) 2022-08-29 08:49:47 -07:00
dependabot[bot] 99b7f8fa4b
Bump cloud.google.com/go/pubsub from 1.24.0 to 1.25.1 (#3542) 2022-08-29 08:49:04 -07:00
dependabot[bot] 85fe01eefa
Bump github.com/linxGnu/grocksdb from 1.7.5 to 1.7.7 (#3543) 2022-08-29 08:48:51 -07:00
dependabot[bot] 75fe6e4ab2
Bump modernc.org/strutil from 1.1.2 to 1.1.3 (#3544) 2022-08-29 08:47:53 -07:00
dependabot[bot] ac939ccb89
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.35.1 to 3.37.1 (#3545) 2022-08-29 08:47:16 -07:00
Patrick Schmidt 00041f6e08
Fix corrupted mount reads after changes on fh (#3536) 2022-08-29 08:27:13 -07:00
chrislu b7a887fea1 3.24 2022-08-28 21:53:13 -07:00
Rain Li 089446f5b1
Fix reload page before all file uploaded bug (#3534) 2022-08-28 07:51:47 -07:00
chrislu cc0c8c5f81 simplify 2022-08-27 00:21:57 -07:00
chrislu 3741779144 minor 2022-08-27 00:20:58 -07:00
chrislu 580a7fa685 adjust for data race 
fix https://github.com/seaweedfs/seaweedfs/issues/3512
 2022-08-27 00:19:13 -07:00
chrislu 87b70a6809 clean up 2022-08-27 00:09:04 -07:00
chrislu 27c7611ebf Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-26 23:47:39 -07:00
chrislu 9920d65bc0 gateway to remote object store: adjust upload concurrency 2022-08-26 23:47:37 -07:00
chrislu c839ce1b19 s3 sink use s3 upload manager 
fix https://github.com/seaweedfs/seaweedfs/issues/3531
 2022-08-26 23:47:12 -07:00
famosss dc4037925d
fix: Build DeletedVids before reset dn's children (#3530) 2022-08-26 22:52:08 -07:00
chrislu 3f3a1341d8 make CodeQL happy 2022-08-26 17:09:11 -07:00
chrislu 301b49b63f atomic operation 2022-08-26 17:04:11 -07:00
chrislu 57a46f46a0 nano level precision 2022-08-26 16:55:15 -07:00
chrislu 8dae81c5ed Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-26 16:49:51 -07:00
chrislu e0f4366f4c more detailed error 
related to https://github.com/seaweedfs/seaweedfs/issues/3528
 2022-08-26 16:49:48 -07:00
Patrick Schmidt 5df105b1f9
Fix a few data races when reading files in mount (#3527) 2022-08-26 16:41:37 -07:00
chrislu f5156cf3a8 rename varaible 2022-08-26 10:23:42 -07:00
Konstantin Lebedev e90ab4ac60
avoid race conditions for OnPeerUpdate (#3525) 
https://github.com/seaweedfs/seaweedfs/issues/3524
 2022-08-26 10:18:49 -07:00
Konstantin Lebedev 4f7a1f67cd
avoid race conditions for diskLocation.MaxVolumeCount (#3526) 2022-08-26 08:41:42 -07:00
Konstantin Lebedev 913a5d0dd4
add target dev_race to MakeFile (#3522) 2022-08-26 01:28:03 -07:00
chrislu 6f483a4d36 atomic add 
fix https://github.com/seaweedfs/seaweedfs/issues/3514
 2022-08-25 22:20:34 -07:00
chrislu 7394f7feee minor: adjust ordering 2022-08-25 00:20:23 -07:00
chrislu dbf0de4ce1 minor clean up 2022-08-25 00:19:08 -07:00
Patrick Schmidt 2930263dfd
Fix race conditions during in-flight size checks (#3505) 2022-08-24 20:03:34 -07:00
Patrick Schmidt 7b424a54dc
Add raft server access mutex to avoid races (#3503) 2022-08-24 09:49:05 -07:00
Chris Lu f7e4359b59
Temporarily enable cgo 
Temporarily enable cgo following findings from https://github.com/seaweedfs/seaweedfs/discussions/3152 by https://github.com/dyus

Resolve build issue:
```
# github.com/mattn/go-ieproxy
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:36:9: undefined: getConf
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:41:9: undefined: reloadConf
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:48:2: undefined: overrideEnvWithStaticProxy
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:53:13: psc.findProxyForURL undefined (type *ProxyScriptConf has no field or method findProxyForURL, but does have FindProxyForURL)
```
 2022-08-24 02:24:21 -07:00
Konstantin Lebedev f7aeb06544
s3: report metadata if the directory is explicitly created (#3498) 
* replace mkdir to mkFile

* ContentLength must be zero

* revert mkDir

* Seaweedfs-Is-Directory-Key return metadata
 2022-08-24 00:15:44 -07:00
Guo Lei c57c79a0ab
optimiz commitig compact (#3388) 
* optimiz vacuuming volume

* fix bugx

* rename parameters

* fix conflict

* change copyDataBasedOnIndexFile to an instance method

* close needlemap

* optimiz commiting Vacuum volume for  leveldb index

* fix bugs

* fix leveldb loading bugs

* refactor

* fix leveldb loading bug

* add leveldb recovery

* add test case for levelDB

* modify test case to cover all the new branches

* use one tmpNm instead of two instances

* refactor

* refactor

* move setWatermark to the end

* add test for watermark and updating leveldb

* fix error logic

* refactor, add test

* check nil before close needlemapeer
add test case
fix metric bug

* add tests, fix bugs

* adjust log level
remove wrong test case
refactor

* avoid duplicate  updating metric for leveldb index
 2022-08-23 23:53:35 -07:00
chrislu 10414fd81c ping timeout at 15 seconds 
this 72 minute timeout setting seems unreasonably long

15 seconds is around the time when a new raft leader should be elected.
 2022-08-23 23:28:16 -07:00
askeipx 2e78a522ab
remove old raft servers if they don't answer to pings for too long (#3398) 
* remove old raft servers if they don't answer to pings for too long

add ping durations as options

rename ping fields

fix some todos

get masters through masterclient

raft remove server from leader

use raft servers to ping them

CheckMastersAlive for hashicorp raft only

* prepare blocking ping

* pass waitForReady as param

* pass waitForReady through all functions

* waitForReady works

* refactor

* remove unneeded params

* rollback unneeded changes

* fix
 2022-08-23 23:18:21 -07:00
Guo Lei 762dc219a6
fix deletecount bug (#3499) 2022-08-23 08:53:24 -07:00