Commit graph

71 commits

Author SHA1 Message Date
mervynzhang df400e6c71
Concurrency works better (#4663)
Co-authored-by: mervyn.zhang <mervyn.zhang@sap.com>
2023-07-12 23:04:54 -07:00
mervynzhang 1ebb549f77
support swift (#4480) 2023-05-19 06:39:25 -07:00
Muhammad Hallaj bin Subery 9bd422d2c9
adding support for B2 region (#4177)
Co-authored-by: Muhammad Hallaj bin Subery <hallaj@tuta.io>
2023-02-05 21:24:21 -08:00
chrislu 81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
aronneagu 180853a2c9
Replace dashes with underscores in x-amz-meta headers (#3965) 2022-11-10 07:09:53 -08:00
Konstantin Lebedev 5431c445cd
fix filer.remote.sync to azure with ContentType (#3949)
* fix filer.remote.sync to azure with ContentType

* fix pass X-Amz-Meta to X-Ms-Meta
2022-11-04 09:10:33 -07:00
chrislu 4193dafce1 azure metadata: skip metadata prefixed with "X-"
fix https://github.com/seaweedfs/seaweedfs/issues/3875
2022-11-02 21:42:02 -07:00
chrislu 9920d65bc0 gateway to remote object store: adjust upload concurrency 2022-08-26 23:47:37 -07:00
chrislu eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 1d0c53ea56 remote storage: stop supporting hdfs as a remote storage 2022-06-20 14:15:59 -07:00
chrislu 61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
chrislu bff1ccc1de fix compilation 2022-05-11 00:52:15 -07:00
chrislu ad01c63b84 conditionally skip hdfs related code 2022-04-21 01:43:01 -07:00
justin d02f13c2d1 remove Redundant type conversion and use strings.TrimSuffix to enhance readability 2022-04-06 14:58:09 +08:00
chrislu a0bad1c997 remove any go mod changes
This reverts commit 6c7f7d6887.
2022-03-21 23:04:00 -07:00
chrislu 6c7f7d6887 Revert "Merge pull request #2782 from SadmiB/upstream"
This reverts commit a644b7236a, reversing
changes made to 349257f822.
2022-03-21 23:00:50 -07:00
SadmiB d12540c9f2 Add contabo api client 2022-03-21 17:16:49 +01:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu 04663c3611 remote.mount: print out metadata sync errors 2021-11-06 11:29:50 -07:00
Eng Zer Jun a23bcbb7ec
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu 2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu 53b9b521c9 adjust error message 2021-09-04 13:46:06 -07:00
Chris Lu da49d25950 auto list of storage types 2021-09-04 00:18:21 -07:00
Chris Lu bbc77f7af4 fix compilation 2021-09-03 22:56:59 -07:00
Chris Lu 0652805236 cloud drive: add createBucket() deleteBucket() 2021-09-03 22:30:55 -07:00
Chris Lu 83cd0fc739 cloud drive: add list buckets 2021-09-03 20:42:02 -07:00
Chris Lu fbfc90fd1e adjust formatting remote location 2021-09-03 18:52:37 -07:00
Chris Lu bca4a9de78 simplify 2021-09-02 23:09:24 -07:00
Chris Lu 958125bd02 conforming to http user agent common practice 2021-09-02 22:55:35 -07:00
Chris Lu 7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu 3bd48c4f29 filer.remote.sync: exit when directory is unmounted
this will not propagate the deletions back to the cloud
2021-09-01 01:29:22 -07:00
Chris Lu 3faaa6e360 ensure cached client with updated storage conf 2021-09-01 01:27:45 -07:00
Chris Lu 1dfcdc53b7 cloud drive: add storj, filebase 2021-08-30 17:43:14 -07:00
Chris Lu 701cc133f3 cloud drive: s3 add option for v4 signature 2021-08-30 17:28:33 -07:00
Chris Lu eacaa44dc2 refactor 2021-08-29 19:13:48 -07:00
Chris Lu a31f2907f0 cloud drive: filer.remote.sync supports remove folder 2021-08-29 18:46:28 -07:00
Chris Lu 001a472057 cloud mount: remote storage support hdfs 2021-08-29 18:41:29 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
Chris Lu c08ac536ed cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu a7a914f120 cloud drive: add support for Baidu BOS 2021-08-24 23:46:33 -07:00
Chris Lu c6c97bd83d add default env variables 2021-08-24 23:25:36 -07:00
Chris Lu a19c728034 cloud drive: add support for Tencent COS 2021-08-24 23:19:45 -07:00
Chris Lu 19a81d25af cloud drive: add support for Aliyun OSS 2021-08-24 23:14:24 -07:00
Chris Lu 47d775cf68 cloud drive: add support for BackBlaze 2021-08-24 22:30:06 -07:00
Chris Lu e9ebe24f2e cloud drive: add support for Azure 2021-08-24 01:18:30 -07:00
Chris Lu a78d0227cd adjust package name 2021-08-23 23:19:31 -07:00
Chris Lu 12631a3f5b cloud drive: gcs simplify a little bit 2021-08-23 14:43:01 -07:00
Chris Lu 00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00