Commit graph

34 commits

Author SHA1 Message Date
chrislu e55076c46f cloud tier: add retry when copying data file
fix https://github.com/seaweedfs/seaweedfs/issues/3828
2022-10-12 00:38:32 -07:00
chrislu 39340f7e42 cloud tier: s3 consume all read response body
fix https://github.com/seaweedfs/seaweedfs/issues/3584
2022-09-04 22:23:28 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 3e20336bde fix atomic add int64
fix https://github.com/chrislusf/seaweedfs/issues/3038#issuecomment-1123269831
2022-05-11 00:44:35 -07:00
chrislu 76148ce0f7 use storage_class for backend configuration
fix https://github.com/chrislusf/seaweedfs/issues/3008
2022-05-02 12:20:01 -07:00
chrislu 1aae7a3f1b volume.tier.upload progress starts negative #2992
fix https://github.com/chrislusf/seaweedfs/issues/2992
2022-04-30 18:10:01 -07:00
chrislu 192983b464 s3 backend support customizing storage class 2022-04-30 17:36:40 -07:00
Chris Lu 0c8dea9de8 go fmt 2021-11-02 23:39:16 -07:00
Chris Lu c857cc7286 cloud tier: remove tagging since not all s3 vendors support this 2021-10-29 12:39:19 -07:00
Chris Lu d04cdcf40d s3 header add user agent 2021-10-29 12:28:24 -07:00
Chris Lu f58ea6a2ee add source name to error message 2021-10-01 02:19:30 -07:00
Chris Lu 7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu c08ac536ed cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu 00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00
Chris Lu f0cc130849 do not force path style for better compatibility 2021-08-23 03:09:41 -07:00
Chris Lu 2cca07b44c cloud tier to non-AWS s3 gateways 2021-02-28 18:59:09 -08:00
Chris Lu 81797a059a volume: sync volume file right before compaction
fix https://github.com/chrislusf/seaweedfs/issues/1237
2020-03-19 23:54:52 -07:00
Chris Lu 709f231e23 tiered storage: add s3 endpoint for private s3 implementation
fix https://github.com/chrislusf/seaweedfs/issues/1238
2020-03-19 21:13:56 -07:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu 2000284435 rename volume tier info to volume info 2019-12-28 11:21:49 -08:00
Chris Lu 48d28d3eb2 tier: support remote file attributes and remember the file extension 2019-12-25 21:37:24 -08:00
Chris Lu eecad6062a go fmt 2019-12-25 09:56:42 -08:00
Chris Lu d960b3474a tier storage: support downloading the remote dat files 2019-12-25 09:53:13 -08:00
Chris Lu 356bd1b629 adjust text 2019-12-10 22:09:29 -08:00
Chris Lu 10bd3c6b4b refactoring 2019-12-08 19:44:16 -08:00
Chris Lu 2b8e20f122 remove unused functions 2019-12-08 18:33:25 -08:00
Chris Lu 70648d35ad go fmt 2019-12-02 20:49:58 -08:00
Chris Lu caae543a9f fix test 2019-12-02 15:54:24 -08:00
Chris Lu ec8de250e2 tiered storage: can copy to s3, read from s3
master not aware tiered volume yet, file assigning is not working yet
2019-12-02 15:08:31 -08:00
Chris Lu 0da7b894cc pass backend config from master to volume servers 2019-11-29 01:05:09 -08:00
Chris Lu f60154f330 master load backend storage config from master.toml 2019-11-28 18:33:18 -08:00
Chris Lu 0f9ba84274 s3 2019-11-27 03:09:45 -08:00
Chris Lu e4c63ff2f8 go fmt 2019-11-22 14:51:09 -08:00
Chris Lu 9711a6ffaa WIP 2019-11-18 19:24:37 -08:00