gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Cloud Cache Benefits (markdown)

Chris Lu 2021-08-16 01:45:34 -07:00
parent 9a41f263c2
commit dfdcb90ae0
1 changed files with 7 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
Cloud-Cache-Benefits.md

@ -51,7 +51,7 @@ However, how to make SeaweedFS work with data already on cloud?
# Possible Use Cases
* Machine learning
## Machine learning
* Problem
* Training jobs need to repeatedly visit a large set of files.
* The randomized access pattern is hard for caching.
@ -59,14 +59,14 @@ However, how to make SeaweedFS work with data already on cloud?
* Users can explicitly ask SeaweedFS Cloud Cache to cache one whole folder.
* Increase training speed and reduce API cost and network cost.
* Users can access data with FUSE mounted folders.
* Data Hoarding
## Data Hoarding
* Problem
* With cloud capacity and storage tiering, saving data files there may be a good idea.
* Recently uploaded files very likely need to be accessed again.
* With SeaweedFS Cloud Cache
* Users can explicitly ask SeaweedFS Cloud Cache to uncache by file age.
* Users can also choose to never uncache, basically treating cloud copy as a backup.
* Big Data
## Big Data
* Problem
* Run MapReduce, Spark, and Flink jobs on cloud data is slow due to metadata operations.
* Repeated data access increases unnecessary cost.
@ -75,24 +75,24 @@ However, how to make SeaweedFS work with data already on cloud?
* Avoiding slow cloud storage metadata access.
* Access data only once.
* Write back data to work with cloud ecosystems.
* Cloud Storage Vendor Agnostic
## Cloud Storage Vendor Agnostic
* Problem
* Different datasets may need to be on different vendors, based on access pattern, latency, cost, etc.
* With SeaweedFS Cloud Cache
* Transparently switch to from one vendor to another.
* Switch S3 Compatible Storage Vendors
## Switch S3 Compatible Storage Vendors
* Problem
* Other S3 compatible vendors are known to have lesser flexibility, scalability and performance as SeaweedFS.
* With SeaweedFS Cloud Cache
* Compare SeaweedFS with the vendor over a long period of time.
* Transparently switch away if SeaweedFS works better.
* Move Off Cloud
## Move Off Cloud
* Problem
* Cloud storage is costly!
* With SeaweedFS Cloud Cache
* Help to transition between on-cloud to off-cloud.
* When you are happy with it, just stop the write back process (and cancel the monthly payment to the cloud vendor!).
* Support multiple access methods.
## Support multiple access methods.
* Problem
* You may need to access cloud data by HDFS, or HTTP, or S3 API, or WebDav, or FUSE Mount.
* With SeaweedFS Cloud Cache