gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Created Mount Remote Storage (markdown)

Chris Lu 2021-08-09 23:36:32 -07:00
parent 38a26ea61c
commit fb01808e7e
1 changed files with 44 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

44
Mount-Remote-Storage.md Normal file

@ -0,0 +1,44 @@
After [[Configure Remote Storage]], you will get a storage name `cloud1`.
## Mount a Remote Storage
Now you can run `remote.mount` in `weed shell`:
```
> help remote.mount
remote.mount # mount remote storage and pull its metadata
# assume a remote storage is configured to name "s3_1"
remote.configure -name=cloud1 -type=s3 -access_key=xxx -secret_key=yyy
# mount and pull one bucket
remote.mount -dir=xxx -remote=cloud1/bucket
# mount and pull one directory in the bucket
remote.mount -dir=xxx -remote=cloud1/bucket/dir1
# after mount, start a separate process to write updates to remote storage
weed filer.remote.sync -filer=<filerHost>:<filerPort> -dir=xxx
```
With `remote.mount`, you can mount one bucket or one of the directory in the bucket.
## Cache Meta Data
This operation will pull down all meta data from the remote storage.
### Speed up meta data operations
This will greatly speed up metadata operations, such as file listing, directory traversal, check file size, read file modification time, etc. This operations usually are repeated often. Adding up all the network calls to the remote storage will slow down operations quite a lot.
### Reduce API cost
Also, this will avoid API calls to the remote storage. The price for API calls usually is not cheap.
For PUT, COPY, POST, LIST requests, they cost $0.005 per 1,000 requests on [[AWS S3|https://aws.amazon.com/s3/pricing/]].
And this is a cost which can often surprise you or your team. Some inadvertent programming logic loop changes may 10x your cost.
With SeaweedFS cacheing meta data, all requests are executed locally. The actual cost to read meta data is reduced to minimum: Just Once!
### Freedom of programmers
Another important factor to use remote storage is that it gives your software engineers FREEDOM!
Cloud gives your scalability, but limit you on cost. Because of cost concern, your software engineers have to jump within the cost hoops.
This costs more time and effort, limits their creativity, and reduces ROI.