gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Mount Remote Storage (markdown)

Chris Lu 2021-08-21 02:08:48 -07:00
parent bb23ed4dff
commit ed25296bb3
1 changed files with 4 additions and 25 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
Mount-Remote-Storage.md

@ -53,39 +53,18 @@ If the data on the cloud can changed often, you can create a cronjob to run it.
## Write Back changes to Remote Storage
If the mounted directory is only for reading, you can start to use it.
If the mounted directory is only for reading, you can skip this step.
If you want to change files in the mounted directory, a separate process `weed filer.remote.sync -dir=xxx` should be started.
If local changes need to be synchronized to the cloud storage, a separate process `weed filer.remote.sync -dir=xxx` should be started.
This process will listen to filer change events, and write any changes back to the remote storage.
```
weed filer.remote.sync -filer=<filerHost>:<filerPort> -dir=xxx
```
The process is designed to be worry-free. It should automatically resume if stopped, and can reconnect automatically.
## Unmount a Remote Storage
Similarly, running `remote.unmount -dir=xxx` can unmount a remote storage. However, this means all cached data and metadata will be lost.
And if `weed filer.remote.sync -filer=<filerHost>:<filerPort> -dir=xxx` was not run, the updated local data have not been uploaded to the remote storage, so these local changes will be lost.
## Benefits to Cache Metadata
### Speed up meta data operations
This will greatly speed up metadata operations, such as file listing, directory traversal, check file size, read file modification time, etc. This operations usually are repeated often. Adding up all the network calls to the remote storage will slow down operations quite a bit.
### Reduce API cost
Also, this will avoid API calls to the cloud storage. The price for API calls usually is not cheap.
For PUT, COPY, POST, LIST requests, they cost $0.005 per 1,000 requests on [[AWS S3|https://aws.amazon.com/s3/pricing/]].
The cost can often surprise you and your team. Some inadvertent programming logic loop changes may 10x your cost.
With SeaweedFS caching meta data, all requests are executed locally. The actual cost to read cloud metadata is reduced to minimum: Just Once!
### Increase Productivity
Another important factor to use remote storage is that it gives your software engineers FREEDOM!
Cloud gives your scalability, but limit you on cost. Because of cost concern, your software engineers have to jump through unnecessary hoops. This costs more time and effort, limits their creativity and productivity. The project will take more resources or get delayed.
And if `weed filer.remote.sync -filer=<filerHost>:<filerPort> -dir=xxx` was not run, the local updates have not been uploaded to the remote storage, so these local updates will be lost.