gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Remote Storage Architecture (markdown)

Chris Lu 2021-08-14 07:55:06 -07:00
parent e2a6c58797
commit 121aba2878
1 changed files with 9 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
Remote-Storage-Architecture.md

@ -41,14 +41,18 @@ There are no format changes to the files on the cloud. The mounted files will be
## Mount Remote Storage
The remote storage, e.g., AWS S3, can be [[configured|Configure Remote Storage]] and [[mounted|Mount Remote Storage]] directly to an empty folder in SeaweedFS.
```
# in "weed shell"
> remote.mount -dir=xxx -remote=cloud1/bucket
```
On mount, all the metadata will be pulled down and cached to the local filer store.
The metadata will be used for all metadata operations, such as listing, directory traversal, read file size, compare file modification time, etc, which will be free and fast as usual, without any API calls to the cloud.
## Cache/Uncache File Content
If the cloud data has any changes, just run `remote.mount -dir=xxx -remote=cloud1/bucket` again in `weed shell`. The local cache will get updated (except deletions). It is a light weight process and you can run it regularly.
A file has metadata and its content.
## Cache/Uncache File Content
By default, the file content is [[cached|Cache Remote Storage]] to local volume servers on the first read.
@ -56,15 +60,13 @@ Sometimes you may want to fetch all file content for a set of files. But trying
Here you can run command `remote.cache -dir=xxx` in `weed shell`, which will cache all files under the specified directory.
Correspondingly, you can also run `remote.uncache -dir=xxx` in `weed shell`, to avoid local duplicated storage.
Correspondingly, you can also run `remote.uncache -dir=xxx` in `weed shell`, to purge local cache.
## Write Back Cache
The cache is write back by the `weed filer.remote.sync` process.
The cache is write back by the `weed filer.remote.sync` process. This asynchronous write back will not slow down any local operations.
If not starting `weed filer.remote.sync`, the cache will be just read only. Changes are not forbidden, but any data changes will not be propagated back to the cloud.
The asynchronous write back will not slow down any local operations.
If not starting `weed filer.remote.sync`, the data changes will not be propagated back to the cloud.
# Possible Use Cases