gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Mount Remote Storage (markdown)

Chris Lu 2021-08-15 02:23:09 -07:00
parent b7d296ed0e
commit 203451f8fe
1 changed files with 15 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
Mount-Remote-Storage.md

@ -16,36 +16,40 @@ Usage of remote.mount:
remote.mount # mount remote storage and pull its metadata
# assume a remote storage is configured to name "s3_1"
remote.configure -name=cloud1 -type=s3 -access_key=xxx -secret_key=yyy
remote.configure -name=cloud1 -type=s3 -access_key=xyz -secret_key=yyy
# mount and pull one bucket
remote.mount -dir=xxx -remote=cloud1/bucket
remote.mount -dir=/xxx -remote=cloud1/bucket
# mount and pull one directory in the bucket
remote.mount -dir=xxx -remote=cloud1/bucket/dir1
remote.mount -dir=/xxx -remote=cloud1/bucket/dir1
# after mount, start a separate process to write updates to remote storage
weed filer.remote.sync -filer=<filerHost>:<filerPort> -dir=xxx
weed filer.remote.sync -filer=<filerHost>:<filerPort> -dir=/xxx
```
With `remote.mount`, you can mount one bucket or one of the directory in the bucket.
With `remote.mount`, you can mount one bucket or any directory in the bucket.
## Cache Metadata
This operation will pull down all metadata from the remote storage.
This `remote.mount` operation will pull down all metadata from the remote storage.
Later, any metadata operations can be invoked locally, and respond very fast.
Later, any metadata operations will be fast to just read local metadata.
`unmount` will drop all local metadata and cached file content.
`remote.unmount` will drop all local metadata and cached file content.
### Repeatedly Update Metadata
Sometimes the data on the cloud has changed and local metadata becomes stale. To `unmount` first and `mount` again is costly since all data has to be cached again. To refresh the metadata changes, you can simply re-run the same command with `-nonempty` option, e.g.:
Sometimes the data on the cloud has changes and local metadata becomes stale. To `unmount` first and `mount` again can work but costly, since all data has to be cached again.
To refresh the metadata changes, you can run this on the mounted directory or any sub directories, e.g.:
```
remote.mount -dir=xxx -remote=cloud1/bucket -nonempty
remote.meta.sync -dir=/xxx
remote.meta.sync -dir=/xxx/sub/dir
```
This will update local metadata accordingly and still keep file contents that are not changed.
If the data on the cloud is changed often, you can add this command to the admin scripts defined in `master.toml`, to run it regularly.
If the data on the cloud can changed often, you can create a cronjob to run it. Or you can add this command to the admin scripts defined in `master.toml`, to run it regularly.
## Write Back changes to Remote Storage