From fb01808e7e1f93879708306807d3c8dccb6c979e Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Mon, 9 Aug 2021 23:36:32 -0700 Subject: [PATCH] Created Mount Remote Storage (markdown) --- Mount-Remote-Storage.md | 44 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 Mount-Remote-Storage.md diff --git a/Mount-Remote-Storage.md b/Mount-Remote-Storage.md new file mode 100644 index 0000000..f800d5e --- /dev/null +++ b/Mount-Remote-Storage.md @@ -0,0 +1,44 @@ +After [[Configure Remote Storage]], you will get a storage name `cloud1`. + +## Mount a Remote Storage +Now you can run `remote.mount` in `weed shell`: +``` +> help remote.mount + remote.mount # mount remote storage and pull its metadata + + # assume a remote storage is configured to name "s3_1" + remote.configure -name=cloud1 -type=s3 -access_key=xxx -secret_key=yyy + + # mount and pull one bucket + remote.mount -dir=xxx -remote=cloud1/bucket + # mount and pull one directory in the bucket + remote.mount -dir=xxx -remote=cloud1/bucket/dir1 + + # after mount, start a separate process to write updates to remote storage + weed filer.remote.sync -filer=: -dir=xxx + +``` + +With `remote.mount`, you can mount one bucket or one of the directory in the bucket. + +## Cache Meta Data +This operation will pull down all meta data from the remote storage. + +### Speed up meta data operations +This will greatly speed up metadata operations, such as file listing, directory traversal, check file size, read file modification time, etc. This operations usually are repeated often. Adding up all the network calls to the remote storage will slow down operations quite a lot. + +### Reduce API cost +Also, this will avoid API calls to the remote storage. The price for API calls usually is not cheap. + +For PUT, COPY, POST, LIST requests, they cost $0.005 per 1,000 requests on [[AWS S3|https://aws.amazon.com/s3/pricing/]]. + +And this is a cost which can often surprise you or your team. Some inadvertent programming logic loop changes may 10x your cost. + +With SeaweedFS cacheing meta data, all requests are executed locally. The actual cost to read meta data is reduced to minimum: Just Once! + +### Freedom of programmers + +Another important factor to use remote storage is that it gives your software engineers FREEDOM! + +Cloud gives your scalability, but limit you on cost. Because of cost concern, your software engineers have to jump within the cost hoops. +This costs more time and effort, limits their creativity, and reduces ROI.