gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated S3 API FAQ (markdown)

Adam Lamar 2022-02-11 15:43:42 -07:00
parent 752f3d7f8d
commit 4c2621c525
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
S3-API-FAQ.md

@ -40,9 +40,9 @@ Having separate LevelDB instance, or separate SQL tables, will help to isolate t
## Filer Metadata Store Growth
Due to the semantics of the S3 API, empty directories (aka prefixes) aren't shown. However, an entry is still stored in the filer metadata store. When access patterns create many unique directories and then remove all the objects inside those directories, the filer metadata store can grow unbounded with orphaned directories. These directories are visible in the filer metadata store itself, but not using the S3 API.
Due to the semantics of the S3 API, empty directories (aka prefixes) aren't shown. However, an entry is still stored in the filer metadata store. When workload access patterns create many unique directories and then remove all the objects inside those directories, the filer metadata store can grow unbounded with orphaned directories. These directories are visible in the filer metadata store itself, but not using the S3 API.
If the filer argument `-allowEmptyFolder=false` is set, the orphaned directories are cleaned up during list requests for non top-level directories. Simply list an existing, non bucket-level directory.
If the filer argument `-allowEmptyFolder=false` is set, the orphaned directories are cleaned up during list requests for non bucket-level directories. Normally this works well, but if the workload never performs a list operation, the orphaned directories may never be cleaned up. To force cleanup, simply list an existing, non bucket-level directory.
Example using rclone:
@ -50,4 +50,4 @@ Example using rclone:
rclone lsf seaweedfs:my-bucket/dir
```
If the directory `dir` exists in `my-bucket`, the orphaned metadata will be cleaned up. Note that `rclone ls` does not trigger cleanup, but `rclone lsf` will.
If the directory `dir` exists in `my-bucket`, the orphaned metadata will be cleaned up. Note that due to slight API usage differences, `rclone ls` does not trigger cleanup, but `rclone lsf` will.