gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated S3 Bucket Quota (markdown)

Chris Lu 2022-01-21 13:28:15 -08:00
parent a6bbe83e76
commit e0bdeab2e9
1 changed files with 13 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
S3-Bucket-Quota.md

@ -1,11 +1,17 @@
# Efficient Usage Collection
To manage quota, the system first needs to know how much storage currently used. Usually this would need some relatively complicated counting, rolling up the directory tree. One way is to count during write time, but the upper tree nodes would become hot spots. This would slow down the normal write operations.
Since SeaweedFS uses one collection for each bucket, the usage can be efficiently collected from the volumes used by the collection. So the usage collection is very cheap and does not interfere with write operations.
# How it works
There are two aspects of quota management: "quota configuration" and "quota enforcement".
* "Quota Configuration" is done in by an admin script in `weed shell`, where you can set/remove/enable/disable bucket quota.
* "Quota Enforcement" is actually also an admin script, which you can run it regularly by adding it to `master.toml`. It will check whether all the buckets are over their limit or not. If one bucket is over quota, it will be set to read only.
* "Quota Enforcement" is actually also an admin script, which you can run it periodically by adding it to `master.toml`. It will check whether all the buckets are over their limit or not. If one bucket is over quota, it will set the bucket to read only.
The bucket read only attribute is checked when filer process write requests. If the bucket is read only, the creation and update requests on this bucket will be denied, except deletion. The read operations will not be affected.
The bucket read only flag is checked when filer process write requests. If the bucket is read only, the creation and update requests on this bucket will be denied, except deletion. The read operations are not be affected.
# Quota Configuration and Enforcement
@ -31,13 +37,17 @@ updated quota for bucket b1
b3 size:10280 file:3
```
**3. Enforce the quota, and found it should be set to read only.**
```
> s3.bucket.quota.enforce -apply
b1 size:527971104 quota:15728640 usage:3356.75%
changing bucket b1 to read only!
```
Note: Internally this readOnly flag is saved into filer configuration for `locationPrefix="/buckets/b1/"`. So you could directly check the bucket `readOnly` property with `fs.configure` also.
```
> fs.configure
{
"locations": [
{
@ -47,6 +57,4 @@ updated quota for bucket b1
}
]
}
```
Actually the filer configuration is changed to read only for this bucket. So you could directly change the bucket `readOnly` property with `fs.configure` also.