gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Created Tiered Storage (markdown)

Chris Lu 2021-02-09 14:41:46 -08:00
parent 2ebafffbf5
commit 66a8db7a28
1 changed files with 31 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
Tiered-Storage.md Normal file

@ -0,0 +1,31 @@
All read and write operation for volume data is O(1) disk seek. However, there are fast, slow, and remote storages.
Since data that are hot, warm, and cold, it would be cost-efficient to move data accordingly.
# Speed up volume index
To write any data, the volume index needs to append one entry. To read any data, the volume index lookup is required. The volume index can be in memory mode or an LevelDB instance. The amount of index content is small while it is accessed frequently.
You can run `weed volume -dir.idx=/fast/disk/dir` or `weed server -volume.dir.idx=/vast/disk/dir` to ensure the volume index is located on a fast disk, e.g., a fast SSD mount.
# Volume Disk Type
If the volume data also needs fast access, you can create them on fast disks.
For example, one volume server can have two disks with two kinds of disk types:
```
weed volume -max=10000 -disk=hdd,ssd -port=8080 -dir=/data,/ssd -compactionMBps=40 -minFreeSpacePercent=7
```
Please notice the `-disk=hdd,ssd & -dir=/data,/ssd`.
Then you can create with `fs.configure` a path specific setting and set buckets/collections that start with (example) "ssd_" to be allocated to the ssd, and all other collection will be created on the hdd. you can also create specific bucket name.
Example (within the weed shell):
```
fs.configure -locationPrefix=/buckets/ssd_ -disk=ssd -apply
```
https://github.com/chrislusf/seaweedfs/wiki/Path-Specific-Configuration
For now, ssd volumes needs to be manually moved from ssd directories to hdd directories, if you want to change the disk type.