gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

add overview page to help orient newcomers

Dave Cottlehuber 2020-12-09 21:29:46 +00:00
parent 9a52345187
commit 21d3da2b4c
1 changed files with 33 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

33
Overview.md Normal file

@ -0,0 +1,33 @@
# SeaweedFS Components
seaweedfs comprises 3 main conceptual components. The master service and the volume service together provide a distributed object store, with user-configurable replication and redundancy. The optional filer and S3 service are additional layers on top of the object store. Each of these services may run one or more instances, on various actual servers.
## master service
The essential master service works smart, not hard. It represents a raft cluster of 1 (or 3 or more servers) that own a consistent view of the entire seaweedfs cluster and communicate it to all participating nodes, through a raft Leader.
The number of servers in the master service must always be odd, to ensure that a majority consensus can be formed. You're best off keeping this number down, a small number of stable servers is better than a large pool of flakey boxes. 1 or 3 is typical.
The leader is arbitrarily chosen from all available master servers, through a periodic raft election. It assigns file ids, appoints which volumes to store objects in, and also owns deciding which nodes are part of the cluster.
All other servers in seaweedfs send heartbeats to the leader, which uses them to decide where to route traffic, and how to handle replication.
If the leader is unavailable, the raft consensus protocol ensures that a new leader is appointed, with the agreement of the entire cluster, and the existing absent leader is demoted until it is able to function correctly again.
## volume service
The volume service is where the hard work is done.
It packs many objects (files and chunks of files) efficiently into larger individual volumes, which can be arbitrarily large blocks on disk. Redundancy and replication of data is managed at the volume level, not on a per-object level.
Each volume server sends periodic heartbeats with status and volume information back to the leader, via a master.
## filer service
The optional filer service does heavy lifting so you don't have to. It translates seaweedfs volumes and objects, into user-visible paths (like URLs or file systems) over HTTP or UNIX FUSE mounts.
While it is not as efficient as the direct object store, it provides a convenient and common abstraction that can be used to provide normal looking filesystems, or web APIs for down/uploads, to existing applications without modification.
## S3 service
This optional service provides AWS style S3 buckets, similar to the filer service. It can be started separately, or together with the filer, as required.