gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Words from SeaweedFS Users (markdown)

Chris Lu 2021-05-19 12:27:15 -07:00
parent a04177753b
commit 74cb8ff69c
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
Words-from-SeaweedFS-Users.md

@ -3,7 +3,7 @@
| Using SeaweedFS as part of the startup [OroraTech](https://ororatech.com/) where we are processing large amounts of infrared and visual satellite data to detect wildfires worldwide. Along with the actual hotspot detections we are generating large amounts of prerendered tile data (Slippy Maps) from the near realtime satellite images.| Right now running two identical dedicated servers with 140TB storage each for the staging and prod deployments of the tile storage. On these servers SeaweedFS is deployed through docker-compose and with a reverse proxy in front of it. This setup should be sufficient for quite some time, but the option of scaling to a distributed deployment in the future is helpful. | We were using AWS S3 for this purpose but wanted to find a scalable solution to handle our rapidly increasing S3 costs ($400/day mainly caused by the number of PUT operations, 40TB in 1 billion objects in the end). |
| [Source Code](https://github.com/EVERYGO111/OStoreBench), [Paper](https://github.com/EVERYGO111/OStoreBench/blob/master/research%20paper-OStoreBench.pdf) | OStoreBench: Open source Benchmarking Distributed Object Storage Systems Using Real-word Application Scenarios, Benchmark SeaweedFS with CEPH, Swift | OStoreBench: The performance of SeaweedFS is the best in three typical scenarios compared to Ceph and Swift. |
| replaced ceph with a seaweedfs under the docker registry in production | Under the registry half a million files. Not big but have intensive exchange. | Killer feature of seaweedfs is that it disign like S3 in yandex and can work in k8s and spread between data centers. Ceph has a bad design in the case of using a huge number of small files over 10 million, cluster recovery takes several days. The next step is to use instead of Glusterfs, which is now barely alive and is bent from 10 million files. |
| we use seaweedfs embedded in our AI products that are deployed on client site (usually AirGapped because of the sensitivity of the data)| clusters ranging from 3-10 servers (and now startiting to get bigger and bigger), usually retaining 7-14 days video and 30-60 days of thumbnails | we comared CEPH & Minio, we checked deployment procedure & maintenance and especially performance of writes and especially single server performance and easy scale out. we went and found that seaweedfs always won. we mainly write intensive and rarely read (usually reading as soon as write, so no real disk access) and 95% of the data is not missing critical, so the easiness of seaweedfs and the amazing performance (all writes are sequential as possible) |
| we use seaweedfs embedded in our AI products that are deployed on client site (usually AirGapped because of the sensitivity of the data)| clusters ranging from 3-10 servers (and now starting to get bigger and bigger), usually retaining 7-14 days video and 30-60 days of thumbnails | we comared CEPH & Minio, we checked deployment procedure & maintenance and especially performance of writes and especially single server performance and easy scale out. we went and found that seaweedfs always won. we mainly write intensive and rarely read (usually reading as soon as write, so no real disk access) and 95% of the data is not missing critical, so the easiness of seaweedfs and the amazing performance (all writes are sequential as possible) |
| [Holding lots of files](https://hypixel.net/threads/dev-blog-5-storing-your-skyblock-island.2190753/) | We've had to develop our own backup script and monitoring, interfacing with SeaweedFS. Backups of the whole dataset are done twice a day and stored in S3 for a few weeks. We run SeaweedFS across 3 volume servers which all use very low resources, always replicating volumes on the 3 servers for availability and peace of mind. The Seaweed FID are stored in Mongo. | It is basically Amazon S3, but self-hosted.|
| [InternetArchive compares SeaweedFS with Minio](https://github.com/internetarchive/sandcrawler/blob/master/proposals/2020_seaweed_s3.md) | SeaweedFS 200M object upload via Python script sucessfully in about 6 days, memory usage was at a moderate 400M (~10% of RAM). Relatively constant performance at about 400 PutObject requests/s (over 5 threads, each thread was around 80 requests/s; then testing with 4 threads, each thread got to around 100 requests/s) | Problem: minio inserts slowed down after inserting 80M or more objects. |
| Store images | Evercam has used Seaweed for a few years. We've 1344TB of mostly jpegs and use the filer for folder structure. It's worked well for us, especially with low cost Hetzner SX boxes. | Question: What about your recovery times when a server fails on 1Gbs port? Answer: Also in almost 5 years we only had one server crash which was due to file-system corruption, and we overcome that as well, it was a few leveldb files which got corrupt due to which the whole XFS file-system was went down, but we recovered it. Just one drawback was: We never used the same filer for saving files, and Get speed was also quite slow on that one, but with time, the volume compaction and vacuum, everything works fine on GET requests. |