From ed41f4f5cf2c06621ee293008fdad24c0e20c811 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Mon, 24 May 2021 00:11:08 -0700 Subject: [PATCH] add sqlite wiki --- Directories-and-Files.md | 2 +- Filer-Cassandra-Setup.md | 2 +- Filer-Store-Replication.md | 4 ++-- Filer-Stores.md | 1 + Hadoop-Compatible-File-System.md | 2 +- 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Directories-and-Files.md b/Directories-and-Files.md index 072d4f0..fd8d744 100644 --- a/Directories-and-Files.md +++ b/Directories-and-Files.md @@ -54,7 +54,7 @@ Filer has a persistent client connecting to Master, to get the location updates There are no network round trip to lookup the volume id location. For file reads: -1. Filer lookup metadata from Filer Store, which can be Cassandra/Mysql/Postgres/Redis/LevelDB/etcd/TiKV. +1. Filer lookup metadata from Filer Store, which can be Cassandra/Mysql/Postgres/Redis/LevelDB/etcd/Sqlite. 2. Filer read from volume servers and pass along to the read request. ![](FilerRead.png) diff --git a/Filer-Cassandra-Setup.md b/Filer-Cassandra-Setup.md index e3381c8..b98c9b6 100644 --- a/Filer-Cassandra-Setup.md +++ b/Filer-Cassandra-Setup.md @@ -1,4 +1,4 @@ -SeaweedFS can utilize existing familiar data store, e.g., Cassandra, Mysql, Postgres, Redis, to store the filer metadata. +SeaweedFS can utilize existing familiar data store, e.g., Cassandra, Mysql, Postgres, Redis, Sqlite, to store the filer metadata. The following takes Cassandra as an example. diff --git a/Filer-Store-Replication.md b/Filer-Store-Replication.md index 5263013..3cdfa25 100644 --- a/Filer-Store-Replication.md +++ b/Filer-Store-Replication.md @@ -25,9 +25,9 @@ It can also enables Active-Active or one-directional replication. ## Use Cases -For filer stores using shared filer stores, such as shared Mysql/Postgres/Cassandra/Redis/ElasticSearch/etc in [[Filer-Stores]], this is not really needed, since all filers are stateless, and there are no need to replicate the meta data back to the same filer store. +For filer stores using shared filer stores, such as shared Mysql/Postgres/Cassandra/Redis/Sqlite/ElasticSearch/etc in [[Filer-Stores]], this is not really needed, since all filers are stateless, and there are no need to replicate the meta data back to the same filer store. -But if each filer has its own filer store, usually with the default local Leveldb, or even with a dedicated Mysql/Postgres/Cassandra/Redis/etc store, this would be very useful. +But if each filer has its own filer store, usually with the default local Leveldb, or even with a dedicated Mysql/Postgres/Cassandra/Redis/Sqlite/etc store, this would be very useful. Sometimes you may want to replicate the existing store to a new filer store, or move to a new filer store, this would also be useful. diff --git a/Filer-Stores.md b/Filer-Stores.md index d3bd356..bdc7da4 100644 --- a/Filer-Stores.md +++ b/Filer-Stores.md @@ -9,6 +9,7 @@ The Filer Store persists all file metadata and directory information. | leveldb2 | O(logN)| unlimited | Local, Very Fast | | Yes| | Similar to leveldb, part of the lookup key is 128bit MD5 instead of the long full file path | | leveldb3 | O(logN)| unlimited | Local, Very Fast | | Yes| Yes| Similar to leveldb2, separate leveldb instance for each bucket | | RocksDB | O(logN)| unlimited | Local, Very Fast | | Native| | Default, fairly scalable | +| Sqlite | O(logN)| unlimited | Local, Very Fast |Atomic| Yes | Yes | Default, fairly scalable, Stream backup | | Mongodb | O(logN)| unlimited | Local or Distributed, Fast | | Yes| | Easy to manage | | Redis | O(1) | limited | Local or Distributed, Fastest ||Native| | one directory's sub file names are stored in one key~value entry | | Cassandra | O(logN)| unlimited | Local or Distributed, Very Fast||Native| | | diff --git a/Hadoop-Compatible-File-System.md b/Hadoop-Compatible-File-System.md index b6dceb8..d7af541 100644 --- a/Hadoop-Compatible-File-System.md +++ b/Hadoop-Compatible-File-System.md @@ -176,7 +176,7 @@ $ bin/hdfs dfs -ls seaweedfs://localhost:8888/ # Notes ## Atomicity -SeaweedFS satisfies the HCFS [requirements](https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/filesystem/introduction.html) that the following operations to be atomic, when using MySql/Postgres database transactions. +SeaweedFS satisfies the HCFS [requirements](https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/filesystem/introduction.html) that the following operations to be atomic, when using MySql/Postgres/Sqlite database transactions. 1. Creating a file. If the overwrite parameter is false, the check and creation MUST be atomic. 1. Deleting a file.