From 0aa4266ed4d576bf5da3db04643342fd64324b95 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Mon, 5 Aug 2019 11:48:45 -0700 Subject: [PATCH] Updated Directories and Files (markdown) --- Directories-and-Files.md | 43 ---------------------------------------- 1 file changed, 43 deletions(-) diff --git a/Directories-and-Files.md b/Directories-and-Files.md index ae36cef..62c8101 100644 --- a/Directories-and-Files.md +++ b/Directories-and-Files.md @@ -72,49 +72,6 @@ For file renaming, it is just O(1) operations, with deleting the old metadata an For directory renaming, it will be O(N) operations, with N as the number of files and folders underneath the to-be-renamed directory. This is because each of them will need to adjust the metadata. But still there are no change for the actual file content on volume servers. -### Comparing Storage Options - -The Filer Store persists all file metadata and directory information. - -| Filer Store Name | Lookup | number of entries in a folder | Scalability | Renaming | TTL | Note | -| ---------------- | -- | -- | -- | -- | -- | -- | -| memory | O(1) | limited by memory | Local, Fast | | | for testing only, no persistent storage | -| leveldb | O(logN)| unlimited | Local, Very Fast | | | Default, fairly scalable | -| leveldb2 | O(logN)| unlimited | Local, Very Fast, faster than leveldb | | | Similar to leveldb, part of the lookup key is 128bit MD5 instead of the long full file path | -| Redis | O(1) | limited | Local or Distributed, Fastest ||Yes| one directory's sub file names are stored in one key~value entry | -| Cassandra | O(logN)| unlimited | Local or Distributed, Very Fast||Yes| | -| MySql | O(logN)| unlimited | Local or Distributed, Fast |Atomic| | Easy to manage, export | -| Postgres | O(logN)| unlimited | Local or Distributed, Fast |Atomic| | Easy to manage, export | -| Etcd | O(logN)| ~10GB | Distributed, 10,000 writes/sec || | No SPOF. High Availability.| - -#### Switching between different Stores -It is easy to switch between different filer stores. - -For example: -```sh - -# first save current filer meta data - -$ weed shell -> fs.cd http://filerHost:filerPort/ -> fs.meta.save -... -total 65 directories, 292 files -meta data for http://localhost:8888/ is saved to localhost-8888-20190417-005421.meta -> exit - -# now switch to a new filer, and load the previously saved metadata -$ weed shell -> fs.meta.load localhost-8888-20190417-005421.meta -... -total 65 directories, 292 files -localhost-8888-20190417-005421.meta is loaded to http://localhost:8888/ - -``` - -### Extending Storage Options - -For any new storage option, please implement the FilerStore interface. It should be fairly straight forward to implement. Welcome to contribute back. ### Use Cases