Updated Filer Change Data Capture (markdown)

Chris Lu 2020-12-23 15:04:36 -08:00
parent b43111b57a
commit 91a1de8ad8

@ -2,6 +2,40 @@ Is it too much a dream to have something similar to [inotify](https://man7.org/l
Actually SeaweedFS can give you more! Actually SeaweedFS can give you more!
# Experience it first
You can watch the SeaweedFS meta data changes.
```
$ weed watch -h
Example: weed watch [-filer=localhost:8888] [-target=/]
Default Usage:
-filer string
filer hostname:port (default "localhost:8888")
-pathPrefix string
path to a folder or file, or common prefix for the folders or files on filer (default "/")
-pattern string
full path or just filename pattern, ex: "/home/?opher", "*.pdf", see https://golang.org/pkg/path/filepath/#Match
-timeAgo duration
start time before now. "300ms", "1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h"
Description:
See recent changes on a filer.
$ weed watch
dir:/home new_entry:{name:"3190.pdf" chunks:{size:1130805 mtime:1608763500740201000 e_tag:"5c745d0a109bde58e0baaa6515c3694a" fid:{volume_id:1 file_key:35 cookie:4265699614}} attributes:{file_size:1130805 mtime:1608763500 file_mode:432 uid:502 gid:20 crtime:1608763500 mime:"application/pdf" replication:"000" md5:"\\t]\n\x10\x9b\xdeXສe\x15\xc3iJ"}} delete_chunks:true new_parent_path:"/home" signatures:1622972984
```
# How it works?
The `weed watch` code is nothing fancy. It is calls a gRPC stream API to subscribe to all meta data changes and simply print out the meta data.
The gRPC API has several important use cases within SeaweedFS:
* Replicate data to other SeaweedFS clusters in `weed filer.sync`.
* Replicate meta data to other filers if not sharing the same filer meta store.
* Replicate meta data to `weed mount` asynchronously.
The gRPC API is also open to public and can support all other languages.
# Example # Example
Here is an example, in Java: Here is an example, in Java: