From f23dccfff88eee255eac3a02304969304b70aab2 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Sat, 23 May 2020 18:55:40 -0700 Subject: [PATCH] Updated Filer Metadata Events (markdown) --- Filer-Metadata-Events.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/Filer-Metadata-Events.md b/Filer-Metadata-Events.md index b322cbd..1d83c0a 100644 --- a/Filer-Metadata-Events.md +++ b/Filer-Metadata-Events.md @@ -1,7 +1,7 @@ -On filer, there is a `/topics/.system/log` folder, it stores all events from filer metadata change events. +On filer, there is a `/topics/.system/log` folder, it stores all filer metadata change events. ## Metadata Event Format -The events are organized by timestamp, `yyyy-MM-dd/hh-mm.segment`. +The events are stored in files organized by timestamp, `yyyy-MM-dd/hh-mm.segment`. The events are encoded by protobuf, defined in https://github.com/chrislusf/seaweedfs/blob/master/weed/pb/filer.proto . The related sections are: ``` @@ -34,7 +34,7 @@ message LogEntry { ``` -The ondisk file is a repeated of the following bytes: +The ondisk file is a repeated bytes of the following format: 1. 4 bytes of `LogEntry` size 2. serialized `LogEntry` @@ -68,7 +68,9 @@ message SubscribeMetadataResponse { This is similar to `inotify` in normal file system, but actually much more powerful. Usually `inotify` can only monitor one directory and its direct children, but SeaweedFS metadata subscription can monitor one directory and all its sub-directories and files. -This could be useful to build powerful event driven data processing pipelines. Please let me know if you have some great ideas or accomplishment! +This API also allows you to read from any point of time and replay all metadata events. + +This API could be useful to build powerful event driven data processing pipelines. Please let me know if you have some great ideas or accomplishment! ### What kind of events it contains?