Commit graph

2004 commits

Author SHA1 Message Date
bukton fd4576651b add new package mongodb and install lib mongodb 2020-04-18 22:26:57 +07:00
Chris Lu 11f5a6d913 messaging can compile now 2020-04-18 01:12:01 -07:00
Chris Lu 5bea77010f refactor 2020-04-17 22:39:21 -07:00
Chris Lu 2e2537a9ea rename files 2020-04-17 22:35:44 -07:00
Chris Lu e0f5996560 fix "call of Unmarshal passes non-pointer as second argument" 2020-04-17 10:04:41 -07:00
Chris Lu 826f96b6f3 fix unreachable code 2020-04-17 10:04:28 -07:00
Chris Lu ffa69b24ae fix travis 2020-04-17 10:01:11 -07:00
Chris Lu 3f3dba5a68 broker: append message logs 2020-04-17 02:29:38 -07:00
Chris Lu 2a45897237 broker: read cipher value from filer 2020-04-17 02:29:00 -07:00
Chris Lu bda82f61bc filer: able to append to a file 2020-04-17 02:28:09 -07:00
Chris Lu 722b14337e add cipher option to meta data updates 2020-04-17 01:26:27 -07:00
Chris Lu 7d3672c60a simplify 2020-04-17 00:00:48 -07:00
Chris Lu 508f3490a0 update messaging proto 2020-04-16 03:29:57 -07:00
Chris Lu f5a748d33c refactoring 2020-04-16 02:55:09 -07:00
Chris Lu ce4b369be2 scaffolding messaging 2020-04-16 02:21:23 -07:00
Reed aada5b9aa7 making volume stats size info readable. 2020-04-16 10:08:08 +08:00
Chris Lu 9cacaf5eca better detect file mime type 2020-04-14 11:32:31 -07:00
Chris Lu 3f1d79512f only detect when not sure about the types 2020-04-14 11:02:05 -07:00
Chris Lu d2d1597850 volume: detect gzip effectiveness only when content type is empty
revert part of 2286eda575
2020-04-14 10:52:53 -07:00
Chris Lu 23715af402
Merge pull request #1273 from wyxloading/config_race
Fix data race about config
2020-04-13 22:24:51 -07:00
Chris Lu 2b5c4fbbf3 tiered caching
1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files
2020-04-13 22:19:27 -07:00
Chris Lu f282ed444b refactoring 2020-04-13 21:58:10 -07:00
wuyuxiang b556d3d035 Fix data race about config 2020-04-14 10:22:40 +08:00
Chris Lu d8f5985e5e add logs 2020-04-13 12:58:45 -07:00
Chris Lu 5c9e91393f 1.74 2020-04-13 10:25:48 -07:00
Chris Lu 7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu 8e23dc078b refactoring 2020-04-12 20:48:21 -07:00
Chris Lu b062393f3f refactoring 2020-04-12 17:51:31 -07:00
Chris Lu 5361f999ed shell: only one shell is allowed to connect to the cluster
fix https://github.com/chrislusf/seaweedfs/issues/1266
2020-04-12 17:48:39 -07:00
Chris Lu eb16bb0287 go mod cleanup 2020-04-12 16:44:58 -07:00
Chris Lu d30483d642 re-enable system logs 2020-04-12 14:03:07 -07:00
Chris Lu 6f948e4887 remove configurable topics folder location 2020-04-12 13:07:59 -07:00
Chris Lu 94e35cdb35 mount: fix
fix bug found by git bisect, but I do not understand why it can cause error!
2020-04-12 03:34:36 -07:00
Chris Lu 1c2e920fa2 fix compilation error 2020-04-12 03:11:19 -07:00
Chris Lu b5a713be68 filer store: add redis2 redis_cluster2 as default redis store 2020-04-12 02:50:41 -07:00
Chris Lu e613695af7 a little optimization 2020-04-12 01:13:57 -07:00
Chris Lu 2a1f396df5 avoid duplicated setting chunks into cache 2020-04-12 01:06:50 -07:00
Chris Lu b9b7da905e handle nil chunk cache 2020-04-12 01:00:12 -07:00
Chris Lu 211d87cf4c mount: option to disable caching 2020-04-12 00:52:54 -07:00
Chris Lu 1c65656fb4 s3: add option to fsync buckets 2020-04-11 23:37:10 -07:00
Chris Lu e4af63a721 volume server: accept fsync=true in write requests 2020-04-11 21:39:16 -07:00
Chris Lu 5987810e5e set default chunk size to 16 2020-04-11 21:22:52 -07:00
Chris Lu df97da25f9 mount: add on disk caching 2020-04-11 21:12:41 -07:00
Chris Lu c8ca234773 refactoring 2020-04-11 14:27:25 -07:00
Chris Lu df9d538044 rename function 2020-04-11 14:19:44 -07:00
Chris Lu d7f3acb2c0 refactor 2020-04-11 12:45:24 -07:00
Chris Lu ed54682598 refactoring 2020-04-11 12:43:17 -07:00
Chris Lu e909b55633 refactoring 2020-04-11 12:39:46 -07:00
Chris Lu b7f0ba3800 refactoring 2020-04-11 12:37:41 -07:00
Chris Lu 417125457e add additional buffers for logs 2020-04-11 02:19:48 -07:00
Chris Lu 4b27afe176 fix sqs message reading 2020-04-10 15:27:47 -07:00
Chris Lu dce2702e1b fix the instruction 2020-04-10 14:50:10 -07:00
Chris Lu 13ab1fdaf0 mount: use file default permission 0666 2020-04-10 02:23:23 -07:00
Chris Lu 1101a42e5c mv : create new folder, move children, and delete old folder 2020-04-10 01:37:03 -07:00
Chris Lu bcf37346ef add timestamp inside lock 2020-04-10 01:35:59 -07:00
Chris Lu 9fa065f600 typo in logs 2020-04-09 23:43:09 -07:00
Chris Lu 8db2120bee add logs 2020-04-09 23:42:59 -07:00
Chris Lu 43c62a8a95 1.73 2020-04-09 21:44:30 -07:00
Chris Lu eb39df2704 avoid dead loop 2020-04-09 21:43:26 -07:00
Chris Lu 56ec8c4513 fix to avoid nil file an dirty pages 2020-04-09 21:43:05 -07:00
Chris Lu 59f40e2027 volume: best effort to detect ip address
fix https://github.com/chrislusf/seaweedfs/issues/1264
2020-04-09 00:26:24 -07:00
Chris Lu f6a7e79dc3 weed shell: simplify CLI option for filer 2020-04-08 23:57:15 -07:00
Chris Lu 2329d9e0c1 add volume.fix.replication to default scaffold 2020-04-08 23:36:22 -07:00
Chris Lu bb78ab9915 wait for master to be aware of the failed volumes
fix https://github.com/chrislusf/seaweedfs/issues/1268
2020-04-08 23:12:37 -07:00
Chris Lu 8764bdb9df 1.72 2020-04-08 22:35:14 -07:00
Chris Lu 8a73276602 refactor a bit 2020-04-08 22:33:36 -07:00
Chris Lu 006f78b70b make a copy of request data
to avoid concurrency issues
2020-04-08 22:31:19 -07:00
Chris Lu 00b993a234 add util for md5 2020-04-08 22:30:38 -07:00
Chris Lu ed98223b08 still log, but not persisting the changes 2020-04-08 20:32:57 -07:00
Chris Lu 6630541399 ensure correct file size when opening existing files 2020-04-08 12:50:59 -07:00
Chris Lu b524a40375 add locking to fs cache 2020-04-08 12:50:34 -07:00
Chris Lu bd56172b82 simplify file handle management 2020-04-08 12:50:20 -07:00
Chris Lu dc08e4098f add etag only for PUT or large chunked uploads 2020-04-08 09:13:26 -07:00
Chris Lu ec2eb8bc48 add If-None-Match and If-Modified-Since
fix https://github.com/chrislusf/seaweedfs/issues/1269
2020-04-08 08:12:00 -07:00
Chris Lu 4c498e73de filer replication: add s3 endpoint 2020-04-07 17:49:00 -07:00
Chris Lu 745f5d2a25 fix setting default value 2020-04-07 01:58:48 -07:00
Chris Lu 54debdc6f7 filer: fix configuration settings 2020-04-07 01:30:53 -07:00
Chris Lu 90d576affe disable meta data change event logging for now. 2020-04-07 01:05:24 -07:00
Chris Lu b51d8f03bb 1.71 2020-04-05 22:04:16 -07:00
Chris Lu 5a5908407d filer: support larger file size
fix https://github.com/chrislusf/seaweedfs/issues/1257
2020-04-05 21:48:45 -07:00
Chris Lu 19edd9c091 comment out debug messages 2020-04-05 20:15:17 -07:00
Chris Lu 78ea937606 fix 2020-04-05 17:34:26 -07:00
Chris Lu 53626734d4 iterate through the log buffer 2020-04-05 16:51:30 -07:00
Chris Lu f71b855db5 adjust caching and notification for recursive directory deletion 2020-04-05 15:03:25 -07:00
Chris Lu f14b6a09a8 remove duplicated update events 2020-04-05 13:22:15 -07:00
Chris Lu 91da7057b1 refactoring 2020-04-05 13:11:43 -07:00
Chris Lu 2d43f85577 watch entries with common path prefix 2020-04-05 12:51:21 -07:00
Chris Lu 7bc3c93512 add util.PathJoin 2020-04-05 12:40:46 -07:00
Chris Lu af1f64d244 change from 'weed tail' to 'weed watch' 2020-04-05 12:39:20 -07:00
Chris Lu b10679fcf0 adjust log file names 2020-04-05 12:38:46 -07:00
Chris Lu 2cb1700ecb set to 4MB meta log buffer size 2020-04-05 12:38:29 -07:00
Chris Lu e78ee087e9 fix tests 2020-04-05 01:17:35 -07:00
Chris Lu dcc6b6a3ed go fmt 2020-04-05 00:52:31 -07:00
Chris Lu bf270d9e8c filer: able to tail meta data changes 2020-04-05 00:51:16 -07:00
Chris Lu f7b5a1d697 add disk stat to logs 2020-04-04 13:17:23 -07:00
Chris Lu 6e776aaafc loop through all masters 2020-04-03 00:47:48 -07:00
Chris Lu 4a5e381feb fail fast if filer configured wrong masters
fix https://github.com/chrislusf/seaweedfs/issues/1257
2020-04-03 00:47:33 -07:00
Chris Lu aaf5e658cc adjust parameter name 2020-04-03 00:41:05 -07:00
Chris Lu 6c96e51414 re-ordering 2020-04-03 00:40:54 -07:00
Chris Lu d2f8e50459 make error a little bit more clearer
fix https://github.com/chrislusf/seaweedfs/issues/1259
2020-04-03 00:24:09 -07:00