gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4837 commits 28 branches 278 tags 135 MiB
Commit graph

199 commits

Author SHA1 Message Date
Chris Lu cc2bd97ad9 refactor 2020-11-26 11:25:56 -08:00
Chris Lu 0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu effa00ed08 refactor 2020-11-26 11:21:58 -08:00
Chris Lu 513bcd6e0d filer: avoid duplicated FindEntry for deletion 2020-11-26 11:14:56 -08:00
Chris Lu eab53ea80d filer leveldb store: a bit more efficient directory listing with prefix 2020-11-22 21:10:41 -08:00
Chris Lu 290b5e2cd0 directly delete file chunks 
keeping current async deletions for now
 2020-11-17 17:20:21 -08:00
Chris Lu dc304342b2 fs.configure: configurable volume growth 2020-11-17 01:00:02 -08:00
Chris Lu 5f19e81dab filer confi: support hierachical configuration 2020-11-16 16:50:12 -08:00
Chris Lu c0d279c54e filere.conf: prefer to use json format 2020-11-15 21:48:17 -08:00
Chris Lu ee2fa14dbe filer conf: delete location specific configuration 2020-11-15 20:15:47 -08:00
Chris Lu 71056dae07 fs.configure: read and local add filer configuration 2020-11-15 18:09:35 -08:00
Chris Lu 0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu 95c0de285d refactoring 2020-11-15 16:58:48 -08:00
Chris Lu 500bcab953 refactoring 2020-11-15 14:41:56 -08:00
Chris Lu 590f02179d filer: load filer conf when starting 2020-11-15 14:06:03 -08:00
Chris Lu 0a406f652e load filer conf and match by prefix 2020-11-15 00:26:05 -08:00
Chris Lu 0fc9ffc603 able to read filer.conf if updated 2020-11-14 21:21:20 -08:00
Chris Lu 442e092995 filer watch "/etc" folder for configuration changes 2020-11-14 14:26:08 -08:00
Konstantin Lebedev aa30604cec fix tests 2020-11-12 08:49:38 +05:00
Konstantin Lebedev 1eec5c8d5d gen pb 2020-11-12 04:10:06 +05:00
Konstantin Lebedev fc7baef5bb fiil serverUrls sorted by data center 2020-11-12 02:13:33 +05:00
Chris Lu 8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu df8d976bb0 refactoring 2020-11-01 01:58:48 -07:00
Chris Lu 16fae84414 fix format error 2020-10-24 20:11:31 -07:00
Chris Lu e9d40b80b2 less verbose logs 2020-10-24 09:42:54 -07:00
Chris Lu b81359823f postgres: support empty user 2020-10-22 14:27:47 -07:00
Chris Lu d75bc62196 to read files that are quickly changing 2020-10-22 12:59:15 -07:00
Chris Lu 5179e559f7 skip empty logs 2020-10-22 00:35:48 -07:00
Chris Lu bd103c143a add lock for vidCache 2020-10-21 19:28:59 -07:00
Chris Lu 68d39c86f1 mysql, postgres, cassandra: change kv key to base64 encoding 
The exisitng key-value operation for stores using mysql, postgres, and maybe cassandra are already broken.

The kv is used to store hardlink, filer store signature and replication progress.

So users using hardlink and also uses mysql, postgres, or cassandra will have broken hard links.

Users using filer.sync will need to re-sync the files.
 2020-10-16 11:10:12 -07:00
Chris Lu c95d3d9dff adjust tests 2020-10-14 12:27:52 -07:00
Chris Lu 93bcf56514 file read report EOF 
fix https://github.com/chrislusf/seaweedfs/issues/1344
 2020-10-14 12:18:24 -07:00
Chris Lu 0542911e29 go fmt 2020-10-13 19:50:46 -07:00
Chris Lu 58fa506491 minor 2020-10-13 19:50:22 -07:00
Chris Lu aac4cb1f0c adds errror on read and write 2020-10-13 13:53:34 -07:00
Chris Lu 3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu 723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu d155f907c2 mount: configurable read wait time 2020-10-10 20:09:43 -07:00
Chris Lu 8a52379ecb add retry if volume can not be found 2020-10-10 16:02:39 -07:00
Chris Lu 70af0ec24c filer: fix hanging on read 2020-10-10 16:02:10 -07:00
Chris Lu cff8bb6554 return proper error 2020-10-10 15:43:22 -07:00
Chris Lu 6da8eef54f fix logic error 2020-10-09 00:01:47 -07:00
Chris Lu 5d01dd28a6 add logging 2020-10-08 23:38:00 -07:00
Chris Lu 8d34eb0050 mount:exponentially backoff if read error for about 10 minutes 2020-10-08 23:31:26 -07:00
Chris Lu 6e1f936efd refactoring 2020-10-08 23:19:42 -07:00
Chris Lu b2ee5873fb fix error not being returned 2020-10-08 23:19:20 -07:00
Chris Lu 8676db0f6a adjust logging 2020-10-08 18:33:37 -07:00
Chris Lu eed492b73b randomize file locations 2020-10-07 23:58:32 -07:00
Chris Lu 62d3d3aea0 go fmt 2020-10-07 23:30:54 -07:00
Chris Lu a8624c2e4f read from alternative replica 
related to https://github.com/chrislusf/seaweedfs/issues/1512
 2020-10-07 22:49:04 -07:00
Chris Lu 36492c47ec adjust 2020-10-05 14:06:18 -07:00
Chris Lu 8f8e9ddbde adjust API 2020-10-05 09:47:07 -07:00
Konstantin Lebedev e4f2d9eb4a We return etag using the same algorithm as aws s3 
https://teppen.io/2018/06/23/aws_s3_etags/
 2020-10-05 14:43:32 +05:00
Chris Lu 6b591b02af adjust log level 2020-10-04 16:21:43 -07:00
Chris Lu 666859f7cd mount: read prefetching 2020-10-04 01:31:04 -07:00
Chris Lu bc47835997 mount: avoid duplicated reads 2020-10-03 20:16:42 -07:00
Chris Lu 8d65ad1444 Revert "mount: adds read prefetching" 
This reverts commit 692f0614d1.
 2020-10-03 19:40:56 -07:00
Chris Lu 692f0614d1 mount: adds read prefetching 2020-10-03 16:50:09 -07:00
limd ed7816681a Add cassandra authenticator mode 2020-09-25 11:22:47 +08:00
Chris Lu 0790c6d605 fix empty hard link id 2020-09-24 11:16:43 -07:00
Chris Lu 1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu 2e7c361a0d hardlink deletion factors in hardlink counter 2020-09-24 09:43:52 -07:00
Chris Lu 1295347958 adjust hardlink update 
simplify logic, pass entity content directly to hard link. The "weed mount" handles the logic to calculate hard link counter.
 2020-09-24 09:43:00 -07:00
Chris Lu 5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
Chris Lu b61d33f251 expose only store wrapper to meta changes 2020-09-22 16:27:36 -07:00
Chris Lu 852e5f7cbc filer: fix mongodb insert 
fix https://github.com/chrislusf/seaweedfs/issues/1471
 2020-09-17 21:50:52 -07:00
Chris Lu 10f9081526 filer: adjust meta data events to received timestamp 
if a client is already connected and start from t0. A message recieved at t+1 but with timestamp t-1 may not be processed by the client.

This commit changes to the event received time, so the replication can be ordered.
 2020-09-15 01:18:33 -07:00
Chris Lu b0c7de186d filer: fix postgres prefixed directory listing problem 
fix https://github.com/chrislusf/seaweedfs/issues/1465
 2020-09-12 13:37:03 -07:00
Chris Lu 446e476a11 go fmt 2020-09-12 04:08:03 -07:00
Chris Lu 2a0925590c filer: etcd store avoid read with nil option 
fix https://github.com/chrislusf/seaweedfs/issues/1463
 2020-09-11 15:29:45 -07:00
ruitao.liu 5b0676049a change elastic initialize process similar as others. 2020-09-10 23:35:20 +08:00
ruitao.liu 9be4e97625 change logs print format. 2020-09-10 16:30:15 +08:00
ruitao.liu 719dc43af1 modify elastic urls from string to array. 2020-09-10 16:24:09 +08:00
ruitao.liu 72f9d7f047 use util to generate md5. 2020-09-10 16:11:18 +08:00
ruitao.liu 6a5b38c0d4 fix elastic kv ops. 2020-09-10 15:59:16 +08:00
ruitao.liu 3f7fbfddca add more basic elastic options. 2020-09-10 14:22:07 +08:00
Chris Lu daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu 44b3f2efc8 filer: Elastic Search return correct kv error 
fix https://github.com/chrislusf/seaweedfs/issues/1452
 2020-09-07 12:44:51 -07:00
Chris Lu d8af6f62df adjust logging 2020-09-06 12:10:40 -07:00
Chris Lu 07f32feed0 print sync progress 2020-09-06 00:50:51 -07:00
Chris Lu 0af6252b2d fix compilation 2020-09-06 00:50:38 -07:00
Chris Lu ac494ff5e8 Update meta_aggregator.go 2020-09-06 00:29:16 -07:00
Chris Lu c9f8f25ba5 read peer filer from start 2020-09-06 00:12:41 -07:00
Chris Lu 30dc365cbd add self filer 2020-09-06 00:11:46 -07:00
Chris Lu 8acd714600 apply meta changes only if store is different 2020-09-05 23:25:03 -07:00
Chris Lu d741ed66db check whether shares the same filer store 2020-09-05 23:02:52 -07:00
Chris Lu bba90ff3c8 read filer signature 2020-09-05 22:52:15 -07:00
Chris Lu 71b0e256c7 filer: setOrLoadFilerStoreSignature 2020-09-05 14:08:59 -07:00
ruitao.liu 450cf07505 skip the index that for kv usage. 2020-09-04 21:49:03 +08:00
ruitao.liu 1384ff9a2f 1.split kv in one file. 
2.disable query for kv in es index.
 2020-09-04 17:34:26 +08:00
ruitao.liu 83080b5e03 ES backended filer support kv ops. 2020-09-04 15:40:13 +08:00
Chris Lu 46f65a84a6 filer: elastic7 adjust default value 2020-09-03 20:12:38 -07:00
Chris Lu f815fce81b
Merge pull request #1446 from taozix/master 
new filer store option to es7.
 2020-09-03 20:10:23 -07:00
Chris Lu 44b4ebf0ff filer: Redis cleanly delete directory 
fix https://github.com/chrislusf/seaweedfs/issues/1448
 2020-09-03 19:17:39 -07:00
Chris Lu b8f32bcab9 filer: compress stored metadata 2020-09-03 11:00:20 -07:00
ruitao.liu 798280e980 change filer2 to filer. 2020-09-03 17:05:26 +08:00
ruitao.liu a93d27d1e8 new filer option to es v7. 2020-09-03 16:34:58 +08:00
Chris Lu 68e878adb5 fix formatting 2020-09-02 21:42:12 -07:00
Chris Lu 0d99a5da3c c*: support kv operations 2020-09-02 18:39:24 -07:00