gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
5010 commits 28 branches 278 tags 135 MiB
Commit graph

255 commits

Author SHA1 Message Date
qieqieplus bbae13e415 impl: TTL per entry for rocksdb; fix package name 2021-01-05 17:58:40 +08:00
qieqieplus ebb223c190 fix for rocksdb kv 2021-01-04 21:21:46 +08:00
chenqieqie 40b3207f18 fix #1726 2021-01-04 18:48:55 +08:00
Chris Lu bdd166c031 adjust readme 2021-01-03 00:47:58 -08:00
Chris Lu f4d87f8b9d works 
performance seems about the same as leveldb for insert entries
 2021-01-02 23:32:58 -08:00
Chris Lu 2c3c2c27d7 separate prefix from namePattern 
fix https://github.com/chrislusf/seaweedfs/issues/1722
 2021-01-01 20:23:23 -08:00
Chris Lu da7e5aaa65 filer: optimize for less number of directory lookup 
bottom up directory lookup
 2020-12-27 20:46:17 -08:00
Chris Lu da87f6b265 remove unused code 2020-12-26 15:21:12 -08:00
Chris Lu 0a7c5f85a9 filer: add namePattern to search in current folder 2020-12-26 15:05:31 -08:00
Chris Lu 0e016bc7bd hbase add ttl 2020-12-24 12:10:35 -08:00
Chris Lu 9bf6c10505 fix prefix 2020-12-24 00:18:59 -08:00
Chris Lu 2fea8cfa0f fix compilation 2020-12-23 23:54:15 -08:00
Chris Lu 53bc1ea25b fix compilation 2020-12-23 23:53:46 -08:00
Chris Lu b5e2be635a adjust for directory listing 2020-12-23 23:49:22 -08:00
Chris Lu c4a202ec41 fix wrong column family 2020-12-23 23:23:05 -08:00
Chris Lu 94e3757c08 fix 2020-12-23 21:45:16 -08:00
Chris Lu 64c48c9724 seems compiling 2020-12-23 21:44:53 -08:00
Chris Lu a0990b929d minor 2020-12-22 17:33:40 -08:00
Chris Lu 97e3432dfe avoid wrong error 
fix https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-22 02:46:24 -08:00
Chris Lu 4f31c1bb94 go fmt 2020-12-22 02:34:08 -08:00
Chris Lu 6c4f32d173 remove unused code 2020-12-22 02:33:31 -08:00
Chris Lu 90d785a15f filer: redis, redis cluster, cassandra support super large directory 2020-12-22 02:26:05 -08:00
Chris Lu fe46411cd4 handle directory listing edge cases 2020-12-21 23:37:43 -08:00
Chris Lu c74bede730 handle listing directories 2020-12-21 23:19:05 -08:00
Chris Lu adf8cb4000 filer: path-specific stores trim out common prefixes 2020-12-21 22:57:13 -08:00
Chris Lu 0823bde8d8 refactor: split into two files 2020-12-21 22:05:15 -08:00
Chris Lu 8ab70ccf76 refactor 2020-12-21 21:46:32 -08:00
Chris Lu 65cc6dc636 refactor to dedicated function 2020-12-21 21:18:34 -08:00
Chris Lu 488c2680e8 change default filer store root 2020-12-21 21:15:22 -08:00
Chris Lu f5a19a87be filer: report error if the path specific store is not found 
related https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-21 12:43:52 -08:00
Chris Lu 41c0f3ad24 filer: support path-specific filer store 2020-12-19 01:27:09 -08:00
Chris Lu 0d5683fb0e todo: load path-specific store from filer.toml 2020-12-18 03:05:42 -08:00
Chris Lu 3269fd7eaf filer: use store by path 2020-12-18 02:57:49 -08:00
Chris Lu e605f1e001 refactor, add getActualStore() by path 2020-12-18 02:55:00 -08:00
Chris Lu 23903aa95b refactoring 2020-12-18 02:35:45 -08:00
Chris Lu 4e6b316913 assert FilerStoreWrapper is VirtualFilerStore 2020-12-18 02:17:06 -08:00
Chris Lu 38fc89d041 filer store: sql update if any insert error happens 
fix https://github.com/chrislusf/seaweedfs/issues/1673
 2020-12-13 20:49:44 -08:00
Chris Lu 23280257df sql put kv: avoid unnecessary update 2020-12-13 19:45:47 -08:00
Chris Lu 14910d035c the pagination size was too big for recursive deletion 2020-12-12 13:26:10 -08:00
Chris Lu 03637d6f57 s3: move "delete-directory-if-empty" to read time 
move "delete-directory-if-empty" to read time instead of entry deletion time

the listing speed for a s3 bucket folder will slow down if it has many sub folders

related to 0d345ac97d

fix https://github.com/chrislusf/seaweedfs/issues/1647

fix https://github.com/chrislusf/seaweedfs/issues/1670
 2020-12-12 03:38:34 -08:00
Chris Lu 37075a414d adjust logs 2020-12-12 03:33:57 -08:00
Chris Lu 3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu c2f18a10cb minor 2020-12-10 22:23:22 -08:00
Chris Lu 4b0c2a846b add logs 2020-12-10 19:55:28 -08:00
Chris Lu 83078ac6ce filer: change to /etc/seaweedfs folder on filer 
fix https://github.com/chrislusf/seaweedfs/issues/1666
 2020-12-10 11:11:02 -08:00
Chris Lu 765b3ef1be save /etc/iam/identity.json inside filer store 2020-12-10 00:15:22 -08:00
Chris Lu a9c619c270 added error handling for saving 2020-12-09 01:34:24 -08:00
Chris Lu 8e78187a97 add back last read chunk cache to reader and properly close the reader 2020-12-08 22:26:46 -08:00
Chris Lu 900d22c6ec mount: avoid memory leaking read buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1654

the reader goes together with the file handle, which may stay for a long time.
 2020-12-08 02:38:53 -08:00
Chris Lu fce8803087 break import cycle 2020-12-07 00:29:17 -08:00
Chris Lu 01e2da5782 refactoring 2020-12-06 23:16:20 -08:00
Chris Lu 9acda432fe fix import cycle 2020-12-06 20:12:52 -08:00
Chris Lu 5931a2f53f filer conf: support json conf only 2020-12-06 19:47:06 -08:00
Chris Lu f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu a9c6be5fc3 set creation time 2020-11-30 03:11:52 -08:00
Chris Lu 0d345ac97d s3: remove empty parent folder on delete 
fix https://github.com/chrislusf/seaweedfs/issues/1637
 2020-11-26 12:21:58 -08:00
Chris Lu cc2bd97ad9 refactor 2020-11-26 11:25:56 -08:00
Chris Lu 0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu effa00ed08 refactor 2020-11-26 11:21:58 -08:00
Chris Lu 513bcd6e0d filer: avoid duplicated FindEntry for deletion 2020-11-26 11:14:56 -08:00
Chris Lu eab53ea80d filer leveldb store: a bit more efficient directory listing with prefix 2020-11-22 21:10:41 -08:00
Chris Lu 290b5e2cd0 directly delete file chunks 
keeping current async deletions for now
 2020-11-17 17:20:21 -08:00
Chris Lu dc304342b2 fs.configure: configurable volume growth 2020-11-17 01:00:02 -08:00
Chris Lu 5f19e81dab filer confi: support hierachical configuration 2020-11-16 16:50:12 -08:00
Chris Lu c0d279c54e filere.conf: prefer to use json format 2020-11-15 21:48:17 -08:00
Chris Lu ee2fa14dbe filer conf: delete location specific configuration 2020-11-15 20:15:47 -08:00
Chris Lu 71056dae07 fs.configure: read and local add filer configuration 2020-11-15 18:09:35 -08:00
Chris Lu 0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu 95c0de285d refactoring 2020-11-15 16:58:48 -08:00
Chris Lu 500bcab953 refactoring 2020-11-15 14:41:56 -08:00
Chris Lu 590f02179d filer: load filer conf when starting 2020-11-15 14:06:03 -08:00
Chris Lu 0a406f652e load filer conf and match by prefix 2020-11-15 00:26:05 -08:00
Chris Lu 0fc9ffc603 able to read filer.conf if updated 2020-11-14 21:21:20 -08:00
Chris Lu 442e092995 filer watch "/etc" folder for configuration changes 2020-11-14 14:26:08 -08:00
Konstantin Lebedev aa30604cec fix tests 2020-11-12 08:49:38 +05:00
Konstantin Lebedev 1eec5c8d5d gen pb 2020-11-12 04:10:06 +05:00
Konstantin Lebedev fc7baef5bb fiil serverUrls sorted by data center 2020-11-12 02:13:33 +05:00
Chris Lu 8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu df8d976bb0 refactoring 2020-11-01 01:58:48 -07:00
Chris Lu 16fae84414 fix format error 2020-10-24 20:11:31 -07:00
Chris Lu e9d40b80b2 less verbose logs 2020-10-24 09:42:54 -07:00
Chris Lu b81359823f postgres: support empty user 2020-10-22 14:27:47 -07:00
Chris Lu d75bc62196 to read files that are quickly changing 2020-10-22 12:59:15 -07:00
Chris Lu 5179e559f7 skip empty logs 2020-10-22 00:35:48 -07:00
Chris Lu bd103c143a add lock for vidCache 2020-10-21 19:28:59 -07:00
Chris Lu 68d39c86f1 mysql, postgres, cassandra: change kv key to base64 encoding 
The exisitng key-value operation for stores using mysql, postgres, and maybe cassandra are already broken.

The kv is used to store hardlink, filer store signature and replication progress.

So users using hardlink and also uses mysql, postgres, or cassandra will have broken hard links.

Users using filer.sync will need to re-sync the files.
 2020-10-16 11:10:12 -07:00
Chris Lu c95d3d9dff adjust tests 2020-10-14 12:27:52 -07:00
Chris Lu 93bcf56514 file read report EOF 
fix https://github.com/chrislusf/seaweedfs/issues/1344
 2020-10-14 12:18:24 -07:00
Chris Lu 0542911e29 go fmt 2020-10-13 19:50:46 -07:00
Chris Lu 58fa506491 minor 2020-10-13 19:50:22 -07:00
Chris Lu aac4cb1f0c adds errror on read and write 2020-10-13 13:53:34 -07:00
Chris Lu 3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu 723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu d155f907c2 mount: configurable read wait time 2020-10-10 20:09:43 -07:00
Chris Lu 8a52379ecb add retry if volume can not be found 2020-10-10 16:02:39 -07:00
Chris Lu 70af0ec24c filer: fix hanging on read 2020-10-10 16:02:10 -07:00
Chris Lu cff8bb6554 return proper error 2020-10-10 15:43:22 -07:00
Chris Lu 6da8eef54f fix logic error 2020-10-09 00:01:47 -07:00
Chris Lu 5d01dd28a6 add logging 2020-10-08 23:38:00 -07:00
Chris Lu 8d34eb0050 mount:exponentially backoff if read error for about 10 minutes 2020-10-08 23:31:26 -07:00