gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
5592 commits 28 branches 278 tags 135 MiB
Commit graph

306 commits

Author SHA1 Message Date
Chris Lu 52a8f1470e filer store: add mysql2 2021-01-19 17:21:50 -08:00
Chris Lu 93b3adba98 fix bucket creation 2021-01-19 15:55:51 -08:00
Chris Lu fa0c8d5283 fix error 2021-01-19 14:05:48 -08:00
Chris Lu 4c5b752b04 restructuring sql stores 2021-01-19 13:53:16 -08:00
Chris Lu ca8f793978 refactor: prepare for bucket specific tables 2021-01-19 12:34:58 -08:00
Chris Lu 90ce1eec19 add more input params 2021-01-19 11:32:55 -08:00
Chris Lu de876c795d minor fix 2021-01-18 01:14:27 -08:00
Chris Lu 09f49d1c04 refactoring 2021-01-16 19:52:15 -08:00
Chris Lu a4063a5437 add stream list directory entries 2021-01-15 23:56:24 -08:00
Chris Lu 01dc8a43ba fix rocksdb 2021-01-15 18:31:29 -08:00
Chris Lu 16ad74f477 go fmt 2021-01-14 23:11:27 -08:00
Chris Lu f002e668de change limit to int64 in case of overflow 2021-01-14 23:10:37 -08:00
Chris Lu 19e52fd414 pass along hasMore 2021-01-14 22:49:29 -08:00
Chris Lu c0bcf6f6e1 implement sql changes 2021-01-14 22:44:22 -08:00
Chris Lu 5d4568b91f implement elastic changes 2021-01-14 22:42:25 -08:00
Chris Lu 893cbc8482 implement c* changes 2021-01-14 22:38:34 -08:00
Chris Lu 5ef43b9b09 implement etcd changes 2021-01-14 22:35:56 -08:00
Chris Lu 54527f0326 implement hbase changes 2021-01-14 22:34:35 -08:00
Chris Lu b5ceffe188 implement leveldb changes 2021-01-14 22:33:05 -08:00
Chris Lu c64bfb0e2e implement mongodb changes 2021-01-14 22:28:51 -08:00
Chris Lu 34a846009d implement rocksdb changes 2021-01-14 22:24:26 -08:00
Chris Lu d2ece29fff implement redis changes 2021-01-14 22:21:31 -08:00
Chris Lu 9a50dbcda0 chagned api 2021-01-14 22:18:38 -08:00
Chris Lu e1c7bc66f9 change interface 2021-01-14 22:17:49 -08:00
Chris Lu 4aaa97c789 skip not found error in case of race condition 2021-01-14 19:56:14 -08:00
Chris Lu fd1d8a2a09 fix compilation 2021-01-14 00:06:15 -08:00
LazyDBA247-Anyvision 8eed763b97 better mysql connection pool management 
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime)
to enable refresh of connections.
 2021-01-14 08:14:21 +02:00
Chris Lu f17aa1d06c skip directory loop if dropping the bucket 2021-01-13 13:49:04 -08:00
Chris Lu ca73013453 leveldb3: ensure repeatable adding removing buckets 2021-01-13 13:48:48 -08:00
Chris Lu 5cd9c9e642 remove println 2021-01-12 18:47:38 -08:00
Chris Lu def6d8e565 fix test 2021-01-12 12:32:54 -08:00
Chris Lu 629c9962e7 filer: add leveldb3 
support dedicated leveldb instance for each bucket
 2021-01-12 02:29:44 -08:00
Chris Lu 361043e6c1 filer store: leveldb2 fix nil entry error if not found 2021-01-12 02:28:57 -08:00
Chris Lu cfb9342a15 avoid concurrent map updates to viper 2021-01-12 02:28:13 -08:00
Chris Lu acc0d5cd2c Revert "Revert "filer store: redis module upgrade"" 
This reverts commit dd0171e35c.
 2021-01-11 02:30:19 -08:00
Chris Lu dd0171e35c Revert "filer store: redis module upgrade" 
This reverts commit 40161c9cec.
 2021-01-11 02:09:18 -08:00
Chris Lu 394513f598 filer: ensure seamless meta data updates 2021-01-11 02:08:55 -08:00
Chris Lu e2e0e30a51 filer: redis default to false for useReadOnly and routeByLatency 2021-01-08 01:12:44 -08:00
Chris Lu 40161c9cec filer store: redis module upgrade 2021-01-08 01:08:44 -08:00
Chris Lu 07f4703bfc
Merge pull request #1738 from qieqieplus/rocksdb 
fix rocksdb crash when list directory
 2021-01-07 02:26:41 -08:00
qieqieplus 0f18592315 alter style 2021-01-07 18:21:48 +08:00
qieqieplus c7e7431fd3 fix seek lastkey may reach EOF 2021-01-07 17:07:56 +08:00
Chris Lu e327385644 go fmt 2021-01-06 04:22:00 -08:00
Chris Lu 2b76854641 add "weed filer.cat" to read files directly from volume servers 2021-01-06 04:22:00 -08:00
qieqieplus 0764fccde7 ignore decode error for non-entry data 2021-01-06 17:42:43 +08:00
qieqieplus b2e50f602f
fix time comparison 2021-01-05 19:36:40 +08:00
qieqieplus bbae13e415 impl: TTL per entry for rocksdb; fix package name 2021-01-05 17:58:40 +08:00
qieqieplus ebb223c190 fix for rocksdb kv 2021-01-04 21:21:46 +08:00
chenqieqie 40b3207f18 fix #1726 2021-01-04 18:48:55 +08:00
Chris Lu bdd166c031 adjust readme 2021-01-03 00:47:58 -08:00
Chris Lu f4d87f8b9d works 
performance seems about the same as leveldb for insert entries
 2021-01-02 23:32:58 -08:00
Chris Lu 2c3c2c27d7 separate prefix from namePattern 
fix https://github.com/chrislusf/seaweedfs/issues/1722
 2021-01-01 20:23:23 -08:00
Chris Lu da7e5aaa65 filer: optimize for less number of directory lookup 
bottom up directory lookup
 2020-12-27 20:46:17 -08:00
Chris Lu da87f6b265 remove unused code 2020-12-26 15:21:12 -08:00
Chris Lu 0a7c5f85a9 filer: add namePattern to search in current folder 2020-12-26 15:05:31 -08:00
Chris Lu 0e016bc7bd hbase add ttl 2020-12-24 12:10:35 -08:00
Chris Lu 9bf6c10505 fix prefix 2020-12-24 00:18:59 -08:00
Chris Lu 2fea8cfa0f fix compilation 2020-12-23 23:54:15 -08:00
Chris Lu 53bc1ea25b fix compilation 2020-12-23 23:53:46 -08:00
Chris Lu b5e2be635a adjust for directory listing 2020-12-23 23:49:22 -08:00
Chris Lu c4a202ec41 fix wrong column family 2020-12-23 23:23:05 -08:00
Chris Lu 94e3757c08 fix 2020-12-23 21:45:16 -08:00
Chris Lu 64c48c9724 seems compiling 2020-12-23 21:44:53 -08:00
Chris Lu 1c7e1295dc Merge branch 'master' into support_ssd_volume 2020-12-22 17:44:52 -08:00
Chris Lu a0990b929d minor 2020-12-22 17:33:40 -08:00
Chris Lu 97e3432dfe avoid wrong error 
fix https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-22 02:46:24 -08:00
Chris Lu 4f31c1bb94 go fmt 2020-12-22 02:34:08 -08:00
Chris Lu 6c4f32d173 remove unused code 2020-12-22 02:33:31 -08:00
Chris Lu 90d785a15f filer: redis, redis cluster, cassandra support super large directory 2020-12-22 02:26:05 -08:00
Chris Lu fe46411cd4 handle directory listing edge cases 2020-12-21 23:37:43 -08:00
Chris Lu c74bede730 handle listing directories 2020-12-21 23:19:05 -08:00
Chris Lu adf8cb4000 filer: path-specific stores trim out common prefixes 2020-12-21 22:57:13 -08:00
Chris Lu 0823bde8d8 refactor: split into two files 2020-12-21 22:05:15 -08:00
Chris Lu 8ab70ccf76 refactor 2020-12-21 21:46:32 -08:00
Chris Lu 65cc6dc636 refactor to dedicated function 2020-12-21 21:18:34 -08:00
Chris Lu 488c2680e8 change default filer store root 2020-12-21 21:15:22 -08:00
Chris Lu f5a19a87be filer: report error if the path specific store is not found 
related https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-21 12:43:52 -08:00
Chris Lu 41c0f3ad24 filer: support path-specific filer store 2020-12-19 01:27:09 -08:00
Chris Lu 0d5683fb0e todo: load path-specific store from filer.toml 2020-12-18 03:05:42 -08:00
Chris Lu 3269fd7eaf filer: use store by path 2020-12-18 02:57:49 -08:00
Chris Lu e605f1e001 refactor, add getActualStore() by path 2020-12-18 02:55:00 -08:00
Chris Lu 23903aa95b refactoring 2020-12-18 02:35:45 -08:00
Chris Lu 4e6b316913 assert FilerStoreWrapper is VirtualFilerStore 2020-12-18 02:17:06 -08:00
Chris Lu 1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu c45d6a865b Merge branch 'master' into support_ssd_volume 2020-12-13 22:14:18 -08:00
Chris Lu 38fc89d041 filer store: sql update if any insert error happens 
fix https://github.com/chrislusf/seaweedfs/issues/1673
 2020-12-13 20:49:44 -08:00
Chris Lu 23280257df sql put kv: avoid unnecessary update 2020-12-13 19:45:47 -08:00
Chris Lu 0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu 14910d035c the pagination size was too big for recursive deletion 2020-12-12 13:26:10 -08:00
Chris Lu 03637d6f57 s3: move "delete-directory-if-empty" to read time 
move "delete-directory-if-empty" to read time instead of entry deletion time

the listing speed for a s3 bucket folder will slow down if it has many sub folders

related to 0d345ac97d

fix https://github.com/chrislusf/seaweedfs/issues/1647

fix https://github.com/chrislusf/seaweedfs/issues/1670
 2020-12-12 03:38:34 -08:00
Chris Lu 37075a414d adjust logs 2020-12-12 03:33:57 -08:00
Chris Lu 3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu c2f18a10cb minor 2020-12-10 22:23:22 -08:00
Chris Lu 4b0c2a846b add logs 2020-12-10 19:55:28 -08:00
Chris Lu 83078ac6ce filer: change to /etc/seaweedfs folder on filer 
fix https://github.com/chrislusf/seaweedfs/issues/1666
 2020-12-10 11:11:02 -08:00
Chris Lu 765b3ef1be save /etc/iam/identity.json inside filer store 2020-12-10 00:15:22 -08:00
Chris Lu a9c619c270 added error handling for saving 2020-12-09 01:34:24 -08:00
Chris Lu 8e78187a97 add back last read chunk cache to reader and properly close the reader 2020-12-08 22:26:46 -08:00
Chris Lu 900d22c6ec mount: avoid memory leaking read buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1654

the reader goes together with the file handle, which may stay for a long time.
 2020-12-08 02:38:53 -08:00