gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6009 commits 28 branches 278 tags 135 MiB
Commit graph

368 commits

Author SHA1 Message Date
Konstantin Lebedev 3325b850cc rm func CheckAllChunkViews 2021-05-21 16:05:59 +05:00
Konstantin Lebedev 26a4f34a57 del checks 
url err logging
stats chunk fetch
 2021-05-21 15:59:12 +05:00
Konstantin Lebedev 42f631f549 fix https://github.com/chrislusf/seaweedfs/issues/2046 2021-05-20 13:19:35 +05:00
Konstantin Lebedev 03d1199d5f Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader" 
This reverts commit ac71117e
 2021-05-20 11:45:21 +05:00
Chris Lu 1737af480a adjust logs 2021-05-10 21:47:51 -07:00
Chris Lu 55a8f57381 go fmt 2021-05-06 03:37:51 -07:00
Chris Lu ac71117ee6 revert PR #1903 avoid http error: superfluous response.WriteHeader 2021-05-05 15:11:39 -07:00
Chris Lu fa0d973113
Merge pull request #2037 from utsl42/reader_at 
make reader_at handle random reads more efficiently for FUSE
 2021-04-28 17:31:46 -07:00
Nathan Hawkins 042de9359c make reader_at handle random reads more efficiently for FUSE 2021-04-28 19:13:37 -04:00
Konstantin Lebedev c2269123d3 fix aws style Etag for chunks 2021-04-28 22:28:05 +05:00
Chris Lu 79f2e780c1 ensure name pattern checking is case sensitive 2021-04-24 11:51:23 -07:00
Chris Lu ddc8643ee0 filer: directory listing adds namePatternExclude 
fix https://github.com/chrislusf/seaweedfs/issues/2023
 2021-04-24 11:49:03 -07:00
Chris Lu 89eb9f6e70 clean up .uploads directory in mysql and postgres tables 
fix https://github.com/chrislusf/seaweedfs/issues/1957

When no uploads are running, you can run this SQL to clean up.
delete from <bucket_name> where directory like '/.uploads/%'
 2021-04-22 23:23:23 -07:00
Chris Lu d41e6826d3 adjust logging 2021-04-18 13:06:38 -07:00
Konstantin Lebedev 5021bea698 GetUserPolicy 2021-04-10 23:57:45 +05:00
Konstantin Lebedev 011e6e90ee Merge branch 'upstreamMaster' into iamapipr 2021-04-06 13:50:33 +05:00
Chris Lu 5e64f65632 ensure tables are created 
fix https://github.com/chrislusf/seaweedfs/issues/1957
 2021-04-02 12:02:26 -07:00
Chris Lu 8e404a1433 go fmt 2021-04-02 02:22:26 -07:00
Chris Lu e06807a55b adjust logs 2021-03-30 20:36:06 -07:00
LazyDBA247-Anyvision 7f44d953b5 fix GetBool 2021-03-30 01:36:02 +03:00
LazyDBA247-Anyvision 4c51e6a660 add enableUpsert=true 
and rename config to upsertQuery
 2021-03-30 00:32:03 +03:00
LazyDBA247-Anyvision 4a02389eb0 Adding custom insertQuery support for postgres/2 mysql/2 2021-03-29 09:58:13 +03:00
Chris Lu bd7471d877 refactor 2021-03-25 12:05:59 -07:00
Konstantin Lebedev 9f26f2815c SaveAs S3 Configuration 2021-03-25 12:34:11 +05:00
Chris Lu 2f7c7afdec refactor: remove unused parameter 2021-03-22 22:13:19 -07:00
Chris Lu d48dd0c738 use []byte directly instead of bytes.Buffer 2021-03-22 22:12:57 -07:00
Konstantin Lebedev eb54993a4e Merge branch 'upstreamMaster' into check_chunkviews_mr 
# Conflicts:
#	weed/filer/filechunk_manifest.go
#	weed/filer/stream.go
#	weed/replication/repl_util/replication_util.go
#	weed/util/fasthttp_util.go
 2021-03-16 15:29:49 +05:00
Chris Lu 69694a17be reverting 7d57664c2d 2021-03-16 02:59:26 -07:00
Konstantin Lebedev 06da02739d CheckAllChunkViews() for HEAD requests only 2021-03-16 14:15:17 +05:00
Chris Lu 4b1ed227d1 revert fasthttp changes 
related to https://github.com/chrislusf/seaweedfs/issues/1907
 2021-03-16 00:33:14 -07:00
Konstantin Lebedev 46b9f5cff4 add debug logging 2021-03-15 19:33:33 +05:00
Konstantin Lebedev 7194a5e7bf avoid http error: superfluous response.WriteHeader 
https://github.com/chrislusf/seaweedfs/issues/1838
 2021-03-15 18:52:59 +05:00
Chris Lu 35d939a1d2 go fmt 2021-03-14 13:21:02 -07:00
Chris Lu 2d4c2db81d filer: leveldb, rocksdb auto create store directory 
fix https://github.com/chrislusf/seaweedfs/issues/1901
 2021-03-14 13:20:14 -07:00
Chris Lu 6d3a96eb56 filer: mysql2, postgres2 trigger actions on bucket creation and deletion 
fix https://github.com/chrislusf/seaweedfs/issues/1877
 2021-03-13 22:07:39 -08:00
Chris Lu 7d57664c2d mount: internals switch to filer.Entry instead of protobuf 2021-03-12 00:36:38 -08:00
Chris Lu d084334ffd
Merge pull request #1883 from kmlebedev/passDelNonEmptyFolder 
s3 delete-objects pass OK for fail to delete non-empty folder
 2021-03-11 11:01:44 -08:00
Konstantin Lebedev 58cdcc6d6e error if urls not found 2021-03-11 23:34:36 +05:00
Konstantin Lebedev 4db21012c1 error if read chunk zero data size 2021-03-11 22:38:59 +05:00
Konstantin Lebedev 15b018da34 s3 delete-objects pass OK for fail to delete non-empty folder 2021-03-10 19:41:35 +05:00
Chris Lu 5be83dd4a9 add comments 2021-03-09 14:13:48 -08:00
Chris Lu 9d8ca513d9 refactor 2021-03-09 13:21:26 -08:00
Chris Lu 9abb041763 filer source: support filerProxy mode 2021-02-28 16:19:47 -08:00
Chris Lu 776f497469 filer: fs.configure should try to read from entry.content also 
related to https://github.com/chrislusf/seaweedfs/issues/1792
 2021-02-18 17:07:02 -08:00
Chris Lu 3575d41009 go fmt 2021-02-17 20:57:08 -08:00
Chris Lu 6daa932f5c refactoring to get master function, instead of passing master values directly 
this will enable retrying later
 2021-02-17 20:55:55 -08:00
Chris Lu 3f8b0da677 filer: do not print password on error 
fix https://github.com/chrislusf/seaweedfs/issues/1809
 2021-02-17 02:13:52 -08:00
Chris Lu b9b5b932c5 filer: postgres2 mysql2 avoid repeatedly creating tables 2021-02-15 16:19:24 -08:00
LazyDBA247-Anyvision 7f458d5e78 better postgres connection pool management 
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime)
to enable refresh of stale connections.
 2021-02-15 07:45:09 +02:00
LazyDBA247-Anyvision 51b4963e2e postgres2 & memsql2 
add escape (quote identifiers) for the dynamic sql
so tables (collections) with special characters will work.
 2021-02-14 13:14:36 +02:00
Chris Lu 7d9dc3c6a2 use fasthttp lib to read 2021-02-12 05:59:52 -08:00
Konstantin Lebedev 94eac4f00e Do reconnect to RabbitMQ 
https://github.com/google/go-cloud/issues/2958
https://github.com/chrislusf/seaweedfs/issues/1773
 2021-02-11 13:59:36 +05:00
Chris Lu 821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu c3af72d950 fier store: fix elastic search regression 
fix https://github.com/chrislusf/seaweedfs/issues/1774
 2021-02-04 01:30:14 -08:00
Chris Lu 2396ac234c filer: fix elastic search pagination 
possible fix for https://github.com/chrislusf/seaweedfs/issues/1774
 2021-02-02 11:34:12 -08:00
Chris Lu 4be51c0701 filer: leveldb and hbase may miss files when listing large directories more than 1024 
fix https://github.com/chrislusf/seaweedfs/issues/1768
 2021-01-31 20:11:44 -08:00
Chris Lu 314dc1c957 filer: etcd store fix listing 
fix https://github.com/chrislusf/seaweedfs/issues/1767
 2021-01-30 13:38:44 -08:00
Chris Lu 990fa69bfe add back AdjustedUrl() related code 2021-01-28 14:36:29 -08:00
Chris Lu 00707ec00f mount: outsideContainerClusterMode proxy through filer 
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
 2021-01-24 19:01:58 -08:00
Chris Lu 6ca10725b8 Revert "mount: when outside cluster network, use filer as proxy to access volume servers" 
This reverts commit 096e088d7b.
 2021-01-24 03:15:19 -08:00
Chris Lu 096e088d7b mount: when outside cluster network, use filer as proxy to access volume servers 2021-01-24 01:41:38 -08:00
Chris Lu d5add83e85 filer store: add postgres2 2021-01-19 18:07:29 -08:00
Chris Lu 52a8f1470e filer store: add mysql2 2021-01-19 17:21:50 -08:00
Chris Lu 93b3adba98 fix bucket creation 2021-01-19 15:55:51 -08:00
Chris Lu fa0c8d5283 fix error 2021-01-19 14:05:48 -08:00
Chris Lu 4c5b752b04 restructuring sql stores 2021-01-19 13:53:16 -08:00
Chris Lu ca8f793978 refactor: prepare for bucket specific tables 2021-01-19 12:34:58 -08:00
Chris Lu 90ce1eec19 add more input params 2021-01-19 11:32:55 -08:00
Chris Lu de876c795d minor fix 2021-01-18 01:14:27 -08:00
Chris Lu 09f49d1c04 refactoring 2021-01-16 19:52:15 -08:00
Chris Lu a4063a5437 add stream list directory entries 2021-01-15 23:56:24 -08:00
Chris Lu 01dc8a43ba fix rocksdb 2021-01-15 18:31:29 -08:00
Chris Lu 16ad74f477 go fmt 2021-01-14 23:11:27 -08:00
Chris Lu f002e668de change limit to int64 in case of overflow 2021-01-14 23:10:37 -08:00
Chris Lu 19e52fd414 pass along hasMore 2021-01-14 22:49:29 -08:00
Chris Lu c0bcf6f6e1 implement sql changes 2021-01-14 22:44:22 -08:00
Chris Lu 5d4568b91f implement elastic changes 2021-01-14 22:42:25 -08:00
Chris Lu 893cbc8482 implement c* changes 2021-01-14 22:38:34 -08:00
Chris Lu 5ef43b9b09 implement etcd changes 2021-01-14 22:35:56 -08:00
Chris Lu 54527f0326 implement hbase changes 2021-01-14 22:34:35 -08:00
Chris Lu b5ceffe188 implement leveldb changes 2021-01-14 22:33:05 -08:00
Chris Lu c64bfb0e2e implement mongodb changes 2021-01-14 22:28:51 -08:00
Chris Lu 34a846009d implement rocksdb changes 2021-01-14 22:24:26 -08:00
Chris Lu d2ece29fff implement redis changes 2021-01-14 22:21:31 -08:00
Chris Lu 9a50dbcda0 chagned api 2021-01-14 22:18:38 -08:00
Chris Lu e1c7bc66f9 change interface 2021-01-14 22:17:49 -08:00
Chris Lu 4aaa97c789 skip not found error in case of race condition 2021-01-14 19:56:14 -08:00
Chris Lu fd1d8a2a09 fix compilation 2021-01-14 00:06:15 -08:00
LazyDBA247-Anyvision 8eed763b97 better mysql connection pool management 
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime)
to enable refresh of connections.
 2021-01-14 08:14:21 +02:00
Chris Lu f17aa1d06c skip directory loop if dropping the bucket 2021-01-13 13:49:04 -08:00
Chris Lu ca73013453 leveldb3: ensure repeatable adding removing buckets 2021-01-13 13:48:48 -08:00
Chris Lu 5cd9c9e642 remove println 2021-01-12 18:47:38 -08:00
Chris Lu def6d8e565 fix test 2021-01-12 12:32:54 -08:00
Chris Lu 629c9962e7 filer: add leveldb3 
support dedicated leveldb instance for each bucket
 2021-01-12 02:29:44 -08:00
Chris Lu 361043e6c1 filer store: leveldb2 fix nil entry error if not found 2021-01-12 02:28:57 -08:00
Chris Lu cfb9342a15 avoid concurrent map updates to viper 2021-01-12 02:28:13 -08:00
Chris Lu acc0d5cd2c Revert "Revert "filer store: redis module upgrade"" 
This reverts commit dd0171e35c.
 2021-01-11 02:30:19 -08:00
Chris Lu dd0171e35c Revert "filer store: redis module upgrade" 
This reverts commit 40161c9cec.
 2021-01-11 02:09:18 -08:00
Chris Lu 394513f598 filer: ensure seamless meta data updates 2021-01-11 02:08:55 -08:00
Chris Lu e2e0e30a51 filer: redis default to false for useReadOnly and routeByLatency 2021-01-08 01:12:44 -08:00