Commit graph

3500 commits

Author SHA1 Message Date
Chris Lu 319743d330 go fmt 2021-04-02 02:22:59 -07:00
Chris Lu 8e404a1433 go fmt 2021-04-02 02:22:26 -07:00
Chris Lu bdf2ddddfd revert to same implementation as before
This reverts commit 7e8edc3c4a.
2021-04-02 02:21:38 -07:00
Chris Lu 67e019d54b add missing changes 2021-04-02 01:11:44 -07:00
Chris Lu 7e8edc3c4a refactoring 2021-04-02 01:10:24 -07:00
Chris Lu cc0df36a9e smaller file chunks are faster 2021-04-01 02:21:59 -07:00
Chris Lu cefe66f159 dedicated upload processor
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu 1f984d2645 refactor buffer pool 2021-04-01 02:20:00 -07:00
Chris Lu b5880334fc refactor 2021-03-30 21:07:34 -07:00
Chris Lu e79e2ddeed remove unused MoveEvents 2021-03-30 20:42:44 -07:00
Chris Lu e06807a55b adjust logs 2021-03-30 20:36:06 -07:00
Chris Lu ac875976c0 filer, volume: add concurrent upload size limit to avoid OOM
add some back pressure when writes are slow
2021-03-30 02:10:53 -07:00
Chris Lu a1e18a1384 server add memory profiling 2021-03-30 02:10:53 -07:00
LazyDBA247-Anyvision 9385e2224d fix postgres2 upsertQuery syntax also 2021-03-30 10:51:52 +03:00
Chris Lu a5662c9e5e adjust ordering 2021-03-30 00:26:57 -07:00
Chris Lu bec74c3e6a
Merge pull request #1948 from LazyDBA247-Anyvision/master
add enableUpsert=true
2021-03-30 00:25:01 -07:00
LazyDBA247-Anyvision 9f1cab179c use a compatible pg syntax for upsert 2021-03-30 10:21:27 +03:00
Chris Lu a95929e53c reduce default concurrentWriters to 32 2021-03-30 00:17:52 -07:00
LazyDBA247-Anyvision 7f44d953b5 fix GetBool 2021-03-30 01:36:02 +03:00
LazyDBA247-Anyvision 4c51e6a660 add enableUpsert=true
and rename config to upsertQuery
2021-03-30 00:32:03 +03:00
Chris Lu b61e9f6a1a adjust ordering 2021-03-29 00:49:50 -07:00
Chris Lu 1ebdbad222
Merge pull request #1946 from LazyDBA247-Anyvision/master
Adding Customisation - insertQuery for postgres/mysql
2021-03-29 00:42:48 -07:00
LazyDBA247-Anyvision 96c62bd34d scaffold - insertQuery
Example for upsert query for mysql & postgres variants
2021-03-29 10:01:12 +03:00
LazyDBA247-Anyvision 4a02389eb0 Adding custom insertQuery support for postgres/2 mysql/2 2021-03-29 09:58:13 +03:00
Chris Lu 6b7aa9633f 2.36 2021-03-28 19:09:06 -07:00
Chris Lu bd7471d877 refactor 2021-03-25 12:05:59 -07:00
李海 06be5dc6c3 log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid 2021-03-25 18:59:40 +08:00
李海 69b2dab9c6 add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id 2021-03-25 18:49:26 +08:00
Chris Lu 82dfe06066
Merge pull request #1933 from kmlebedev/missingKeysThreshold
add missingKeysThreshold for fsck
2021-03-24 10:56:25 -07:00
Konstantin Lebedev 020a5d40c3 avoid counter is null 2021-03-24 22:51:22 +05:00
Konstantin Lebedev df6cf0a2fa nonRepairThreshold 2021-03-24 22:07:13 +05:00
Konstantin Lebedev 9ce82a9cd2 GOMAXPROCS use defaults to the value of runtime.NumCPU
https://github.com/chrislusf/seaweedfs/issues/1942
2021-03-24 18:45:58 +05:00
Konstantin Lebedev 60972f1c97 rename option repairThreshold 2021-03-24 13:24:49 +05:00
Chris Lu a801332b0d filer: return 409 if file conflicts with a directory
fix https://github.com/chrislusf/seaweedfs/issues/1938
2021-03-24 01:03:11 -07:00
Chris Lu c42b95c596 more help message
fix https://github.com/chrislusf/seaweedfs/issues/1937
2021-03-23 17:27:57 -07:00
Konstantin Lebedev 40a9e88c07 rename var 2021-03-23 21:42:35 +05:00
Konstantin Lebedev d21a5bf5d0 add missingKeysThreshold for fsck 2021-03-23 15:04:07 +05:00
Chris Lu 85cff10787 mysql: ensure name column is case sensitive
fix https://github.com/chrislusf/seaweedfs/issues/1931
2021-03-23 00:46:50 -07:00
Chris Lu 4abb511db3 make a local copy of the in memory cached data 2021-03-22 22:33:07 -07:00
Chris Lu 3cbc40fa48 avoid creating multiple reader 2021-03-22 22:32:47 -07:00
Chris Lu 2f7c7afdec refactor: remove unused parameter 2021-03-22 22:13:19 -07:00
Chris Lu d48dd0c738 use []byte directly instead of bytes.Buffer 2021-03-22 22:12:57 -07:00
Chris Lu 8eb14967ef shell: volume.check.disk adds optional slow mode
fix https://github.com/chrislusf/seaweedfs/issues/1930
2021-03-22 21:01:43 -07:00
Chris Lu 288369cfc7 mount: release resources when Forget() is called
address https://github.com/chrislusf/seaweedfs/issues/1929
2021-03-22 16:22:19 -07:00
Chris Lu 5d931eff27 avoid possible nil
fix https://github.com/chrislusf/seaweedfs/issues/1928

The nil was because of `dn.Parent().UnlinkChildNode(dn.Id())` in topo.UnRegisterDataNode() function, when the dn leaves the cluster.
2021-03-22 13:24:07 -07:00
Chris Lu f315eb2bb8 fix build error 2021-03-22 00:12:53 -07:00
Chris Lu 1dd5bc134c 2.35 2021-03-22 00:05:09 -07:00
Chris Lu b465095db1 shell: add volume.check.disk to fix inconsistency for replicated volumes
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu df461402cc ensure entry attribute is not nill
fix https://github.com/chrislusf/seaweedfs/issues/1926
2021-03-21 22:27:30 -07:00
Chris Lu 102a951377 refactor, split into 2 files 2021-03-21 13:05:53 -07:00