Commit graph

5478 commits

Author SHA1 Message Date
Chris Lu 7e8edc3c4a refactoring 2021-04-02 01:10:24 -07:00
Chris Lu cc0df36a9e smaller file chunks are faster 2021-04-01 02:21:59 -07:00
Chris Lu cefe66f159 dedicated upload processor
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu 1f984d2645 refactor buffer pool 2021-04-01 02:20:00 -07:00
Chris Lu b5880334fc refactor 2021-03-30 21:07:34 -07:00
Chris Lu e79e2ddeed remove unused MoveEvents 2021-03-30 20:42:44 -07:00
Chris Lu e06807a55b adjust logs 2021-03-30 20:36:06 -07:00
Chris Lu a4db1fe32f patch https://github.com/advisories/GHSA-5mg8-w23w-74h3
https://github.com/advisories/GHSA-5mg8-w23w-74h3
2021-03-30 02:30:34 -07:00
Chris Lu ac875976c0 filer, volume: add concurrent upload size limit to avoid OOM
add some back pressure when writes are slow
2021-03-30 02:10:53 -07:00
Chris Lu a1e18a1384 server add memory profiling 2021-03-30 02:10:53 -07:00
Chris Lu 48188abc7b
Merge pull request #1952 from LazyDBA247-Anyvision/master
fix postgres2 upsertQuery syntax also
2021-03-30 01:01:59 -07:00
Chris Lu e5164a3c34
Merge pull request #1951 from chrislusf/dependabot/maven/other/java/client/com.google.guava-guava-29.0-jre
Bump guava from 28.0-jre to 29.0-jre in /other/java/client
2021-03-30 01:01:14 -07:00
Chris Lu 3472c87c21
Merge pull request #1950 from chrislusf/dependabot/maven/test/random_access/com.google.guava-guava-29.0-jre
Bump guava from 28.0-jre to 29.0-jre in /test/random_access
2021-03-30 01:00:54 -07:00
LazyDBA247-Anyvision 9385e2224d fix postgres2 upsertQuery syntax also 2021-03-30 10:51:52 +03:00
dependabot[bot] 9401890f18
Bump guava from 28.0-jre to 29.0-jre in /other/java/client
Bumps [guava](https://github.com/google/guava) from 28.0-jre to 29.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-30 07:46:34 +00:00
dependabot[bot] fb04fda28b
Bump guava from 28.0-jre to 29.0-jre in /test/random_access
Bumps [guava](https://github.com/google/guava) from 28.0-jre to 29.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-30 07:46:24 +00:00
Chris Lu a5662c9e5e adjust ordering 2021-03-30 00:26:57 -07:00
Chris Lu bec74c3e6a
Merge pull request #1948 from LazyDBA247-Anyvision/master
add enableUpsert=true
2021-03-30 00:25:01 -07:00
LazyDBA247-Anyvision 9f1cab179c use a compatible pg syntax for upsert 2021-03-30 10:21:27 +03:00
Chris Lu a95929e53c reduce default concurrentWriters to 32 2021-03-30 00:17:52 -07:00
Chris Lu be25bc6766 Java client 1.6.4 2021-03-30 00:17:35 -07:00
Chris Lu e72e3453e8 Java client: ByteBuffer.clear supports higher JDK versions 2021-03-29 21:42:28 -07:00
LazyDBA247-Anyvision 7f44d953b5 fix GetBool 2021-03-30 01:36:02 +03:00
LazyDBA247-Anyvision 4c51e6a660 add enableUpsert=true
and rename config to upsertQuery
2021-03-30 00:32:03 +03:00
Chris Lu 3cf84b5fae
Merge pull request #1947 from kmlebedev/minio-gateway-compose
minio gateway s3 docker compose file
2021-03-29 09:42:07 -07:00
Konstantin Lebedev 6b3cc94ac4 minio gateway s3 docker compose file 2021-03-29 14:59:59 +05:00
Chris Lu b61e9f6a1a adjust ordering 2021-03-29 00:49:50 -07:00
Chris Lu 1ebdbad222
Merge pull request #1946 from LazyDBA247-Anyvision/master
Adding Customisation - insertQuery for postgres/mysql
2021-03-29 00:42:48 -07:00
LazyDBA247-Anyvision 96c62bd34d scaffold - insertQuery
Example for upsert query for mysql & postgres variants
2021-03-29 10:01:12 +03:00
LazyDBA247-Anyvision 4a02389eb0 Adding custom insertQuery support for postgres/2 mysql/2 2021-03-29 09:58:13 +03:00
Chris Lu 6b7aa9633f 2.36 2021-03-28 19:09:06 -07:00
Chris Lu bd7471d877 refactor 2021-03-25 12:05:59 -07:00
Chris Lu b8d3f11e92
Merge pull request #1945 from bash99/master
Add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id
2021-03-25 10:29:07 -07:00
李海 06be5dc6c3 log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid 2021-03-25 18:59:40 +08:00
李海 69b2dab9c6 add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id 2021-03-25 18:49:26 +08:00
Chris Lu 82dfe06066
Merge pull request #1933 from kmlebedev/missingKeysThreshold
add missingKeysThreshold for fsck
2021-03-24 10:56:25 -07:00
Konstantin Lebedev 020a5d40c3 avoid counter is null 2021-03-24 22:51:22 +05:00
Konstantin Lebedev df6cf0a2fa nonRepairThreshold 2021-03-24 22:07:13 +05:00
Chris Lu 26657f1d72
Merge pull request #1943 from kmlebedev/maxCPUs
GOMAXPROCS use defaults to the value of runtime.NumCPU
2021-03-24 09:29:38 -07:00
Konstantin Lebedev 9ce82a9cd2 GOMAXPROCS use defaults to the value of runtime.NumCPU
https://github.com/chrislusf/seaweedfs/issues/1942
2021-03-24 18:45:58 +05:00
Konstantin Lebedev 60972f1c97 rename option repairThreshold 2021-03-24 13:24:49 +05:00
Chris Lu a801332b0d filer: return 409 if file conflicts with a directory
fix https://github.com/chrislusf/seaweedfs/issues/1938
2021-03-24 01:03:11 -07:00
Chris Lu c42b95c596 more help message
fix https://github.com/chrislusf/seaweedfs/issues/1937
2021-03-23 17:27:57 -07:00
Konstantin Lebedev 40a9e88c07 rename var 2021-03-23 21:42:35 +05:00
Konstantin Lebedev d21a5bf5d0 add missingKeysThreshold for fsck 2021-03-23 15:04:07 +05:00
Chris Lu 85cff10787 mysql: ensure name column is case sensitive
fix https://github.com/chrislusf/seaweedfs/issues/1931
2021-03-23 00:46:50 -07:00
Chris Lu 5f0e68c21e revert wrong change 2021-03-22 23:28:40 -07:00
Chris Lu b5e10bf511 Java client 1.6.3 2021-03-22 23:25:47 -07:00
Chris Lu 5a78c6ca99 Java client: support windows 2021-03-22 23:20:41 -07:00
Chris Lu 4abb511db3 make a local copy of the in memory cached data 2021-03-22 22:33:07 -07:00