Commit graph

3177 commits

Author SHA1 Message Date
Chris Lu d848d08944 use public url also for deletions 2020-03-18 11:16:45 -07:00
Chris Lu 6566c8e114 weed upload: add usePublicUrl option
fix https://github.com/chrislusf/seaweedfs/issues/1236
2020-03-18 10:50:53 -07:00
Chris Lu db9854fa0a 1.64 2020-03-17 10:58:35 -07:00
Chris Lu bec6ec7db6 go fmt 2020-03-17 10:01:55 -07:00
Chris Lu b964bbab3d fix compilation 2020-03-17 10:01:24 -07:00
Chris Lu c3cb6fa1d7 volume: compaction can cause readonly volumes
address https://github.com/chrislusf/seaweedfs/issues/1233
2020-03-17 09:43:57 -07:00
Chris Lu 12df236def
Update README.md 2020-03-16 15:41:15 -07:00
Chris Lu 1e251b8cdf
Update README.md 2020-03-16 15:34:23 -07:00
Chris Lu 08bea09841
Update README.md 2020-03-16 00:03:56 -07:00
Chris Lu f9b3750ad1 volume: handle repeated writes response 2020-03-15 04:16:00 -07:00
Chris Lu 22400c6633 consistent gzip logic
local store gzip same as replicated writes
2020-03-15 04:15:40 -07:00
Chris Lu 560df51def refactoring 2020-03-15 03:11:26 -07:00
Chris Lu 7edbee6f57 volume: proxy writes to remote volume server, with replication or not
the panic is triggered by uploading a file to a volume server not holding the designated replica.
2020-03-15 10:20:14.365488 I | http: panic serving 127.0.0.1:57124: runtime error: invalid memory address or nil pointer dereference
goroutine 119 [running]:
net/http.(*conn).serve.func1(0xc0001a8000)
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:1772 +0x139
panic(0x2316fe0, 0x3662900)
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/runtime/panic.go:973 +0x396
github.com/chrislusf/seaweedfs/weed/topology.getWritableRemoteReplications(0xc00009c000, 0x2, 0x7ffeefbffbd2, 0xe, 0x0, 0xa, 0x0, 0x0, 0xbb4bf1f7)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/store_replicate.go:157 +0x53
github.com/chrislusf/seaweedfs/weed/topology.ReplicatedWrite(0x7ffeefbffbd2, 0xe, 0xc00009c000, 0xc000000002, 0xc000472750, 0xc0001b2200, 0x0, 0x1, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/store_replicate.go:29 +0xc7
github.com/chrislusf/seaweedfs/weed/server.(*VolumeServer).PostHandler(0xc0001513f0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/volume_server_handlers_write.go:52 +0x56f
github.com/chrislusf/seaweedfs/weed/server.(*VolumeServer).privateStoreHandler(0xc0001513f0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/volume_server_handlers.go:37 +0x21f
net/http.HandlerFunc.ServeHTTP(0xc0004420e0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2012 +0x44
net/http.(*ServeMux).ServeHTTP(0xc0001fc800, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2387 +0x1a5
net/http.serverHandler.ServeHTTP(0xc0001781c0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2807 +0xa3
net/http.(*conn).serve(0xc0001a8000, 0x2934420, 0xc000212400)
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:1895 +0x86c
created by net/http.(*Server).Serve
	/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2933 +0x35c
Eg:
server A (datacenter 1) and server B (datacenter 2) hold replica (100) for volume 1.
If you upload a file with a key 1,xxxxx to server C (datacenter 3) will trigger the panic on server C.
The server C should either proxy upload file to the correct volume server or should return an HTTP error code and not panic.
2020-03-15 02:50:42 -07:00
Chris Lu 7b37178716 filer: close stores if interrupted 2020-03-14 20:30:26 -07:00
Chris Lu d022b6bc0e fix compilation 2020-03-14 16:32:16 -07:00
Chris Lu 0576a27f44 protect against possible nil
which is unlikely to happen though
2020-03-14 16:06:06 -07:00
Chris Lu 43ed730e90
Update README.md 2020-03-14 15:05:21 -07:00
Chris Lu cc52e8ca87 1.63 2020-03-14 14:25:07 -07:00
Chris Lu ee635dcc36 refactoring 2020-03-14 14:08:00 -07:00
Chris Lu 18cda6adbb cleaner fix 2020-03-14 13:55:32 -07:00
Chris Lu 17efa361f9 fix possible nil 2020-03-14 13:53:03 -07:00
Chris Lu 91e0a987e4 1.62 2020-03-14 01:03:59 -07:00
Chris Lu de1ba85346 HDFS support encrypted data storage 2020-03-14 00:27:57 -07:00
Chris Lu e2e691d9c2 clean up, add test 2020-03-13 23:53:15 -07:00
Chris Lu 7213f446db update grpc 2020-03-13 16:31:40 -07:00
Chris Lu 3cc9e85895 volume: vacuum pass preallocate variable 2020-03-13 16:17:44 -07:00
Chris Lu 4ce23e8809 fix compilation error 2020-03-13 15:51:38 -07:00
Chris Lu c90eb0da1f volume: handling readonly volumes after compaction
ensure readonly volumes are not added as writable
2020-03-13 15:41:27 -07:00
Chris Lu 025e586c91 testing vacuum 2020-03-13 15:41:27 -07:00
Chris Lu 2cc8e2ea3a
Update README.md 2020-03-12 21:54:48 -07:00
Chris Lu 595dd4fc9a
Update README.md 2020-03-12 21:53:43 -07:00
Chris Lu a7b027121a
Update README.md 2020-03-12 21:37:08 -07:00
Chris Lu 41bbea0300 tweaking snap 2020-03-12 13:23:25 -07:00
Chris Lu c3226ce75a
Merge pull request #758 from popey/add-snapcraft
Create snapcraft.yaml
2020-03-12 12:45:26 -07:00
Chris Lu e85da50ad4 remove tikv, move its support to "tikv" branch 2020-03-11 20:59:00 -07:00
Chris Lu 9797420ac8 go mod tidy 2020-03-11 20:34:04 -07:00
Chris Lu d439d83772 volume: follow compactionBytePerSecond
related to https://github.com/chrislusf/seaweedfs/issues/1108
2020-03-11 10:32:17 -07:00
Chris Lu 4237a813cc skip printouts 2020-03-11 01:13:54 -07:00
Chris Lu ebc739afb5 filer store postgres: skip empty parameters
better fix https://github.com/chrislusf/seaweedfs/issues/1227
2020-03-11 01:13:54 -07:00
Chris Lu f59fcba265
Merge pull request #1228 from HongyanShen/master
fix #1226
2020-03-11 00:02:10 -07:00
HongyanShen 81610ed006 fix: #1226 2020-03-11 14:37:14 +08:00
HongyanShen 03529fc0c2
Merge pull request #1 from chrislusf/master
sync
2020-03-11 12:55:24 +08:00
Chris Lu 60f5f05c78 1.61 2020-03-10 13:52:49 -07:00
Chris Lu 7f3ce7b6d6 adjust tikv build env 2020-03-10 13:36:45 -07:00
Chris Lu ead756981b adjust go mod 2020-03-10 13:35:05 -07:00
Chris Lu d3e1ef64b3 ensure error checking on windows 2020-03-10 13:33:26 -07:00
Chris Lu 88a110e67e release file handle 2020-03-10 13:32:22 -07:00
Chris Lu 43e6261579 volume: fix memory leak 2020-03-10 13:31:23 -07:00
Chris Lu d214cefc2e filer: list diretory pagination make up for the expired entries 2020-03-09 23:28:01 -07:00
Chris Lu e6de42f888 1.60 2020-03-09 22:32:40 -07:00