Chris Lu
bc8d34143d
Revert "fix "weed backup" rerunning"
...
This reverts commit 1937be2bb2
.
2018-09-09 02:38:30 -07:00
Chris Lu
1937be2bb2
fix "weed backup" rerunning
...
"weed backup" rerunning will already have ReplicaPlacement set, while version is not set.
2018-09-09 02:31:12 -07:00
Chris Lu
76cbe8bf33
instant notification of new volumes added or deleted
2018-08-24 01:26:56 -07:00
Chris Lu
ac793a3c5a
clean remove all left over files when deleting a collection
2018-08-23 23:33:16 -07:00
Chris Lu
d361a1ddf5
fix id String() instead
2018-08-23 22:44:30 -07:00
Chris Lu
694f93de80
parse needle id correctly
...
fix https://github.com/chrislusf/seaweedfs/issues/713
2018-08-23 20:48:43 -07:00
Chris Lu
1f030af06f
adjusting startup logging
2018-08-22 13:59:04 -07:00
Chris Lu
75d63db60d
randomize raft server startup
...
also some go fmt
2018-08-12 14:27:14 -07:00
Chris Lu
495a776671
support 128 bit NeedleId
2018-07-31 23:25:26 -07:00
Chris Lu
10bb76fe6b
go fmt
2018-07-27 02:10:42 -07:00
Chris Lu
4d322df95e
avoid reparsing the multipart form
2018-07-27 02:10:10 -07:00
Chris Lu
57ded4fd80
speed up leveldb bolddb loading
2018-07-24 22:17:56 -07:00
Chris Lu
ea8ecf35f4
set needle AppendAtNs value during writes
2018-07-24 02:44:33 -07:00
Chris Lu
77fadab560
go fmt
2018-07-24 01:38:08 -07:00
Chris Lu
4f317c7e3d
working version3
2018-07-24 01:36:04 -07:00
Chris Lu
852af28f91
needle priting format
2018-07-21 17:41:21 -07:00
Chris Lu
7e2031b18f
go fmt
2018-07-21 17:39:10 -07:00
Chris Lu
c98df05ed0
support PUT
2018-07-21 15:58:48 -07:00
Chris Lu
feb8eeb830
refactor: separate multipart parsing file
2018-07-21 12:01:35 -07:00
Chris Lu
3edfe1d28f
extend export command to show tombstone + change output format to CSV
...
merging https://github.com/chrislusf/seaweedfs/pull/610 and add "-limit" option
2018-07-14 20:51:17 -07:00
Chris Lu
a51aa4c586
avoid extra log messages
...
fix https://github.com/chrislusf/seaweedfs/issues/689
2018-07-13 22:07:32 -07:00
Chris Lu
8a12fd9ec4
add error checking when loading dat file
2018-07-09 23:31:25 -07:00
Chris Lu
d0982cafa7
refactoring
2018-07-09 00:22:50 -07:00
Chris Lu
5b1fd374be
fix all tests
2018-07-08 02:39:04 -07:00
Chris Lu
d4d7ced922
refactoring: add type for needle id, offset
...
later the type size can possibly be adjusted
2018-07-08 02:28:04 -07:00
Chris Lu
5bfb72d058
faster loading boltdb or leveldb needle map metrics by bloomfilter
...
avoid btree
2018-07-07 00:51:17 -07:00
Chris Lu
aba1fe01b3
use constant NeedleIndexSize instead of the number 16
2018-07-07 00:50:14 -07:00
Chris Lu
defcd4edc5
adjust log level
2018-07-07 00:48:58 -07:00
Chris Lu
9bcaa65574
avoid divided by 0
...
fix https://github.com/chrislusf/seaweedfs/issues/650
2018-06-25 01:20:15 -07:00
Chris Lu
233a9a9b6c
fix log error
2018-06-24 16:12:41 -07:00
Chris Lu
49f4216280
fix compilation error
2018-06-24 15:37:54 -07:00
Chris Lu
7f6500ae07
super block reserves a couple of bits for future extension
2018-06-24 15:23:03 -07:00
Chris Lu
f339e9c284
add support for extra super block info
2018-06-24 15:19:57 -07:00
Chris Lu
9ba335a7c4
prepare for flexible super block
2018-06-24 11:37:08 -07:00
Chris Lu
3a6f9c9bcf
increase test size
2018-06-23 18:30:13 -07:00
Chris Lu
7f7e4e9885
fix error for deleted files during compaction
...
deletion during commit may cause trouble when make up the difference during commitCompact()
2018-06-23 18:24:59 -07:00
Chris Lu
3699996985
add compaction test
2018-06-23 16:48:19 -07:00
Chris Lu
0467195f07
fix file closing on windows
2018-06-13 00:36:51 -07:00
Chris Lu
eaf000eff1
fix error on go tip
2018-06-11 23:39:42 -07:00
Chris Lu
43e3f5724c
use fixed list of masters in both filer and volume servers
2018-06-01 00:39:39 -07:00
Chris Lu
458ada173e
go fmt
2018-05-27 11:52:26 -07:00
Chris Lu
1675243f29
maybe speed up a little when loading index
2018-05-23 02:38:11 -07:00
Chris Lu
69b9d8c3c2
fix boltdb variable usage
2018-05-22 10:18:09 -07:00
Chris Lu
942c2cbd7b
mv pb to master_pb
2018-05-09 23:11:54 -07:00
Chris Lu
57c5405657
add todo
2018-05-09 04:18:31 -07:00
Chris Lu
6db1c8bafb
Revert "adjust the needle mapper"
...
This reverts commit e2c7be3a2e
.
2018-05-09 03:57:07 -07:00
Chris Lu
e2c7be3a2e
adjust the needle mapper
...
as discussed on
https://github.com/chrislusf/seaweedfs/pull/642#issuecomment-387606960
2018-05-09 03:50:14 -07:00
WenLong LI
b9068c1304
use volumes variable
...
fix leveldb mount bug
2018-05-03 12:13:53 +08:00
liwenlong05
02a1597642
fix leveldb mount bug
2018-05-03 11:20:38 +08:00
mtolman
e4b1e5c3c8
Fix for issue #635
...
Fix for the following issue on master: https://github.com/chrislusf/seaweedfs/issues/635
2018-04-17 13:26:48 -06:00
Yang Pengcheng
a2112e1be0
fix ldb dir deleting
2017-08-30 22:55:03 +08:00
Chris Lu
58344980e4
preallocate disk space during compaction also, add cleanup for failed compaction
2017-08-29 23:59:53 -07:00
Chris Lu
5047bdb4a2
skip bytes cache
2017-06-03 11:44:24 -07:00
Chris Lu
82c3ccc8dd
add btree for volume index
2017-05-26 22:51:25 -07:00
Chris Lu
bd97cbc523
skip busy reporting back to master
...
possible fix for https://github.com/chrislusf/seaweedfs/issues/499
2017-05-23 21:12:03 -07:00
Chris Lu
043b7a7c65
add locking to access l.volumes
2017-01-20 10:18:43 -08:00
brstgt
0656838fe5
Don't return actual file size from CheckVolumeDataIntegrity, it will be 0 if last needle is a tombstone, so it's not reliable anyway
2017-01-20 16:37:45 +01:00
brstgt
4fb5bb09b6
Remove obsolete property Volume.dataFileSize
2017-01-20 16:31:11 +01:00
brstgt
e074a54a20
Delete volumes online without restarting volume server
2017-01-20 13:02:37 +01:00
brstgt
492f93416d
Mount and unmount volumes online without restarting volume server
2017-01-20 12:49:20 +01:00
brstgt
18b3afc97a
Log volumeId if dataFileSize != actual data file size - improve log types
2017-01-20 12:22:25 +01:00
brstgt
0c898df430
Log volumeId if dataFileSize != actual data file size
2017-01-20 11:52:23 +01:00
Chris Lu
5b0f5f456e
volume server get notified of leader change
2017-01-18 09:34:27 -08:00
Chris Lu
59022b6fe0
fix byte counter on loading index file
...
fix https://github.com/chrislusf/seaweedfs/issues/441
2017-01-18 08:40:39 -08:00
Chris Lu
3065506b38
volume servers always connect to the master leader
2017-01-13 10:20:40 -08:00
Chris Lu
e46c3415f7
gRpc for master~volume heartbeat
2017-01-10 01:01:12 -08:00
Chris Lu
4beaaa0650
simplify the gzip optimization logic
...
fix https://github.com/chrislusf/seaweedfs/issues/436
2017-01-09 19:31:58 -08:00
Chris Lu
ed44f12f6d
support Fallocate on linux
2017-01-08 11:01:46 -08:00
Chris Lu
7b6837cbc2
move back the section
2017-01-08 10:35:47 -08:00
sparklxb
da9b672d1b
support additional header name-value pairs
2017-01-08 22:34:42 +08:00
sparklxb
86a7c56275
support additional header name-value pairs
2017-01-08 09:16:40 +08:00
Chris Lu
13e7069eb9
keep track of total data file size
2017-01-06 10:22:20 -08:00
lixianbin
d96d0a87cf
fix bug: upload big .gz file more than maxMB
2017-01-04 11:23:40 +08:00
Chris Lu
dcaf1796fe
add option to enable caching
2016-11-16 07:09:57 -08:00
Chris Lu
f54f530ada
adjust verbose logging
2016-11-13 14:07:51 -08:00
霍晓栋
b9f385bd66
refactor concurrent loading code logic
2016-11-13 13:24:52 +08:00
霍晓栋
096ffa9744
concurrent loading volume
2016-11-11 11:53:22 +08:00
Chris Lu
36f9633223
add locks for location.volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/392
2016-11-06 20:55:22 -08:00
霍晓栋
7d73bbb073
comment UT case
2016-10-07 16:40:51 +08:00
霍晓栋
7d382ba5fe
comment UT case
2016-10-07 16:34:22 +08:00
霍晓栋
ce1f7ab662
makediff func with UT case
2016-10-07 16:22:24 +08:00
霍晓栋
ed848425c7
supplemental data between compacting and commit compacting
2016-09-29 13:57:23 +08:00
Chris Lu
dffad65f2f
fix compilation
2016-09-26 22:30:44 -07:00
Chris Lu
7e29218327
add a template for makeupDiff
2016-09-26 22:26:41 -07:00
Chris Lu
1bc041b46d
add a new way to manually compact corrupted volume
...
fix https://github.com/chrislusf/seaweedfs/issues/371
2016-09-22 20:31:20 -07:00
Chris Lu
01cbd5cb58
lock
...
fix https://github.com/chrislusf/seaweedfs/issues/367
2016-09-08 09:50:31 -07:00
霍晓栋
b9b3651a98
deleted index entry could not point to deleted needle
2016-07-25 14:56:58 +08:00
霍晓栋
09bd3d015d
deleted index entry could not point to deleted needle
2016-07-25 14:54:40 +08:00
霍晓栋
78678f4bcb
deleted needle does not need checksum verification
2016-07-25 11:40:35 +08:00
Chris Lu
d981eb282f
refactor volume_read_write.go out of volume.go
2016-07-03 00:10:27 -07:00
Chris Lu
576573711a
refactoring
2016-07-02 23:58:23 -07:00
Chris Lu
3d8df0f709
refactor volume_loading.go out of volume.go
2016-07-02 23:56:49 -07:00
Chris Lu
582d5d526c
refactoring
2016-07-02 23:50:58 -07:00
Chris Lu
3713119ab9
adjust verification logic.
...
1. log file name for easier debugging
2. remove unused version in readIndexEntryAtOffset()
3. if indexSize==0, the dataFile usually still has a super block of
size 8. So skipping this case.
2016-07-02 23:45:37 -07:00
霍晓栋
16f6984d02
refactor data integrity checking code v2
2016-07-03 14:11:25 +08:00
霍晓栋
dda13def2a
refactor data integrity checking code
2016-07-03 12:53:03 +08:00
霍晓栋
2e74fb60c8
add data integrity checking
2016-07-02 21:34:30 +08:00
Chris Lu
101e784577
add locking on possible concurrent map access
...
fix https://github.com/chrislusf/seaweedfs/issues/328
2016-06-23 09:10:25 -07:00
霍晓栋
1ef81ac518
filer could detect master nodes healthy status and choose a working one when encountering SPOF
2016-06-07 11:38:40 +08:00
Chris Lu
5ce6bbf076
directory structure change to work with glide
...
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
2016-06-02 18:09:14 -07:00