gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9922 commits 28 branches 278 tags 135 MiB
Commit graph

6381 commits

Author SHA1 Message Date
Chris Lu 043b7a7c65 add locking to access l.volumes 2017-01-20 10:18:43 -08:00
brstgt 0656838fe5 Don't return actual file size from CheckVolumeDataIntegrity, it will be 0 if last needle is a tombstone, so it's not reliable anyway 2017-01-20 16:37:45 +01:00
brstgt 4fb5bb09b6 Remove obsolete property Volume.dataFileSize 2017-01-20 16:31:11 +01:00
brstgt e074a54a20 Delete volumes online without restarting volume server 2017-01-20 13:02:37 +01:00
brstgt 492f93416d Mount and unmount volumes online without restarting volume server 2017-01-20 12:49:20 +01:00
brstgt 18b3afc97a Log volumeId if dataFileSize != actual data file size - improve log types 2017-01-20 12:22:25 +01:00
brstgt 0c898df430 Log volumeId if dataFileSize != actual data file size 2017-01-20 11:52:23 +01:00
Chris Lu c8f54aad8b adjust timing of leader election 2017-01-18 09:54:43 -08:00
Chris Lu 8de0027df5 Release 0.74 2017-01-18 09:39:47 -08:00
Chris Lu 5b0f5f456e volume server get notified of leader change 2017-01-18 09:34:27 -08:00
Chris Lu 59022b6fe0 fix byte counter on loading index file 
fix https://github.com/chrislusf/seaweedfs/issues/441
 2017-01-18 08:40:39 -08:00
Chris Lu 76da296799 upgrade to 0.73 2017-01-15 12:29:53 -08:00
Chris Lu 3065506b38 volume servers always connect to the master leader 2017-01-13 10:20:40 -08:00
Chris Lu d953ed442c auto detect volume server ip address 2017-01-12 13:42:53 -08:00
Chris Lu 1be354bdce default IP to "localhost" 
so that the master can know the ip address.
 2017-01-10 23:57:59 -08:00
Chris Lu adc8d344a5 update proto buffer 2017-01-10 23:57:16 -08:00
Chris Lu e767c3ea4f disable master connection timeout 
temporarily disable master connection timeout due to heartbeat
connection timeout
 2017-01-10 01:30:00 -08:00
Chris Lu e46c3415f7 gRpc for master~volume heartbeat 2017-01-10 01:01:12 -08:00
Chris Lu 4beaaa0650 simplify the gzip optimization logic 
fix https://github.com/chrislusf/seaweedfs/issues/436
 2017-01-09 19:31:58 -08:00
Chris Lu 9fa648e570 Merge pull request #435 from sparklxb/master 
support uploading files to specific dataCenter
 2017-01-09 10:07:07 -08:00
sparklxb 66e7013dfe suport uploading files to specific dataCenter 2017-01-09 23:34:47 +08:00
wangjie 90a6f43c56 fix the bug than we can't get filename when download file. 2017-01-09 14:55:21 +08:00
Chris Lu 53cf1b4900 change the basic unit to MB 2017-01-08 14:57:32 -08:00
Chris Lu ed44f12f6d support Fallocate on linux 2017-01-08 11:01:46 -08:00
Chris Lu 7b6837cbc2 move back the section 2017-01-08 10:35:47 -08:00
sparklxb da9b672d1b support additional header name-value pairs 2017-01-08 22:34:42 +08:00
sparklxb 86a7c56275 support additional header name-value pairs 2017-01-08 09:16:40 +08:00
Chris Lu 13e7069eb9 keep track of total data file size 2017-01-06 10:22:20 -08:00
Chris Lu e61c9af523 check Accept instead of content type 
fix https://github.com/chrislusf/seaweedfs/issues/429
 2017-01-04 19:58:27 -08:00
Chris Lu dc42d15352 release 0.72 2017-01-03 21:32:59 -08:00
Chris Lu 4d3353750e fix build issue 2017-01-03 21:28:30 -08:00
Chris Lu 297f64e286 Merge pull request #428 from sparklxb/master 
fix bug: upload big .gz file more than maxMB
 2017-01-03 21:23:08 -08:00
Chris Lu 8fb29e5230 change to 0.71 version 2017-01-03 21:17:54 -08:00
lixianbin d96d0a87cf fix bug: upload big .gz file more than maxMB 2017-01-04 11:23:40 +08:00
listeng 5b18cf09e7 Return json data when content-type is "application/json" 
Before #344, the filer will return json data, and cschiano make a template that render to html. But sometimes need json data, so I add some code will return json data when content-type is "application/json".
 2016-12-26 16:49:43 +08:00
Mike Tolman 43e655afc8 Updating PostgreSQL file store integration to support directories. 2016-12-12 13:03:32 -07:00
Mike Tolman e88795be13 Adding PostgreSQL Support 2016-12-07 17:24:40 -07:00
Mike Tolman b18ec59c56 Revert "Adding PostgreSQL Filer Support" 
This reverts commit 10853e4d2f.
 2016-12-07 17:16:10 -07:00
Mike Tolman 10853e4d2f Adding PostgreSQL Filer Support 2016-12-07 17:13:03 -07:00
Mike Tolman a1783a14cc Revert "Adding PostgreSQL Filer Support" 
This reverts commit 855058858a.
 2016-12-07 16:54:06 -07:00
Mike Tolman 855058858a Adding PostgreSQL Filer Support 2016-12-07 16:51:36 -07:00
Chris Lu 64509dd6ca fix text. 2016-11-28 20:07:27 -08:00
Chris Lu 924f797c58 Merge pull request #406 from eshujiushiwo/master 
Fix the filer connection of cassandra
 2016-11-28 20:05:45 -08:00
eshujiushiwo e025fc00a4 let filer use all cassandra server 
let filer use all cassandra server
 2016-11-28 18:23:39 +08:00
eshujiushiwo 5ee7fdc0cc fix hosts strings 
fix hosts strings
 2016-11-28 16:03:39 +08:00
eshujiushiwo 3450eff288 fix info log 
fix info log
 2016-11-28 15:32:16 +08:00
eshujiushiwo 3e29d0f75b Update cassandra_store.go 2016-11-28 15:14:19 +08:00
eshujiushiwo 809aa028ec fix the connection of cassandra 
fix the connection of cassandra ,let filer can connect to multi cassandra nodes which are in the one cluster
 2016-11-28 15:01:04 +08:00
eshujiushiwo 89ccb6be05 Update cassandra_store.go 2016-11-28 12:16:51 +08:00
eshujiushiwo 5385bf9555 update cassandra connections 
update cassandra connections
 2016-11-28 11:27:40 +08:00
Chris Lu 9ba52db585 add writable checking 
fix https://github.com/chrislusf/seaweedfs/issues/405
 2016-11-26 12:14:06 -08:00
Chris Lu dcaf1796fe add option to enable caching 2016-11-16 07:09:57 -08:00
Chris Lu f54f530ada adjust verbose logging 2016-11-13 14:07:51 -08:00
霍晓栋 b9f385bd66 refactor concurrent loading code logic 2016-11-13 13:24:52 +08:00
霍晓栋 096ffa9744 concurrent loading volume 2016-11-11 11:53:22 +08:00
Chris Lu 3320e495f4 Merge pull request #394 from bedragon/master 
fix bug : join post is empty
 2016-11-07 19:09:19 -08:00
Yang zhixiang 57005ed7f8 fix bug : join post is empty 2016-11-08 11:00:46 +08:00
Chris Lu 36f9633223 add locks for location.volumes 
fix https://github.com/chrislusf/seaweedfs/issues/392
 2016-11-06 20:55:22 -08:00
Yang zhixiang 4194ae33fd fix raft_server.go isPeersChanged bug 2016-11-03 15:46:59 +08:00
霍晓栋 7d73bbb073 comment UT case 2016-10-07 16:40:51 +08:00
霍晓栋 7d382ba5fe comment UT case 2016-10-07 16:34:22 +08:00
霍晓栋 ce1f7ab662 makediff func with UT case 2016-10-07 16:22:24 +08:00
霍晓栋 ed848425c7 supplemental data between compacting and commit compacting 2016-09-29 13:57:23 +08:00
Chris Lu dffad65f2f fix compilation 2016-09-26 22:30:44 -07:00
Chris Lu 7e29218327 add a template for makeupDiff 2016-09-26 22:26:41 -07:00
Chris Lu 1bc041b46d add a new way to manually compact corrupted volume 
fix https://github.com/chrislusf/seaweedfs/issues/371
 2016-09-22 20:31:20 -07:00
Chris Lu 01cbd5cb58 lock 
fix https://github.com/chrislusf/seaweedfs/issues/367
 2016-09-08 09:50:31 -07:00
霍晓栋 78474409a5 filer mysqlstore bug fix 2016-09-08 11:35:54 +08:00
霍晓栋 1c2affea8e Merge remote-tracking branch 'upstream/master' 2016-09-08 10:34:24 +08:00
Chris Lu 0559aa9673 use Lock instead of RLock 
fix https://github.com/chrislusf/seaweedfs/issues/364
 2016-09-07 18:21:05 -07:00
Chris Lu c4b7966dbe minor help message change 2016-09-07 18:21:05 -07:00
霍晓栋 3aa021a812 refactoring mysql store code 2016-09-05 14:10:22 +08:00
霍晓栋 e7b237c8da UT case fix 2016-08-31 11:55:02 +08:00
霍晓栋 b0035747e3 add filer support 2016-08-31 11:32:30 +08:00
霍晓栋 f04d8fcbcc if replicated volume has one copy in readonly mode at one node,it should be removed from writable list 2016-08-09 20:12:39 +08:00
Mike Tolman 01d3f69c52 Adding AutoChunk/MaxMB Support to Filer API 
This is related to the following issue I added to chrislusf/seaweedfs:
https://github.com/chrislusf/seaweedfs/issues/342
 2016-08-05 16:01:30 -06:00
Mike Tolman a89a3c86d0 Revert "Add AutoChunking to the Filer API, so that you can upload really large files through the filer API." 
This reverts commit 09059bfdcc.
 2016-08-05 15:47:46 -06:00
Mike Tolman 0d331c1e3a Revert "Changing needle_byte_cache so that it doesn't grow so big when larger files are added." 
This reverts commit 87fee21ef5.
 2016-08-05 15:46:45 -06:00
Mike Tolman 0f4c7dd8fd Revert "Ooops. Missed a line." 
This reverts commit 14d4252904.
 2016-08-05 15:46:08 -06:00
Mike Tolman ce99bb927d Revert "Adding HTTP verb whitelisting options." 
This reverts commit 34837afc7a.
 2016-08-05 15:45:48 -06:00
Mike Tolman 34837afc7a Adding HTTP verb whitelisting options. 2016-08-05 15:23:43 -06:00
Mike Tolman 14d4252904 Ooops. Missed a line. 2016-08-05 15:22:46 -06:00
Mike Tolman 09059bfdcc Add AutoChunking to the Filer API, so that you can upload really large files through the filer API. 2016-08-05 15:19:10 -06:00
Mike Tolman 87fee21ef5 Changing needle_byte_cache so that it doesn't grow so big when larger files are added. 2016-08-05 15:14:24 -06:00
霍晓栋 52e55508da deleted index entry could not point to deleted needle 2016-07-25 15:07:11 +08:00
霍晓栋 b9b3651a98 deleted index entry could not point to deleted needle 2016-07-25 14:56:58 +08:00
霍晓栋 09bd3d015d deleted index entry could not point to deleted needle 2016-07-25 14:54:40 +08:00
霍晓栋 78678f4bcb deleted needle does not need checksum verification 2016-07-25 11:40:35 +08:00
Chris Lu 185a916f5e adjusting command options 2016-07-21 15:00:07 -07:00
Chris Lu a5be4a6d40 fix package name 2016-07-21 01:23:56 -07:00
Chris Lu a57162e8bf delete operation does not need this checking 2016-07-21 00:40:13 -07:00
Chris Lu cdae9fc680 add "weed copy" command to copy files to filer 2016-07-20 23:45:55 -07:00
霍晓栋 574cc41f0b recover filer read handler logic 2016-07-21 12:20:22 +08:00
cschiano af905a3ff7 Add limit parameter and pagination for files 2016-07-20 10:46:28 +02:00
cschiano c63cebaee1 Add a visualization for the filer 2016-07-18 10:28:24 +02:00
Chris Lu d3ae3a8e14 collect proof when stopping 2016-07-05 10:23:14 -07:00
Chris Lu b38ff39e8b add master profiling option 2016-07-03 15:39:57 -07:00
Chris Lu d981eb282f refactor volume_read_write.go out of volume.go 2016-07-03 00:10:27 -07:00
Chris Lu 576573711a refactoring 2016-07-02 23:58:23 -07:00
Chris Lu 3d8df0f709 refactor volume_loading.go out of volume.go 2016-07-02 23:56:49 -07:00
Chris Lu 582d5d526c refactoring 2016-07-02 23:50:58 -07:00
Chris Lu 3713119ab9 adjust verification logic. 
1. log file name for easier debugging
2. remove unused version in readIndexEntryAtOffset()
3. if indexSize==0, the dataFile usually still has a super block of
size 8. So skipping this case.
 2016-07-02 23:45:37 -07:00
霍晓栋 16f6984d02 refactor data integrity checking code v2 2016-07-03 14:11:25 +08:00
霍晓栋 dda13def2a refactor data integrity checking code 2016-07-03 12:53:03 +08:00
霍晓栋 2e74fb60c8 add data integrity checking 2016-07-02 21:34:30 +08:00
霍晓栋 339856eb78 fix bug when choosing the healthy master in filer service 2016-06-30 19:54:05 +08:00
霍晓栋 079fdbad13 fix volume_layout bug, add new feature for volume etag write,filer pass through query params' 2016-06-29 16:05:00 +08:00
Chris Lu b617b13c43 remember oversized volumes 
fix https://github.com/chrislusf/seaweedfs/issues/331
 2016-06-27 15:28:23 -07:00
Chris Lu d0dbf6d2ea adjust logging 2016-06-26 12:49:10 -07:00
霍晓栋 1f63094542 refactor Volume Assign function 2016-06-26 10:50:18 +08:00
霍晓栋 c601ef03b1 Merge remote-tracking branch 'upstream/master' 2016-06-26 10:27:50 +08:00
Chris Lu 0647c75a06 update dependency versions 2016-06-24 14:20:32 -07:00
Chris Lu 101e784577 add locking on possible concurrent map access 
fix https://github.com/chrislusf/seaweedfs/issues/328
 2016-06-23 09:10:25 -07:00
霍晓栋 79b0a58132 add options params for the Assign func 2016-06-23 11:19:09 +08:00
Chris Lu 356b8048c5 fix compilation problem due to API changes 2016-06-18 18:57:33 -07:00
Chris Lu cc01c352c3 refactoring 2016-06-13 15:07:17 -07:00
Chris Lu a61453d3ec fix nil crash 
fix https://github.com/chrislusf/seaweedfs/issues/320
 2016-06-13 14:38:35 -07:00
DaLin fa2a7b5f2f fix bug: filer DELETE 2016-06-11 01:16:19 +08:00
霍晓栋 3a25af223f refactor work for filer write handler 2016-06-08 15:46:14 +08:00
霍晓栋 26bb9094d4 made a little change for Amazon S3 API's compatibility 2016-06-08 12:30:20 +08:00
Chris Lu 0e3140c54c a little refactoring 2016-06-06 23:50:27 -07:00
霍晓栋 d514b6f3f9 read write handler update 2016-06-07 11:45:00 +08:00
霍晓栋 1ef81ac518 filer could detect master nodes healthy status and choose a working one when encountering SPOF 2016-06-07 11:38:40 +08:00
Chris Lu 96d2f11f81 adjust log 2016-06-05 09:05:03 -07:00
Chris Lu b456173eb8 delete file storage when POST to update 
PUT is used to update. But sometimes POST is also used to post the same
file again. In this case, the actual file storage was not marked as
deleted.

This fix will try its best to delete the orphan files.
 2016-06-03 00:24:55 -07:00
Chris Lu 535b1610c8 add fid and url to filer post result 
fix https://github.com/chrislusf/seaweedfs/issues/274
 2016-06-02 20:44:50 -07:00
Chris Lu 4f47b8cff9 refactoring 2016-06-02 20:05:34 -07:00
Chris Lu 5ce6bbf076 directory structure change to work with glide 
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
 2016-06-02 18:09:14 -07:00
Chris Lu 5071f528f6 testing compilation with remove package 2013-02-10 03:49:51 -08:00
Chris Lu 55f2627fcf testing new directory structure 2013-02-10 03:31:40 -08:00
Chris Lu ab6fb13ad7 avoid the "src" folder 2013-02-10 03:25:35 -08:00