gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Hadoop Compatible File System (markdown)

Chris Lu 2020-11-08 23:19:53 -08:00
parent a7903c3d81
commit 4e5ce3410b
1 changed files with 8 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
Hadoop-Compatible-File-System.md

@ -10,12 +10,12 @@ $ mvn install
# build for hadoop2 # build for hadoop2
$cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs2 $cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs2
$ mvn package $ mvn package
$ ls -al target/seaweedfs-hadoop2-client-*.jar $ ls -al target/seaweedfs-hadoop2-client-1.5.3.jar
# build for hadoop3 # build for hadoop3
$cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs3 $cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs3
$ mvn package $ mvn package
$ ls -al target/seaweedfs-hadoop3-client-*.jar $ ls -al target/seaweedfs-hadoop3-client-1.5.3.jar
``` ```
Maven Maven
@ -46,8 +46,8 @@ Or you can download the latest version from MavenCentral
Suppose you are getting a new Hadoop installation. Here are the minimum steps to get SeaweedFS to run. Suppose you are getting a new Hadoop installation. Here are the minimum steps to get SeaweedFS to run.
You would need to start a weed filer first, build the seaweedfs-hadoop2-client-xxx.jar You would need to start a weed filer first, build the seaweedfs-hadoop2-client-1.5.3.jar
or seaweedfs-hadoop3-client-xxx.jar, and do the following: or seaweedfs-hadoop3-client-1.5.3.jar, and do the following:
``` ```
# optionally adjust hadoop memory allocation # optionally adjust hadoop memory allocation
@ -60,12 +60,12 @@ $ echo "<configuration></configuration>" > etc/hadoop/mapred-site.xml
# on hadoop2 # on hadoop2
$ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \ $ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \
-Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \ -Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \
-libjars ./seaweedfs-hadoop2-client-x.x.x.jar \ -libjars ./seaweedfs-hadoop2-client-1.5.3.jar \
-ls / -ls /
# or on hadoop3 # or on hadoop3
$ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \ $ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \
-Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \ -Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \
-libjars ./seaweedfs-hadoop3-client-x.x.x.jar \ -libjars ./seaweedfs-hadoop3-client-1.5.3.jar \
-ls / -ls /
``` ```
@ -106,9 +106,9 @@ $ bin/hadoop classpath
# Copy SeaweedFS HDFS client jar to one of the folders # Copy SeaweedFS HDFS client jar to one of the folders
$ cd ${HADOOP_HOME} $ cd ${HADOOP_HOME}
# for hadoop2 # for hadoop2
$ cp ./seaweedfs-hadoop2-client-x.x.x.jar share/hadoop/common/lib/ $ cp ./seaweedfs-hadoop2-client-1.5.3.jar share/hadoop/common/lib/
# or for hadoop3 # or for hadoop3
$ cp ./seaweedfs-hadoop3-client-x.x.x.jar share/hadoop/common/lib/ $ cp ./seaweedfs-hadoop3-client-1.5.3.jar share/hadoop/common/lib/
``` ```
Now you can do this: Now you can do this: