gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Updated Hadoop Compatible File System (markdown)

Chris Lu 2020-11-08 23:19:53 -08:00
parent a7903c3d81
commit 4e5ce3410b
1 changed files with 8 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
Hadoop-Compatible-File-System.md

@ -10,12 +10,12 @@ $ mvn install
# build for hadoop2
$cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs2
$ mvn package
$ ls -al target/seaweedfs-hadoop2-client-*.jar
$ ls -al target/seaweedfs-hadoop2-client-1.5.3.jar
# build for hadoop3
$cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs3
$ mvn package
$ ls -al target/seaweedfs-hadoop3-client-*.jar
$ ls -al target/seaweedfs-hadoop3-client-1.5.3.jar
```
Maven
@ -46,8 +46,8 @@ Or you can download the latest version from MavenCentral
Suppose you are getting a new Hadoop installation. Here are the minimum steps to get SeaweedFS to run.
You would need to start a weed filer first, build the seaweedfs-hadoop2-client-xxx.jar
or seaweedfs-hadoop3-client-xxx.jar, and do the following:
You would need to start a weed filer first, build the seaweedfs-hadoop2-client-1.5.3.jar
or seaweedfs-hadoop3-client-1.5.3.jar, and do the following:
```
# optionally adjust hadoop memory allocation
@ -60,12 +60,12 @@ $ echo "<configuration></configuration>" > etc/hadoop/mapred-site.xml
# on hadoop2
$ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \
-Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \
-libjars ./seaweedfs-hadoop2-client-x.x.x.jar \
-libjars ./seaweedfs-hadoop2-client-1.5.3.jar \
-ls /
# or on hadoop3
$ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \
-Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \
-libjars ./seaweedfs-hadoop3-client-x.x.x.jar \
-libjars ./seaweedfs-hadoop3-client-1.5.3.jar \
-ls /
```
@ -106,9 +106,9 @@ $ bin/hadoop classpath
# Copy SeaweedFS HDFS client jar to one of the folders
$ cd ${HADOOP_HOME}
# for hadoop2
$ cp ./seaweedfs-hadoop2-client-x.x.x.jar share/hadoop/common/lib/
$ cp ./seaweedfs-hadoop2-client-1.5.3.jar share/hadoop/common/lib/
# or for hadoop3
$ cp ./seaweedfs-hadoop3-client-x.x.x.jar share/hadoop/common/lib/
$ cp ./seaweedfs-hadoop3-client-1.5.3.jar share/hadoop/common/lib/
```
Now you can do this: