gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #1166 from BruceWangNo1/s3-select

Browse source 
temporary s3-select feature documentation
This commit is contained in:
Chris Lu 2019-12-23 11:23:56 -08:00 committed by GitHub
parent 8d068dc5d6 fa4f599ad4
commit a0a9f6c134
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 47 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

47
weed/s3api/S3SelectREADME.md Normal file
View file

@ -0,0 +1,47 @@
# AWS S3 Select feature in Seaweedfs
## Start weed
```bash
git checkout s3-select
# Compile weed
weed server -s3
```
## Install And Configure AWSCLI
```bash
pip install awscli
aws configure // default all the way
aws configure set default.s3.signature_version s3v4
```
## Make a bucket in S3
```bash
aws --endpoint-url http://localhost:8333 s3 mb s3://newbucketgoogle
```
## Download googleplaystore.csv
Link: https://www.kaggle.com/lava18/google-play-store-apps
## Copy the file to the S3 bucket
```bash
aws --endpoint-url http://localhost:8333 s3 copy google-play-store-apps/googleplaystore.csv s3://newbucketgoogle
```
## Perform S3 Select Query
```bash
aws --endpoint-url http://localhost:8333 s3api select-object-content --bucket "newbucketgoogle" --key
"googleplaystore.csv" --expression "select * from s3object limit 5" --expression-type 'SQL' --input-serialization '{"CSV": {"FileHeaderInfo": "USE","RecordDelimiter": "\n","FieldDelimiter": ","}, "CompressionType": "NONE"}' --output-serialization '{"CSV": {"RecordDelimiter": "\n","FieldDelimiter": ","}}' "sample.csv"
```
## Check out result
```bash
vim sample.csv
```