AWS - goofys 설치 및 S3 마운트

goofys 성능 비교

s3를 마운트 하는 대표적인 서비스는 s3fs-fuse와 goofys다. 아래는 goofys와 다른 서비스들 간의 성능 비교다. goofys를 선택한 가장 큰 이유는 속도 차이가 다른 서비스에 비해 월등히 높기 때문이다.

 

Golang fuse 설치

1. golang 설치

yum install -y golang fuse

go version 확인

$ go version
go version go1.18.4 linux/amd64

 

go 환경 변수 설정

$ which go
/usr/bin/go

$ export GOBIN=/usr/bin

$ go env | egrep 'GOROOT|GOBIN|GOPATH'
GOBIN="/usr/bin"
GOPATH="/root/go"
GOROOT="/usr/lib/golang"

 

goofys 설치

$ cd /home/tomcat

$ wget http://bit.ly/goofys-latest -O /usr/local/bin/goofys

$ chmod +x /usr/local/bin/goofys

 

goofys 버전확인

$ goofys --version
goofys version 0.19.0-943e017724ea820eb4185419ef3c41d6f921a324

 

AWS Access Key / Secret Key 등록 

aws configure 명령어로 Access Key 등록

$ aws configure
AWS Access Key ID [None]: accesskey
AWS Secret Access Key [None]: secretkey
Default region name [None]: ap-northeast-2
Default output format [None]: json

edit로  Access Key / Secret Key등록

$ vi ~/.aws/credentials
[default]
aws_access_key_id = ****************67M
aws_secret_access_key = ****************8LTA

$ vi ~/.aws/config
[default]
region = ap-southeast-1

Access Key 리스트 확인

$ aws configure list
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ****************E67M shared-credentials-file
secret_key     ****************8LTA shared-credentials-file
    region           ap-southeast-1      config-file    ~/.aws/config

AWS S3 마운트(Goofys 마운트)

마운트 시 해당 일반계정으로 접속하여

--sse 옵션은 AWS 내 버킷에 암호화 정책이 있을 경우 사용

 

root 권한으로 실행 시 일반 계정은 해당 폴더에 권한이 없으니 유의하자.

goofys -o allow_other -o user_id=1001 -o group_id=1001 --sse 버킷명:접두사 /마운트 경로

 

이렇게 마운트가 끝났다.

 

파일 업로드 테스트를 해보자.

 

자동 마운트를 할 경우 sudo 권한이 있는 경우 추천한다.

--sse는 버킷 내 기본 암호화 정책이 있는 경우 넣어준다.

$ vi /etc/fstab

goofys#버킷명:접두사 /마운트경로 fuse _netdev,user_id=1001,group_id=1001,allow_other 0 0 --sse

 

fstab에 등록한 경우 자동 마운트 테스트를 하기 위해 원마운트 시키고 

umount -l /home/tomcat/폴더명

 

mount -a로 정상 마운트가 되는지 확인한다.

mount -a

 

 

마지막 goofys help TIP

Error: goofys takes exactly two arguments.

NAME:
   goofys - Mount an S3 bucket locally

USAGE:
   goofys [global options] bucket[:prefix] mountpoint

VERSION:
   0.19.0-943e017724ea820eb4185419ef3c41d6f921a324

GLOBAL OPTIONS:
   -o value            Additional system-specific mount options. Be careful!
   --cache value       Directory to use for data cache. Requires catfs and `-o a                                                           llow_other'. Can also pass in other catfs options (ex: --cache "--free:10%:$HOME                                                           /cache") (default: off)
   --dir-mode value    Permission bits for directories. (default: 0755) (default                                                           : 493)
   --file-mode value   Permission bits for files. (default: 0644) (default: 420)
   --uid value         UID owner of all inodes. (default: 67491)
   --gid value         GID owner of all inodes. (default: 101)
   --endpoint value    The non-AWS endpoint to connect to. Possible values: http                                                           ://127.0.0.1:8081/
   --profile value     Use a named profile from $HOME/.aws/credentials instead o                                                           f "default"
   --use-content-type  Set Content-Type according to file extension and /etc/mim                                                           e.types (default: off)

TUNING OPTIONS:
   --cheap                 Reduce S3 operation costs at the expense of some perf                                                           ormance (default: off)
   --no-implicit-dir       Assume all directory objects ("dir/") exist (default:                                                            off)
   --stat-cache-ttl value  How long to cache StatObject results and inode attrib                                                           utes. (default: 1m0s)
   --type-cache-ttl value  How long to cache name -> file/dir mappings in direct                                                           ory inodes. (default: 1m0s)

AWS S3 OPTIONS:
   --region value         The region to connect to. Usually this is auto-detecte                                                           d. Possible values: us-east-1, us-west-1, us-west-2, eu-west-1, eu-central-1, ap                                                           -southeast-1, ap-southeast-2, ap-northeast-1, sa-east-1, cn-north-1 (default: "u                                                           s-east-1")
   --storage-class value  The type of storage to use when writing objects. Possi                                                           ble values: REDUCED_REDUNDANCY, STANDARD, STANDARD_IA. (default: "STANDARD")
   --sse                  Enable basic server-side encryption at rest (SSE-S3) i                                                           n S3 for all writes (default: off)
   --sse-kms key-id       Enable KMS encryption (SSE-KMS) for all writes using t                                                           his particular KMS key-id. Leave blank to Use the account's CMK - customer maste                                                           r key (default: off)
   --acl value            The canned ACL to apply to the object. Possible values                                                           : private, public-read, public-read-write, authenticated-read, aws-exec-read, bu                                                           cket-owner-read, bucket-owner-full-control (default: off)

MISC OPTIONS:
   --help, -h     Print this help text and exit successfully.
   --debug_fuse   Enable fuse-related debugging output.
   --debug_s3     Enable S3-related debugging output.
   -f             Run goofys in foreground.
   --version, -v  print the version

[spark45@AWSADJSGPL000Q ~]$ goofys -h
Error: goofys takes exactly two arguments.

NAME:
   goofys - Mount an S3 bucket locally

USAGE:
   goofys [global options] bucket[:prefix] mountpoint

VERSION:
   0.19.0-943e017724ea820eb4185419ef3c41d6f921a324

GLOBAL OPTIONS:
   -o value            Additional system-specific mount options. Be careful!
   --cache value       Directory to use for data cache. Requires catfs and `-o allow_other'. Can also pass in other catfs options (ex: --cache "--free:10%:$HOME/cache") (default: off)
   --dir-mode value    Permission bits for directories. (default: 0755) (default: 493)
   --file-mode value   Permission bits for files. (default: 0644) (default: 420)
   --uid value         UID owner of all inodes. (default: 67491)
   --gid value         GID owner of all inodes. (default: 101)
   --endpoint value    The non-AWS endpoint to connect to. Possible values: http://127.0.0.1:8081/
   --profile value     Use a named profile from $HOME/.aws/credentials instead of "default"
   --use-content-type  Set Content-Type according to file extension and /etc/mime.types (default: off)

TUNING OPTIONS:
   --cheap                 Reduce S3 operation costs at the expense of some performance (default: off)
   --no-implicit-dir       Assume all directory objects ("dir/") exist (default: off)
   --stat-cache-ttl value  How long to cache StatObject results and inode attributes. (default: 1m0s)
   --type-cache-ttl value  How long to cache name -> file/dir mappings in directory inodes. (default: 1m0s)

AWS S3 OPTIONS:
   --region value         The region to connect to. Usually this is auto-detected. Possible values: us-east-1, us-west-1, us-west-2, eu-west-1, eu-central-1, ap-southeast-1, ap-southeast-2, ap-northeast-1, sa-east-1, cn-north-1 (default: "us-east-1")
   --storage-class value  The type of storage to use when writing objects. Possible values: REDUCED_REDUNDANCY, STANDARD, STANDARD_IA. (default: "STANDARD")
   --sse                  Enable basic server-side encryption at rest (SSE-S3) in S3 for all writes (default: off)
   --sse-kms key-id       Enable KMS encryption (SSE-KMS) for all writes using this particular KMS key-id. Leave blank to Use the account's CMK - customer master key (default: off)
   --acl value            The canned ACL to apply to the object. Possible values: private, public-read, public-read-write, authenticated-read, aws-exec-read, bucket-owner-read, bucket-owner-full-control (default: off)

MISC OPTIONS:
   --help, -h     Print this help text and exit successfully.
   --debug_fuse   Enable fuse-related debugging output.
   --debug_s3     Enable S3-related debugging output.
   -f             Run goofys in foreground.
   --version, -v  print the version