s3fs is a FUSE filesystem that allows you to mount an Amazon S3 bucket as a local filesystem. It stores files natively and transparently in S3. The maximum size of objects that s3fs can handle depends on Amazon S3. For example, up to 5 GB when using a single PUT API. And up to 5 TB is supported when Multipart Upload API is used. S3 bucket can take all RESTFul API calls like GET, PUT, LIST, and DELETE. HowTo Mount S3 bucket in Linux Using S3FS.

s3fs is stable and is being used in a number of production environments, e.g., Rsync backup to s3. S3 bucket can also be used for archival purposes best suitable solution for archival. Keeping retention for data like frequently accessed data can be tiered using its own mechanism

Configure EPEL Repository RHEL / Centos

---

 

To Install and configure S3FS configure EPEL repository

# yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
# yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Now remove old packages if already installed and install fresh packages for the latest and stable versions

RHEL / Centos

# yum remove fuse fuse-s3fs
# yum install gcc libstdc++-devel gcc++ curl-devel libxml2-devel openssl-devel mailcap git ntp fuse s3fs-fuse

Ubuntu / Debian

$ sudo apt-get install build-essential libcurl4-openssl-dev libxml2-dev mime-support ntp

Check NTP server is in sync otherwise you will get an error while mounting

# yum install ntpupdate
# yum install ntp
# ntpdate -q pool.ntp.org

Create S3 Bucket in Amazon

Login to S3 Console Click –> S3 –> Create bucket

Provide Bucket Name (should be unique), Select region, Click Next, Click Next, Set Permissions, Review, and Click Finish.

Go To –> Security & Identity Management –> IAM –> Users

The recommendation is to create a new user with programmatic access. Create Access Key

Download Access Key this key contains Secret Key ID and Secret. Filename must be passwd-s3fs otherwise mount will fail

# echo BKIAIY5KVWP6P7YN4TPQ:RVCEQPW1HgaTSH65yGx3RGvsJS1LDnLkG99H285Y > ~/.passwd-s3fs
# chmod 600 ~/.s3passwd
# mkdir -p /mnt/s3 /tmp/s3cache
# chmod 777 /mnt/s3 /tmp/s3cache

HowTo Mount S3 Bucket in Linux S3FS

Mount your S3 bucket in Linux using below s3fs command

# s3fs -o use_cache=/tmp/s3cache Server-ComputerBucket /mnt/s3

S3 Bucket fstab Entry

Edit /etc/fstab file and add below entry

<Bucket Name> <Mount Point> <File System> <Password File,Entries> <Options>

Server-ComputerBucket /mnt/s3 fuse.s3fs passwd_file=/root/.passwd-s3fs,allow_other,use_cache=/tmp/s3cache 0 0

That’s it. S3 Bucket Mounted successfully in Linux using S3FS.

Related Articles

For more AWS Articles