How do I use s3fs on Linode Object Storage?

This has been updated to reflect contributions by @mattwatts and @jyoo.

Mounting a Linode Object Storage bucket on your Linode

Installation

We want to install s3fs-fuse:

Depending on your distribution would do (as root):

Example Notes

This example will use mybucket as our bucket name, us-east-1 as our region and /mount/s3 as our mount point.

USEYOURREALACCESSKEY123 will be our access key, and
USEyourREALsecretKEYhereITisLONGER456 will be our secret key.

Wherever these values appear, feel free to substitute your own.

Create a mount point

mkdir -p /mount/s3

Edit your /etc/fstab file

Add the following on a single line:

mybucket /mount/s3 fuse.s3fs _netdev,allow_other,use_path_request_style,url=https://us-east-1.linodeobjects.com/ 0 0

Credentials

Put your credentials in /etc/passwd-s3fs separated by a colon:

USEYOURREALACCESSKEY123:USEyourREALsecretKEYhereITisLONGER456

Set the passwd-s3fs file's permissions:

chmod 600 /etc/passwd-s3fs

Mount your bucket

mount -a

Limitations

From: https://github.com/s3fs-fuse/s3fs-fuse

Generally S3 cannot offer the same performance or semantics as a local file system. More specifically:

• random writes or appends to files require rewriting the entire file
• metadata operations such as listing directories have poor performance due to network latency
• no atomic renames of files or directories
• no coordination between multiple clients mounting the same bucket
• no hard links
• inotify detects only local modifications, not external ones by other clients or tools

Following these instructions I am able read the contents of my s3 bucket. I can drop files into s3 via the linode interface and immediately see them on my mount. However, when I do 'touch test.txt' in the mounted folder the file is created but does not propagate to s3. Is this the expected behavior?

nobrega7121: You may need to change the permissions on the file in order to view them from other locations. The s3fs GitHub has some information about this. You could also use s3cmd by running s3cmd put example.txt s3://my-example-bucket -P, though this is different than creating a file in the mounted directory.

Hey there,

I was attempting to follow these instructions on my Ubuntu 20.04 machine and I ran into a couple of issues that I wanted to share with the community.

Essentially, when following these instructions verbatim, I received the error below:

terminate called after throwing an instance of 'std::invalid_argument'
what(): s3fs_strtoofft
Aborted (core dumped)

I took a look at the s3fs-fuse repo and noticed some discrepancies in the way that they recommend formulating /etc/fstab and what Harry has above. Essentially I needed to remove the s3fs# at the beginning of the line, change the fuse option after the mount point to fuse.s3fs, and remove the ,uid=userid,gid=groupid after the Linode Object Storage URL.

My /etc/fstab entry now looks like this:

{mybucket} {/path/to/mountpoint}  fuse.s3fs _netdev,allow_other,use_path_request_style,url=https://{region}.linodeobjects.com/ 0 0

After making this update I was receiving an error regarding my credentials that I had in ${HOME}/.passwd-s3fs and determined that I needed to move them to /etc/passwd-s3fs. After making that change, I was able to run the mount -a command and get this bucket mounted \o/.

Anyway, I hope this information helps anyone who ends up getting stuck with that same error that I was seeing.

-delete-