How do I use s3fs on Linode Object Storage?

Linode Staff

I want to mount my Linode Object Storage as a filesytem in my Linode. How do I do this?

4 Replies

Mounting an s3 bucket on your Linode

You can install fuse and s3fs:

On Debian you would just do:

sudo apt-get update && \
sudo apt-get upgrade && \
sudo apt-get install fuse s3fs

Edit your /etc/fstab file and add the following line (replace {mybucket}, {/path/to/mountpoint}, and {region} as appropriate:

s3fs#{mybucket} {/path/to/mountpoint} fuse _netdev,allow_other,use_path_request_style,url=https://{region}.linodeobjects.com/,uid=userid,gid=groupid 0 0

and put your credentials in /etc/passwd-s3fs or ~/.passwd-s3fs in this format:

S3_ACCESS_KEY:S3_SECRET_ACCESS_KEY

Set the passwd-s3fs file's permissions:

sudo chmod 600 /etc/passwd-s3fs

To mount the bucket, do:

mount -a

Limitations (from https://github.com/s3fs-fuse/s3fs-fuse)

Generally S3 cannot offer the same performance or semantics as a local file system. More specifically:

  • random writes or appends to files require rewriting the entire file
  • metadata operations such as listing directories have poor performance due to network latency
  • eventual consistency can temporarily yield stale data(Amazon S3 Data Consistency Model)
  • no atomic renames of files or directories
  • no coordination between multiple clients mounting the same bucket
  • no hard links
  • inotify detects only local modifications, not external ones by other clients or tools

Following these instructions I am able read the contents of my s3 bucket. I can drop files into s3 via the linode interface and immediately see them on my mount. However, when I do 'touch test.txt' in the mounted folder the file is created but does not propagate to s3. Is this the expected behavior?

nobrega7121: You may need to change the permissions on the file in order to view them from other locations. The s3fs GitHub has some information about this. You could also use s3cmd by running s3cmd put example.txt s3://my-example-bucket -P, though this is different than creating a file in the mounted directory.

Hey there,

I was attempting to follow these instructions on my Ubuntu 20.04 machine and I ran into a couple of issues that I wanted to share with the community.

Essentially, when following these instructions verbatim, I received the error below:

terminate called after throwing an instance of 'std::invalid_argument'
what(): s3fs_strtoofft
Aborted (core dumped)

I took a look at the s3fs-fuse repo and noticed some discrepancies in the way that they recommend formulating /etc/fstab and what Harry has above. Essentially I needed to remove the s3fs# at the beginning of the line, change the fuse option after the mount point to fuse.s3fs, and remove the ,uid=userid,gid=groupid after the Linode Object Storage URL.

My /etc/fstab entry now looks like this:

{mybucket} {/path/to/mountpoint}  fuse.s3fs _netdev,allow_other,use_path_request_style,url=https://{region}.linodeobjects.com/ 0 0

After making this update I was receiving an error regarding my credentials that I had in ${HOME}/.passwd-s3fs and determined that I needed to move them to /etc/passwd-s3fs. After making that change, I was able to run the mount -a command and get this bucket mounted \o/.

Anyway, I hope this information helps anyone who ends up getting stuck with that same error that I was seeing.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct