Can I Upload Entire Directories into an Object Storage Bucket?

Linode Staff

Do I have upload individual files to Object Storage or could I just upload a whole directory to my Object Storage Bucket?

1 Reply

You can upload entire directories into an Object Storage bucket. You can do this a few different ways. I would recommend using our Cloud Manager or installing s3cmd to interact with Object Storage. We have a guide on how to upload files and directories through our Cloud Manager.

Upload Objects to a Bucket

I do want to point out that uploading through the Cloud Manager, individual object uploads are limited to a size of 5GB each, though larger object uploads can be facilitated with multipart uploads. s3cmd and Cyberduck do this for you automatically if a file exceeds this limit as part of the uploading process.

We do have a guide that goes over how to install and use s3cmd and I will link that here:

Using s3cmd with Object Storage

When uploading a whole directory using s3cmd you may notice that you get an error if you just use the put command.

s3cmd put dir2 s3://ikuhn/
ERROR: Parameter problem: Use --recursive to upload a directory: dir2

You will need to put the --recursive tag to upload the whole directory.

s3cmd put dir2 s3://ikuhn/ --recursive
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
upload: 'dir2/file1' -> 's3://ikuhn/dir2/file1'  [1 of 4]
 0 of 0     0% in    0s     0.00 B/s  done
upload: 'dir2/file2' -> 's3://ikuhn/dir2/file2'  [2 of 4]
 0 of 0     0% in    0s     0.00 B/s  done
upload: 'dir2/file3' -> 's3://ikuhn/dir2/file3'  [3 of 4]
 0 of 0     0% in    0s     0.00 B/s  done
upload: 'dir2/file4' -> 's3://ikuhn/dir2/file4'  [4 of 4]
 0 of 0     0% in    0s     0.00 B/s  done

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct