How do I upload a file to Object Storage using python?

Hello,

I try to access the object storage using python. However, I always receive the following error message:

Traceback (most recent call last):
  File "C:/dev/python-playground/s3sync/Test.py", line 16, in <module>
    print(s3.bucket_list('my-bucket'))
  File "C:\Python38\lib\site-packages\S3\S3.py", line 321, in bucket_list
    for truncated, dirs, objects in self.bucket_list_streaming(bucket, prefix, recursive, uri_params, limit):
  File "C:\Python38\lib\site-packages\S3\S3.py", line 355, in bucket_list_streaming
    response = self.bucket_list_noparse(bucket, prefix, recursive, uri_params, max_keys)
  File "C:\Python38\lib\site-packages\S3\S3.py", line 386, in bucket_list_noparse
    response = self.send_request(request)
  File "C:\Python38\lib\site-packages\S3\S3.py", line 1334, in send_request
    handler_fn = self._http_400_handler(request, response, self.send_request, request)
  File "C:\Python38\lib\site-packages\S3\S3.py", line 1215, in _http_400_handler
    return fn(*args, **kwargs)
  File "C:\Python38\lib\site-packages\S3\S3.py", line 1344, in send_request
    raise err
S3.Exceptions.S3Error: 400 (InvalidArgument)

I use this script:

from S3.Config import Config
from S3.S3 import S3

access_key = 'abc'
secret_key = 'abcde'

cfg = Config(None, access_key, secret_key)
cfg.host_base = 'eu-central-1.linodeobjects.com'
cfg.bucket_location = 'EU'
cfg.host_bucket = 'my-bucket.eu-central-1.linodeobjects.com'
cfg.website_endpoint = 'http://my-bucket.eu-central-1.linodeobjects.com'
cfg.use_https = False

s3 = S3(cfg)
print(s3.bucket_list('my-bucket'))

Can you please tell me what's wrong?

3 Replies

Hey there -

I did some consulting with our resident Object Storage experts, and they recommend that you try doing this with boto/boto3, which are the official Python libraries.

Here is a doc that shows you how to do this. If you follow this, you should be able to get this working:

Ceph - Python S3 Examples

This also doesn’t quite look right:

cfg.bucket_location = 'EU'

Should this be

cfg.bucket_location = 'eu-central-1'

?

Do you get any more information in your error message, a HTTP 400 error means something in the request was wrong. Most AWS libraries give detailed reasons what’s wrong.

Hello and thank your for the answers.
Unfortunately, I do not have any more error messages than that one above.
I got it to work by first running "python3 s3cmd --configure" in the "Scripts" folder of the python installation and filled in the data according to this documentation: https://www.linode.com/docs/platform/object-storage/how-to-use-object-storage/#install-and-configure-s3cmd

After that, I could connect using the above approach (of course without the need to configure anything before connecting).
Since s3cmd is just one file with a lot of python functions, I take this as reference now. However, I did not dig deep into it for now.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct