How do I upload a file to Object Storage using python?
I try to access the object storage using python. However, I always receive the following error message:
Traceback (most recent call last): File "C:/dev/python-playground/s3sync/Test.py", line 16, in <module> print(s3.bucket_list('my-bucket')) File "C:\Python38\lib\site-packages\S3\S3.py", line 321, in bucket_list for truncated, dirs, objects in self.bucket_list_streaming(bucket, prefix, recursive, uri_params, limit): File "C:\Python38\lib\site-packages\S3\S3.py", line 355, in bucket_list_streaming response = self.bucket_list_noparse(bucket, prefix, recursive, uri_params, max_keys) File "C:\Python38\lib\site-packages\S3\S3.py", line 386, in bucket_list_noparse response = self.send_request(request) File "C:\Python38\lib\site-packages\S3\S3.py", line 1334, in send_request handler_fn = self._http_400_handler(request, response, self.send_request, request) File "C:\Python38\lib\site-packages\S3\S3.py", line 1215, in _http_400_handler return fn(*args, **kwargs) File "C:\Python38\lib\site-packages\S3\S3.py", line 1344, in send_request raise err S3.Exceptions.S3Error: 400 (InvalidArgument)
I use this script:
from S3.Config import Config from S3.S3 import S3 access_key = 'abc' secret_key = 'abcde' cfg = Config(None, access_key, secret_key) cfg.host_base = 'eu-central-1.linodeobjects.com' cfg.bucket_location = 'EU' cfg.host_bucket = 'my-bucket.eu-central-1.linodeobjects.com' cfg.website_endpoint = 'http://my-bucket.eu-central-1.linodeobjects.com' cfg.use_https = False s3 = S3(cfg) print(s3.bucket_list('my-bucket'))
Can you please tell me what's wrong?
Hey there -
I did some consulting with our resident Object Storage experts, and they recommend that you try doing this with boto/boto3, which are the official Python libraries.
Here is a doc that shows you how to do this. If you follow this, you should be able to get this working:
This also doesn’t quite look right:
cfg.bucket_location = 'EU'
Should this be
cfg.bucket_location = 'eu-central-1'
Do you get any more information in your error message, a HTTP 400 error means something in the request was wrong. Most AWS libraries give detailed reasons what’s wrong.
Hello and thank your for the answers.
Unfortunately, I do not have any more error messages than that one above.
I got it to work by first running "python3 s3cmd --configure" in the "Scripts" folder of the python installation and filled in the data according to this documentation: https://www.linode.com/docs/platform/object-storage/how-to-use-object-storage/#install-and-configure-s3cmd
After that, I could connect using the above approach (of course without the need to configure anything before connecting).
Since s3cmd is just one file with a lot of python functions, I take this as reference now. However, I did not dig deep into it for now.