A few questions regarding Object Storage lifecycle policies

While I'm no expert on Lifecycle policies for Object Storage, I've set up policies for a few of my buckets previously. If you haven't seen it yet, we have a great guide that covers setting lifecycle policies: How To Manage Objects with Lifecycle Policies. I highly recommend reading through that guide! As for your questions, I'll break them down below:

1. Yes, you can delete failed uploads after a day or two through setting AbortIncompleteMultipartUpload and its child DaysAfterInitiation. In the following example (taken from our guide) failed uploads are deleted after 3 days. See Additional Actions section of the aforementioned guide.

   ...
   <AbortIncompleteMultipartUpload>
     <DaysAfterInitiation>3</DaysAfterInitiation>
   </AbortIncompleteMultipartUpload>
   ...
   

2. To keep objects indefinitely, don't set the Expiration directive in a lifecycle policy. This directive will automatically delete objects after a certain number of days, which sounds like something you don't want.

3. Lifecycle policies are set on a per bucket level. I'm not aware of setting them per-file, but others may know of a workaround that I do not.

And just for your own knowledge, lifecycle policies can be managed through the command line through s3cmd or Cyberduck. You'll have more control using raw XML through s3cmd, but Cyberduck may be an easier tool depending on your experience.

You can filter lifecycle policies by prefix, file size, and tag in your lifecycle rules. Each strategy has its pros and cons.

Prefixes

Remember that prefixes are just that, prefixes, and there is no root '/' folder.

You can create a strategy to match a single file uniquely, but you cannot enforce that it will always be unique. To set a lifecycle policy for a single file, just assign a rule with a Prefix of the complete path and filename. NOTE: If another file appends on the original filename, the rule will apply to it as well. For example:

The prefix logs/exampleFile will also match:

logs/exampleFile2
logs/exampleFiles/anotherFile

Tags

S3 lets you implement custom tags to use in searching for your objects. Every time you create a new version of the file you would have to remember to apply the tag to it.

You could use the following to apply the tag:

aws s3api put-object-tagging --bucket bucket123 --key logs/exampleFile \
     --tagging LCRule=1

Your lifecycle rule would be something like:

<LifecycleConfiguration>
    <Rule>
        <ID>RemoveNonCurrentVersions</ID>
        <Filter>
            <Tag>
                <Key>LCRule</Key>
                <Value>1</Value>
            </Tag>
        </Filter>
        <Status>Enabled</Status>
        <NonCurrentVersionExpiration>
            <NonCurrentDays>30</NonCurrentDays>
        </NonCurrentVersionExpiration>
    </Rule>
</LifecycleConfiguration>