My lifecycle policies in linode storage not work

There seems to be an issue with your configuration file. Lifecycle policy must be written in a specific way to work. Each lifecycle policy file needs a LifecycleConfiguration block and a nested Rule block. Writing them on the same line will not work.

This is what a sample policy looks like:

<LifecycleConfiguration>
    <Rule>
        <ID>delete-all-objects</ID>
        <Prefix></Prefix>
        <Status>Enabled</Status>
        <Expiration>
            <Days>1</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>

Our guide on Managing Objects with Lifecycle Policies has good information on how to setup a lifecycle policy.

Hi @isackey

Thank you for your response quickly
I think no error in my lifecycle file
Please check code in below
root@gitlab:/var/opt/gitlab/backups# cat lifecycle_policy.xml
<lifecycleconfiguration> <rule> <id>delete-all-objects</id> <prefix></prefix> <status>Enabled</status> <expiration> <days>3</days> </expiration> </rule> </lifecycleconfiguration>
root@gitlab:/var/opt/gitlab/backups# s3cmd getlifecycle s3://gitlab

<lifecycleconfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <rule> <id>delete-all-objects</id> <prefix> <status>Enabled</status> <expiration> <days>3</days> </expiration> </prefix></rule> </lifecycleconfiguration>

Hi @isackey

Thank you for your response quickly
I think no error in my lifecycle file
Please check code in below
root@gitlab:/var/opt/gitlab/backups# cat lifecycle_policy.xml

 <lifecycleconfiguration> 
     <rule> 
         <id>delete-all-objects</id> 
         <prefix></prefix> 
         <status>Enabled</status> 
         <expiration> 
            <days>3</days> 
         </expiration> 
     </rule> 
 </lifecycleconfiguration>

root@gitlab:/var/opt/gitlab/backups# s3cmd getlifecycle s3://gitlab

      <lifecycleconfiguration xmlns="http://s3.amazonaws.com/doc/2006- 
 03-01/"> 
     <rule> 
         <id>delete-all-objects</id> 
         <prefix> 
         <status>Enabled</status> 
         <expiration> 
            <days>3</days> 
         </expiration> 
     </prefix></rule> </lifecycleconfiguration>

I asked a similar question last week. There is definitely something up with the lifecycle policy. This morning it did run and deleted files stored between December 4th to December 16th in one go. I have made no changes whatsoever. In fact I was going to delete the old files manually as leaving it longer would have incurred additional charges.

https://www.linode.com/community/questions/22247/object-storage-lifecycle-config-not-working

I have a data processing app that uploads one file per minute from 4 servers - so 5-6k files per day, which are stored with a prefix of “processing” and the date - e.g. processing/2021-12-28.

I have a lifecycle rule that removes files with a prefix of “processing” after 7 days of being uploaded.

If I look back, there are a handful (probably no more than 20) still left each day prior to the current 7 day period.

So it’s definitely not quite working 100%.

Hi @isackey and all

Now my lifecycle policy work correctly

I dont update any line in code of lifecycle policy

Thank you

After it mass deleted 12 days worth of data in one go, it worked correctly for 6 days before it stopped working again. Currently have 20 days worth of data with a 13 day expiration policy.

A few notes:

• The xmlns element is not required in your lifecycle policy.

• A lifecycle policy may not complete due to processing overhead on any given day

• @truong-vu I see that the output of s3cmd getlifecycle s3://gitlab put some elements out of order and put the <status> and <expiration> tags inside the <prefix> tag. It looks like the <filter> element was originally omitted. You might want to try:

<LifecycleConfiguration> 
     <Rule> 
         <ID>delete-all-objects</ID>
         <Filter>
             <Prefix></Prefix>
         </Filter>
         <Status>Enabled</Status> 
         <Expiration> 
            <Days>3</Days> 
         </Expiration> 
     </Rule> 
 </LifecycleConfiguration>

A lifecycle policy may not complete due to processing overhead on any given day

What happens if an Expiration rule to expire files when they are 7 days old, doesn't run completely on day 7 due to this overhead.

Would this catch up on day 8 or onwards?

A lifecycle policy may not complete due to processing overhead on any given day

Since my last post 3 weeks ago:

After it mass deleted 12 days worth of data in one go, it worked correctly for 6 days before it stopped working again. Currently have 20 days worth of data with a 13 day expiration policy.

…the lifecycle policy has not run a single time. No changes whatsoever to the configuration. I'm currently manually deleting files in order to stay on top of things but I think we have to write a tool to manage the lifecycle ourselves at this point.

Was there any resolution to this?

I am trying to delete failed multipart uploads with the code from Linode's own documentation: https://www.linode.com/docs/products/storage/object-storage/guides/lifecycle-policies/#additional-actions

I also get this response when I run getlifecycle:

<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-    03-01/">
        <Rule>
                <ID>delete-incomplete-multipart-uploads</ID>
                <Prefix/>
                <Status>Enabled</Status>
                <AbortIncompleteMultipartUpload>
                        <DaysAfterInitiation>1</DaysAfterInitiation>
                </AbortIncompleteMultipartUpload>
        </Rule>
</LifecycleConfiguration>

Yet, it doesn't work.

Since it's failed multipart uploads, "ls" doesn't list the multipart objects either.

Super frustrating that I am getting billed without recourse.