Some of the cloud services, (like MCG Noobaa/Azure Namespace store),
may not be able to map and store the objects which contain s3 style
metadata keys.
To help such services determine if the objects being transitioned
are from RGW and whether or not ignore such attrs, added below two
headers for the objects being copied/transitioned from RGW -
1) x-rgw-cloud : true/false
(set to "true" if the object is being transitioned/synced from RGW)
2) x-rgw-cloud-keep-attrs : true/false
- if set to default "true" , the cloud service should store all the
x-amz-meta-* attrs. If cannot be mapped/stored, the operation
should fail
- if set to "false", the destination cloud can ignore such attrs and
just store the object data being sent.
Also fixed a bug in the cloudtier module wherein the user-defined attrs
were not being copied to the cloud endpoint as part of transition
Fixes: https://tracker.ceph.com/issues/57980
Fixes: https://tracker.ceph.com/issues/58796
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
With commit#81ad226, aws auth v4 rquires region name for remote
endpoint connection. Include the same in the tier parameters.
& misc fixes
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
For versioned and locked objects, similar semantics as that of LifecycleExpiration are applied as stated below -
If the bucket versioning is enabled and the object transitioned to cloud is
- current version, irrespective of what the config option "retain_object" value is, the object is not deleted but instead delete marker is created on the source rgw server.
- noncurrent version, it is deleted or retained based on the config option "retain_object" value.
If the object is locked, and is
- current version, it is transitioned to cloud post which it is made noncurrent with delete marker created.
- noncurrent version, transition is skipped.
Also misc rebase fixes and cleanup -
* Rename config option to "retain_head_object"
to reflect its functionality to keep head object post transitioning
to cloud if enabled
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Currently the transition is supported to cloud providers
that are compatible with AWS/S3. Hence change the tier-type to
cloud-s3 to configure the S3 style endpoint details.
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
If the object is versioned, to avoid objects getting overwritten
post transition to cloud, append object versionID to the target
object name
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Also to avoid object name collisions across various buckets
post cloud transition, add bucket name to the object prefix.
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
