0

I am currently facing a problem.
Currently I am trying to mirror the cloned image using clone v2.

Below is the information of the image I cloned.

[root@ossrccephm1 ~]# rbd info data/clone1
rbd image 'clone1':
        size 1 GiB in 256 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 6752b7edc746d
        block_name_prefix: rbd_data.6752b7edc746d
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, journaling, operations
        op_features: clone-child
        flags:
        create_timestamp: Thu Jul 20 12:20:35 2023
        access_timestamp: Thu Jul 20 12:20:35 2023
        modify_timestamp: Thu Jul 20 12:20:35 2023
        parent: data/image1@75fcc859-c93c-437b-b6e7-0cbc5da0a415
        overlap: 1 GiB
        journal: 6752b7edc746d
        mirroring state: enabled
        mirroring global id: 75372d13-b86d-4992-8435-62f280aa1c2b
        mirroring primary: true

And one thing to note here is that I removed the snapshot after cloning the image.

[root@ossrccephm1 ~]# rbd snap ls data/image1 --all
SNAPID NAME                                 SIZE  PROTECTED TIMESTAMP                NAMESPACE
    16 75fcc859-c93c-437b-b6e7-0cbc5da0a415 1 GiB           Thu Jul 20 12:19:45 2023 trash (snap1)
    19 7a093c89-b516-4ada-9e52-eb016f62ecc0 1 GiB           Thu Jul 20 13:10:50 2023 trash (snap2)

And I mirrored the created image to the remote cluster, and as a result, the mirroring setup was successful, but the rbd-mirror daemon located in the remote cluster did not find the corresponding snapshot, so mirroring did not work.

2023-07-20T05:54:39.354+0000 7f1730edb640 -1 librbd::image::RefreshRequest: failed to locate snapshot: 75fcc859-c93c-437b-b6e7-0cbc5da0a415
2023-07-20T05:54:39.354+0000 7f1730edb640 -1 librbd::image::OpenRequest: failed to find snapshot 75fcc859-c93c-437b-b6e7-0cbc5da0a415
2023-07-20T05:54:39.354+0000 7f1722ebf640 -1 rbd::mirror::image_replayer::CreateImageRequest: 0x55c76eb58d20 handle_clone_image: failed to clone image data/674f9a2134e0e to clone1

It sounds like it's impossible to reference a snapshot that has been removed, so how does an image that is referencing that snapshot refer to its parent's data? I am curious about the principle of clone v2 and what actions are performed when the snapshot used in the clone is removed and falls into the trash.

I couldn't find any in-depth information about clone v2. help!

EDIT:

What I want to do is:

  1. set mirroring between source pool and target pool.
  2. create a image(image1)
  3. create a snapshot of image1, then the same snapshot will be created at the target cluster's image1
  4. make a clone image of image1's snapshot on the target cluster.
  5. remove the snapshot of image1 in the source cluster, then the snapshot of image1 in target cluster will be removed. But, because I made the clone image with clone v2 api at the target cluster.
  6. Since the clone image was referencing the removed snapshot, the actual snapshot was not completely removed and was left in the namespace trash. So, finally, the cloned image is a validly usable image.
  7. In this state, I tried to replicate the clone image referencing the snapshot that was removed (automatically stored in the trash namespace to be exact) by mirroring it back to the source cluster.
  8. In this situation, mirroring fails because the rbd-mirror daemon cannot find the removed snapshots (snapshots that currently exist in the namespace trash). Specifically, the rbd mirror image enable command completes successfully, but looking at the log of the rbd-mirror daemon, the removed snapshot is not found, so mirroring is not actually possible.

I would love to be able to find a snapshot with the rbd-mirror removed. Also, it would be nice if there was a way to revert snapshots that exist in the namespace trash. Since the flatten command is also executed, it seems correct that the removed snapshot itself is a valid snapshot, but I have not been able to find a command or method to access the snapshot.

Would you have understood my situation?

Thank you so much for reading my long question.

gyunn35
  • 3
  • 2
  • It's unclear to me what exactly you tried to accomplish, first you mention an image "clone1", then it's "image1". Apparently, you're using journal-based mirroring. So you clone a primary image (which is mirrored), create a snapshot (manually?) on the clone or on the image? And what is your expectation? That a removed snapshot from the primary image is still present in the mirrored cluster? You see, there are lots of questions, please clarify and edit your question so it becomes clearer what doesn't work as you expect it. – eblock Jul 21 '23 at 07:13
  • @eblock I explained my situation in more detail via edit. Thank you for your attention. – gyunn35 Jul 22 '23 at 10:49
  • I'm still not sure if I understand everything you write, but I tried to reproduce parts of it, here's what I understand so far: If you want the clone to be functional I recommend to flatten it before removing the original snapshot (in the source cluster), in my tests flattening wasn't done automatically. But flattening it removes the snapshot from the trash namespace, so it's kind of "cleaning up". IMO the process is working as expected, you have everything mirrored on the destination cluster so in case of a failure you can switch to the remote cluster and make those images primary. – eblock Jul 24 '23 at 06:25
  • If you remove snapshots in source they're removed from the destination cluster as well since that seems to be the state you want the image to be in, otherwise you wouldn't delete the snapshot, right? You can't have both, mirroring between sites to have identical images (and snapshots) but at the same time expect to keep them in different states (removed origin snap "survives" on remote cluster). So if you want to actually work with images on the remote cluster you'll need to clone and flatten your images to keep the original mirrored images in sync. – eblock Jul 24 '23 at 06:28
  • @eblock My final goal was to leave a clone image referencing the snapshot on the target cluster without leaving a snapshot on the source cluster while maintaining mirroring between the source cluster and the target cluster, and then mirror the clone image back to the source cluster. In fact, I know that this goal is difficult to achieve, but I was wondering if there was something I didn't know, so I posted a question. As you said, it seems impossible to make the state of the source cluster and target cluster different with mirroring set up. – gyunn35 Jul 24 '23 at 08:05
  • @eblock I was curious about the implementation principle of a clone image that effectively refers to a deleted snapshot even when the snapshot is deleted, and I couldn't find any documentation about that part, so I posted a question. Is there any way to get knowledge about rbd clone v2 api? Thank you so much for your continued interest. – gyunn35 Jul 24 '23 at 08:05
  • I'm not a developer so I can't really provide more details, you could probably contact the ceph developers in the mailing list, maybe you get a response, I don't know. – eblock Jul 24 '23 at 10:40

0 Answers0