14

ARKit 1.5 introduces image recognition. In the code you have to create a Set of the reference images like this:

let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil)

and then images contained in the Set can be recognised.

I wanted to know if it is possible to add images on the fly to this AR Resources folder. For example a user could take a picture and send it to a server, it is then recognised by the ARKit. Or a user could receive a set of images based on his location, etc.

Gaurav Bharti
  • 1,065
  • 1
  • 14
  • 22
magohamote
  • 1,484
  • 1
  • 17
  • 29
  • 2
    Probably "On-Demand Resources"? https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/On_Demand_Resources_Guide/ Or add each image programmatically: https://developer.apple.com/documentation/arkit/arreferenceimage/2942252-init – Karlis Mar 15 '18 at 13:10
  • thank you, I will look at it! – magohamote Mar 15 '18 at 13:11

2 Answers2

22

You cannot amend the contents of the default folder at runtime, although you can create images on the fly.

enter image description here

As an example let's put an image into the Assets Folder (not the ARResources one), which in my case is called 'moonTarget'.

We could then create a function which we call in viewDidLoad etc:

/// Create ARReference Images From Somewhere Other Than The Default Folder
func loadDynamicImageReferences(){

    //1. Get The Image From The Folder
    guard let imageFromBundle = UIImage(named: "moonTarget"),
    //2. Convert It To A CIImage
    let imageToCIImage = CIImage(image: imageFromBundle),
    //3. Then Convert The CIImage To A CGImage
    let cgImage = convertCIImageToCGImage(inputImage: imageToCIImage)else { return }

    //4. Create An ARReference Image (Remembering Physical Width Is In Metres)
    let arImage = ARReferenceImage(cgImage, orientation: CGImagePropertyOrientation.up, physicalWidth: 0.2)

    //5. Name The Image
    arImage.name = "CGImage Test"

    //5. Set The ARWorldTrackingConfiguration Detection Images
    configuration.detectionImages = [arImage]
}


/// Converts A CIImage To A CGImage
///
/// - Parameter inputImage: CIImage
/// - Returns: CGImage
func convertCIImageToCGImage(inputImage: CIImage) -> CGImage? {
    let context = CIContext(options: nil)
    if let cgImage = context.createCGImage(inputImage, from: inputImage.extent) {
     return cgImage
    }
    return nil
}

We can then test this in the ARSCNViewDelegate:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {

    //1. If Out Target Image Has Been Detected Than Get The Corresponding Anchor
    guard let currentImageAnchor = anchor as? ARImageAnchor else { return }

    let x = currentImageAnchor.transform
    print(x.columns.3.x, x.columns.3.y , x.columns.3.z)

    //2. Get The Targets Name
    let name = currentImageAnchor.referenceImage.name!

    //3. Get The Targets Width & Height In Meters
    let width = currentImageAnchor.referenceImage.physicalSize.width
    let height = currentImageAnchor.referenceImage.physicalSize.height

    print("""
    Image Name = \(name)
    Image Width = \(width)
    Image Height = \(height)
    """)

    //4. Create A Plane Geometry To Cover The ARImageAnchor
    let planeNode = SCNNode()
    let planeGeometry = SCNPlane(width: width, height: height)
    planeGeometry.firstMaterial?.diffuse.contents = UIColor.white
    planeNode.opacity = 0.25
    planeNode.geometry = planeGeometry

    //5. Rotate The PlaneNode To Horizontal
    planeNode.eulerAngles.x = -.pi/2

    //The Node Is Centered In The Anchor (0,0,0)
    node.addChildNode(planeNode)

    //6. Create AN SCNBox
    let boxNode = SCNNode()
    let boxGeometry = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)

    //7. Create A Different Colour For Each Face
    let faceColours = [UIColor.red, UIColor.green, UIColor.blue, UIColor.cyan, UIColor.yellow, UIColor.gray]
    var faceMaterials = [SCNMaterial]()

    //8. Apply It To Each Face
    for face in 0 ..< 5{
        let material = SCNMaterial()
        material.diffuse.contents = faceColours[face]
        faceMaterials.append(material)
    }
    boxGeometry.materials = faceMaterials
    boxNode.geometry = boxGeometry

    //9. Set The Boxes Position To Be Placed On The Plane (node.x + box.height)
    boxNode.position = SCNVector3(0 , 0.05, 0)

    //10. Add The Box To The Node
    node.addChildNode(boxNode)   
}

As you can see the same could also be applied from a live feed as well.

Hope this helps...

As @Karlis said you could also look at using OnDemandResouces and then converting them to desired specs of an ARReferenceImage.

Update: For anyone looking to see an example of creating dynamic reference images from a Server please take a look at the following project I have created: Dynamic Reference Images Sample Code

BlackMirrorz
  • 7,217
  • 2
  • 20
  • 31
  • Many thanks! So here when you set `configuration.detectionImages = [arImage]` the detectable images are limited to the one you just created right? Or does it add up to the ones already present in the AR Resource folder? I think I need to access the configuration of the current session, then access the actual set of images, then add my new image to this set, sounds correct? – magohamote Mar 15 '18 at 16:28
  • 1
    From my testing if you need to use the ARBundle you need to use: ARReferenceImage.referenceImages(inGroupNamed: AR_BUNDLE, bundle: nil), then if you want to use dynamic ones you will need to create a new set. You can alternate between them though :) – BlackMirrorz Mar 15 '18 at 23:32
  • I don't get why @op didn't accept this answer. anyways, thank you! it solved my problem :) – MaX Apr 20 '18 at 09:01
  • Hi @BlackMirrorz!! Would you have time to help me with ARkit related question too? https://stackoverflow.com/questions/62927167/swift-get-the-truthdepth-camera-parameters-for-face-tracking-in-arkit :( – swiftlearneer Jul 17 '20 at 05:23
5
let documentDirectory = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]
let pathToObject = documentDirectory + "/reference.jpg"
let imageUrl = URL(fileURLWithPath: pathToObject)
let imageData: Data = try! Data(contentsOf: imageUrl)
let image = UIImage(data: imageData, scale: UIScreen.main.scale)
let cgImage = image!.cgImage

let referenceImages = ARReferenceImage.init(cgImage!, orientation: .up, physicalWidth: 0.595)
referenceImages.name = "Reference Image"
let configuration = ARImageTrackingConfiguration()
configuration.trackingImages = [referenceImages]
session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
Aayushi
  • 787
  • 10
  • 14