How to load a model and textures from a remote server using ARKit?

Question

I am working on an ARKit app using Swift in Xcode trying to load models from a remote server but am having an issue getting the model and the textures / materials to load together and have the textures / materials display correctly.

I have gone through a few links and tutorials and get the model to load but the materials do not show. I have created models in the scene editor, or downloaded them and converted to .scn files, locate them in Finder and then uploaded them to a web-server. Just the .scn file and material (images).

//Tap Gesture 
@objc func handleTap(_ gesture: UITapGestureRecognizer) {

    //hittest
    let results = self.sceneView.hitTest(gesture.location(in: gesture.view), types: ARHitTestResult.ResultType.featurePoint)

    //return first tap
    guard let result: ARHitTestResult = results.first else {
        return
    }

    //Set URL of location of model
    let myURL = NSURL(string: "https://www.website.com/scnfiles/iPhoneX.scn")

    //Try getting this url or return 
    guard let scene = try? SCNScene(url: myURL! as URL, options: nil) else {return}

    //Set the node to be the model
    let node = scene.rootNode.childNode(withName: "SketchUp", recursively: true)

    //Set scale
    node?.scale = SCNVector3(0.025,0.025,0.025)

    //The material image is located in the same directory
    node?.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "https://website/scnfiles/iPhoneX_Screen.jpg")

    //set the position of the model
    let position = SCNVector3Make(result.worldTransform.columns.3.x, result.worldTransform.columns.3.y, result.worldTransform.columns.3.z)
        node?.position = position

    //Add to scene
    self.sceneView.scene.rootNode.addChildNode(node!)
}

I would like the model to load correctly as it would locally, showing the model with the applied textures but instead I get the model with no textures, just colors or white where the material is expected.

I receive the following error which looks like it is trying to load it locally?

ARKitModels[10386:3406637] [SceneKit] Error: Failed to load : <C3DImage 0x281e45180 src:file:///var/containers/Bundle/Application/233AE78F-748F-420B-96AD-30F591ADF80C/ARKitModels.app/material/iPhoneX_Screen.jpg [0.000000x0.000000]>

Any help is appreciated and if there is a better way to do this please let me know. Thank you!

score 4 · Answer 1 · edited Jan 23 '20 at 04:54

So, since I want to load everything from a remote server, nothing locally here is what I came up with thanks to some help from @ATV.

Solution: Loop though child nodes and set texture to the same name as the node.name. You can set the name of the node in scene editor and then have the same named image files in the proper folder. This allows for dynamic loading. This only works if you have control over the models / server or have explicit instructions.

@objc func handleTap(_ gesture: UITapGestureRecognizer) {

    let results = self.sceneView.hitTest(gesture.location(in: gesture.view), types: ARHitTestResult.ResultType.featurePoint)

    guard let result: ARHitTestResult = results.first else {
        return
    }

    let myURL = NSURL(string: "https://www.website.com/scnfiles/model/model.scn")

    guard let scene = try? SCNScene(url: myURL! as URL, options: nil) else {
        return
    }

    let node = scene.rootNode.childNode(withName: "SketchUp", recursively: true)

    // Solution //
    let children = (node?.childNodes)!

    for child in children {
        child.geometry?.materials.forEach{$0.diffuse.contents = "https://www.website.com/scnfiles/model/materials/" + child.name! + ".jpg"
    }

    node?.scale = SCNVector3(0.025, 0.025, 0.025)

    let position = SCNVector3Make(result.worldTransform.columns.3.x, 
                                  result.worldTransform.columns.3.y, 
                                  result.worldTransform.columns.3.z)
    node?.position = position
    self.sceneView.scene.rootNode.addChildNode(node!)
}

@ Eric .18 can you please tell me in which format to take models from remote url.Please share any remote one of the url — Gangireddy Rami Reddy, Jan 27 '20 at 11:16
@ Eric .18 Can you please tell to me how we can put .scn modes in server side. — Gangireddy Rami Reddy, Jan 27 '20 at 11:17

score 0 · Answer 2 · answered May 08 '19 at 12:46

0

Try to set pre-downloaded UIImage as contents to all materials from your geometry. Something like:

geometry.materials.forEach{$0.diffuse.contents = textureImage}

answered May 08 '19 at 12:46

ATV

338
1
4
19

Thanks! How does this work with all textures / a model with many textures and different images on different surfaces? It just placed the texture around all surfaces. – Eric.18 May 08 '19 at 12:53
Usually, if you use multi textures - you have this list of files: - `.obj` (or any model's format) file with defined texture coordinates - `.mtl` file with defined materials, lighting settings etc - resources (.jpg, .png) files that defined in `.mtl` as textures for materials You can read more about this [here](https://en.wikipedia.org/wiki/Wavefront_.obj_file) – ATV May 08 '19 at 13:53
@Eric.18 All this files, listed above should be located in one directory - in this case textures will appear automatically on inited SCNScene. – ATV May 08 '19 at 14:49
All the files are in the same directory on a server. The textures do not appear as expected. In your answer, what is textureImage? You think just having the images and .scn in the same directory will just show without setting any materials? Should I be calling init SCNScene in a specific way? – Eric.18 May 08 '19 at 15:21
1

@Eric.18 "should be located in one directory" - I mean locally on phone, not on a server. As a first step - you should download (e.g. with `URLSession.shared.downloadTask...`) model and texture files and save them to local storage (e.g. with `FileManager.default` to `NSTemporaryDirectory()`). After this you should initiated your `SCNScene` by passing URL to **local** model. Textures should appear on model without manual setup. – ATV May 08 '19 at 15:32
Thanks. Seems to defeat the purpose of everything being loaded remotely instead of locally... I would have to update and re-submit the app each time I need a new model? So why not just have the .scn model locally as well? I created a solution to load each texture from the server and will post the solution once I test it a bit. – Eric.18 May 08 '19 at 15:36
Not sure that I caught the thought about "re-submit the app each time I need a new model". It depends on your needs - you can provide me more details about your plans about using models and textures if needed. – ATV May 08 '19 at 15:57

score 0 · Answer 3 · answered Jun 28 '22 at 02:22

0

Save the trouble and use single .usdz files for your models, instead of folders of .obj with textures.

answered Jun 28 '22 at 02:22

elemento

345
3
11

How to load a model and textures from a remote server using ARKit?

3 Answers3

Linked