Embed widgets with pythreejs: wrong perspective and camera look-at

Question

I am using pythreejs to visualize some 3D models.

When visualizing the models on a Jupyter notebook, everything works as expected.

But when trying to embed the widget in an HTML document, I am facing two problems:

It seems the camera, on load, is looking at (0, 0, 0), not as expected, and once you interact with the widget, the camera will "jump" and start looking at the expected coordinate
The projection (ortographic camera mode) is lost too

Here is the code to reproduce the error and an animation of the mentioned problem:

from ipywidgets import embed
from pythreejs import *
from IPython.display import display

base = Mesh(
    BoxBufferGeometry(20, 0.1, 20), 
    MeshLambertMaterial(color='green', opacity=0.5, transparent=True),
    position=(0, 0, 0),
)
cube = Mesh(
    BoxBufferGeometry(10, 10, 10), 
    MeshLambertMaterial(color='green', opacity=0.5, transparent=False),
    position=(0, 5, 0),
)
target = (0, 5, 0)
view_width = 600
view_height = 400
camera = CombinedCamera(position=[60, 60, 60], width=view_width, height=view_height)
camera.mode = 'orthographic'
lights = [
    PointLight(position=[100, 0, 0], color="#ffffff"),
    PointLight(position=[0, 100, 0], color="#bbbbbb"),
    PointLight(position=[0, 0, 100], color="#888888"),
    AmbientLight(intensity=0.2),
]
orbit = OrbitControls(controlling=camera, target=target)
camera.lookAt(target)
scene = Scene(children=[base, cube, camera] + lights)
renderer = Renderer(scene=scene, camera=camera, controls=[orbit],
                    width=view_width, height=view_height)
camera.zoom = 4

embed.embed_minimal_html('export.html', views=renderer, title='Renderer')
display(renderer)

The result looks good in the notebook:

But when opening the export.html file:

Note how the view of the cube "jumps" suddenly on interaction and how the projection is different: perspective instead of orthographic (parallel) projection.

Could it be an issue with ipywidgets? Since the view is okay when displayed in the notebook.

How could it be fixed?

score 3 · Accepted Answer · answered Jan 16 '20 at 15:43

This is a bug in the CombinedCamera code in pythreejs (https://github.com/jupyter-widgets/pythreejs/issues/308). The sync logic will need to call updateProjectionMatrix on the camera when any of the attributes change. The OrbitController calls this when you interact with it, which is why the view is 'fixed' when interacting.

ZF007 · Answer 2 · 2020-01-09T19:40:13.463

It took a few days, didn't get cadquery working properly, neither the code from your first question on this topic without cadquery The code here made it possible to look at the issue...

The jumping happens because orbit.update() for target does not occurs and the function update() is not available in python; only in c++ or c#, etc. From the docs:

When animating the camera rotation above, we used the camera’s quaternion . This is the most robust method for animating free-form rotations. For example, the animation above was created by first moving the camera manually, and then reading out its position and quaternion properties at the wanted views...

The text can be found here on page 12. And also discussed here at github.

However, the jumping can be reproduced in IPython if you apply the following:

renderer = Renderer(scene=scene, camera=camera, controls=[orbit], position=target, width=view_width, height=view_height)

here position is added with target coordinates [0, 5, 0] but the update for this is only done when you mouse-click and adjust to position of the cube/camera. The jump is similar/equal to the jump as seen in the export.HTML.

Conclusion: the programmed camera position is seen as a jump after manual interference due to the absense of the .update() function of the OrbitControls python class and thus not a bug or mistake.

Update 1 - Ipython Renderer output while not ran in jupyter-notebook:

target = (0,5,0)
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), target=(0.0, 5.0, 0.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

target = (0,0,0)
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

target = (0, 5, 0) and position=target
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), target=(0.0, 5.0, 0.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

target = (0,0,0)  and position=(0, 5, 0)
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

To test the .update() for the export.html I consider a search for free SDD so I'm able to test it in a linux conda environment using my current yml-file.

I think this does not answer the questions: is it an `ipywidgets` issue? (since the view is properly rendered in Jupyter, but not in the exported HTML; how could it be fixed? How does `OrbitControls.update()` Python method have anything to do with this? The exported HTML has no Python code and the rendered view in Jupyter works as expected (so does not have any issues with that `.update()` it seems. — Peque, Jan 09 '20 at 17:49
Also, you seem to focus on the "jump", but there is also the loss of perspective/mode. The camera should have an ortographic view in the exported document, but that is not the case. It only works well in the Jupyter notebook. — Peque, Jan 09 '20 at 17:53
If you set target to (0,0,0) and position to (0,5,0) you get identical views from the export and Ipython. I've tested it using komodo edit 11.1 and at a friends place with sublime 3.2.1. I've updated the answer with output from four options. The first option shows the jump and all other don't. Camera view angle in all case exactly the same. — ZF007, Jan 09 '20 at 19:35
Yeah, but I do not want to set the target to `(0, 0, 0)`. I want the target to be `(0, 5, 0)` as in the example I provided. Also, do any of your options result in an exported HTML view with orthographic camera? — Peque, Jan 09 '20 at 20:37
"orthographic" representation in 1, 2, 3 or curvilinear projection? See also [here](https://en.wikipedia.org/wiki/Orthographic_projection) for views. I think a still image of your preferred view in the question will help us to compare the exact viewpoints so that there is no confusion about it as I suspect is right now :-) For me its in both cases (t=0,0,0 and t,0,5,0) I see a trimetric/2-point view. — ZF007, Jan 10 '20 at 11:01
Ortographic as per [the image in that Wikipedia article](https://en.wikipedia.org/wiki/Orthographic_projection#/media/File:Comparison_of_graphical_projections.svg) and as opposed to a perspective projection. An example of the expected output is showed in my question: the first animation shows an ortographic view, whereas the second animation shows a perspective view. Maybe the difference is tiny, but it exists. :-) — Peque, Jan 10 '20 at 12:21

Embed widgets with pythreejs: wrong perspective and camera look-at

2 Answers2

Linked