Vertex Displacment with Audio Context (THREE.JS R76)

Question

I am trying to map vertices from the AudioContext api in Three.js.

Now, I have successfully done this with planes(non shader) but am running into problems trying to apply it to a cylinder. Since the cylinder vertices are full vectors, instead of 0's for a plane, I don't know how I would map them to the frequencyData.

I am including some extra functions for future viewers looking for Audio Context.

Audio Context

function audioLink(){
player = document.getElementById('musicPlayer'),
context = new (window.AudioContext || window.webkitAudioContext),
analyser = context.createAnalyser(),
source = context.createMediaElementSource(player);

source.connect(analyser);
analyser.connect(context.destination);
analyser.fftSize = 256;
frequencyData = new Uint8Array(analyser.frequencyBinCount);
analyser.getByteTimeDomainData(frequencyData);      
}

Here is my code for a top and bottom plane...

function updateVertWave(){
for (var i = 0, len = waveBottom.geometry.vertices.length; i < len; i++) {
    waveBottomVert[i].z = frequencyData[i]*6;   
    waveTopVert[i].z = frequencyData[i]*-6; 
}

waveBottom.geometry.verticesNeedUpdate = true;
waveTop.geometry.verticesNeedUpdate = true;
}

STUCK HERE

function updateVertCylinder(){
for (var i = 0, len = cylinder.geometry.vertices.length; i < len; i++) {
    (STUCK)
}
cylinder.geometry.verticesNeedUpdate = true;
cylinder.geometry.computeFaceNormals();
cylinder.geometry.computeVertexNormals(); 
scene.getObjectByName("cylinder").rotation.y += 0.004;
}

Render

function render() {
renderFrame = requestAnimationFrame(render);
analyser.getByteFrequencyData(frequencyData);
if (planeViz) {
    updateVertWave();
} else { 
    updateVertCylinder();
}
renderer.render(scene, camera);
};

I understand doing this with shaders makes more sense but I don't know enough for that yet. I suppose you would pass in the frequency data as a uniform but then I am right back to my original problem of freq to vector manipulation.

Interesting. So you mutate the z of each vertex according to the frequency data. Instead of depending on the order of the plane (which you could compute for the cylinder) why not just depend on x,y position to lookup in your frequency data? something like Math.floor(cylinderVerts[i].x/cylinderLengthInX * numOfFrequencyDatums) — user01, Apr 29 '16 at 17:22
I don't really understand what you're proposing but changing any vert.x , vert.y, or vert.z alone cause the cylinder to disappear which is why I need to change the entire vector3. I had played around with multiplying by a scalar and adding a scalar but it didnt work the way I need. — Michael Paccione, Apr 30 '16 at 14:38
Sorry if I was unclear. I meant you leave the x,y as they are and update the z for each vert. That will distort the cylinder in someway, but perhaps not in a useful or desired way. Could post a demo scene on blo.ocks.org or somewhere? — user01, Apr 30 '16 at 16:38
The relevant code is here I will comment back soon with some url links to view it live. I have tried.... cylinderVert[i] = cylinderVertArray[i].multiplyScalar(frequencyData[i]); .......cylinderVert[i].z = cylinderVert[i].z * frequencyData[i]).... etc etc but it is not visualizing how it i intended. I don't know enough about vectors to know what i have to do here. — Michael Paccione, Apr 30 '16 at 17:17
Your not describing how you want the vertices in the cylider to behave? On the plane you are setting the z-component to be the fequency * 6. I understand that will look like a wave. but for a cylinder, I guess you want the it to expand "outwards", increasing the radius with the frequency? — micnil, May 07 '16 at 01:39
essentially increase the amplitude of the displacement with the frequency. I have implemented this with a shader by now though. It was hidden deep in the three js examples here: http://threejs.org/examples/#webgl_custom_attributes — Michael Paccione, May 07 '16 at 02:19

score 1 · Accepted Answer · answered May 07 '16 at 02:21

1

Found a shader example deep in the three.js examples...

https://github.com/mrdoob/three.js/blob/master/examples/webgl_custom_attributes.html

answered May 07 '16 at 02:21

Michael Paccione

2,467
6
39
74

Vertex Displacment with Audio Context (THREE.JS R76)

1 Answers1