Drawing with SVG across different phone screens using React-Native-Vision-Camera

Question

I'm trying to render a pose onto a camera screen with SVG and react-native-vision camera. It works fine on the normal 16:9 aspect ratio but on my friends iPhone Pro Max, which is 19.5:9 then the pose doesn't match up with the body.

It's in the correct shape, just doesn't grow with the body it seems.

Functions for Rending Pose

  const [frameWidth, setFrameWidth] = React.useState(Dimensions.get('screen').width);
  const [frameHeight, setFrameHeight] = React.useState(Dimensions.get('screen').height);

  const getFrameSize = (): number[] => {
    let width: number, height: number;
    if (Platform.OS === "android") {
      if (
        frameWidth > frameHeight &&
        Dimensions.get("window").width > Dimensions.get("window").height
      ) {
        width = frameWidth;
        height = frameHeight;
      } else {
        width = frameHeight;
        height = frameWidth;
      }
    } else {
      width = frameWidth;
      height = frameHeight;
    }
    return [width, height];
  };

  const getViewBox = () => {
    const frameSize = getFrameSize();
    const viewBox = "0 0 " + frameSize[0] + " " + frameSize[1];
    return viewBox;
  };

const renderPose = () => {
    if (poses != null && poses.length > 0) {
      const selectedJoints = [
        poses[0].LeftShoulder,
        poses[0].RightShoulder,
        poses[0].LeftHip,
        poses[0].RightHip,
        poses[0].RightKnee,
        poses[0].LeftKnee,
        poses[0].RightAnkle,
        poses[0].LeftAnkle,
        poses[0].RightShoulder,
        poses[0].LeftToe,
        poses[0].RightToe,
        poses[0].RightElbow,
        poses[0].LeftElbow,
        poses[0].RightThumb,
        poses[0].LeftThumb,
      ]
        .filter((k) => (k.inFrameLikelihood ?? 0) > 0.3)
        .map((k, index) => {
          // Flip horizontally on android or when using back camera on iOS.
          return (
            <Svg >
              <Circle
                key={`skeletonkp_${k.type}`}
                cx={k.position.x}
                cy={k.position.y}
                r={25}
                strokeWidth="2"
                fill="transparent"
                stroke={Theme.COLORS.WHITE}
              />

              {/* <Circle
                key={`skeletonkp_${k.type}${index}`}
                cx={k.position.x}
                cy={k.position.y}
                r={10}
                strokeWidth="0"
                fill="#FAF9F6"
                stroke="transparent"
              /> */}
            </Svg>
          );
        });
      return (
        <Svg
          viewBox={getViewBox()}
        >
          {selectedJoints}
        </Svg>
      );
    } else {
      return <View></View>;
    }
  };

Camera component

  <PinchGestureHandler onGestureEvent={onPinchGesture} enabled={isActive}>
          <Reanimated.View style={StyleSheet.absoluteFill}>
            <ReanimatedCamera
              ref={camera}
              style={StyleSheet.absoluteFill}
              device={device}
              format={format}
              fps={30}
              hdr={enableHdr}
              lowLightBoost={device.supportsLowLightBoost && enableNightMode}
              isActive={isActive}
              onInitialized={onInitialized}
              onError={onError}
              enableZoomGesture={false}
              animatedProps={cameraAnimatedProps}
              photo={true}
              video={true}
              audio={hasMicrophonePermission}
              frameProcessor={
                device.supportsParallelVideoProcessing && isActive
                  ? frameProcessor
                  : undefined
              }
              orientation="portrait"
              frameProcessorFps={30}
            />
          </Reanimated.View>
        </PinchGestureHandler>

score 0 · Accepted Answer · answered Jun 05 '23 at 23:31

For anyone that encounters same issue, I needed to find the correct camera format available and then style the camera component to be 16:9. With an iPhone pro max being 19.5:9, absolute fill made the image stretched and messed with the pose placement.

Drawing with SVG across different phone screens using React-Native-Vision-Camera

1 Answers1