1

I am using p5 and ml5.js to train an ML model by adding some images via my webcam. The train function works fine like this. However, if I uncomment the if statement within the train function:

if (lossValue == null)

the value of classifier because undefined after that and it would throw an error in the next step. Why is this happening and how can I fix this? If I just use a console.log inside the if statement, it doesn't create a problem. However, if I use the setState inside it, it sets the classifier to undefined.

export const Component: React.FC<ComponentProps> = (props: ComponentProps) => {
    const [prediction, setPrediction] = useState<string>();
    const [confidence, setConfidence] = useState<string>();
    const [trainingComplete, setTrainingComplete] = useState<boolean>();
    //const [lossValue, setLoss] = useState<any>();

    let capture: p5Types.Element;
    let classifier: any;
    const setup = (p5: p5Types, canvasParentRef: Element) => {
        capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
        const featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
         classifier = featureExtractor.classification(capture, videoReady);
    }

    const draw = (p5: p5Types) => {
    }

    function modelReady() {
        console.log('Model Ready');
    }

    function videoReady() {
        console.log('Video Ready');
    }


    function gotResult() {
        console.log('classifier in results', classifier);
        classifier.classify(capture, (err: any, result: any) => {
            setPrediction(result[0].label);
            
        });
    }

    function train() {
        console.log('classifier in train', classifier);
        classifier.train((lossValue: any) => {
            console.log('Loss is', lossValue);
            // if (lossValue == null){
            //  setTrainingComplete(true);
            // }
        });
        console.log('classifier in train', classifier);
    }



    return (<div><Sketch setup={setup} draw={draw} className="sketch" />
        <div className="button">
            <Button variant="contained" color="primary" onClick={() => {classifier.addImage('first');console.log('image added')}}>First</Button>
            <Button variant="contained" color="primary" onClick={() => {classifier.addImage('second');console.log('image added')}}>Second</Button>
        </div>
        <div className="secondbutton">
            <Button variant="contained" color="primary" onClick={() => train()}>Train!</Button>
            <Button variant="contained" color="primary" onClick={() => gotResult()}>Test!</Button>
            <br />
            {trainingComplete && (<span>Training Complete!</span>)}<br />
        </div>
    </div>)
        ;
};

Codesandbox:

https://codesandbox.io/s/hardcore-solomon-zb34l?file=/src/Component.tsx

  • Maybe the called method `setTrainingComplete(true)` sets classifier to undefined?! –  Apr 18 '21 at 23:00
  • Yup, that's what's causing the issue, as mentioned in the qs @Lynx242 –  Apr 19 '21 at 00:24
  • Are you and @x89 working together? I just answered another question about this same code. Different problem, same core issue. You need to use `useRef`. https://stackoverflow.com/questions/67116772/typeerror-cannot-read-property-classify-of-undefined-save-ml5-js-model/67154884#67154884 – Linda Paiste Apr 19 '21 at 01:22
  • 1
    The solution in the answer below does not work. Could someone else try in the codesandboy? –  Apr 20 '21 at 00:13

2 Answers2

3

You are defining classifier as a let variable. It is not a state so it does not persist across re-renders. Every time your component gets re-rendered the code let classifier: any; gets executed and classifier becomes undefined.

When you call a setState function such as setTrainingComplete(true) this changes the state of your component and causes it to re-render. Therefore you lose the value of classifier.

You want to keep the value of classifier so you need to store it using a useState or useRef hook. I generally use useRef when dealing with complex objects from external libraries which have their own internal state. So that's what I suggest here.

const classifierRef = useRef<any>();
function train() {
    const classifier = classifierRef.current;
    console.log('classifier in train', classifier);
    classifier?.train((lossValue: any) => {
        console.log('Loss is', lossValue);
        if (lossValue == null) {
            setTrainingComplete(true);
        }
    });
    console.log('classifier in train', classifier);
}
Linda Paiste
  • 38,446
  • 6
  • 64
  • 102
0

You have to add the useRef for the capture variable as well. https://dementorwriter.medium.com/picture-classification-with-react-ml5-c45672aeb961

x89
  • 2,798
  • 5
  • 46
  • 110