4

I'm working on Java APP that will process the stream from the IP Camera (Milesight MS-C2682-P) located on Local network. It will detect objects and trigger actions depending on what's in the image (let´s say it will start an alarm, when a person is detected) - for that I need it to be with minimal delay.

I have an RTSP link "rtsp://username:password@ip_addr:rtsp_port/main", to access stream from my IP Camera, but in my JAVA app there is a 12 seconds delay (and it's increasing). This happens, when images are not handled fast enough, so they are buffered. There are "hacks" and "workarounds" (OpenCV VideoCapture lag due to the capture buffer), but I believe there has to be a prettier solution.

The other link I was able to get is an HTTP one, that uses also H.264 codec (can be used with MJPEG and MPEG4, if there is a possible way to use them effectively). "http://username:password@ip_addr:http_port/ipcam/mjpeg.cgi" - works like a charm.. in Python and browser. However, it doesn´t work in Java, an error is thrown:

OpenCV(4.2.0) C:\build\master_winpack-bindings-win64-vc14-static\opencv\modules\videoio\src\cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): HTTP_URL in function 'cv::icvExtractPattern'

Both links work smoothly in VLC.

So, the network is not a problem ('cause VLC handles stream with minimal delay) and Python using OpenCV is also doing a good job. It all comes down to Java implementation of OpenCV.. I guess.

Here is a Java code:

VideoPlayer videoPlayer = new VideoPlayer();    // My Class, just creates and updates JFrame, works like a charm with laptop's webcam, so certainly no issues here
Mat image = new Mat();
VideoCapture ipCamera = new VideoCapture(RTSP_URL);
// or the HTTP link
// VideoCapture ipCamera = new VideoCapture(HTTP_URL);

// verify if u got access to camera
if (!ipCamera.isOpened()) {
       System.out.println("ERROR: Camera isn't working !!! ");
       return;
}
System.out.println("OK: Connected to camera.");

while (true) {
        ipCamera.read(image);

        videoPlayer.updateVideo_MatImage(image);
}

And this is the Python code I'm using:

import cv2

cap = cv2.VideoCapture(RTSP_URL)
# or the HTTP link
# cap = cv2.VideoCapture(HTTP_URL)

while True:
    ret, image = cap.read()
    cv2.imshow("Test", image)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cv2.destroyAllWindows()

I just need to get the latest image, when a request is made. So I need to avoid any kind of buffering. It has to be implemented in Java since it's a requirement for this project. So is there a way to get only latest image from camera? What could cause the error mentioned above?

Thank you guys for any advice.

Jake_3H
  • 372
  • 4
  • 14
  • What did you do in your VideoPlayer class? – Bahramdun Adil Feb 19 '20 at 21:31
  • VideoPlayer class creates JFrame, imageIcon and JLabel - only necessary stuff for displaying image. Then there is a function to convert Mat to BufferedImage. This is certainly OK. Because when I'm using a laptop webcam instead of IP Cam, I don't have any problem with delay. – Jake_3H Feb 20 '20 at 11:00
  • I can tell you that, the OpenCV has wrappers for Java and Python, so all staff are done in the lower layer which is C/C++, Java or Python almost do nothing just calling the low layer functions, but in Java when you new the Mat, you must release it manually, because JVM will not release the memory for you because it is not a Java object, if you do not release it then you will have memory leak problem and your program will work very slow. I also have developed such apps in Java, which has no delay at any time and works well as in pure C++. – Bahramdun Adil Feb 20 '20 at 14:40
  • Alright. That sounds good. I'll try it as soon as I get to the camera (Monday I suppose). I've googled a bit about that. To "free" the memory, I should set the Mat to null and garbage collector will handle the rest. Is that right? Or is there another, more efficient way? I've looked here. Last comment even linked oracle, saying to null the objects, to help garbage collector: https://stackoverflow.com/questions/1567979/how-to-free-memory-in-java – Jake_3H Feb 21 '20 at 09:39
  • Setting the Mat object to null, GC will not help you release its memory, it is why OpenCV Mat class has provided a method called release(). Mat's data is not a Java object when you new a Mat then Java will call the C/C++ so C/C++ will create an object in the heap memory, so the object will live outside the JVM heap, so the JVM has no direct access to this memory space only by Mat object. So JVM only manages the Mat object inside the JVM heap, not its data outside of the JVM heap. So if you did not release the Mat object, then your PC will be hanged because of lack of memory. – Bahramdun Adil Feb 21 '20 at 13:01
  • With release() delay is still the same. News are, that even first .grap() funtion takes several seconds to return image - whole thread is stuck at this line (again, not problem in python). And then grab() returns true for around 150 times (it's a while loop, where only grab() is called and number of iteration displayed) and then stuck for around 5 seconds and then again around 150 returns of true. It ranges from 120 to 180. all those images it grabs are images captured during this "stuck" phase. Python grab() has no "stuck" phase. Do you have any other idea, what can cause this behavior? – Jake_3H Feb 27 '20 at 14:54

0 Answers0