Program crashes when camera is facing plane with RemoteCam

[hedkaoz]

Hi, I'm working with a remote camera feed in Unigine and using OpenCV to capture and display frames in real time on a plane. The basic workflow is as follows:
I'm capturing frames from a remote camera using OpenCV and saving these frames as temporary images to disk. I'm then continuously loading the images to update as textures in a material of an ObjectMeshStatic object. The texture updates correctly, but the problem is that the program only crashes when the camera is facing the plane with the material, not when it's anywhere else in the scene.

Additional details: The problem doesn't occur if I'm not facing the plane with the material, suggesting that there might be something related to the texture rendering on that specific material.
The code doesn't throw any obvious exceptions, but the game freezes after a few seconds or minutes of facing the plane.

HERES THE CODE:

using System;

using System.Threading;

using Unigine;

using OpenCvSharp;

 

[Component(PropertyGuid = "d17ac209a67a4e27ae0648e2f78f1b1f5e1bb254")]

public class RemoteCameraStream : Component

{

    private Material videoMaterial;

    private VideoCapture videoCapture;

    private volatile bool isRunning = false;

    private volatile bool reloadTexture = false;    

    private readonly string streamUrl = "X";   // URL del stream HLS , edit: sorry i cant leave here the URL

 

    public void Init()

    {

        Node screenNode = World.GetNodeByName("VideoScreenRemoto");

        if (screenNode == null)

        {

            Log.Error("Nodo 'VideoScreenRemoto' no encontrado.");

            return;

        }

        ObjectMeshStatic screenMesh = screenNode as ObjectMeshStatic;

        if (screenMesh == null)

        {

            Log.Error("'VideoScreenRemoto' no es un ObjectMeshStatic.");

            return;

        }

        videoMaterial = screenMesh.GetMaterial(0);

        if (videoMaterial == null)

        {

            Log.Error("Material no encontrado.");

            return;

        }

        videoMaterial.TwoSided = true;

        videoMaterial.SetParameterFloat4("albedo_color", new vec4(1, 1, 1, 1));

        videoMaterial.SetState("transparent", 0);

        isRunning = true;

        new Thread(CaptureAndSaveFrames).Start();

    }

 

    private void CaptureAndSaveFrames()

    {

        try

        {

            videoCapture = new VideoCapture(streamUrl);

            if (!videoCapture.IsOpened())

            {

                Log.Error("No se pudo abrir el stream remoto.");

                return;

            }

            Log.Message("Stream remoto abierto correctamente.\n");

            Mat frame = new Mat();

            while (isRunning)

            {

                if (videoCapture.Read(frame) && !frame.Empty())

                {

                    Cv2.CvtColor(frame, frame, ColorConversionCodes.BGR2BGRA);

                    string path = Engine.AppPath + "/remote_frame.png";

                    Cv2.ImWrite(path, frame);

                    reloadTexture = true;

                }

                Thread.Sleep(100); //frames por segundo

            }

        }

        catch (Exception e)

        {

            Log.Error($"Error en stream remoto: {e.Message}");

        }

        finally

        {

            videoCapture?.Release();

        }

    }

 

    public void Update()

    {

        if (reloadTexture)

        {

            reloadTexture = false;

 

            string path = Engine.AppPath + "/remote_frame.png";

            if (Unigine.FileSystem.IsFileExist(path))

            {

                Texture tex = new Texture();

                if (tex.Load(path))

                {

                    videoMaterial.SetTexture("albedo", tex); //actualiza los frames en la pantalla

                    Log.Message("Frame remoto actualizado desde archivo.\n");

                }

                else

                {

                    Log.Error("Falló la carga de textura remota.");

                }

            }

        }

    }

 

    public void Shutdown()

    {

        isRunning = false;

        videoCapture?.Release();

        videoCapture = null;

        Log.Message("Stream remoto finalizado.\n");

    }

}

Edited April 25, 2025 by hedkaoz

[cash-metall]

Hello!

when you create a texture from file, it is not immediately loaded into memory. Texture Streaming mechanism is used - i.e. it will be loaded from disk only when needed + try to generate a mipmap for it. 
during this time, you change the file in another thread right at the moment of loading this file - I assume this is the reason for the crash.

some tips that will help solve your problem:

1. manage thread sync mechanizm to aviod simultaneous writing and reading file. (Simple Mutex lock around ImWrite and Load / use additional buffers for reading and writing from different files)

2. use Image and setTextureImage to disable Texture Streaming mechanism.

string path = Engine.AppPath + "/remote_frame.png";
if (Unigine.FileSystem.IsFileExist(path))
{
    Image image = new Image();
    if (image.Load(path))
    {
        int index = videoMaterial.FindTexture("albedo");
        videoMaterial.SetTextureImage(index, image); //actualiza los frames en la pantalla
        Log.Message("Frame remoto actualizado desde archivo.\n");
    }
    else
    {
        Log.Error("Falló la carga de textura remota.");
    }
}

3. try to avoid writing and reading file from disk at all - it can reduce perfimance and leed to spikes and low frame rate. (Cv2.ImEncode to write in buffer and Image.Load for create Unigine::Image from buffer)

Thanks!