0

I'm currently working on a project where I receive frames as bytearray and display them on the GUI (WPF). Right now im unsatisfied with the performance, because the frames are getting displayed delayed. So I thought about doing Multithreading, so that the socket and the GUI are working independent from each other. However If I try to run the socketRoutine in a seperate Thread I get an error, that a thread cant access the ressource of the other thread.

Im not sure which ressource is meant. I guess its either the bytearray im passing to the GUI or its the GUI component which has to be accessed.

These is my code right now.

namespace Subscriber_WPF{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
/// 

public partial class MainWindow : Window{

    /*Kinect Datentypen*/
    private WriteableBitmap colorBitmap = null;
    static readonly String publisherID = "TYPE1";

    SocketHandler socketHandler;
    Thread socketThread;

    public MainWindow(){     
        ConsoleManager.Show();
        colorBitmap = new WriteableBitmap(1920, 1080, 96.0, 96.0, PixelFormats.Bgra32, null);
        this.DataContext = this;

        //initializeKinectComponents();
       socketHandler = new SocketHandler(5555, publisherID, this);
       socketThread = new Thread(()=>socketHandler.runSocketRoutine());

        Console.WriteLine("GUI-Components initialized. Press Enter to start receiving Frames.");

        this.KeyDown += MainWindow_KeyDown;

    }

    private void MainWindow_KeyDown(object sender, KeyEventArgs e){
        if (e.Key.Equals(Key.Return)) {
            socketThread.Start();
        }
    }

    public void refreshGUI(byte[]content) {
       Action EmptyDelegate = delegate () { };
       BitmapSource source = BitmapSource.Create(1920, 1080, 72, 72, PixelFormats.Bgra32, BitmapPalettes.Gray256, content, 1920 * 4);
       videoView.Source = source;

        /*interrupt the socket-Loop to update GUI=> that is the current method without multithreading*/
       //this.Dispatcher.Invoke(DispatcherPriority.Render, EmptyDelegate);

    }


}

SocketHandler

namespace Subscriber_WPF{

class SocketHandler{

    private int port;
    private string publisherID;
    private MainWindow window;
    private static ZContext context;
    private static ZSocket subscriber;

    public SocketHandler(int port, string publisherID, MainWindow window) {
        this.port = port;
        this.publisherID = publisherID;
        this.window = window;
        this.initializeZMQSocket(this.port, this.publisherID);
    }

    private void initializeZMQSocket(int port, String publishID){
        context = new ZContext();
        subscriber = new ZSocket(context, ZSocketType.SUB);
        /*initialize sockets*/
        subscriber.Connect("tcp://127.0.0.1:" + port);
        subscriber.Subscribe(publishID);
        Console.WriteLine("subscriber is ready!");
    }


    public void runSocketRoutine(){
        Console.WriteLine("Waiting for Messages.");

        while (true)
        {
            byte[] content = new byte[8294400];

            using (ZMessage message = subscriber.ReceiveMessage())
            {
                Console.WriteLine("Message received!");
                string pubID = message[0].ReadString();
                /**/
                if (pubID.Equals(publisherID))
                {

                    content = message[1].Read();
                    Console.WriteLine("size of content: " + message[1].Length);
                    window.refreshGUI(content);
                }

            }
        }

    }


}

If someone has an idea on how to stop that delaying display or how i could easily handle that Thread-issue I would be very grateful!

Many Greets!

  • 1
    I guess you try to update the UI from another thread than the MainThread=UIThread. You have to use `Dispatcher.Invoke`http://stackoverflow.com/questions/4253088/updating-gui-wpf-using-a-different-thread – Mat Feb 13 '17 at 14:10
  • Possible duplicate of [Updating GUI (WPF) using a different thread](http://stackoverflow.com/questions/4253088/updating-gui-wpf-using-a-different-thread) – Mat Feb 13 '17 at 14:10
  • 3
    So you are regularily recreating a 1920 x 1080 bitmap for displaying and wonder about performance issues? Maybe you should find a different way to present your visual data. – grek40 Feb 13 '17 at 14:10
  • Maybe it's enough to `.Freeze()` your bitmap source after creation and before pushing it to the display thread. Can't test right now. – grek40 Feb 13 '17 at 14:13
  • @grek40 - it is 2017 by now. Creating a handful of 2MPx pictures per second shouldn't be too hard on a modern PC. On a phone it may be a little too much. – H H Feb 13 '17 at 14:13
  • As mentioned the UI thread might be running on a different thread. This might help explain the different ways to update the UI thead https://www.codeproject.com/Articles/52752/Updating-Your-Form-from-Another-Thread-without-Cre – jason.kaisersmith Feb 13 '17 at 14:13
  • Applications *don't* redraw the entire screen, since the 1980s. They use double buffering *at least* so they only have to swap bitmaps instead of drawing an entire bitmap on scren. – Panagiotis Kanavos Feb 13 '17 at 14:14
  • @HenkHolterman you're on... show me the part of question where this is limited to a hand ful per second. – grek40 Feb 13 '17 at 14:17
  • @HenkHolterman the reason that 2017 PCs have no issue with drawing is that they don't redraw the full screen. The eye is **very** sensitive to jitter, missed frames and redrawing. Windows Forms supports double buffering, but the coder still has to take care and invalidate only parts of the screen. WPF takes care of updating only the parts that correspond to changed bound data, and contains rudimentary hardware accelaration – Panagiotis Kanavos Feb 13 '17 at 14:17
  • @blackexpery - at a minimum the `videoView.Source = source;` should be invoked to the main thread. But I'm weary about that being a videoView. – H H Feb 13 '17 at 14:19
  • @PanagiotisKanavos - and yet again you seem to be seeing a very different question. – H H Feb 13 '17 at 14:20
  • @HenkHolterman `it is 2017 by now. Creating a handful of 2MPx pictures per second shouldn't be too hard`. Actually this *is* a hard problem, not related to a machine's power. Kevin Ross's answer is a very good starting point – Panagiotis Kanavos Feb 13 '17 at 14:21
  • On a side-note, get rid of that `= new byte[8294400]` part. – H H Feb 13 '17 at 14:23
  • @Mat Yes that was the missing link. Thanks! – BlackExupery Feb 16 '17 at 12:12

1 Answers1

2

I have done a similar thing in the past using the excellent RX framework. Your run socket routine becomes and IObservable where T is the type you are returning, in your case byte[]. You can then subscribe to this stream on the UI thread. Using the RX framework you can also do cool things like rate limiting so if your routine spits out lots of "records" you can limit that to 1 per timespan.

Kevin Ross
  • 7,185
  • 2
  • 21
  • 27
  • This won't solve the redrawing issue. It will make redrawing smoother but it will still take a lot of CPU. – Panagiotis Kanavos Feb 13 '17 at 14:19
  • 1
    True it would still be asking a lot of the CPU but by rate limiting the subscription you can find a good compromise between CPU usage and visual redraw speed – Kevin Ross Feb 13 '17 at 14:22
  • @HenkHolterman sorry, when you've worked with near-real time GIS systems, you get to know all the problems tied with specific implementations, like tight uncontrolled loops, redrawing, flickering, etc. – Panagiotis Kanavos Feb 13 '17 at 14:29
  • For example jitter - you'd need Rx to smooth frames even if repainting didn't cost anything. That's because packets will have random lag *and* the UI thread can be preempted at any point. – Panagiotis Kanavos Feb 13 '17 at 14:31