how do I convert an IplImage to a standard web image format to display in url?

Question

so i understand the basics of socket programming however what i get when i go to localhost:8888 are unreadable strings instead of video or image frames. i was wondering if anyone had any pointer as to how to display images on my port rather than strings. the code i have is below:

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

void* streamServer(void* arg);
void  quit(char* msg, int retval);

int main(int argc, char** argv)
{
    pthread_t   thread_s;
    int         key;

    if (argc == 2) {
        capture = cvCaptureFromFile(argv[1]);
    } else {
        capture = cvCaptureFromCAM(0);
    }

    if (!capture) {
        quit("cvCapture failed", 1);
    }

    img0 = cvQueryFrame(capture);
    img1 = cvCreateImage(cvGetSize(img0), IPL_DEPTH_8U, 1);

    cvZero(img1);
    cvNamedWindow("stream_server", CV_WINDOW_AUTOSIZE);

    /* print the width and height of the frame, needed by the client */
    fprintf(stdout, "width:  %d\nheight: %d\n\n", img0->width, img0->height);
    fprintf(stdout, "Press 'q' to quit.\n\n");

    /* run the streaming server as a separate thread */
    if (pthread_create(&thread_s, NULL, streamServer, NULL)) {
        quit("pthread_create failed.", 1);
    }

    while(key != 'q') {
        /* get a frame from camera */
        img0 = cvQueryFrame(capture);
        if (!img0) break;

        img0->origin = 0;
        cvFlip(img0, img0, 1); //-1);

        /**
         * convert to grayscale
         * note that the grayscaled image is the image to be sent to the client
         * so we enclose it with pthread_mutex_lock to make it thread safe
         */
        pthread_mutex_lock(&mutex);
        cvCvtColor(img0, img1, CV_BGR2GRAY);
        is_data_ready = 1;
        pthread_mutex_unlock(&mutex);

        /* also display the video here on server */
        cvShowImage("stream_server", img0);
        key = cvWaitKey(30);
    }

    /* user has pressed 'q', terminate the streaming server */
    if (pthread_cancel(thread_s)) {
        quit("pthread_cancel failed.", 1);
    }

    /* free memory */
    cvDestroyWindow("stream_server");
    quit(NULL, 0);
}

/**
 * This is the streaming server, run as a separate thread
 * This function waits for a client to connect, and send the grayscaled images
 */
void* streamServer(void* arg)
{
    struct sockaddr_in server;

    /* make this thread cancellable using pthread_cancel() */
    pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
    pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);

    /* open socket */
    if ((serversock = socket(PF_INET, SOCK_STREAM, 0)) == -1) {
        quit("socket() failed", 1);
    }

    /* setup server's IP and port */
    memset(&server, 0, sizeof(server));
    server.sin_family = AF_INET;
    server.sin_port = htons(PORT);
    server.sin_addr.s_addr = INADDR_ANY;

    /* bind the socket */
    if (bind(serversock, (const void*)&server, sizeof(server)) == -1) {
        quit("bind() failed", 1);
    }

    /* wait for connection */
    if (listen(serversock, 10) == -1) {
        quit("listen() failed.", 1);
    }

    /* accept a client */
    if ((clientsock = accept(serversock, NULL, NULL)) == -1) {
        quit("accept() failed", 1);
    }

    /* the size of the data to be sent */
    int imgsize = img1->imageSize;
    int bytes, i;

    /* start sending images */
    while(1)
    {
        /* send the grayscaled frame, thread safe */
        pthread_mutex_lock(&mutex);
        if (is_data_ready) {
            bytes = send(clientsock, img1->imageData, imgsize, 0);
            is_data_ready = 0;
        }
        pthread_mutex_unlock(&mutex);

        /* if something went wrong, restart the connection */
        if (bytes != imgsize) {
            fprintf(stderr, "Connection closed.\n");
            close(clientsock);

            if ((clientsock = accept(serversock, NULL, NULL)) == -1) {
                quit("accept() failed", 1);
            }
        }

the whole code: http://pastebin.com/jnuZhx8C

Answer 1

Unless img1->imageData is in a format the browser is supposed to recognize (png, jpg, gif), unrecognizable text is all you should expect.

Answer 2

Before we go into the technical details, ask yourself the question: what is standard web image format?

In your opinion, would it be PNG? JPG? TIFF? BMP? GIF? All of these?

An image format like that stores much more than just pixels. If you take a look at the specification of PNG, you'll see that it stores a lot of information about the image besides the pixels.

The problem is that when you send just the pixels over the network, like you are doing now, whoever is receiving the data on the other side of the connection can't do anything useful with it because it doesn't even know what's the dimension of the image.

The other misconception we need to clear is: there's no such thing as display images on my port rather than strings. What is a port?

Transferring data across networks is one thing. Doing something useful with the data (like displaying it on the screen) on the other side of the connection is an entirely different task.

If you need a practical example on how to accomplish these tasks, I suggest you take a look at this awesome tutorial: Streaming OpenCV Videos Over the Network. I just noticed that the code from your question came from there, so pay extra attention to section 4. Implementation of the Client-side.