RPC Multithreaded Server on Linux - Pthreads

Question

I am working on a RPC project for university. I have to implement a RPC multithreaded Server. I've been working on a server with only one procedure that sums two numbers. My code is at: https://github.com/alvmatias/pdytr

This is what i do to handle client requests

  /* Register RPC Service */
  /* serve client's requests asynchronously */
  while(1){
  rfds = svc_fdset;
  /* get max value that newly created file descriptor can have in "nfds" */
    switch (select(nfds, &rfds, NULL,NULL,NULL)){
        case -1:   
        case 0 :
            break;
        default: 
            /* Handle RPC request on each file descriptor */
            svc_getreqset(&rfds);
    }

Then i create the threads:

static void ej_prg_1(struct svc_req *rqstp, register SVCXPRT *transp)
{
    /* 2 threads possible for now */
    pthread_t th[2];
    pthread_attr_t attr[2];
    static int id=0;
    /* Used to pass arguments to "funcionA" */
    struct data_str{
        struct svc_req *rqstp;
        SVCXPRT *transp;
        int id;
    } *data_ptr=(struct data_str*)malloc(sizeof(struct data_str));
    /* Set parameters */
    data_ptr-> rqstp = rqstp;
    data_ptr-> transp = transp;
    data_ptr-> id = id;
    /* Create thread */
    printf("Crating thread %d\n", id);
    pthread_attr_init(&attr[id]);
    pthread_attr_setdetachstate(&attr[id],PTHREAD_CREATE_DETACHED);
    pthread_create(&th[id],&attr[id],&funcionA,(void *)data_ptr);
    printf("Thread %d created\n", id);
    id=(id+1)%2;
}

This is the function that each thread executes:

void *funcionA(void *data) {
    /* Structure for parameters */
    struct thr_data{
        struct svc_req *rqstp;
        SVCXPRT *transp;
        int id;
    } *ptr_data; 
    union {
        operands add_1_arg;
    } argument;
    union {
        int add_1_res;
    } result;
    bool_t retval;
    xdrproc_t _xdr_argument, _xdr_result;

    bool_t (*local)(char *, void *, struct svc_req *);

    /* Get the parameters */
    ptr_data = (struct thr_data  *)data;
    struct svc_req *rqstp = ptr_data-> rqstp;
    register SVCXPRT *transp = ptr_data-> transp;

    printf("Hello thread: %d\n", ptr_data-> id);

    /*Code generated by rpcgen */

    switch (rqstp->rq_proc) {
    case NULLPROC:
        (void) svc_sendreply (transp, (xdrproc_t) xdr_void, (char *)NULL);
        return;

    case ADD:
        _xdr_argument = (xdrproc_t) xdr_operands;
        _xdr_result = (xdrproc_t) xdr_int;
        local = (bool_t (*) (char *, void *,  struct svc_req *))add_1_svc;
        break;

    default:
        svcerr_noproc (transp);
        return;
    }

    memset ((char *)&argument, 0, sizeof (argument));
    if (!svc_getargs (transp, (xdrproc_t) _xdr_argument, (caddr_t) &argument)) {
        svcerr_decode (transp);
        return;
    }

    retval = (bool_t) (*local)((char *)&argument, (void *)&result, rqstp);
    if (retval > 0 && !svc_sendreply(transp, (xdrproc_t) _xdr_result, (char *)&result)) {
        svcerr_systemerr (transp);
    }
    if (!svc_freeargs (transp, (xdrproc_t) _xdr_argument, (caddr_t) &argument)) {
        fprintf (stderr, "%s", "unable to free arguments");
        exit (1);
    }
    if (!ej_prg_1_freeresult (transp, _xdr_result, (caddr_t) &result)){
        fprintf (stderr, "%s", "unable to free results");
    }
    /* End of rpcgen code */

    /* Exit thread */ /* Remember it's detached */
    printf("Bye thread: %d\n", ptr_data-> id);
    pthread_exit(0);
}

This is the remote procedure:

bool_t add_1_svc(operands *argp, int *p, struct svc_req *rqstp)
{
    bool_t result;
    printf("Got request: adding %d, %d\n", argp->x, argp->y);
    /* Add two numbers */
    *p=argp->x + argp->y;
    /* Execute a 4 seconds sleep to check multithreading */
    sleep(4);
    /* Return ok */
    result = 1;
    return (result);
}

My problem is that i execute two clients, for example:

./ej_client localhost 20 20 ./ej_client localhost 30 30

Both get 60 as answer and not 40 and 60 as it is expected. What's wrong with my program?

Looked at this question and the answer says that -M flag does generate the stubs but svc_calls are not MT-safe under linux. Does this mean multithreaded servers won't work? RPC can't decode arguments for TCP transport

EDIT: The problem seems to be here

if (retval > 0 && !svc_sendreply(transp, (xdrproc_t) _xdr_result, (char *)&result)) {
        svcerr_systemerr (transp);
    }

I think "transp" gets overwritten with the latest call. So, the client that makes the latest call receives an answer for a previous one. What can i do to solve this problem?

EDIT2: I've found this: In the current implementation, the service transport handle SVCXPRT contains a single data area for decoding arguments and encoding results. Therefore, this structure cannot be freely shared between threads that call functions that do this. Check: https://docs.oracle.com/cd/E19683-01/816-0214/6m6nf1p6o/index.html What can I do to make it MT safe,so it can be freely shared between multiple threads?

Thanks, Matias.

Without the rest of your code, I can't tell what's happening. However, I would strongly advise against typecasting as this could hide bugs. I don't see anywhere in your code you actually need casts. — MFisherKDX, Jan 22 '18 at 19:21
Full code is at https://github.com/alvmatias/pdytr. In which part of the code you say typecasting is probably making some trouble? — Matias, Jan 22 '18 at 19:23
I'm not saying any of the typecasts are causing trouble. I am saying don't do it. They aren't needed. None of those casts can help you -- only possibly hurt you. Especially casting the function pointer. If you incorrectly cast something to the wrong type, you will end up with undefined behavior ... possible crashes or subtle bugs. — MFisherKDX, Jan 22 '18 at 19:34
By the way, you should add the relevant portions of your multi-threaded code to the question as the problem is probably there. You likely have a race condition which is magnified by your `sleep(4)` statement in `add_1_svc` — MFisherKDX, Jan 22 '18 at 19:42
The sleep(4) is to force the threads to work at the same time. Otherwise, the first remote call returns before the second one is made. — Matias, Jan 22 '18 at 20:01

score 0 · Answer 1 · answered Apr 15 '18 at 18:01

svc_getargs() is not thread safe. this function copies the data from main rpc thread. when you call this function it copies the current request data, you want to call this from the callback function, copy the data to a struct and then use the data in the thread. Otherwise you might see some unpredictable results.

I would use rpcgen -MNC <.x file name> and then create a wrapper function, to call svc_getargs() to copy the current request data and then pass the data through struct variable to thread for processing

RPC Multithreaded Server on Linux - Pthreads

1 Answers1