Polling a website using Rx

Question

just trying to get my head around Rx

I am using Rx to poll a website every 2 seconds

var results = new List<MyDTO>();
var cx = new WebserviceAPI( ... );
var callback = cx.GetDataAsync().Subscribe(rs => { results.AddRange(rs); });
var poller = Observable.Interval(TimeSpan.FromSeconds(2)).Subscribe( _ => { cx.StartGetDataAsync(); });

(The webservice API exposes a getItemsAsync/getItemsCompleted event handler type mechanism from which I am creating an observable).

When the web site returns, I am unpacking the "business part of" the response into an IEnumerable of DTOs

public IObservable<IEnumerable<MyDTO>> GetDataAsync()
{
    var o = Observable.FromEventPattern<getItemsCompletedEventHandler,getItemsCompletedEventArgs>(
        h => _webService.getItemsCompleted += h,
        h => _webService.getItemsCompleted -= h);

    return o.Select(c=> from itm in c.EventArgs.Result.ItemList
                        select new MyDTO()
                        {
                           ...
                        });
}

My reasoning being that given that all the data was just there in the string, it made sense just to pack it up there an then into an IEnumerable ... but now I'm not sure if that is right!

If the website takes longer than 2 secs to respond I am finding that MSTest is crashing out. When debugging, the error being generated is

"There was an error during asynchronous processing. Unique state object is required for multiple asynchronous simultaneous operations to be outstanding"

with the inner exception

"Item has already been added. Key in dictionary: 'System.Object' Key being added: 'System.Object'"

I am supposing that the problem is one of reentrancy in that the next call is starting and returning data before the previous call has finished populating the data.

So I'm not sure whether

1. I have put the thing together quite right
2. I should be throttling the connection in some way so as to avoid re-entrancy.
3. I should use a different intermediate data structure (or mechanism) instead of an IEnumerable

I would appreciate some guidance.

EDIT 1: So I have changed the web call to include a unique state object

public void StartGetDataAsync()
{
    ...
    //  was: _webService.getItemsAsync(request);
    _webService.getItemsAsync(request, Guid.NewGuid());
}

and made it work. But I am still unsure if that is the right way to do it

EDIT 2 - Web service sigs: I'm consuming a soap web service which the webServiceApi class wraps. The references.cs created contains the following methods

public void getItemsAsync(GetItemsReq request, object userState) 
{
    if ((this.getItemsOperationCompleted == null)) 
    {
        this.getItemsOperationCompleted = new System.Threading.SendOrPostCallback(this.OngetItemsOperationCompleted);
    }
    this.InvokeAsync("getItems", new object[] {
                    request}, this.getItemsOperationCompleted, userState);
}

private System.Threading.SendOrPostCallback getItemsOperationCompleted;

public event getItemsCompletedEventHandler getItemsCompleted;

public delegate void getItemsCompletedEventHandler(object sender, getItemsCompletedEventArgs e);

public partial class getItemsCompletedEventArgs : System.ComponentModel.AsyncCompletedEventArgs 
{
    ...
}

private void OngetItemsOperationCompleted(object arg) 
{
    if ((this.getItemsCompleted != null)) 
    {
        System.Web.Services.Protocols.InvokeCompletedEventArgs invokeArgs = ((System.Web.Services.Protocols.InvokeCompletedEventArgs)(arg));
        this.getItemsCompleted(this, new getItemsCompletedEventArgs(invokeArgs.Results, invokeArgs.Error, invokeArgs.Cancelled, invokeArgs.UserState));
    }
 }

Probably given you too much (or missed something)!

Thx

Answer 1

I think I've got a decent starting point for you.

Basically I think you need to abstract away the complexity of the web service and create a nice clean function to get your results.

Try something like this:

Func<GetItemsReq, IObservable<getItemsCompletedEventArgs>> fetch =
    rq =>
        Observable.Create<getItemsCompletedEventArgs>(o =>
        {
            var cx = new WebserviceAPI(/* ... */);
            var state = new object();
            var res =
                Observable
                    .FromEventPattern<
                        getItemsCompletedEventHandler,
                        getItemsCompletedEventArgs>(
                        h => cx.getItemsCompleted += h,
                        h => cx.getItemsCompleted -= h)
                    .Where(x => x.EventArgs.UserState == state)
                    .Take(1)
                    .Select(x => x.EventArgs);
            var subscription = res.Subscribe(o);
            cx.getItemsAsync(rq, state);
            return subscription;
        });

Personally I would go one step further and define a return type, say GetItemsReq, that doesn't include the user state object, but is basically the same as getItemsCompletedEventArgs.

You should then be able to use Observable.Interval to create the polling that you need.

If your web service implements IDisposable then you should add an Observable.Using call into the above function to correctly dispose of the web service when it is complete.

Let me know if this helps.

Answer 2

Enigmativity has a good solution.

          var state = new object();

          cx.getItemsAsync(rq, state);

These statements are the key to solving the error.

Each time an Async method is called, it needs to have a unique object passed to it (unless of course only one Async method will be running at a time, which is highly unlikely). I scratched my head over this for hours until I discovered that you can pass an object parameter. Without the parameter, it will see multiple methods as the same call and give you that "Item has already been added. Key in dictionary: 'System.Object' Key being added: 'System.Object'" message.