How do I use OAuth to connect to the Etrade API?

Question

E-Trade released their API recently and provided technical documentation which is somewhat useful but not complete.

Does anyone have a fully working example in C# that shows how this works?

I have been able to do the authentication using OAuth correctly, but when it comes to getting information out of my account or market data, the servers fail.

Answer 1

I was able to connect using the DevDefined OAuth Library, but i had to make some tweeks to the source to get it to work properly. I forked the repo so you can download the src i used, and build you a .dll.

Repo: GitHub

Example Class:

 public abstract class BaseOAuthRepository
{

    private static string REQUEST_URL = "https://etws.etrade.com/oauth/request_token";
    private static string AUTHORIZE_URL = "https://us.etrade.com/e/t/etws/authorize";
    private static string ACCESS_URL = "https://etws.etrade.com/oauth/access_token";

    private readonly TokenBase _tokenBase;
    private readonly string _consumerSecret;

    protected BaseOAuthRepository(TokenBase tokenBase, 
                                  string consumerSecret)
    {
        _tokenBase = tokenBase;
        _consumerSecret = consumerSecret;
    }

    public TokenBase MyTokenBase
    {
        get { return _tokenBase; }
    }

    public string MyConsumerSecret
    {
        get { return _consumerSecret; }
    }


    public OAuthSession CreateSession()
    {
        var consumerContext = new OAuthConsumerContext
        {
            ConsumerKey = MyTokenBase.ConsumerKey,
            ConsumerSecret = MyConsumerSecret,
            SignatureMethod = SignatureMethod.HmacSha1,
            UseHeaderForOAuthParameters = true,
            CallBack = "oob"
        };

        var session = new OAuthSession(consumerContext, REQUEST_URL, AUTHORIZE_URL, ACCESS_URL);    
        return session;
    }

    public IToken GetAccessToken(OAuthSession session)
    {
        IToken requestToken = session.GetRequestToken();
        string authorizationLink = session.GetUserAuthorizationUrlForToken(MyTokenBase.ConsumerKey, requestToken);
        Process.Start(authorizationLink);
        Console.Write("Please enter pin from browser: ");
        string pin = Console.ReadLine();
        IToken accessToken = session.ExchangeRequestTokenForAccessToken(requestToken, pin.ToUpper());

        return accessToken;
    }

    public string GetResponse(OAuthSession session, string url)
    {
        IToken accessToken = MyTokenBase;

        var response = session.Request(accessToken).Get().ForUrl(url).ToString();
        return response;
    }

    public XDocument GetWebResponseAsXml(HttpWebResponse response)
    {
        XmlReader xmlReader = XmlReader.Create(response.GetResponseStream());
        XDocument xdoc = XDocument.Load(xmlReader);
        xmlReader.Close();
        return xdoc;
    }

    public string GetWebResponseAsString(HttpWebResponse response)
    {
        Encoding enc = System.Text.Encoding.GetEncoding(1252);
        StreamReader loResponseStream = new
        StreamReader(response.GetResponseStream(), enc);
        return loResponseStream.ReadToEnd();
    }
}

Answer 2

Here's the code I've used to connect to the ETrade API (tested and works).

One caveat: You need to implement your own storage of user tokens. I've not included that here since the code I created is highly domain specific.

First, I added DotNetOpenAuth to the project and created an ETradeConsumer (it derives from DotNetOpenAuth's WebConsumer):

EtradeConsumer.cs

public static class ETradeConsumer
{
    public static string AccessUrl 
    { 
        get 
        { 
            return "https://etws.etrade.com/oauth/access_token"; 
        } 
    }

    public static string RequestUrl 
    { 
        get 
        { 
            return "https://etws.etrade.com/oauth/request_token"; 
        } 
    }

public static string UserAuthorizedUrl 
    {  
        get 
        { 
            return "https://us.etrade.com/e/t/etws/authorize"; 
        } 
    }
private static readonly ServiceProviderDescription ServiceProviderDescription = new ServiceProviderDescription()
{
    AccessTokenEndpoint = new MessageReceivingEndpoint(AccessUrl, HttpDeliveryMethods.PostRequest | HttpDeliveryMethods.AuthorizationHeaderRequest),
    ProtocolVersion = ProtocolVersion.V10a,
    RequestTokenEndpoint = new MessageReceivingEndpoint(RequestUrl, HttpDeliveryMethods.PostRequest | HttpDeliveryMethods.AuthorizationHeaderRequest),
    TamperProtectionElements = new ITamperProtectionChannelBindingElement[] { new HmacSha1SigningBindingElement() },
    UserAuthorizationEndpoint = new MessageReceivingEndpoint(new Uri(UserAuthorizedUrl), HttpDeliveryMethods.GetRequest | HttpDeliveryMethods.AuthorizationHeaderRequest)
};

public static DesktopConsumer CreateConsumer(IConsumerTokenManager tokenManager)
{
    return new DesktopConsumer(ServiceProviderDescription, tokenManager);
}

public static Uri PrepareRequestAuthorization(DesktopConsumer consumer, out string requestToken)
{
    if (consumer == null)
    {
        throw new ArgumentNullException("consumer");
    }

        Uri authorizationUrl = consumer.RequestUserAuthorization(null, null, out requestToken);

    authorizationUrl = new Uri(string.Format("{0}?key={1}&token={2}", ServiceProviderDescription.UserAuthorizationEndpoint.Location.AbsoluteUri, consumer.TokenManager.ConsumerKey, requestToken));
    return authorizationUrl;
}

public static AuthorizedTokenResponse CompleteAuthorization(DesktopConsumer consumer, string requestToken, string userCode)
    {
    var customServiceDescription = new ServiceProviderDescription
    {
            RequestTokenEndpoint = ServiceProviderDescription.RequestTokenEndpoint,
        UserAuthorizationEndpoint =
                new MessageReceivingEndpoint(
                string.Format("{0}?key={1}&token={2}",  ServiceProviderDescription.UserAuthorizationEndpoint.Location.AbsoluteUri,
                              consumer.TokenManager.ConsumerKey, requestToken),
                HttpDeliveryMethods.AuthorizationHeaderRequest | HttpDeliveryMethods.GetRequest),
        AccessTokenEndpoint = new MessageReceivingEndpoint(
        ServiceProviderDescription.AccessTokenEndpoint.Location.AbsoluteUri + "?oauth_verifier" + userCode + string.Empty,
                HttpDeliveryMethods.AuthorizationHeaderRequest | HttpDeliveryMethods.GetRequest),
        TamperProtectionElements = ServiceProviderDescription.TamperProtectionElements,
        ProtocolVersion = ProtocolVersion.V10a
    };

    var customConsumer = new DesktopConsumer(customServiceDescription, consumer.TokenManager);
    var response = customConsumer.ProcessUserAuthorization(requestToken, userCode);
    return response;
    }

}

Secondly, you need to create a class to manage Etrade tokens. As an example, I created the following class. It manages the tokens through an InMemoryCollection, but it really should be held somewhere else (a database, or a cookie, or something so that the user doesn't have to authenticate/authorize every single time). The ConsumerKey and ConsumerSecret tokens are things you sign up for through Etrade:

public class ETradeTokenManager : IConsumerTokenManager 
{
    private Dictionary<string, string> tokensAndSecrets = new Dictionary<string, string>();
public string ConsumerKey { get { return "YourConsumerKey"; } }
public string ConsumerSecret { get { return "YourConsumerSecret";  } }
    
public string GetTokenSecret(string token)
{
    return tokensAndSecrets[token];
}

public void StoreNewRequestToken(UnauthorizedTokenRequest request, ITokenSecretContainingMessage response)
{
    tokensAndSecrets[response.Token] = response.TokenSecret;
}

public void ExpireRequestTokenAndStoreNewAccessToken(string consumerKey, string requestToken, string accessToken, string accessTokenSecret)
{
    tokensAndSecrets.Remove(requestToken);
    tokensAndSecrets[accessToken] = accessTokenSecret;
}

public TokenType GetTokenType(string token)
{
    throw new NotImplementedException();
}
}

Finally, put the following in (I used ASP.NET MVC 3. Your framework may differ):

public ActionResult EtradeAuthorize(string returnUrl)
   {
        var consumer = ETradeConsumer.CreateConsumer(TokenManager);
    string requestToken;
    Uri popupWindow = ETradeConsumer.PrepareRequestAuthorization(consumer, out requestToken);
    var etradeViewModel = new ETradeAuthorizeViewModel(popupWindow, requestToken);
    return View(etradeViewModel);
    }

    [HttpPost]
    public ActionResult CompleteAuthorization(FormCollection formCollection)
    {
    string accessToken = "";
    var consumer = ETradeConsumer.CreateConsumer(TokenManager);
    var authorizationReponse = ETradeConsumer.CompleteAuthorization(consumer, formCollection["requestToken"], formCollection["userCode"]);
    if (authorizationReponse != null)
    {
        accessToken = authorizationReponse.AccessToken;
    }
    var etradeViewModel = new ETradeCompleteAuthorizeViewModel(formCollection["requestToken"], formCollection["userCode"], accessToken);
    return View(etradeViewModel);
    }

If you get a 400 Bad Request, take out the callbackUrl for Etrade. For some reason it throws a bad request whenever a callback URL is used. They prefer oob (Out of Band). In order to use oob, set null to the Callback URL in the Consumer.Channel.Send() method.

There are other issues. This issue: Due to a logon delay or other issue, your authentication could not be completed at this time. Please try again. is caused by the authorize portion of the call not being processed properly. Specifically, Etrade requires that the authorize URL looks as follows:

https://us.etrade.com/e/t/etws/authorize?key={yourConsumerKey}&token={requestToken}

The OAuth specification requires that the request token should be request_token={requestToken} and not token={requestToken}.

I couldn't get the Etrade API to authorize correctly with the WebConsumer but once I switched to the Desktop Consumer and manipulated the request myself, it worked correctly.

Answer 3

If you get "Due to a logon delay or other issue, your authentication could not be completed at this time. Please try again.",

then I think you put wrong keys in the authorize url.

You should follow the document by replacing the appropriate keys in this format

https://us.etrade.com/e/etws/authorize?key=&token=

Answer 4

To use the example class + GitHub's code from jejernig's answer, I used the following:

TokenBase token = new TokenBase { ConsumerKey = "oauth_consumer_key from ETRADE" }; // OAuthRepository only seems to use the consumer key
OAuthRepository rep = new OAuthRepository(token, "consumer_secret from ETRADE");
OAuthSession session = rep.CreateSession();
IToken accessToken = rep.GetAccessToken(session);

I just removed the abstract from BaseOAuthRepository, and had to fix the GitHub code because IOAuthSession.GetUserAuthorizationUrlForToken()'s parameters were reversed (I changed the rest of the code to match the interface's parameters).

I am getting the dreaded Due to a logon delay or other issue, your authentication could not be completed at this time. Please try again. message, but this may be due to an actual logon issue that I have to resolve.

Answer 5

Clear cookies and try again. I am not sure, why this happens. But, once you get this error, unless otherwise you clear the cookies, you will get the same error. I am able to login successfully and call some of the REST services.