Federated authentication in Sharepoint 2013: getting rtFa and FedAuth cookies

Question

The scenario is the following: I need to perform a federated authentication of a user (which uses his university account) into the Sharepoint site of his university and to obtain both the FedAuth and rtFa cookies (which I have to pass to SharePoint REST webservices in order to access resources).

I made some attempts but there is at least an issue in each one:

1) Using Microsoft.SharePoint.Client library

ClientContext context = new ClientContext(host);
SharePointOnlineCredentials creds = new SharePointOnlineCredentials(user, passw);
context.Credentials = creds;

Uri sharepointuri = new Uri(host);
string authCookie = creds.GetAuthenticationCookie(sharepointuri);

Web web = context.Web;
context.Load(web, w=>w.Lists);
context.ExecuteQuery();

fedAuthString = authCookie.Replace("SPOIDCRL=", string.Empty);

This way I manage to get the FedAuth cookie but I am unable to get the rtFa cookie.

How can I get the rtFa cookie at this point? Can I intercept the HTTP request involved in such an operation (i.e., context.ExecuteQuery()) -- which presumably contains the rtFa cookie in the headers? Or, can I get the rtFa cookie by only leveraging on the FedAuth cookie?

2) Using MsOnlineClaimsHelper

This is a helper class which can be found on the Internet (e.g., here http://blog.kloud.com.au/tag/msonlineclaimshelper/ ).

This class, as it is, works with normal authentication but fails with federated authentication.

So I adjusted it in order to make it work in this case. As long as I understand, the steps are the following:

1. Authenticate using username and password to the STS ADFS service of the university (the "federated party" or the ISSUER) -- here the Relying Party is Sharepoint O365 STS ("https://login.microsoftonline.com/extSTS.srf")
2. If the auth succeeds, I get back a SAML assertion containing the claims and a security token
3. Now, I authenticate to the SharePoint site by passing the Security Token
4. If the token is recognized, I get back a response which contains the two cookies (FedAuth and rtFa)

I am not an expert in this matter, and I came out with the following code:

This is the code that calls the method above and try to get FedAuth and rtFa from credentials in two steps (step 1: get SAML token from Federated Party; step 2: pass token from Federated Party to Sharepoint):

     private List<string> GetCookies(){
            // 1: GET SAML XML FROM FEDERATED PARTY THE USER BELONGS TO
            string samlToken = getResponse_Federation(sts: "https://sts.FEDERATEDDOMAIN.com/adfs/services/trust/13/usernamemixed/",
                realm: "https://login.microsoftonline.com/extSTS.srf");

            // 2: PARSE THE SAML ASSERTION INTO A TOKEN 
            var handlers = FederatedAuthentication.ServiceConfiguration.SecurityTokenHandlers;
            SecurityToken token = handlers.ReadToken(new XmlTextReader(new StringReader(samlToken )));

            // 3: REQUEST A NEW TOKEN BASED ON THE ISSUED TOKEN
            GenericXmlSecurityToken secToken = GetO365BinaryTokenFromToken(token);

            // 4: NOW, EASY: I PARSE THE TOKEN AND EXTRACT FEDAUTH and RTFA
            ...............
    }


    private string getResponse_Federation(string stsUrl, string relyingPartyAddress)
    {
        var binding = new Microsoft.IdentityModel.Protocols.WSTrust.Bindings.UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential);
        binding.ClientCredentialType = HttpClientCredentialType.None;

        var factory = new WSTrustChannelFactory(binding,  stsUrl);

        factory.Credentials.UserName.UserName = "username";
        factory.Credentials.UserName.Password = "password";
        factory.Credentials.SupportInteractive = false;
        factory.TrustVersion = TrustVersion.WSTrust13;

        IWSTrustChannelContract channel = null;
        try
        {
            var rst = new RequestSecurityToken
            {
                RequestType = WSTrust13Constants.RequestTypes.Issue,
                AppliesTo = new EndpointAddress(relyingPartyAddress), //("urn:sharepoint:MYFEDERATEDPARTY"),
                ReplyTo = relyingPartyAddress,
                KeyType = WSTrust13Constants.KeyTypes.Bearer,
                TokenType =  "http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0",
                RequestDisplayToken = true,
            };
            channel = (WSTrustChannel)factory.CreateChannel();

            RequestSecurityTokenResponse response = null;
            SecurityToken st = channel.Issue(rst, out response);
            var genericToken = st as GenericXmlSecurityToken;
            return genericToken.TokenXml.OuterXml;
        }
        catch (Exception e)
        {
            return null;
        }
    }

    private GenericXmlSecurityToken GetO365BinaryTokenFromToken(SecurityToken issuedToken)
    {
        Uri u = new Uri("https://login.microsoftonline.com/extSTS.srf");

        WSHttpBinding binding = new WSHttpBinding(SecurityMode.TransportWithMessageCredential);
        binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.None;
        binding.Security.Mode = SecurityMode.TransportWithMessageCredential;
        binding.Security.Message.ClientCredentialType = MessageCredentialType.IssuedToken;

        Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannelFactory channel =
        new Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannelFactory(
            binding, new EndpointAddress("https://login.microsoftonline.com/extSTS.srf"));

        channel.TrustVersion = TrustVersion.WSTrust13;
        channel.Credentials.SupportInteractive = false;

        GenericXmlSecurityToken token = null;

        try
        {
            RequestSecurityToken rst = new RequestSecurityToken(WSTrust13Constants.RequestTypes.Issue, WSTrust13Constants.KeyTypes.Bearer)
            {
            };
            rst.AppliesTo = new EndpointAddress("urn:sharepoint:MYFEDERATEDPARTY");
            channel.ConfigureChannelFactory();
            var chan = (Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel)channel.CreateChannelWithIssuedToken(issuedToken);

            RequestSecurityTokenResponse rstr = null;

            token = chan.Issue(rst, out rstr) as GenericXmlSecurityToken;

            return token;
        }
        catch (Exception ex){
            Trace.TraceWarning("WebException in getO365BinaryTokenFromADFS: " + ex.ToString());
            throw;
        }
    }

I managed to get back a SAML token from the university STS. However, when parsed, the resulting SecurityToken has no security keys (i.e., the SecurityKeys collection is empty)

With no keys, I get on GetO365BinaryTokenFromToken() but when I try to send the token to the SharePoint Authentication service -- I get the following error: "The signing token Microsoft.IdentityModel.Tokens.Saml2.Saml2SecurityToken has no keys. The security token is used in a context that requires it to perform cryptographic operations, but the token contains no cryptographic keys. Either the token type does not support cryptographic operations, or the particular token instance does not contain cryptographic keys. Check your configuration to ensure that cryptographically disabled token types (for example, UserNameSecurityToken) are not specified in a context that requires cryptographic operations (for example, an endorsing supporting token)."

I think that there are also some configuration issues that I cannot control directly, on both sides (the university STS ADFS and the Sharepoint STS).

I hope that more expert people would bring clarity in this process and even provide advice to actually make this scenario work.

File download function

With the following function, I am able to download a file (given an URL such as https://myfederatedparty.sharepoint.com/sites/MYSITE/path/myfile.pdf) by issuing BOTH the FedAuth and the rtFa cookie. If I do not pass the rtFa cookie, I get an "Unauthorized" response.

    public static async Task<byte[]> TryRawWsCall(String url, string fedauth, string rtfa, CancellationToken ct, TimeSpan? timeout = null) {
        try {
            HttpClientHandler handler = new HttpClientHandler();
            handler.CookieContainer = new System.Net.CookieContainer();
            CookieCollection cc = new CookieCollection();
            cc.Add(new Cookie("FedAuth", fedauth));
            cc.Add(new Cookie("rtFa", rtfa));
            handler.CookieContainer.Add(new Uri(url), cc);

            HttpClient _client = new HttpClient(handler);
            if (timeout.HasValue)
                _client.Timeout = timeout.Value;
            ct.ThrowIfCancellationRequested();

            var resp = await _client.GetAsync(url);
            var result = await resp.Content.ReadAsByteArrayAsync();
            if (!resp.IsSuccessStatusCode)
                return null;
            return result;
        }
        catch (Exception) { return null; }
    }

Answer 1

In fact, only FedAuth cookie is mandatory when it comes to SharePoint Online/Office 365 authentication.

According to Remote Authentication in SharePoint Online Using Claims-Based Authentication:

The FedAuth cookies enable federated authorization, and the rtFA cookie enables signing out the user from all SharePoint sites, even if the sign-out process starts from a non-SharePoint site.

So, it is enough to provide SPOIDCRL HTTP header in order to perform authentication in SharePoint Online/Office 365, for example:

var request = (HttpWebRequest)WebRequest.Create(endpointUri);
var credentials = new SharePointOnlineCredentials(userName,securePassword);
var authCookie = credentials.GetAuthenticationCookie(webUri);
request.Headers.Add(HttpRequestHeader.Cookie, authCookie);

---

The following examples demonstrates how to perform active authentication in SharePointOnline/Office 365 by providing FedAuth cookie.

Example 1: Retrieve FormDigest via SharePoint 2013 REST API (uisng MsOnlineClaimsHelper class)

public static string GetFormDigest(Uri webUri, string userName, string password)
{
   var claimsHelper = new MsOnlineClaimsHelper(webUri, userName, password);
   var endpointUri = new Uri(webUri,"/_api/contextinfo");
   var request = (HttpWebRequest)WebRequest.Create(endpointUri);
   request.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");
   request.Method = WebRequestMethods.Http.Post;
   request.Accept = "application/json;odata=verbose";
   request.ContentType = "application/json;odata=verbose";
   request.ContentLength = 0;

   var fedAuthCookie = claimsHelper.CookieContainer.GetCookieHeader(webUri); //FedAuth are getting here
   request.Headers.Add(HttpRequestHeader.Cookie, fedAuthCookie); //only FedAuth cookie are provided here
   //request.CookieContainer = claimsHelper.CookieContainer;
   using (var response = (HttpWebResponse) request.GetResponse())
   {
        using (var streamReader = new StreamReader(response.GetResponseStream()))
        {
                var content = streamReader.ReadToEnd();
                var t = JToken.Parse(content);
                return t["d"]["GetContextWebInformation"]["FormDigestValue"].ToString();
        }     
    }
}

Example 2: Retrieve FormDigest via SharePoint 2013 REST API (using SharePointOnlineCredentials class)

public static string GetFormDigest(Uri webUri, string userName, string password)
{
   var endpointUri = new Uri(webUri, "/_api/contextinfo");
   var request = (HttpWebRequest)WebRequest.Create(endpointUri);
   request.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");
   request.Method = WebRequestMethods.Http.Post;
   request.Accept = "application/json;odata=verbose";
   request.ContentType = "application/json;odata=verbose";
   request.ContentLength = 0;

   var securePassword = new SecureString();
   foreach (char c in password)
   {
       securePassword.AppendChar(c);
   }
   request.Credentials = new SharePointOnlineCredentials(userName,securePassword);

   using (var response = (HttpWebResponse)request.GetResponse())
   {
       using (var streamReader = new StreamReader(response.GetResponseStream()))
       {
           var content = streamReader.ReadToEnd();
           var t = JToken.Parse(content);
           return t["d"]["GetContextWebInformation"]["FormDigestValue"].ToString();
        }
   }
}

---

Update

The modified version of the example for downloading a file:

public static async Task<byte[]> DownloadFile(Uri webUri,string userName,string password, string relativeFileUrl, CancellationToken ct, TimeSpan? timeout = null)
{
        try
        {

            var securePassword = new SecureString();
            foreach (var c in password)
            {
                securePassword.AppendChar(c);
            }
            var credentials = new SharePointOnlineCredentials(userName, securePassword);
            var authCookie = credentials.GetAuthenticationCookie(webUri);
            var fedAuthString = authCookie.TrimStart("SPOIDCRL=".ToCharArray());
            var cookieContainer = new CookieContainer();
            cookieContainer.Add(webUri, new Cookie("SPOIDCRL", fedAuthString));


            HttpClientHandler handler = new HttpClientHandler();
            handler.CookieContainer = cookieContainer;

            HttpClient _client = new HttpClient(handler);
            if (timeout.HasValue)
                _client.Timeout = timeout.Value;
            ct.ThrowIfCancellationRequested();

            var fileUrl = new Uri(webUri, relativeFileUrl);
            var resp = await _client.GetAsync(fileUrl);
            var result = await resp.Content.ReadAsByteArrayAsync();
            if (!resp.IsSuccessStatusCode)
                return null;
            return result;
        }
        catch (Exception) { return null; }
 }

Answer 2

This is what I did. Might be useful to future readers!

For my usecase, I am running a WinForms Windows client side app. I was able to grab the FedAuth and rtFA cookie using a embedded WebBrowser Control.

I have uploaded my sample test project to github here: https://github.com/OceanAirdrop/SharePointOnlineGetFedAuthAndRtfaCookie

Here is what I did:

Step 01: Naviagte to SharePoint Url in WebBrowser Control

Using the WebBrowser control, first navigate to a web page on your sharepoint site that you have access to. It can be any page. The aim here is to get the cookies from the loaded page. This step only needs to be done once in the app.

webBrowser1.Navigate(@"https://xx.sharepoint.com/sites/xx/Forms/AllItems.aspx");

Step 02: Grab Cookies from WebBrowser Control

Next, override the Navigated event in the WebBrowser control. This lets you know the page has fully loaded.

Now, heres the wrinkle!! The FedAuth cookies are written with an HTTPOnly flag, which means they cannot be accessed from the .NET Framework. This means if you try to access the cookies of the WebBrowser control, you will get null string back!

// This line of code wont work and will return null
var cookies = webBrowser1.Document.Cookie;

So, to get around this, you instead need to call InternetGetCookieEx in the WININET.dll. I took the code from here. This is what the Navigated function handler looks like:

private void webBrowser1_Navigated(object sender, WebBrowserNavigatedEventArgs e)
{
    try
    {
        if (webBrowser1.Url.AbsoluteUri == "about:blank")
            return;

        // This line calls through to InternetGetCookieEx
        var cookieData = GetWebBrowserCookie.GetCookieInternal(webBrowser1.Url, false);

        if (string.IsNullOrEmpty(cookieData) == false)
        {
            textBoxCookie.Text = cookieData;

            var dict = ParseCookieData(cookieData);
            textBoxFedAuth.Text = dict["FedAuth"];
            textBoxrtFa.Text = dict["rtFa"];
        }
    }
    catch (Exception)
    {
    }
}

Step 03: Progrmatically Make WebRequest Using Cookies

Now that we have the FedAuth and rtFA cookies we can continue on and use the HttpClient to call any andpoint we need. In my case calling many endpoints that contain images. The code looks like this:

private void buttonDownloadImage_Click(object sender, EventArgs e)
{
    try
    {
        var url = $"https://xx.sharepoint.com/sites/xx/xx/Images/{textBoxImageName.Text}";

        var handler = new HttpClientHandler();

        handler.CookieContainer = new System.Net.CookieContainer();

        // Add our cookies to collection
        var cc = new CookieCollection();
        cc.Add(new Cookie("FedAuth", textBoxFedAuth.Text));
        cc.Add(new Cookie("rtFa", textBoxrtFa.Text));

        handler.CookieContainer.Add(new Uri(url), cc);

        var httpClient = new HttpClient(handler);
        var resp = httpClient.GetAsync(url).Result;
        var byteData = resp.Content.ReadAsByteArrayAsync().Result;
               
        if (resp.IsSuccessStatusCode)
        {
            pictureBox1.Image = byteArrayToImage(byteData);
        }
    }
    catch (Exception) 
    { 
            
    }
}

Thats it. And it works like a charm.

Answer 3

I created a github project based on https://stackoverflow.com/users/1375553/vadim-gremyachev 's answer https://github.com/nddipiazza/SharepointOnlineCookieFetcher with a project that can generate these cookies.

It has releases for Windows, Centos7 and Ubuntu16 and I used mono develop to build it so that it is platform independent.

Intended for users who are not making programs with CSOM in c# but still want to be able to easily get the cookies.

Usage

One Time Only Step: (see Access to the path "/etc/mono/registry" is denied)

sudo mkdir /etc/mono
sudo mkdir /etc/mono/registry
sudo chmod uog+rw /etc/mono/registry

Run program:

Linux: ./SharepointOnlineSecurityUtil -u youruser@yourdomain.com -w https://tenant.sharepoint.com

Windows: SharepointOnlineSecurityUtil.exe -u youruser@yourdomain.com -w https://tenant.sharepoint.com

Enter a password when promped

Result of stdout will have SPOIDCRL cookie.

Answer 4

I still needed both FedAuth and rtFa cookies for my purposes. I tried using just FedAuth, but it wouldn't work without both. Another developer confirmed he saw the same behavior.

NOTE: Legacy authentication must be enabled in your tenant for this to work.

Here is a thread to help obtain both FedAuth and rtFa.

1. Send Post request to https://login.microsoftonline.com/extSTS.srf with the following body.
   Replace UserName, Password, EndPoint Address with relevant values.

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope"
      xmlns:a="http://www.w3.org/2005/08/addressing"
      xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:Header>
    <a:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>
    <a:ReplyTo>
      <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
    </a:ReplyTo>
    <a:To s:mustUnderstand="1">https://login.microsoftonline.com/extSTS.srf</a:To>
    <o:Security s:mustUnderstand="1"
       xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <o:UsernameToken>
        <o:Username>[username]</o:Username>
        <o:Password>[password]</o:Password>
      </o:UsernameToken>
    </o:Security>
  </s:Header>
  <s:Body>
    <t:RequestSecurityToken xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">
      <wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
        <a:EndpointReference>
          <a:Address>[endpoint]</a:Address>
        </a:EndpointReference>
      </wsp:AppliesTo>
      <t:KeyType>http://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</t:KeyType>
      <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>
      <t:TokenType>urn:oasis:names:tc:SAML:1.0:assertion</t:TokenType>
    </t:RequestSecurityToken>
  </s:Body>
</s:Envelope>

2. Note the content of the wsse:BinarySecurityToken node within the response data.
3. Send Post request to https://YourDomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0.
   Replace 'YourDomain with relevant value. Provide the wsse:BinarySecurityToken content within the body of the request.

The response header will contain the FedAuth and rtFa cookies.