109

Android 7 introduced some changes to the way certificates are handled (http://android-developers.blogspot.com/2016/07/changes-to-trusted-certificate.html) and somehow I cannot make my Charles proxy work any more.

My network_security_config.xml:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <base-config>
        <trust-anchors>
            <certificates src="system" />
        </trust-anchors>
    </base-config>
    <debug-overrides>
        <trust-anchors>
            <certificates src="user" />
        </trust-anchors>
    </debug-overrides>
</network-security-config>

I'm running in debug mode. But no matter what, I get javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found..

Needless to say, I did install a pfx certificate from Settings -> Security -> Install from storage. The certificate shows in User Credentials but not in Trusted credentials -> User. On my lollipop device, the certificates are listed there.

I'm using okhttp3 as HTTP library.

Any idea what I am doing wrong ?

mbonnin
  • 6,893
  • 3
  • 39
  • 55
  • 1
    As a quick test, try adding/moving the `user` one into the `` and see if that changes matters. It shouldn't, but it'll only take a moment to try. – CommonsWare Aug 29 '16 at 21:10
  • Wouldn't you need to add the explicit CA for Charles in https://developer.android.com/training/articles/security-config.html : 'Trusting Additional CAs' as the Charles CA is a self generated one and wouldn't be in the Android System trust chain? – Morrison Chang Aug 29 '16 at 21:35
  • @Morisson Chang: I don't want to embed the Charles CA into the app, I want to be able to add it manually on my development phone like I used to do. From `Settings -> Security -> Install from storage` – mbonnin Aug 29 '16 at 21:42
  • 1. How are you referencing the network_security_config.xml in your app's AndroidManifest.xml? 2. Would you please link the .pfx file? I presume it contains only the CA cert (no private keys) and thus should be fine to publish. I'm asking because, if the .pfx contains a private key, it'll be assumed to be a client cert file and thus CA from the file won't be installed as trusted for server authentication. – Alex Klyubin Aug 30 '16 at 00:10
  • @Alex Klyubin: My original .pfx file indeed contained a private key. I just switched to .crt with just one cert inside. No luck so far. I'll try harder, reboot all the things and keep you updated. – mbonnin Aug 30 '16 at 00:18
  • 1
    @AlexKlyubin hurra, it's working now :). I guess I needed to reboot my charles or app or whatever other thing. I was confused because I have installed the same .pfx file on countless android devices before nougat and they were recognized ok. But everything is good now. Could you write an answer so I can approve it ? – mbonnin Aug 30 '16 at 00:24

6 Answers6

124

The solution is do not use .p12, just navigate with Chrome (with configured proxy on wifi) to http://charlesproxy.com/getssl and install downloaded .pem file.

I had exactly the same problem on my Nexus 5X running Android 7.0. There was previously exported .p12 from Charles 3.11.5 (Help->SSL Proxying->Export Charles Root certificate and Private key). When I tried to install .p12 from phone (Settings->Security->Install from storage) it appears only under "User credentials" and never at "Trusted credentials", and of course SSL with Charles proxy did not work.

The total "how-to" for Android 7.0 would be like that:

  1. Configure WiFi + proxy (how Charles requires it). Connect it.
  2. On device, navigate with Chrome to http://charlesproxy.com/getssl, accept request for download .pem, then press "open", it launches "Certificate installer" app. Use it to install the certificate as "VPN and apps".
  3. Put the attribute android:networkSecurityConfig="@xml/network_security_config" to <application> at Manifest.xml
  4. Create res/xml/network_security_config.xml with content from the first post (it is totally correct).
  5. Launch Charles and app and have fun.

P.S. Check date/time on the device. It should be correct.

Alexander Skvortsov
  • 2,696
  • 2
  • 17
  • 31
  • 2
    An important note on this great answer: if your app is targeting API 23 or lower, you don't need steps 3 and 4. In fact you won't be able to compile the network-security-config stuff if you have API 23 or lower. – David Ferrand Sep 19 '16 at 09:59
  • 1
    This is a great solution for Android 7.0. Much clearer than the official Charles Proxy information. – Adam Link Jan 04 '17 at 21:43
  • 3
    Likely only me but these steps do not work for me anymore in 7.1.1. I recently did an update – satyajit Feb 10 '17 at 01:01
  • @satyajit This solution is working for me on Android 7.1.1 on a Google Pixel. – Dick Lucas Feb 25 '17 at 02:21
  • with or without steps 3 and 4? – satyajit Feb 25 '17 at 18:16
  • I'm struck in this weired state where after downloading certificate, tapping on certificate it asks for PIN, even after entering correct PIN multiple times nothing happens ( Motorola 7.0 ). I tried removing PIN on device, restarted, tried same step each time I am taken to "Confirm your PIN" and just gets struck there. – vikramvi Aug 22 '17 at 12:51
  • @vikramvi, Probably something wrong with the phone. Did you try another one? – Alexander Skvortsov Aug 22 '17 at 13:41
  • What if I want to see the proxy log, not from my apps? I can't do the step 3, what should I do? – Minji Song Sep 13 '18 at 05:41
26

Based on the troubleshooting thread of comments for the OP, the answer is to install just the proxy's CA cert as trusted, not its cert + private key.

The issue was caused by two factors:

  1. Installing not just the MiTM proxy's CA cert but also its private key (thus enabling VPN apps on the device to decrypt/MiTM network traffic from other apps). You don't need the MiTM proxy's private key on the device.

  2. Android Nougat change in behavior of the Settings -> Security -> Install from storage flow for files which contain a private key in addition to cert(s). This change in behavior unmasked the above issue.

Prior to Nougat, the Settings -> Security -> Install from storage flow for files containing a private key in addition to certs erroneously installed the certs as trusted for server authentication (e.g., HTTPS, TLS, thus making your MiTM succeed), in addition to being correctly installed as client certs used for authenticating this Android device to servers. In Nougat, the bug was fixed and these certs are no longer installed as trusted for server authentication. This prevents client authentication credentials from affecting (weaking) the security of connections to servers. In your scenario, this prevents your MiTM from succeeding.

What complicates matters is that the Settings -> Security -> Install from storage does not provide an explicit way for the user to specify whether they are installing a client authentication credential (private key + cert chain) or a server authentication trust anchor (just a CA cert -- no private key needed). As a result, the Settings -> Security -> Install from storage flow guesses whether it's dealing with client/user authentication credential or server authentication trust anchor by assuming that, if a private key is specified, it must be a client/user authentication credential. In your case, it incorrectly assumed that you are installing a client/user authentication credential rather than a server authentication trust anchor.

P. S. With regards to your Network Security Config, you should probably configure the app to also trust "system" trust anchors in debug mode (debug-overrides section). Otherwise debug builds of the app won't work unless connections are MiTM'd by a proxy whose CA cert is installed as trusted on the Android device.

Alex Klyubin
  • 5,554
  • 2
  • 29
  • 24
  • 1
    Regarding the PS: the documentation says `Trust anchors specified in debug-overrides are added to all other configurations`. So I was under the impression that it added to 'base-config' and not replaced it completely ? – mbonnin Aug 30 '16 at 17:16
  • Oh, you're right. You don't need to explicitly list "system" trust anchors in debug-overrides. – Alex Klyubin Aug 30 '16 at 20:49
  • 4
    This answer is unclear. Please refer to @stkent's answer below. – AdamHurwitz Jan 22 '17 at 22:24
  • 4
    Agree this answer explains what is happening but is very unclear what steps one should do. – StarWind0 Sep 07 '17 at 16:52
1

I also had the same error and tried every way until I read the following message which was key: "The certificate shows in User Credentials but not in Trusted credentials".

So the problem was how to make the certificate appear there and these were the steps I followed:

  1. Download the certificate in the emulator from http://chls.pro/ssl
  2. Install the certificate & Trust
  • Setting app -> Security -> Encryption & Credentials -> Install a Certificate -> Select CA Certificate option
  • Select Install Anyway

enter image description here

  • Select Charles Proxy Certificate that you downloaded on your storage
  1. Verify that you're trusted the certificate
  • Open Trusted Credentials -> User Tab and you can see your certificate here enter image description here
  1. Start intercepting HTTPS Traffic
Jorge Casariego
  • 21,948
  • 6
  • 90
  • 97
0

For me SSL proxy was not working in release build variant. In debug worked.

k4dima
  • 6,070
  • 5
  • 41
  • 39
0

In 2023

Worth to mention that you need charles certificate file installed in your android phone and exist in your project directory.

This is how I make Charles work on Android 12. One screenshot would be perfect.

  • Put the certificate file(charlescert.pem) to res/raw/charlescert.pem. Well in case you cannot download it in your Android phone, download it in your computer's browser and upload that to your Android phone.

network_security_config.xml

If I don't add this line tools:ignore="NetworkSecurityConfig" the code will complain, come from iOS dev... not sure the reason.

<?xml version="1.0" encoding="utf-8"?>
<network-security-config xmlns:tools="http://schemas.android.com/tools">
    <base-config>
        <trust-anchors>
            <certificates src="@raw/charlescert"
                tools:ignore="NetworkSecurityConfig" />
            <certificates src="system" />
        </trust-anchors>
    </base-config>
</network-security-config>

Eventually, add <application android:networkSecurityConfig="@xml/network_security_config"...> in AndroidManifest.xml

enter image description here

Zhou Haibo
  • 1,681
  • 1
  • 12
  • 32
-1

I'm on Android 7.1.1, here's how I setup on my device (OnePlus One) - without the change of manifest (I was targeting API 21 for my app):

In Charles Proxy:

  1. Help > SSL Proxying > Install Charles Root Certificate on a Mobile Device or Remote Browser. This steps gives you the proxy IP and port number and also the link to where you should download charles proxy SSL.

On your phone:

  1. Wifi Settings > Modify Network > Advanced Options. Set Proxy to Manual and enter the IP and Port number you received from Charles into Proxy hostname and Proxy port respectively.

  2. (OPTIONAL) You may or may not be able to access the chls.pro/ssl link provided by Charles earlier. On my device, I was always notified that I had no network connection. I added the charlesproxy.com to the Bypass proxy for field.

  3. On your browser, go to the link in step 3 and download whatever certificate necessary (if it doesn't work on Chrome, download Dolphin Browser).You can name your certificate with whatever name.

Back on Charles Proxy:

  1. You should get the prompt to either Allow or Deny your phone to use the proxy if your settings is defaulted to prompt you for remote connections.

You can now use Charles on Nougat 7.1.1.

Tiffany
  • 111
  • 1
  • 9