How to capture HTTPS traffice on an Android device

Packet capture is an important means for debugging problems on a daily basis. It can help us clarify the data transmission problem between the client and the server,make it easier to find the root cause

In recent years, the http protocol has been gradually being phased out, and almost all network requests have replaced by the https protocol

In this post I will explain the steps to capture HTTPS package , yes no kidding, I mean HTTPS 🙂

Tools for packet capturing

To capture network requests, you must first select a packet capture tool.

There are many professional packet capture tools. in this post I am going to use Fiddler

Please install Fiddler on your computer first

After the installation is complete, start Fiddler, It will then automatically grab all the network request packets on your current computer.

Configuration on computer

But if we want to capture network requests on the android device, we need to do some additional configuration.
First click View -> Preferences -> Connections from the top toolbar of Fiddler

Pay attention to the following two options

• Port number, the default value is 8866, if there are no special requirements, just leave it as it is
• Make sure Allow remote computers to connect must be checked, otherwise the network request on the android device will not be caught.

Check the second option and click SAVE

Configurations on android device

Next, we need to do some simple configurations on the mobile phone.

• Make sure that your mobile phone and the computer used to capture packets are in the same local network
• Then modify the advanced options of the mobile phone currently connected to Wifi, change the proxy type to manual, change the proxy host name to the computer’s ip address, and change the proxy port to 8866:

Capture HTTP traffic

Now we can use Fiddler to capture the network request on the mobile phone. visit a https website from your device’s browser:

http://printf.pro/

Then go to Fiddler to check, you can find that the network request packet on the mobile phone has been successfully captured by Fiddler (sometimes there are too many packet information displayed in Fiddler, which is inconvenient to view, you can use Ctrl+X to clear the information):

The details of this network request can be seen in Fiddler, including the header information of the request, the header information of the response, the body content of the response, and so on.

Capture https requests

The https protocol is an encrypted transmission protocol. the data is is encrypted and then transmitted
This protocol ensures the security of data,for example, we can try to access Bing in the mobile browser, and then watch the packet information captured in Fiddler, as you can see, this kind of package is not helpful for us to analyze

So how do we capture the network packets over https? follow the steps below

• First, you need to enable the https packet capture feature in Fiddler. Click View -> Preferences -> HTTPS from the toolbar at the top of Fiddler.In the HTTPS settings page, first click Trust root certificate to install the certificate, then check the Capture HTTPS traffic option, then click SAVE

• Install certificate provided by Fiddler
  Visit the following address in your mobile browser:

http://ipv4.fiddler:8866/

You will see a web page built by Fiddler:

Click on the FiddlerRoot certificate link to download and install the mobile phone certificate provided by Fiddler.

After the installation is complete, visit Bing again, then you can see that the network packet requesting the Bing homepage has also been successfully captured

Capture traffice of Android applications

Notice that above solution is only suitable for capturing traffice from browser. If you want to capture packets of other applications, you still cannot capture them.
This is because Android has a security upgrade in the 7.0 . Starting from Android 7.0, only the certificate of the packet capture tool is installed on the device, but it is still impossible to capture the https request, and a network security configuration must be added to the code of the application.

This upgrade makes every application more secure, because capturing https packets is indeed a dangerous behavior, you can only capture packets and debug your own programs.
Therefore, this security upgrade was made in the Android 7.0 system. By default, we cannot capture the https request of other App.

If you want to capture the https request of your own App, create a network_security_config.xml file in the res/xml directory and add the following configuration:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <base-config cleartextTrafficPermitted="true">
        <trust-anchors>
            <certificates src="user"/>
            <certificates src="system"/>
        </trust-anchors>
    </base-config>
</network-security-config>

Next, you need to configure the android:networkSecurityConfig property in AndroidManifest.xml to make the above configuration take effect:

<application
    ...
    android:networkSecurityConfig="@xml/network_security_config">
    ...
</application>

In this way, we can capture the https network request sent by the current application.

Doubts

Since Android 7.0 starts, you must add network security configuration to your application to capture packets for https requests. but why for browser we did nothing in the first test?
Let’s take a look at the AndroidManifest.xml file in the Chromium source code.

As you can see, a section of android:networkSecurityConfig configuration has already been added to the Chromium source code, so let’s continue to follow up to see what is configured inside: