Monday, April 18, 2016

Using the ICS KeyChain API

Update: Sample app code is now available on github.

The recently released Android 4.0 (Ice Cream Sandwich, ICS) introduces a new, unified UI for both tablets and handsets, lots of 'people-centric' communication and sharing features and other convenient improvements such as a better camera app and the much-hyped face unlock. Since everyone is talking about those, we will have a look at some of the less-user visible, but nonetheless important security-related improvements.

Android is often said to be missing crucial security features to be seriously accepted in the corporate world, which has long been the  domain of RIM's BlackBerry. Two of those missing features were the ability to control the system's trusted CA certificates and offer a centralized secure credential storage. Since many companies use private PKI's, the ability to install trusted certificates system-wide is essential for using corporate services secured by those PKI's. Until now, the only way to use those was to embed the needed CA certificates in each application and create custom TrustStores to be able to connect using SSL. A system-wide credential storage has actually been available for a while, but it was only usable by the built-in VPN and WiFi (EAP) clients. One could install a private key/certificate pair using the Settings app, but there was no public API to access the installed keys from applications. ICS offers SDK API's for both trusted certificate management and the secure credential storage via the KeyChain class. We will have a look at how it is used in the following sections.

The KeyChain class is deceptively simple: it offers only 4 public static methods, but those are sufficient to do most certificate-related tasks. Let's first see how one would install a private key/certificate pair and use those to sign and verify some data. The KeyChain API lets you install a private key/certificate pair bundled in a PKCS#12 file. Instead of offering an API to directly install the key and certificate, KeyChain provides a factory method, createInstallIntent() that returns a system intent to parse and install keys/certificates (that is actually the same intent offered by the Settings app in previous versions). To install a PKCS#12 file, you have to read it to a binary array, store it under the EXTRA_PKCS12 key in the intent's extras, and start the associated activity:

Intent intent = KeyChain.createInstallIntent();
byte[] p12 = readFile("keystore-test.pfx");
intent.putExtra(KeyChain.EXTRA_PKCS12, p12);
startActivity(intent);

This will prompt you for the PKCS#12 password in order to extract and parse the key and certificate. If the password is correct, you will be prompted for a 'certificate name' as shown in the screenshot below. If the PKCS#12 has a friendly name attribute it will be shown as the default, if not you will just get a long hexadecimal hash string. The string you enter here is the key/certificate alias you will use to access those later via the KeyChain API. You will be prompted to set a lock screen PIN or password to protect the credential storage if you haven't already set one.


To use a private key stored in the system credential storage, you need to call KeyChain.choosePrivateKeyAlias()and provide a callback implementation that receives the selected alias:

public class KeystoreTest extends Activity implements OnClickListener,
     KeyChainAliasCallback {

    @Override
    public void onClick(View v) {
        KeyChain.choosePrivateKeyAlias(this, this, 
           new String[] { "RSA" }, null, null, -1, null);
    }

    @Override
    public void alias(final String alias) {
        Log.d(TAG, "Thread: " + Thread.currentThread().getName());
        Log.d(TAG, "selected alias: " + alias);
    }
}

The first parameter is the current context, the second -- the callback to invoke, and the third and forth specify the acceptable keys (RSA, DSA or null for any) and acceptable certificate issuers for the certificate matching the private key (Edit: it turns out both keyTypes and issuers are currently unused, so just pass null). The next two parameters are the host and port number of the server requesting a certificate, and the last one is the alias to preselect. We leave all but the key type as unspecified (null or -1) here to be able to select from all available certificates. One thing to note here is that the alias() callback will not be called on the main thread, so you shouldn't try to directly manipulate the UI (it is called on a binder thread).  Using the key requires user authorization, so Android will display a key selection dialog which also serves to allow access to the selected key.


In order to get a reference to a private key, you need to call the KeyChain.getPrivateKey() method passing the key alias name received in the previous step. This doesn't seem to be documented but if you try to call this method on the main thread you will get an exception saying that this may 'lead to a deadlock'. Here we call it on a background thread using AsyncTask (which is almost always the right thing to do when dealing with potentially time-consuming I/O operations).

new AsyncTask<Void, Void, Boolean>() {

    private Exception error;

    @Override
    protected Boolean doInBackground(Void... arg) {
        try {
            PrivateKey pk = KeyChain.getPrivateKey(ctx,
                    alias);
            X509Certificate[] chain = KeyChain.getCertificateChain(ctx, 
                    alias);
       
            byte[] data = "foobar".getBytes("ASCII");
            Signature sig = Signature.getInstance("SHA1withRSA");
            sig.initSign(pk);
            sig.update(data);
            byte[] signed = sig.sign();

            PublicKey pubk = chain[0].getPublicKey();
            sig.initVerify(pubk);
            sig.update(data);
            boolean valid = sig.verify(signed);
            Log.d(TAG, "signature is valid: " + valid);

            return valid;
       } catch (Exception e) {
           e.printStackTrace();
           error = e;

           return null;
       }
   }

   @Override
   protected void onPostExecute(Boolean valid) {
        if (error != null) {
            Toast.makeText(ctx, "Error: " + error.getMessage(),
                    Toast.LENGTH_LONG).show();

            return;
        }

        Toast.makeText(ctx, "Signature is valid: " + valid,
            Toast.LENGTH_SHORT).show();
    }
}.execute();

We first get the private key and certificate chain using the key alias and then create and verify a signature to check if the key is actually usable. Since we are using a self-signed certificate the 'chain' consists of a single entry, but for a certificate signed by a CA you will need to find the actual end entity certificate in the returned array.

Installing a CA certificate is not very different from installing a PKCS#12 file: you load the certificate in a byte array and pass it as an extra to the install intent.

Intent intent = KeyChain.createInstallIntent();
intent.putExtra(KeyChain.EXTRA_CERTIFICATE, cert);
startActivity(intent);

Android will parse the certificate, and if it's Basic Constraints extension is set to CA:TRUE it will consider it a CA certificate and import it into the user trust store. You will need to authenticate to import the certificate, but the funny thing is that the import dialog does not show neither the certificate DN, nor its hash value. The user has no way of knowing what they are importing, until it's done. Very few people will bother to actually check, so this could be a potential security threat: malicious applications might trick people into installing rogue certificates. Here's how the import dialog looks:


After the certificate is imported, it will show up in the 'Trusted credentials' screen's 'User' tab (Settings->Security->Trusted credentials). Tapping the certificate entry displays a details dialog, where you can (finally!) check the subject, issuer, validity period, serial number and  SHA-1/SHA-256 fingerprints. You can also remove the certificate by pressing the 'Remove' button (scroll down to display it).


While you can delete individual CA certificates, there is no way to delete individual keys and user certificates. You can delete all by using the 'Clear credentials' option in the Credential storage section of the security settings. Another thing to note is that, as long as you have keys in the credential storage, you cannot remove the screen lock, since it is used to protect access to the keystore. In previous Android versions, there was a separate 'credential storage password', but it seems in ICS they decided to simplify things by using the screen lock password to protect credential storage as well.

The newly introduced KeyChain API lets you install and access private keys in a centralized and secure credential storage, as well as add system-wide trusted certificates. It doesn't provide low-level access to the underlying keystore, utilizing the Android intent dispatching mechanism instead to call a system activity that does the actual work. The CA certificate install dialog is missing a crucial feature (displaying details about the certificate), but all in all, providing the access to the system keystore service is a step in the right direction.

That wraps the first part of our Android keystore introduction. In the next part we will look into all that is hidden behind theKeyChain facade, and try to give some details about the underlying implementation.

ICS Credential Storage Implementation

In the previous entry, we looked at how the new ICS KeyChain API is used and tried installing a user private key/certificate pair and a CA certificate. Now we'll will try to find out where  they are actually stored and how they are protected.

Looking at frameworks/base/keystore/java/android/security, we notice several interesting classes that are not mentioned in the SDK documentation. The most promising is the KeyStore class, so let's have a look. Sure enough, it is marked as hidden (using the dreaded @hide comment). It does have methods for interacting with the key store (get()put()delete()reset(), etc.), but where is the actual key store? As it turns out, all methods send command to a local socket aptly named 'keystore'. With a little creative grepping, we find out that there is native daemon with the same name listening on that socket. The source is in frameworks/base/cmds/keystore/keystore.cpp, so let's have a look. The file has some helpful comments, and we learn that keys are encrypted, checksummed and saved as files (one key per file). But where are the actual files? Looking at /init.rc we find the keystore daemon startup command looks like this:

service keystore /system/bin/keystore /data/misc/keystore
    class main
    user keystore
    group keystore
    socket keystore stream 666

Next step is, of course, peeking into /data/misc/keystore

# ls -la /data/misc/keystore
-rw------- keystore keystore       84 2011-11-30 15:26 .masterkey
-rw------- keystore keystore      980 2011-11-30 15:56 1000_CACERT_testca
-rw------- keystore keystore      820 2011-11-30 15:55 1000_USRCERT_test
-rw------- keystore keystore      932 2011-11-30 15:55 1000_USRPKEY_test

Here each file name consists of the UID of the user that created it (1000 is system), the entry type (CA certificate, user certificate or private key), and the key name (alias) connected with underscores. And, of course, there is a .masterkey. Going back to the keystore daemon source, we find out that:
  • each key is encrypted with a 128-bit AES master key in CBC mode
  • each key blob contains an info header, the initial vector (IV) used for encryption, an MD5 hash value of the encrypted data and the encrypted data itself
  • the master key (in .masterkey) is itself encrypted with an AES key. The encryption key is derived from the password using the PBKDF2 key-derivation function with 8192 iterations (it may take a while...). The salt is randomly generated and is stored in the .masterkey file's info header.
What this means in practice is that the Android key store is pretty secure for a software solution: even if you had access to a rooted device and managed to extract the key blobs, you would still need the keystore password to derive the master key. Trying out different password to decrypt the master key would require at least 8192 iterations to derive a key, which is prohibitively expensive. In addition, the derivation function is seeded  with a 128-bit random number, so pre-calculated password tables cannot be used.

Key blobs are owned by the keystore user, so on a regular (not rooted) device, you need to go through the daemon to access the keys. As it turns out, there is a helpful command line utility that talks to the daemon and lets us manipulate the key store: keystore_cli. It has commands for initializing the key store, listing, getting and deleting keys, etc. Experimenting with it shows that the keystore daemon is additionally checking the calling process's UID to grant or deny access to each command:

# keystore_cli unlock
keystore_cli unlock
6 Permission denied
# keystore_cli get CACERT_testca
keystore_cli get CACERT_testca
1 No error
-----BEGIN CERTIFICATE-----
MIICiTCCAfKgAwI...

# su system
su system
$ keystore_cli insert foo bar
keystore_cli insert foo bar
1 No error
$ keystore_cli saw ""
keystore_cli saw ""
1 No error
foo
USRPKEY_test
USRCERT_test
CACERT_testca
$ keystore_cli get foo
keystore_cli get foo
1 No error
bar
$ exit

# su app_44
su app_44
$ keystore_cli saw ""
keystore_cli saw ""
1 No error
$ keystore_cli insert baz boo
keystore_cli insert baz boo
1 No error
$ keystore_cli get baz
keystore_cli get baz
1 No error
boo

This basically translates to:
  • root cannot lock/unlock the key store, but can access system keys
  • the system user can do pretty much anything (initialize or reset the key store, etc.)
  • regular users can insert, delete and access keys, but can only see their own keys

The android.security.KeyStore class we found while browsing the framework's source is almost a one-to-one port of the keystore_cli command's functionality to Java. By using it Java apps can get direct access to the keystoredaemon, but as we said, that class is not part of the public API. There are a couple of reasons for this:

  • even if they had access to it, normal apps wouldn't have the needed permissions to initialize or unlock the key store
  • it's interface exposes the current implementation: keys are returned as raw blobs which wouldn't be possible if the key store and related cryptographic operations were implemented in hardware (such as in a TPM).

As mentioned in the previous article, most of the described credential storage functionality has been available in Android since at least Donut (1.5), but the key store was only accessible to system applications such as Settings, and the WiFi and VPN clients. What ICS adds are a few layers on top of this that make it possible to offer user applications access to the system key store and assert fine-grained control over what keys each app is allowed to use. In the next part of the series we will look at the implementation of the new credential storage functionality added in ICS.

ICS Credential Storage Implementation, Part 2

In the previous entry, we found how Android's keystore daemon manages keys and certificates, and how to connect to it using the provided keystore_cli utility. Now we will look at the intermediate layers between the OS daemon and the public KeyChain API introduced in ICS.

Browsing the android.security package, we find two AIDL files: IKeyChainService.aidl andIKeyChainAliasCallback.aidl. This is a hint that the actual key store functionality, like most Android OS services, is implemented as a remote service that the public API's bind to. IKeyChainAliasCallback is just the callback called when you select a key via KeyStore#choosePrivateKeyAlias(), so it's of little interest. IKeyChainService has the actual methods KeyChain uses to get a handle to a private key or a certificate, plus some internal API's used by the Settings and certificate installer applications. Naturally, the whole interface is marked as hidden, so SDK applications cannot directly bind to the service.

The IKeyChainService interface has one implementation, the KeyChainService in the KeyChain.apk system package. We find the source in packages/apps/KeyChain, so let's explore the app's configuration. Looking at the manifest reveals that it consists of three components: the KeyChainService, a KeyChainActivity, and a broadcast receiver, you guessed it, KeyChainBroadcastReceiver. The package is com.android.keychain and itssharedUserId is set to 'android.uid.system', which, as we saw in the previous article, is necessary to be able to send management commands to the native keystore daemon. Let's first examine the service.

As can be expected, the KeyChainService is a wrapper for the android.security.KeyStore class that directly communicates with the native keystore daemon. It provides 4 sets of functionality: 
  • key store management: methods for getting private keys and certificates
  • trust store management: methods for installing and deleting CA certificates in the user trust store
  • key and trust store initialization: a reset() method that deletes all key store entries, including the master key, thus returning the key store to a 'not initialized' state; it also removes all user-installed trusted certificates
  • methods for querying and adding entries to the key access grant database (more on this later)
Since the KeyChain application is running as the system user, any process that binds to its remote interface would technically be able to perform all key and trust store operations. To prevent this, the KeyChainService imposes additional access control on its users. It employs two mechanisms to achieve this: controlling access based on the caller's UID and a key access grant database. Deleting a CA certificate and resetting the key and trust stores are only allowed to the system user (those operations are typically called via the Settings app's UI, which runs as system), and installing a trusted CA certificate is only allowed to the system user or the certificate installer application (com.android.certinstaller package). Controlling access to the key store is a little bit more interesting:KeyChainService maintains a grants database (in/data/data/com.android.keychain/databases/grants.db) that maps UID's to the key aliases they are allowed to use. Let's have a look inside:
# cd /data/data/com.android.keychain/databases
cd /data/data/com.android.keychain/databases
# ls
ls
grants.db
grants.db-journal
# sqlite3 grants.db
sqlite3 grants.db
sqlite> .schema
.schema
CREATE TABLE android_metadata (locale TEXT);
CREATE TABLE grants (  alias STRING NOT NULL,  uid INTEGER NOT NULL,  UNIQUE (al
ias,uid));
sqlite> select * from grants;
select * from grants;
test|10044
key1|10044

In this example, the application with UID 10044 (our test application) is granted access to the keys with the test andkey1 aliases.

Each call to getPrivateKey() or getCertificate() is subject to a check against the grants database, and results in a exception if a grant for the required alias is not found. As stated before, KeyChainService has API's for adding and querying grants, and only the system user is allowed to call them. But who is responsible for actually granting and revoking access? Remember the private key selection dialog from the first article? When you callKeyChain#choosePrivateKeyAlias(), it will start the KeyChainActivity introduced above, which will check if the key store is unlocked, and if so, show they key selection dialog. Clicking the 'Allow' button will return to theKeyChainActivity, which will then call KeyChainService#setGrant() with the selected alias, adding it to the grants database. Thus, even if the activity requesting access to a private key has the needed permissions, the user has to unlock the key store and explicitly authorize access to each individual key.

Besides controlling private key storage, the KeyChainService also offers trust store management by using the newly added TrustedCertificateStore class (part of libcore). This class provides both the ability to add user-installed trusted CA certificates and remove (mark as not trusted) system (pre-installed) CA's. Since the implementation is fairly complex and rather interesting, it will be the topic of another post.

The last component of the KeyChain app is the KeyChainBroadcastReceiver. It listens for aandroid.intent.action.PACKAGE_REMOVED broadcast and simply forwards control to the KeyChainService. On receiving the PACKAGE_REMOVED action, the service does some grant database maintenance: it goes through all entries and deletes those referencing packages that are no longer available (i.e., uninstalled ones). With this we now have the (almost) complete picture (click to enlarge):


ICS introduces a new service that grants access to both the system key store (managed by the keystore daemon) and trust store (manged by the TrustedCertificateStore class) that backs the KeyChain API exposed in the public SDK. That makes it possible to control access to keys based on both the calling process's UID and the key access grant database, thus allowing for fine-grained, user-driven control over what keys each application can access. We've discussed most of the components this framework consists of in this and the previous entry. What remains is to look into the new trust store implementation introduced in Android 4.0. That will be the focus of the next post of this series.

ICS Trust Store Implementation

In the previous two posts we looked at the internal implementation of the Android credential storage, and how it is linked to the new KeyChain API introduced in ICS. As briefly mentioned in the second post, there is also a newTrustedCertificateStore class that manages user installed CA certificates. In this entry we will examine how the new trust store is implemented and how it is integrated in the framework and system applications.

Storing user credentials such as passwords and private keys securely is of course essential, but why should we care about the trust store? As the name implies, the trust store determines who we trust when connecting to Internet servers or validating signed messages. While credentials are usually used proactively only when we authenticate to a particular service, the trust store is used every time we connect to a secure server. For example, each time you check GMail, Android connects to Google's severs using SSL and validates their certificates based on the device's trust store. Most users are unaware of this, unless some error occurs. Since the trust store is used practically all the time, and usually in the background, one could argue that it's even more important then credential storage. Up till Android 4.0, the OS trust store was hard wired into the firmware, and users had no control over it whatsoever. Certificates bundled in the store were chosen solely by the device manufacturer or carrier. The only way to make changes was to root your device, re-package the trusted certificates file and replace the original one (instructions from cacert.org here). That is obviously not too practical, and a major obstacle to using Android in enterprise PKI's. In the wake of major CA's being compromised practically each month this year, tools that make changing the default trusted certificates in place have been developed, but using them still requires a rooted phone. Fortunately, ICS has made managing the trust store much more flexible, and gives the much needed control over who to trust to the user. Let's see what has changed.

Pre-ICS, the trust store was a single file: /system/etc/security/cacerts.bks, a Bouncy Castle (one of the JCE cryptographic providers used in Android) native keystore file. It contains all the CA certificates Android trusts and is used both by system apps such as the email client and browser, and applications developed using the SDK. Since it resides on the read-only system partition, it cannot be changed even by system-level applications. The newly introduced in ICSTrustedCertificateStore class still reads system trusted certificates from /system/etc/security, but adds two new, mutable locations to store CA certificates in /data/misc/keychain: the cacerts-added and cacerts-removed directories. Let's see what's inside:

ls -l /data/misc/keychain
drwxr-xr-x system   system            2011-11-30 12:56 cacerts-added
drwxr-xr-x system   system            2011-12-02 15:21 cacerts-removed
# ls -l /data/misc/keychain/cacerts-added
ls -l /data/misc/keychain/cacerts-added
-rw-r--r-- system   system        653 2011-11-29 18:34 30ef493b.0
-rw-r--r-- system   system        815 2011-11-30 12:56 9a8df086.0
# ls -l /data/misc/keychain/cacerts-removed
ls -l /data/misc/keychain/cacerts-removed
-rw-r--r-- system   system       1060 2011-12-02 15:21 00673b5b.0

Each file contains one CA certificate. The file names may look familiar: they are hashes of the CA subject names, as used in mod_ssl and other cryptographic software implemented using OpenSSL. This makes it easy to quickly find certificates without scanning the entire store. Also note the permissions of the directories: 0775 system systemguarantees that only the system user is able to add or remove certificates, but anyone can read them. As can be expected, adding trusted CA certificates is implemented by storing the certificate in cacerts-added under the appropriate file name. The two files above, 30ef493b.0 and 9a8df086.0, correspond to the certificates displayed in the 'User' tab of the Trusted credential system application (Settings->Security->Trusted credentials). But how are OS-trusted certificates disabled? Since pre-installed CA certificates are still stored in /system/etc/security (read-only), a CA is marked as not trusted by placing a copy of its certificate in cacerts-removed. Re-enabling is performed by simply removing the file. In this particular case, 00673b5b.0 is the thawte Primary Root CA, shown as disabled in the 'System' tab:


TrustedCertificateStore is not available in the SDK, but it has a wrapper accessible via the standard JCEKeyStore API, TrustedCertificateKeyStoreSpi, that applications can use. Here's how we can use it to get the current list of trusted certificates::

KeyStore ks = KeyStore.getInstance("AndroidCAStore");
ks.load(null, null);
Enumeration aliases = ks.aliases();
while (aliases.hasMoreElements()) {
    String alias = aliases.nextElement();
    X09Certificate cert = (X509Certificate) 
       ks.getCertificate(alias);
    Log.d(TAG, "Subject DN: " + 
       cert.getSubjectDN().getName());
    Log.d(TAG, "Issuer DN: " + 
       cert.getIssuerDN().getName());
}

If you examine the output of this code, you would notice that certificate aliases start with either the user: (for user installed certificates) or system: (for pre-installed ones) prefix, followed by the subject's hash value. This lets us easily access the OS's trusted certificates, but a real word application would be more interested in whether it should trust a particular server certificate, not what the current trust anchors are. ICS makes this very easy by integrating theTrustedCertificateKeyStoreSpi with Android's JSSE (secure sockets) implementation. The defaultTrustManagerFactory uses it to get a list of trust anchors, thus automatically validating server certificates against the system's currently trusted certificates. Higher-level code that uses HttpsURLConnection or HttpClient (both built on top of JSSE) should thus just work without needing to worry about creating and initializing a customSSLSocketFactory. Here's how we can use the TrustManager to validate a certificate issued by a private CA (the CA certificate is already installed in the user trust store).

X509Certificate[] chain = KeyChain.getCertificateChain(ctx,
    "keystore-test-ee");
Log.d(TAG, "chain length: " + chain.length);
for (X509Certificate x : chain) {
    Log.d(TAG, "Subject DN: "
        + x.getSubjectDN().getName());
    Log.d(TAG, "Issuer DN: "
        + x.getIssuerDN().getName());
}

TrustManagerFactory tmf = TrustManagerFactory.getInstance("X509");
tmf.init((KeyStore) null);

TrustManager[] tms = tmf.getTrustManagers();
X509TrustManager xtm = (X509TrustManager) tms[0];
Log.d(TAG, "checking chain with " + xtm);
xtm.checkClientTrusted(chain, "RSA");
Log.d(TAG, "chain is valid");

Works pretty well, but there is one major problem with this code: it does not check revocation. Android's defaultTrustManager explicitly turns off revocation when validating the certificate chain. So even if the certificate had a valid CDP (CRL distribution point) extension, pointing to a valid CRL, and the certificate was actually revoked, it would still validate fine in Android. What's missing here is the ability to dynamically fetch, cache and update revocation information as needed, based on information available in certificate extensions. Hopefully future version of Android will add this functionality to make Android's PKI support complete.

Of course, system applications such as the browser, email and VPN clients are also taking advantage of the new trust store, so connecting to a corporate Exchange server or a secure Web application should be as easy as installing the appropriate certificates. We'll see how well that works out in practice once I get a real ICS device (shouldn't be too long now...). 

That concludes our discussion of the new credential and trust stores introduced in Android 4.0. To sum things up: users can now freely install and remove private keys and trusted certificates, as well as disable pre-installed CA certificates via the Settings app. Third-party applications can also do this via the new KeyChain API, if the user grants the app the necessary permissions. The key and trust stores are fully integrated into the OS, so using standard secure communication and cryptographic Java API's should just work, without the need for applications-specific key stores. A key element required for full PKI support -- revocation checking, is still missing, but the key and trust store functionality added in ICS is a huge step in making Android more secure, flexible and enterprise-friendly.