A VPN establishes a cryptographically secure networkusing the existing insecure infra structure of the Internet. Anumber of protocols, including IPSec have been designed toestablish VPNs. However, keys must be shared between thecommunicating peers before a VPN can be established. IKEprotocol is used for exchanging keys between authenticatedpeers over the Internet. However, IKE is vulnerable to DoSattacks and has security holes. A number of protocols havebeen proposed to replace IKE but these protocols also havevulnerabilities of their own. In this paper we present ananalysis of IKE and identify its security holes and designweaknesses. We also propose a more secure and efficientkey exchange protocol, Arcanum, and carry out its securityanalysis and comparison with existing protocols. Arcanumis more secure, robust to DoS attacks and efficient in termsof time and number of messages.
1. IntroductionThe Internet is highly insecure, because the traffic passes
unencrypted through a shared media. Private networks areexpensive and face the problem of physically securing theirlines. Virtual Private Networks (VPN) offer an economicalsolution to this problem by establishing a virtually privatenetwork while physically sharing the Internet media. Pri-vacy is achieved by creating a cryptographically secure tun-nel between authenticated peers, making the traffic virtuallyinvisible to the rest of the Internet. Many protocols havebeen designed to form VPNs including IPSec [4], L2TPand PPTP. However, before these protocols can function,keys must be exchanged between the communicating peers.Manual exchange of keys is not possible in large and geo-graphically spread networks. Moreover, keys must be re-freshed frequently to limit the amount of data encryptedwith a single key. Therefore, it is essential to design a pro-tocol that can exchange the keys automatically.
Internet Key Exchange (IKE) [2] is the proposed key ex-change standard by the IETF for IPSec. IKE has been de-
rived from two protocols, namely, the Oakley [10] and theSKEME protocol [5]. IKE is presently in the form of anRFC and has not been declared as a standard because it hascertain shortcomings and security holes. Many researchershave carried out the analysis of IKE [8][11][12]. A numberof possible successors have also been proposed to replaceIKE which include IKEv2 [3], Just Fast Keying (JFK) [1]and SIGMA [6]. However, these protocols also have prob-lems. IKEv2 adds complexity by keeping variable num-ber of messages and does not fully exploit the strength of��� authentication. It is also vulnerable to a DoS attack.JFK compromises on perfect forward secrecy (���) by re-using the ����. SIGMA has odd number of messages,hence, the delivery of the last message cannot be confirmed.
In this paper we will analyse the IKE protocol and give abrief introduction and comparison of famous key exchangeprotocols. We will also propose a more secure, efficientand DoS resistant key exchange protocol and carry out itscomprehensive security analysis. The rest of this paper isorganized as follows. Section 2 contains a list of notation.In Section 3 we present the analysis of IKE. In Section 4and 5 we propose a new protocol, Arcanum and discuss itssalient features. Some qualitative results are presented inSection 6. In Section 7 we carry out the security analysis ofArcanum. Finally conclusions are given in Section 8.
2. Notation
���� Authenticator of message�� Certificate (public key)� Cookie, a pseudorandom value (to avoid DoS)�� Certificate request��� Diffie-Hellman (� public value, � group,
�� public/private value pair, �� shared secret)��� ISAKMP header (* encrypted payload) �� IP address of � (� for source, � for destination)� ISAKMP payload used to exchange��� values� Nonce, a random value (to avoid replay attacks)��� Public key of �, Public Key Encryption�� Public Key Encryption
��� Pre-Shared Key�� Security Association���, �� Digital signature (payload)������ A pseudorandom function (e.g. HMAC)
applied on message � using key � �, � � is optional, is encrypted with key �, �, � Initiator, responder, gateway in key exchange� Message concatenation
3. Analysis of IKE Protocol
3.1. Security Holes of IKE Protocol
IKE has no protection against simple IP flooding DoS at-tack because of improper use of cookies. The use of cook-ies was first introduced in Photuris [13] and the idea wasto keep the responder stateless until the initiator proves thatit can receive at its claimed IP address. But in IKE, theresponder does keep state after receiving message 1 (seeFig. 1, Fig. 2 and Fig. 3). Hence, an illegitimate initiator canexhaust its memory by sending �� requests from bogus IPaddresses. Even if IKE somehow avoids this attack, a CPUexhaustion attack can be launched against the responder byinitiating a large number of ��s up to message 3. Message5, in which the initiator authenticates itself, is never sent.�� based aggressive mode of IKE passes identities in
the clear whereas both identities could have been hidden[11]. Aggressive mode has no protection against DoS at-tacks and comprises of three messages. If the final messageis lost, the initiator will keep sending data into an incom-plete �� for some time. In ��� authentication (Fig. 3)it is assumed that the initiator already has the responder’spublic key, an unlikely case in large networks. Anotherproblem with ��� is that in case the private keys are es-crowed, which is likely the case with encryption/decryptionkeys, the escrow agent can impersonate all its clients.
IKE claims that its ��� mode (Fig. 2) hides both theidentities from active attackers whereas in practice both theidentities are exposed to an eavesdropper [12]. The respon-der doesn’t know who he is talking to when he receives mes-sage 5 and doesn’t know which ��� to use. IKE addressesthis problem by binding the identities to the correspondingIP addresses that can be seen by eavesdroppers. A reflectionattack is also possible on the quick mode of IKE [8]. Sincethe same encryption key is used in both the directions, allthe intruder has to do is to swap the IP addresses and reflectthe message. The end result is that the initiator thinks thatit shares two ��s with the responder whereas it has none.
3.2. Design Weaknesses of IKE Protocol
An �� proposal in IKE must be accepted in full or re-jected. Therefore, the initiator must propose every accept-able algorithm for one purpose (say encryption) with allacceptable algorithms for another purpose (say authentica-tion). This leads to a combinatorial explosion of �� pro-
HDR, SA
HDR, SA
HDR, KE, Ni
HDR, KE, Nr
HDR*, IDi, [CERTi], SIGi
HDR*, IDr, [CERTr], SIGr
RESPONDERINITIATOR INITIATOR RESPONDER
HDR, SAi1, KEi, Ni
HDR, SAr1
HDR, SAi1, KEi, Ni
[IDr], SIGi, SAi2, TSi, TSr
HDR, SAr1, KEr, Nr, [CRQ]
SIGr, SAr2, TSi, TSrHDR*, IDr, [CERTr],
HDR*, IDi, [CERTi], [CRQ]
HDR, SAi1, KEi, Ni
HDR, SAr1, KEr,
KErid, Nr, [CRQ]
[CRQ], SAi2, SIGi}Ke, AUTH
Ni, Nr, {IDi, IDr, [CERTi],
HDR, SAi1, SAr1, KEi, KErid,
SAr2, SIGr}Ke, AUTHHDR, {IDr, [CERTr],
INITIATOR RESPONDER
IKEv2 ARCANUMIKE
Figure 1. DS authentication. The small arrowsindicate the saving of state by the responder.
posals. Moreover, in IKE the number of negotiable param-eters is very large which adds complexity and increases thenumber of round trip messages. Another complexity in IKEis the eight possible ways in which Phase 1 can be accom-plished. In IKE, it is possible for two different ��s to havethe same set of cookies [3]. This can cause problems sinceIKE uses the cookie pair for �� identification. Moreover,identity hiding in IKE is done since there can be many iden-tities behind the same IP address. But IKE does not specifyhow the responder comes to know about the identity withwhom the initiator wants to communicate.
4. The Arcanum Key Exchange Protocol
We propose a new protocol, Arcanum (“sacred secret”in Latin), which is inspired by JFK and IKEv2. Authors ofJFK propose a single phased approach and argue that twophases add unnecessary overhead and complexity. How-ever, in Arcanum, we used a two phased approach becausewe believe that this overhead will be distributed amongmany Phase 2 ��s and will make the protocol more effi-cient in the long run. A single Phase 2 negotiation can beused to establish many ��s, further distributing the over-head. Moreover, the first Phase 2 �� is piggy backed on thePhase 1 �� (this approach has also been used in [3]). Witha Phase 1 �� established, it is very easy to setup Phase 2��s, rekey existing ��s and pass control/error messages.
Arcanum uses weak cookies, which are random valuesand strong cookies, which are pseudorandom values. Weakcookies are used by the initiator who keeps state and strongcookies are used by the responder when it wants to elimi-nate state after replying. Strong cookies are regenerated bythe responder for validating message 3.
IKEv2 suggested proposing more than one algorithm foreach purpose in a single proposal (e.g. three encryption al-gorithms with two authentication algorithms) so that the re-sponder can choose one algorithm of each type. This avoidsthe combinatorial explosion of proposals but introduces an-other problem in which an algorithm of one type may notwork with an algorithm of another type. Therefore, in ouropinion, proposing algorithms in the form of suits is a bet-
ter technique after all. Moreover, it is logical to proposesuits with algorithms providing the same level of security.Arcanum proposes algorithms in suits but instead of writ-ing individual algorithms, pointers to pre-configured (andpublically available) algorithm suits are used. This greatlysimplifies the �� payload and reduces its size.��� is said to be achieved when keys of different ses-
sions are totally independent. It requires a �� key ex-change each time a new �� is setup or an old �� isrekeyed. Generation of ���� involve heavy computationsand may become a problem if a large number of �� orrekeying requests arrive simultaneously e.g. a new peercomes online or DoS attack is launched. We have devel-oped a strong technique in Arcanum that solves this prob-lem without compromising on ���. Arcanum calculatesa stack of ����, when the processor is idle. When an ��request arrives, it uses the current value from this stack andincrements the pointer. If the �� establishes successfully itdeletes the ���� associated with it in the stack and calcu-lates a new one to fill up the stack. If the �� does not com-plete successfully, it reuses the���� once the pointer com-pletes a round trip. The stack size is large enough to avoiduse of the same ���� for multiple ��s. Thus Arcanumavoids wasting time calculating ���� for illegitimate re-quests and legitimate �� requests which do not reach com-pletion. This technique improves efficiency as well as pro-vides protection against DoS attacks.
5. Arcanum Message ExchangesAll message of Arcanum begin with the ISAKMP header
(which also contains the cookies). The encryption bit in theheader is set only if the entire message is encrypted. Theinitiator chooses the authentication method and guesses the��� in case of �� and ��� modes. Once a Phase 1 ��is in place, Phase 2 ��s can easily be established with onlytwo messages according to the Phase 2 of IKEv2 [3].
5.1. SA Establishment using DS
Fig. 1 shows the message exchanges of �� mode ofIKE, IKEv2 and Arcanum. IKE comprises of six messagesand the responder saves state at message 1. The first twomessages of IKEv2 (dotted lines) are optional and are onlyperformed if the responder thinks it is under DoS attack.In IKEv2 the responder saves state at message 3. Arcanumonly consists of four messages and the responder saves stateafter message 3 which is equivalent to message 5 of IKEand IKEv2. Message 1 of Arcanum contains Phase 1 ��
proposals, initiator’s ��� and a nonce. Message 2 com-prises of an �� containing selected proposal, responders��� taken from the pre-calculated ���� stack, an index(���) to the ���� stack and a nonce. The responder in-cludes a strong cookie (Eqn. 1) in message 2. In Eqn. 1, �is a local secret and is changed frequently to counter cookiejar attack. Maximum fields of the first two messages are
included in the cookie calculation for their integrity protec-tion. ��� serves two purposes. First, the initiator doesnot have to echo the responder’s complete���, hence sav-ing message space. Second, it protects against replay at-tacks by providing freshness to the cookie. After message2, the responder can delete all state. Receiving a duplicatemessage 1 is just like receiving a new message 1.
The initiator upon receiving message 2 echoes it backin message 3, except for the responder’s ���, and repeatsits �� proposal and nonce. The rest of message 3 is en-crypted and contains the initiator’s identity, the intendedresponder’s identity, the initiator’s certificate, a request forthe responder’s certificate, Phase 2 �� proposals and theinitiator’s ��. The entire message is authenticated with���� . Upon receiving message 3, the responder firstchecks if the value of ��� is currently in use. If not,it drops the message otherwise it retrieves the correspond-ing ��� value from the ���� stack and recalculates thecookie for verification. If the cookie is valid, the responderproceeds with calculating the ���� and derives all the re-quired keys. Then it verifies the authenticator and decryptsthe encrypted part and verifies the initiator’s ��. If thesignature is valid the responder replies with an encryptedmessage 4 containing its identity, its certificate, an �� con-taining selected proposal for Phase 2 and its ��. Message4 is also authenticated with ���� . The following keyingmaterial is derived for a Phase 1 ��.
�� � ������ �������� ������������
���������������� (1)
�� � ������������ �����
��� � ������� ������������� ���� ������
�� � ������� ����������������� ���� ������
��� � ������� ���������������� ���� ������
���� � ������������������������� ����
�������������
�� � �������� � � �
�� � ������ � ��� �� � ������ ���� � � �
���� � ��������������� ��� � ��������
Where �� can be ��� or �� depending upon whichkey is calculated. �� and � are the first required numberof bits of ��. Note that different keys are being used inthe two directions of the �� to avoid reflection attacks.
5.2. SA Establishment using PSK
Fig. 2 shows the message exchanges of ��� mode ofIKE, IKEv2 and Arcanum. IKE comprises of six mes-sages and the responder saves state at message 1. IKEv2is similar to its �� mode except that instead of �� it uses���� (calculated with ���) for authentication. Encryp-tion starts from message 5 in IKE and IKEv2 whereas in Ar-canum, the very first message is encrypted (excluding�)
Figure 2. PSK authentication.with a key derived as per Eqn. 3. Encrypted part of message1 comprises of Phase 1 �� proposals, initiator’s ��� anda nonce. ��� (Eqn. 2) is an identifier to the ��� beingused and is different every time a new �� is established.Since only a legitimate initiator is able to produce a valid���, a DoS attack cannot be launched against this mode.The ���s must be unguessable and unlinkable. To avoidthe same ��� being used by both the peers in simultaneousinitiation of ��s, one peer uses odd numbers and the otheruses even numbers (a similar approach is used in [6]).
The responder varifies the ���, saves state and replies.Message 2 (encrypted with ��) contains an �� with theresponder’s selected proposal, its ����, a nonce and����� which authenticates the responder and also pro-vides integrity protection to ���� and ���� so that the re-sponder is not tricked into selecting the weakest proposal.The initiator verifies ����� and replies with message 3.Message 3 (encrypted and authenticated with keys derivedfrom ����) comprises of the initiator’s identity, the in-tended responder’s identity, a proposal for the Phase 2 ��
and ����� to authenticate the initiator. The responderreplies after verifying the authenticator and �����. Mes-sage 4 is also encrypted and authenticated and comprises ofthe responder’s identity and selected Phase 2 ��. �� is cal-culated according to Eqn. 4 whereas the rest of the keyingmaterial is calculated according to Section 5.1.
��� � ������ ��� ��� � ������ ��� � � � (2)
�� � ������ ��� ��������� (3)
�� � �������������������� (4)
5.3. SA Establishment using PKE
Fig. 3 shows the IKE and Arcanum protocols with ���
authentication. IKEv2 does not have a ��� mode. Ar-canum uses two different types of public keys for the re-sponder. The public key of the gateway (outer identity) as-sociated with the IP address and the public key of the actualresponder. Therefore, the initiator need not know the ac-tual responder’s public key in advance. Advantages of thismethod are that ��� can also be negotiated, ��� valuesare passed encrypted providing additional security and both
Figure 3. PKE authentication.identities are protected against active attacks. Message 1contains Phase 1 �� proposals and ��� options. The re-sponder selects a proposal and a ��� and sends a strongcookie (Eqn. 5) and its IP address certificate ���. Sinceit’s IP address is fixed, it does not require secrecy. Message3 contains the proposed �� and selected ��, the initia-tors nonce encrypted with ���. The rest of the message isencrypted with a symmetric key �� derived from the ini-tiator’s nonce (Eqn. 7) and contains the initiators identity,��� value, optionally the initiators certificate, the intendedresponder’s identity, a second nonce encrypted with the in-tended responder’s public key in case it is available and theinitiator wants to establish keys directly with the responder.Message 3 is also authenticated with �� (Eqn. 8).
The responder varifies its cookie in message 3, savesstate and replies with message 4 containing its nonce en-crypted with ���. The rest of the message is encryptedwith a symmetric key �� (Eqn. 7) and contains its identityand ���. Message 5 is encrypted with a key derived fromthe ���� and contains ����� to authenticate the initia-tor, proposals for the Phase 2 �� and optionally a ���
value in case ��� is desired for the Phase 2 ��. Note thatthis option was not included in �� and ��� modes sincethe message size was already too large. Message 6 is simi-lar to 5 except that it contains a single selected �� proposal.�� is calculated according to Eqn. 6 and the remaining key-ing material is calculated according to Section 5.1.
�� � ���� ��� ����������������������� (5)
�� � ���������� ���������������� (6)
�� � ������� ������ (7)
�� � ��������������� (8)
6. ResultsWe have tested a prototype implementation of Arcanum
protocol in Java (code available at [9]). We tested the pro-tocol by launching two types of attacks namely, IP floodingDoS attack (message 1 only) and Man-in-the-Middle DoSattack (message 1 and message 3). The attacks were notsuccessful in providing DoS but only caused a minor delayin the legitimate SA establishment. The delay was more inthe case of the latter attack as compared to the former attack.
Arcanum uses strong cookies in �� and ��� mode tocounter IP flooding attacks. In ��� mode, a valid ���is required for each acceptable message 1. Therefore, thisattack is not possible against ��� mode. Arcanum is ro-bust to Man-in-the-Middle DoS attacks because it does notrequire the responder to perform heavy computations be-fore the initiator has authenticated itself. An attacker havingcontrol of a router can easily launch a CPU exhaustion DoSattack against the remaining key exchange protocols. Theattacker sends message 1, receives a reply and sends mes-sage 3 to make the responder calculate ���� and all thekeying material only to find out that the initiator is illegiti-mate. The attacker repeats the attack with different IP ad-dresses causing DoS. Arcanum is robust to such attacks be-cause it calculates a new ���� only if a pair out of its pre-calculated stack has been used in a successful ��. ��is not compromised here because a new ���� is used forevery successful ��. JFK counters this situation by reusingthe ���� and thus compromises on ��.
7.2. Protection Against Other Attacks
Reflection attack is not possible against Arcanum be-cause all ��s are unidirectional. Moreover, it has a mes-sage counter to differentiate requests from responses anduses sequence numbers to differentiate between different��s in progress. Replay attack is countered through theuse of strong cookies. Strong cookies also provide protec-tion against three other attacks. First, they provide integrityprotection to maximum fields of message 1 and message2, including the ����. This provides protection against anattack in which the responder is tricked into selecting theweakest �� proposal. Second, a cookie jar attack cannotbe launched against the protocol because the value of localsecret is changed frequently. Third, all the fields required bythe responder to recalculate its cookie appear first in mes-sage 3. In case it is fragmented the responder can calculateits cookie from the first fragment to decide whether it shouldwait for the remaining fragments or not.
7.3. Extra Security in PSK Mode and Efficiency
Arcanum fully exploits the power of ��� authentica-tion by using ��� which provides a low level authentica-tion of the initiator right at the first message. When a mes-sage with a valid ��� arrives, the responder knows that itis most likely a legitimate initiator. The �� and ��� pay-loads are encrypted in message 1 and message 2 providingextra security. Arcanum is more efficient in terms of timeand message sizes. It completes the protocol in minimumnumber of messages without compromising on the securityof the protocol. The responder replies instantly to message1, by using a���� from a stack which is filled up when the
processor is idle. The message sizes are greatly reduced byintroducing���� and by using protocol suit identifiers in�� payload.
8. ConclusionWe have identified the security holes and design weak-
nesses of IKE and have presented a brief literature reviewof other famous key exchange protocols. We have alsoproposed a new key exchange protocol, Arcanum which ismore secure, efficient and robust to DoS attacks. Somequalitative results have also been presented. Finally, wehave carried out a detailed security analysis of Arcanum andhave compared it to IKE, JFK and IKEv2.
References[1] W. Aiello, S. Bellovin, M. Blaze, R. Canetti, J. Loannidis, A.
Keromytis and O. Reingold, “Efficient, DoS-Resistant, Se-cure Key Exchange for Internet Protocols”, 9th ACM Confer-ence on Computer and Comm. Security, pp. 48–58, 2002.
[2] D. Harkins and D. Carrel, “The Internet Key ExchangeProtocol (IKE)”, IETF RFC 2409, Nov, 1998. available athttp://www.ietf.org/rfc/rfc2409.txt
[3] D. Harkins, C. Kaufman, S. Kent, T. Kivinen and R. Perlman,“Proposal for the IKEv2 Protocol”, Internet Draft (draft-ietf-ipsec-ikev2-02.txt), April, 2002 (expired Oct, 2002).
[4] S. Kent and R. Atkinson, “Security Architecture for the Inter-net Protocol (IPSec)”, IETF RFC 2401, Nov, 1998. availableat http://www.ietf.org/rfc/rfc2401.txt
[5] H. Krawczyk, “SKEME: a versatile secure key exchangemechanism for Internet”, IEEE Symposium on Network andDistributed System Security, pp. 114–117,1996.
[6] H. Krawczyk, “The IKE-SIGMA Protocol”, Internet Draft(draft-krawczyk-ipsec-ike-sigma-00.txt), Nov, 2001 (expiredMay, 2002).
[7] D. Maughan et al., “Internet Security Association and KeyManagement Protocol (ISAKMP)”, IETF RFC 2408, Nov,1998. available at http://www.ietf.org/rfc/rfc2408.txt
[8] C. Meadows, “Analysis of the IKE Protocol Using NRL Pro-tocol Analyzer”, IEEE Symposium on Security and Privacy,pp. 216–231, 1999.
[9] A. S. Mian, “Implementation of Arcanum Protocol”,http://www.cs.uwa.edu.au/�ajmal/key.html, January, 2004.
[10] H. Orman, “The Oakley Key Determination Pro-tocol”, IETF RFC 2412, Nov, 1998. available athttp://www.ietf.org/rfc/rfc2412.txt
[11] R. Perlman and C. Kaufman, “Key Exchange in IPSec: Anal-ysis of IKE”, IEEE Internet Computing, Vol. 4, No. 6, pp.50–56, 2000.
[12] R. Perlman and C. Kaufman, “Analysis of the IPSec Key Ex-change Standard”, Tenth IEEE International Workshop on En-abling Technologies: Infrastructure for Collaborative Enter-prises, pp. 150–156, 2001.
[13] P.K. Qualum and W. Simpson and DayDreamer, “Photuris:Session Key Management Protocol”, IETF RFC 2522, March,1999. available at http://www.ietf.org/rfc/rfc2522.txt
