This is a continuation post of part1 and part2 of my "Integrated Windows Authentication blog series" and last one in this series where we are going to discuss about what we can do when Kerberos Authentication fails, how to detect it and correct it! Let me start by mentioning this --> C:\Windows\System32\Wininet.dll file calls the InitializeSecurityContext function to build the Kerberos ticket. The InitializeSecurityContext (Kerberos) function initiates the client side, outbound security context from a credential handle. The function is used to build a security context between the client application and a remote peer. InitializeSecurityContext (Kerberos) returns a token that the client must pass to the remote peer, which the peer in turn submits to the local security implementation through the AcceptSecurityContext (Kerberos) call. The token generated should be considered opaque by all callers. Typically, the InitializeSecurityContext (Kerberos) function is called in a loop until a sufficient security context is established. From <https://msdn.microsoft.com/en-us/library/aa375507(VS.85).aspx > I have explained in my Part1 post about how the kerberos authentication works, I strongly recommend reading that first before troubleshooting so we know what we are doing. So, when the Kerberos AUTHETICATION FAILS or maybe not yet, This is how you verify/check/resolve : Firewall Configuration:
Transcript
This is a continuation post of part1 and part2 of my "Integrated Windows Authentication blog series" and last one in this series where we are going to discuss about what we can do when Kerberos Authentication fails, how to detect it and correct it! Let me start by mentioning this --> C:\Windows\System32\Wininet.dll file calls the InitializeSecurityContext function to build the Kerberos ticket.
The InitializeSecurityContext (Kerberos) function initiates the client side, outbound security context from a credential handle. The function is used to build a security context between the client application and a remote peer. InitializeSecurityContext (Kerberos) returns a token that the client must pass to the remote peer, which the peer in turn submits to the local security implementation through the AcceptSecurityContext (Kerberos) call. The token generated should be considered opaque by all callers.Typically, the InitializeSecurityContext (Kerberos) function is called in a loop until a sufficient security context is established.
From <https://msdn.microsoft.com/en-us/library/aa375507(VS.85).aspx>
I have explained in my Part1 post about how the kerberos authentication works, I strongly recommend reading that first before troubleshooting so we know what we are doing.
So, when the Kerberos AUTHETICATION FAILS or maybe not yet, This is how you verify/check/resolve :
Firewall Configuration:Open firewall ports to allow HTTP traffic in on default and non-default ports. Ensure clients can connect to Kerberos Ports on the Active Directory. Before testing authentication, ensure clients can access the SharePoint Server web applications on the configured HTTP ports. In addition, ensure clients can authenticate with Active Directory and request Kerberos tickets from the KDC over the standard Kerberos ports. Open firewall ports to allow HTTP traffic in on default and non-default ports:Typically you have to configure the firewall on each front-end Web to allow incoming requests over ports TCP 80 and TCP 443. Open Windows Firewall with Advanced Security and browse to the
following Inbound Rules: World Wide Web Services (HTTP Traffic-In) World Wide Web Services (HTTPS Traffic-In)
Make sure the appropriate ports are open in your environment. In our example, we access SharePoint Server over HTTP (port 80), so this rule was enabled. In addition, we have to open the non-default port used in our example (TCP 5555). If you have web sites running on non-default ports, you also have to configure custom rules to allow HTTP traffic on those ports. Ensure that clients can connect to Kerberos ports on the Active Directory roleTo use Kerberos authentication, clients will have to request ticket granting tickets (TGT) and service tickets (ST) from the Key Distribution Center (KDC) over UDP or TCP port 88. By default, when you install the Active Directory Role in Windows Server 2008 and later, the role will configure the following incoming rules to allow this communication by default:
Kerberos Key Distribution Center – PCR (TCP-In) Kerberos Key Distribution Center – PCR (UDP-In) Kerberos Key Distribution Center (TCP-In) Kerberos Key Distribution Center (UDP-In)
In your environment ensure these rules are enabled and that clients can connect to the KDC (domain controller) over port 88.
From <https://technet.microsoft.com/en-us/library/gg502602(v=office.14).aspx>
Test browser authentication: After configuring Active Directory, DNS and SharePoint Server you can now test whether Kerberos authentication is configured correctly by browsing to your web applications. When testing in the browser, ensure the following conditions are met:
1. The test user is logged into a Windows 7 or higher computer joined to the domain that SharePoint Server is installed in, or is logged into a domain trusted by the SharePoint Server domain.
2. Integrated Windows authentication is enabled in the browser. Under Internet Options in the Advanced tab, make sure Enable Integrated Windows Authentication* is enabled in the Security section:
3. Local intranet is configured to automatically logon clients. Under Internet explorer option, in the Security tab, select Local Intranet and click the Custom level button. Scroll down and make sure that Automatic logon only in Intranet zone is selected.
4. Ensure that Automatically detect intranet network is selected in Internet options->Security->Intranet Zone->Sites.
5. If you are using fully qualified domain names to access the SharePoint Server web applications, ensure that the FQDNs are included in the intranet zone, either explicitly or by wildcard inclusion (for example, “*.vmlab.local”).
The easiest way to determine if Kerberos authentication is being used is by logging into a test workstation and navigating to the web site in question. If the user isn’t prompted for credentials and the site is rendered correctly, you can assume Integrated Windows authentication is working.
From <https://technet.microsoft.com/en-us/library/gg502602(v=office.14).aspx>
If any of the above is not set properly, then Delegation fails even when Kerberos authentication works:o Check Internet Explorer Zone used for the URL.
Kerberos delegation is only allowed for the « Intranet » and « Trusted Sites » zones (in other words, IE sets the ISC_REQ_DELEGATE flag when it calls InitializeSecurityContext only if the zone computed is « Intranet » or « Trusted Sites » )
o The account used for the application pool identity must have the « Trusted for delegation » flag set
If delegation still fails, you may consider using DELEGCONFIG. This tool allows to diagnose - and resolve - dozens of issues preventing Kerberos authentication from working correctly. Check the following link for more details: http://blogs.iis.net/bretb/archive/2008/03/27/How-to-Use-DelegConfig.aspx
The next step is to determine if the negotiate protocol was used to negotiate Kerberos authentication as the authentication provider for the request. This can be done in the following ways:
SharePoint server Front-end Windows security logs If Kerberos authentication is working correctly you will see Logon events in the security event logs on the front-end webs with event ID = 4624. In the general information for these events you should see the security ID being logged onto the computer and the Logon Process used, which should be Kerberos. Event is here below: Log Name: SecuritySource: Microsoft-Windows-Security-AuditingDate: 5/6/2018 10:27:48 PMEvent ID: 4624Task Category: LogonLevel: InformationKeywords: Audit SuccessUser: N/AComputer: SP.contoso.comDescription:An account was successfully logged on. Subject:
We should see same event on KDC (Domain controller) too:
Check if the IIS server is actually configured to send the WWW-Authenticate: Negotiate header: If IIS doesn't send this header, you'll need to use ADSUTIL to set the Negotiate header though the NTAuthenticationProviders metabase property (see http://support.microsoft.com/kb/215383)Note: by default, the NTAuthenticationProviders property is not set which causes IIS to send both Negotiate and NTLM headers:
To check or set the property: Locate the directory that contains the Adsutil.vbs file. By default, this directory is C:\Inetpub\Adminscripts.Use the following command to retrieve the current values for the NTAuthenticationProviders metabase property: cscript adsutil.vbs get w3svc/WebSite/root/NTAuthenticationProvidersIn this command, WebSite is a placeholder for the Web site ID number. The Web site ID number of the default Web site is 1.
For Example: PS C:\Users\Administrator.contoso> cscript "C:\inetpub\AdminScripts\adsutil.vbs" get w3svc/1834077867/root/NTAuthenticationProvidersMicrosoft (R) Windows Script Host Version 5.8Copyright (C) Microsoft Corporation. All rights reserved. NTAuthenticationProviders : (STRING) "Negotiate,NTLM" If the command above does not return the string "Negotiate,NTLM," use the following command to enable the Negotiate process: cscript adsutil.vbs set w3svc/WebSite/root/NTAuthenticationProviders "Negotiate,NTLM"
Example: cscript adsutil.vbs set w3svc/1834077867/root/NTAuthenticationProviders "Negotiate,NTLM"
Warning: If you edit the metabase incorrectly, you can cause serious problems that may require that you reinstall any product that uses the metabase. Microsoft cannot guarantee that problems that result if you incorrectly edit the metabase can be solved. Edit the metabase at your own risk.Note: Always back up the metabase before you edit it.
From <https://support.microsoft.com/en-us/help/215383/how-to-configure-iis-to-support-both-the-kerberos-protocol-and-the-ntl>
Check if the client and server are on the same machine: Kerberos is not enabled in this configuration and a hard coded loopback check will always force usage of NTLM in this scenario. Note that NTLM may also not work in this configuration (see http://support.microsoft.com/kb/896861 for more details). Workarounds:Set DisableStrictNameChecking to 1 And Either [Preferred method if NTLM is needed]Create "Multi-String Value" BackConnectionHostNames = "HostName of the site" at location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0ORCreate Dword "DisableLoopbackCheck" = 1 at location HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
Duplicate or Wrong Service principal name set for Application Pool identity: If a wrong SPN is set for the APP Pool identity or if there are Duplicate SPNs set, kerberos AUTHENTICATION fails because it cannot determine which account to use to Decrypt the Ticket. We can use these SETSPN.EXE commands to check any SPN related issues: SETSPN -L [Application Pool account] --> Will list all SPNs registered for that account.SETSPN -X --> Will show any Duplicate SPNsSETSPN -Q --> Can be used to Query for Specific or all SPNs. SETSPN.EXE Usage: C:\Windows\system32\setspn.exe [modifiers switch] [accountname] Where "accountname" can be the name or domain\name of the target computer or user account Edit Mode Switches: -R = reset HOST ServicePrincipalName
Usage: setspn -R accountname -S = add arbitrary SPN after verifying no duplicates exist Usage: setspn -S SPN accountname -D = delete arbitrary SPN Usage: setspn -D SPN accountname -L = list SPNs registered to target account Usage: setspn [-L] accountname Query Mode Switches: -Q = query for existence of SPN Usage: setspn -Q SPN -X = search for duplicate SPNs Usage: setspn -X Some Example commands with outputs are as below: PS C:\Users\Administrator.contoso> SetSPN -S HTTP/Kerberos.contoso.com:9090 Contoso\SPAppPoolChecking domain DC=contoso,DC=com Registering ServicePrincipalNames for CN=SPAppPool,CN=Users,DC=contoso,DC=com HTTP/Kerberos.contoso.com:9090Updated object PS C:\Users\Administrator.contoso> setspn -l contoso\SPAppPoolRegistered ServicePrincipalNames for CN=SPAppPool,CN=Users,DC=contoso,DC=com: HTTP/Kerberos.contoso.com:9090 HTTP/Kerberos:9090 HTTP/Kerberos.contoso.com HTTP/Kerberos PS C:\Users\Administrator.contoso> SetSPN -S HTTP/Kerberos Contoso\SPAppPoolChecking domain DC=contoso,DC=comCN=SPAppPool,CN=Users,DC=contoso,DC=com HTTP/Kerberos.contoso.com:9090 HTTP/Kerberos:9090 HTTP/Kerberos.contoso.com HTTP/Kerberos Duplicate SPN found, aborting operation! PS C:\Users\Administrator.contoso> setspn -Q http/spChecking domain DC=contoso,DC=comNo such SPN found.
PS C:\Users\Administrator.contoso> setspn -Q */SQLChecking domain DC=contoso,DC=comCN=SQL,CN=Computers,DC=contoso,DC=com MSSQLSvc/SQL.contoso.com:1433 MSSQLSvc/SQL.contoso.com TERMSRV/SQL TERMSRV/SQL.contoso.com WSMAN/SQL WSMAN/SQL.contoso.com RestrictedKrbHost/SQL HOST/SQL RestrictedKrbHost/SQL.contoso.com HOST/SQL.contoso.com Existing SPN found! PS C:\Users\Administrator.contoso> setspn -Q */spChecking domain DC=contoso,DC=comCN=SP,CN=Computers,DC=contoso,DC=com TERMSRV/SP TERMSRV/SP.contoso.com WSMAN/SP WSMAN/SP.contoso.com RestrictedKrbHost/SP HOST/SP RestrictedKrbHost/SP.contoso.com HOST/SP.contoso.com Existing SPN found! PS C:\Users\Administrator.contoso> setspn -XChecking domain DC=contoso,DC=comProcessing entry 0found 0 group of duplicate SPNs. So, for same Service there should not be 2 different Service accounts for which SPNs are registered otherwise IIS will not know which account to decrypt from.
Tools:
A. KlistB. Kerberos Authentication testerC. FiddlerD. Microsoft network monitor
A. Klist.EXE:
this command-line tool helps you list and purge tickets on your client machine. You can list tickets to verify the validity of tickets. You can also check for other details, such as the permissible flags, SPN type, the client, and server details.
KLIST output (in this example, for a Site URL: http://kerberos:9090): PS C:\Users\Administrator.contoso> klist Current LogonId is 0:0x16958c Cached Tickets: (2) #0> Client: Administrator @ CONTOSO.COM Server: krbtgt/CONTOSO.COM @ CONTOSO.COM KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96 Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize Start Time: 5/6/2018 21:43:29 (local) End Time: 5/7/2018 7:43:29 (local) Renew Time: 5/13/2018 21:43:29 (local) Session Key Type: AES-256-CTS-HMAC-SHA1-96 Cache Flags: 0x1 -> PRIMARY Kdc Called: DC.contoso.com #1> Client: Administrator @ CONTOSO.COM Server: HTTP/kerberos @ CONTOSO.COM KerbTicket Encryption Type: RSADSI RC4-HMAC(NT) Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize Start Time: 5/6/2018 21:43:29 (local) End Time: 5/7/2018 7:43:29 (local) Renew Time: 5/13/2018 21:43:29 (local) Session Key Type: RSADSI RC4-HMAC(NT) Cache Flags: 0 Kdc Called: DC.contoso.com So, if server does not have same Decryption mechanism (Say RC4) so it can't decrypt tickets. In that case, make sure you enable AES encryption for Service account properties.Run these two commands below if you want to Purge user and System level Kerberos Tickets. PS C:\Users\Administrator.contoso> klist purge Current LogonId is 0:0x16958c Deleting all tickets: Ticket(s) purged! PS C:\Users\Administrator.contoso> klist -li 0x3e7 purge
Current LogonId is 0:0x16958cTargeted LogonId is 0:0x3e7 Deleting all tickets: Ticket(s) purged!
B. Kerberos Authentication Tester (It can be found here:
It shows what authentication method is used in a web request: None, Basic, NTLM or Kerberos It shows the SPN used in case of Kerberos It shows the HTTP status It shows the HTTP Headers of the request. It shows the version of NTLM used (v1 or v2) It has a detailed view with a complete breakdown of the Authorization header. (Yep, went
through all the RFCs to dissect the Kerberos and NTLM packages) It shows your current Kerberos tickets and allows you to remove them (like klist.exe)
The blue details link at the top right shows a new screen with the details of the request, including a breakdown of the Authorization HTTP Header. You can also save this XML.
It shows all kerb Tickets.
C. Network Monitor/WireShark: network trace capture and review tools.
You can review trace to identify Kerberos errors.
Best way to get the network capture is: 1. To reduce the possibility of caching data, do one of the following:•Close/Reopen client application•Logoff/Logon client workstation•Reboot client workstation2. Start the network capture3. Clear DNS cache using:ipconfig /flushdns4. Clear NetBIOS cache using:nbtstat –RR5. Clear user Kerberos tickets using:klist purge
6. Clear system / computer Kerberos tickets using (Vista or higher only):Klist –li 0x3e7 purge7. Reproduce the authentication failure with the application in question8. Stop the network capture
Now that you have the capture, you can filter the traffic using the string ‘Kerberosv5’ if you are using Network Monitor. If you are using Wireshark, you can filter using the string ‘Kerberos’.To be more thorough, load the Authentication Traffic filter that shows packets containing Kerberos tickets as well. You can do this by clicking the Load Filter button, choose Standard Filters, and then click Authentication Traffic. OR add this Filter: KerberosV5 OR KerberosV5.KrbError.Sname OR KerberosV5.AsReqOR KerberosV5.AsRepOR KerberosV5.TgsReqOR KerberosV5.TgsRep Click 'Apply' to begin filtering…
In my Network trace examples below:I am using SQL as my Client machine192.168.2.50 is DC (KDC)SP is SharePoint server Analysing Network trace………………………. First, we see AS (Authentication Server) request and response.DC given AS response Ticket (TGT) to client. (Service name) Sname requested and sent is krbtgt:Sname: krbtgt/CONTOSO.COM
Then Client uses AS response Ticket to make TGS request (Service ticket) to KDC.TGS Response ticket (Service Ticket) is sent to client by KDC.(Service name) Sname requested and sent is HTTP:Sname: HTTP/WebApp name
D.FIDDLER: 1st request will be anonymous if user is not authenticated.401 is what we see first ……………..
Then we see 302 client Presenting kerberos Ticket to Server (SP) where server responds with a Kerberos Reply after decrypting the ticket with App Pool Identity.
If the Ticket presented by Client is valid then server moves request (302) to the page Client should be going to.. It could be home page of the Site, as in this example is: Location: http://teams/_layouts/15/start.aspx#/SitePages/Home.aspx
In Next step, Client uses it's Original Kerberos ticket to obtain data.SO, we see 200 …………….. And we get the page..
Then further, client uses same ticket to access the Site resources (pages, links, files etc.).------------------------------------------------------------------------------------------------------------------------------------ And that's all you might need for almost all of your Kerberos related issues - Of Course, wrt SharePoint :)
Happy SharePointing !! Check other Part links in this series too:Part1: Explaining Integrated Windows Authentication in SharePoint and How NTLM fails but Kerberos works in Double-Hop authenticationPart2: Step by step Guide for Configuring Kerberos Authentication in SharePoint 2013/2016 Additional References: Explained: Windows Authentication in ASP.NET 2.0Windows Authentication Deep Dive What Every Administrator Should KnowKerberos Survival GuideKerberos Authentication DemoKerberos ExplainedHow the Kerberos Version 5 Authentication Protocol WorksKerberos messages and ticketsDouble-hop authentication: Why NTLM fails and Kerberos worksKerberos configuration known issues (SharePoint Server 2010)
Things to check when Kerberos authentication fails using IIS/IE…Kerberos and LDAP Error MessagesKerberos errors in network capturesService Principal Name (SPN) checklist for Kerberos authentication with IIS 7.0/7.5Kerberos RFC:http://www.ietf.org/rfc/rfc1510.txt http://www.ietf.org/rfc/rfc4120.txt
