This post disappeared from a previous blog of mine so I'm reposting it.
—
A couple of weeks ago I was having difficulty turning on Windows Integrated security for a couple of virtual directories that were interfaces to the SOAP and HTTP adapter. I initially had Anonymous and Integrated Windows turned on. As soon as I turned off anonymous and left only Integrated Windows security turned on I would get prompted to enter my credentials. After three failed attempts to login I would get the following error message: ‘HTTP Error 401.2 – Unauthorized: Access is denied due to server configuration.’
I assumed that I didn’t have the ACLs properly configured on the directories and files that the virtual directories managed but after reviewing their configuration everything checked out.
I stumbled across a reference to AuthDiag, a new tool from MS that helps you troubleshoot authentication and authorization issues with IIS 5.x and 6.0. I installed the tool, ran the ‘Check Authentication’ test on the virtual directories and quickly discovered that my issues had something to do with the fact the identity for the application pools that my virtual directories were running in wasn’t configured properly. Auth Diag kept giving me a failure with the message ‘Service principal name (SPN) for user xx\yy not found in Active Directory’ where xx was the domain and yy was the user name for the account I was using as the application pool identities. (As you probably know, if you have multiple BizTalk Servers in your BizTalk Group then you must use domain accounts for you host instance credentials and I was using the same account for the application pool identities.)
After a suggestion from a colleague, I switched the identities for the application pools to a local account (Network Service) and the prompt and 401 immediately went away. Of course the requests couldn’t make it into BizTalk because the local Network Service Account didn’t have the permissions to reach the BizTalk MessageBox database.
Another colleague suggested using wfetch.exe, part of the IIS 6.0 Resource Kit to help debug my issue. I ran wfetch.exe using NTLM (for the ‘Auth’ value in the ‘Authentication’ group box) and could access the virtual directories without problems. However, when using either Kerberos or Negotiate I saw the following error message in the response from the IIS: ‘0x80090322 (The target principal name is incorrect.): Unable to InitializeSecurityContext’. This article describes the differences between these three options although it describes these in the context of the IIS authentication and not wfectch.exe
I was stumped until I found the Configuration Application Pool Identity in the IIS 6.0 documentation. According to that page, there are issues with using Kerberos (which may be used when you use Integrated Windows Security.) The solution to my problem was that I needed to run the setspn.exe tool in order to set a Service Principal Name (SPN) on the account configured as the Identity for the application pools. We ran setspn.exe using the machine name and the machine’s fully qualified domain name, and the issue immediately went away:
setspn -A HTTP/servername domain\account
setspn -A HTTP/servername.fullyqualifieddomainname domain\account
You can find the setspn.exe tool on the Windows 2003 CD-ROM or .ISO at \support\tools\suptools.msi
One thing to note about setspn.exe is that you must be a domain administrator, of the domain that the account is in, that you use as a parameter to setnspn.exe (which is also the account you are using for the application pool identity. There is a note here (click on the ‘Troubleshooting Setspn’ link) that says the permission necessary for running the setspn.exe can be delegated.
In summary, you must ex