Shibboleth IDP File Permissions

In light of recent developments in the thread landscape involving JNDI, and Java library vulnerabilities such as Log4Shell (CVE-2021-44228), we note the critical importance of file permissions to the Shibboleth deployment.

Note: This document currently discusses only Linux-based deployments of Shibboleth IDP. We believe that the Windows installation handles permissions specific to that system automatically when performing an MSI-based installation procedure outlined by the wiki. We are investigating best practices for Windows installations and will update this document accordingly with our findings.

Linux Service Privileges

For a proper, secure installation of Shibboleth IDP, you should configure the servlet container (Jetty) to run as a non-root user. The proper configuration for this depends upon the servlet container of choice, i.e. for Eclipse Jetty (the recommended container):

• Jetty 9.x – Setting Port 80 Access for a Non-Root User

Linux Filesystem Permissions

The most critical element of the file system permissions is to ensure that the user which runs the servlet container does not have write permissions for the IDP configuration files, except in select circumstances.

Below you’ll find a small shell script to set appropriate permissions for a “standard” deployment, meaning:

• all files are by-default owned by root,
• select permissions are provided to the user account under which the servlet container runs, i.e. jetty,
• Shibboleth is installed in the default location: /opt/shibboleth-idp, and
• the $JETT_BASE directory of the Jetty installation is /opt/jetty-base

#!/bin/sh
chown -R root:jetty /opt/shibboleth-idp/;
chown -R jetty:jetty /opt/shibboleth-idp/{credentials,logs,metadata};
find /opt/shibboleth-idp -type d -exec chmod 750 {} \;
find /opt/shibboleth-idp -type f -exec chmod 640 {} \;
chmod -R 750 /opt/shibboleth-idp/bin;
chmod -R u-w /opt/shibboleth-idp/credentials;
chown -R root:jetty /opt/jetty-base/;
chown -R jetty:jetty /opt/jetty-base/{logs,tmp};
find /opt/jetty-base -type d -exec chmod 750 {} \;
find /opt/jetty-base -type f -exec chmod 640 {} \;

Essentially, the jetty user requires read-access to most files within the IDP installation directory, hence we set root to own those files and jetty as the group, with permissions for the owner (root) to be allowed to edit those files, and the group (jetty) permission to read. Other users have no access.

jetty needs to own the credentials, logs, and metadata directories, with write permissions only for logs and metadata. If jetty doesn’t own credentials it’s unable to unlock the cryptographic keys required for SAML, and will not start. We remove the jetty users permissions to write to those files separately (chmod -R u-w /opt/shibboleth-idp/credentials).

Likewise, jetty needs to be able to write to the logs and tmp directories from $JETTY_BASE.

Essentially, the goal is to allow the jetty user to have the absolute minimal permissions to run the IDP software.

N.B. With this goal in mind, we note that if you do not leverage any <MetadataProvider> elements which fetch metadata from external locations, you shouldn’t even need to allow write access for the metadata directory. One option is to ensure that you specify a backingFilePath to a directory other than <IDP-HOME>/metadata, i.e. something like /var/cache/shibboleth. Then only that directory needs write access for jetty.

Need help assessing a security issue? Contact us to discuss your needs. IDM Engineering is a team of dedicated, honest SSO support engineers that are standing by to help!

Shibboleth Logging 101

If you’re installing, configuring, or managing a single sign-on environment, you will inevitably find yourself wanting (or needing) to understand what’s going on under the hood. That’s where log files come in. In this space, we’ve collected some useful general information about Shibboleth logging for both:

• Shibboleth Logging 101
  • Shibboleth IdP Logging
    • Logging Options for idp.properties
    • “Debug” Logging
  • Shibboleth SP Logging
    • “Debug” Logging
    • Debugging assertions

Shibboleth IdP Logging

Logging on Shibboleth IdP is implemented via an abstract layer (SLF4J) which directs control of logging to the Logback facility. Since the project depends upon these logging implementations, Shibboleth is somewhat beholden to configuring via these external methods. Thankfully, they are relatively generic and highly customizable.

Logging is configured in %{idp.home}/conf/logback.xml, where %{idp.home} is the location where Shibboleth IdP is installed (typically, and by default, /opt/shibboleth-idp). Importantly, you don’t usually need to adjust this file unless you want to make specific changes to the logging constructs, e.g. changing the format of the logged strings. Most of the major settings you’ll need to adjust can be edited from %{idp.home}/conf/idp.properties.

Logs are stored within %{idp.home}/logs.

Logging Options for idp.properties

“Debug” Logging

When we say “turn up logging to DEBUG” we really mean that you should adjust one or more of the above properties in order to see more useful information. There’s no fixed set, but in general:

• If you’re doing any kind of debugging you should set idp.loglevel.idp = DEBUG
• If you want to see the actual SAML assertions, you should use a combination such as:

idp.loglevel.idp = DEBUG
idp.loglevel.messages = DEBUG
idp.loglevel.opensaml = DEBUG
idp.loglevel.encryption = DEBUG

• If you’re working on an issue with a data connector or attribute resolver, you might find:

idp.loglevel.idp = DEBUG
idp.loglevel.ldap = INFO

• to be all that you really need, however, you can always take idp.loglevel.ldap to DEBUG as well (though be aware, it’s quite chatty).

WARNING: There is no reason to keep debug logging turned on in a production environment. This is especially true if you are capturing raw or decrypted SAML assertions. Don’t do it. Tune things back to default by commenting out your changes when you’re done! You have been warned.

More information about Shibboleth IdP Logging can be found on the wiki!

Shibboleth SP Logging

The Shibboleth SP software writes to two separate diagnostic log files by default, as configured by the shibd.logger and native.logger logging setup files. The first file governs most of the interesting “SAML” bits, like assertion receipt, decryption, and attribute resolution. These events will be logged into a file named shibd.log within the default log directory (unless modified):

• Linux systems: /var/log/shibboleth
• Windows systems: C:\opt\shibboleth-sp\var\log\shibboleth

native.logger controls messages related to RequestMapping, and more often than not isn’t needed. However, once caveat is that on Windows systems using IIS the default configuration leads to no creation of a simple native.log file. This can be easily addressed.

“Debug” Logging

Overall behavior is specified by the log4j.rootCategory parm in shibd.logger, which is by default:

log4j.rootCategory=INFO, shibd_log, warn_log

bumping this to DEBUG is minimally necessary for most debugging.

Debugging assertions

If you are interested in seeing the SAML assertions themselves, set:

log4j.category.OpenSAML.MessageDecoder=DEBUG
log4j.category.OpenSAML.MessageEncoder=DEBUG

by un-commenting these lines in shibd.loggger.

WARNING: There is no reason to keep debug logging turned on in a production environment. This is especially true if you are capturing raw or decrypted SAML assertions. Don’t do it. Tune things back to default by commenting out your changes when you’re done! You have been warned.

More information about Shibboleth SP Logging can be found on the wiki!

Need help debugging a Shibboleth issue? Contact us to discuss your needs. IDM Engineering is a team of dedicated, honest SSO support engineers that are standing by to help!

Assessing Attribute Release Policies with AACLI

Shibboleth Identity Provider (IdP) includes an incredibly useful and powerful tool for determining, without doing an actual authentication sequence, what attributes will be released for a given user (principal) and service provider.

That tool is the Attribute Authority Command Line Interface (AACLI).

You can invoke the AACLI tool by executing a script in the terminal, i.e. {idp.home}/bin/aacli.sh (or {idp.home}/bin/aacli.bat for Windows installations):

[user @ /opt/shibboleth-idp/]$ .bin/aacli.sh -n user -r https://sp.idmengineering.com/shibbolet

If you have correctly configured Access Controls for Administrative Functions, you may access the output of the script via a special resolvertest endpoint as such:

https://localhost.idmengineering.com/idp/profile/admin/resolvertest?principal=jdoe&requester=https%3A%2F%2Fsp.example.org%2Fsp

This URL you could access via curl for use in custom scripting.

Parameters

Examples

Shell Script, Simple Output

[idp.idmengineering.com shibboleth-idp]# ./bin/aacli.sh -n john -r https://sp.idmengineering.com/shibboleth

Result:

{
"requester": "https://sp.idmengineering.com/shibboleth",
"principal": "john",
"attributes": [


  {
    "name": "uid",
    "values": [
              "John"          ]
  },

  {
    "name": "mail",
    "values": [
              "support@idmengineering.com"          ]
  },

  {
    "name": "sn",
    "values": [
              "Doe"          ]
  }

]
}

Shell Script, Output formatted as SAML 2.0 Assertion

[idp.idmengineering.com shibboleth-idp]# ./bin/aacli.sh -n john -r https://sp.idmengineering.com/shibboleth

Result:

<?xml version="1.0" encoding="UTF-8"?>
<saml2:Assertion ID="_057aa390d3cebb0d9c7b90524667edd1"
    IssueInstant="2020-09-18T16:20:55.242Z" Version="2.0" xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion">
    <saml2:Issuer>https://localhost.idmengineering.com/idp/shibboleth</saml2:Issuer>
    <saml2:Subject>
        <saml2:NameID
            Format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient"
            NameQualifier="https://localhost.idmengineering.com/idp/shibboleth" SPNameQualifier="https://sp.idmengineering.com/shibboleth">AAdzZWNyZXQx49glc8r4c80yYO2LWKJ9yHk4GV3IzMIZvBYsEKNnbmxuRfySoLSAZBu7H3OTxNzJKTPIpTJ0o2Ye9YnyMIve0at0+QWNSGz/Rjuu1PW/wvse24m40MFlYWQoWu2EDO5cmYWYUWze/jBPtuyCN0XqM6MJczyAujM=</saml2:NameID>
    </saml2:Subject>
    <saml2:AttributeStatement>
        <saml2:Attribute FriendlyName="uid"
            Name="urn:oid:0.9.2342.19200300.100.1.1" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
            <saml2:AttributeValue>John</saml2:AttributeValue>
        </saml2:Attribute>
        <saml2:Attribute FriendlyName="mail"
            Name="urn:oid:0.9.2342.19200300.100.1.3" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
            <saml2:AttributeValue>support@idmengineering.com</saml2:AttributeValue>
        </saml2:Attribute>
        <saml2:Attribute FriendlyName="sn" Name="urn:oid:2.5.4.4" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
            <saml2:AttributeValue>Doe</saml2:AttributeValue>
        </saml2:Attribute>
    </saml2:AttributeStatement>
</saml2:Assertion>

URL Request, Simple Output

URL: https://localhost.idmengineering.com/idp/profile/admin/resolvertest?principal=john&requester=https%3A%2F%2Fsp.idmengineering.com%2Fshibboleth

URL Request, Output formatted as SAML 2.0 Assertion

URL: https://localhost.idmengineering.com/idp/profile/admin/resolvertest?saml2&principal=john&requester=https%3A%2F%2Fsp.idmengineering.com%2Fshibboleth

Attribute Resolution Trouble?

You can use AACLI to debug issues related to attribute release… but why is a given attribute not being released? Here are some common issues:

• The attribute isn’t being provided by a data connector. This is perhaps because the attribute is null for that principal.
• There is no attribute definition defined for that attribute.
• The attribute definition does not define a dependency from which to pull the source attribute (i.e. explicitly specify the attribute or say which resolver it’s from).
• The attribute definition is marked as a dependency only attribute and thus is not released from the resolver.
• The attribute definition does not define an encoder appropriate for the given request protocol (i.e. SAML1 encoder exists but SAML2 doesn’t).
• The attribute is being filtered out by the attribute filter policy.

The last point, the lack of an appropriate policy releasing the attribute for a given SP in attribute-filter.xml is the most likely cause of a missing attribute, and as such should most likely be checked first.

Need help debugging an attribute issue? Contact us to discuss your needs. IDM Engineering is a team of dedicated, honest SSO support engineers that are standing by to help!

SameSite Cookies and Shibboleth

Google Chrome v.80 is slated to be deployed to the stable channel on February 4th, 2020. (Note: some sources indicate Feb. 17th as the release target.) With this update comes a fundamental shift in the default handling of cookies within Chrome. Starting with this release, cookies will by default be treated as though they have the property SameSite=lax, instead of this property being unset.

The SameSite cookie attribute is a IETF draft written by Google Inc. which instructs the user-agent not to send the SameSite cookie during a cross-site HTTP request. The aim of the SameSite property is to help prevent certain forms of cross site request forgery. Cross-site HTTP requests are those for which the top level site (i.e. that shown in an address bar) changes during navigation.

The SameSite attribute can take three values:

• strict – only attach cookies for ‘same-site’ requests.
• lax – send cookies for ‘same-site’ requests, along with ‘cross-site’ top level navigations using safe HTTP methods e.g. (GET HEAD OPTIONS TRACE).
• none – send cookies for all ‘same-site’ and ‘cross-site’ requests.

The previous behavior of the Chrome browser would be equivalent to SameSite=None.

Ramification for Shibboleth Identity Provider

Per the Shibboleth Consortium, which conducted extensive testing of the Identity Provider software:

the IdP should continue to function when its cookies are being defaulted to SameSite=Lax by browsers (currently tested on Chrome 78-81 and Firefox 72 with the same-site default flags set). Typically, we have only seen the IdP itself break when the JSESSIONID is set to SameSite=strict, which should not happen apart from when explicitly trying to set SameSite=none with older versions of Safari on MacOS <=10.14 and all WebKit browsers on iOS <=12 (Source)

They go on to list the following scenarios wherein SSO breaks, namely:

• When using client side session storage, with htmlLocalStorage set to false, HTTP-POST SSO will not work (show login page again) with defaulted SameSite=Lax IdP cookies. However, when using client side session storage, with htmlLocalStorage set to true, and all bean references in shibboleth.ClientStorageServices are left as they are, HTTP-POST SSO will work with defaulted SameSite=Lax.
• When using server side session storage, if either htmlLocalStorage or the bean references in shibboleth.ClientStorageServices are commented out, HTTP-POST SSO will not work (show login page again) with defaulted SameSite=Lax. Once again, however, when using sever side session storage, with htmlLocalStorage set to true, or all bean references in shibboleth.ClientStorageServices are left as they are, HTTP-POST SSO will work with defaulted SameSite=Lax.

Therefore, to take the relevant IdP-side steps necessary to guarantee SSO on existing installations of the IdP v3, you should enable the HTML Local Storage plugin whether you use client-side storage or server-side storage. This is achieved by setting the idp.storage.htmlLocalStorage property to true in idp.properties.

See the Shibboleth Wiki section re: StorageConfiguration for more information and the implications of this setting.

For more details on the Consortium’s investigatory work related to SameSite please visit this wiki page.

Ramification for Shibboleth Service Provider

Unfortunately, as noted by the Shibboleth Consortium, the bulk of the issues likely to arise within service provider deployments lie solely within the application space, and are likely outside of the Shibboleth domain.

Nor can we provide any sort of yes/no or good/bad conclusion for anybody as to whether “their system is affected”. That is going to depend entirely on the individual case and the only real answer is to test. (Source)

Effectively, SPs should really just test their systems prior to the expected launch of the SameSite changes in Chrome. You can test in either Firefox or Chrome:

Firefox

• Enter about:config in the URL bar, Accept Risk and Continue
• Type samesite to filter options to display: network.cookie.sameSite.laxByDefault
• Set network.cookie.sameSite.laxByDefault to true

Chrome

• Enter chrome://flags in URL bar
• Type SameSite
• Enable “SameSite by default cookies”

If your application fails to work under the test conditions, you can adjust the relevant (blocked) cookies in order to specify the SameSite=none directive. None of the cookies set by Shibboleth SP should require this directive, however, it is quite likely that you will need to adjust cookies within your application.

In particular, the notes from the Shibboleth Consortium further state that

… a typical source of problems for most applications is going to be load balancer behavior. If you’re using cookies for node affinity, you’re going to have problems with SameSite unless you do something about it.

and in particular, you should adjust your load balancer to specify SameSite=none for these affinity cookies.

Need help understanding how the SameSite cookie attribute affects your application or SSO environment? Contact us to discuss your needs. IDM Engineering is a team of dedicated, honest SSO support engineers that are standing by to help!

Configuring native.log for Shibboleth SP3 on IIS

One of the complaints we receive the most from clients utilizing Shibboleth Service Provider (SP) on Windows Server / IIS, is that native.log is empty or missing.

This log, while under typical operational loads is generally indistinguishable in its content from shibd.log, the native logger provides valuable information about internal request mapper routing. This can vary from unimportant, to relevant, to absolutely critical information when it comes to diagnosing SP issues.

On Windows systems, the default native logging is to the Windows Event Log, which is observable with the Event Viewer (under Application and Services Logs -> Shibboleth).

The following configuration snippet is taken from the distributed native.logger file, and configures the log4j (logging provider for Shibboleth) appender for the Event Viewer:

log4j.appender.native_log=org.apache.log4j.NTEventLogAppender
log4j.appender.native_log.source=Shibboleth Service Provider
log4j.appender.native_log.layout=org.apache.log4j.PatternLayout
log4j.appender.native_log.layout.ConversionPattern=%c %x: %m

org.apache.log4j.NTEventLogAppender triggers the logging to the Windows Event log. You can adjust the other parameters of this logging here if you are content with this behavior.

To get more ‘natural’ behavior, you can instead define the following appender in native.log

log4j.appender.native_log=org.apache.log4j.RollingFileAppender


log4j.appender.native_log.fileName=C:/opt/shibboleth-sp/var/log/shibboleth/native.log

log4j.appender.native_log.maxFileSize=1000000


log4j.appender.native_log.maxBackupIndex=10

log4j.appender.native_log.layout=org.apache.log4j.PatternLayout

log4j.appender.native_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n

and Shibboleth will write this information into the native.log file of your specification. This is the default behavior on Linux-based installs, but isn’t done on Windows because this alone is insufficient. You must also ensure that the file you want Shibboleth to write to is given writable by the IIS service account, IUSR:

The above screenshot demonstrates the permissions that we utilize on one of our dev IIS servers which is writing logs to native.log.

Looking for support with Shibboleth on IIS? Look no further! Contact us for all of you SSO support needs!

LDAP SSL/TLS Config for Shibboleth IdP

Many (if not most) of our clients utilize LDAP as the authentication source for Shibboleth IdP.

That said, we all too often encounter environments without properly configured encryption of the connection. All-to-often this arises because of uncertainty about the options and how to properly configure SSL/TLS for the IdP.

Shibboleth supports two mechanisms for encrypted connections:

TLS

The most basic connection is TLS (transport layer security). Confusingly, this is usually referred to in logs and configs as SSL (even outside of Shibboleth), but does not refer to true SSL by itself. As a protocol, SSL has long been deprecated, but we still use the term colloquially to refer to TLS.

The Shibboleth configuration options (in the ldap.properties file) for TLS look something like this:

idp.authn.LDAP.ldapURL       = ldaps://ldap.example.org:636
idp.authn.LDAP.useStartTLS   = false
idp.authn.LDAP.useSSL        = true

We’ve set idp.authn.LDAP.useSSL to true to indicate that our IdP is to connect via TLS (I know, it’s silly nomenclature), and our ldapURL includes both the protocol specification as ldaps:// and the port number that’s typically used for LDAPS connections (TCP/636).

Start TLS

The second connection type is called StartTLS. In this mode, the connection starts out as clear text communication over the standard port (TCP/389), during which the IdP indicates to the LDAP server that it should communicate via a secure connection. It’s at this point that TLS negotiation occurs, and afterwards communication is secure.

The configuration in Shibboleth looks like this:

idp.authn.LDAP.ldapURL       = ldap://ldap.example.org:389
idp.authn.LDAP.useStartTLS   = true
idp.authn.LDAP.useSSL        = false

Note that the major differences are that we’ve explicitly flagged to use StartTLS as opposed to SSL, and the LDAP URL includes the non-secure port number (TCP/389) as well as a regular ldap:// protocol identifier.

There are advantages to either choice. StartTLS can make networking firewall rules simpler, but not all LDAP deployments support it, and so it’s not a universal mechanism.

What’s important is that it doesn’t matter which mechanism you choose, so long as you do choose one.

No matter how secure you think the internal network is between the IdP server and the LDAP server, you don’t want to trust the data between the two traveling in the clear.

Configuring Reloadable Services for Shibboleth IdP

Shibboleth Identity Provider Version 3 introduced the ability to reload individual services within the Shibboleth framework. Prior to IdP v3, if you wanted to onboard a new Service Provider by adding new <MetadataProvider> and <RelyingParty> elements, you would be required to restart the servlet container. Now, nearly every class of change that you may need to make to the Shib IdP can be done without restarting Jetty.

Reloadable Services

There are two functional methods to achieve this, either a direct request to the administrative handler URL:

http(s)://idp.example.org/idp/profile/admin/reload-service?id=[SOME SERVICE]

or by utilizing the included sample shell script which will make this call for you:

$ /opt/shibboleth-idp/bin/reload-service.sh -id [SOME SERVICE]

In each case [SOME SERVICE] represents the name of the Shibboleth service that you would like to reload, i.e. one of the service identifiers (id) listed below.

Access Control

In order to be able to make requests to the reloadable services identified above, you must ensure that you have white-listed the IP address of the host making the call. In the case of the reload-service.sh script, that includes ensuring that the interface of the IdP server itself has been whitelisted.

To do this, ensure that the IP addresses are listed within /opt/shibboleth-idp/conf/access-control.xml as in the following example:

<entry key="AccessByIPAddress">
    <bean id="AccessByIPAddress" parent="shibboleth.IPRangeAccessControl"
        p:allowedRanges="#{ {'127.0.0.1/32', '::1/128'} }" />
</entry>

Services

The following services can be reloaded:

Example: Onboarding a new SP

In the following examples, we will only use reload-service.sh, however you can easily adjust the call to the HTTP GET as above.

After you’ve added the SP’s <MetadataProvider> element to /opt/shibboleth-idp/conf/metadata-providers.xml you should first reload that service:

[root@idp.example.org shibboleth-idp]# ./bin/reload-service.sh -id shibboleth.MetadataResolverService

Configuration reloaded for 'shibboleth.MetadataResolverService'

Then, after you have added an appropriate attribute filter policy for this entity in attribute-filter.xml, you should reload that service:

[root@idp.example.org shibboleth-idp]# ./bin/reload-service.sh -id shibboleth.AttributeFilterService

Configuration reloaded for 'shibboleth.AttributeFilterService'

Lastly, presuming you need to configure a <RelyingPartyOverride> for this entity to adjust the particulars of the single sign on integration, you should reload that service:

[root@idp.example.org shibboleth-idp]# ./bin/reload-service.sh -id shibboleth.RelyingPartyResolverService

Configuration reloaded for 'shibboleth.RelyingPartyResolverService'

Common Issues

The most common issue we encounter with reloadable services is that calls to reload-service.sh fail, either because Access Control was not established properly, or because the IDP cannot properly make calls to http(s)://localhost/idp which is the default IDP_BASE_URL environment variable.

Setting up IDP_BASE_URL within your Shibboleth service’s startup (typically /etc/default/jetty) script usually resolves this issue. For example, a typical startup script might need to look something like this:

export INST_BASE=/opt
export JAVA_HOME=$INST_BASE/java
export JAVA=$JAVA_HOME/bin/java
export JETTY_HOME=$INST_BASE/jetty
export JETTY_BASE=$INST_BASE/idp_jetty
export IDP_HOME=$INST_BASE/shibboleth-idp
export IDP_BASE_URL=http://idp.example.org/idp