SMB3 encryption does not work with "native" Kerberos provider on Linux

[pseudometric]

I work at Arcesium, where we use smbj together with MIT Kerberos, by setting sun.security.jgss.native = true to have JGSS use the host’s MIT Kerberos libraries instead of the the default Kerberos implementation in the JDK. We have run into the following problem while trying to use SMB3 encryption.

Standard GSSAPI does not expose the Kerberos session key directly. To protect data with a security context, one uses GSS “wrapping” or explicit MIC generation (GSS_wrap(), GSS_GetMIC(), etc.); these operations in turn use the session key opaquely, in the case of a Kerberos security mechanism.

SMB3 however was specified by Microsoft with SSPI in mind, not GSSAPI, and SMB3 session encryption uses the Kerberos session key directly. smbj retrieves that session key by using an “extended” GSSAPI inquireSecContext() call; see KRB5_GET_SESSION_KEY here. With the native Kerberos provider, this throws an exception:

Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at com.hierynomus.smbj.auth.ExtendedGSSContext.krb5GetSessionKey(ExtendedGSSContext.java:57)
    ... 37 more
Caused by: GSSException: Operation unavailable (Mechanism level: Inquire type not supported.)

indicating that the JGSS-to-host-GSSAPI glue layer does not support this inquiry.

MIT Kerberos does support this, however: it has defined its own similar extended GSSAPI calls to retrieve either the Kerberos acceptor subkey or ticket session key, using gss_inquire_sec_context_by_oid() with the OIDs GSS_C_INQ_SSPI_SESSION_KEY and GSS_C_INQ_ODBC_SESSION_KEY, respectively. As the names suggest, these are there specifically for Windows compatibility. I tried these out and they do indeed work on Linux; code like so in an existing C GSSAPI client using Kerberos:

gss_buffer_set_t skey;
major = gss_inquire_sec_context_by_oid(&minor,
                                       context,
                                       GSS_C_INQ_ODBC_SESSION_KEY,
                                       &skey);
debug("** count %d", skey->count);
debug("**   key %d %s",
      skey->elements[0].length,
      hexstring(skey->elements[0].value, skey->elements[0].length));

produces this:

client: ** count 2
client: **   key 32 3A5011B4FB173F9F829ADC809766099A4908C89F25316672E0FB1F160B400C7A

which I verified indeed matches the actual ticket session key in the test.

So, all the pieces seem to be available to fix this; it’s just a question of how best to go about it. Ideally we would augment ExtendedGSSContext to handle the “native” case, but it might be more practical to just add a native host-GSSAPI call to retrieve the session key as above. With that, the existing smbj code for SMB3 encryption ought to work.

At the moment we are not planning to do this work, since we have a workaround and it’s not clear that it makes sense for us to spend the time on it. But I thought we’d open an issue and put our findings here, in case anyone else runs into this.

Thanks,

Richard E. Silverman [email protected]

[pseudometric]

These session-key context inquiries (GSS_C_INQ_{ODBC,SSPI}_SESSION_KEY) were added recently, in MIT Kerberos 1.21; with earlier versions you'll get an error:

major: The operation or option is not available or unsupported
minor: Invalid argument