False "compliant" result

[gerhard-tinned]

ssh_scan (gihub tags/0.0.13) reports correctly as follows.

"compliance": { "policy": "Mozilla Modern", "compliant": true, "recommendations": [ ], "references": [ ] }

While ssh_scan (github tags/0.0.14 until tags/0.0.16) report with the same policy a compliant false.

"compliance": { "policy": "Mozilla Modern", "compliant": false, "recommendations": [ ], "references": [ ] }

[claudijd]

@gerhard-tinned would you mind sharing all of the scan result JSON and any log output from your scan? I did look around to see what changed in https://github.com/mozilla/ssh_scan/releases/tag/0.0.14 and it looks like we added a new policy control for ssh_version. Let me know, I'd be happy to help troubleshoot this with you and get this issue resolved.

[gerhard-tinned]

Sorry for the delay. Here is what I have done and used. I use CentOS which sadly still uses OpenSSH 5.3 which does not support all Mozilla Modern ciphers but more then the intermediate.

What did I do:
I took the mozilla modern policy and commented out the not supported ciphers. This is the end result after commenting out the not-supported algorithms from openssh 5.3:

(temporary for a test, i have added the auth_methods other then publickey)

---
name: Mozilla Modern
ssh_version: 2.0
auth_methods:
- publickey
- password
- gssapi-keyex
- gssapi-with-mic
kex:
#- [email protected]
- ecdh-sha2-nistp521
- ecdh-sha2-nistp384
- ecdh-sha2-nistp256
- diffie-hellman-group-exchange-sha256
encryption:
#- [email protected]
#- [email protected]
#- [email protected]
- aes256-ctr
- aes192-ctr
- aes128-ctr
macs:
#- [email protected]
#- [email protected]
#- [email protected]
- hmac-sha2-512
- hmac-sha2-256
#- [email protected]
compression:
- none
- [email protected]
references:
- https://wiki.mozilla.org/Security/Guidelines/OpenSSH

Then I checkout the version 0.0.13 via the follwing command:

git checkout tags/0.0.13

And execute it against one ofmy servers. The result looks like this (removed a vew sections for prvacy (if you need all details, I can send them to you privatly):

I, [2016-12-07T09:12:14.627386 #10756]  INFO -- : Started ssh_scan of host.domain.tld:22222
I, [2016-12-07T09:12:16.622638 #10756]  INFO -- : Completed ssh_scan of host.domain.tld:22222
[
  {
    "ssh_scan_version": "0.0.13",
    "ip": "...",
    "port": "22222",
    "server_banner": "SSH-2.0-OpenSSH_5.3",
    "ssh_version": 2.0,
    "os": "unknown",
    "os_cpe": "o:unknown",
    "ssh_lib": "openssh",
    "ssh_lib_cpe": "a:openssh:openssh:5.3",
    "cookie": "3...5",
    "key_algorithms": [
      "ecdh-sha2-nistp521",
      "ecdh-sha2-nistp384",
      "ecdh-sha2-nistp256",
      "diffie-hellman-group-exchange-sha256"
    ],
    "server_host_key_algorithms": [
      "ssh-rsa",
      "ssh-dss"
    ],
    "encryption_algorithms_client_to_server": [
      "aes256-ctr",
      "aes192-ctr",
      "aes128-ctr"
    ],
    "encryption_algorithms_server_to_client": [
      "aes256-ctr",
      "aes192-ctr",
      "aes128-ctr"
    ],
    "mac_algorithms_client_to_server": [
      "hmac-sha2-512",
      "hmac-sha2-256"
    ],
    "mac_algorithms_server_to_client": [
      "hmac-sha2-512",
      "hmac-sha2-256"
    ],
    "compression_algorithms_client_to_server": [
      "none",
      "[email protected]"
    ],
    "compression_algorithms_server_to_client": [
      "none",
      "[email protected]"
    ],
    "languages_client_to_server": [

    ],
    "languages_server_to_client": [

    ],
    "hostname": "host.domain.tld",
    "auth_methods": [
      "publickey",
      "gssapi-keyex",
      "gssapi-with-mic",
      "password"
    ],
    "fingerprints": {
      "md5": "c6:....:13",
      "sha1": "3a:1a:...:20:1a",
      "sha256": "10:d9:ca:...:ab:1b:4d"
    },
    "compliance": {
      "policy": "Mozilla Modern",
      "compliant": true,
      "recommendations": [

      ],
      "references": [
        "https://wiki.mozilla.org/Security/Guidelines/OpenSSH"
      ]
    }
  }
]

(Sorry gihub formats the code a bit funny)

[gerhard-tinned]

With version tags/0.0.14 , the output shows like this.

I, [2016-12-07T09:20:05.524726 #10773]  INFO -- : Started ssh_scan of host.domain.tld:22222
I, [2016-12-07T09:20:07.604864 #10773]  INFO -- : Completed ssh_scan of host.domain.tld:22222
[
  {
    "ssh_scan_version": "0.0.14",
    "ip": "...",
    "port": "22222",
    "server_banner": "SSH-2.0-OpenSSH_5.3",
    "ssh_version": 2.0,
    "os": "unknown",
    "os_cpe": "o:unknown",
    "ssh_lib": "openssh",
    "ssh_lib_cpe": "a:openssh:openssh:5.3",
    "cookie": "7...5",
    "key_algorithms": [
      "ecdh-sha2-nistp521",
      "ecdh-sha2-nistp384",
      "ecdh-sha2-nistp256",
      "diffie-hellman-group-exchange-sha256"
    ],
    "server_host_key_algorithms": [
      "ssh-rsa",
      "ssh-dss"
    ],
    "encryption_algorithms_client_to_server": [
      "aes256-ctr",
      "aes192-ctr",
      "aes128-ctr"
    ],
    "encryption_algorithms_server_to_client": [
      "aes256-ctr",
      "aes192-ctr",
      "aes128-ctr"
    ],
    "mac_algorithms_client_to_server": [
      "hmac-sha2-512",
      "hmac-sha2-256"
    ],
    "mac_algorithms_server_to_client": [
      "hmac-sha2-512",
      "hmac-sha2-256"
    ],
    "compression_algorithms_client_to_server": [
      "none",
      "[email protected]"
    ],
    "compression_algorithms_server_to_client": [
      "none",
      "[email protected]"
    ],
    "languages_client_to_server": [

    ],
    "languages_server_to_client": [

    ],
    "hostname": "host.domain.tld",
    "auth_methods": [
      "publickey",
      "gssapi-keyex",
      "gssapi-with-mic",
      "password"
    ],
    "fingerprints": {
      "md5": "c6:....:13",
      "sha1": "3a:1a:...:20:1a",
      "sha256": "10:d9:ca:...:ab:1b:4d"
    },
    "compliance": {
      "policy": "Mozilla Modern",
      "compliant": false,
      "recommendations": [

      ],
      "references": [
        "https://wiki.mozilla.org/Security/Guidelines/OpenSSH"
      ]
    },
    "start_time": "2016-12-07 09:20:05 +0000",
    "end_time": "2016-12-07 09:20:07 +0000",
    "scan_duration_seconds": 2.07894367
  }
]

Just to state the obvious, This is the same commands line used, with the same parameters, the same policy, and it connects and checks the same host without any configuration changes. :) This behaviour with "compliant: false " is the same all the way up to 0.0.16.

[ashishgaurav13]

@claudijd @gerhard-tinned
I think this is being caused by a typo in lib/ssh_scan/policy_manager.rb, where in the function compliant?, we have a out_of_policy_ssh_version instead of !out_of_policy_ssh_version. This line was added in between 0.0.13 and 0.0.14.

[claudijd]

nice catch @agaurav77, I will land the fix in a second.

[claudijd]

This has been fixed in master, @gerhard-tinned would you mind giving it another try on master and seeing that resolves your issue? Also, many thanks for the detailed troubleshooting and hugs to @agaurav77 for fixing what looks to have been the bug.

Also, to your point above @gerhard-tinned about 5.3, I'd like to eventually extend the policy manager to support conditional policies, but we haven't quite hit that maturity level just yet. If a bug doesn't already exist, I will create one for that to let people know we're thinking about it.

[claudijd]

Created #293 to cover the conditional banner policies.

[gerhard-tinned]

@claudijd, I will try to test it again with the master version. Most likely on Monday.

[claudijd]

@gerhard-tinned sounds good, thanks!

[gerhard-tinned]

Sorry for the delay. Was sick.

The current master version seems to work perfectly. ;)

Thanks for solving this issue.

[claudijd]

@gerhard-tinned hope you feel better! Thanks for confirming, we're gearing up for a gem release soon for some major API feature set, the fix will be included in that release.