add additional json fields: port,ip,scheme,url

[tarunKoyalwar]

Proposed Changes

• fix bug related to host field (only host should be shown here not url)
• add additional fields like ip,scheme,url,port where relevant
• closes host field bug + additional jsonl fields (improvement) #4205

Note

Expected Behavior:

-  "host": "https://scanme.sh/this_is_test",
+  "host": "scanme.sh",

Additional fields:

+  "port": "443",
+  "ip": "128.199.158.128",
+  "scheme": "https",
+  "url": "https://scanme.sh",

Steps To Reproduce:

echo https://scanme.sh/this_is_test | nuclei -id tech-detect -ms -j -or -duc -silent| jq .

Before

$  echo https://scanme.sh/this_is_test | nuclei -id tech-detect -ms -j -or -duc -silent| jq .  
{
  "template": "http/technologies/tech-detect.yaml",
  "template-url": "https://templates.nuclei.sh/public/tech-detect",
  "template-id": "tech-detect",
  "template-path": "/Users/tarun/nuclei-templates/http/technologies/tech-detect.yaml",
  "info": {
    "name": "Wappalyzer Technology Detection",
    "author": [
      "hakluke"
    ],
    "tags": [
      "tech"
    ],
    "severity": "info",
    "metadata": {
      "max-request": 1
    }
  },
  "type": "http",
  "host": "https://scanme.sh/this_is_test",
  "timestamp": "2023-11-24T20:26:18.783136+05:30",
  "matcher-status": false
}

Now

$ echo https://scanme.sh/this_is_test | ./nuclei -id tech-detect -ms -j -or -duc -silent| jq .
{
  "template": "http/technologies/tech-detect.yaml",
  "template-url": "https://templates.nuclei.sh/public/tech-detect",
  "template-id": "tech-detect",
  "template-path": "/Users/tarun/nuclei-templates/http/technologies/tech-detect.yaml",
  "info": {
    "name": "Wappalyzer Technology Detection",
    "author": [
      "hakluke"
    ],
    "tags": [
      "tech"
    ],
    "severity": "info",
    "metadata": {
      "max-request": 1
    }
  },
  "type": "http",
  "host": "scanme.sh",
  "port": "443",
  "scheme": "https",
  "url": "https://scanme.sh/this_is_test",
  "path": "/this_is_test",
  "ip": "128.199.158.128",
  "timestamp": "2023-11-24T20:25:47.358938+05:30",
  "matcher-status": false
}

[dogancanbakir]

LGTM

[ehsandeep]

return host as ip when only ip is available

[tarunKoyalwar]

$ echo 128.199.158.128 | ./nuclei -id tech-detect -ms -j -silent | jq .
{
  "template": "http/technologies/tech-detect.yaml",
  "template-url": "https://templates.nuclei.sh/public/tech-detect",
  "template-id": "tech-detect",
  "template-path": "/Users/tarun/nuclei-templates/http/technologies/tech-detect.yaml",
  "info": {
    "name": "Wappalyzer Technology Detection",
    "author": [
      "hakluke"
    ],
    "tags": [
      "tech"
    ],
    "severity": "info",
    "metadata": {
      "max-request": 1
    }
  },
  "type": "http",
  "host": "128.199.158.128",
  "port": "443",
  "scheme": "https",
  "url": "https://128.199.158.128",
  "request": "GET / HTTP/1.1\r\nHost: 128.199.158.128\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2866.71 Safari/537.36\r\nConnection: close\r\nAccept: */*\r\nAccept-Language: en\r\nAccept-Encoding: gzip\r\n\r\n",
  "response": "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 2\r\nContent-Type: text/plain; charset=utf-8\r\nDate: Mon, 27 Nov 2023 14:19:25 GMT\r\n\r\nok",
  "ip": "128.199.158.128",
  "timestamp": "2023-11-27T19:49:25.221005+05:30",
  "matcher-status": false