Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new field 'port' in network protocol #4123

Merged
merged 5 commits into from
Sep 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 45 additions & 1 deletion docs/template-guide/network.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,49 @@ host:

If a port is specified in the host, the user supplied port is ignored and the template port takes precedence.

### Port

Starting from Nuclei v2.9.15, a new field called `port` has been introduced in network templates. This field allows users to specify the port separately instead of including it in the host field.

Previously, if you wanted to write a network template for an exploit targeting SSH, you would have to specify both the hostname and the port in the host field, like this:
```yaml
host:
- "{{Hostname}}"
- "{{Host}}:22"
```

In the above example, two network requests are sent: one to the port specified in the input/target, and another to the default SSH port (22).

The reason behind introducing the port field is to provide users with more flexibility when running network templates on both default and non-default ports. For example, if a user knows that the SSH service is running on a non-default port of 2222 (after performing a port scan with service discovery), they can simply run:

```bash
$ nuclei -u scanme.sh:2222 -id xyz-ssh-exploit
```

In this case, Nuclei will use port 2222 instead of the default port 22. If the user doesn't specify any port in the input, port 22 will be used by default. However, this approach may not be straightforward to understand and can generate warnings in logs since one request is expected to fail.

Another issue with the previous design of writing network templates is that requests can be sent to unexpected ports. For example, if a web service is running on port 8443 and the user runs:

```bash
$ nuclei -u scanme.sh:8443
```

In this case, `xyz-ssh-exploit` template will send one request to `scanme.sh:22` and another request to `scanme.sh:8443`, which may return unexpected responses and eventually result in errors. This is particularly problematic in automation scenarios.

To address these issues while maintaining the existing functionality, network templates can now be written in the following way:

```yaml
host:
- "{{Hostname}}"
port: 22
```
In this new design, the functionality to run templates on non-standard ports will still exist, except for the default reserved ports (`80`, `443`, `8080`, `8443`, `8081`, `53`). Additionally, the list of default reserved ports can be customized by adding a new field called exclude-ports:

```yaml
exclude-ports: 80,443
```
When `exclude-ports` is used, the default reserved ports list will be overwritten. This means that if you want to run a network template on port `80`, you will have to explicitly specify it in the port field.

#### Matchers / Extractor Parts

Valid `part` values supported by **Network** protocol for Matchers / Extractor are -
Expand All @@ -105,7 +148,7 @@ id: input-expressions-mongodb-detect

info:
name: Input Expression MongoDB Detection
author: pd-team
author: pdteam
severity: info
reference: https://github.com/orleven/Tentacle

Expand All @@ -114,6 +157,7 @@ tcp:
- data: "{{hex_decode('3a000000a741000000000000d40700000000000061646d696e2e24636d640000000000ffffffff130000001069736d6173746572000100000000')}}"
host:
- "{{Hostname}}"
port: 27017
read-size: 2048
matchers:
- type: word
Expand Down
21 changes: 21 additions & 0 deletions integration_tests/network/network-port.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
id: network-port-example

info:
name: Example Template with Network Port
author: pdteam
severity: high
description: This is an updated description for the network port example.
reference: https://updated-reference-link

tcp:
- host:
- "{{Hostname}}"
port: 23846
inputs:
- data: "PING\r\n"
read-size: 4
matchers:
- type: word
part: data
words:
- "PONG"
61 changes: 61 additions & 0 deletions v2/cmd/integration-test/network.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package main

import (
"net"
"strings"

"github.com/projectdiscovery/nuclei/v2/pkg/testutils"
osutils "github.com/projectdiscovery/utils/os"
Expand All @@ -14,6 +15,7 @@ var networkTestcases = []TestCaseInfo{
{Path: "network/self-contained.yaml", TestCase: &networkRequestSelContained{}},
{Path: "network/variables.yaml", TestCase: &networkVariables{}},
{Path: "network/same-address.yaml", TestCase: &networkBasic{}},
{Path: "network/network-port.yaml", TestCase: &networkPort{}},
}

const defaultStaticPort = 5431
Expand Down Expand Up @@ -145,3 +147,62 @@ func (h *networkVariables) Execute(filePath string) error {

return expectResultsCount(results, 1)
}

type networkPort struct{}

func (n *networkPort) Execute(filePath string) error {
ts := testutils.NewTCPServer(nil, 23846, func(conn net.Conn) {
defer conn.Close()

data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
}
if string(data) == "PING" {
_, _ = conn.Write([]byte("PONG"))
}
})
defer ts.Close()

results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug)
if err != nil {
return err
}

if err := expectResultsCount(results, 1); err != nil {
return err
}

// even though we passed port 443 in url it is ignored and port 23846 is used
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, strings.ReplaceAll(ts.URL, "23846", "443"), debug)
if err != nil {
return err
}

if err := expectResultsCount(results, 1); err != nil {
return err
}

// this is positive test case where we expect port to be overridden and 34567 to be used
ts2 := testutils.NewTCPServer(nil, 34567, func(conn net.Conn) {
defer conn.Close()

data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
}
if string(data) == "PING" {
_, _ = conn.Write([]byte("PONG"))
}
})
defer ts2.Close()

// even though we passed port 443 in url it is ignored and port 23846 is used
// instead of hardcoded port 23846 in template
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, ts2.URL, debug)
if err != nil {
return err
}

return expectResultsCount(results, 1)
}
34 changes: 34 additions & 0 deletions v2/pkg/protocols/common/contextargs/contextargs.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,17 @@ package contextargs

import (
"net/http/cookiejar"
"strings"

mapsutil "github.com/projectdiscovery/utils/maps"
sliceutil "github.com/projectdiscovery/utils/slice"
stringsutil "github.com/projectdiscovery/utils/strings"
urlutil "github.com/projectdiscovery/utils/url"
)

var (
// reservedPorts contains list of reserved ports for non-network requests in nuclei
reservedPorts = []string{"80", "443", "8080", "8443", "8081", "53"}
)

// Context implements a shared context struct to share information across multiple templates within a workflow
Expand Down Expand Up @@ -49,6 +58,31 @@ func (ctx *Context) hasArgs() bool {
return ctx.isInitialized() && !ctx.args.IsEmpty()
}

// UseNetworkPort updates input with required/default network port for that template
// but is ignored if input/target contains non-http ports like 80,8080,8081 etc
func (ctx *Context) UseNetworkPort(port string, excludePorts string) error {
ignorePorts := reservedPorts
if excludePorts != "" {
// TODO: add support for service names like http,https,ssh etc once https://github.com/projectdiscovery/netdb is ready
ignorePorts = sliceutil.Dedupe(strings.Split(excludePorts, ","))
}
if port == "" {
// if template does not contain port, do nothing
return nil
}
target, err := urlutil.Parse(ctx.MetaInput.Input)
if err != nil {
return err
}
inputPort := target.Port()
if inputPort == "" || stringsutil.EqualFoldAny(inputPort, ignorePorts...) {
// replace port with networkPort
target.UpdatePort(port)
ctx.MetaInput.Input = target.Host
}
return nil
}

// Get the value with specific key if exists
func (ctx *Context) Get(key string) (interface{}, bool) {
if !ctx.hasArgs() {
Expand Down
8 changes: 8 additions & 0 deletions v2/pkg/protocols/network/network.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,14 @@ type Request struct {
// Inputs contains inputs for the network socket
Inputs []*Input `yaml:"inputs,omitempty" json:"inputs,omitempty" jsonschema:"title=inputs for the network request,description=Inputs contains any input/output for the current request"`
// description: |
// Port is the port to send network requests to. this acts as default port but is overriden if target/input contains
// non-http(s) ports like 80,8080,8081 etc
Port string `yaml:"port,omitempty" json:"port,omitempty" jsonschema:"title=port to send requests to,description=Port to send network requests to"`

// description: |
// ExcludePorts is the list of ports to exclude from being scanned . It is intended to be used with `Port` field and contains a list of ports which are ignored/skipped
ExcludePorts string `yaml:"exclude-ports,omitempty" json:"exclude-ports,omitempty" jsonschema:"title=exclude ports from being scanned,description=Exclude ports from being scanned"`
// description: |
// ReadSize is the size of response to read at the end
//
// Default value for read-size is 1024.
Expand Down
10 changes: 9 additions & 1 deletion v2/pkg/protocols/network/request.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,18 @@ func (request *Request) Type() templateTypes.ProtocolType {
}

// ExecuteWithResults executes the protocol requests and returns results instead of writing them.
func (request *Request) ExecuteWithResults(input *contextargs.Context, metadata, previous output.InternalEvent, callback protocols.OutputEventCallback) error {
func (request *Request) ExecuteWithResults(target *contextargs.Context, metadata, previous output.InternalEvent, callback protocols.OutputEventCallback) error {
var address string
var err error

input := target.Clone()
// use network port updates input with new port requested in template file
// and it is ignored if input port is not standard http(s) ports like 80,8080,8081 etc
// idea is to reduce redundant dials to http ports
if err := input.UseNetworkPort(request.Port, request.ExcludePorts); err != nil {
gologger.Debug().Msgf("Could not network port from constants: %s\n", err)
}

if request.SelfContained {
address = ""
} else {
Expand Down
Loading