Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug fixes in js , network protocol and flow #4313

Merged
merged 17 commits into from
Nov 2, 2023
Merged

Conversation

tarunKoyalwar
Copy link
Member

@tarunKoyalwar tarunKoyalwar commented Oct 30, 2023

Proposed Changes

  • closes Incomplete Data Received with Nuclei's JavaScript NetConn ('nuclei/net')  #4285
  • TCP is stream protocol and when we call conn.Read it reads whatever is available at that moment in buffer and does not necessarily mean everything sent by server. same is the case if we specify conn.Read(N)
  • this issue is fixed by reading it in a loop (which is general behaviour)
  • New change: if no length is specified or N is 0 . then all data sent by server is read

@tarunKoyalwar tarunKoyalwar self-assigned this Oct 30, 2023
pkg/js/libs/net/net.go Dismissed Show dismissed Hide dismissed
pkg/js/libs/net/net.go Dismissed Show dismissed Hide dismissed
@tarunKoyalwar tarunKoyalwar changed the title fix net read js: fix read from connection in net module Oct 30, 2023
@tarunKoyalwar tarunKoyalwar marked this pull request as ready for review October 30, 2023 12:42
@tarunKoyalwar
Copy link
Member Author

tarunKoyalwar commented Oct 30, 2023

example template

id: js_test

info:
  name: Testing JS
  author: rahulmaini
  severity: info


javascript:
  - code: |
      let m = require('nuclei/net');
      let name=Host+':'+Port;
      let conn = m.OpenTLS('tcp', name);
      conn.Send('GET / HTTP/1.1\r\nHost:'+name+'\r\nConnection: close\r\n\r\n');
      resp = conn.RecvString()
      log(resp)

    args:
      Host: "{{Host}}"
      Port: "443"
$ ./nuclei -t simplejs.yaml  -u vps.noob.ninja 

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.0.2

		projectdiscovery.io

[INF] Current nuclei version: v3.0.2 (latest)
[INF] Current nuclei-templates version: v9.6.8 (latest)
[INF] New templates added in latest release: 79
[INF] Templates loaded for current scan: 1
[WRN] Executing 1 unsigned templates. Use with caution.
[INF] Targets loaded for current scan: 1
[JS] HTTP/1.1 200 OK
Date: Mon, 30 Oct 2023 12:44:01 GMT
Server: Apache/2.4.18 (Ubuntu)
Access-Control-Allow-Origin: *
Cache-Control: no-cache
Transfer-Encoding: chunked
Content-Type: application/json

155
{
    "swagger": "2.0",
    "info": {
        "title": "xxxxxxx",
        "description": "test",
        "version": "2017-06-04T22:56:06+00:00",
        "contact": {
            "name": "test",
            "url": "javascript:alert(document.domain)",
            "email": "xx.team@%0D%0Ajavascript%3Aalert(1)%2F%2F%0A.com"
        }
    }
}


0

[INF] No results found. Better luck next time!

@tarunKoyalwar tarunKoyalwar marked this pull request as draft October 30, 2023 12:53
Copy link

@iamnoooob iamnoooob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. In the initial commit there was this issue where it would always fetch complete data even on specifying N bytes. But, this just fixed it.

@tarunKoyalwar
Copy link
Member Author

tarunKoyalwar commented Oct 30, 2023

TODO

  • fix same issue in network protocol
id: network-port-example

info:
  name: Example Template with Network Port
  author: pdteam
  severity: high
  description: This is an updated description for the network port example.
  reference: https://updated-reference-link

tcp:
  - host: 
      - "tls://{{Hostname}}"
    port: 443
    inputs:
      - data: "GET / HTTP/1.1\r\nHost: {{Hostname}}\r\nConnection: close\r\n\r\n"
    read-size: 5000
    extractors:
    - type: dsl
      dsl:
        - "len(data)"
$ ./nuclei -u projectdiscovery.io -t network-https.yaml 

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.0.2

		projectdiscovery.io

[INF] Current nuclei version: v3.0.2 (latest)
[INF] Current nuclei-templates version: v9.6.8 (latest)
[INF] New templates added in latest release: 79
[INF] Templates loaded for current scan: 1
[WRN] Executing 1 unsigned templates. Use with caution.
[INF] Targets loaded for current scan: 1
[network-port-example] [tcp] [high] projectdiscovery.io:443 [1369]

@tarunKoyalwar tarunKoyalwar linked an issue Oct 31, 2023 that may be closed by this pull request
@tarunKoyalwar
Copy link
Member Author

New Changes

$ ./nuclei -u scanme.sh -t ~/test-templates/jshelper.yaml                                           

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.0.2

		projectdiscovery.io

[INF] Current nuclei version: v3.0.2 (latest)
[INF] Current nuclei-templates version: v9.6.8 (latest)
[INF] New templates added in latest release: 79
[INF] Templates loaded for current scan: 1
[WRN] Executing 1 unsigned templates. Use with caution.
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[js_test] [http] [info] https://scanme.sh/6f6b

template available in issue description

@tarunKoyalwar tarunKoyalwar marked this pull request as ready for review October 31, 2023 22:23
@tarunKoyalwar tarunKoyalwar added the Status: Review Needed The issue has a PR attached to it which needs to be reviewed label Oct 31, 2023
@tarunKoyalwar
Copy link
Member Author

tests are passing locally . only interactsh related test is failing in gh workflow (which is expected)

@tarunKoyalwar tarunKoyalwar changed the title js: fix read from connection in net module bug fixes in js , network protocol and flow Nov 1, 2023
@tarunKoyalwar
Copy link
Member Author

New Changes

$ TESTS="flow/flow-hide-matcher.yaml" ./run.sh
::group::Build nuclei
::endgroup::
::group::Build nuclei integration-test
::endgroup::
::group::Installing nuclei templates

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.0.2

		projectdiscovery.io

[INF] No new updates found for nuclei templates
[✓] Test "flow/flow-hide-matcher.yaml" passed!

Copy link
Member

@dogancanbakir dogancanbakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ehsandeep ehsandeep merged commit 595ba8e into dev Nov 2, 2023
12 checks passed
@ehsandeep ehsandeep deleted the issue-4285-opentls branch November 2, 2023 08:03
@ehsandeep ehsandeep removed the Status: Review Needed The issue has a PR attached to it which needs to be reviewed label Nov 2, 2023
olearycrew pushed a commit to olearycrew/nuclei that referenced this pull request Nov 3, 2023
* fix net read

* only return N bytes if extra available

* use ConnReadN from readerutil

* add integration test

* print unsigned warning in stderr

* fix js protocol in flow projectdiscovery#4318

* fix integration test: url encoding issue

* fix network protocol issue + integration tests

* multiple improvements to integration test

* replace all conn.Read() from tests

* disable network-basic.yaml in windows

* disable code protocol in win CI

* fix bitwise login  ps1-snippet.yaml

* hide previous matcher events in flow

* remove dead code+ update integration tests

---------

Co-authored-by: Sandeep Singh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants