disable non default templates as default #5370
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ehsandeep
requested changes
Jul 4, 2024
There was a problem hiding this comment.
For self-contained template we need to see if it expecting input from CLI variable or env variable to exclude, for example below template is good to run without any additional input.
id: basic-example
info:
name: Test HTTP Template
author: pdteam
severity: info
self-contained: true
http:
- method: GET
path:
- "https://example.com/test"
matchers:
- type: word
words:
- "This is test matcher text"
$ go run . -t test_template.yaml -v
__ _
____ __ _______/ /__ (_)
/ __ \/ / / / ___/ / _ \/ /
/ / / / /_/ / /__/ / __/ /
/_/ /_/\__,_/\___/_/\___/_/ v3.3.0-dev
projectdiscovery.io
[VER] Started metrics server at localhost:9092
[INF] Current nuclei version: v3.3.0-dev (development)
[INF] Current nuclei-templates version: v9.9.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 164
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[VER] [basic-example] Sent HTTP request to https://example.com/test
[INF] No results found. Better luck next time! |
tarunKoyalwar
requested changes
Jul 15, 2024
There was a problem hiding this comment.
$ ./nuclei -l data.txt -t js-analyse.yaml
__ _
____ __ _______/ /__ (_)
/ __ \/ / / / ___/ / _ \/ /
/ / / / /_/ / /__/ / __/ /
/_/ /_/\__,_/\___/_/\___/_/ v3.3.0-dev
projectdiscovery.io
[WRN] Skipping 1 file template[s], use file or directory as an input to run file templates
[INF] Current nuclei version: v3.3.0-dev (development)
[INF] Current nuclei-templates version: v9.9.1 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] Targets loaded for current scan: 1
[INF] No results found. Better luck next time!
[FTL] Could not run nuclei: no templates provided for scancurrently it doesn't handle cases where
- input is given using
-lor target list - input is given from stdin
echo "pkg" | ./nuclei -t js-analyse.yaml alternative implementation:
- as mentioned in other comment we can precheck this and add state variables
HasFileInputwhile loading targets ( see: pkg/input ). since this package deals with all input types adding this check in that package would apply this check on all supported input types
Comment on lines
+552
to
+597
| var ( | ||
| numericalExpressionRegex = regexp.MustCompile(`^[0-9+\-/\W]+$`) | ||
| unresolvedVariablesRegex = regexp.MustCompile(`(?:%7[B|b]|\{){2}([^}]+)(?:%7[D|d]|\}){2}["'\)\}]*`) | ||
| ) | ||
|
|
||
| // copy of the original function from pkg/protocols/common/expressions/variables.go:ContainsUnresolvedVariables | ||
| func templateContainsUnresolvedVariables(templatePath string) bool { | ||
| data, err := os.ReadFile(templatePath) | ||
| if err != nil { | ||
| return false | ||
| } | ||
|
|
||
| matches := unresolvedVariablesRegex.FindAllStringSubmatch(string(data), -1) | ||
| if len(matches) == 0 { | ||
| return false | ||
| } | ||
|
|
||
| var unresolvedVariables []string | ||
| for _, match := range matches { | ||
| if len(match) < 2 { | ||
| continue | ||
| } | ||
|
|
||
| // Skip if the match is an expression | ||
| if numericalExpressionRegex.MatchString(match[1]) { | ||
| continue | ||
| } | ||
| // or if it contains only literals (can be solved from expression engine) | ||
| if hasLiteralsOnly(match[1]) { | ||
| continue | ||
| } | ||
| unresolvedVariables = append(unresolvedVariables, match[1]) | ||
| } | ||
|
|
||
| return len(unresolvedVariables) > 0 | ||
| } | ||
|
|
||
| func hasLiteralsOnly(data string) bool { | ||
| expr, err := govaluate.NewEvaluableExpressionWithFunctions(data, dsl.HelperFunctions) | ||
| if err != nil { | ||
| return false | ||
| } | ||
| if expr != nil { | ||
| _, err = expr.Evaluate(nil) | ||
| return err == nil | ||
| } |
There was a problem hiding this comment.
if this is duplicated i think we can move it to pkg/utils because to avoid any inconsistencies in future and we can reuse it for other purpose as well
Comment on lines
+473
to
+488
|
|
||
| if parsed.SelfContained && | ||
| store.config.ExecutorOptions.Options.Vars.IsEmpty() && !store.config.ExecutorOptions.Options.EnvironmentVariables && | ||
| templateContainsUnresolvedVariables(templatePath) { | ||
| stats.Increment(templates.SkippedSelfContainedStats) | ||
| return | ||
| } | ||
|
|
||
| if parsed.HasFileProtocol() && | ||
| lo.NoneBy(store.config.ExecutorOptions.Options.Targets, func(target string) bool { | ||
| return fileutil.FileOrFolderExists(target) | ||
| }) { | ||
| stats.Increment(templates.SkippedFileStats) | ||
| return | ||
| } | ||
|
|
There was a problem hiding this comment.
not sure if we should compute this check here , the stats definetly belong here but checking everytime if target is a file for every template might increase latency on small hosts/boxes . i think it would be better if we precalcuate and put it in types.Options ( we do this for headless,DAST and other filtering ) so we could also add 2 options
HasGlobalVars bool
HasFileInput bool
or something similar
another reason for this is that nuclei supports target from various flags -u , -l and even input mode and this logic currently only checks for values of -u and not -l
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Closes #5231
file:
self-contained:
both:
Checklist