Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(secret): trim excessively long lines #7192

Merged
merged 7 commits into from
Jul 23, 2024
Merged

fix(secret): trim excessively long lines #7192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a3f8cf3
initial commit
afdesk Jul 19, 2024
6cb43b6
feat: trim long lines
afdesk Jul 21, 2024
f9baa72
chore: add a const instead of hard-coded figures
afdesk Jul 22, 2024
36ea011
test: add obfuscated js for testing
afdesk Jul 22, 2024
7460068
fix: down max line length to 100
afdesk Jul 22, 2024
ef12409
test: update long contnet
afdesk Jul 22, 2024
70dd895
added a comment
afdesk Jul 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions pkg/fanal/secret/scanner.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,7 +476,10 @@ func toFinding(rule Rule, loc Location, content []byte) types.SecretFinding {
}
}

const secretHighlightRadius = 2 // number of lines above + below each secret to include in code output
const (
secretHighlightRadius = 2 // number of lines above + below each secret to include in code output
maxLineLength = 100 // all lines longer will be cut off
)

func findLocation(start, end int, content []byte) (int, int, types.Code, string) {
startLineNum := bytes.Count(content[:start], lineSep)
Expand Down Expand Up @@ -511,9 +514,16 @@ func findLocation(start, end int, content []byte) (int, int, types.Code, string)
rawLines := lines[codeStart:codeEnd]
var foundFirst bool
for i, rawLine := range rawLines {
strRawLine := string(rawLine)
realLine := codeStart + i
inCause := realLine >= startLineNum && realLine <= endLineNum

var strRawLine string
if len(rawLine) > maxLineLength {
strRawLine = lo.Ternary(inCause, matchLine, string(rawLine[:maxLineLength]))
} else {
strRawLine = string(rawLine)
}
Comment on lines +521 to +525
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like 100 characters is enough.

  1. For lines before and after secret - we don't need such a big line.
  2. For secret line - if length of secret with secret > 100 - we use 30 + secret + 20 characters:

    trivy/pkg/fanal/secret/scanner.go

    Lines 501 to 504 in 36ea011

    if lineEnd-lineStart > 100 {
    lineStart = lo.Ternary(start-30 < 0, 0, start-30)
    lineEnd = lo.Ternary(end+20 > len(content), len(content), end+20)
    }

Now 2000 is a length for backward compatibility with RSA keys.

IIUC you want to include full RSA key to result (-----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY-----), but I think this is not necessary.
But when we use 2000, we can see case when secret length is 10, but the user sees 1900 unneeded characters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DmitriyLewen I generally agree with you. 2k is too long line.


code.Lines = append(code.Lines, types.Line{
Number: codeStart + i + 1,
Content: strRawLine,
Expand Down
46 changes: 38 additions & 8 deletions pkg/fanal/secret/scanner_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,8 +353,8 @@ func TestSecretScanner(t *testing.T) {
Lines: []types.Line{
{
Number: 1,
Content: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa GITHUB_PAT=**************************************** bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
Highlighted: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa GITHUB_PAT=**************************************** bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
Content: "aaaaaaaaaaaaaaaaaa GITHUB_PAT=**************************************** bbbbbbbbbbbbbbbbbbb",
Highlighted: "aaaaaaaaaaaaaaaaaa GITHUB_PAT=**************************************** bbbbbbbbbbbbbbbbbbb",
IsCause: true,
FirstCause: true,
LastCause: true,
Expand Down Expand Up @@ -462,8 +462,8 @@ func TestSecretScanner(t *testing.T) {
Lines: []types.Line{
{
Number: 1,
Content: "{\"key\": \"-----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************-----END RSA PRIVATE KEY-----\\n\"}",
Highlighted: "{\"key\": \"-----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************-----END RSA PRIVATE KEY-----\\n\"}",
Content: "----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************-----END RSA PRIVATE",
Highlighted: "----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************-----END RSA PRIVATE",
Comment on lines +465 to +466
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DmitriyLewen I've decreased maxLineLength and updated the test.
it shows before/after.

IsCause: true,
FirstCause: true,
LastCause: true,
Expand All @@ -483,8 +483,8 @@ func TestSecretScanner(t *testing.T) {
Lines: []types.Line{
{
Number: 1,
Content: "-----BEGIN RSA PRIVATE KEY-----****************************************************************************************************************************************************************************************-----END RSA PRIVATE KEY-----",
Highlighted: "-----BEGIN RSA PRIVATE KEY-----****************************************************************************************************************************************************************************************-----END RSA PRIVATE KEY-----",
Content: "----BEGIN RSA PRIVATE KEY-----****************************************************************************************************************************************************************************************-----END RSA PRIVATE",
Highlighted: "----BEGIN RSA PRIVATE KEY-----****************************************************************************************************************************************************************************************-----END RSA PRIVATE",
IsCause: true,
FirstCause: true,
LastCause: true,
Expand All @@ -504,8 +504,8 @@ func TestSecretScanner(t *testing.T) {
Lines: []types.Line{
{
Number: 1,
Content: "-----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************-----END RSA PRIVATE KEY-----",
Highlighted: "-----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************-----END RSA PRIVATE KEY-----",
Content: "----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************-----END RSA PRIVATE",
Highlighted: "----BEGIN RSA PRIVATE KEY-----**************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************-----END RSA PRIVATE",
IsCause: true,
FirstCause: true,
LastCause: true,
Expand Down Expand Up @@ -667,6 +667,27 @@ func TestSecretScanner(t *testing.T) {
},
},
}
wantFindingTokenInsideJs := types.SecretFinding{
RuleID: "stripe-publishable-token",
Category: "Stripe",
Title: "Stripe Publishable Key",
Severity: "LOW",
StartLine: 1,
EndLine: 1,
Match: "){case a.ez.PRODUCTION:return\"********************************\";case a.ez.TEST:cas",
Code: types.Code{
Lines: []types.Line{
{
Number: 1,
Content: "){case a.ez.PRODUCTION:return\"********************************\";case a.ez.TEST:cas",
Highlighted: "){case a.ez.PRODUCTION:return\"********************************\";case a.ez.TEST:cas",
IsCause: true,
FirstCause: true,
LastCause: true,
},
},
},
}

tests := []struct {
name string
Expand Down Expand Up @@ -982,6 +1003,15 @@ func TestSecretScanner(t *testing.T) {
Findings: []types.SecretFinding{wantMultiLine},
},
},
{
name: "long obfuscated js code with secrets",
configPath: filepath.Join("testdata", "skip-test.yaml"),
inputFilePath: filepath.Join("testdata", "obfuscated.js"),
want: types.Secret{
FilePath: filepath.Join("testdata", "obfuscated.js"),
Findings: []types.SecretFinding{wantFindingTokenInsideJs},
},
},
}

for _, tt := range tests {
Expand Down
1 change: 1 addition & 0 deletions pkg/fanal/secret/testdata/obfuscated.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.