2.3.0

Whispers 2.3.0 release notes

• New Feature: 💫 Static Code Analysis 💫 is now supported!

  • The present release complements classic Whispers' structured text analysis with Semgrep's AST generator for common programming languages like Python, PHP, Java/Scala/Kotlin, JavaScript/TypeScript, Go, etc etc.
  • New argument --ast for enabling this feature via the CLI (it is disabled by default)
  • New setting ast: true for enabling this feature via a custom config file (set to ast: false by default)
  • Replaced astroid Python AST generator with semgrep

• Detection rule improvements

  • Known API keys
  • AWS account ID
  • Passwords
  • Creditcards

• Drop end-of-life Python support

  • Versions 3.6 and 3.7 are no longer supported. Oldest supported version is Python 3.8.
  • Last release that supports Python 3.6 and 3.7 is Whispers 2.2.1

• Dependency tracking improvements

  • New requirements-dev.txt file allows Dependabot updates for dev dependencies
  • Modified setup.py to read from requirements.txt and requirements-dev.txt
  • Updated build CI to use Python 3.12.3

• Debugging and troubleshooting

  • Modified config.yml to exclude known false positives
  • Fixed Dockerfile to work with docker build -t whispers . or the same make image
  • New arg --dump for generating an AST of a file: whispers --dump src/example.ts

💫 New Feature: Static Code Analysis 💫

With the release of Whispers 2.3, it is now possible to accurately apply Whispers' secret detection techniques for structured text to static code. Before this release, Whispers only supported structured text formats, such as JSON or XML. Semgrep is an open source SAST tool, which has a built-in feature for generating Abstract Structure Trees (ASTs) for many common programming languages. Generating an AST for static code yields an accurate structured text representation, which can be checked for secrets with Whispers' rules and plugins. As such, generating ASTs requires an additional "format conversion" step, which naturally affects runtime speed. When AST is enabled it will take longer to scan the same scope if any source code files are present. The increased amount of runtime time would be however long it takes to run the following command on all static code files in scope:

semgrep scan --metrics=off --quiet --dump-ast --json --lang $LANG $SRCFILE

Consider the following benchmarks:

time whispers -F " " tests/fixtures
# 313 detected secrets
# 0,51s user 0,03s system 99% cpu 0,540 total
# 0,60s user 0,04s system 99% cpu 0,642 total

time whispers -a -F " " tests/fixtures
# 421 detected secrets
# 2,20s user 0,40s system 100% cpu 2,589 total
# 2,32s user 0,46s system 100% cpu 2,772 total

AST conversion is disabled by default - semgrep will not execute at all unless explicitly enabled. Custom config files that are missing ast: false or ast: true will default to false.

ast: true  # enable AST in config.yml

whispers --ast target/dir/or/file  # enable AST in CLI

❌ Breaking changes ❌

❌ Replaced astroid with semgrep ❌

Before Whispers 2.3, only Python AST scanning was natively supported by astroid, and integrated via plugins/python.py. With the release of Whispers 2.3, this functionality is superseded by semgrep, and integrated via plugins/semgrep.py. As a base line, the new semgrep plugin detects the same findings as the astroid plugin, but supports more programming languages.

Unfortunately semgrep has telemetry enabled by default, but can be turned off via --metrics=off. In any case, semgrep will not execute unless explicitly enabled via args or config.

⚠️ NOTE: At the time of writing, semgrep does not support Windows OS natively, and can only be installed through WSL. As such, compiled Whispers PE32+ executable comes without Static Code Analysis support. Installing Whispers on Windows via WSL with pip3 install whispers does have Static Code Analysis support.

Changelog

Date	Version	Release notes
2021-12-07	2.0.0	https://github.com/adeptex/whispers/releases/tag/2.0.0
2022-07-29	2.1.0	https://github.com/adeptex/whispers/releases/tag/2.1.0
2023-10-23	2.2.0	https://github.com/adeptex/whispers/releases/tag/2.2.0
2024-06-16	2.3.0	https://github.com/adeptex/whispers/releases/tag/2.3.0

What's Changed

• Bump lxml from 5.2.0 to 5.2.1 by @dependabot in #130
• 2.2.2 by @adeptex in #132
• Bump typing-extensions from 4.10.0 to 4.11.0 by @dependabot in #131
• 2.3.0 by @adeptex in #139
• Bump jellyfish from 1.0.3 to 1.0.4 by @dependabot in #136
• Bump lxml from 5.2.1 to 5.2.2 by @dependabot in #134

Full Changelog: 2.2.1...2.3.0