Skip to content

Commit

Permalink
Merge branch 'release/v0.18.0'
Browse files Browse the repository at this point in the history
  • Loading branch information
targodan committed Dec 9, 2022
2 parents ed58b19 + 8ec674c commit b861cf1
Show file tree
Hide file tree
Showing 32 changed files with 1,101 additions and 170 deletions.
6 changes: 5 additions & 1 deletion .github/actions/crossbuild-windows/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,17 @@ inputs:
yara-version:
description: "Yara version"
required: true
go-version:
description: "Golang version"
required: true
runs:
using: "composite"
steps:
- name: Crossbuilding for windows
env:
OPENSSL_VERSION: ${{ inputs.openssl-version }}
YARA_VERSION: ${{ inputs.yara-version }}
run: ./crossBuildForWindows.sh
GO_VERSION: ${{ inputs.go-version }}
run: ./crossBuildForWindows.sh --pull
working-directory: cicd
shell: bash
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ jobs:
with:
openssl-version: ${{ matrix.openssl-version }}
yara-version: ${{ matrix.yara-version }}
go-version: ${{ matrix.go-version }}
- name: Upload windows build
uses: actions/upload-artifact@v3
if: ${{ matrix.go-version == '1.19' }}
Expand Down
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,10 @@ vendor/

# Logfiles
*.log

# Archives, resulting from builds and release prep
*.zip
*.tar.gz
*.tar.zst

.build-cache/
59 changes: 32 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ The most notable differences to stock yara are (see section [Usage](#usage)),

Other quality-of-life features include

- (Experimental) Report server: Run `yapscan receive :8000` on a secure machine and run `yapscan scan` with the `--report-server` flag on infected machines. No missing reports due to ransomware anymore.
- Statically built, dependency free exe for Windows
- Listing running processes
- Listing and dumping memory segments of a specific process
Expand All @@ -38,11 +39,11 @@ COMMANDS:
list-process-memory, lsmem lists all memory segments of a process
dump dumps memory of a process
scan scans processes or paths with yara rules
receive starts a server receiving reports from other yapscan clients (see --report-server flag of scan command)
anonymize anonymize reports
zip-rules creates an encrypted zip containing compiled yara rules
join joins dumps with padding
crash-process, crash crash a process
as-service executes yapscan as a windows service (windows only)
help, h Shows a list of commands or help for one command
```

Expand All @@ -55,32 +56,36 @@ USAGE:
yapscan scan [command options] [pid/path...]
OPTIONS:
--rules value, -r value, -C value path to yara rules file or directory, if it's a file it can be a yara rules file or a zip containing a rules file encrypted with password "infected"
--rules-recurse, --recurse-rules, --rr if --rules specifies a directory, compile rules recursively (default: false)
--all-processes, --all-p scan all running processes (default: false)
--all-drives, --all-d scan all files in all local drives, implies --recurse (default: false)
--all-shares, --all-s scan all files in all mounted net-shares, implies --recurse (default: false)
--file-extensions value, -e value list of file extensions to scan, use special extension "-" as no extension, use --file-extensions "" to allow any (default: "-", "so", "exe", "dll", "sys")
--threads value, -t value number of threads (goroutines) used for scanning files (default: 6)
--full-report create a full report (default: false)
--scan-mapped-files when encountering memory-mapped files also scan the backing file on disk (default: false)
--report-dir value the directory to which the report archive will be written (default: current working directory)
--store-dumps store dumps of memory regions that match rules, implies --full-report, the report will be encrypted with --password (default: false)
--password value setting this will encrypt the report with the given password; ignored without --full-report
--pgpkey value setting this will encrypt the report with the public key in the given file; ignored without --full-report
--anonymize anonymize any output, hashing any usernames, hostnames and IPs with a salt (default: false)
--salt value the salt (base64 string) to use for anonymization, ignored unless --anonmyize is provided (default: random salt)
--verbose, -v show more information about rule matches (default: false)
--filter-permissions value, --f-perm value only consider segments with the given permissions or more, examples: "rw" includes segments with rw, rc and rwx
--filter-permissions-exact value, --f-perm-e value comma separated list of permissions to be considered, supported permissions: r, rw, rc, rwx, rcx
--filter-type value, --f-type value comma separated list of considered types, supported types: image, mapped, private
--filter-state value, --f-state value comma separated list of considered states, supported states: free, commit, reserve (default: "commit")
--filter-size-max value, --f-size-max value maximum size of memory segments to be considered, can be absolute (e.g. "1.5GB"), percentage of total RAM (e.g. "10%T") or percentage of free RAM (e.g. "10%F") (default: "10%F")
--filter-size-min value, --f-size-min value minimum size of memory segments to be considered
--filter-rss-ratio-min value, --f-rss-min value minimum RSS/Size ratio of memory segments to eb considered
--suspend, -s suspend the process before reading its memory (default: false)
--force, -f don't ask before suspending a process (default: false)
--help, -h show help (default: false)
--rules value, -r value, -C value path to yara rules file or directory, if it's a file it can be a yara rules file or a zip containing a rules file encrypted with password "infected"
--rules-recurse, --recurse-rules, --rr if --rules specifies a directory, compile rules recursively (default: false)
--all-processes, --all-p scan all running processes (default: false)
--all-drives, --all-d scan all files in all local drives, implies --recurse (default: false)
--all-shares, --all-s scan all files in all mounted net-shares, implies --recurse (default: false)
--file-extensions value, -e value [ --file-extensions value, -e value ] list of file extensions to scan, use special extension "-" as no extension, use --file-extensions "" to allow any (default: "-", "so", "exe", "dll", "sys")
--threads value, -t value number of threads (goroutines) used for scanning files (default: 6)
--full-report create a full report (default: false)
--scan-mapped-files when encountering memory-mapped files also scan the backing file on disk (default: false)
--report-dir value the directory to which the report archive will be written (default: current working directory)
--report-server value the address of the server, the reports will be sent to
--server-ca value CA.pem to use when validating the server
--client-cert value certificate.pem to use for client authentication
--client-key value key.pem to use for client authentication
--store-dumps store dumps of memory regions that match rules, implies --full-report, the report will be encrypted with --password (default: false)
--password value, -p value setting this will encrypt the report with the given password; ignored without --full-report
--pgpkey value, -k value setting this will encrypt the report with the public key in the given file; ignored without --full-report
--anonymize anonymize any output, hashing any usernames, hostnames and IPs with a salt (default: false)
--salt value the salt (base64 string) to use for anonymization, ignored unless --anonmyize is provided (default: random salt)
--verbose, -v show more information about rule matches (default: false)
--filter-permissions value, --f-perm value only consider segments with the given permissions or more, examples: "rw" includes segments with rw, rc and rwx
--filter-permissions-exact value, --f-perm-e value [ --filter-permissions-exact value, --f-perm-e value ] comma separated list of permissions to be considered, supported permissions: r, rw, rc, rwx, rcx
--filter-type value, --f-type value [ --filter-type value, --f-type value ] comma separated list of considered types, supported types: image, mapped, private
--filter-state value, --f-state value [ --filter-state value, --f-state value ] comma separated list of considered states, supported states: free, commit, reserve (default: "commit")
--filter-size-max value, --f-size-max value maximum size of memory segments to be considered, can be absolute (e.g. "1.5GB"), percentage of total RAM (e.g. "10%T") or percentage of free RAM (e.g. "10%F") (default: "10%F")
--filter-size-min value, --f-size-min value minimum size of memory segments to be considered
--filter-rss-ratio-min value, --f-rss-min value minimum RSS/Size ratio of memory segments to eb considered
--suspend, -s suspend the process before reading its memory (default: false)
--force, -f don't ask before suspending a process (default: false)
--help, -h show help (default: false)
```

Here are some additional example usages
Expand Down
57 changes: 57 additions & 0 deletions app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,22 @@ func MakeApp() *cli.App {
Usage: "the directory to which the report archive will be written",
DefaultText: "current working directory",
},
&cli.StringFlag{
Name: "report-server",
Usage: "the address of the server, the reports will be sent to",
},
&cli.StringFlag{
Name: "server-ca",
Usage: "CA.pem to use when validating the server",
},
&cli.StringFlag{
Name: "client-cert",
Usage: "certificate.pem to use for client authentication",
},
&cli.StringFlag{
Name: "client-key",
Usage: "key.pem to use for client authentication",
},
&cli.BoolFlag{
Name: "store-dumps",
Usage: "store dumps of memory regions that match rules, implies --full-report, the report will be encrypted with --password",
Expand Down Expand Up @@ -399,6 +415,47 @@ func MakeApp() *cli.App {
},
}, segmentFilterFlags...), suspendFlags...),
},
{
Name: "receive",
Usage: "starts a server receiving reports from other yapscan clients (see --report-server flag of scan command)",
Action: receive,
ArgsUsage: "<listen-address>",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "report-dir",
Usage: "the directory to which the report archives will be written",
DefaultText: "current working directory",
},
&cli.StringFlag{
Name: "password",
Aliases: []string{"p"},
Usage: "setting this will encrypt the report with the given password; ignored without --full-report",
},
&cli.StringFlag{
Name: "pgpkey",
Aliases: []string{"k"},
Usage: "setting this will encrypt the report with the public key in the given file; ignored without --full-report",
},
&cli.StringFlag{
Name: "server-cert",
Usage: "certificate.pem to use for TLS",
},
&cli.StringFlag{
Name: "server-key",
Usage: "key.pem to use for TLS",
},
&cli.StringFlag{
Name: "client-ca",
Usage: "CA.pem to use for client authentication",
DefaultText: "no client authentication by default",
},
&cli.BoolFlag{
Name: "verbose",
Aliases: []string{"v"},
Usage: "activate debug output of the http server",
},
},
},
{
Name: "anonymize",
Usage: "anonymize reports",
Expand Down
111 changes: 111 additions & 0 deletions app/receive.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
package app

import (
"context"
"fmt"
"net/http"
"os"
"os/signal"
"time"

"github.com/sirupsen/logrus"

"github.com/gin-gonic/gin"

"github.com/fkie-cad/yapscan/archiver"

"github.com/fkie-cad/yapscan/pgp"

"github.com/fkie-cad/yapscan/output"
"github.com/targodan/go-errors"
"github.com/urfave/cli/v2"
)

func receive(c *cli.Context) error {
err := initAppAction(c)
if err != nil {
return err
}

if c.NArg() != 1 {
return errors.Newf("expected exactly one argument <listen address>")
}

if c.Bool("verbose") {
gin.SetMode(gin.DebugMode)
} else {
gin.SetMode(gin.ReleaseMode)
}

wcBuilder := output.NewWriteCloserBuilder()
if c.String("password") != "" && c.String("pgpkey") != "" {
return fmt.Errorf("cannot encrypt with both pgp key and a password")
}
if c.String("password") != "" {
wcBuilder.Append(output.PGPSymmetricEncryptionDecorator(c.String("password"), true))
}
if c.String("pgpkey") != "" {
ring, err := pgp.ReadKeyRing(c.String("pgpkey"))
if err != nil {
return fmt.Errorf("could not read specified public pgp key, reason: %w", err)
}
wcBuilder.Append(output.PGPEncryptionDecorator(ring, true))
}
wcBuilder.Append(output.ZSTDCompressionDecorator())

reportServer := archiver.NewArchiverServer(
c.Args().First(),
c.String("report-dir"),
wcBuilder.SuggestedFileExtension(),
wcBuilder)

signalChan := make(chan os.Signal)
signal.Notify(signalChan, os.Interrupt)
shutdownStarted := make(chan bool, 1)
shutdownCompleted := make(chan interface{})

if c.String("server-cert") != "" ||
c.String("server-key") != "" ||
c.String("client-ca") != "" {
err = reportServer.EnableTLS(
c.String("server-cert"),
c.String("server-key"),
c.String("client-ca"))
if err != nil {
return cli.Exit(err, 1)
}
}

go func() {
var err error
<-signalChan
defer close(shutdownCompleted)

shutdownStarted <- true

shutdownTimeout := 5 * time.Second
logrus.Infof("Received interrupt, shutting down server (timeout: %v)...", shutdownTimeout)

ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
defer cancel()

err = reportServer.Shutdown(ctx)
if err != nil {
logrus.WithError(err).Error("Error during closing of open reports.")
} else {
logrus.Info("Closed open reports.")
}
}()

err = reportServer.Start()

switch {
case <-shutdownStarted:
<-shutdownCompleted
default:
}
if err != http.ErrServerClosed {
return cli.Exit(err, 10)
}
return nil
}
Loading

0 comments on commit b861cf1

Please sign in to comment.