init for release

.gitignore

@@ -0,0 +1,69 @@
+/.gradle/
+/.settings/
+.project
+.classpath
+.antProperties.xml
+
+.DS_Store
+
+# Eclipse.gitignore below
+.metadata
+bin/
+tmp/
+*.tmp
+*.bak
+*.swp
+*~.nib
+local.properties
+.settings/
+.loadpath
+.recommenders
+
+# External tool builders
+.externalToolBuilders/
+
+# Locally stored "Eclipse launch configurations"
+*.launch
+
+# PyDev specific (Python IDE for Eclipse)
+*.pydevproject
+
+# CDT-specific (C/C++ Development Tooling)
+.cproject
+
+# CDT- autotools
+.autotools
+
+# Java annotation processor (APT)
+.factorypath
+
+# PDT-specific (PHP Development Tools)
+.buildpath
+
+# sbteclipse plugin
+.target
+
+# Tern plugin
+.tern-project
+
+# TeXlipse plugin
+.texlipse
+
+# STS (Spring Tool Suite)
+.springBeans
+
+# Code Recommenders
+.recommenders/
+
+# Annotation Processing
+.apt_generated/
+.apt_generated_test/
+
+# Scala IDE specific (Scala & Java development for Eclipse)
+.cache-main
+.scala_dependencies
+.worksheet
+
+# Uncomment this line if you wish to ignore the project description file.
+# Typically, this file would be tracked if it contains build/dependency configurations:
+#.project

README.md

@@ -0,0 +1,103 @@
+# Ghostrings
+
+Scripts for recovering string definitions in Go binaries with P-Code analysis.
+Tested with x86, x86-64, ARM, and ARM64.
+
+There are two main parts to the string recovery flow with Ghostrings:
+
+* Find dynamic string structure definitions on the stack via P-Code analysis, then use their start address and length values to define strings in the `go.string.*` string data blob
+* Fill the remaining gaps in `go.string.*` using some mathematical checks, based on the ascending length order of the strings
+
+These two techniques greatly simplify recovering all string definitions in the `go.string.*` blob with minimal manual intervention.
+
+### Background
+
+A well-known issue with reverse engineering Go programs is that the lack of null terminators in Go strings makes recovering string definitions from compiled binaries difficult. Many of a Go program's constant string values are stored together in one giant blob in the compiled build, without any terminator characters built into the string data to mark where one string ends and another begins. Even a simple program that just prints "Hello world!" has over 1,500 strings in it related to the Go runtime system and other standard libraries. This can cause typical ASCII string discovery implementations, such as the one provided by Ghidra, to create false positive string definitions that are tens of thousands of characters long.
+
+Instead of null terminated strings, Go uses a string structure that consists of a pointer and length value. Many of these string structures are created on the program’s stack at runtime, so recovering individual string start locations and length values requires analyzing the compiled machine code. There are a few existing scripts that perform this analysis by checking for certain patterns of x86-64 instructions, but they miss structures created with unhandled variations of instructions that ultimately have the same effect on the stack, and they're also restricted to a specific ISA.
+
+Ghostrings avoids both these problems by working with the simplified, architecture independent P-Code operations produced by Ghidra’s decompiler analysis.
+
+## Setup
+
+### Installing
+
+In Ghidra:
+
+1. File -> Install Extensions -> Add extension
+2. Select the extension ZIP file
+
+### Development
+
+For Eclipse with GhidraDev plugin:
+
+1. Clone repo
+2. File -> Import -> Projects from Folder or Archive
+3. Select repo directory
+4. GhidraDev -> Link Ghidra...
+5. Select imported project
+
+### Building
+
+Build with Eclipse:
+
+* GhidraDev -> Export -> Ghidra Module Extension...
+
+Build directly with Gradle:
+
+```console
+$ cd Ghostrings
+$ gradle -PGHIDRA_INSTALL_DIR=<ghidra_install_dir>
+```
+
+## Scripts
+
+### Golang
+
+These can be found in the Golang category in the Script Manager.
+
+* `GoDynamicStrings.java`
+  * Analyzes P-Code to find string structures created on the stack. Uses the lower level "register" style analysis.
+* `GoDynamicStringsSingle.java`
+  * Performs the same analysis as `GoDynamicStrings.java`, but uses a single decompiler process. Use this if analyzing a large binary causes the parallel decompiler processes to exhaust system memory.
+* `GoDynamicStringsHigh.java`
+  * Experimental, uses P-Code output from the higher level "normalize" style analysis. Currently depends on a hack that turns off deadcode elimination in the decompiler.
+* `GoKnownStrings.java`
+  * Searches for standard unique strings and defines them.
+* `GoStringFiller.java`
+  * Fills in gaps in `go.string.*` after initial analysis, based on strings being ordered by ascending length.
+
+### P-Code
+
+This can be found in the PCode category in the Script Manager.
+
+* `PrintHighPCode.java`
+  * Prints high P-Code output for the currently selected function to the console, with a selector for the decompiler simplification style to use.
+
+
+## String recovery flow
+
+Here’s the general flow for using these scripts to recover string definitions in a Go binary:
+
+1. Clear all automatically defined strings in the `.rodata` (ELF) or `.rdata` (PE) memory block. The goal is to eliminate incorrect string definitions caused by the lack of null terminators.
+   1. In the "Defined Strings" window, add the "Mem Block" column to the display
+   2. Create a filter on the memory block column to only show strings in the target block
+   3. Select all strings in the window, then in the listing right-click and choose "Clear Code Bytes"
+2. Run `GoDynamicStrings.java` or `GoDynamicStringsHigh.java`.
+3. *(Optional)* Run `GoKnownStrings.java` to detect some standard strings.
+4. Run `GoStringFiller.java`.
+   * If it detects false positive short strings (strings that violate the ascending length order), clear them and re-run the script. There is an option to do this automatically.
+   * There’s an option to allow the script to define strings even when a unique set of string lengths can’t be identified, as a last resort. Specifically, there’s one rule that checks if a gap’s size is evenly divisible only by a single string length. It’s possible there are actually strings of different lengths in the gap, but this works often enough to be useful. I recommend running the script without allowing false positives until all the short strings have been fixed.
+   * If the binary is stripped, locate the area of one byte strings found by the dynamic strings script.
+     Ensure it's the start of the grouped together non-null-terminated strings (more strings should be defined after with length in ascending order).
+     Create the label `go.string.*` at the first one byte string. 
+5. Check for remaining gaps in `go.string.*`, and define any strings with obvious start and end points. Sometimes defining one or two strings and re-running `GoStringFiller.java` is sufficient to fill in remaining gaps.
+6. *(Optional)* Re-run Ghidra's built-in ASCII String analysis tool.
+   * Disable overwriting existing strings. Run with and then without the null terminator requirement.
+
+
+## Release Info
+
+Copyright 2022 NCC Group. Released under the GPLv3 license (see LICENSE).
+
+Main project author: James Chambers <[email protected]>

build.gradle

@@ -0,0 +1,60 @@
+/* ###
+ * IP: GHIDRA
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Builds a Ghidra Extension for a given Ghidra installation.
+//
+// An absolute path to the Ghidra installation directory must be supplied either by setting the 
+// GHIDRA_INSTALL_DIR environment variable or Gradle project property:
+//
+//     > export GHIDRA_INSTALL_DIR=<Absolute path to Ghidra> 
+//     > gradle
+//
+//         or
+//
+//     > gradle -PGHIDRA_INSTALL_DIR=<Absolute path to Ghidra>
+//
+// Gradle should be invoked from the directory of the project to build.  Please see the
+// application.gradle.version property in <GHIDRA_INSTALL_DIR>/Ghidra/application.properties
+// for the correction version of Gradle to use for the Ghidra installation you specify.
+
+//----------------------START "DO NOT MODIFY" SECTION------------------------------
+def ghidraInstallDir
+
+if (System.env.GHIDRA_INSTALL_DIR) {
+	ghidraInstallDir = System.env.GHIDRA_INSTALL_DIR
+}
+else if (project.hasProperty("GHIDRA_INSTALL_DIR")) {
+	ghidraInstallDir = project.getProperty("GHIDRA_INSTALL_DIR")
+}
+
+if (ghidraInstallDir) {
+	apply from: new File(ghidraInstallDir).getCanonicalPath() + "/support/buildExtension.gradle"
+}
+else {
+	throw new GradleException("GHIDRA_INSTALL_DIR is not defined!")
+}
+//----------------------END "DO NOT MODIFY" SECTION-------------------------------
+
+repositories {
+	// Declare dependency repositories here.  This is not needed if dependencies are manually 
+	// dropped into the lib/ directory.
+	// See https://docs.gradle.org/current/userguide/declaring_repositories.html for more info.
+	// Ex: mavenCentral()
+}
+
+dependencies {
+	// Any external dependencies added here will automatically be copied to the lib/ directory when
+	// this extension is built.	
+}

build/.gitignore

@@ -0,0 +1,3 @@
+*
+*/
+!.gitignore

data/README.txt

@@ -0,0 +1,15 @@
+The "data" directory is intended to hold data files that will be used by this module and will
+not end up in the .jar file, but will be present in the zip or tar file.  Typically, data
+files are placed here rather than in the resources directory if the user may need to edit them.
+
+An optional data/languages directory can exist for the purpose of containing various Sleigh language
+specification files and importer opinion files.  
+
+The data/buildLanguage.xml is used for building the contents of the data/languages directory.
+
+The skel language definition has been commented-out within the skel.ldefs file so that the 
+skeleton language does not show-up within Ghidra.
+
+See the Sleigh language documentation (docs/languages/index.html) for details Sleigh language 
+specification syntax.
+

dist/.gitignore

@@ -0,0 +1,3 @@
+*
+*/
+!.gitignore

extension.properties

@@ -0,0 +1,5 @@
+name=@extname@
+description=Tools for recovering strings in Go binaries.
+author=James Chambers
+createdOn=4/4/2022
+version=@extversion@