Skip to content

Commit

Permalink
Merge pull request #184 from cashapp/krueger/example-readme
Browse files Browse the repository at this point in the history
Add a README.md to the Example project
  • Loading branch information
kruegermj authored Jan 22, 2024
2 parents adb65ef + af80956 commit 60222bf
Show file tree
Hide file tree
Showing 2 changed files with 115 additions and 34 deletions.
79 changes: 79 additions & 0 deletions Example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Example Project

This is an example project that includes snapshot tests that are also used by the CI system to confirm the framework continues to work as expected.

## Development Requirements

Your local environment should match one these supported versions.

| Software | Versions |
| --- | --- |
| Xcode | 12.5.1 (iOS 13 and 14), 14.3.1 (iOS 16) [.github/workflows/ci.yml](https://github.com/cashapp/AccessibilitySnapshot/blob/master/.github/workflows/ci.yml) |
| Ruby | 3.2.2 [Example/Gemfile](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Example/Gemfile) |
| Bundler | 2.4.22 [Gemfile.lock](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Gemfile.lock) |
| Simulators | iOS 13.7 - iPhone 12 Pro, iOS 14.5 - iPhone 12 Pro, iOS 16.4 - iPhone 14 Pro [Scripts/build.swift](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Scripts/build.swift) |

### Setting up environment

1. Install the Xcode IDE

- To install Xcode versions you can visit the [Apple developer downloads](https://developer.apple.com/download/all/) site directly.
- Verify your Xcode version and installation with:

```sh
xcode-select -p
```

1. RVM can be used for Ruby version management. Instructions here: https://rvm.io/rvm/install

- Install the current Ruby version

```sh
rvm install ruby-3.2.2 --with-openssl-dir=/opt/homebrew/opt/openssl@3
```

NOTE: This can take a while as RVM will typically build Ruby from source.

- Restart your shell session after installing RVM and Ruby or: `source ~/.zshrc` / `source ~/.bashrc`

- Verify your installation of RVM and Ruby

```sh
which ruby # /Users/<LDAP>/.rvm/rubies/ruby-3.2.2/bin/ruby
```

1. Install Bundler

```sh
gem install bundler -v 2.4.22
```

Verify your installation of Bundler

```sh
bundle -v # Bundler version 2.4.22
```

### Building the project

1. Install the required Gems in the `Gemfile`

```sh
bundle install
```

1. Install CocoaPod dependencies from `Podfile` and update the CocoaPod source repositories

```sh
bundle exec pod install
```

1. Open the newly generated workspace

```sh
xed AccessibilitySnapshot.xcworkspace
```

### Getting Snapshot Images from CI

Test results are archived for CI jobs. When there is a failure because of a snapshot test image changing those images can be extracted from the archive. See [Scripts/ExtractImagesFromTestResults.swift](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Scripts/ExtractImagesFromTestResults.swift) for instructions.
70 changes: 36 additions & 34 deletions Scripts/ExtractImagesFromTestResults.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,40 +2,42 @@

// Usage:
//
// 1. Download the Test Results bundle from the CI build. From your PR, go to
// 1. Install xcparse: brew install chargepoint/xcparse/xcparse
//
// 2. Download the Test Results bundle from the CI build. From your PR, go to
// Checks > CI, then click on "Test Results" in the Artifacts section.
//
// 2. Run this script with the path to this download.
// 3. Run this script with the path to this download.
//
// ./Scripts/ExtractImagesFromTestResults.sh /path/to/Test\ Results
// ./Scripts/ExtractImagesFromTestResults.swift /path/to/Test\ Results
//
// 3. This will create a directory called "tmp" that contains all of the
// 4. This will create a directory called "tmp" that contains all of the
// extracted images (reference, failed, and diff) organized by OS version,
// test class name, and test name.
// test class name, and test name.

import Foundation

enum TaskError: Error {
case code(Int32)
case code(Int32)
}

func execute(commandPath: String, arguments: [String]) throws {
let task = Process()
task.launchPath = commandPath
task.arguments = arguments
let task = Process()
task.launchPath = commandPath
task.arguments = arguments

task.launch()
task.launch()

task.waitUntilExit()
task.waitUntilExit()

guard task.terminationStatus == 0 else {
throw TaskError.code(task.terminationStatus)
}
guard task.terminationStatus == 0 else {
throw TaskError.code(task.terminationStatus)
}
}

if CommandLine.arguments.count < 2 || CommandLine.arguments[1] == "--help" || CommandLine.arguments[1] == "-h" {
print("usage: CompareRenamedSnapshots SNAPSHOT_LIST_FILE OLD_REPO_PATH NEW_DEVICE_ID OLD_DEVICE_ID")
exit(0)
print("usage: CompareRenamedSnapshots SNAPSHOT_LIST_FILE OLD_REPO_PATH NEW_DEVICE_ID OLD_DEVICE_ID")
exit(0)
}

let testResultsContainerPath = CommandLine.arguments[1]
Expand All @@ -47,38 +49,38 @@ guard let testResultsEnumerator = fileManager.enumerator(at: testResultsContaine
}

func parseResults(testResultsPath: String, outputPath: String) throws {
try execute(
commandPath: "/opt/homebrew/bin/xcparse",
arguments: ["screenshots", "--os", "--test", testResultsPath, outputPath]
)
try execute(
commandPath: "/opt/homebrew/bin/xcparse",
arguments: ["screenshots", "--os", "--test", testResultsPath, outputPath]
)
}

for case let fileURL as URL in testResultsEnumerator {
if fileURL.path.hasSuffix(".xcresult") {
try parseResults(testResultsPath: fileURL.path, outputPath: "tmp")
try parseResults(testResultsPath: fileURL.path, outputPath: "tmp")
}
}

/*

As a future enhancement, this script could also rename the failed snapshot images. One complicating
factor of this is the tests with multiple associated reference images (e.g. when using identifiers)
don't differentiate the resulting images in the test result bundle. We'll also need to handle
renaming based on which snapshot engine was used:
As a future enhancement, this script could also rename the failed snapshot images. One complicating
factor of this is the tests with multiple associated reference images (e.g. when using identifiers)
don't differentiate the resulting images in the test result bundle. We'll also need to handle
renaming based on which snapshot engine was used:

FBSnapshotTestCase
FBSnapshotTestCase

/Example/SnapshotTests/ReferenceImages/_64/<TestClass>/
/Example/SnapshotTests/ReferenceImages/_64/<TestClass>/

<TestName>[email protected]
<TestName>[email protected]
<TestName>[email protected]
<TestName>[email protected]

SnapshotTesting
SnapshotTesting

/Example/SnapshotTests/__Snapshots__/<TestClass>/
/Example/SnapshotTests/__Snapshots__/<TestClass>/

<TestName>.390x844-14-5-3x.png
<TestName>.375x812-13-7-3x.png
<TestName>.390x844-14-5-3x.png
<TestName>.375x812-13-7-3x.png

*/
*/

0 comments on commit 60222bf

Please sign in to comment.