-
Notifications
You must be signed in to change notification settings - Fork 70
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #184 from cashapp/krueger/example-readme
Add a README.md to the Example project
- Loading branch information
Showing
2 changed files
with
115 additions
and
34 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
# Example Project | ||
|
||
This is an example project that includes snapshot tests that are also used by the CI system to confirm the framework continues to work as expected. | ||
|
||
## Development Requirements | ||
|
||
Your local environment should match one these supported versions. | ||
|
||
| Software | Versions | | ||
| --- | --- | | ||
| Xcode | 12.5.1 (iOS 13 and 14), 14.3.1 (iOS 16) [.github/workflows/ci.yml](https://github.com/cashapp/AccessibilitySnapshot/blob/master/.github/workflows/ci.yml) | | ||
| Ruby | 3.2.2 [Example/Gemfile](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Example/Gemfile) | | ||
| Bundler | 2.4.22 [Gemfile.lock](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Gemfile.lock) | | ||
| Simulators | iOS 13.7 - iPhone 12 Pro, iOS 14.5 - iPhone 12 Pro, iOS 16.4 - iPhone 14 Pro [Scripts/build.swift](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Scripts/build.swift) | | ||
|
||
### Setting up environment | ||
|
||
1. Install the Xcode IDE | ||
|
||
- To install Xcode versions you can visit the [Apple developer downloads](https://developer.apple.com/download/all/) site directly. | ||
- Verify your Xcode version and installation with: | ||
|
||
```sh | ||
xcode-select -p | ||
``` | ||
|
||
1. RVM can be used for Ruby version management. Instructions here: https://rvm.io/rvm/install | ||
|
||
- Install the current Ruby version | ||
|
||
```sh | ||
rvm install ruby-3.2.2 --with-openssl-dir=/opt/homebrew/opt/openssl@3 | ||
``` | ||
|
||
NOTE: This can take a while as RVM will typically build Ruby from source. | ||
|
||
- Restart your shell session after installing RVM and Ruby or: `source ~/.zshrc` / `source ~/.bashrc` | ||
|
||
- Verify your installation of RVM and Ruby | ||
|
||
```sh | ||
which ruby # /Users/<LDAP>/.rvm/rubies/ruby-3.2.2/bin/ruby | ||
``` | ||
|
||
1. Install Bundler | ||
|
||
```sh | ||
gem install bundler -v 2.4.22 | ||
``` | ||
|
||
Verify your installation of Bundler | ||
|
||
```sh | ||
bundle -v # Bundler version 2.4.22 | ||
``` | ||
|
||
### Building the project | ||
|
||
1. Install the required Gems in the `Gemfile` | ||
|
||
```sh | ||
bundle install | ||
``` | ||
|
||
1. Install CocoaPod dependencies from `Podfile` and update the CocoaPod source repositories | ||
|
||
```sh | ||
bundle exec pod install | ||
``` | ||
|
||
1. Open the newly generated workspace | ||
|
||
```sh | ||
xed AccessibilitySnapshot.xcworkspace | ||
``` | ||
|
||
### Getting Snapshot Images from CI | ||
|
||
Test results are archived for CI jobs. When there is a failure because of a snapshot test image changing those images can be extracted from the archive. See [Scripts/ExtractImagesFromTestResults.swift](https://github.com/cashapp/AccessibilitySnapshot/blob/master/Scripts/ExtractImagesFromTestResults.swift) for instructions. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,40 +2,42 @@ | |
|
||
// Usage: | ||
// | ||
// 1. Download the Test Results bundle from the CI build. From your PR, go to | ||
// 1. Install xcparse: brew install chargepoint/xcparse/xcparse | ||
// | ||
// 2. Download the Test Results bundle from the CI build. From your PR, go to | ||
// Checks > CI, then click on "Test Results" in the Artifacts section. | ||
// | ||
// 2. Run this script with the path to this download. | ||
// 3. Run this script with the path to this download. | ||
// | ||
// ./Scripts/ExtractImagesFromTestResults.sh /path/to/Test\ Results | ||
// ./Scripts/ExtractImagesFromTestResults.swift /path/to/Test\ Results | ||
// | ||
// 3. This will create a directory called "tmp" that contains all of the | ||
// 4. This will create a directory called "tmp" that contains all of the | ||
// extracted images (reference, failed, and diff) organized by OS version, | ||
// test class name, and test name. | ||
// test class name, and test name. | ||
|
||
import Foundation | ||
|
||
enum TaskError: Error { | ||
case code(Int32) | ||
case code(Int32) | ||
} | ||
|
||
func execute(commandPath: String, arguments: [String]) throws { | ||
let task = Process() | ||
task.launchPath = commandPath | ||
task.arguments = arguments | ||
let task = Process() | ||
task.launchPath = commandPath | ||
task.arguments = arguments | ||
|
||
task.launch() | ||
task.launch() | ||
|
||
task.waitUntilExit() | ||
task.waitUntilExit() | ||
|
||
guard task.terminationStatus == 0 else { | ||
throw TaskError.code(task.terminationStatus) | ||
} | ||
guard task.terminationStatus == 0 else { | ||
throw TaskError.code(task.terminationStatus) | ||
} | ||
} | ||
|
||
if CommandLine.arguments.count < 2 || CommandLine.arguments[1] == "--help" || CommandLine.arguments[1] == "-h" { | ||
print("usage: CompareRenamedSnapshots SNAPSHOT_LIST_FILE OLD_REPO_PATH NEW_DEVICE_ID OLD_DEVICE_ID") | ||
exit(0) | ||
print("usage: CompareRenamedSnapshots SNAPSHOT_LIST_FILE OLD_REPO_PATH NEW_DEVICE_ID OLD_DEVICE_ID") | ||
exit(0) | ||
} | ||
|
||
let testResultsContainerPath = CommandLine.arguments[1] | ||
|
@@ -47,38 +49,38 @@ guard let testResultsEnumerator = fileManager.enumerator(at: testResultsContaine | |
} | ||
|
||
func parseResults(testResultsPath: String, outputPath: String) throws { | ||
try execute( | ||
commandPath: "/opt/homebrew/bin/xcparse", | ||
arguments: ["screenshots", "--os", "--test", testResultsPath, outputPath] | ||
) | ||
try execute( | ||
commandPath: "/opt/homebrew/bin/xcparse", | ||
arguments: ["screenshots", "--os", "--test", testResultsPath, outputPath] | ||
) | ||
} | ||
|
||
for case let fileURL as URL in testResultsEnumerator { | ||
if fileURL.path.hasSuffix(".xcresult") { | ||
try parseResults(testResultsPath: fileURL.path, outputPath: "tmp") | ||
try parseResults(testResultsPath: fileURL.path, outputPath: "tmp") | ||
} | ||
} | ||
|
||
/* | ||
|
||
As a future enhancement, this script could also rename the failed snapshot images. One complicating | ||
factor of this is the tests with multiple associated reference images (e.g. when using identifiers) | ||
don't differentiate the resulting images in the test result bundle. We'll also need to handle | ||
renaming based on which snapshot engine was used: | ||
As a future enhancement, this script could also rename the failed snapshot images. One complicating | ||
factor of this is the tests with multiple associated reference images (e.g. when using identifiers) | ||
don't differentiate the resulting images in the test result bundle. We'll also need to handle | ||
renaming based on which snapshot engine was used: | ||
|
||
FBSnapshotTestCase | ||
FBSnapshotTestCase | ||
|
||
/Example/SnapshotTests/ReferenceImages/_64/<TestClass>/ | ||
/Example/SnapshotTests/ReferenceImages/_64/<TestClass>/ | ||
|
||
<TestName>[email protected] | ||
<TestName>[email protected] | ||
<TestName>[email protected] | ||
<TestName>[email protected] | ||
|
||
SnapshotTesting | ||
SnapshotTesting | ||
|
||
/Example/SnapshotTests/__Snapshots__/<TestClass>/ | ||
/Example/SnapshotTests/__Snapshots__/<TestClass>/ | ||
|
||
<TestName>.390x844-14-5-3x.png | ||
<TestName>.375x812-13-7-3x.png | ||
<TestName>.390x844-14-5-3x.png | ||
<TestName>.375x812-13-7-3x.png | ||
|
||
*/ | ||
*/ | ||
|