diff --git a/TULLY_APP_SMOKE.md b/TULLY_APP_SMOKE.md
new file mode 100644
index 000000000..6ce1dfa8d
--- /dev/null
+++ b/TULLY_APP_SMOKE.md
@@ -0,0 +1,9 @@
+# Tully GitHub App Smoke
+
+This fork-local file validates the private Tully Review GitHub App installation on the thegeekmachine Hystrix fork.
+
+Expected outputs:
+
+- Tully Review receives a signed pull request webhook.
+- Tully Review can create a check run on the pull request head commit.
+- Tully Review can create or update a summary pull request comment.