RNTester: Setup E2E tests for iOS and Android with Appium, Jest and Cucumber #35824

adzironman · 2023-01-13T12:27:59Z

please check out this one first ➡️ #35016

Summary

The motivation is to create an e2e testing solution for the rn-tester default app. With cooperation with Callstack developers, we are trying to make a working tool. This PR is adding configuration for the BDD approach with cucumber.
Next steps:

integrate with devices' cloud
build an app automatically

Changelog

[General] [Added] - Added first working configuration for e2e testing with Appium, Jest and Cucumber

Test Plan

Follow Readme File

Comparison of work for approach with and without Cucumber

In a clean jest+wdio setup, test scenarios are written in one file:

In order to use the BDD approach you need to do a few additional steps in comparison to jest+wdio solution implemented here.

Additional steps:

Add .feature file with test scenarios written in Gherkin syntax

2. Add a runner file

call appropriate methods in the same order as the steps in the feature file. This is a drawback of that solution.

implement methods(used by gherking steps) that will be used only in that feature. Steps shared between features can be implemented in common_steps file

Videos

Android:

final_andorid.mp4

iOS:

final_ios.mp4

…/react-native into feature/rn-tester-e2e

RNTester - e2e cucumber new approach

facebook-github-bot · 2023-01-13T12:28:05Z

Hi @adzironman!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

github-actions · 2023-01-13T12:29:02Z

	Warnings
⚠️	🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against b3d21be

mohit23x · 2023-01-16T06:54:23Z

This Appium setup is cool, we made a babel plugin to find elements with testID, do check if it's useful in this setup. https://www.npmjs.com/package/babel-plugin-format-testid

facebook-github-bot · 2023-01-16T08:07:06Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

cipolleschi

I left a couple of comments that are iOS specific. A part from that, great work!

A question out of ignorance: if we have to implement and call the JS methods, can we get rid of Gerkin? It looks a repetition of the test description and they could go out of sync easily (or become maintenance burden). If we have JS methods with descriptive names, we can use a code-as-documentation approach and simplify the setup.
Do you think that it is possible? (Another approach could be to generate the Gerkin file from the JS one)

cipolleschi · 2023-01-16T14:13:39Z

packages/rn-tester-e2e/README.md

+If you are M1 mac user:
+1. Install ffi package `gem install ffi -v '1.15.5' --source 'https://rubygems.org/'`
+2. Install pods with new architecture, e.g. using JSC `USE_HERMES=0 arch -x86_64 pod install`
+3. Open the generated `RNTesterPods.xcworkspace`.
+4. Build the app.


This should not be required anymore.

cipolleschi · 2023-01-16T14:15:25Z

packages/rn-tester-e2e/README.md

+3. Open the generated `RNTesterPods.xcworkspace`.
+4. Build the app.
+
+Find the **RNTester.app** in `~/Library/Developer/Xcode/DerivedData/RNTesterPods-{id}/Build/Products/Debug-iphonesimulator` and move the app to the following directory `/react-native/packages/rn-tester-e2e/apps`


I think we can use xcodebuild from the command line and tell it to generate the app in some specific folder: above listing 8 and 9

NickGerleman

Leaving some very superficial feedback. This will need some cleanup I think.

More generally, I would try to think of the scenarios we want to test E2E, and what we need more than this to be able to accomplish it.

The basis here lets the test runner navigate, check for existence of controls/text. But I think the more valuable bits are understanding the impact a props, etc has on the native component. Either visually via screenshot testing, or via approaches like snapshotting the native component tree.

With the screen objects, I would try to settle on some pattern. E.g. an object per RNTester page, if we are going to have them. Though unless we solve the problem of engineers adding to the screen object when adding RNTester examples, we should not assume these will be kept up to date.

I also expect that Appium may balloon the number of dependencies we have and are caching. Please keep an eye out for its effect on the lockfile, especially for anything which will immediately come back as a security bit.

NickGerleman · 2023-01-16T14:34:07Z

packages/rn-tester-e2e/jest.setup.js

+import { beforeEach, afterEach, jest } from '@jest/globals';
+
+
+jest.setTimeout(40000);


This contradicts the timeout we set in the config?

NickGerleman · 2023-01-16T14:34:24Z

packages/rn-tester-e2e/jest.setup.js

+  host: 'localhost',
+  port: 4723,
+  waitforTimeout: 30000,
+  logLevel: 'silent',


Suggested change

logLevel: 'silent',

logLevel: 'error',

NickGerleman · 2023-01-16T14:34:53Z

packages/rn-tester-e2e/package.json

+    "private": true,
+    "keywords": [],
+    "author": "",
+    "license": "ISC",


Suggested change

"license": "ISC",

"license": "MIT",

NickGerleman · 2023-01-16T14:35:47Z

packages/rn-tester-e2e/package.json

+        "@babel/eslint-parser": "^7.19.1",
+        "@babel/runtime": "^7.12.5",
+        "@react-native-community/eslint-config": "^2.0.0",
+        "appium": "^2.0.0-beta.40",


Not seeing reflected in yarn lockfile. Could we yarn install (also so we can see how many deps added)

NickGerleman · 2023-01-16T14:36:18Z

packages/rn-tester-e2e/package.json

+        "jest-cucumber": "^3.0.1",
+        "jest-html-reporter": "^3.7.0",
+        "metro-react-native-babel-preset": "^0.70.3",
+        "react-test-renderer": "18.0.0",


Seems unused?

NickGerleman · 2023-01-16T14:38:30Z

packages/rn-tester-e2e/test/screenObjects/components.screen.js

+
+  buttonComponentElement = Utils.platformSelect({
+    ios: '[label="Button Simple React Native button component."]',
+    android: '~Button Simple React Native button component.',


Can we search by test id instead of string?

NickGerleman · 2023-01-16T14:39:22Z

packages/rn-tester-e2e/test/screenObjects/buttonComponent.screen.js

+class ButtonComponentScreen {
+
+    buttonScreenElement = Utils.platformSelect({
+        ios: '[label="Button"]',


Can we search by test id instead of any string? Also means we can remove this per-platform bits.

NickGerleman · 2023-01-16T14:41:17Z

packages/rn-tester-e2e/test/common_steps/common.steps.js

+// Common steps reusable between different features
+export const userIsOnMainScreen = (given) => {
+    given('User is on the main screen', async () => {
+        await driver.pause(2000);


Harcoded waits like this should be avoided

NickGerleman · 2023-01-16T14:42:14Z

packages/rn-tester-e2e/test/features/buttonComponentScreen.feature

@@ -0,0 +1,19 @@
+Feature: Button component screen


Do we really need cucumber, if we don't use it anywhere else.

NickGerleman · 2023-01-16T14:42:48Z

packages/rn-tester-e2e/test/helpers/utils.js

+    await driver.$(locator).waitForDisplayed();
+    return driver.$(locator).getText();
+  }
+


All of these should be formatted/linted, have copyright headers.

kelset · 2023-01-16T15:59:42Z

@NickGerleman @cipolleschi please ignore this PR for now, this is part 2 of #35786 but the team is still working on them both (you'll see that a good part of your comments I've already covered in the other PR).

I'll ping you once this 2nd PR should be looked at ;)

kelset · 2023-02-23T12:13:58Z

let's close this one in favour of #36267, things have changed enough that adding cucumber will need (if we decide to do it) much further down the line

mateuszm22 and others added 30 commits October 18, 2022 13:06

e2e for rn-tester

2d7b8d2

added apps folder

c9183a7

Merge branch 'facebook:main' into feature/rn-tester-e2e

d45231d

ios xcuitest conf

83a9cb0

specs and screens adjustments

1cc5c9c

Add ios readme

ae7efd4

switch to jest runner, moved capabilities from package.json, refactor

566bcbd

Merge branch 'feature/rn-tester-e2e' of https://github.com/mateuszm22…

698a0d6

…/react-native into feature/rn-tester-e2e

Cucumber implemented

e82b77b

app removed

f884711

refactor

320dc82

change readme name

7fb9ec0

Merge branch 'facebook:main' into feature/rn-tester-e2e

0a186ee

cucumber feature and runner adjustments

253ac0b

switch typo fix

faff18a

refactor

1602579

Merge branch 'feature/rn-tester-e2e' of https://github.com/mateuszm22…

192ae42

…/react-native into feature/rn-tester-e2e

refactor

ee7fbb9

e2e folder moved to rn-tester-e2e

78a70a4

e2e rn-tester package moved

a304fc9

Instruction on how to add a test and how to execute

d3e745b

all folders and files described in readme

da78d7c

describedAllFoldersAndFiles md extension added

1b0d0f1

Updating readme with building app info

82e4786

Updates for rn-tester-e2e markdown

5e2a54e

Small fixes to e2e cucumbers markdowns

a87021d

Updating readme files

f52dbfe

Updating android config

1d6c99b

Adding common steps file

f9cce01

Removing unnecessary files

f2ef916

adiorsz and others added 6 commits January 10, 2023 15:57

Updating cucumber tests

2789ed0

github-actions fixes

3ebe200

removing commented lines

de05312

Removing some user specific values

97d6045

Updating typos in readme

be4415d

Merge pull request #1 from mateuszm22/rn-e2e-cucumber-new-approach

6e655c3

RNTester - e2e cucumber new approach

react-native-bot added the No CLA Authors need to sign the CLA before a PR can be reviewed. label Jan 14, 2023

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 16, 2023

react-native-bot removed the No CLA Authors need to sign the CLA before a PR can be reviewed. label Jan 16, 2023

github actions fixes

fe819d1

react-native-bot added the Type: Enhancement A new feature or enhancement of an existing feature. label Jan 16, 2023

adzironman marked this pull request as ready for review January 16, 2023 12:11

cipolleschi reviewed Jan 16, 2023

View reviewed changes

NickGerleman reviewed Jan 16, 2023

View reviewed changes

kelset mentioned this pull request Jan 16, 2023

RNTester: Setup E2E tests for iOS and Android with Appium and Jest #35016

Closed

package.json cleanup

b3d21be

kelset marked this pull request as draft January 20, 2023 13:25

kelset closed this Feb 23, 2023

		import { beforeEach, afterEach, jest } from '@jest/globals';


		jest.setTimeout(40000);

RNTester: Setup E2E tests for iOS and Android with Appium, Jest and Cucumber #35824

RNTester: Setup E2E tests for iOS and Android with Appium, Jest and Cucumber #35824

Uh oh!

Conversation

adzironman commented Jan 13, 2023 • edited by kelset Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

please check out this one first ➡️ #35016

Summary

Changelog

Test Plan

Comparison of work for approach with and without Cucumber

Videos

Uh oh!

facebook-github-bot commented Jan 13, 2023

Action Required

Process

Uh oh!

github-actions bot commented Jan 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mohit23x commented Jan 16, 2023

Uh oh!

facebook-github-bot commented Jan 16, 2023

Uh oh!

cipolleschi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NickGerleman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kelset commented Jan 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kelset commented Feb 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

adzironman commented Jan 13, 2023 •

edited by kelset

Loading

github-actions bot commented Jan 13, 2023 •

edited

Loading

kelset commented Jan 16, 2023 •

edited

Loading

kelset commented Feb 23, 2023 •

edited

Loading