Included files and folders
The elgentos testing suite comes with a variety of files and folders. To understand the testing suite - and therefore to make the most out of it - it's important to understand the purpose of these files and folders. To make things easier for you, we've added the following files and functionalities.
The base-tests folder contains all tests written by us. When you install the testing suite, another folder will be
created called tests. This is where you can overwrite all our tests. To do this, simply copy the file you want to make
changes to from the base-tests folder into the tests folder. The testing suite will automatically look for files in
tests and use those instead of its base-tests counterpart. You can find this functionality in playwright.config.ts
(or playwright.config.example.ts if you're reading this before installing - as you should!)
Warning
You should not make changes directly to the base-tests folder. Whenever the testing suite is updated, these tests will be replaced by their updated versions. If you make a change to these tests and then run an update, you'll lose all changes you've made.
TL;DR: the tests folder is a copy of base-tests in which you can make your changes. The testing suite will always
prefer your custom code over the base tests.
The config folder, unsurprisingly, contains a variety of files needed to run the tests. We've placed these files here
to make it easier for you to customize your testing suite. Let's go over the files you'll find in the config folder:
Playwright best practices
tell us to prefer 'user-facing attributes' over XPath or CSS selectors for your locators. We've collected all of these
identifiers in this JSON file, grouped together in alphabetical order. For example: on line 38, you'll find the variable
simpleProductTitle, with the value Push It Messenger Bag. You can simply change the value here to whatever product
you want to test with, and all tests that need to confirm the name of the product will now check for this new value.
You're welcome! Note that this file is imported as follows:
import { UIReference } from '@config';This file contains a variety of input values you can use for your tests. Most notably is addresCountries, which is a
list of countries that can be used to create/fill in an address. Note that this file is imported as follows:
import { inputValues } from '@config';Similar to the input values and element identifiers, we'll also need to check to ensure the desired outcome of a test
has been reached. This is done with what we call 'outcome markers', which can be found in outcomer-markers.json. We've
decided to call them outcome marker because they can be a variety of things, but they're strings more often than not.
Note that this file is imported as follows:
import { outcomeMarker } from '@config';slugs.json contains a list of slugs used to navigate to the correct page. 🤷🏻 Note that this file is imported as follows:
import { slugs } from '@config';Test toggles used to be part of the testing suite, as it would allow users to control what tests to run. However, since Playwright essentially offers us the same kind of functionality in tags but better, we've decided to remove these. This file will be removed in a future update. You can read more about this in Running the suite.
index.ts contains the method that allows the import to be the way it's described above. Essentially, index.ts checks
for the presence of the config files in both tests and base-tests. The file provides a way to load default JSON
configs, optionally override them with project-specific files in tests/config, and export the merged results. This
avoids hardcoding the selectors/test data in your tests and makes them configurable per project.
First things first, what is a pom? Well, Playwright uses Page Object Models to help with structuring your testing suite. This makes maintenance a lot easier. A page object represents a part of your web application. An e-commerce web application might have a home page, a listings page and a checkout page. Each of them can be represented by page object models. You can read more about how that works in the Playwright docs about Page Object Models.
For our testing suite, we've divided them into two folders: adminhtml and frontend. The former deals with the Magento
admin part of your site, and is therefore mostly used in setup.spec.ts. frontend contains all other pom files, for
all the separate parts of your website.
When you run the setup, you're essentially running most of what is in magentoAdmin.page.ts. This file contains the
steps needed to create a coupon through addCartPriceRule(), check if a customer exists before creating one through
checkIfCustomerExists(), disable the login CAPTCHA (disableLoginCaptcha()), enable multiple admin logins
(enableMultipleAdminLogins()) and logging in to the admin section (login()). Please check setting up for more
information.
Note
You can decide what steps are needed for your tests. For example, if the website you want to run tests on does not
use discount codes, you can simply add a test.skip() to the actual test in setup.spec.ts. You can read more about
this in Setting Up.
These files all contain the methods needed to perform the tests. For example, account.page.ts contains a function to
add an address to the test account. Here's what you need to know about these files from a design philosophy perspective:
- Whenever possible, variables are declared in the
constructor. Note that this is not always possible. - Most functions end with an
expectto check if the test has been completed correctly. We are currently working on creating more defined standards for our testing suite, and these will likely move to the test files themselves. For now, however, most are defined in the{pagename}.page.ts.
This folder only contains magewire.d.ts, which defines the magewire property for TypeScript. You can read more about
Magewire for Hyvä here: What is Magewire? - Hyvä Docs.
Utils contains a few files that help the testing suite in generic ways, but are not really part of actual testing.
These are:
-
notificationValidator.utils.ts: can be used to catch messages/notifications from Magento 2 and push them to the report, as well asreturnit to the test/function. We are currently still working on making this the standard throughout the testing suite, as we want to avoidconsole.logandechoas must as possible. -
magewire.utils.ts: The functions here help with handling everything that runs through Magewire. -
env.utils.ts: this simple function is to better adhere to the DRY principle (Don't Repeat Yourself): this function is called whenever an environment variable is required. If that variable is not set, this throws an error. -
loggerfolder and files: these essentially create a customizable logger utility that prints clean, formatted log messages to the console (or terminal), with support for different log levels (log, info, warn, error) and conditional debug logging.
Right in the tests folder are all the spec files: these are the files that contain the actual tests. We recommend
reading through these files before running them (make sure to open the corresponding page file as well), so you know
what is being tested. We won't explain all the files, but here's a general overview of what you'll find in these files.
Note
After reading this, we recommend checking out contact.spec.ts and the related contact.page.ts! This is a
relatively easy test and can really help with understanding the 'grammar' of the testing suite.
These files always start with the relevant imports. Certain files might have a beforeEach() or beforeAll() at the
top of the file. These are steps that Playwright will perform before each test, or before all tests. In the case of
account.spec.ts, the beforeEach() is placed within a test.describe(). test.describe() is used to group tests
together. By placing beforeEach() within describe, it will run only before the tests within this group.
Each test comes with a description using Gherkin 'grammar' within JSDoc syntax. For example, this is the description for
the test Place_order_for_simple_product() from checkout.spec.ts:
/**
* @feature Place order for simple product
* @scenario User places an order for a simple product
* @given I have a product in my cart
* @and I am on any page
* @when I navigate to the checkout
* @and I fill in the required fields
* @and I click the button to place my order
* @then I should see a confirmation that my order has been placed
* @and a order number should be created and show to me
*/In a test, you can usually expect the following elements, roughly in the order they're listed in. Let's go through it
by looking at the test Add_coupon_code_in_cart(), starting on line 117 in cart.spec.ts:
- an instance is created of the pom class needed for the test, as well as variables that are required. In this case,
these are
const browserEngineandconst cart. - If an
envvariable is required, we use therequireEnv()function to ensure it's set, or throw an error otherwise. - Then, a call is made to the function that performs the steps necessary to complete the action we want to test (in this case, adding a coupon code to the cart).
- Usually, you'll see an
expect()at the end of the test. If you don't see them, it's very likely you can instead find them in the correspondingpage.tsfile. We're working on standardizing where we place the assertions.
Other than everything mentioned above, here are a few more files that are important to ensuring the testing suite can function.
This fill will be copied to .env, and contains a set of variables. When you install the testing suite (see Setting
up), you will be asked to fill in these variables. The setup also suggests defaults to speed up the process. Here's
an overview of which variables you'll find and what they do:
-
PLAYWRIGHT_BASE_URL - This is the url Playwright will run the tests against. Slugs are appended to this URL. Default is
https://hyva-demo.elgentos.io/. -
PLAYWRIGHT_PRODUCTION_URL and PLAYWRIGHT_REVIEW_URL are currently not in use, but can be used to differentiate your tests and environments, if that's necessary for your tests.
-
MAGENTO_ADMIN_SLUG - the slug used to log in to the admin section. Default is
admin. -
MAGENTO_ADMIN_USERNAME - the username used to log in to the admin section. Runs a shell command using Node.js to retrieve the current operating system user.
-
MAGENTO_ADMIN_PASSWORD - the password used to log in to the admin section. Default is
Test1234!. -
MAGENTO_THEME_LOCALE - sets the locale for your theme. Currently not in active use. Default is
nl_NL. -
MAGENTO_NEW_ACCOUNT_PASSWORD is the password used by the test that creates a new account. Default is
NewTest1234! -
MAGENTO_EXISTING_ACCOUNT_EMAIL_{BROWSERNAME} (CHROMIUM, FIREFOX, WEBKIT) - these are the email addresses used by the testing suite to log in and perform tests. When you run
setup.spec.ts, these are used to create test accounts (the password will be MAGENTO_EXISTING_ACCOUNT_PASSWORD). Those accounts are then used to perform tests for which an account is necessary. Default is[email protected],[email protected]and[email protected]. -
MAGENTO_EXISTING_ACCOUNT_PASSWORD - The password used to create the test accounts.
-
MAGENTO_EXISTING_ACCOUNT_CHANGED_PASSWORD - The password used for the test
Change_password. Default isAanpassenKan@0212. -
MAGENTO_COUPON_CODE_{BROWSERNAME} (CHROMIUM, FIREFOX, WEBKIT) - these values will be used in
setup.spec.tsto create coupon codes. This value is what tests will later fill in to activate the coupon code.
these help with the set-up of the testing suite.
these help with the set-up of the testing suite.
a file with global Playwright settings
this file allows the existence of base-tests and tests.
We've included a Makefile.playwright file that you can include in your own Makefile. This will allow you to run the
testing suite from the main magento2 folder.
this file will automatically translate your csv files from the i18n folder.
This is currently not used by our suite, and will most likely be phased out.