[🐛 Bug]: DefaultSlotMatcher is considering extension capabilities to determine whether the slot matches the requested capabilities

[sbabcoc]

What happened?

My Selenium framework includes the ability to manage a local Grid instance, including Appium nodes. Because the Appium server needs an execution context to function, I have the ability to run the server via PM2. When this configuration is employed, the shutdown procedure is different. Consequently, I need to include a property in the capabilities object that indicates this configuration. This is done via an extension capability names nord:options. The expectation is that this capability will be available to query, but won't affect slot matching. Unfortunately, this isn't currently the case. Adding this extension capability causes the corresponding node to no longer match Appium session requests.

The scenario that caused my immediate issue is that both the node capabilities and the new session request define values for nord:options, each specifying a different informational property. The slot matcher compares the values of these capabilities and rejects the slot because the objects differ.

How can we reproduce the issue?

Create a grid with a node whose capabilities include a `nord:options` extension capability. Try to acquire a session from this node with capabilities that specify a different value for `nord:options`.

EXPECT: The slot matcher should provide a session
ACTUAL: No matching session is provided

Relevant log output

FAILED: com.nordstrom.automation.selenium.android.AndroidTest.testSearchActivity
        {
  "constName": "ANDROID",
  "description": "",
  "className": "com.nordstrom.automation.selenium.platform.TargetType",
  "class": "com.nordstrom.automation.selenium.listeners.PlatformInterceptor$PlatformIdentity"
}
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at org.openqa.selenium.support.ui.FluentWait.propagateIfNotIgnored(FluentWait.java:252)
	at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:213)
	at com.nordstrom.automation.selenium.core.DriverManager.injectDriver(DriverManager.java:169)
	at com.nordstrom.automation.selenium.core.DriverManager.beforeInvocation(DriverManager.java:105)
	at com.nordstrom.automation.selenium.listeners.DriverListener.beforeInvocation(DriverListener.java:55)
	at com.nordstrom.automation.testng.AbstractListenerChain.beforeInvocation(AbstractListenerChain.java:231)
	at org.testng.internal.invokers.InvokedMethodListenerInvoker.invokeListener(InvokedMethodListenerInvoker.java:56)
	at org.testng.internal.invokers.BaseInvoker.runInvokedMethodListeners(BaseInvoker.java:67)
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:642)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:969)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.testng.TestRunner.privateRun(TestRunner.java:829)
	at org.testng.TestRunner.run(TestRunner.java:602)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:437)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:431)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:391)
	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
	at org.testng.TestNG.runSuites(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1067)
	at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:115)
	at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:293)
	at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:91)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at com.nordstrom.automation.selenium.core.GridUtility.getDriver(GridUtility.java:166)
	at com.nordstrom.automation.selenium.core.GridUtility.getDriver(GridUtility.java:128)
	at com.nordstrom.automation.selenium.core.DriverManager$1.apply(DriverManager.java:284)
	at com.nordstrom.automation.selenium.core.DriverManager$1.apply(DriverManager.java:1)
	at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:203)
	... 29 more
Caused by: org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Could not start a new session. New session request timed out 
Host info: host: 'DESKTOP-21IB3RG', ip: '192.168.254.20'
Build info: version: '4.23.0', revision: '4df0a231af'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.4'
Driver info: driver.version: unknown
Build info: version: '4.23.0', revision: '4df0a231af'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.4'
Driver info: io.appium.java_client.android.AndroidDriver
Command: [null, newSession {capabilities=[Capabilities {appium:app: https://github.com/appium/a..., appium:automationName: Espresso, appium:forceEspressoRebuild: true, appium:host: 0.0.0.0, appium:showGradleLog: true, nord:options: {personality: Espresso}, platformName: ANDROID}]}]
Capabilities {appium:app: https://github.com/appium/a..., appium:automationName: Espresso, appium:forceEspressoRebuild: true, appium:host: 0.0.0.0, appium:showGradleLog: true, nord:options: {personality: Espresso}, platformName: ANDROID}
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:114)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:75)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:61)
	at io.appium.java_client.remote.AppiumCommandExecutor.createSession(AppiumCommandExecutor.java:176)
	at io.appium.java_client.remote.AppiumCommandExecutor.execute(AppiumCommandExecutor.java:237)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:527)
	at io.appium.java_client.AppiumDriver.startSession(AppiumDriver.java:270)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
	at io.appium.java_client.AppiumDriver.<init>(AppiumDriver.java:91)
	at io.appium.java_client.AppiumDriver.<init>(AppiumDriver.java:103)
	at io.appium.java_client.android.AndroidDriver.<init>(AndroidDriver.java:109)
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	... 36 more

Operating System

Windows 10

Selenium version

4.23.0

What are the browser(s) and version(s) where you see this issue?

All

What are the browser driver(s) and version(s) where you see this issue?

AndroidDriver (RemoteWebDriver)

Are you using Selenium Grid?

4.23.0

[github-actions]

@sbabcoc, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[sbabcoc]

The wrinkle here is that we now have Appium extension capabilities which are being matched through this improper behavior. Fixing this is going to require either adding explicit matching for Appium capabilities in DefaultNodeMatcher or requiring a specific Appium node matcher.

[sbabcoc]

There was an issue filed about one year ago that seems related to what I'm seeing: #12640

[sbabcoc]

Also #14216
The Appium-specific logic added to RelaySessionFactory is a direct result of incorporating too much processing on new session requests. Decisions about how to handle unspecified values should be made by the consumer, not applied by generic transport-layer implementation.

[sbabcoc]

The W3C specification confirms my instincts. https://www.w3.org/TR/webdriver1/#new-session

... an intermediary node cannot forward extension capabilities specific to that implementation to an endpoint node...

The spec strongly recommends that vendor-specific features be passed via top-level parameters, not via requested capabilities. Regardless, these sorts of vendor-specific features should be processed by vendor-specific slot matchers. The default slot matcher should not include handling of vendor-specific features.

[VietND96]

If taking your example, capabilities include a nord:options. When starting Node, did you add config detect-drivers = false under [node] section?
It looks like you are setting custom capabilities for matching specific Nodes - https://www.selenium.dev/documentation/grid/configuration/toml_options/#setting-custom-capabilities-for-matching-specific-nodes
Any cap with format ext:name (not special extension e.g se:, goog:, etc.). It should be considered in SlotMatcher.
We also added tests in docker-selenium to test this feature
https://github.com/SeleniumHQ/docker-selenium/blob/d43142dedd1242f7e5f62b09e9cda525adff5fbc/tests/SeleniumTests/__init__.py#L162-L163

[sbabcoc]

@VietND96 Capabilities with "simple" values (Boolean/Number/String) should be considered for slot matching, even if their capability names specify a prefix. Capabilities with "complex" values (Array/Object) should be ignored for slot matching if their capability names specify a prefix.
We shouldn't give special treatment to any prefixes, because this produces inconsistent behavior. We also shouldn't be altering clients' requested capabilities in the node matching process.

[sbabcoc]

@VietND96 Regarding the nord:options extension capability, this is specified as an object, which will be ignored for purposes of slot matching.
Regarding the test statements you've specified, these capabilities have string values, which will be considered for purposes of slot matching.

[github-actions]

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 14 days.