Skip to content

Add support for screen reader blur and focus (accessibility) #24642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Add support for screen reader blur and focus (accessibility) #24642

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
df0f2be
add onAccessibilityBlur and onAccessibilityFocus for Android and iOS
Apr 17, 2019
2266cac
update component props
Apr 24, 2019
67c4a04
add support for TextInput
Apr 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion Libraries/Components/Button.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,16 @@ const View = require('View');

const invariant = require('invariant');

import type {PressEvent} from 'CoreEventTypes';
import type {SyntheticEvent, PressEvent} from 'CoreEventTypes';

type TargetEvent = SyntheticEvent<
$ReadOnly<{|
target: number,
|}>,
>;

type AccessibilityBlurEvent = TargetEvent;
type AccessibilityFocusEvent = TargetEvent;

type ButtonProps = $ReadOnly<{|
/**
Expand Down Expand Up @@ -92,6 +101,17 @@ type ButtonProps = $ReadOnly<{|
* Used to locate this view in end-to-end tests.
*/
testID?: ?string,

onAccessibilityBlur?: ?(AccessibilityBlurEvent) => void,

/**
* When `accessible` is true and VoiceOver (iOS) or TalkBack (Android) is
* enabled, this event is fired immediately once the element gains the screen
* reader focus.
*
* See http://facebook.github.io/react-native/docs/view.html#onaccessibilityfocus
*/
onAccessibilityFocus?: ?(AccessibilityFocusEvent) => void,
|}>;

/**
Expand Down Expand Up @@ -127,6 +147,8 @@ class Button extends React.Component<ButtonProps> {
const {
accessibilityLabel,
color,
onAccessibilityFocus,
onAccessibilityBlur,
onPress,
title,
hasTVPreferredFocus,
Expand Down Expand Up @@ -165,6 +187,8 @@ class Button extends React.Component<ButtonProps> {
<Touchable
accessibilityLabel={accessibilityLabel}
accessibilityRole="button"
onAccessibilityBlur={onAccessibilityBlur}
onAccessibilityFocus={onAccessibilityFocus}
accessibilityStates={accessibilityStates}
hasTVPreferredFocus={hasTVPreferredFocus}
nextFocusDown={nextFocusDown}
Expand Down
14 changes: 14 additions & 0 deletions Libraries/Components/TextInput/TextInput.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1098,6 +1098,8 @@ const TextInput = createReactClass({
<TouchableWithoutFeedback
onLayout={props.onLayout}
onPress={this._onPress}
onAccessibilityBlur={this._onAccessibilityBlur}
onAccessibilityFocus={this._onAccessibilityFocus}
rejectResponderTermination={true}
accessible={props.accessible}
accessibilityLabel={props.accessibilityLabel}
Expand Down Expand Up @@ -1150,6 +1152,8 @@ const TextInput = createReactClass({
<TouchableWithoutFeedback
onLayout={props.onLayout}
onPress={this._onPress}
onAccessibilityBlur={this._onAccessibilityBlur}
onAccessibilityFocus={this._onAccessibilityFocus}
rejectResponderTermination={props.rejectResponderTermination}
accessible={props.accessible}
accessibilityLabel={props.accessibilityLabel}
Expand Down Expand Up @@ -1208,6 +1212,8 @@ const TextInput = createReactClass({
<TouchableWithoutFeedback
onLayout={props.onLayout}
onPress={this._onPress}
onAccessibilityBlur={this._onAccessibilityBlur}
onAccessibilityFocus={this._onAccessibilityFocus}
accessible={this.props.accessible}
accessibilityLabel={this.props.accessibilityLabel}
accessibilityRole={this.props.accessibilityRole}
Expand Down Expand Up @@ -1325,6 +1331,14 @@ const TextInput = createReactClass({
}
},

_onAccessibilityBlur: function(event: BlurEvent) {
Copy link
Contributor

@blavalla blavalla May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shoudln't this be AccessibilityBlurEvent not BlurEvent?

this.props.oncAccessibilityBlur && this.props.oncAccessibilityBlur(event);
Copy link
Contributor

@blavalla blavalla May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo oncAccessibilityBlur -> onAccessibilityBlur

},

_onAccessibilityFocus: function(event: BlurEvent) {
Copy link
Contributor

@blavalla blavalla May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shoudln't this be AccessibilityFocusEvent not BlurEvent?

this.props.oncAccessibilityFocus && this.props.oncAccessibilityFocus(event);
Copy link
Contributor

@blavalla blavalla May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo oncAccessibilityFocus -> onAccessibilityFocus

},

_onTextInput: function(event: TextInputEvent) {
this.props.onTextInput && this.props.onTextInput(event);
},
Expand Down
2 changes: 2 additions & 0 deletions Libraries/Components/Touchable/TouchableHighlight.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,8 @@ const TouchableHighlight = ((createReactClass({
clickable={
this.props.clickable !== false && this.props.onPress !== undefined
}
onAccessibilityBlur={this.props.onAccessibilityBlur}
onAccessibilityFocus={this.props.onAccessibilityFocus}
onClick={this.touchableHandlePress}
onStartShouldSetResponder={this.touchableHandleStartShouldSetResponder}
onResponderTerminationRequest={
Expand Down
11 changes: 10 additions & 1 deletion Libraries/Components/Touchable/TouchableNativeFeedback.android.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,14 @@ const TouchableNativeFeedback = createReactClass({
this._dispatchPressedStateChange(false);
},

touchableHandleAccessibilityBlur: function(e: PressEvent) {
this.props.onAccessibilityBlur && this.props.onAccessibilityBlur(e);
},

touchableHandleAccessibilityFocus: function(e: PressEvent) {
this.props.onAccessibilityFocus && this.props.onAccessibilityFocus(e);
},

touchableHandlePress: function(e: PressEvent) {
this.props.onPress && this.props.onPress(e);
},
Expand Down Expand Up @@ -329,7 +337,8 @@ const TouchableNativeFeedback = createReactClass({
this.props.clickable !== false &&
this.props.onPress !== undefined &&
!this.props.disabled,
onClick: this.touchableHandlePress,
onAccessibilityBlur: this.touchableHandleAccessibilityBlur,
onAccessibilityFocus: this.touchableHandleAccessibilityFocus,
onStartShouldSetResponder: this.touchableHandleStartShouldSetResponder,
onResponderTerminationRequest: this
.touchableHandleResponderTerminationRequest,
Expand Down
3 changes: 2 additions & 1 deletion Libraries/Components/Touchable/TouchableOpacity.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,8 @@ const TouchableOpacity = ((createReactClass({
<Animated.View
accessible={this.props.accessible !== false}
accessibilityLabel={this.props.accessibilityLabel}
onAccessibilityFocus={this.props.onAccessibilityFocus}
onAccessibilityBlur={this.props.onAccessibilityBlur}
accessibilityHint={this.props.accessibilityHint}
accessibilityRole={this.props.accessibilityRole}
accessibilityStates={this.props.accessibilityStates}
Expand All @@ -327,7 +329,6 @@ const TouchableOpacity = ((createReactClass({
clickable={
this.props.clickable !== false && this.props.onPress !== undefined
}
onClick={this.touchableHandlePress}
onStartShouldSetResponder={this.touchableHandleStartShouldSetResponder}
onResponderTerminationRequest={
this.touchableHandleResponderTerminationRequest
Expand Down
4 changes: 4 additions & 0 deletions Libraries/Components/Touchable/TouchableWithoutFeedback.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ type TargetEvent = SyntheticEvent<

type BlurEvent = TargetEvent;
type FocusEvent = TargetEvent;
type AccessibilityBlurEvent = TargetEvent;
type AccessibilityFocusEvent = TargetEvent;

const PRESS_RETENTION_OFFSET = {top: 20, left: 20, right: 20, bottom: 30};

Expand Down Expand Up @@ -67,6 +69,8 @@ export type Props = $ReadOnly<{|
disabled?: ?boolean,
hitSlop?: ?EdgeInsetsProp,
nativeID?: ?string,
onAccessibilityBlur?: ?(AccessibilityBlurEvent) => void,
onAccessibilityFocus?: ?(AccessibilityFocusEvent) => void,
onBlur?: ?(e: BlurEvent) => void,
onFocus?: ?(e: FocusEvent) => void,
onLayout?: ?(event: LayoutEvent) => mixed,
Expand Down
35 changes: 34 additions & 1 deletion Libraries/Components/View/ViewPropTypes.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@

'use strict';

import type {PressEvent, Layout, LayoutEvent} from 'CoreEventTypes';
import type {
SyntheticEvent,
PressEvent,
Layout,
LayoutEvent,
} from 'CoreEventTypes';
import type {EdgeInsetsProp} from 'EdgeInsetsPropType';
import type React from 'React';
import type {ViewStyleProp} from 'StyleSheet';
Expand All @@ -20,6 +25,15 @@ import type {AccessibilityRole, AccessibilityStates} from 'ViewAccessibility';
export type ViewLayout = Layout;
export type ViewLayoutEvent = LayoutEvent;

type TargetEvent = SyntheticEvent<
$ReadOnly<{|
target: number,
|}>,
>;

export type AccessibilityBlurEvent = TargetEvent;
export type AccessibilityFocusEvent = TargetEvent;

type DirectEventProps = $ReadOnly<{|
/**
* When `accessible` is true, the system will try to invoke this function
Expand All @@ -29,6 +43,25 @@ type DirectEventProps = $ReadOnly<{|
*/
onAccessibilityAction?: ?(string) => void,

/**
* When `accessible` is true and VoiceOver (iOS) or TalkBack (Android) is
* enabled, this event is fired immediately once the element loses the screen
* reader focus.
*
* See http://facebook.github.io/react-native/docs/view.html#onaccessibilityblur
*/

onAccessibilityBlur?: ?(AccessibilityBlurEvent) => void,

/**
* When `accessible` is true and VoiceOver (iOS) or TalkBack (Android) is
* enabled, this event is fired immediately once the element gains the screen
* reader focus.
*
* See http://facebook.github.io/react-native/docs/view.html#onaccessibilityfocus
*/
onAccessibilityFocus?: ?(AccessibilityFocusEvent) => void,

/**
* When `accessible` is true, the system will try to invoke this function
* when the user performs accessibility tap gesture.
Expand Down
2 changes: 2 additions & 0 deletions React/Views/RCTView.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ extern const UIAccessibilityTraits SwitchAccessibilityTrait;
@property (nonatomic, copy) RCTDirectEventBlock onAccessibilityTap;
@property (nonatomic, copy) RCTDirectEventBlock onMagicTap;
@property (nonatomic, copy) RCTDirectEventBlock onAccessibilityEscape;
@property (nonatomic, copy) RCTDirectEventBlock onAccessibilityFocus;
@property (nonatomic, copy) RCTDirectEventBlock onAccessibilityBlur;

/**
* Accessibility properties
Expand Down
18 changes: 18 additions & 0 deletions React/Views/RCTView.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,24 @@ - (BOOL)accessibilityPerformEscape
}
}

- (void)accessibilityElementDidLoseFocus
{
[super accessibilityElementDidLoseFocus];

if (_onAccessibilityBlur) {
_onAccessibilityBlur(nil);
}
}

- (void)accessibilityElementDidBecomeFocused
{
[super accessibilityElementDidBecomeFocused];

if (_onAccessibilityFocus) {
_onAccessibilityFocus(nil);
}
}

- (NSString *)description
{
NSString *superDescription = super.description;
Expand Down
2 changes: 2 additions & 0 deletions React/Views/RCTViewManager.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,8 @@ - (RCTShadowView *)shadowView
RCT_REMAP_VIEW_PROPERTY(onAccessibilityTap, reactAccessibilityElement.onAccessibilityTap, RCTDirectEventBlock)
RCT_REMAP_VIEW_PROPERTY(onMagicTap, reactAccessibilityElement.onMagicTap, RCTDirectEventBlock)
RCT_REMAP_VIEW_PROPERTY(onAccessibilityEscape, reactAccessibilityElement.onAccessibilityEscape, RCTDirectEventBlock)
RCT_REMAP_VIEW_PROPERTY(onAccessibilityBlur, reactAccessibilityElement.onAccessibilityBlur, RCTDirectEventBlock)
RCT_REMAP_VIEW_PROPERTY(onAccessibilityFocus, reactAccessibilityElement.onAccessibilityFocus, RCTDirectEventBlock)
RCT_REMAP_VIEW_PROPERTY(testID, reactAccessibilityElement.accessibilityIdentifier, NSString)

RCT_EXPORT_VIEW_PROPERTY(backgroundColor, UIColor)
Expand Down
62 changes: 47 additions & 15 deletions ReactAndroid/src/main/java/com/facebook/react/uimanager/AccessibilityDelegateUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,44 @@
package com.facebook.react.uimanager;

import android.content.Context;
import androidx.core.view.AccessibilityDelegateCompat;
import androidx.core.view.ViewCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat.CollectionItemInfoCompat;
import android.text.SpannableString;
import android.text.style.URLSpan;
import android.view.View;
import android.view.accessibility.AccessibilityEvent;
import androidx.core.view.AccessibilityDelegateCompat;
import androidx.core.view.ViewCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat.CollectionItemInfoCompat;
import android.view.View;
import com.facebook.react.bridge.ReadableArray;
import com.facebook.react.R;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;

import java.util.Locale;
import javax.annotation.Nullable;

import com.facebook.react.R;
import com.facebook.react.bridge.Arguments;
import com.facebook.react.bridge.ReactContext;
import com.facebook.react.bridge.ReadableArray;
import com.facebook.react.bridge.ReadableMap;
import com.facebook.react.bridge.WritableMap;
import com.facebook.react.uimanager.events.RCTEventEmitter;

/**
* Utility class that handles the addition of a "role" for accessibility to
* either a View or AccessibilityNodeInfo.
*/

public class AccessibilityDelegateUtil {

public static final String BLUR_EVENT_NAME = "onAccessibilityBlur";
public static final String FOCUS_EVENT_NAME = "onAccessibilityFocus";

public static String getBlurEventName() {
return BLUR_EVENT_NAME;
}

public static String getFocusEventName() {
return FOCUS_EVENT_NAME;
}

/**
* These roles are defined by Google's TalkBack screen reader, and this list
* should be kept up to date with their implementation. Details can be seen in
Expand Down Expand Up @@ -108,17 +123,34 @@ public static void setDelegate(final View view) {
// if a view already has an accessibility delegate, replacing it could cause
// problems,
// so leave it alone.
if (!ViewCompat.hasAccessibilityDelegate(view)
&& (accessibilityRole != null || view.getTag(R.id.accessibility_states) != null)) {
if (!ViewCompat.hasAccessibilityDelegate(view)) {
ViewCompat.setAccessibilityDelegate(view, new AccessibilityDelegateCompat() {
@Override
public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfoCompat info) {
super.onInitializeAccessibilityNodeInfo(host, info);
setRole(info, accessibilityRole, view.getContext());
// states are changable.
ReadableArray accessibilityStates = (ReadableArray) view.getTag(R.id.accessibility_states);
if (accessibilityStates != null) {
setState(info, accessibilityStates, view.getContext());

if (accessibilityRole != null || view.getTag(R.id.accessibility_states) != null) {
setRole(info, accessibilityRole, view.getContext());
// states are changable.
ReadableArray accessibilityStates = (ReadableArray) view.getTag(R.id.accessibility_states);
if (accessibilityStates != null) {
setState(info, accessibilityStates, view.getContext());
}
}
}

@Override
public void onInitializeAccessibilityEvent(View host, AccessibilityEvent accessibilityEvent) {
super.onInitializeAccessibilityEvent(host, accessibilityEvent);
ReactContext reactContext = (ReactContext)host.getContext();

WritableMap event = Arguments.createMap();
event.putInt("target", host.getId());

if (accessibilityEvent.getEventType() == AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUS_CLEARED) {
reactContext.getJSModule(RCTEventEmitter.class).receiveEvent(host.getId(), BLUR_EVENT_NAME, event);
Copy link
Contributor

@blavalla blavalla Apr 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how React Native handles the call to receiveEvent here, but it's worth pointing out that not all events that get passed through onInitializeAccessibilityEvent will actually be fired. Every ancestor of the view has the chance to cancel the event before it bubbles to the top. Only if it makes it all the way to the top of the view hierarchy will it be passed off to a running accessibility service.

The AccessibilityEvent cycle goes like this:

1.) View.sendAccessibilityEvent() (intent to send event started)
2.) View.onInitializeAccessibilityEvent() (event populated with view info)
3.) View.dispatchPopulateAccessibilityEvent() (event text populated with additional info)
4.) ViewParent.requestSendAccessibilityEvent() (parent of View is free to modify or cancel the event, or pass it up to its parent)
5.) ViewRootImpl calls sendAccessibilityEvent on the AccessibilityManager (this is what actually passes the event to any running accessibility service).

The best way to actually handle this would probably be to override performAccessibilityAction instead. When Talkback (or any other accessibility service) receives a focus event, it's not guaranteed to actually change focus. For example, if the already focused view sent another TYPE_VIEW_ACCESSIBILITY_FOCUSED event, it would be ignored. If it does decide to change focus, it will fire the performAccessibilityAction method on the View and pass in an AccessibilityNodeInfo.ACTION_ACCESSIBILITY_FOCUS action.

Here is the code in Talkback doing that for reference:
https://github.com/google/talkback/blob/e69d4731fce02bb9e69613d0e48c29033cad4a98/talkback/src/main/java/eventprocessor/ProcessorFocusAndSingleTap.java#L871

Copy link
Author

@estevaolucas estevaolucas Apr 30, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bravalla Thanks for this detailed feedback.

I applied your suggestions, which works fine for focus event, but for the blur event it doesn't work as expected. Using performAccessibilityAction, blur is triggered just when the focus moves to a non react-native element (my assumption). When the focus is moving around react-native elements, nothing happens.

In this video you can understand better what I say: blur just trigger when focus moves to status bar.

code: https://gist.github.com/elucaswork/242c098a78577ddcdef4d75a60d2a4a5

Do you have an idea of what can be happening?
Thank you.

Copy link
Contributor

@blavalla blavalla May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elucaswork, The code here looks good to me. I've verified the approach in a non-ReactNative app to make sure that Talkback was correctly calling performAccessibilityAction on both focus and blur, and it is.

It's possible that the issue is at the ReactNative side (I commented above on a few typos), but I am definitely not an expert on that layer so I'll defer to others if thats where the issue lies.

You could verify that the Android side is working correctly by adding some logging into the performAccessibilityAction method itself to see if the expected actions are ever being passed in.

} else if (accessibilityEvent.getEventType() == AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUSED) {
reactContext.getJSModule(RCTEventEmitter.class).receiveEvent(host.getId(), FOCUS_EVENT_NAME, event);
}
}
});
Expand Down
10 changes: 10 additions & 0 deletions ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@
import java.util.HashMap;

import com.facebook.react.R;
import com.facebook.react.common.MapBuilder;
import com.facebook.react.bridge.ReadableArray;
import com.facebook.react.uimanager.AccessibilityDelegateUtil.AccessibilityRole;
import com.facebook.react.uimanager.annotations.ReactProp;
import com.facebook.react.uimanager.util.ReactFindViewUtil;

import java.util.Map;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;

Expand Down Expand Up @@ -288,4 +290,12 @@ protected void onAfterUpdateTransaction(@Nonnull T view) {
super.onAfterUpdateTransaction(view);
updateViewAccessibility(view);
}

@Override
public @Nullable Map<String, Object> getExportedCustomDirectEventTypeConstants() {
return MapBuilder.<String, Object>builder()
.put(AccessibilityDelegateUtil.getBlurEventName(), MapBuilder.of("registrationName", "onAccessibilityBlur"))
.put(AccessibilityDelegateUtil.getFocusEventName(), MapBuilder.of("registrationName", "onAccessibilityFocus"))
.build();
}
}