Skip to content

Implement UiaConnectionBoundObject in abstraction API #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Implement UiaConnectionBoundObject in abstraction API #95

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7979556
microsoft.ui.uiAutomation winrt library:
michaelDCurran Jun 2, 2022
fde3a9b
Uia abstraction library:
michaelDCurran Jun 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion src/UIAutomation/Microsoft.UI.UIAutomation/Microsoft.UI.UIAutomation.idl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,10 @@ namespace Microsoft.UI.UIAutomation

unsealed runtimeclass AutomationRemoteExtensionTarget : AutomationRemoteObject
{
AutomationRemoteBool IsExtensionTarget();
/* Deprecated */ AutomationRemoteBool IsExtensionTarget();
Copy link

@rahulkhanna1605 rahulkhanna1605 Oct 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be marked as deprecated? @mavangur could you also confirm if this would be required.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know when this got deprecated Rahul, At the time I introduced this API, we are in a view that it would be useful. I don't know who marked it deprecated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Sorry for parachuting in, just hoping to provide some historic context :))

Is<Type> and As<Type> methods should be on AutomationRemoteAnyObject, since that's what facilitates "casting" between the generic "any" type and a "concrete" type. Basically it helps "inspect" the actual type and convert between them safely. (Sort of like a QI...)

This method was originally added to AutomationRemoteExtensionTarget where it doesn't do much good -- it's already representing a specific type. This PR is moving it to the right place and deprecating this one, presumably so that if anyone took a dependency on this so far for some reason they're not immediately broken.

Consider just introducing a (technically) breaking change and removing this, though.

Mick also did have a question in the PR:

Can someone please verify that the IsExtensionTarget op code in the remote ops VM really does ask if an anyObject is an extensionTarget (similar to all the other Is* op codes)?

I don't know for sure - I no longer have access to the Windows code base :( - but I would assume that it'd follow the same pattern as the others; I think someone would've had to go out of their way to implement it any differently given how things were originally structured.

void CallExtension(AutomationRemoteGuid extensionId, AutomationRemoteObject[] operands);
AutomationRemoteBool IsExtensionSupported(AutomationRemoteGuid extensionId);
void Set(AutomationRemoteExtensionTarget rhs);
}

runtimeclass AutomationRemoteArray : AutomationRemoteObject
Expand Down Expand Up @@ -268,6 +269,8 @@ namespace Microsoft.UI.UIAutomation
AutomationRemoteCacheRequest AsCacheRequest();
AutomationRemoteBool IsByteArray();
AutomationRemoteByteArray AsByteArray();
AutomationRemoteBool IsExtensionTarget();
AutomationRemoteExtensionTarget AsExtensionTarget();
}

enum AutomationActiveEnd
Expand Down
19 changes: 18 additions & 1 deletion src/UIAutomation/Microsoft.UI.UIAutomation/Standins.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -694,7 +694,7 @@ namespace winrt::Microsoft::UI::UIAutomation::implementation
winrt::AutomationRemoteElement AutomationRemoteElement::GetUpdatedCacheElement(const winrt::AutomationRemoteCacheRequest& cacheRequest)
{
auto result = m_parent->NewNull().AsElement();
result.Set(*this);
result.Set(static_cast<winrt::AutomationRemoteElement>(*this));
result.PopulateCache(cacheRequest);
return result;
}
Expand Down Expand Up @@ -1008,4 +1008,21 @@ namespace winrt::Microsoft::UI::UIAutomation::implementation
{
return As<AutomationRemoteByteArray>();
}

winrt::AutomationRemoteBool AutomationRemoteAnyObject::IsExtensionTarget()
{
const auto resultId = m_parent->GetNextId();
m_parent->InsertInstruction(bytecode::IsExtensionTarget{
resultId,
m_operandId
});

const auto result = Make<AutomationRemoteBool>(resultId);
return result;
}

winrt::AutomationRemoteExtensionTarget AutomationRemoteAnyObject::AsExtensionTarget()
{
return As<AutomationRemoteExtensionTarget>();
}
}
9 changes: 8 additions & 1 deletion src/UIAutomation/Microsoft.UI.UIAutomation/Standins.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -541,9 +541,14 @@ namespace winrt::Microsoft::UI::UIAutomation::implementation
public:
AutomationRemoteExtensionTarget(bytecode::OperandId operandId, AutomationRemoteOperation& parent);

winrt::AutomationRemoteBool IsExtensionTarget();
/* Deprecated */ winrt::AutomationRemoteBool IsExtensionTarget();
void CallExtension(const winrt::AutomationRemoteGuid& extensionId, winrt::array_view<const winrt::AutomationRemoteObject> operands);
winrt::AutomationRemoteBool IsExtensionSupported(const winrt::AutomationRemoteGuid& extensionId);

void Set(const class_type& rhs)
{
AutomationRemoteObject::Set<AutomationRemoteExtensionTarget>(rhs);
}
};

struct AutomationRemoteStringMap : AutomationRemoteStringMapT<AutomationRemoteStringMap, Microsoft::UI::UIAutomation::implementation::AutomationRemoteObject>
Expand Down Expand Up @@ -654,6 +659,8 @@ namespace winrt::Microsoft::UI::UIAutomation::implementation
winrt::AutomationRemoteCacheRequest AsCacheRequest();
winrt::AutomationRemoteBool IsByteArray();
winrt::AutomationRemoteByteArray AsByteArray();
winrt::AutomationRemoteBool IsExtensionTarget();
winrt::AutomationRemoteExtensionTarget AsExtensionTarget();

#include "AutomationRemoteAnyObjectMethods.g.h"

Expand Down
19 changes: 19 additions & 0 deletions src/UIAutomation/UiaOperationAbstraction/UiaOperationAbstraction.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1045,6 +1045,25 @@ namespace UiaOperationAbstraction
}
}

void UiaOperationDelegator::ConvertVariantDataToRemote(std::variant<winrt::com_ptr<IUnknown>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteExtensionTarget>& localExtensionTargetVariant) const
{
if (m_useRemoteApi && m_remoteOperation)
{
if (auto localExtensionTarget = std::get_if<winrt::com_ptr<IUnknown>>(&localExtensionTargetVariant))
{
if (*localExtensionTarget)
{
localExtensionTargetVariant = m_remoteOperation.ImportConnectionBoundObject(FromAbi<AutomationConnectionBoundObject>(localExtensionTarget->get()));
}
else
{
localExtensionTargetVariant = m_remoteOperation.NewNull().AsExtensionTarget();
}
}
}
}

void UiaOperationDelegator::ConvertVariantDataToRemote(std::variant<winrt::com_ptr<IUIAutomationElement>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteElement>& localElementVariant) const
{
Expand Down
3 changes: 3 additions & 0 deletions src/UIAutomation/UiaOperationAbstraction/UiaOperationAbstraction.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,9 @@ namespace UiaOperationAbstraction
void ConvertVariantDataToRemote(std::variant<
std::shared_ptr<wil::unique_variant>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteObject>& localVariantVariant) const;
void ConvertVariantDataToRemote(std::variant<
winrt::com_ptr<IUnknown>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteExtensionTarget>& localExtensionTargetVariant) const;
void ConvertVariantDataToRemote(std::variant<
winrt::com_ptr<IUIAutomationElement>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteElement>& localElementVariant) const;
Expand Down
114 changes: 94 additions & 20 deletions src/UIAutomation/UiaOperationAbstraction/UiaTypeAbstraction.g.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -806,7 +806,99 @@
}
};

class UiaTextRange: public UiaTypeBase<
template<typename LocalType, typename RemoteType> class UiaTypeBaseWithExtensionSupport: public UiaTypeBase<
typename LocalType,
typename RemoteType>
{
public:
static constexpr VARTYPE c_comVariantType = VT_UNKNOWN;
static constexpr auto c_variantMember = &VARIANT::punkVal;

using UiaTypeBase<typename LocalType, typename RemoteType>::UiaTypeBase;

UiaBool IsExtensionSupported(UiaGuid guid)
{
if (UiaOperationAbstraction::ShouldUseRemoteApi())
{
this->ToRemote();
guid.ToRemote();
auto remoteValue = std::get<typename RemoteType>(m_member);
return remoteValue.IsExtensionSupported(guid);
}

// No local equivalent
throw winrt::hresult_not_implemented();
}

template<typename... ArgTypes> void CallExtension(UiaGuid guid, ArgTypes&... args)
{
if (UiaOperationAbstraction::ShouldUseRemoteApi())
{
this->ToRemote();
guid.ToRemote();
auto remoteValue = std::get<typename RemoteType>(m_member);
(args.ToRemote(),...);
remoteValue.CallExtension(guid, {static_cast<winrt::Microsoft::UI::UIAutomation::AutomationRemoteObject>(args)...});
return;
}

// No local equivalent
throw winrt::hresult_not_implemented();
}
};

class UiaConnectionBoundObject: public UiaTypeBaseWithExtensionSupport<
winrt::com_ptr<IUnknown>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteExtensionTarget>
{
public:
static constexpr VARTYPE c_comVariantType = VT_UNKNOWN;
static constexpr auto c_variantMember = &VARIANT::punkVal;
static constexpr auto c_anyTest = &winrt::Microsoft::UI::UIAutomation::AutomationRemoteAnyObject::IsExtensionTarget;
static constexpr auto c_anyCast = &winrt::Microsoft::UI::UIAutomation::AutomationRemoteAnyObject::AsExtensionTarget;

UiaConnectionBoundObject(_In_ IUnknown* connectionBoundObject);
UiaConnectionBoundObject(winrt::com_ptr<IUnknown> const& connectionBoundObject);
UiaConnectionBoundObject(winrt::Microsoft::UI::UIAutomation::AutomationRemoteExtensionTarget const& extentionTarget);
explicit UiaConnectionBoundObject(winrt::Microsoft::UI::UIAutomation::AutomationRemoteAnyObject const& extensionTarget);
UiaConnectionBoundObject(const UiaConnectionBoundObject& other) = default;
UiaConnectionBoundObject& operator=(const UiaConnectionBoundObject& other);

operator winrt::com_ptr<IUnknown>() const;
operator wil::com_ptr<IUnknown>() const;

IUnknown* get() const;
IUnknown* operator->() const {return get();}
void reset();
IUnknown** operator&();
operator bool() const {return !!get();}
UiaBool operator!() const {return IsNull();}
operator UiaBool() const {return !IsNull();}

template <typename T>
void copy_to(T** to) const try
{
std::get<winrt::com_ptr<IUnknown>>(m_member).as(to);
}
catch(...)
{
THROW_HR(static_cast<HRESULT>(winrt::to_hresult()));
}

template <>
void copy_to(IUnknown** to) const try
{
std::get<winrt::com_ptr<IUnknown>>(m_member).copy_to(to);
}
catch(...)
{
THROW_HR(static_cast<HRESULT>(winrt::to_hresult()));
}

UiaBool IsNull() const;
};

class UiaTextRange: public UiaTypeBaseWithExtensionSupport<
winrt::com_ptr<IUIAutomationTextRange>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteTextRange>
{
Expand Down Expand Up @@ -2010,7 +2102,7 @@
}
};

class UiaElement: public UiaTypeBase<
class UiaElement: public UiaTypeBaseWithExtensionSupport<
winrt::com_ptr<IUIAutomationElement>,
winrt::Microsoft::UI::UIAutomation::AutomationRemoteElement>
{
Expand Down Expand Up @@ -2154,24 +2246,6 @@

UiaVariant GetMetadataValue(UiaPropertyId propertyId, UiaMetadata metadataId);

UiaBool IsExtensionSupported(UiaGuid guid);

template<typename... ArgTypes> void CallExtension(UiaGuid guid, ArgTypes&... args)
{
if (UiaOperationAbstraction::ShouldUseRemoteApi())
{
this->ToRemote();
guid.ToRemote();
auto remoteValue = std::get<winrt::Microsoft::UI::UIAutomation::AutomationRemoteElement>(m_member);
(args.ToRemote(),...);
remoteValue.CallExtension(guid, {static_cast<winrt::Microsoft::UI::UIAutomation::AutomationRemoteObject>(args)...});
return;
}

// No local equivalent
throw winrt::hresult_not_implemented();
}

void FromRemoteResult(const winrt::Windows::Foundation::IInspectable& result)
{
m_member = result.as<IUIAutomationElement>();
Expand Down
87 changes: 71 additions & 16 deletions src/UIAutomation/UiaOperationAbstraction/UiaTypeAbstractionImpl.g.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2008,24 +2008,24 @@
}

UiaTextRange::UiaTextRange(_In_ IUIAutomationTextRange* object):
UiaTypeBase(MakeWinrtComPtr(object))
UiaTypeBaseWithExtensionSupport(MakeWinrtComPtr(object))
{
ToRemote();
}

UiaTextRange::UiaTextRange(winrt::com_ptr<IUIAutomationTextRange> const& object):
UiaTypeBase(object)
UiaTypeBaseWithExtensionSupport(object)
{
ToRemote();
}

UiaTextRange::UiaTextRange(winrt::Microsoft::UI::UIAutomation::AutomationRemoteTextRange const& object):
UiaTypeBase(object)
UiaTypeBaseWithExtensionSupport(object)
{
}

UiaTextRange::UiaTextRange(winrt::Microsoft::UI::UIAutomation::AutomationRemoteAnyObject const& object):
UiaTypeBase(object.AsTextRange())
UiaTypeBaseWithExtensionSupport(object.AsTextRange())
{
}

Expand Down Expand Up @@ -5031,24 +5031,24 @@
}

UiaElement::UiaElement(_In_ IUIAutomationElement* element):
UiaTypeBase(MakeWinrtComPtr(element))
UiaTypeBaseWithExtensionSupport(MakeWinrtComPtr(element))
{
ToRemote();
}

UiaElement::UiaElement(winrt::com_ptr<IUIAutomationElement> const& element):
UiaTypeBase(element)
UiaTypeBaseWithExtensionSupport(element)
{
ToRemote();
}

UiaElement::UiaElement(winrt::Microsoft::UI::UIAutomation::AutomationRemoteElement const& element):
UiaTypeBase(element)
UiaTypeBaseWithExtensionSupport(element)
{
}

UiaElement::UiaElement(winrt::Microsoft::UI::UIAutomation::AutomationRemoteAnyObject const& element):
UiaTypeBase(element.AsElement())
UiaTypeBaseWithExtensionSupport(element.AsElement())
{
}

Expand Down Expand Up @@ -7224,16 +7224,71 @@
return metadataValue;
}

UiaBool UiaElement::IsExtensionSupported(UiaGuid guid)
UiaConnectionBoundObject::UiaConnectionBoundObject(_In_ IUnknown* target):
UiaTypeBaseWithExtensionSupport(MakeWinrtComPtr(target))
{
ToRemote();
}

UiaConnectionBoundObject::UiaConnectionBoundObject(winrt::com_ptr<IUnknown> const& target):
UiaTypeBaseWithExtensionSupport(target)
{
ToRemote();
}

UiaConnectionBoundObject::UiaConnectionBoundObject(winrt::Microsoft::UI::UIAutomation::AutomationRemoteExtensionTarget const& target):
UiaTypeBaseWithExtensionSupport(target)
{
}

UiaConnectionBoundObject::UiaConnectionBoundObject(winrt::Microsoft::UI::UIAutomation::AutomationRemoteAnyObject const& target):
UiaTypeBaseWithExtensionSupport(target.AsExtensionTarget())
{
}

UiaConnectionBoundObject& UiaConnectionBoundObject::operator=(const UiaConnectionBoundObject& other)
{
AssignCopyTo<UiaConnectionBoundObject>(m_member, other.m_member);
return *this;
}

UiaConnectionBoundObject::operator winrt::com_ptr<IUnknown>() const
{
return std::get<winrt::com_ptr<IUnknown>>(m_member);
}

UiaConnectionBoundObject::operator wil::com_ptr<IUnknown>() const
{
return wil::com_ptr<IUnknown>(std::get<winrt::com_ptr<IUnknown>>(m_member).get());
}

IUnknown* UiaConnectionBoundObject::get() const
{
return std::get<winrt::com_ptr<IUnknown>>(m_member).get();
}

void UiaConnectionBoundObject::reset()
{
std::get<winrt::com_ptr<IUnknown>>(m_member) = nullptr;
}

IUnknown** UiaConnectionBoundObject::operator&()
{
reset();
return std::get<winrt::com_ptr<IUnknown>>(m_member).put();
}

UiaBool UiaConnectionBoundObject::IsNull() const
{
if (UiaOperationAbstraction::ShouldUseRemoteApi())
{
this->ToRemote();
guid.ToRemote();
auto remoteValue = std::get<winrt::Microsoft::UI::UIAutomation::AutomationRemoteElement>(m_member);
return remoteValue.IsExtensionSupported(guid);
auto remoteValue = std::get_if<AutomationRemoteExtensionTarget>(&m_member);
if (remoteValue)
{
return remoteValue->IsNull();
}
}

// No local equivalent
throw winrt::hresult_not_implemented();
}
return !get();
}