3

TL;DR: Is there any way to have custom button style (custom pressed state) in SwiftUI on tvOS while it is still working correctly with Accessibility Focus API and therefore its hasFocus works in UI tests?

Please visit dedicated Github repo for a longer version and a sample project with UI tests example.

What is the problem?

To create a custom button in SwiftUI on tvOS and can customize it based on pressed state, you can implement custom ButtonStyle.

Button(...)
    .buttonStyle(AnyCustomButtonStyle())

and then use ButtonConfiguration.isPressed in the view.

However, I have found that although the button visually looks focused, it does not really report as Focused in Accessibility API. See the sample project for example and actual test showing the problem.

Among other problems, it makes the button quite difficult to work with in tvOS UI tests. tvOS is relying on focus for navigation and because the button's hasFocus always stays false (even if the button renders in focused appearance), it can prevent a lot of useful APIs to work in tests.

Why do you need custom ButtonStyle on tvOS?

I know Apple provides some custom PrimitiveButtonStyle implementations (like CardButtonStyle), but those don't provide enough flexibility. They all modify your button (e.g. add background).

Not being able to use custom buttonStyle makes it impossible to implement for example Capsule-style buttons like this...

First button focused First button focused and pressed
First button focused First button focused and pressed

Please let's leave aside the discussion if it is a good idea or not ... Just trying to find if there is a solution or if it is eventually a bug of SwiftUI.

What is the issue with Accessibility?

Without custom button, the button reports as Focused to Accessibility in UI tests.

Without custom style

(lldb) po app
Attributes: Application, 0x12ed10b30, pid: 61273, label: 'FocusSwiftUI'
Element subtree:
 →Application, 0x12ed10b30, pid: 61273, label: 'FocusSwiftUI'
...
              Button, 0x12ed0c660, {{468.0, 477.0}, {298.0, 126.0}}, label: 'Button 1'
                Button, 0x12ed0c770, {{453.0, 470.7}, {328.0, 138.7}}, Focused
              Button, 0x12ed0c1e0, {{806.0, 477.0}, {303.0, 126.0}}, label: 'Button 2'
                Button, 0x12ed0c2f0, {{791.0, 470.8}, {333.0, 138.5}}
...

However, the moment you set buttonStyle to any custom one, the output changes drastically...

(lldb) po app
Attributes: Application, 0x106d0cd40, pid: 70156, label: 'FocusSwiftUI'
Element subtree:
 →Application, 0x106d0cd40, pid: 70156, label: 'FocusSwiftUI'
 ...
              Button, 0x106d0ce50, {{707.0, 505.0}, {142.0, 71.0}}, label: 'Button 1'
              Button, 0x106d0cf60, {{885.0, 505.0}, {146.0, 71.0}}, label: 'Button 2'
              Button, 0x106d0d180, {{1067.0, 505.0}, {147.0, 71.0}}, label: 'Button 3'
...

Notice there is no more Focused button anywhere... We will never get any button with hasFocus == true in queries for example...

What else did I try?

I tried to experiment with many (probably all) .accessibility... modifiers before asking. Many different results, but none of them ever had proper focus behavior in UI tests...

  • .accessibilityChildren: close to default button behavior in terms of accessibility structure (e.g. nested buttons)
  • .accessibilityRepresentation: button never gets visually highlighted

Temporary workaround

For now there seems to be no solution in sight. I have implemented rather complicated hack to get at least UI tests working for now.

In a nutshell: When the app is running in a context of UI tests (determined through Launch Arg), each affected button adds clear color background (it has no visual or behavioral impact) when focused. The background color then has a constant accessibilityIdentifier (e.g. "MY_FOCUSED", I call it "custom focus marker"). When evaluating if element is focused in UI tests, I then check if the button contains child element where accessibilityIdentifier == "MY_FOCUSED".

It is nasty, but somehow good enough for UI tests and actually works reliably so far. It works thanks to the fact there is always only one focused item at the same time and the "if focused -> set background" takes care of the automatic update of the "custom focus marker".

Lukas Kukacka
  • 7,604
  • 2
  • 25
  • 50
  • Does this answer your question https://stackoverflow.com/a/63172333/12299030? – Asperi Jan 11 '22 at 14:45
  • Thank you @Asperi, but unfortunately not. I found your answer before posting my question. Your answer is valid for _visual_ styling. But it is basically the same as my sample code and has the same issue: missing Accessibility Focus. I tested your code in XCUI tests. – Lukas Kukacka Jan 11 '22 at 16:19
  • @LukasKukacka any updates on this issue? I'm encountering the same behavior. – Jason Barker Apr 12 '22 at 20:41
  • @JasonBarker Unfortunately no solution in sight. I have updated my question with short description of my current temporary hack. – Lukas Kukacka Apr 21 '22 at 11:20
  • @Asperi , Lukas: Can you guys help me on this: https://stackoverflow.com/questions/72156195/swiftui-tvos-navigationlink-with-button-style-custom-disables-auto-scroll-in-li – Abhishek Thapliyal May 30 '22 at 12:42

2 Answers2

0

You can try to set accessibilityRepresentstion - does it help?

Button(..)
    .buttonStyle(AnyCustomButtonStyle())
    .accessibilityRepresentation {
        Button(..)
    }

Edit: Another idea - what about having the “correct” Button in the background of your custom? Something like:

Button(..)
    .buttonStyle(AnyCustomButtonStyle())
    .background(
        Button(..).opacity(0.0001) //it may work even with opacity 0
    )

You may additionally improve it with making the visible button hidden to accessibility with .accessibility(hidden: true)

cluelessCoder
  • 948
  • 6
  • 19
0

If you're looking for actual VoiceOver accessibility (not using it as a UI test tool I mean) you can use the isFocused environment variable and on change, call UIAccessibility.post(notification: .announcement, argument: "Your message")

Kris9000
  • 31
  • 4