onTouch() method doesn't receive ACTION_DOWN

Question

I have an app, where I need to display multiple RecyclerViews inside ConstraintLayout inside NestedScrollView inside SwipeRefreshLayout.

All RecyclerViews have LinearLayoutManager with horizontal orientation.

So, my current layout tree looks like this:

<SwipeRefreshLayout>
    <NestedScrollView>
        <ConstraintLayout>
            <RecyclerView></RecyclerView>
            <RecyclerView></RecyclerView>
            <RecyclerView></RecyclerView>
        ...
        </ConstraintLayout>
    </NestedScrollView>
</SwipeRefreshLayout>

What I'm trying to achieve is something similar to what Google Play app has on Home screen.

My problem is such, that RecyclerViews are way too picky about what can be considered a horizontal swipe.

I tried to setNestedScrollingEnabled(false) but to no avail.

Then I started to look into onTouch() methods of everything above RecyclerViews and I found, that ACTION_DOWN of MotionEvent does appear only if I scroll very slowly - anything, that can be considered a swipe/fling doesn't produce ACTION_DOWN, only ACTION_MOVE and ACTION_UP.

Is there something I don't know about this construction? Does some view consume ACTION_DOWN?

And side question - are there examples of recreating not only appearance of Google Play Home screen, but also a feel? Their horizontal lists are way more forgiving, when it comes to horizontal scroll.

EDIT: Added OnItemTouchListener to all my RecyclerViews. It intercepts ALL swipes/flings/scrolls correctly, but not all trigger OnSwipeListener. Here's code. Side note: OnSwipeListener is taken from this answer with slight addition of onScroll method

public class MyOnItemTouchListener extends RecyclerView.SimpleOnItemTouchListener {

    private GestureDetectorCompat mDetector;
    private OnSwipeListener mListener = new OnSwipeListener() {

        @Override public boolean onSwipe(Direction direction) {
            Log.d("Tag", "OnItemListener direction=" + direction);
            return super.onSwipe(direction);
        }
    };

    public HomeOnItemTouchListener(Context context) {
        mDetector = new GestureDetectorCompat(context, mListener);
    }

    @Override public boolean onInterceptTouchEvent(RecyclerView rv, MotionEvent e) {
        Log.d("Tag", "OnItemListener intercept MotionEvent=" + e);
        mDetector.onTouchEvent(e);
        return super.onInterceptTouchEvent(rv, e);
    }

    @Override public void onTouchEvent(RecyclerView rv, MotionEvent e) {
        Log.d("Tag", "OnItemListener onTouch MotionEvent=" + e);
        mDetector.onTouchEvent(e);
        super.onTouchEvent(rv, e);
    }
}