Android: How to swipe between fragments (left - right)?

Question

I don't want to use a viewpager, because I must to add, delete, reorder fragments easily (to complicated with ViewPager to do that).

So in each fragment I use a gesture detector:

final GestureDetector mGesture = new GestureDetector(getActivity(),
        new GestureDetector.SimpleOnGestureListener() {

            @Override
            public boolean onDown(MotionEvent e) {
                return true;
            }

            @Override
            public boolean onFling(MotionEvent e1, MotionEvent e2, float velocityX,
                                   float velocityY) {
                LogUtils.LOGD(TAG, "onFling has been called!");
                final int SWIPE_MIN_DISTANCE = 120;
                final int SWIPE_MAX_OFF_PATH = 250;
                final int SWIPE_THRESHOLD_VELOCITY = 200;
                try {
                    if (Math.abs(e1.getY() - e2.getY()) > SWIPE_MAX_OFF_PATH)
                        return false;
                    if (e1.getX() - e2.getX() > SWIPE_MIN_DISTANCE
                            && Math.abs(velocityX) > SWIPE_THRESHOLD_VELOCITY) {
                        EventBus.getDefault().post(new SwipePartTrainingEvent(true));
                        LogUtils.LOGD(TAG, "Right to Left");
                    } else if (e2.getX() - e1.getX() > SWIPE_MIN_DISTANCE
                            && Math.abs(velocityX) > SWIPE_THRESHOLD_VELOCITY) {
                        EventBus.getDefault().post(new SwipePartTrainingEvent(false));
                        LogUtils.LOGD(TAG, "Left to Right");
                    }
                } catch (Exception e) {
                    // nothing
                }
                return super.onFling(e1, e2, velocityX, velocityY);
            }
        });

In each fragment, I have a recyclerview with different items. The gesture works fine if I swipe outside of recyclerview (wrap_content on height). But in the recyclerview (on an item) the swipe doesn't work.

So how can I do to swipe everywhere on the fragment?

Thank you very much