Simple answer: Don't keep a view alive between activities. You can save and pass states/variables, but it is not smart to keep it in memory. So either :
Switch to a fragment structure and create a fragment containing the SurfaceView attached to a View and move it to the background when not needed
Or keep an instance (or the states) of the surfaceview and detach it from the attached View and reattach it when needed
I think these are the only legit ways to go for. I once did this just using a Singleton for the surfaceview and attaching/detaching it from a view when needed, but this is not the cleanest way.
Comment Pseudo code transaction
You should have the following: An Activity, FragmentA, FragmentB and VideoFragment. The Activity layout only consists about two viewholders, one that is full screen (for FragmentA and B) and the other one that is for the video view. To create fragments just right click New->Fragment->Blank. First you'll have to init the fragment in the view to do this and eventually replace FragmentA with FragmentB you could use the following code:
Fragment fragmentB = new FragmentB();
FragmentManager fm = getSupportFragmentManager();
FragmentTransaction transaction = fm.beginTransaction();
transaction.replace(R.id.viewHolder, fragmentB);
transaction.commit();
Your activity layout should look something like this:
<?xml version="1.0" encoding="utf-8"?>
<android.support.design.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:fitsSystemWindows="true"
tools:context="nl.coffeeit.clearvoxnexxt.activities.TestActivity">
<RelativeLayout
android:layout_width="match_parent"
android:layout_height="match_parent" >
<FrameLayout
android:id="@+id/viewHolderFragment"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="#F00"/>
<FrameLayout
android:id="@+id/videoHolder"
android:layout_width="300dp"
android:layout_height="200dp"
android:layout_alignParentBottom="true"
android:layout_alignParentRight="true"
android:layout_margin="16dp"
android:alpha="0.5"
android:background="#00F"/>
</RelativeLayout>
</android.support.design.widget.CoordinatorLayout>