I don't want my app to open when user presses my play/pause button in the app's notification

Question

I've made a notification for my media app (xamarin/android) that allows the user to play and pause the audio straight from the notification so the user doesn't have to get back into the app if they want to pause or play quickly.

However, each time the user hits my notification's button it also brings them back to the app. How can get it so that the user can be in a different application, go to the notification, hit the play/pause button but then return to whatever they were doing? Not bring them back to my app.

I thought it might have to do something with setting the launchmode to my activity but I have been messing around with it and haven't gotten anything to change yet.

This is my activity

[Activity]
    public class SecondActivity : Activity
    {
        DabPlayer player = GlobalResources.playerPodcast;
        EpisodeViewModel Episode;
        DroidDabNativePlayer droid = new DroidDabNativePlayer();
        protected override void OnCreate(Bundle bundle)
        {
            base.OnCreate(bundle);

            if (player.IsReady)
            {
                if (player.IsPlaying)
                {
                    player.Pause();
                }
                else
                {
                    player.Play();
                }
            }
            else
            {
                if (player.Load(Episode.Episode))
                {
                    player.Play();
                }
                else
                {
                    //DisplayAlert("Episode Unavailable", "The episode you are attempting to play is currently unavailable. Please try again later.", "OK");
                }

            }

            Finish();
        }

This is my android manifest

<application android:label="DAB">
    <activity android:name="SecondActivity" launchMode="singleTop" />

I don't think you would really need to see the notification build but here it is in case it's helpful.

var builder = new NotificationCompat.Builder(Application.Context, CHANNEL_ID)
.SetStyle(new Android.Support.V4.Media.App.NotificationCompat.MediaStyle()                                            
.SetShowActionsInCompactView(0))                               
.SetContentIntent(firstPendingIntent) // Start up this activity when the user clicks the intent.                                  
.SetVisibility(NotificationCompat.VisibilityPublic)
.AddAction(Resource.Drawable.ic_media_play_dark, "Play", pendingIntent)
.SetMediaSession(mSession.SessionToken)
.SetContentText(GlobalResources.playerPodcast.EpisodeTitle)
.SetContentTitle(GlobalResources.playerPodcast.ChannelTitle);
.SetDeleteIntent(MediaButtonReceiver.BuildMediaButtonPendingIntent(Application.Context, PlaybackState.ActionStop))
.SetSmallIcon(Resource.Drawable.app_icon) // This is the icon to display                                                                                                     

Any ideas how to call this play/pause function without having to bring the user back to the app?