Is there a way to noindex follow a section of a page to remove duplicate text?

Question

I have a private website with many articles. Located at the bottom of the page is the author of the page, which is built into Wordpress through custom PHP.

When running a site test, it shows I have 600+ duplicate elements because of all the author paragraphs generating on the page. Is there a way to noindex follow that specific section without upsetting Google?

The code for that portion is located in its own PHP file (see attached). I know I can add <meta name="robots" content="noindex, nofollow"> in the header but I'm worried that it will hide the ENTIRE page and not just this section.

Let me know if I need to clarify anything!

@unless (App\is_tree('about-us'))
  @set($author, "user_{$GLOBALS['post']->post_author}")
  @set($reviewer, get_field('medical_reviewer'))

  <footer class="ContentFooter">
    <div class="ContentFooter__section">
      <a class="UserLink" data-toggle="Author" role="button">
        @if (get_field('author_photo', $author))
          <img
            class="UserLink__img"
            src="{{ get_field('author_photo', $author)['sizes']['small'] }}"
            alt="{{ __('Photo of', 'sage') }} {{ get_the_author() }}"
          >
        @endif

        <span class="UserLink__info">
          <span class="UserLink__label">{{ __('Author', 'sage') }}</span>
          <span class="UserLink__name">{{ get_the_author() }}</span>

          @if (get_field('author_title', $author))
            <span class="UserLink__title">
              {{ get_field('author_title', $author) }}
            </span>
          @endif
        </span>

        <span class="Icon Icon--UserLink Icon--plus"></span>
      </a>

      <div
        class="ContentFooter__author-about ContentFooter__section__content"
        id="Author"
        data-toggler=".js-expanded"
      >
        {!! get_the_author_meta('description') !!}
      </div>
    </div> 

    @if ($reviewer)
      <div class="ContentFooter__section">
        <a class="UserLink" data-toggle="Reviewer" role="button">
          @if (has_post_thumbnail($reviewer))
            <img
              class="UserLink__img"
              src="{{ get_the_post_thumbnail_url($reviewer, 'thumbnail') }}"
              alt="{{ __('Photo of', 'sage') }} {{ get_the_title($reviewer) }}"
            >
          @endif

          <span class="UserLink__info">
            <span class="UserLink__label">{{ __('Reviewer', 'sage') }}</span>
            <span class="UserLink__name">{{ get_the_title($reviewer) }}</span>
            <span class="UserLink__title">
              {{ __('Last Reviewed:') }} {{ $medical_review_date }}
            </span>
          </span>

          <span class="Icon Icon--UserLink Icon--plus"></span>
        </a>

        <div
          class="ContentFooter__author-about ContentFooter__section__content"
          id="Reviewer"
          data-toggler=".js-expanded"
        >
          {!! get_the_content(null, false, $reviewer) !!}
        </div>
      </div>
    @endif

    @if (get_field('citations'))
      <div class="ContentFooter__section">
        <p class="ContentFooter__section__heading">
          <a
            data-toggle="Citations"
            role="button"
          >
            {{ __('Sources', 'sage') }}
          </a>
        </p>

        <div
          class="ContentFooter__section__content ContentFooter__citations"
          id="Citations"
          data-toggler=".js-expanded"
        >
          {!! get_field('citations') !!}
        </div>
      </div>
    @endif

    @unless (is_singular('post'))
      <div class="ContentFooter__nav">
        @if ($GLOBALS['post']->post_parent)
          {!! previous_post_link('%link', '
            <span class="Icon Icon--angle-left"></span>
            <span class="ContentFooter__nav__text-wrap">
              <span class="ContentFooter__nav__label ContentFooter__nav__label--prev">
                Previous Page
              </span>
              <span class="ContentFooter__nav__text">%title</span>
            </span>
          ') !!}
        @endif

        {!! next_post_link('%link', '
          <span class="ContentFooter__nav__text-wrap">
            <span class="ContentFooter__nav__label ContentFooter__nav__label--next">
              Next Page
            </span>
            <span class="ContentFooter__nav__text">%title</span>
          </span>
          <span class="Icon Icon--angle-right"></span>
        ') !!}
      </div>
    @endunless
  </footer>
@endunless

I would read through this: https://stackoverflow.com/q/3207211/231316 — Chris Haas, Jul 20 '21 at 20:27
Can you clarify what those 600 duplicate elements are? And how does an article have 600 authors? if that was the case, then technically your page is at the wrong here. — choz, Jul 21 '21 at 04:33
@choz In the "Users" section of Wordpress they have a written paragraph of their bio. The PHP code above takes that bio paragraph and applies it to the bottom of each page they publish. It's about 12 authors in total, but all of the authors combined have written over 600 pages/articles (so it is show duplicate text for all of them. Ex. One author has 40 duplicate text paragraphs because they wrote 40 articles). — Erica, Jul 21 '21 at 15:50

score 0 · Answer 1 · answered Jul 21 '21 at 16:08

TL;DR; There is no straightforward way to tell crawlers to not look into specific section in a page. Your best chance is to not show it to them.

What crawlers see is what they index. Unless robots meta is set to noindex, modern robots will respect that and stop indexing the whole page as you already know.

As per Chris' comment that's referring to this thread by using google{on|off} - This will not affect your Google Web Search at all. Thanks to John Mueller to share this information here.

The only way I see it, and the good thing is, your page is not SPA but rather the content is being handled in the server before responding to the client.

Therefore, I'd suggest that it's better to;

Track if it's a bot whose accessing your page - There's an answer here that explains how you can do it in PHP.

These are the list of user agents that I consider as a bot in my app;

/((Google|Apple|bing|linkedin|duckduck|Yandex)bot|Yahoo|Baiduspider|Teoma|Slurp|google-structured-data-testing-tool)/i;

Implement a logic to hide the portion of those duplicated contents to not display when the bots accessing it.

This is a practice that I use to include or exclude keywords from Google search result. Most common scenario this happened to me by far was wiring this to Show more.. or Show less.. feature on a description. Which is where we want bots to pick up all the contents in it, but only gives humans some of it, which I think it's the same case as your case above.

Is there a way to noindex follow a section of a page to remove duplicate text?

1 Answers1