Infinite scroll pagination and filter with hotwire

Question

I have a table with infinite scroll working perfectly without reloading entire page. I'm now having issues with adding filter. Thanks to Phil Reynolds' article https://purpleriver.dev/posts/2022/hotwire-handbook-part-2 I was able to implement infinite load.

controller action

def index
  if params[:query].present?
    search = "%#{params[:query]}%"
    alerts = Alert.where( "title ILIKE ?", search )
  else
    alerts = Alert.all
  end

  @pagy, @alerts = pagy(alerts, items: 100)
end

the table

<%= turbo_frame_tag "page_handler" %>
<table class="w-full border border-t-gray-300 table-auto">
  <thead>
    <tr class="bg-gray-200 text-gray-600 uppercase text-sm leading-3">
      <th class="py-3 px-4 text-left">Severity</th>
      <th class="py-3 px-3 text-left">Title</th>
      ...
    </tr>
  </thead>
    <tbody id="alerts" class="text-gray-600 text-sm font-light">
      <%= render "alerts_table", alerts: @alerts %>
    </tbody>
</table>
<%= render "shared/index_pager", pagy: @pagy %>

alerts_pager partial

<div id="<%= controller_name %>_pager" class="min-w-full my-8 flex justify-center">
  <% if pagy.next %>
    <%= link_to 'Loading',
                "#{controller_name}?query=#{params[:query]}&page=#{pagy.next}",
                data: {
                  turbo_frame: 'page_handler',
                  controller: 'autoclick'
                },
                class: 'rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800'%>
  <% end %>
</div>

turbo frame response

<%= turbo_frame_tag "page_handler" do %>
    <%= turbo_stream_action_tag(
        "append",
        target: "alerts",
        template: %(#{render "alerts_table", alerts: @alerts}) 
    ) %>
    <%= turbo_stream_action_tag(
        "replace",
        target: "alerts_pager",
        template: %(#{render "shared/index_pager", pagy: @pagy})
        ) %>
<% end %>

autoclick controller

import { Controller } from "@hotwired/stimulus"
import { useIntersection } from 'stimulus-use'

export default class extends Controller {
  options = {
    threshold: 0.5
  }

  connect() {
    useIntersection(this, this.options)
  }

  appear(entry) {
    this.element.click()
  }
}

I also managed to make it working together with filter but it reloads full page.

<div id="<%= controller_name %>_filter" class="bg-gray-200 p-1 shadow-lg">
    <div class="p-1 lg:w-1/3">
        <%= form_with url: alerts_path,  method: :get do %>
            <%= text_field_tag "query", 
                                                    nil, 
                                                    placeholder: "Filter", 
                                                    class: "inline-block rounded-md border border-gray-200 outline-none px-3 py-2 w-full" %>
        <% end %>
    </div>
</div>

I want to update content in the same turbo frame. But the problem is that turbo_stream_action_tag in the page_handler frame appends data. Do need to have another turbo_frame_tag that serves filter? How to implement it?

I tried to add <%= turbo_frame_tag "filter_handler" %> to the index page and added sections below to turbo frame response

<%= turbo_frame_tag "filter_handler" do %>
    <%= turbo_stream_action_tag(
        "replace",
        target: "alerts",
        template: %(#{render "alerts_table", alerts: @alerts}) 
    ) %>
<% end %>

and added data: {turbo_frame: "filter_handler"} attr to the filter. But it works incorrectly

Answer 1

You can add turbo_stream response for your form and do an update or replace action instead of append. Just so I can test it, I made a simpler version of the infinite scroll but it should work the same:

# app/controllers/posts_controller.rb

class PostsController < ApplicationController
  include Pagy::Backend

  # GET /posts
  def index
    scope = Post.order(id: :desc)
    scope = scope.where(Post.arel_table[:title].matches("%#{params[:query]}%")) if params[:query]
    @pagy, @posts = pagy(scope)

    respond_to do |format|
      # this will be the response to the search form request
      format.turbo_stream do
        render turbo_stream: turbo_stream.replace(:infinite_scroll, partial: "infinite_scroll")
      end
      # this is regular navigation response
      format.html
    end
  end
end

# app/views/posts/index.html.erb

# NOTE: set up a GET form and make it submit as turbo_stream
#                                     vvv          vvvvvvvvvvvvvvvvvv
<%= form_with url: "/posts", method: :get, data: { turbo_stream: true } do |f| %>
  <%= f.search_field :query %>
<% end %>
<%= render "infinite_scroll" %>

# app/views/posts/_infinite_scroll.html.erb

<div id="infinite_scroll">
  <%= turbo_frame_tag "page_#{params[:page] || 1}", target: :_top do %>
    <hr><%= tag.h3 "Page # #{params[:page] || 1}", class: "text-2xl font-bold" %>

    <% @posts.each do |post| %>
      <%= tag.div post.title %>
    <% end %>

    <% if @pagy.next %>
      # NOTE: technically there is no need for `turbo_stream.append` here
      #       but without it turbo frames will be nested inside each other
      #       which works just fine.
      #       also, i'm not sure why `turbo_stream_action_tag` is used.
      <%= turbo_stream.append :infinite_scroll do %>
        <%= turbo_frame_tag "page_#{@pagy.next}", target: :_top, loading: :lazy, src: "#{controller_name}?query=#{params[:query]}&page=#{@pagy.next}" %>
        # NOTE:                       this bit is also important ^^^^^^^^^^^^^^
      <% end %>
    <% end %>
  <% end %>
</div>

---

You can also just wrap the whole thing in another frame:

<%= form_with url: "/posts", method: :get, data: { turbo_frame: :infinite_frame, turbo_action: :advance } do |f| %>
  <%= f.search_field :query %>
<% end %>
<%= turbo_frame_tag :infinite_frame do %>
  <%= render "infinite_scroll" %>
<% end %>

In this case, there is no need for format.turbo_stream response in index action.

---

In case anyone is wondering how it works, it's easier to see than explain, so this is what it renders initially:

<div id="infinite_scroll">
  <turbo-frame id="page_1" target="_top">
    <hr><h3 class="text-2xl font-bold">Page # 1</h3>
    <!-- page 1 posts -->
  </turbo-frame>

  <!-- NOTE: this frame is not loaded yet -->
  <turbo-frame loading="lazy" id="page_2" src="posts?query=&amp;page=2" target="_top"></turbo-frame>
</div>

Once you scroll down to page_2 frame, it sends next page request, which will have page_2 frame and not yet loaded page_3 frame:

<div id="infinite_scroll">
  <turbo-frame id="page_1" target="_top">
    <hr><h3 class="text-2xl font-bold">Page # 1</h3>
    <!-- page 1 posts -->
  </turbo-frame>

  <!-- NOTE: page 2 frame is loaded and updated -->
  <turbo-frame loading="lazy" id="page_2" src="http://localhost:3000/posts?query=&amp;page=2" target="_top" complete="">
    <hr><h3 class="text-2xl font-bold">Page # 2</h3>
    <!-- page 2 posts -->
  </turbo-frame>

  <!-- NOTE: and just keep scrolling -->
  <turbo-frame loading="lazy" id="page_3" src="posts?query=&amp;page=3" target="_top"></turbo-frame>
</div>

---

Infinite scroll with table

It doesn't work with table because you can't have <turbo-frame> tag inside <tbody> tag. Just gonna have to do scrolling outside of the table and append the rows, which is what you were doing before. But here is a working example, everything fits into a single template, no partials:

<!-- app/views/posts/index.html.erb -->

<!-- when searching, just replace the whole inifinite scroll part -->
<%= form_with url: "/posts", method: :get, data: { turbo_frame: :infinite_frame, turbo_action: :advance } do |f| %>
  <%= f.search_field :query, value: params[:query] %>
<% end %>

<!-- you can put this into a partial instead -->
<% rows = capture do %>
  <tr colspan="2">
    <th class="px-3 py-3 text-left">Page <%= params[:page]||1 %></th>
  </tr>
  <% @posts.each do |post| %>
    <tr class="">
      <th class="px-3 py-3 text-left"><%= post.id %></th>
      <th class="px-3 py-3 text-left"><%= post.title %></th>
    </tr>
  <% end %>
<% end %>

<!--
  to avoid appending the first page and just render it, we need to 
  differentiate the first request from subsequent page_2, page_3
  turbo frame requests
-->
<% infinite_scroll_request = request.headers["Turbo-Frame"] =~ /page_/ %>

<!--
  the search will also work without this frame
  but this way it won't update the whole page
-->
<%= turbo_frame_tag :infinite_frame, target: :_top do %>
  <!--
    render the first page on initial request, we don't need the whole 
    table again on subsequent requests
  -->
  <% unless infinite_scroll_request  %>
    <table class="w-full border table-auto border-t-gray-300">
      <thead>
        <tr class="text-sm text-gray-600 uppercase bg-gray-200 leading-3">
          <th class="px-3 py-3 text-left">ID</th>
          <th class="px-3 py-3 text-left">Title</th>
        </tr>
      </thead>
      <tbody id="infinite_rows" class="text-sm font-light text-gray-600">
        <%= rows %>
      </tbody>
    </table>
  <% end %>

  <div id="infinite_scroll">
    <%= turbo_frame_tag "page_#{params[:page] || 1}", target: :_top do %>
      <!-- render the next page and append it to tbody -->
      <% if infinite_scroll_request %>
        <%= turbo_stream.append :infinite_rows do %>
          <%= rows %>
        <% end %>
      <% end %>
      <% if @pagy.next %>
        <%= turbo_stream.append :infinite_scroll do %>
          <%= turbo_frame_tag "page_#{@pagy.next}", target: :_top, loading: :lazy, src: "#{controller_name}?query=#{params[:query]}&page=#{@pagy.next}" do %>
            <b>loading...</b>
          <% end %>
        <% end %>
      <% end %>
    <% end %>
  </div>
<% end %>

Answer 2

I end up with the approach below. It works like a charm. But Alex's solution also works and may explain things better, so it's accepted

def index
    search_params = params.permit(:format, :page, q: [:title_cont])
    @q = Alert.ransack(search_params[:q])
    alerts = @q.result(distinct: true).order(created_at: :asc)
    @pagy, @alerts = pagy_countless(alerts, items: 50)
end

<!-- app/views/alerts/index.html.erb -->

<div class="bg-gray-200 p-1 shadow-lg">
    <div class="p-1 lg:w-1/3">
    <%= search_form_for @q, data: { turbo_frame: :results } do |f| %>
        <%= f.search_field :title_or_asset_cont,
                            placeholder: "Filter",
                            oninput: 'this.form.requestSubmit()',
                            autofocus: true,
                            autocomplete: 'off',
                            class: "inline-block rounded-md w-full" %>
    <% end %>
    </div>
</div>

<%= turbo_frame_tag :results, data: { turbo_action: 'advance' } do %>
<table class="w-full table-auto">
    <thead>
...
    </thead>
        <tbody id="alerts">
            
        </tbody>
</table>
    <%= turbo_frame_tag :pagination, loading: :lazy,
            src: alerts_path(format: :turbo_stream, q: params[:q]&.to_unsafe_h) %>
<% end %>

<!-- app/views/alerts/index.turbo_stream.erb -->

<%= turbo_stream.append :alerts do %>
  <%= render "alerts_table", alerts: @alerts %>
<% end %>

<% if @pagy.next.present? %>
  <%= turbo_stream.replace :pagination do %>
  
    <%= turbo_frame_tag :pagination, 
                        loading: :lazy,
                        src: alerts_path(format: :turbo_stream, page: @pagy.next, q: params[:q]&.to_unsafe_h) %>
  <% end %>
<% end %>