Daniel Colson

March 3, 2021

Adventures in Infinite Scrolling

"Searching bored to death" by Eze Matteo.gif


HEY uses infinite-scroll style pagination in a bunch of places. We've got a Stimulus controller that does the heavy lifting (check out the JavaScript source maps at app.hey.com if you'd like to see the full controller). To use it, we add two data attributes to our HTML—one for the controller and a "nextPageLink" target.

<ul data-controller="pagination">
  <li>Result 1</li>
  <li>Result 2</li>
  <li>Result 3</li>
  <a data-pagination-target="nextPageLink" href="/results?page=2" tabindex="-1">Loading more results...</a>
</ul>

The Stimulus controller uses an Intersection Observer to identify when the link enters the browser viewport, at which point it fetches the next set of results, and updates the page. Something like:

new IntersectionObserver(([entry], observer) => {
  if (entry.isIntersecting) {
    // fetch the next results, then update the page
    observer.disconnect()
  }
}).observe(this.nextPageLinkTarget))

This was working quite well for us, until we stuck it in a fixed-position container.

Keyboard Navigation

On mobile HEY, searching navigates to a dedicated search page, /search/new. But on larger screens, the results open up in a modal container on the original page. (We use a Turbo Frame here, which allows us to use the same url, controller, and view for both cases, but more on that later.)

This all works fine when scrolling with a mouse—we get to the bottom of the results, the link comes into view, and the next page loads. But we've been working hard to improve accessibility for HEY, and part of that is ensuring that everything works with keyboard navigation as well. Unfortunately, keyboard navigation was not working as expected in Safari when we first released.

HEY allows you to navigate up and down the search results using arrow keys. The link to the next page is intentionally not reachable by keyboard navigation, since the next page is meant to load automatically. When you reach the last result with the keyboard, the link should intersect with the browser viewport and kick off the next page load.

And that's exactly what was happening in most cases, but not when navigating the modal in Safari. Even though the link was within the document viewport, it was still outside the modal, and thus not yet visible. That's not good enough for Safari. Safari will only detect the intersection if the element is visible.

The link to the next page is inside the browser viewport, but not yet visible within the search modal.png


Intersection Options

To get this working, we needed to pass a custom root element to use as the intersection viewport, along with a margin to allow the intersection to happen a bit before the link comes into view. So we updated our HTML with some additional Stimulus attributes:

<ul data-controller="pagination" data-pagination-root-margin-value="40px"
data-pagination-target="root">

And then passed those options to the Intersection Observer:

const intersectionOptions = { root: this.rootTarget, margin: this.rootMarginValue }

new IntersectionObserver(([entry], observer) => {
  if (entry.isIntersecting) {
    // fetch the next results, then update the page
    observer.disconnect()
  }
}, intersectionOptions).observe(this.nextPageLinkTarget))

This seemed to work. Or did it? We had this in production for a while, but it turns out it worked because entry.isIntersecting returned true all the time, causing us to eagerly loading every page of results (at the rate of about two pages per second).

A flurry of search requests being sent one after the other.png


Turbo Frames

It turns out we had chosen the wrong element for the intersection root. Instead of the ul, we needed to use the modal container (in this case a turbo-frame element) as the intersection root. The modal container, with it's fixed position and overflow, has a scrollable viewport that can intersect with the link or not.

The "viewport" of the ul, on the other hand, is always the full height of the ul. It's height is not constrained, and so the link will always intersect with it. This is a bit confusing, since the "viewport" of the ul  isn't fully visible (it overflows from the modal viewport), but that doesn't make a difference here (not even in Safari).

So we updated our observer to use the turbo-frame as the root instead. All good, right? Nope. In fixing keyboard navigation in the modal, we broke pagination altogether on /search/new. entry.isIntersecting now returned false all the time, because of some styling we have on turbo-frame.

turbo-frame {
  display: contents;
}

Using display: contents means the turbo-frame doesn't produce a box like other standard elements do. No box means no height, no viewport, and no possibility for anything to intersect with it.

The box model for display contents shows it has no height or width.png



This doesn't happen with the modal because the modal styles include display: block. We could try that on /search/new, but then we'd be back to eagerly loading every page of results.

Scrollable Offset Parent

We had /search/new working fine when we used the document viewport as the intersection root. And we had the modal working fine when we used the modal container as the intersection root. So why not both?

With help from Javan, we ended up with:

get scrollableOffsetParent() {
  const root = this.element.offsetParent
  return root.scrollHeight > root.clientHeight ? root : null
}

We take the offsetParent (i.e. the closest positioned ancestor), and then check whether that element's scrollHeight (which includes the height of the overflow content that isn't visible yet) is greater than its clientHeight (which does not include the overflow). (Be mindful of layout thrashing when using these methods.)

It helps that this is also pretty much what the docs recommend:

Typically, you'll want to watch for intersection changes with regard to the element's closest scrollable ancestor, or, if the element isn't a descendant of a scrollable element, the viewport.

Loading Next Page

That's all I got. Happy scrolling!