Paged Scopes: A Will_paginate Alternative
The first time I needed to paginate data in a Rails site, I went straight for the de-facto standard, which, since Rails 2.0, has undoubtedly been will_paginate. However, it didn’t take me long to discover it couldn’t do all that I wanted it to.
Most importantly, I wanted to be able to redirect from a resource member action (the update action, say) back to the index action, with the page set so that the edited resource would be part of the paged list. I couldn’t see a way to do that with will_paginate. I found the will_paginate helper a bit messy – ever heard of block helpers? And finally, I wanted my pages to be objects, not just numbers. This would let me load them in controllers and pass them to named routes and have them just work. Will_paginate didn’t seem to fit the bill.
Now don’t get me wrong; will_paginate must be pretty awesome – it’s the third most watched repo on GitHub as I write this. But choice is always good, and to me, will_paginate seems a bit bloated and ill-fitting to the way I like to structure my code.
So, naturally, I rolled my own pagination solution. I’ve finally packaged it up and released it as a new ActiveRecord pagination gem, Paged Scopes. It’s everything I need in Rails pagination and nothing I don’t. It’s also lightweight and pretty solid. Check it out!
Features
The bullet-point summary of the Paged Scopes gem goes something like this:
- Pages are instances of a class which belongs to the collection it’s paginating;
- Pages can be found by number or by contained object;
- Each page has its own paged collection, which is a scope on the underlying collection; and
- Flexible, Digg-style pagination links are achieved using a block helper.
A Console Session Is Worth a Thousand Words
Let’s take a look at how pagination works with Paged Scopes. Consider a collection of articles obtained using a published named scope.
@articles = Article.published => [#<Article id: 1, title: "Article #1">, ..., #<Article id: 5, title: "Article #5">] @articles.count => 5
The Paged Scopes gem adds a per_page attribute directly to named_scope collections (and to association collections, too). This value determines how many objects each page contains, and needs to be set before we can paginate the collection:
@articles.per_page = 2 => 2
Paginating this collection will now give us three pages.
How do we access these pages? By calling pages, the other main method added to ActiveRecord collections. It returns an enumerated class, the instances of which represent the pages of the collection. We can interact with the pages class in some familiar ways:
@articles.pages => #<Class:0x24ea99c> @articles.pages.count => 3 @articles.pages.first => #<Page, for: Article, number: 1> @articles.pages.find(1) => #<Page, for: Article, number: 1> @articles.pages.last => #<Page, for: Article, number: 3> @articles.pages.find(4) => # PagedScopes::PageNotFound: couldn't find page number 4 @articles.pages.all => [#<Page, for: Article, number: 1>, #<Page, for: Article, number: 2>, #<Page, for: Article, number: 3>] @articles.first.to_param => "1"
Looks just like any other model – each page is its own self-contained object, as it should be. We can access the collection objects in the page using the same name as the underlying model. In our example, our collection contains Article instances, so the articles in the page are accessed using an articles method:
@articles.pages.first.articles => [#<Article id: 1, title: "Article #1">, #<Article id: 2, title: "Article #2">] @articles.pages.last.articles => [#<Article id: 5, title: "Article #5">] @articles.pages.map(&:articles).map(&:size) => [2, 2, 1] @articles.pages.map { |page| page.articles.map(&:title) } => [["Article #1", "Article #2"], ["Article #3", "Article #4"], ["Article #5"]]
So far, so good. But what, exactly, is return by the articles method? Let’s see:
@articles.pages.first.articles.class => ActiveRecord::NamedScope::Scope @articles.pages.first.articles.send(:scope, :find) => {:conditions=>"published_at IS NOT NULL", :offset=>0, :limit=>2} @articles.pages.last.articles.send(:scope, :find) => {:conditions=>"published_at IS NOT NULL", :offset=>4, :limit=>2} @articles.send(:scope, :find) => {:conditions=>"published_at IS NOT NULL"}
Yep, it’s just a scope on the parent collection, with :limit and :offset added according to the page number. This is kinda important. It means that the objects in the paged collection will not load from the database until they are referenced. We can pass around page objects in view helpers and named routes and so on, without worrying about inadvertently loading the paged data.
Finding a Page By Its Contents
One particularly nice feature of the library is that we can find a page by identifying an object the page contains.
article = Article.find(3) => #<Article id: 3, title: "Article #3"> @articles.pages.find_by_article(article) => #<Page, for: Article, number: 2> article = articles.find(8) => #<Article id: 8, title: "Article #8"> @articles.pages.find_by_article(article) => nil @articles.pages.find_by_article!(article) => # PagedScopes::PageNotFound: #<Article id: 8, title: "Article #8"> not found in scope
This is really handy if you want to redirect from a resource member action to the paged of the index containing the edited object. (More on this later.)
This is implemented using the code I described in my previous post. As a result you get a couple of freebies on your ActiveRecord objects:
article = Article.scoped(:order => "title ASC").find(3) => #<Article id: 3, title: "Article #3"> article.next => #<Article id: 4, title: "Article #4"> article = Article.scoped(:order => "title DESC").find(3) => #<Article id: 3, title: "Article #3"> article.next => #<Article id: 2, title: "Article #2"> article.previous => #<Article id: 4, title: "Article #4">
In other words, you can find the next and previous objects for any object in a collection. This provides an easy way to link to neighbouring objects (e.g. older and newer posts in a blog).
A Caveat
It’s important to store the paged scope or association collection in a variable, rather than refer to it directly. In other words:
# Do this: @articles = @user.articles.published # or whatever => [#<Article ...>, ..., #<Article ...>] @articles.per_page = 5 => 5 @articles.per_page => 5 # Don't do this: @user.articles.published.per_page = 5 => 5 @user.articles.published.per_page => nil
This is because paged scopes and association collections return new instances each time they’re called. You need to hang onto them to set the per_page and then get the pages.
In Part II …
In my next article I’ll describe some extensions to ActionController that you get with the Paged Scopes gem. They include some handy routing and controller methods which encapsulate my preferred usage pattern for pagination. And I’ll describe the paginator, an object which makes rendering pagination links a cinch.
Get It!
In the meantime, you can install the Paged Scopes gem as follows:
gem sources -a http://gems.github.com # just once sudo gem install mholling-paged_scopes
And in your config/environment.rb, if you’re on Rails:
config.gem "mholling-paged_scopes", :lib => "paged_scopes", :source => "http://gems.github.com"
[ UPDATE: The gem is now hosted on Gemcutter, with slightly different installation instructions – see here. ]
Peruse the code at GitHub.
Have at it!
Thanks, I can see usefulness for this right away. I will try it out sometime soon and tinker with it.
(This is a comment I gave at RailsFire before I realized they just have a repost of this.)
Nice. In one of my projects I was using something I called "poor man's will_paginate" which was a will_paginate re-implementation with named scopes (it wasn't so object-oriented as your design). I was playing with it for a while, but decided not to build a library strictly around ActiveRecord features because I support DataMapper, Sequel and others.
People like will_paginate because using it is only 2 lines of code—one in controller and the other in the view—but it's opinionated so some things are intentionally hard to change (like HTML markup pagination, as you mentioned). Your project seems really nice for people who need more power and hence take a more low-level approach.
@Mislav: hey, thanks for reading! :) Any bravado in my article is just theatre of course. Respect to anyone who's doing open-source, I say. I just wanted to throw something of my own into the ring. I have obviously not used will_paginate much as I didn't even know it was ORM-agnostic. I would say the main drawcard for my code is the ability to find a page from an object in it. (For all I know, you can do that with will_paginate too.)
Anyway, thanks again for having a read.
Thank you, that's great. Could you please advise how I can use it with remote AJAX links? I need go to a page which contains an object with a known id
@Anna: I've not tried AJAX pagination myself, so you're on your own with that one, sorry! You shouldn't have any problems using PagedScopes though.
Still using will_paginate on my current project and its doing a good job, but sometimes it feels kinda clumsy.
Thanks for taking the time to create and publish your work, I will dive straight in into it!
Hi Matthew! It's always refreshing to see new alternatives to de-facto standard libraries.
I've noticed a big delay when paginating very populated tables and just would like to be sure: when I do a "MyModel.my_scope.pages", it doesn't load the whole scoped items, right?
@Raul: No, the call to #pages just returns a Page class for that scope, there shouldn't be any pre-loading of the database going on. You could tail your log to check what SQL commands are being issued when your code is lagging. I have not done any performance testing myself so it could be worth your while to compare it against will_paginate if you intend to use it for something where speed is important!