Pagination with 500000 items

Rails
1

hi

I’ve got a pagination with about 500000 items, and I don’t know if it
will be slow. I need only the first 1000, so what I’m trying to do is
@item_pages, @items = paginate(:products, :order => “title”, :per_page
=> 15, :limit => 1000)

But paginate seems to have no limit option. Will the pagination be
fast without this option?

Luma

2

Luma wrote:

But paginate seems to have no limit option. Will the pagination be
fast without this option?

Pagination works by running two queries, one that returns a count of the
total number of results, and one that returns the requested page of data
only. This second query returning the page of data already uses LIMIT
and OFFSET in the SQL query, so there is no need for an additional limit
option.

Cheers,

Adam

3

In reality no user of your application will page through 500,000 items.

If you only need the first 1,000 items, why not limit the results to
1,000
and paginate those in viewable chunks?

You can use a combination of tagging and searching to help users get the
“right” 1,000 records to look through.

4

Luma wrote:

But paginate seems to have no limit option. Will the pagination be
fast without this option?

The built-in paginator is a known performance and scalability bottleneck
and is likely to be removed from the Rails core in upcoming releases.
Luckily, there’s a couple of very good plugins out there that’ve tackled
the issue: paginating_find [1] and will_paginate [2].

[1] Welcome cardboardrocket.com - BlueHost.com
[2] http://errtheblog.com/post/929

–
Roderick van Domburg

5

Just use will_paginate and not look back.

On 7/29/07, Luma [email protected] wrote:

fast without this option?

Luma

–
Cheers!

6

I second that…
The Paginator in rails has an issue with getting all the records for
you…
We use paginating_find plugin here…works good so far.

cheers…

On Jul 29, 8:55 pm, Roderick van Domburg <rails-mailing-l…@andreas-

7

The built-in paginator is a known performance and scalability bottleneck
and is likely to be removed from the Rails core in upcoming releases.
Luckily, there’s a couple of very good plugins out there that’ve tackled
the issue: paginating_find [1] and will_paginate [2].

It’s gone in edge rails, though there’s a plugin for backwards
compatibility purposes.

–
Rick O.
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com

8

Watch this #51 will_paginate - RailsCasts

Will explains things for you

9

I’ve got a pagination with about 500000 items, and I don’t know if it
will be slow. I need only the first 1000, so what I’m trying to do is
@item_pages, @items = paginate(:products, :order => “title”, :per_page
=> 15, :limit => 1000)

The built in rails pagination will explode with that many items. I
strongly recomment find_by_sql with a :limit parameter.

I recently used some the info here:
http://www.igvita.com/blog/2006/09/22/eager-find-by-sql-pagination-in-rails/
along with the paginating_find plugin from here:

Workign well with 100s of 1000s or records

10

Hi,

thanks all for the pagination tips, and thanks too for pointing out
the RailsCasts. For some reason I had not picked up on these yet,
but have just been and had a look and they are really cool.

Tonypm

11

thanks for that many answers. I’ll try the plugins.