On May 26, 2008, at 2:37 AM, Joel VanderWerf wrote:
Does anyone know how to keep sqlite from going to disk so much, or
is that a necessary part of the locking mechanism? Running with
strace, the write() is immediately followed by fsync(). It keeps my
disk very busy for what should be cheap insert operations to the end
of a table with no indexes
Take a look at PRAGMA synchronous = FULL | NORMAL | OFF
http://www.sqlite.org/pragma.html
Make sure to read on “Atomic Commit In SQLite” before changing
anything though:
http://www.sqlite.org/atomiccommit.html
Petite A. wrote:
Take a look at PRAGMA synchronous = FULL | NORMAL | OFF
Thanks to you and Philip for the good suggestions… will digest them
for a while…
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Joel VanderWerf wrote:
| John C. wrote:
|> But unless one or more of those reasons are biting you hard, the
|> simplicity of sqllite is a major win.
|
| On the subject of sqlite (and tangentially ruby), and with all you smart
| folks on the list, a question:
|
| Does anyone know how to keep sqlite from going to disk so much, or is
| that a necessary part of the locking mechanism? Running with strace, the
| write() is immediately followed by fsync(). It keeps my disk very busy
| for what should be cheap insert operations to the end of a table with no
| indexes. Cpu usage is low too (and I’m using SQLite3::Database#prepare
| from the sqlite3 gem, to keep query overhead low).
Two suggestions:
- Use indexes, as that will improve your performance, and it might just
help with making sure SQLite actually appends directly, instead of
seeking a position first. And the performance improvement even with
small DBs is remarkable: milliseconds instead of seconds…).
- Are you using transactions? If you don’t use transactions, every
insert or update is written to disk immediately, requiring a sync to
disk (we don’t want inconsistent data). However, with a transaction,
every operation gets committed to the DB in one go (which is faster,
allows rollbacks in case an operation fails, and should require less
fsync() calls).
Phillip G.
Twitter: twitter.com/cynicalryan
Blog: http://justarubyist.blogspot.com
Terminate input by end-of-file or marker, not by count.
~ - The Elements of Programming Style (Kernighan & Plaugher)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkg6HHEACgkQbtAgaoJTgL8ZVACeNM4+TOOBRISib7rlqRoHefNI
OFIAnRFavTkTEaIGZKw3It0V95i37yV+
=h05V
-----END PGP SIGNATURE-----