Skip to content

feat(sqlite): add `StatementSync.prototype.iterate` method

Hello,

I wanted an iterate method for StatementSync. An SQL query can return lot of rows, for memory efficiency it seems important we can fetch rows on demand, instead collect all in an array.

I'm not a C/C++ developer, so this was really challenging to create an object with a next callback with v8. I tried to obtain Iterator.prototype so the results extends Iterator. I did not found how to do it with v8.

I'm sure my code is not ready to be merged:

  • memory leak.
    • follow similar algorithm than All
    • variables for next and return callback are wrapped by v8::External added to iterable_iterator JS Object, accessed from context (.This())
  • no unit tests
    • I did not found unit tests for sqlite module, I can do some in JS, but in C++...
    • Maybe they are not mandatory for experimental modules ?
  • solution to have an iterable iterator from iterate() is a bit hacky. in js sqlite module I decorate the method to return Iterator.from(<result from cpp iterate>).
    • If someone can help me to extend Iterator from v8 it could be great, else I'll keep the decorator.
  • documentation

I'm hoping to get some help here to finalize this PR, so that we all get a quality iterate method.

Manual test:

// % ./node --experimental-sqlite
sqlite = require('node:sqlite');
db = new sqlite.DatabaseSync(':memory:');
stmt = db.prepare(`WITH cte(a) AS (VALUES (1), (2), (3)) SELECT * FROM cte`);
iterator = stmt.iterate();
// > Object [Iterator] {}
iterator.map(({a}) => a).map(x => x*x).toArray();
// > [ 1, 4, 9 ]

So it's properly integrated with JS iterator protocol, it's an iterable iterator, it's integrated with IteratoHelpers.

Merge request reports

Loading