Skip to content

Re-use queries in reference parsers

yorickpeterse-staging requested to merge reuse-queries-in-reference-parsers into master

What does this MR do?

This MR changes the reference parsing pipeline to cache queried objects and re-use them on subsequent runs. Data is cached in the RequestStore module so it's automatically flushed after a request.

Are there points in the code the reviewer needs to double check?

Mostly the style of things.

Why was this MR needed?

Certain references are processed multiple times. For example, for every system note we check if it's a cross reference by getting the issues, MRs and commits it references. When redacting data we may end up querying these very same objects. By caching this we can quite drastically reduce timings and SQL query counts.

What are the relevant issue numbers?

#15607 (closed)

Screenshots (if relevant)

Staging timings (method calls):

timings

Production timings (method calls):

production_query_counts

timing_issues_for_nodes_production

timings_issue_references

Does this MR meet the acceptance criteria?

Merge request reports