Skip to content

[WIP] tracing: replacing trace_events with perfetto

Rodrigo Muino Tomonari requested to merge github/fork/jasnell/perfetto into master

This is a work in progress preview of integrating perfetto as a replacement for the trace events subsystem.

This is no where near complete. I'm opening it early to start getting feedback.

Perfetto (https://perfetto.dev) is a tracing subsystem created by Google and used in Android and Chrome. It supports both in-process and out-of-process with the ability to integrate application and system traces.

The traces generated are emitted as encoded protobufs rather than JSON, and the subsystem is much more stable and robust than the existing unfinished tracing support we currently have.

Unfortunately, it's also much more complex, with large new dependencies and a complicated configuration model.

Perfetto will allow richer and more flexible diagnostic tracing use cases, including the ability to stream tracing data in-process (something we currently cannot do easily). The traces are more efficient at runtime, take less memory and disk space, and generally work better.

Currently, this will likely only build on Linux systems (I've only tested on Ubuntu to this point). It will definitely fail on Windows, and DEFINITELY will not pass CI in any way shape or form. There's still a ton of work to do here (including reducing the size of the added dependencies)

Notes

To build with perfetto:

$ ./configure --use-perfetto
$ make -j4

(there's currently a bug in the building step, first time through, it will likely fail, just run make -4 again and it should be ok. It's an open todo to fix it)

Once built, running with the regular command line options (e.g. --trace-event-categories=v8,node) will cause the trace event file to be emitted. Currently, perfetto is being invoked with verbose logging enabled to allow easier debugging, that will be turned off a bit later once I'm sure everything is working right.

The trace event file will be a binary sequence of serialized protobufs rather than a json document. It needs to be loaded into the perfetto UI to be processed. The legacy chrome://tracing will not work

Traces from JavaScript (e.g. trace events for AsyncResource and console.time() are emitted as legacy trace events. I will be converting those over into the new model soon.

The entire current trace_events module will be completely redone. The existing API no longer makes sense and is not supportable. Fortunately, it's still considered experimental but we should still consider it a breaking change. A new API is being implemented that allows for some very interesting options... this is not yet fully implemented but here's an example of what will be possible:

const { TracingSession } = require('trace_events');

const session = new TracingSession({
  categories: 'v8,node.async_hooks',
  duration: 10, * 1000 /* Trace for 10 seconds */
  flushPeriod: 200  /* Flush traces every 200 ms */
});

for await (const chunk of session) {
  // Process chunk of trace event data
}

This will create a new tracing session that pipes traces into a stream rather than out to file. The file trace and the stream trace are independent of each other but can have overlapping information if both are enabled at the same time.

There are a number of other features that are planned as this progresses, such as the ability to limit trace file size, apply compression to the trace output automatically, support tracing via the Inspector protocol, provide trace triggers, provide a user-accessible JavaScript API for emitting traces, and more.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Merge request reports