tid dropped if >max(uint32)

[d4l3k]

The thread IDs from JSON traces are getting dropped if the tid doesn't fit in a uint32. These are displayed finewith chrome://tracing but doesn't work with Perfetto.

I have py-spy chrometrace formatted traces in the format of:

{"args":{"filename":"\u003cfrozen importlib._bootstrap\u003e","line":688},"cat":"py-spy","name":"_load_unlocked","ph":"B","pid":10402,"tid":139627874611776,"ts":9720432}

These are I believe the pthreads thread ID and doesn't fit in a uint32.

perfetto/src/trace_processor/importers/json/json_trace_parser_impl.cc

Line 121 in f8a5e8a

opt_tid = json::CoerceToUint32(value["tid"]);

[LalitMaganti]

I'd suggest this is WAI from the Perfetto perspective: please see https://perfetto.dev/docs/faq#why-does-perfetto-not-support-lt-some-obscure-json-format-feature-gt-

[d4l3k]

@LalitMaganti looks like TrackEvent protos are also using int32 for thread ids so doesn't really solve the underlying problem regardless

As a hack it, does look fairly straightforward to just cast it as a string and set it as a threadname instead

perfetto/src/trace_processor/importers/json/json_trace_parser_impl.cc

Lines 112 to 123 in f8a5e8a

	std::optional<uint32_t> opt_tid;
	if (value.isMember("tid")) {
	if (value["tid"].isString()) {
	// See the comment for |pid| string handling above: the same applies here.
	StringId thread_name_id = storage->InternString(value["tid"].asCString());
	opt_tid = thread_name_id.raw_id();
	procs->UpdateThreadName(*opt_tid, thread_name_id,
	ThreadNamePriority::kOther);
	} else {
	opt_tid = json::CoerceToUint32(value["tid"]);
	}
	}

[LalitMaganti]

looks like TrackEvent protos are also using int32 for thread ids so doesn't really solve the underlying problem regardless

Right: the main reason is that supporting int64 thread IDs can be quite a performance concern as it doubles the memory use of all places where tids are stored and when people use us for multi-GB traces, this adds up.

My response was more dealing with "why does this work on chrome tracing but not in Perfetto". You're correct that if you need int64 tid support and there's not a way to emit 32 bit tids instead then yes you'll need some hack.

As a hack it, does look fairly straightforward to just cast it as a string and set it as a threadname instead

That is quite a hack :) can we maybe instead directly cast from int64 -> int32 instead and store the tids in the casted form? We can expose the tids as the name as you suggest then.