Decluttering a DequntizeLinearF32 followed by QuantizeLinearI8 results in an error at model patching

[mmagician]

Quantizing the first part of Tensorflow's model from this tutorial fails when trying to declutter:
cargo run path-to-model/model_quant.onnx -i 1,784,u8

2024-01-10T13:33:10.956561599Z ERROR tract] ModelBuildingError

    Caused by:
        0: Error at stage declutter
        1: running pass declutter
        2: declutter node #1 "serving_default_flatten_input:0_dequant" DequantizeLinearF32
        3: Trying to substitute a 1,784,I8 by 1,784,U8.

The issue seems to be here:

if !original_fact.compatible_with(new_fact) {
    bail!("Trying to substitute a {:?} by {:?}.\n{:?}", original_fact, new_fact, self);
}

Since U8 is not "compatible" with an I8, the code bails out.
Two very uninformed ideas would be to either:

1. implement compatible_with for U8 <-> I8 (in fact, removing the compatibility check seems to work and replaces the two Dequant, Quant operations with a single LookupTable. I'm not sure whether that's the expected behaviour, but looking at the code I think yes).
2. Call shunt_outside_unchecked instead of shunt_outside inside the if incoming_dt == DatumType::I8 || incoming_dt == DatumType::U8 condition, where we're guaranteed (?) to have u8/i8 only.

Without decluttering, all works fine:

cargo run path-to-model/model_quant.onnx -i 1,784,u8 --declutter-step 0

Resulting in:

┏ 0 Source serving_default_flatten_input:0
┃   ━━━ 1,784,U8
┣ 1 DequantizeLinearF32 serving_default_flatten_input:0_dequant
┃   ━━━ 1,784,F32
┣ 2 QuantizeLinearI8 tfl.quantize
┃   ━━━ 1,784,I8
┣ 3 DequantizeLinearF32 sequential/flatten/Reshape_dequant
┃   ━━━ 1,784,F32

[kali]

Can you share the model onnx file so I can have a look ?

[mmagician]

It escaped my attention, apologies. Here's the model model_quant.onnx.zip

[SludgePhD]

I'm also seeing an error like that:

called `Result::unwrap()` on an `Err` value: running pass declutter

Caused by:
    0: declutter node #1 "serving_default_images:0_dequant" DequantizeLinearF32
    1: Trying to substitute a 1,3,320,320,I8 by 1,3,320,320,U8.
       ModelPatch { context: [], dont_apply_twice: None, model: Graph { nodes: [Node { id: 0, name: "tap.serving_default_images:0-0/0", inputs: [], op: TypedSource { fact: 1,3,320,320,U8 }, outputs: [1,3,320,320,U8 >1/0] }, Node { id: 1, name: "serving_default_images:0_dequant", inputs: [0/0>], op: ElementWiseOp(LookupTable { table: LutImpl { table: 256,U8 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139..., _boo: PhantomData<tract_linalg::generic::lut::GenericLut8> } }, None), outputs: [1,3,320,320,U8 ] }], inputs: [0/0>], outputs: [], outlet_labels: {}, properties: {}, symbol_table:  }, inputs: {}, taps: {0/0>: 0/0>}, shunts: {}, obliterate: [] }

The model causing this is a tf2onnx-converted version of the int8-quantized EfficientDet-Lite0 model from MediaPipe attached here:

mp_object_detection_efficientdet_lite0_int8.onnx.zip