Generating a YOLO-WORLD (S) tflite model.

[Tomvdsch]

Hello everyone,

I'm trying to generate a YOLO-WORLD tflite model (small version) but i can not figure it out. I'm using the tflite_deploy.md and reparameterize.md guides but i just can not figure out how to do it correctly. I'm also a bit confused with config i need to use for the tflite_deploy process. I tried using the included (extra large) model version but its too slow for what i'm trying to achieve. I was wondering if anyone managed to create a tflite version of the small model version already or if someone can help me generate one successfully.

[wondervictor]

Hi,could you share the steps you did?

[Tomvdsch]

I managed to create a tflite model but are still have some problems with it.

First i used step 1 of reparameterize.md to generate some text prompts. After that i ran step two using these prompts and the yolo_world_v2_s_obj365v1_goldg_pretrain-55b943ea.pth model. For the model config i could not figure out what to change besides the num_classes so i only changed that. I also did not do step 4 of reparameterize.md.

When i finished with reparameterize.md i went into tflite_deploy.md, installed the priliminaries and ran python deploy/export_onnx.py path/to/config path/to/weights --custom-text path/to/customtexts --opset 11 --without-bbox-decoder
using the config and model i created during the reparameterize.md process. As custom texts i used data\texts\obj365v1_class_texts.json.

When the onnx model was generated and i finished downloading the "data/coco/val2017/" i ran this code:
import os
import random
from PIL import Image, ImageOps
import cv2
import glob
import numpy as np

root = "data/coco/val2017/"
image_list = os.listdir(root)
image_list = [os.path.join(root, f) for f in image_list]
random.shuffle(image_list)

img_datas = []
for idx, file in enumerate(image_list[:100]):
image = Image.open(file).convert('RGB')
# Get sample input data as a numpy array in a method of your choosing.
img_width, img_height = image.size
size = max(img_width, img_height)
image = ImageOps.pad(image, (size, size), method=Image.BILINEAR)
image = image.resize((640, 640), Image.BILINEAR)
tensor_image = np.asarray(image).astype(np.float32)
tensor_image /= 255.0
tensor_image = np.expand_dims(tensor_image, axis=0)
img_datas.append(tensor_image)

calib_datas = np.vstack(img_datas)
print(f'calib_datas.shape: {calib_datas.shape}')
np.save(file='tflite_calibration_data_100_images_640.npy', arr=calib_datas)

And as last i ran
python onnx2tf.py -i "yolo_world_v2_s_obj365v1_goldg_pretrain-55b943ea_rep_conv.onnx" -oiqt -cind "images" "tflite_calibration_data_100_images_640.npy" "[[[[0.,0.,0.]]]]" "[[[[1.,1.,1.]]]]" -onimc "scores" "bboxes" --verbosity debug

This generated a .tflite model what i used in a slightly rewritten version of tflite_demo.py that takes in a .mp4 file instead of a image and also saved a .mp4 file after processing each frame. But when i'm running this code it keeps detecting my walls and other objects as a person with a confidence score of 1.00 even though i'm also in the frame. Any ideas how i can fix or improve on what i did?

[Tomvdsch]

The yolo_world_x_coco_zeroshot_rep_integer_quant.tflite file also given in tflite_deploy.md seems to work correctly but is way too big and thus slow.

[Tomvdsch]

The problem seems to be the structure of my models output

• With yolo_world_x_coco_zeroshot_rep_integer_quant.tflite the output_details are:
  [{'name': 'PartitionedCall:0', 'index': 858, 'shape': array([ 1, 8400, 4]), 'shape_signature': array([ 1, 8400, 4]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'PartitionedCall:1', 'index': 839, 'shape': array([ 1, 8400, 80]), 'shape_signature': array([ 1, 8400, 80]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]

• With my model the output_details are:
  [{'name': 'PartitionedCall:0', 'index': 331, 'shape': array([ 1, 8400, 80]), 'shape_signature': array([ 1, 8400, 80]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]

But i have no idea how to solve this. Is this because of a wrong config file?