Fix propeller crash when inferring literal type for an offloaded literal #4822

Job	Run time
unpack-envvars	1s
flyteadmin integration tests image / build_docker	5m 0s
Build Images / Build & Push Image (datacatalog) / Push to Github Registry	4m 8s
Build Images / Build & Push Image (flyteadmin) / Push to Github Registry	5m 46s
Build Images / Build & Push Image (flytescheduler) / Push to Github Registry	5m 50s
Build Images / Build & Push Image (flytecopilot) / Push to Github Registry	4m 13s
Build Images / Build & Push Image (flytepropeller) / Push to Github Registry	6m 48s
Dry Run Goreleaser	7m 22s
Test Getting started	4m 13s
Lint (datacatalog) / Run Lint	1m 55s
Lint (flytectl) / Run Lint	1m 38s
Lint (flyteadmin) / Run Lint	2m 50s
Lint (flyteplugins) / Run Lint	2m 20s
Lint (flyteidl) / Run Lint	1m 26s
Lint (flytepropeller) / Run Lint	2m 38s
Lint (flytestdlib) / Run Lint	2m 15s
Unit Tests (datacatalog) / Run Unit Test	2m 36s
Unit Tests (flyteadmin) / Run Unit Test	4m 5s
Unit Tests (flytecopilot) / Run Unit Test	2m 16s
Unit Tests (flyteplugins) / Run Unit Test	3m 29s
Unit Tests (flytepropeller) / Run Unit Test	3m 41s
Unit Tests (flytectl) / Run Unit Test	3m 8s
Unit Tests (flytestdlib) / Run Unit Test	3m 15s
Check Go Generate (datacatalog) / Go Generate	1m 3s
Check Go Generate (flyteadmin) / Go Generate	1m 44s
Check Go Generate (flytecopilot) / Go Generate	1m 1s
Check Go Generate (flytectl) / Go Generate	1m 21s
Check Go Generate (flytepropeller) / Go Generate	1m 41s
Integration Test (flyteadmin) / Integration tests	5m 45s
	1h 33m 28s

Provide feedback