Extended enum values are not handled as enums when used as Map keys

[ankulikov]

Hello, guys!
I've faced with a problem of inconsistent enum serialization:

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.util.Map;

public class JacksonEnumTest {

    public static void main(String[] args) throws JsonProcessingException {
        final ObjectMapper mapper = new ObjectMapper();
        final Map<MyEnum, String> map = Map.of(
            MyEnum.A, "1",
            MyEnum.B, "2");
        System.out.println(mapper.writeValueAsString(map));
    }

    enum MyEnum {
        A,
        B() {
            @Override
            public void foo() {
                // also do nothing
            }
        }

        public void foo() {
            // do nothing
        }

        @Override
        public String toString() {
            return name() + " as string";
        }
    }
}

Expected result:

{"A":"1", "B":"2"}

Actial result:

{"A":"1", "B as string":"2"}

Research:

Actually B.getClass() is not MyEnum but MyEnum$1 which is not enum anymore!

MyEnum.B.getClass().isEnum() == false

So, my proposal is to add this check to com.fasterxml.jackson.databind.JavaType:

 @Override
    public final boolean isEnumType() { 
    return  _class.isEnum() || Enum.class.isAssignableFrom(_class);
 }

[cowtowncoder]

To me that seems to work as expected: by default, Enum values (including subtypes which are detected as Enums just fine) are serialized using Enum.name(): overriding toString() has no effect.

But if you do want to use toString() instead, enable SerializationFeature.WRITE_ENUMS_USING_TO_STRING and it should as expected.
If so, you usually also want to enable DeserializationFeature.READ_ENUMS_USING_TO_STRING

[XakepSDK]

Can confirm issue. Test case.
jackson-databind 2.9.9.3

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class Test {
    public static void main(String[] args) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        Map<Pojo, String> testData = new HashMap<>();
        testData.put(Pojo.FIRST, "firstVal");
        testData.put(Pojo.SECOND, "secondVal");

        // uses #toString() instead of #name()
        // so json is {"FIRST":"firstVal","toString()":"secondVal"}
        // instead of {"FIRST":"firstVal","SECOND":"secondVal"}
        String json = mapper.writeValueAsString(testData);
        System.out.println(json);
        // crash here
        Map<Pojo, String> res = mapper.readValue(json, new TypeReference<Map<Pojo, String>>() {});

        // Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidFormatException:
        // Cannot deserialize Map key of type `dk.xakeps.jacksontest.Test$Pojo`from String "toString()":
        // not a valid representation,
        // problem: (com.fasterxml.jackson.databind.exc.InvalidFormatException)
        // Cannot deserialize Map key of type `dk.xakeps.jacksontest.Test$Pojo` from String "toString()":
        // not one of values excepted for Enum class: [SECOND, FIRST]
        //  at [Source: (String)"{"FIRST":"firstVal","toString()":"secondVal"}"; line: 1, column: 21]
    }

    public enum Pojo {
        FIRST, SECOND() {
            @Override
            public String toString() {
                return "toString()";
            }
        }
    }
}

[cowtowncoder]

Ah. Enums as Map Keys are behaving in unexpected way... that makes more sense.

One thing I would suggest, which should work (and may be required) is to force serialization type, so use mapper.writerFor(new TypeReference<....>).writeValueAsString(....) and see if that helps.

I will keep this open for investigation since handling of Enums as keys is not as well developed as that for values.

[cowtowncoder]

Root cause was that

EnumSubType.class.isEnum()

returns -- surprisingly! -- false (not sure if that'd be considered JDK bug or not).
What does work is instead

Enum.class.isAssignableFrom(EnumSubType.class)

check.