qh.i: Class 'j' is not registered for polymorphic serialization in the scope of 'h0'.

[dawidhyzy]

Describe the bug

After building the app in the release variant polymorphic serialization doesn't work.

To Reproduce

SocketEvent.kt

@Serializable
sealed interface SocketEvent

AnalyticsEvent.kt

@Serializable
sealed interface AnalyticsEvent : SocketEvent {
    @Serializable
    data class PlayerState(
        @SerialName("player_state") val playerState: State,
     ) : AnalyticsEvent

    @Serializable
    data class ButtonClick(
        @SerialName("screen_name") val screenName: String,
     ) : AnalyticsEvent
}

SocketConnectionEvent.kt

@Serializable
sealed class SocketConnectionEvent(
    @SerialName("event") open val event: String,
) : SocketEvent {

    @Serializable
    class PingEvent : SocketConnectionEvent(
        event = SocketEventName.PING.event
    )
}

WebSocketClient.kt

override suspend fun send(event: SocketEvent) {
        val trimmed = event.trimType()
        session?.run {
            sendSerialized(trimmed)
        } ?: Timber.e("No session")
    }

    /**
     * As kotlinx-serialization adds type discriminator to serialized object we need to trim it.
     * See: https://github.com/Kotlin/kotlinx.serialization/issues/464
     */
    private fun SocketEvent.trimType(): JsonElement {
        val json = Json.encodeToJsonElement(this) // <- here we get the Exception
        val valuesWithoutType = json.jsonObject.filterNot { it.key == "type" }
        return Json.encodeToJsonElement(valuesWithoutType)
    }

Proguard configuration

# Keep `Companion` object fields of serializable classes.
# This avoids serializer lookup through `getDeclaredClasses` as done for named companion objects.
-if @kotlinx.serialization.Serializable class **
-keepclassmembers class <1> {
    static <1>$Companion Companion;
}

# Keep `serializer()` on companion objects (both default and named) of serializable classes.
-if @kotlinx.serialization.Serializable class ** {
    static **$* *;
}
-keepclassmembers class <2>$<3> {
    kotlinx.serialization.KSerializer serializer(...);
}

# Keep `INSTANCE.serializer()` of serializable objects.
-if @kotlinx.serialization.Serializable class ** {
    public static ** INSTANCE;
}
-keepclassmembers class <1> {
    public static <1> INSTANCE;
    kotlinx.serialization.KSerializer serializer(...);
}

# @Serializable and @Polymorphic are used at runtime for polymorphic serialization.
-keepattributes RuntimeVisibleAnnotations,AnnotationDefault

Expected behavior

Object serialization works for release build as it works for debug

Environment

• Kotlin version: 1.7.10
• Library version: 1.4.0
• Kotlin platforms: JVM
• Gradle version: 7.5.1
• Android Gradle Plugin: 7.4.0-beta02

[bubenheimer]

I just ran into this same problem, except I even have a custom serializer on the base class to eclipse the polymorphism. So it seems that the custom serializer is lost entirely as well, despite using the standard recommended Proguard config from the kotlinx.serialization repo. Or at least this is what I think the error implies. (I use R8.)

The custom serializer class is still there, but it seems like it is not being used, as if the annotation parameter had gone missing or did not get obfuscated the same way.

[bubenheimer]

I have investigated my problem at great length and I am certain now that there is a bug in my version of R8 (3.3.70) that incorrectly strips annotations in some cases. In particular the @Serializable annotation on my sealed interface is stripped, which removes the custom serializer in my case, and might contribute to the problem that @dawidhyzy is seeing. I looked at the bytecode, and it's gone, despite -keepattributes RuntimeVisibleAnnotations. An explicit -keep package.for.AnalyticsEvent { *; } or similar will hold on to the annotation, customize it to what you need.

The bug is also present in R8 3.3.84, which is the latest R8 for AGP 7.3.x (Dolphin) right now, and R8 4.0.31, which is the latest for AGP 7.4.x (Electric Eel).

However, this happens only in R8 fullMode. The default android.enableR8.fullMode=false works fine. So this may not be the issue that @dawidhyzy is seeing.

FWIW: https://issuetracker.google.com/issues/252388315

[bubenheimer]

@dawidhyzy do you use android.enableR8.fullMode=true? If not then I'll stop spamming this issue.

[dawidhyzy]

@bubenheimer sorry for the delay but I couldn't test the explicit proguard rule till now. It fixed the issue.
I don't think we use fullMode, at least I don't see it enabled in gradle.properties. Is there any other place where it can be applied?

[bubenheimer]

There are a few other ways to set Gradle properties, Gradle docs are your best friend there, I don't deal with these often. I bet you can get a Gradle properties dump somehow.

The rule I gave here simply holds on to everything in that class, so it's not that meaningful in terms of its implications, really, yet it will keep your code working for sure. In the end you can look at the bytecode in the APK for your couple classes and compare with & without R8/Proguard to see what's really missing. You don't really need to understand bytecode for this exercise, you're just visually diffing.

In the linked issuetracker issue I offer another, general rule for the fullMode case, you could see if that happens to help in your case, but without doing the analysis you're just randomly trying out stuff at that point.

[bubenheimer]

So to wrap this up as far as R8 fullMode goes, I suggest adding the following to the sample Proguard in the README. It should not hurt or change anything when not using R8 fullMode, yet without these rules, running fullMode is borked.

-keep, allowoptimization, allowobfuscation class kotlinx.serialization.Serializable { *; }

-if @kotlinx.serialization.Serializable class **
-keep, allowshrinking, allowoptimization, allowobfuscation class <1>

There is more context in the linked Android issuetracker reference.

Edit: based on the update to the R8 FAQ, the least restrictive rule would actually be as follows, but this one only works for R8 users, it sounds like Proguard would show an error:

-if @kotlinx.serialization.Serializable class **
-keep, allowshrinking, allowoptimization, allowobfuscation, allowaccessmodification class <1>

[sandwwraith]

Thanks for your investigation. As far as I understood, the action point for maintainers is to update suggested proguard rules?

[bubenheimer]

Yes, I think it can make sense to add the rule set to the suggested Proguard rules, to accommodate any R8 full mode users. However, I am not sure how many people actually use full mode, I suspect it's few. I've given up on it for production use for the time being.