Yuting Chen presented his workd on JVM testing in ICSE 2019

Abstract: The Java Virtual Machine (JVM) is the cornerstone
of the widely-used Java platform. Thus, it is critical to ensure
the reliability and robustness of popular JVM implementations.
However, little research exists on validating production JVMs.
One notable effort is classfuzz, which mutates Java bytecode
syntactically to stress-test different JVMs. It is shown that
classfuzz mainly produces illegal bytecode files and uncovers
defects in JVMs’ startup processes. It remains a challenge to
effectively test JVMs’ bytecode verifiers and execution engines
to expose deeper bugs.
This paper tackles this challenge by introducing classming, a
novel, effective approach to performing deep, differential JVM
testing. The key of classming is a technique, live bytecode
mutation, to generate, from a seed bytecode file f , likely valid,
executable (live) bytecode files: (1) capture the seed f ’s live
bytecode, the sequence of its executed bytecode instructions; (2)
repeatedly manipulate the control- and data-flow in f ’s live
bytecode to generate semantically different mutants; and (3)
selectively accept the generated mutants to steer the mutation
process toward live, diverse mutants. The generated mutants are
then employed to differentially test JVMs.
We have evaluated classming on mainstream JVM implementations,
including OpenJDK’s HotSpot and IBM’s J9, by mutating
the DaCapo benchmarks. Our results show that classming
is very effective in uncovering deep JVM differences. More than
1,800 of the generated classes exposed JVM differences, and
more than 30 triggered JVM crashes. We analyzed and reported
the JVM runtime differences and crashes, of which 14 have
already been confirmed/fixed, including a highly critical security
vulnerability in J9 that allowed untrusted code to disable the
security manager and elevate its privileges (CVE-2017-1376).