Personal Website of Yakov Shafranovich

« Huntin down a String bug in Java VM         ASRG rechartered »

The bug hunt has finished!

After some digging in the JVM source code, our CTO came out with the exact source of the problem and a possible workaround. Of course this is all at your risk, so don’t blame us or Sun if you have problems. A copy of the message we submited to Sun appears here:

SOURCE CODE TRACEWe took a look at the source code of the JVM. The problem stems from the fact that float values are used to indicate the maximum value of bytes per characters in java.nio.charset.CharsetEncoder.maxBytesPerChar.

The issue is that floats cannot accuratly hold more than 2^24 integer values which is equals to 16,777,216. After that value is reached, the encoding operation in the character set classes incorrectly rounds down the amount of memory needed for the buffer. The correct solution would be to use doubles instead, or account for the round off problem by increasing the buffer size.

SUGGESTED WORKAROUND

The workaround that we are using, is to use to .getBytes() on a substring that is smaller than 16MB, and combined the results by either using a ByteArrayOutputStream or a ByteBuffer.

NOTE: If you are planning on using more than one-byte characters sets, than you have to make sure that your buffer is set accordingly.

Tags: bug, java —

Permalink | Trackback URL | This post has