Java Code Examples for java.nio.CharBuffer#clear()

The following examples show how to use java.nio.CharBuffer#clear() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: AcceptanceFuncTest.java    From ph-commons with Apache License 2.0 6 votes vote down vote up
private void _verifyChunkedOutDecode (final int i, final ByteBuffer in, final String decoded)
{
  final CharBuffer out = CharBuffer.allocate (i);
  int decodeCount = 0;
  final StringBuffer sb = new StringBuffer ();
  CoderResult result = CoderResult.OVERFLOW;
  while (decodeCount < decoded.length ())
  {
    assertEquals ("at position: " + decodeCount, CoderResult.OVERFLOW, result);
    result = decoder.decode (in, out, true);
    assertTrue (out.position () > 0);
    decodeCount += out.position ();
    out.flip ();
    sb.append (out.toString ());
    out.clear ();
  }
  assertEquals (decoded, sb.toString ());
  in.rewind ();
}
 
Example 2
Source File: CsvReporterTest.java    From metrics with Apache License 2.0 6 votes vote down vote up
private String fileContents(String filename) throws IOException {
    final StringBuilder builder = new StringBuilder();
    final FileInputStream input = new FileInputStream(new File(dataDirectory, filename));
    try {
        final InputStreamReader reader = new InputStreamReader(input);
        final BufferedReader bufferedReader = new BufferedReader(reader);
        final CharBuffer buf = CharBuffer.allocate(1024);
        while (bufferedReader.read(buf) != -1) {
            buf.flip();
            builder.append(buf);
            buf.clear();
        }
    } finally {
        input.close();
    }
    return builder.toString();
}
 
Example 3
Source File: CharStreams.java    From codebuff with BSD 2-Clause "Simplified" License 6 votes vote down vote up
/**
 * Copies all characters between the {@link Readable} and {@link Appendable} objects. Does not
 * close or flush either object.
 *
 * @param from the object to read from
 * @param to the object to write to
 * @return the number of characters copied
 * @throws IOException if an I/O error occurs
 */

@CanIgnoreReturnValue
public static long copy(Readable from, Appendable to) throws IOException {
  checkNotNull(from);
  checkNotNull(to);
  CharBuffer buf = createBuffer();
  long total = 0;
  while (from.read(buf) != -1) {
    buf.flip();
    to.append(buf);
    total += buf.remaining();
    buf.clear();
  }
  return total;
}
 
Example 4
Source File: BufferFormatterTest.java    From gflogger with Apache License 2.0 6 votes vote down vote up
@Test
public void testAppendInt() throws Exception {
	final CharBuffer buffer = ByteBuffer.allocateDirect(30).asCharBuffer();

	for (int i = Short.MIN_VALUE - 100; i < Short.MAX_VALUE + 100; i++) {
		BufferFormatter.append(buffer, i);
		assertEquals(Integer.toString(i), toString(buffer));
		buffer.clear();
	}

	final int[] numbers = new int[]{9123123, Integer.MAX_VALUE, Integer.MIN_VALUE};
	for (int i = 0; i < numbers.length; i++) {
		BufferFormatter.append(buffer, numbers[i]);
		buffer.append(' ');
		assertEquals(Integer.toString(numbers[i]) + " ", toString(buffer));
		// check
		buffer.clear();
	}
}
 
Example 5
Source File: StreamEvent.java    From staedi with Apache License 2.0 6 votes vote down vote up
static CharBuffer put(CharBuffer buffer, CharArraySequence data) {
    final int length = data.length();

    if (buffer == null || buffer.capacity() < length) {
        buffer = CharBuffer.allocate(length);
    }

    buffer.clear();

    if (length > 0) {
        data.putToBuffer(buffer);
    }

    buffer.flip();

    return buffer;
}
 
Example 6
Source File: CharStreams.java    From codebuff with BSD 2-Clause "Simplified" License 6 votes vote down vote up
/**
 * Copies all characters between the {@link Readable} and {@link Appendable} objects. Does not
 * close or flush either object.
 *
 * @param from the object to read from
 * @param to the object to write to
 * @return the number of characters copied
 * @throws IOException if an I/O error occurs
 */

@CanIgnoreReturnValue
public static long copy(Readable from, Appendable to) throws IOException {
  checkNotNull(from);
  checkNotNull(to);
  CharBuffer buf = createBuffer();
  long total = 0;
  while (from.read(buf) != -1) {
    buf.flip();
    to.append(buf);
    total += buf.remaining();
    buf.clear();
  }
  return total;
}
 
Example 7
Source File: StockName.java    From dragonwell8_jdk with GNU General Public License v2.0 5 votes vote down vote up
private static void test(CharBuffer cb, String exp) {
    cb.limit(cb.position());
    cb.rewind();
    if (!cb.toString().equals(exp))
        throw new RuntimeException("expect: '" + exp + "'; got: '"
                                   + cb.toString() + "'");
    cb.clear();
}
 
Example 8
Source File: BinaryPrimitivesTest.java    From openjdk-jdk9 with GNU General Public License v2.0 5 votes vote down vote up
@Test
public void stringIdentity() {
    final int MAX_STRING_LENGTH = 4096;
    ByteBuffer bytes = ByteBuffer.allocate(MAX_STRING_LENGTH + 6); // it takes 6 bytes to encode string length of Integer.MAX_VALUE
    CharBuffer chars = CharBuffer.allocate(MAX_STRING_LENGTH);
    StringReader reader = new StringReader();
    StringWriter writer = new StringWriter();
    for (int len = 0; len <= MAX_STRING_LENGTH; len++) {
        for (int i = 0; i < 64; i++) {
            // not so much "test in isolation", I know... we're testing .reset() as well
            bytes.clear();
            chars.clear();

            byte[] b = new byte[len];
            rnd.nextBytes(b);

            String expected = new String(b, StandardCharsets.ISO_8859_1); // reference string

            boolean written = writer
                    .configure(CharBuffer.wrap(expected), 0, expected.length(), false)
                    .write(bytes);

            if (!written) {
                fail("please increase 'bytes' size");
            }
            bytes.flip();
            reader.read(bytes, chars);
            chars.flip();
            assertEquals(chars.toString(), expected);
            reader.reset();
            writer.reset();
        }
    }
}
 
Example 9
Source File: AbstractCommand.java    From gemfirexd-oss with Apache License 2.0 5 votes vote down vote up
protected CharBuffer getFirstLineBuffer() {
  CharBuffer buffer = firstLineBuffer.get();
  if (buffer == null) {
    buffer = CharBuffer.allocate(256);
    firstLineBuffer.set(buffer);
  }
  buffer.clear();
  return buffer;
}
 
Example 10
Source File: UriConnectorResolver.java    From gridgo with MIT License 5 votes vote down vote up
private String extractPlaceholderValue(String schemePart, int i, CharBuffer buffer) {
    buffer.clear();
    char c;

    if (i >= schemePart.length())
        return "";

    boolean insideBracket = schemePart.charAt(i) == '[';
    if (insideBracket) {
        buffer.put('[');
        i++;
    }
    while (i < schemePart.length() && isPlaceholder(c = schemePart.charAt(i), insideBracket)) {
        buffer.put(c);
        i++;
    }
    if (insideBracket) {
        if (schemePart.charAt(i) != ']') {
            throw new MalformedEndpointException(
                    String.format("Malformed endpoint, invalid token at %d, expected ']', actualy '%c': %s", i,
                            schemePart.charAt(i), schemePart));
        }
        buffer.put(']');
    }

    buffer.flip();
    return buffer.toString();
}
 
Example 11
Source File: CharStreams.java    From codebuff with BSD 2-Clause "Simplified" License 5 votes vote down vote up
/**
 * Reads and discards data from the given {@code Readable} until the end of the stream is
 * reached. Returns the total number of chars read. Does not close the stream.
 *
 * @since 20.0
 */

@CanIgnoreReturnValue
public static long exhaust(Readable readable) throws IOException {
  long total = 0;
  long read;
  CharBuffer buf = createBuffer();
  while ((read = readable.read(buf)) != -1) {
    total += read;
    buf.clear();
  }
  return total;
}
 
Example 12
Source File: StockName.java    From TencentKona-8 with GNU General Public License v2.0 5 votes vote down vote up
private static void test(CharBuffer cb, String exp) {
    cb.limit(cb.position());
    cb.rewind();
    if (!cb.toString().equals(exp))
        throw new RuntimeException("expect: '" + exp + "'; got: '"
                                   + cb.toString() + "'");
    cb.clear();
}
 
Example 13
Source File: JISAutoDetect.java    From openjdk-jdk9 with GNU General Public License v2.0 4 votes vote down vote up
protected CoderResult decodeLoop(ByteBuffer src, CharBuffer dst) {
    if (detectedDecoder == null) {
        copyLeadingASCII(src, dst);

        // All ASCII?
        if (! src.hasRemaining())
            return CoderResult.UNDERFLOW;
        // Overflow only if there is still ascii but no out buffer.
        if (!dst.hasRemaining() &&
            isPlainASCII(src.get(src.position())))
            return CoderResult.OVERFLOW;

        // We need to perform double, not float, arithmetic; otherwise
        // we lose low order bits when src is larger than 2**24.
        int cbufsiz = (int)(src.limit() * (double)maxCharsPerByte());
        CharBuffer sandbox = CharBuffer.allocate(cbufsiz);

        // First try ISO-2022-JP, since there is no ambiguity
        Charset cs2022 = Charset.forName("ISO-2022-JP");
        DelegatableDecoder dd2022
            = (DelegatableDecoder) cs2022.newDecoder();
        ByteBuffer src2022 = src.asReadOnlyBuffer();
        CoderResult res2022 = dd2022.decodeLoop(src2022, sandbox);
        if (! res2022.isError())
            return decodeLoop(dd2022, src, dst);

        // We must choose between EUC and SJIS
        Charset csEUCJ = Charset.forName(EUCJPName);
        Charset csSJIS = Charset.forName(SJISName);

        DelegatableDecoder ddEUCJ
            = (DelegatableDecoder) csEUCJ.newDecoder();
        DelegatableDecoder ddSJIS
            = (DelegatableDecoder) csSJIS.newDecoder();

        ByteBuffer srcEUCJ = src.asReadOnlyBuffer();
        sandbox.clear();
        CoderResult resEUCJ = ddEUCJ.decodeLoop(srcEUCJ, sandbox);
        // If EUC decoding fails, must be SJIS
        if (resEUCJ.isError())
            return decodeLoop(ddSJIS, src, dst);
        ByteBuffer srcSJIS = src.asReadOnlyBuffer();
        CharBuffer sandboxSJIS = CharBuffer.allocate(cbufsiz);
        CoderResult resSJIS = ddSJIS.decodeLoop(srcSJIS, sandboxSJIS);
        // If SJIS decoding fails, must be EUC
        if (resSJIS.isError())
            return decodeLoop(ddEUCJ, src, dst);

        // From here on, we have some ambiguity, and must guess.

        // We prefer input that does not appear to end mid-character.
        if (srcEUCJ.position() > srcSJIS.position())
            return decodeLoop(ddEUCJ, src, dst);

        if (srcEUCJ.position() < srcSJIS.position())
            return decodeLoop(ddSJIS, src, dst);

        // end-of-input is after the first byte of the first char?
        if (src.position() == srcEUCJ.position())
            return CoderResult.UNDERFLOW;

        // Use heuristic knowledge of typical Japanese text
        sandbox.flip();
        return decodeLoop(looksLikeJapanese(sandbox) ? ddEUCJ : ddSJIS,
                          src, dst);
    }

    return detectedDecoder.decodeLoop(src, dst);
}
 
Example 14
Source File: XProtocolDecoder.java    From lams with GNU General Public License v2.0 4 votes vote down vote up
@Override
public <T> T decodeDecimal(byte[] bytes, int offset, int length, ValueFactory<T> vf) {
    try {
        CodedInputStream inputStream = CodedInputStream.newInstance(bytes, offset, length);
        // packed BCD format (c.f. wikipedia)
        // TODO: optimization possibilities include using int/long if the digits is < X and scale = 0
        byte scale = inputStream.readRawByte();
        // we allocate an extra char for the sign
        CharBuffer unscaledString = CharBuffer.allocate(2 * inputStream.getBytesUntilLimit());
        unscaledString.position(1);
        byte sign = 0;
        // read until we encounter the sign bit
        while (true) {
            int b = 0xFF & inputStream.readRawByte();
            if ((b >> 4) > 9) {
                sign = (byte) (b >> 4);
                break;
            }
            unscaledString.append((char) ((b >> 4) + '0'));
            if ((b & 0x0f) > 9) {
                sign = (byte) (b & 0x0f);
                break;
            }
            unscaledString.append((char) ((b & 0x0f) + '0'));
        }
        if (inputStream.getBytesUntilLimit() > 0) {
            throw AssertionFailedException
                    .shouldNotHappen("Did not read all bytes while decoding decimal. Bytes left: " + inputStream.getBytesUntilLimit());
        }
        switch (sign) {
            case 0xa:
            case 0xc:
            case 0xe:
            case 0xf:
                unscaledString.put(0, '+');
                break;
            case 0xb:
            case 0xd:
                unscaledString.put(0, '-');
                break;
        }
        // may have filled the CharBuffer or one remaining. need to remove it before toString()
        int characters = unscaledString.position();
        unscaledString.clear(); // reset position
        BigInteger unscaled = new BigInteger(unscaledString.subSequence(0, characters).toString());
        return vf.createFromBigDecimal(new BigDecimal(unscaled, scale));
    } catch (IOException e) {
        throw new DataReadException(e);
    }
}
 
Example 15
Source File: JISAutoDetect.java    From jdk8u_jdk with GNU General Public License v2.0 4 votes vote down vote up
protected CoderResult decodeLoop(ByteBuffer src, CharBuffer dst) {
    if (detectedDecoder == null) {
        copyLeadingASCII(src, dst);

        // All ASCII?
        if (! src.hasRemaining())
            return CoderResult.UNDERFLOW;
        if (! dst.hasRemaining())
            return CoderResult.OVERFLOW;

        // We need to perform double, not float, arithmetic; otherwise
        // we lose low order bits when src is larger than 2**24.
        int cbufsiz = (int)(src.limit() * (double)maxCharsPerByte());
        CharBuffer sandbox = CharBuffer.allocate(cbufsiz);

        // First try ISO-2022-JP, since there is no ambiguity
        Charset cs2022 = Charset.forName("ISO-2022-JP");
        DelegatableDecoder dd2022
            = (DelegatableDecoder) cs2022.newDecoder();
        ByteBuffer src2022 = src.asReadOnlyBuffer();
        CoderResult res2022 = dd2022.decodeLoop(src2022, sandbox);
        if (! res2022.isError())
            return decodeLoop(cs2022, src, dst);

        // We must choose between EUC and SJIS
        Charset csEUCJ = Charset.forName(EUCJPName);
        Charset csSJIS = Charset.forName(SJISName);

        DelegatableDecoder ddEUCJ
            = (DelegatableDecoder) csEUCJ.newDecoder();
        ByteBuffer srcEUCJ = src.asReadOnlyBuffer();
        sandbox.clear();
        CoderResult resEUCJ = ddEUCJ.decodeLoop(srcEUCJ, sandbox);
        // If EUC decoding fails, must be SJIS
        if (resEUCJ.isError())
            return decodeLoop(csSJIS, src, dst);

        DelegatableDecoder ddSJIS
            = (DelegatableDecoder) csSJIS.newDecoder();
        ByteBuffer srcSJIS = src.asReadOnlyBuffer();
        CharBuffer sandboxSJIS = CharBuffer.allocate(cbufsiz);
        CoderResult resSJIS = ddSJIS.decodeLoop(srcSJIS, sandboxSJIS);
        // If SJIS decoding fails, must be EUC
        if (resSJIS.isError())
            return decodeLoop(csEUCJ, src, dst);

        // From here on, we have some ambiguity, and must guess.

        // We prefer input that does not appear to end mid-character.
        if (srcEUCJ.position() > srcSJIS.position())
            return decodeLoop(csEUCJ, src, dst);

        if (srcEUCJ.position() < srcSJIS.position())
            return decodeLoop(csSJIS, src, dst);

        // end-of-input is after the first byte of the first char?
        if (src.position() == srcEUCJ.position())
            return CoderResult.UNDERFLOW;

        // Use heuristic knowledge of typical Japanese text
        sandbox.flip();
        Charset guess = looksLikeJapanese(sandbox) ? csEUCJ : csSJIS;
        return decodeLoop(guess, src, dst);
    }

    return detectedDecoder.decodeLoop(src, dst);
}
 
Example 16
Source File: FastMatcher.java    From netbeans with Apache License 2.0 4 votes vote down vote up
/**
 * Compute lenght of this sequence - quite expensive operation, indeed.
 */
@Override
public int length() {
    if (length != -1) {
        return length;
    }
    long start = System.currentTimeMillis();
    int charactersRead = 0;
    long bytesRead = 0;
    MappedByteBuffer mappedByteBuffer = null;
    CharBuffer charBuffer = CharBuffer.allocate(SIZE_LIMIT);
    CharsetDecoder decoder = prepareDecoder(charset);
    decoder.onUnmappableCharacter(CodingErrorAction.IGNORE);

    try {
        while (bytesRead < fileSize) {
            mappedByteBuffer = fileChannel.map(
                    FileChannel.MapMode.READ_ONLY, bytesRead,
                    Math.min(SIZE_LIMIT, fileSize - bytesRead));
            CoderResult result;
            do {
                charBuffer.clear();
                result = decoder.decode(
                        mappedByteBuffer, charBuffer,
                        bytesRead + SIZE_LIMIT >= fileSize);
                if (result.isUnmappable() || result.isMalformed()
                        || result.isError()) {
                    throw new IOException("Error decoding file: "
                            + result.toString() + " ");
                }
                if (bytesRead + SIZE_LIMIT >= fileSize) {
                    LOG.info("Coding end");
                }
                charactersRead += charBuffer.position();
            } while (result.isOverflow());

            int readNow = mappedByteBuffer.position();
            bytesRead += readNow;
            unmap(mappedByteBuffer);
        }
        charBuffer.clear();
        boolean repeat;
        do {
            repeat = decoder.flush(charBuffer).isOverflow();
            charactersRead += charBuffer.position();
            charBuffer.clear();
        } while (repeat);
    } catch (IOException ex) {
        if (mappedByteBuffer != null) {
            unmap(mappedByteBuffer);
        }
        Exceptions.printStackTrace(ex);
    }
    length = charactersRead;
    LOG.log(Level.INFO, "Length computed in {0} ms.", //NOI18N
            System.currentTimeMillis() - start);
    return length;
}
 
Example 17
Source File: JISAutoDetect.java    From openjdk-jdk8u with GNU General Public License v2.0 4 votes vote down vote up
protected CoderResult decodeLoop(ByteBuffer src, CharBuffer dst) {
    if (detectedDecoder == null) {
        copyLeadingASCII(src, dst);

        // All ASCII?
        if (! src.hasRemaining())
            return CoderResult.UNDERFLOW;
        if (! dst.hasRemaining())
            return CoderResult.OVERFLOW;

        // We need to perform double, not float, arithmetic; otherwise
        // we lose low order bits when src is larger than 2**24.
        int cbufsiz = (int)(src.limit() * (double)maxCharsPerByte());
        CharBuffer sandbox = CharBuffer.allocate(cbufsiz);

        // First try ISO-2022-JP, since there is no ambiguity
        Charset cs2022 = Charset.forName("ISO-2022-JP");
        DelegatableDecoder dd2022
            = (DelegatableDecoder) cs2022.newDecoder();
        ByteBuffer src2022 = src.asReadOnlyBuffer();
        CoderResult res2022 = dd2022.decodeLoop(src2022, sandbox);
        if (! res2022.isError())
            return decodeLoop(cs2022, src, dst);

        // We must choose between EUC and SJIS
        Charset csEUCJ = Charset.forName(EUCJPName);
        Charset csSJIS = Charset.forName(SJISName);

        DelegatableDecoder ddEUCJ
            = (DelegatableDecoder) csEUCJ.newDecoder();
        ByteBuffer srcEUCJ = src.asReadOnlyBuffer();
        sandbox.clear();
        CoderResult resEUCJ = ddEUCJ.decodeLoop(srcEUCJ, sandbox);
        // If EUC decoding fails, must be SJIS
        if (resEUCJ.isError())
            return decodeLoop(csSJIS, src, dst);

        DelegatableDecoder ddSJIS
            = (DelegatableDecoder) csSJIS.newDecoder();
        ByteBuffer srcSJIS = src.asReadOnlyBuffer();
        CharBuffer sandboxSJIS = CharBuffer.allocate(cbufsiz);
        CoderResult resSJIS = ddSJIS.decodeLoop(srcSJIS, sandboxSJIS);
        // If SJIS decoding fails, must be EUC
        if (resSJIS.isError())
            return decodeLoop(csEUCJ, src, dst);

        // From here on, we have some ambiguity, and must guess.

        // We prefer input that does not appear to end mid-character.
        if (srcEUCJ.position() > srcSJIS.position())
            return decodeLoop(csEUCJ, src, dst);

        if (srcEUCJ.position() < srcSJIS.position())
            return decodeLoop(csSJIS, src, dst);

        // end-of-input is after the first byte of the first char?
        if (src.position() == srcEUCJ.position())
            return CoderResult.UNDERFLOW;

        // Use heuristic knowledge of typical Japanese text
        sandbox.flip();
        Charset guess = looksLikeJapanese(sandbox) ? csEUCJ : csSJIS;
        return decodeLoop(guess, src, dst);
    }

    return detectedDecoder.decodeLoop(src, dst);
}
 
Example 18
Source File: JISAutoDetect.java    From openjdk-8 with GNU General Public License v2.0 4 votes vote down vote up
protected CoderResult decodeLoop(ByteBuffer src, CharBuffer dst) {
    if (detectedDecoder == null) {
        copyLeadingASCII(src, dst);

        // All ASCII?
        if (! src.hasRemaining())
            return CoderResult.UNDERFLOW;
        if (! dst.hasRemaining())
            return CoderResult.OVERFLOW;

        // We need to perform double, not float, arithmetic; otherwise
        // we lose low order bits when src is larger than 2**24.
        int cbufsiz = (int)(src.limit() * (double)maxCharsPerByte());
        CharBuffer sandbox = CharBuffer.allocate(cbufsiz);

        // First try ISO-2022-JP, since there is no ambiguity
        Charset cs2022 = Charset.forName("ISO-2022-JP");
        DelegatableDecoder dd2022
            = (DelegatableDecoder) cs2022.newDecoder();
        ByteBuffer src2022 = src.asReadOnlyBuffer();
        CoderResult res2022 = dd2022.decodeLoop(src2022, sandbox);
        if (! res2022.isError())
            return decodeLoop(cs2022, src, dst);

        // We must choose between EUC and SJIS
        Charset csEUCJ = Charset.forName(EUCJPName);
        Charset csSJIS = Charset.forName(SJISName);

        DelegatableDecoder ddEUCJ
            = (DelegatableDecoder) csEUCJ.newDecoder();
        ByteBuffer srcEUCJ = src.asReadOnlyBuffer();
        sandbox.clear();
        CoderResult resEUCJ = ddEUCJ.decodeLoop(srcEUCJ, sandbox);
        // If EUC decoding fails, must be SJIS
        if (resEUCJ.isError())
            return decodeLoop(csSJIS, src, dst);

        DelegatableDecoder ddSJIS
            = (DelegatableDecoder) csSJIS.newDecoder();
        ByteBuffer srcSJIS = src.asReadOnlyBuffer();
        CharBuffer sandboxSJIS = CharBuffer.allocate(cbufsiz);
        CoderResult resSJIS = ddSJIS.decodeLoop(srcSJIS, sandboxSJIS);
        // If SJIS decoding fails, must be EUC
        if (resSJIS.isError())
            return decodeLoop(csEUCJ, src, dst);

        // From here on, we have some ambiguity, and must guess.

        // We prefer input that does not appear to end mid-character.
        if (srcEUCJ.position() > srcSJIS.position())
            return decodeLoop(csEUCJ, src, dst);

        if (srcEUCJ.position() < srcSJIS.position())
            return decodeLoop(csSJIS, src, dst);

        // end-of-input is after the first byte of the first char?
        if (src.position() == srcEUCJ.position())
            return CoderResult.UNDERFLOW;

        // Use heuristic knowledge of typical Japanese text
        sandbox.flip();
        Charset guess = looksLikeJapanese(sandbox) ? csEUCJ : csSJIS;
        return decodeLoop(guess, src, dst);
    }

    return detectedDecoder.decodeLoop(src, dst);
}
 
Example 19
Source File: JISAutoDetect.java    From openjdk-jdk8u-backup with GNU General Public License v2.0 4 votes vote down vote up
protected CoderResult decodeLoop(ByteBuffer src, CharBuffer dst) {
    if (detectedDecoder == null) {
        copyLeadingASCII(src, dst);

        // All ASCII?
        if (! src.hasRemaining())
            return CoderResult.UNDERFLOW;
        if (! dst.hasRemaining())
            return CoderResult.OVERFLOW;

        // We need to perform double, not float, arithmetic; otherwise
        // we lose low order bits when src is larger than 2**24.
        int cbufsiz = (int)(src.limit() * (double)maxCharsPerByte());
        CharBuffer sandbox = CharBuffer.allocate(cbufsiz);

        // First try ISO-2022-JP, since there is no ambiguity
        Charset cs2022 = Charset.forName("ISO-2022-JP");
        DelegatableDecoder dd2022
            = (DelegatableDecoder) cs2022.newDecoder();
        ByteBuffer src2022 = src.asReadOnlyBuffer();
        CoderResult res2022 = dd2022.decodeLoop(src2022, sandbox);
        if (! res2022.isError())
            return decodeLoop(cs2022, src, dst);

        // We must choose between EUC and SJIS
        Charset csEUCJ = Charset.forName(EUCJPName);
        Charset csSJIS = Charset.forName(SJISName);

        DelegatableDecoder ddEUCJ
            = (DelegatableDecoder) csEUCJ.newDecoder();
        ByteBuffer srcEUCJ = src.asReadOnlyBuffer();
        sandbox.clear();
        CoderResult resEUCJ = ddEUCJ.decodeLoop(srcEUCJ, sandbox);
        // If EUC decoding fails, must be SJIS
        if (resEUCJ.isError())
            return decodeLoop(csSJIS, src, dst);

        DelegatableDecoder ddSJIS
            = (DelegatableDecoder) csSJIS.newDecoder();
        ByteBuffer srcSJIS = src.asReadOnlyBuffer();
        CharBuffer sandboxSJIS = CharBuffer.allocate(cbufsiz);
        CoderResult resSJIS = ddSJIS.decodeLoop(srcSJIS, sandboxSJIS);
        // If SJIS decoding fails, must be EUC
        if (resSJIS.isError())
            return decodeLoop(csEUCJ, src, dst);

        // From here on, we have some ambiguity, and must guess.

        // We prefer input that does not appear to end mid-character.
        if (srcEUCJ.position() > srcSJIS.position())
            return decodeLoop(csEUCJ, src, dst);

        if (srcEUCJ.position() < srcSJIS.position())
            return decodeLoop(csSJIS, src, dst);

        // end-of-input is after the first byte of the first char?
        if (src.position() == srcEUCJ.position())
            return CoderResult.UNDERFLOW;

        // Use heuristic knowledge of typical Japanese text
        sandbox.flip();
        Charset guess = looksLikeJapanese(sandbox) ? csEUCJ : csSJIS;
        return decodeLoop(guess, src, dst);
    }

    return detectedDecoder.decodeLoop(src, dst);
}
 
Example 20
Source File: BufferUtils.java    From morfologik-stemming with BSD 3-Clause "New" or "Revised" License 3 votes vote down vote up
/**
 * Ensure the buffer's capacity is large enough to hold a given number
 * of elements. If the input buffer is not large enough, a new buffer is allocated
 * and returned.
 * 
 * @param elements The required number of elements to be appended to the buffer.
 * 
 * @param buffer
 *          The buffer to check or <code>null</code> if a new buffer should be
 *          allocated.
 *
 * @return Returns the same buffer or a new buffer with the given capacity. 
 */
public static CharBuffer clearAndEnsureCapacity(CharBuffer buffer, int elements) {
  if (buffer == null || buffer.capacity() < elements) {
    buffer = CharBuffer.allocate(elements);
  } else {
    buffer.clear();
  }
  return buffer;
}