Java Code Examples for io.netty.buffer.ByteBuf#retainedSlice()

The following examples show how to use io.netty.buffer.ByteBuf#retainedSlice() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: Utils.java    From ambry with Apache License 2.0 6 votes vote down vote up
/**
 * Transfer {@code dataSize} bytes of data from the given crc stream to a newly create {@link ByteBuf}. The method
 * would also update the crc value in the crc stream.
 * @param crcStream The crc stream.
 * @param dataSize The number of bytes to transfer.
 * @return the newly created {@link ByteBuf} which contains the transferred data.
 * @throws IOException Any I/O error.
 */
public static ByteBuf readNettyByteBufFromCrcInputStream(CrcInputStream crcStream, int dataSize) throws IOException {
  ByteBuf output;
  InputStream inputStream = crcStream.getUnderlyingInputStream();
  if (inputStream instanceof NettyByteBufDataInputStream) {
    ByteBuf nettyByteBuf = ((NettyByteBufDataInputStream) inputStream).getBuffer();
    // construct a java.nio.ByteBuffer to create a ByteBufferInputStream
    int startIndex = nettyByteBuf.readerIndex();
    output = nettyByteBuf.retainedSlice(startIndex, dataSize);
    crcStream.updateCrc(output.nioBuffer());
    nettyByteBuf.readerIndex(startIndex + dataSize);
  } else {
    ByteBuffer buffer = getByteBufferFromInputStream(crcStream, dataSize);
    output = Unpooled.wrappedBuffer(buffer);
  }
  return output;
}
 
Example 2
Source File: FtdcProtocol.java    From ftdc with Apache License 2.0 6 votes vote down vote up
private void extractBodyAndExt(int extLength, int bodyLength, ByteBuf buf, boolean compress) {
	try{
		if(hasExt()) {
			extBuf = buf.retainedSlice(buf.readerIndex(), extLength);
			buf.skipBytes(extLength);
			readExt();
		}
		if(hasBody()) {
			bodyBuf = buf.retainedSlice(buf.readerIndex(), bodyLength);
			buf.skipBytes(bodyLength);
			if(compress) {
				readFtdc();
			}else {
				readFtdcUnUnCompress();
			}
		}
	}finally {
		ReferenceCountUtil.release(buf);
	}
}
 
Example 3
Source File: FtdcProtocol.java    From ftdc with Apache License 2.0 6 votes vote down vote up
public Ftdc(ByteBuf body) {
	this.version = body.readByte();
	this.type = body.readByte();
	this.unencodeLength = body.readByte();
	this.chain = body.readByte();
	this.sequenceType = body.readShort();
	this.sequence = body.readInt();
	this.currentSequence = body.readInt();
	this.numData = body.readShort();
	this.ftdcLen = body.readUnsignedShort();
	this.reqId = body.readUnsignedInt();
	//没有数据的情况是不存在tid的
	if(ftdcLen != 0) {
		this.tid = body.readUnsignedInt();
		int actualLen = this.ftdcLen - 4;
		if(body.readableBytes() < actualLen) {
			throw new IllegalArgumentException("bad body length");
		}
		ftdcBody = body.retainedSlice(body.readerIndex(), actualLen);
	}
	
}
 
Example 4
Source File: LengthLimitingContentPreviewer.java    From armeria with Apache License 2.0 5 votes vote down vote up
private static ByteBuf duplicateData(HttpData httpData, int length) {
    if (httpData instanceof ByteBufHolder) {
        final ByteBuf content = ((ByteBufHolder) httpData).content();
        if (content.readableBytes() == length) {
            return content.retainedDuplicate();
        }
        return content.retainedSlice(content.readerIndex(), length);
    } else {
        return Unpooled.wrappedBuffer(httpData.array(), 0, length);
    }
}
 
Example 5
Source File: Http1ObjectEncoder.java    From armeria with Apache License 2.0 5 votes vote down vote up
private static ByteBuf dataChunk(HttpData data, int offset, int chunkSize) {
    if (data instanceof ByteBufHolder) {
        final ByteBuf buf = ((ByteBufHolder) data).content();
        return buf.retainedSlice(offset, chunkSize);
    } else {
        return Unpooled.wrappedBuffer(data.array(), offset, chunkSize);
    }
}
 
Example 6
Source File: InternalKey.java    From Distributed-KV with Apache License 2.0 5 votes vote down vote up
/**
 * 反序列化得到internal key对象,字节数据包含大小信息
 * @param byteBuf byteBuf 字节数据
 * @return internal key对象
 */
@Deprecated
public static InternalKey decodeWithPrefix(ByteBuf bytebuf) {
	Preconditions.checkNotNull(bytebuf);
	bytebuf = bytebuf.slice();
	// bytebuf前 8B 即为info信息
	long info = bytebuf.readLong();
	// TODO 将读取的info解析成state和seq信息
	int state = (int) (info & 0xff);
	long seq = info >> 8;
	// bytebuf剩余部分即user key
	ByteBuf userKey = bytebuf.retainedSlice();
	Preconditions.checkArgument(state == 0 || state == 1);
	return new InternalKey(userKey, seq, InternalKeyType.getType(state));
}
 
Example 7
Source File: FecPacket.java    From java-Kcp with Apache License 2.0 5 votes vote down vote up
public static FecPacket newFecPacket(ByteBuf byteBuf){
    FecPacket pkt = FEC_PACKET_RECYCLER.get();
    pkt.seqid =byteBuf.readUnsignedIntLE();
    pkt.flag = byteBuf.readUnsignedShortLE();
    pkt.data = byteBuf.retainedSlice(byteBuf.readerIndex(),byteBuf.capacity()-byteBuf.readerIndex());
    pkt.data.writerIndex(byteBuf.readableBytes());
    return pkt;
}
 
Example 8
Source File: DotStuffing.java    From NioSmtpClient with Apache License 2.0 5 votes vote down vote up
/**
 * Returns a {@link CompositeByteBuf} that contains the same data as {@code sourceBuffer}, but with
 * SMTP dot-stuffing applied, and (if {@code} appendCRLF is true) a CRLF appended.
 *
 * <p>If dot-stuffing is not required, and {@code appendCRLF} is false, {@code sourceBuffer} is
 * returned. In all other cases, {@code allocator} will be used to create a new {@code ByteBuf}
 * with a {@code refCnt} of one.
 *
 * <p>The {@code previousBytes} parameter is used to maintain dot-stuffing across a series
 * of buffers. Pass the last two bytes of a previous buffer here to ensure an initial dot
 * will be escaped if necessary. Passing null indicates this is the first or only buffer
 * for this message.
 *
 * @param allocator the {@code ByteBufAllocator} to use for new {@code ByteBuf}s
 * @param sourceBuffer the source message data
 * @param previousBytes the previous two bytes of the message, or null
 * @param termination whether to append CRLF to the end of the returned buffer
 */
public static ByteBuf createDotStuffedBuffer(ByteBufAllocator allocator, ByteBuf sourceBuffer, byte[] previousBytes, MessageTermination termination) {
  int dotIndex = findDotAtBeginningOfLine(sourceBuffer, 0, normalisePreviousBytes(previousBytes));

  try {
    if (dotIndex == -1) {
      if (termination == MessageTermination.ADD_CRLF) {
        return allocator.compositeBuffer(2).addComponents(true, sourceBuffer.retainedSlice(), CR_LF_BUFFER.slice());
      } else {
        return sourceBuffer.retainedSlice();
      }
    }

    // Build a CompositeByteBuf to avoid copying
    CompositeByteBuf compositeByteBuf = allocator.compositeBuffer();
    compositeByteBuf.addComponents(true, sourceBuffer.retainedSlice(0, dotIndex), DOT_DOT_BUFFER.slice());

    int nextDotIndex;
    while ((nextDotIndex = findDotAtBeginningOfLine(sourceBuffer, dotIndex + 1, NOT_CR_LF)) != -1) {
      compositeByteBuf.addComponents(true, sourceBuffer.retainedSlice(dotIndex + 1, nextDotIndex - dotIndex - 1), DOT_DOT_BUFFER.slice());
      dotIndex = nextDotIndex;
    }

    compositeByteBuf.addComponent(true, sourceBuffer.retainedSlice(dotIndex + 1, sourceBuffer.readableBytes() - dotIndex - 1));

    if (termination == MessageTermination.ADD_CRLF) {
      compositeByteBuf.addComponent(true, CR_LF_BUFFER.slice());
    }

    return compositeByteBuf;
  } finally {
    sourceBuffer.release();
  }
}
 
Example 9
Source File: ThriftFramedDecoder.java    From drift with Apache License 2.0 4 votes vote down vote up
private Optional<ByteBuf> decode(ByteBuf buffer)
{
    if (bytesToDiscard > 0) {
        discardTooLongFrame(buffer);
        return Optional.empty();
    }

    int initialReaderIndex = buffer.readerIndex();

    if (buffer.readableBytes() < Integer.BYTES) {
        return Optional.empty();
    }
    long frameSizeInBytes = buffer.readUnsignedInt();

    if (frameSizeInBytes > maxFrameSizeInBytes) {
        // this invocation doesn't move the readerIndex
        Optional<FrameInfo> frameInfo = frameInfoDecoder.tryDecodeFrameInfo(buffer);
        if (frameInfo.isPresent()) {
            tooLongFrameInfo = frameInfo;
            tooLongFrameSizeInBytes = frameSizeInBytes;
            bytesToDiscard = frameSizeInBytes;
            discardTooLongFrame(buffer);
            return Optional.empty();
        }
        // Basic frame info cannot be decoded and the max frame size is already exceeded.
        // Instead of waiting forever, fail without providing the sequence ID.
        if (buffer.readableBytes() >= maxFrameSizeInBytes) {
            tooLongFrameInfo = Optional.empty();
            tooLongFrameSizeInBytes = frameSizeInBytes;
            bytesToDiscard = frameSizeInBytes;
            discardTooLongFrame(buffer);
            return Optional.empty();
        }
        buffer.readerIndex(initialReaderIndex);
        return Optional.empty();
    }

    if (buffer.readableBytes() >= frameSizeInBytes) {
        // toIntExact must be safe, as frameSizeInBytes <= maxFrameSize
        ByteBuf frame = buffer.retainedSlice(buffer.readerIndex(), toIntExact(frameSizeInBytes));
        buffer.readerIndex(buffer.readerIndex() + toIntExact(frameSizeInBytes));
        return Optional.of(frame);
    }

    buffer.readerIndex(initialReaderIndex);
    return Optional.empty();
}
 
Example 10
Source File: CompositeMetadataCodec.java    From rsocket-java with Apache License 2.0 4 votes vote down vote up
/**
 * Decode the next metadata entry (a mime header + content pair of {@link ByteBuf}) from a {@link
 * ByteBuf} that contains at least enough bytes for one more such entry. These buffers are
 * actually slices of the full metadata buffer, and this method doesn't move the full metadata
 * buffer's {@link ByteBuf#readerIndex()}. As such, it requires the user to provide an {@code
 * index} to read from. The next index is computed by calling {@link #computeNextEntryIndex(int,
 * ByteBuf, ByteBuf)}. Size of the first buffer (the "header buffer") drives which decoding method
 * should be further applied to it.
 *
 * <p>The header buffer is either:
 *
 * <ul>
 *   <li>made up of a single byte: this represents an encoded mime id, which can be further
 *       decoded using {@link #decodeMimeIdFromMimeBuffer(ByteBuf)}
 *   <li>made up of 2 or more bytes: this represents an encoded mime String + its length, which
 *       can be further decoded using {@link #decodeMimeTypeFromMimeBuffer(ByteBuf)}. Note the
 *       encoded length, in the first byte, is skipped by this decoding method because the
 *       remaining length of the buffer is that of the mime string.
 * </ul>
 *
 * @param compositeMetadata the source {@link ByteBuf} that originally contains one or more
 *     metadata entries
 * @param entryIndex the {@link ByteBuf#readerIndex()} to start decoding from. original reader
 *     index is kept on the source buffer
 * @param retainSlices should produced metadata entry buffers {@link ByteBuf#slice() slices} be
 *     {@link ByteBuf#retainedSlice() retained}?
 * @return a {@link ByteBuf} array of length 2 containing the mime header buffer
 *     <strong>slice</strong> and the content buffer <strong>slice</strong>, or one of the
 *     zero-length error constant arrays
 */
public static ByteBuf[] decodeMimeAndContentBuffersSlices(
    ByteBuf compositeMetadata, int entryIndex, boolean retainSlices) {
  compositeMetadata.markReaderIndex();
  compositeMetadata.readerIndex(entryIndex);

  if (compositeMetadata.isReadable()) {
    ByteBuf mime;
    int ridx = compositeMetadata.readerIndex();
    byte mimeIdOrLength = compositeMetadata.readByte();
    if ((mimeIdOrLength & STREAM_METADATA_KNOWN_MASK) == STREAM_METADATA_KNOWN_MASK) {
      mime =
          retainSlices
              ? compositeMetadata.retainedSlice(ridx, 1)
              : compositeMetadata.slice(ridx, 1);
    } else {
      // M flag unset, remaining 7 bits are the length of the mime
      int mimeLength = Byte.toUnsignedInt(mimeIdOrLength) + 1;

      if (compositeMetadata.isReadable(
          mimeLength)) { // need to be able to read an extra mimeLength bytes
        // here we need a way for the returned ByteBuf to differentiate between a
        // 1-byte length mime type and a 1 byte encoded mime id, preferably without
        // re-applying the byte mask. The easiest way is to include the initial byte
        // and have further decoding ignore the first byte. 1 byte buffer == id, 2+ byte
        // buffer == full mime string.
        mime =
            retainSlices
                ?
                // we accommodate that we don't read from current readerIndex, but
                // readerIndex - 1 ("0"), for a total slice size of mimeLength + 1
                compositeMetadata.retainedSlice(ridx, mimeLength + 1)
                : compositeMetadata.slice(ridx, mimeLength + 1);
        // we thus need to skip the bytes we just sliced, but not the flag/length byte
        // which was already skipped in initial read
        compositeMetadata.skipBytes(mimeLength);
      } else {
        compositeMetadata.resetReaderIndex();
        throw new IllegalStateException("metadata is malformed");
      }
    }

    if (compositeMetadata.isReadable(3)) {
      // ensures the length medium can be read
      final int metadataLength = compositeMetadata.readUnsignedMedium();
      if (compositeMetadata.isReadable(metadataLength)) {
        ByteBuf metadata =
            retainSlices
                ? compositeMetadata.readRetainedSlice(metadataLength)
                : compositeMetadata.readSlice(metadataLength);
        compositeMetadata.resetReaderIndex();
        return new ByteBuf[] {mime, metadata};
      } else {
        compositeMetadata.resetReaderIndex();
        throw new IllegalStateException("metadata is malformed");
      }
    } else {
      compositeMetadata.resetReaderIndex();
      throw new IllegalStateException("metadata is malformed");
    }
  }
  compositeMetadata.resetReaderIndex();
  throw new IllegalArgumentException(
      String.format("entry index %d is larger than buffer size", entryIndex));
}
 
Example 11
Source File: RawMessageImpl.java    From pulsar with Apache License 2.0 4 votes vote down vote up
public RawMessageImpl(MessageIdData id, ByteBuf headersAndPayload) {
    this.id = id;
    this.headersAndPayload = headersAndPayload.retainedSlice();
}
 
Example 12
Source File: ContentFrame.java    From ballerina-message-broker with Apache License 2.0 4 votes vote down vote up
public static ContentFrame parse(ByteBuf buf, int channel, long payloadSize) {
    ByteBuf payload = buf.retainedSlice(buf.readerIndex(), (int) payloadSize);
    buf.skipBytes((int) payloadSize);

    return new ContentFrame(channel, payloadSize, payload);
}
 
Example 13
Source File: TdsMessage.java    From vertx-sql-client with Apache License 2.0 4 votes vote down vote up
public static TdsMessage newTdsMessage(MessageType type, MessageStatus status, int processId, ByteBuf data) {
  return new TdsMessage(type, status, processId, data.retainedSlice());
}
 
Example 14
Source File: HAProxyMessage.java    From netty-4.1.22 with Apache License 2.0 4 votes vote down vote up
private static HAProxyTLV readNextTLV(final ByteBuf header) {

        // We need at least 4 bytes for a TLV
        if (header.readableBytes() < 4) {
            return null;
        }

        final byte typeAsByte = header.readByte();
        final HAProxyTLV.Type type = HAProxyTLV.Type.typeForByteValue(typeAsByte);

        final int length = header.readUnsignedShort();
        switch (type) {
        case PP2_TYPE_SSL:
            final ByteBuf rawContent = header.retainedSlice(header.readerIndex(), length);
            final ByteBuf byteBuf = header.readSlice(length);
            final byte client = byteBuf.readByte();
            final int verify = byteBuf.readInt();

            if (byteBuf.readableBytes() >= 4) {

                final List<HAProxyTLV> encapsulatedTlvs = new ArrayList<HAProxyTLV>(4);
                do {
                    final HAProxyTLV haProxyTLV = readNextTLV(byteBuf);
                    if (haProxyTLV == null) {
                        break;
                    }
                    encapsulatedTlvs.add(haProxyTLV);
                } while (byteBuf.readableBytes() >= 4);

                return new HAProxySSLTLV(verify, client, encapsulatedTlvs, rawContent);
            }
            return new HAProxySSLTLV(verify, client, Collections.<HAProxyTLV>emptyList(), rawContent);
        // If we're not dealing with a SSL Type, we can use the same mechanism
        case PP2_TYPE_ALPN:
        case PP2_TYPE_AUTHORITY:
        case PP2_TYPE_SSL_VERSION:
        case PP2_TYPE_SSL_CN:
        case PP2_TYPE_NETNS:
        case OTHER:
            return new HAProxyTLV(type, typeAsByte, header.readRetainedSlice(length));
        default:
            return null;
        }
    }
 
Example 15
Source File: JsonObjectDecoder.java    From netty-4.1.22 with Apache License 2.0 4 votes vote down vote up
/**
 * Override this method if you want to filter the json objects/arrays that get passed through the pipeline.
 * 如果您想要过滤通过管道传递的json对象/数组,请重写此方法。
 */
@SuppressWarnings("UnusedParameters")
protected ByteBuf extractObject(ChannelHandlerContext ctx, ByteBuf buffer, int index, int length) {
    return buffer.retainedSlice(index, length);
}
 
Example 16
Source File: Entry.java    From distributedlog with Apache License 2.0 2 votes vote down vote up
/**
 * Set the entry buffer of the serialized bytes data of this record set.
 *
 * @param buffer
 *          input stream
 * @return builder
 */
public Builder setEntry(ByteBuf buffer) {
    this.buffer = buffer.retainedSlice();
    return this;
}
 
Example 17
Source File: DelayPacket.java    From java-Kcp with Apache License 2.0 2 votes vote down vote up
public void init(ByteBuf src) {
    this.ptr = src.retainedSlice();

}
 
Example 18
Source File: FtdcDecoder.java    From ftdc with Apache License 2.0 2 votes vote down vote up
/**
 * Extract the sub-region of the specified buffer.
 * <p>
 * If you are sure that the frame and its content are not accessed after
 * the current {@link #decode(ChannelHandlerContext, ByteBuf)}
 * call returns, you can even avoid memory copy by returning the sliced
 * sub-region (i.e. <tt>return buffer.slice(index, length)</tt>).
 * It's often useful when you convert the extracted frame into an object.
 * Refer to the source code of {@link ObjectDecoder} to see how this method
 * is overridden to avoid memory copy.
 */
protected ByteBuf extractFrame(ChannelHandlerContext ctx, ByteBuf buffer, int index, int length) {
    return buffer.retainedSlice(index, length);
}
 
Example 19
Source File: LengthFieldBasedFrameDecoder.java    From netty-4.1.22 with Apache License 2.0 2 votes vote down vote up
/**
 * Extract the sub-region of the specified buffer.
 * <p>
 * If you are sure that the frame and its content are not accessed after
 * the current {@link #decode(ChannelHandlerContext, ByteBuf)}
 * call returns, you can even avoid memory copy by returning the sliced
 * sub-region (i.e. <tt>return buffer.slice(index, length)</tt>).
 * It's often useful when you convert the extracted frame into an object.
 * Refer to the source code of {@link ObjectDecoder} to see how this method
 * is overridden to avoid memory copy.
 * 提取指定缓冲区的子区域。
 如果您确信在当前解码(ChannelHandlerContext, ByteBuf)调用返回后,框架及其内容没有被访问,您甚至可以通过返回切片子区域(即返回缓冲区)来避免内存复制。片(指数、长度))。当您将提取的帧转换为对象时,它通常是有用的。请参阅ObjectDecoder的源代码,了解如何重写此方法以避免内存复制。
 */
protected ByteBuf extractFrame(ChannelHandlerContext ctx, ByteBuf buffer, int index, int length) {
    return buffer.retainedSlice(index, length);
}