Java Code Examples for org.apache.lucene.codecs.simpletext.SimpleTextCodec

The following examples show how to use org.apache.lucene.codecs.simpletext.SimpleTextCodec. These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source Project: lucene-solr   Source File: BaseIndexFileFormatTestCase.java    License: Apache License 2.0 4 votes vote down vote up
/** This test is a best effort at verifying that checkIntegrity doesn't miss any files. It tests that the
 *  combination of opening a reader and calling checkIntegrity on it reads all bytes of all files. */
public void testCheckIntegrityReadsAllBytes() throws Exception {
  assumeFalse("SimpleText doesn't store checksums of its files", getCodec() instanceof SimpleTextCodec);
  FileTrackingDirectoryWrapper dir = new FileTrackingDirectoryWrapper(newDirectory());
  applyCreatedVersionMajor(dir);

  IndexWriterConfig cfg = new IndexWriterConfig(new MockAnalyzer(random()));
  IndexWriter w = new IndexWriter(dir, cfg);
  final int numDocs = atLeast(100);
  for (int i = 0; i < numDocs; ++i) {
    Document d = new Document();
    addRandomFields(d);
    w.addDocument(d);
  }
  w.forceMerge(1);
  w.commit();
  w.close();

  ReadBytesDirectoryWrapper readBytesWrapperDir = new ReadBytesDirectoryWrapper(dir);
  IndexReader reader = DirectoryReader.open(readBytesWrapperDir);
  LeafReader leafReader = getOnlyLeafReader(reader);
  leafReader.checkIntegrity();

  Map<String, FixedBitSet> readBytesMap = readBytesWrapperDir.getReadBytes();

  Set<String> unreadFiles = new HashSet<>(dir.getFiles());System.out.println(Arrays.toString(dir.listAll()));
  unreadFiles.removeAll(readBytesMap.keySet());
  unreadFiles.remove(IndexWriter.WRITE_LOCK_NAME);
  assertTrue("Some files have not been open: " + unreadFiles, unreadFiles.isEmpty());

  List<String> messages = new ArrayList<>();
  for (Map.Entry<String, FixedBitSet> entry : readBytesMap.entrySet()) {
    String name = entry.getKey();
    FixedBitSet unreadBytes = entry.getValue().clone();
    unreadBytes.flip(0, unreadBytes.length());
    int unread = unreadBytes.nextSetBit(0);
    if (unread != Integer.MAX_VALUE) {
      messages.add("Offset " + unread + " of file " + name + "(" + unreadBytes.length() + "bytes) was not read.");
    }
  }
  assertTrue(String.join("\n", messages), messages.isEmpty());
  reader.close();
  dir.close();
}
 
Example 2
Source Project: mtas   Source File: MtasSimpleTextCodec.java    License: Apache License 2.0 4 votes vote down vote up
/**
 * Instantiates a new mtas simple text codec.
 */
public MtasSimpleTextCodec() {
  super("MtasSimpleTextCodec", new SimpleTextCodec());
}