phaistos-networks/TANK

TANK-master
- service.cpp
- service_ro_segments.cpp
- service_verify.cpp
- common.h
- Switch
  - switch_bitops.h
  - compress.h
  - switch.h
  - Makefile
  - sparsefixedbitset.h
  - switch_algorithms.h
  - switch_atomicops.h
  - text.cpp
  - timings.h
  - switch_print.h
  - portability.h
  - switch_refcnt.h
  - crypto.cpp
  - tl
    - optional.hpp
    - expected.hpp
    - make_array.hpp
    - function_ref.hpp
  - base64.h
  - data.h
  - switch_security.cpp
  - switch_ll.h
  - thread.h
  - data.cpp
  - switch_vector.h
  - switch_ranges.h
  - fs.h
  - date.h
  - network.h
  - prioqueue.h
  - switch_numops.h
  - switch_dictionary.h
  - ext
    - FastPFor
      - msvc
        codecs.vcxproj
        entropy.vcxproj
        getopt
        getopt.h
        getopt.c
        FastPFor.vcxproj
        FastPFor.sln
        getopt.vcxproj
        unit.vcxproj
        inmemorybenchmark.vcxproj
        csv2maropu.vcxproj
        gapstats.vcxproj
        benchbitpacking.vcxproj
        example.vcxproj
      - src
        streamvbyte.c
        csv2maropu.cpp
        varintdecode.c
        codecs.cpp
        entropy.cpp
        bitpackingunaligned.cpp
        gapstats.cpp
        benchbitpacking.cpp
        unit.cpp
        bitpacking.cpp
        horizontalbitpacking.cpp
        inmemorybenchmark.cpp
        partitionbylength.cpp
      - Makefile
      - CHANGELOG
      - headers
        simple9_rle.h
        usimdbitpacking.h
        rolledbitpacking.h
        simdbitpacking.h
        simdbinarypacking.h
        varintgb.h
        stringutil.h
        common.h
        optpfor.h
        snappydelta.h
        bitpackingunaligned.h
        memutil.h
        packingvectors.h
        VarIntG8IU.h
        simdnewpfor.h
        maropuparser.h
        cpubenchmark.h
        synthetic.h
        streamvariablebyte.h
        codecfactory.h
        util.h
        simple8b.h
        bitpackingaligned.h
        fastpfor.h
        pfor.h
        bitpacking.h
        bitpackinghelpers.h
        blockpacking.h
        csv.h
        pfor2008.h
        newpfor.h
        simdoptpfor.h
        entropy.h
        externalvector.h
        simple8b_rle.h
        mersenne.h
        simdvariablebyte.h
        ztimer.h
        simdpfor.h
        vsencoding.h
        simdfastpfor.h
        simple9.h
        compositecodec.h
        horizontalbitpacking.h
        simple16.h
        codecs.h
        deltautil.h
        variablebyte.h
      - cmake_install.cmake
      - LICENSE
      - gapstats
      - partitionbylength
      - cmake_modules
        DetectCPUFeatures.cmake
        Findsnappy.cmake
        AppendCompilerFlags.cmake
      - CMakeFiles
        TargetDirectories.txt
        CMakeRuleHashes.txt
        feature_tests.cxx
        FastPFor.dir
        CXX.includecache
        cmake_clean_target.cmake
        flags.make
        cmake_clean.cmake
        build.make
        C.includecache
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        inmemorybenchmark.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        unit.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        gapstats.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        check.dir
        cmake_clean.cmake
        build.make
        DependInfo.cmake
        progress.make
        feature_tests.c
        feature_tests.bin
        Makefile.cmake
        codecs.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        CMakeOutput.log
        inmemorybenchmarksnappy.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        csv2maropu.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        partitionbylength.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        progress.marks
        entropy.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        Makefile2
        example.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        codecssnappy.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        3.10.2
        CMakeSystem.cmake
        CompilerIdC
        CMakeCCompilerId.c
        CMakeDetermineCompilerABI_CXX.bin
        CMakeDetermineCompilerABI_C.bin
        CompilerIdCXX
        CMakeCXXCompilerId.cpp
        CMakeCCompiler.cmake
        CMakeCXXCompiler.cmake
        3.5.1
        CMakeSystem.cmake
        CompilerIdC
        CMakeCCompilerId.c
        CMakeDetermineCompilerABI_CXX.bin
        CMakeDetermineCompilerABI_C.bin
        CompilerIdCXX
        CMakeCXXCompilerId.cpp
        CMakeCCompiler.cmake
        CMakeCXXCompiler.cmake
        CMakeDirectoryInformation.cmake
        benchbitpacking.dir
        CXX.includecache
        flags.make
        cmake_clean.cmake
        build.make
        depend.internal
        link.txt
        DependInfo.cmake
        depend.make
        progress.make
        cmake.check_cache
      - csv2maropu
      - data
        README
        ClueWeb09LogSizeFreqs.1
      - CMakeCache.txt
      - CMakeLists.txt
      - README.md
      - example.cpp
      - tools
        clang-format.sh
      - entropy
    - ebtree
      - ebtree
        ebsttree.h
        Makefile
        ebistree.c
        compiler.h
        ebmbtree.h
        ebpttree.h
        eb32tree.h
        LICENSE
        common
        defaults.h
        splice.h
        syscall.h
        tools.h
        config.h
        errors.h
        accept4.h
        ticks.h
        compiler.h
        regex.h
        debug.h
        mini-clist.h
        base64.h
        version.h
        uri_auth.h
        rbtree.h
        hash.h
        sessionhash.h
        compat.h
        time.h
        memory.h
        appsession.h
        standard.h
        epoll.h
        chunk.h
        template.h
        cfgparse.h
        buffer.h
        eb64tree.h
        ebsttree.c
        eb64tree.c
        ebimtree.c
        ebpttree.c
        ebimtree.h
        ebmbtree.c
        ebtree.h
        ebtree.c
        ebistree.h
        eb32tree.c
      - ebsttree.h
      - Makefile
      - ebistree.c
      - compiler.h
      - ebmbtree.h
      - ebpttree.h
      - eb32tree.h
      - LICENSE
      - common
        defaults.h
        splice.h
        syscall.h
        tools.h
        config.h
        errors.h
        accept4.h
        ticks.h
        compiler.h
        regex.h
        debug.h
        mini-clist.h
        base64.h
        version.h
        uri_auth.h
        rbtree.h
        hash.h
        sessionhash.h
        compat.h
        time.h
        memory.h
        appsession.h
        standard.h
        epoll.h
        chunk.h
        template.h
        cfgparse.h
        buffer.h
      - eb64tree.h
      - ebsttree.c
      - eb64tree.c
      - ebimtree.c
      - ebpttree.c
      - ebimtree.h
      - ebmbtree.c
      - ebtree.h
      - ebtree.c
      - ebistree.h
      - eb32tree.c
    - flat_hash_map.h
    - martinus
      - robin_hood.h
    - ext_xxhash.c
    - tl
      - optional.hpp
      - expected.hpp
      - make_array.hpp
      - function_ref.hpp
    - json
      - .github
        PULL_REQUEST_TEMPLATE.md
        ISSUE_TEMPLATE
        Feature_request.md
        Bug_report.md
        CONTRIBUTING.md
      - cmake
        config.cmake.in
      - Makefile
      - include
        nlohmann
        adl_serializer.hpp
        detail
        json_pointer.hpp
        iterators
        primitive_iterator.hpp
        iter_impl.hpp
        internal_iterator.hpp
        iteration_proxy.hpp
        json_reverse_iterator.hpp
        json_ref.hpp
        output
        output_adapters.hpp
        binary_writer.hpp
        serializer.hpp
        macro_unscope.hpp
        macro_scope.hpp
        value_t.hpp
        exceptions.hpp
        meta
        cpp_future.hpp
        detected.hpp
        is_sax.hpp
        type_traits.hpp
        void_t.hpp
        input
        input_adapters.hpp
        json_sax.hpp
        parser.hpp
        binary_reader.hpp
        lexer.hpp
        conversions
        from_json.hpp
        to_chars.hpp
        to_json.hpp
        json_fwd.hpp
      - third_party
        amalgamate
        config.json
        amalgamate.py
        CHANGES.md
        LICENSE.md
        README.md
      - single_include
        nlohmann
      - nlohmann_json.natvis
      - LICENSE.MIT
      - CMakeLists.txt
      - README.md
      - CODE_OF_CONDUCT.md
      - meson.build
      - ChangeLog.md
      - doc
        binary_formats.md
        Makefile
        faq.md
        usages
        Doxyfile
        images
        range-begin-end.svg
        range-rbegin-rend.svg
        examples
        other_error.output
        type_error.output
        get__PointerType.link
        type_error.cpp
        dump.cpp
        is_number_unsigned.output
        front.cpp
        swap__object_t.cpp
        swap__reference.cpp
        insert__ilist.link
        operator__equal.cpp
        operatorarray__size_type_const.link
        crbegin.link
        crbegin.cpp
        get__PointerType.output
        is_number.cpp
        patch.cpp
        rbegin.cpp
        crend.link
        back.cpp
        operator__greater.link
        swap__string_t.cpp
        basic_json__InputIt_InputIt.output
        parse__istream__parser_callback_t.cpp
        parse__string__parser_callback_t.cpp
        operatorjson_pointer.cpp
        basic_json__value_ptr.link
        swap__object_t.link
        patch.output
        at__object_t_key_type_const.output
        at__object_t_key_type_const.cpp
        sax_parse.link
        push_back__initializer_list.output
        end.link
        emplace_back.link
        is_number_float.output
        is_number_integer.cpp
        merge_patch.cpp
        is_object.link
        out_of_range.output
        from_ubjson.link
        is_number_unsigned.link
        rend.link
        operator__equal__nullptr_t.output
        end.output
        iterator_wrapper.cpp
        items.cpp
        items.link
        json_pointer.cpp
        swap__reference.output
        basic_json__list_init_t.output
        operator__greaterequal.cpp
        basic_json__moveconstructor.output
        operator_deserialize.output
        size.output
        erase__IteratorType.cpp
        front.output
        parse__array__parser_callback_t.link
        parse__array__parser_callback_t.output
        erase__IteratorType_IteratorType.output
        rend.output
        is_structured.output
        is_number.output
        update.output
        push_back__initializer_list.link
        from_cbor.link
        parse__array__parser_callback_t.cpp
        sax_parse.output
        at_json_pointer_const.cpp
        cend.link
        get_ptr.cpp
        emplace_back.cpp
        invalid_iterator.link
        operator__greater.output
        is_structured.cpp
        push_back.link
        erase__size_type.output
        swap__object_t.output
        items.output
        update.cpp
        cbegin.output
        type.cpp
        array.cpp
        at__size_type_const.cpp
        insert__count.link
        iterator_wrapper.link
        insert.cpp
        insert__ilist.cpp
        at_json_pointer.cpp
        push_back__object_t__value.output
        basic_json__basic_json.link
        insert.output
        operator__notequal.output
        swap__reference.link
        from_ubjson.cpp
        basic_json__moveconstructor.cpp
        is_boolean.link
        operatorjson_pointer.link
        merge_patch.link
        get_ref.output
        at_json_pointer.output
        erase__key_type.cpp
        begin.cpp
        basic_json__CompatibleType.output
        operatorarray__size_type_const.output
        operatorjson_pointer_const.cpp
        basic_json__moveconstructor.link
        meta.cpp
        operator__ValueType.cpp
        at__object_t_key_type.link
        type.output
        erase__IteratorType.link
        operatorarray__size_type.cpp
        insert__range_object.link
        type_name.link
        is_string.cpp
        basic_json__basic_json.cpp
        operator__value_t.link
        diff.cpp
        empty.cpp
        json_pointer.link
        find__key_type.link
        is_discarded.link
        basic_json__size_type_basic_json.output
        update__range.link
        operator__equal.link
        begin.output
        to_cbor.link
        is_primitive.output
        operator__less.link
        basic_json__value_t.link
        rbegin.output
        parse_error.cpp
        unflatten.cpp
        object.link
        basic_json__list_init_t.link
        crbegin.output
        crend.output
        type_error.link
        array.output
        operatorjson_pointer_const.link
        operator__lessequal.cpp
        type.link
        iterator_wrapper.output
        parse__iteratortype__parser_callback_t.cpp
        push_back.cpp
        invalid_iterator.cpp
        basic_json__CompatibleType.link
        parse_error.output
        push_back__initializer_list.cpp
        from_ubjson.output
        basic_json__CompatibleType.cpp
        operator__less.cpp
        is_number_float.link
        is_number_float.cpp
        emplace.cpp
        count.cpp
        is_number.link
        operator__greater.cpp
        is_array.link
        get__PointerType.cpp
        parse__string__parser_callback_t.output
        basic_json__nullptr_t.link
        to_msgpack.cpp
        basic_json__nullptr_t.output
        at__size_type_const.output
        at__size_type.cpp
        operator__notequal__nullptr_t.cpp
        get_ref.link
        parse__contiguouscontainer__parser_callback_t.output
        README.cpp
        is_discarded.cpp
        from_msgpack.cpp
        unflatten.output
        at__size_type.link
        basic_json__value_ptr.cpp
        basic_json__copyassignment.cpp
        is_null.cpp
        erase__IteratorType_IteratorType.cpp
        insert__range.output
        basic_json__InputIt_InputIt.cpp
        front.link
        is_string.output
        erase__size_type.link
        find__key_type.output
        is_boolean.cpp
        basic_json__size_type_basic_json.link
        operator_serialize.output
        size.link
        operatorarray__key_type.link
        object.cpp
        operator__equal.output
        operator__lessequal.output
        push_back__object_t__value.cpp
        parse_error.link
        count.output
        at__object_t_key_type_const.link
        merge_patch.output
        empty.output
        is_string.link
        get_ref.cpp
        flatten.cpp
        basic_json__size_type_basic_json.cpp
        operator__value_t.cpp
        update.link
        erase__key_type.output
        diff.output
        cend.cpp
        insert__range_object.output
        max_size.link
        emplace_back.output
        operator__notequal.cpp
        is_array.cpp
        operatorarray__size_type_const.cpp
        out_of_range.link
        max_size.output
        basic_json__copyassignment.output
        operator__ValueType.link
        clear.cpp
        cbegin.cpp
        patch.link
        back.output
        cbegin.link
        basic_json__list_init_t.cpp
        operator__lessequal.link
        at__size_type_const.link
        exception.output
        json_pointer.output
        to_msgpack.link
        from_cbor.output
        begin.link
        operatorarray__key_type.cpp
        get_ptr.output
        is_discarded.output
        is_number_integer.link
        find__key_type.cpp
        insert__ilist.output
        at_json_pointer_const.link
        get__ValueType_const.cpp
        insert__range_object.cpp
        swap__string_t.output
        basic_json__nullptr_t.cpp
        end.cpp
        to_cbor.cpp
        out_of_range.cpp
        parse__iteratortype__parser_callback_t.output
        get__ValueType_const.link
        flatten.output
        operatorjson_pointer_const.output
        other_error.link
        parse__contiguouscontainer__parser_callback_t.cpp
        at__object_t_key_type.output
        basic_json__InputIt_InputIt.link
        array.link
        get_ptr.link
        basic_json__copyassignment.link
        operatorjson_pointer.output
        is_structured.link
        operator__notequal__nullptr_t.output
        exception.link
        sax_parse.cpp
        emplace.link
        is_primitive.link
        erase__size_type.cpp
        basic_json__value.cpp
        erase__key_type.link
        README.output
        to_ubjson.output
        operator_deserialize.link
        clear.output
        insert__count.cpp
        parse__istream__parser_callback_t.output
        push_back.output
        operatorarray__key_type_const.link
        meta.link
        rbegin.link
        other_error.cpp
        type_name.cpp
        at__size_type.output
        rend.cpp
        operator__equal__nullptr_t.link
        basic_json__value_t.cpp
        is_null.output
        push_back__object_t__value.link
        is_array.output
        operator_deserialize.cpp
        basic_json__basic_json.output
        erase__IteratorType.output
        cend.output
        insert__range.cpp
        exception.cpp
        back.link
        to_ubjson.cpp
        from_cbor.cpp
        operator__value_t.output
        parse__string__parser_callback_t.link
        is_primitive.cpp
        max_size.cpp
        flatten.link
        diff.link
        at_json_pointer_const.output
        at_json_pointer.link
        dump.link
        update__range.output
        operatorarray__size_type.link
        clear.link
        invalid_iterator.output
        json_pointer__to_string.output
        to_msgpack.output
        json_pointer__to_string.link
        at__object_t_key_type.cpp
        parse__istream__parser_callback_t.link
        swap__array_t.cpp
        operator_serialize.cpp
        basic_json__value_ptr.output
        README.link
        operator_serialize.link
        emplace.output
        operator__less.output
        operator__greaterequal.output
        meta.output
        insert__count.output
        dump.output
        is_object.output
        operator__notequal__nullptr_t.link
        count.link
        operator__notequal.link
        operator__equal__nullptr_t.cpp
        to_cbor.output
        size.cpp
        is_object.cpp
        to_ubjson.link
        operatorarray__key_type_const.output
        basic_json__value_t.output
        object.output
        unflatten.link
        empty.link
        crend.cpp
        insert__range.link
        get__ValueType_const.output
        insert.link
        operator__ValueType.output
        from_msgpack.link
        operatorarray__key_type_const.cpp
        is_null.link
        operatorarray__size_type.output
        type_name.output
        swap__array_t.output
        operator__greaterequal.link
        basic_json__value.link
        erase__IteratorType_IteratorType.link
        is_number_unsigned.cpp
        basic_json__value.output
        swap__array_t.link
        operatorarray__key_type.output
        update__range.cpp
        parse__contiguouscontainer__parser_callback_t.link
        is_boolean.output
        from_msgpack.output
        is_number_integer.output
        json_pointer__to_string.cpp
        parse__iteratortype__parser_callback_t.link
        swap__string_t.link
        index.md
        scripts
        send_to_wandbox.py
        git-update-ghpages
        css
        mylayout.css
        mylayout_docset.css
  - switch_compiler_aux.h
  - ansifmt.h
  - README.md
  - switch_hash.h
  - base64.cpp
  - crypto.h
  - switch_security.h
  - switch_exceptions.h
  - switch_common.h
  - text_table.h
  - text.h
  - ext_snappy
    - config.h
    - Makefile
    - snappy-sinksource.h
    - snappy-stubs-public.h
    - snappy-internal.h
    - snappy.h
    - snappy.cc
    - snappy-stubs-internal.cc
    - snappy-stubs-internal.h
    - snappy-sinksource.cc
    - snappy-test.h
  - buffer.h
  - switch_mallocators.h
- tank_client.h
- client_common.h
- client_reactor.cpp
- Makefile
- client.cpp
- client_api.cpp
- service_indices.cpp
- service_compact.cpp
- service_main.cpp
- service_json.cpp
- service_consul_io.cpp
- service_reactor.cpp
- client_api_srv_status.cpp
- service_wip.cpp
- service_pragmas.h
- service_consul.cpp
- tank_protocol.md
- service_bootstrap.cpp
- LICENSE
- service_cluster_util.cpp
- service.h
- service_cluster_reconcile.cpp
- service_utils.cpp
- client_api_reload_config.cpp
- client_brokers_util.cpp
- service_api_produce.cpp
- client_api_discover_partitions.cpp
- service_wait.cpp
- service_cluster.cpp
- service_api.cpp
- service_isr.cpp
- client_api_fast_consume.cpp
- service_write.cpp
- client_api_create_topic.cpp
- service_common.h
- service_replication.cpp
- client_api_produce.cpp
- client_pragmas.h
- README.md
- client_utils.cpp
- client_resources.cpp
- service_read.cpp
- client_api_consume.cpp
- cli.cpp
- service_cluster_verify.cpp
- Dockerfile
- service_hw.cpp
- tank_encoding.md
- service_cluster_apply_state_updates.cpp
- service_api_consume.cpp
- service_cluster_gen_partition_nodes_updates.cpp
- service_log.cpp

TANK is a very high performance distributed log, inspired in part by Kafka, and other similar services and technologies. This is the second major public release, TANK 2. Read about this new public release here.

Introduction

You should begin by reading about the core concepts and the client API (A new Java Tank Client is now available).

Please see building instructions. You may also want to run Tank using its Docker image.

Features include:

Very high performance
Very tight encoding of messages(into bundles)
Simple design (Simple is Beautiful; Convenience is Key)
compactions based on message keys and in the future based on programmable logic
A powerful CLI tool for managing, querying, setting messages, mirroring across brokers, etc.
Prometheus Support

TANK's goal is highest performance and simplicity. If you need very high performance, operation simplicity and no reliance on other services (when running TANK in stand-alone mode), consider TANK.

Please see the wiki for more information.

We chose the name TANK because its a storage chamber, suitable for liquids and gas - which we think is analogous to a storage container for data that flows, from and to other containers and other systems via 'pipes' (connections).