Project: hadoop-map-reduce-patterns (GitHub Link)

hadoop-map-reduce-patterns-master
- src
  - main
    - java
      - com
        eftimoff
        mapreduce
        organization
        binning
        BinningTags.java
        partitioning
        LastAccessDate.java
        hierarchical
        PostCommentHierarchy.java
        ReduceSideDataDiscriminator.java
        advanced
        QuestionAnswerBuilder.java
        HierarchicalXmlFragment.java
        shuffling
        Anonymize.java
        totalordering
        TotalOrderSortingStage.java
        utils
        MRDPUtils.java
        CommentTag.java
        summarization
        counters
        CountNumUsersByState.java
        invertedindex
        WikipediaExtractor.java
        numerical
        wordcount
        CommentWordCount.java
        medianstddevbyhour
        MedianAndStandardDeviationCommentLengthByHour.java
        average
        CountAverageTuple.java
        Average.java
        medianstddev
        MedianStdDev.java
        MedianStdDevTuple.java
        minmaxcount
        MinMaxCount.java
        MinMaxCountTuple.java
        filtering
        sampling
        SimpleRandomSampling.java
        filtering
        DistributedGrep.java
        bloomfilter
        BloomFilter.java
        BloomFilterDriver.java
        topten
        TopTenUsersByReputation.java
        Pair.java
        distinct
        DistinctUser.java
        joins
        reducesidejoin
        ReduceSideJoin.java
        bloomfilter
        ReduceSideJoinBloomFilter.java
        compositejoin
        CompositeUserJoin.java
        cartesianproduct
        CartesianCommentComparison.java
        replicatedjoin
        ReplicatedUserJoin.java
        meta
        jobchaining
        BasicJobChaining.java
  - test
    - java
      - com
        eftimoff
        mapreduce
        utils
        MRDPUtilsTest.java
        summarization
        numerical
        wordcount
        CommentWordCountMapperTest.java
        CommentWordCountReducerTest.java
        minmaxcount
        MinMaxCountMapperTest.java
        MinMaxCountReducerTest.java
- examples
  - Hierarchical
  - MedianAndStandardDeviationCommentLengthByHour
    - output.txt
  - SimpleRandomSampling
    - output.txt
  - QuestionAnswerBuilder
  - TopTenUsersByReputation
    - output.txt
  - BinningTags
  - WikipediaExtractor
    - output.txt
  - CommentWordCount
    - output.txt
  - BloomFilter
    - countries
    - countries.gz
  - MedianStdDev
    - output.txt
  - DistributedGrep
    - output.txt
  - MinMaxCount
  - CountNumUsersByState
  - Average
    - output.txt
- pom.xml
- LICENSE
- readme
  - LastAccessDate.md
  - TopTen.md
  - DistinctUser.md
  - MedianStdDev.md
  - img
  - Distinct.md
  - Average.md
  - BloomFilteringHotList.md
  - ReplicatedJoin.md
  - SimpleRandomSampling.md
  - NumericalSummarization.md
  - Anonymize.md
  - CompositeUserJoin.md
  - ReduceSideJoin.md
  - Shuffling.md
  - TopTenUsersByReputation.md
  - BloomFiltering.md
  - InvertedIndexSummarization.md
  - MedianAndStandardDeviationCommentLengthByHour.md
  - UserJoin.md
  - DistributedGrep.md
  - CompositeJoin.md
  - CommentWordCount.md
  - WikipediaExtractor.md
  - Hierarchical.md
  - Binning.md
  - MinMaxCount.md
  - TotalOrderSortingStage.md
  - CartesianProduct.md
  - BinningTags.md
  - Partitioning.md
  - CountNumUsersByState.md
  - ReplicatedUserJoin.md
  - UserJoinBloomFilter.md
  - Filtering.md
  - CountingCounters.md
  - BasicJobChaining.md
  - QuestionAnswerBuilder.md
  - TotalOrderSorting.md
  - PostCommentHierarchy.md
  - CartesianCommentComparison.md
- .travis.yml
- README.md
- .gitignore

MapReduce

Hadoop Map-Reduce Design Patterns

How to use the MapReduce:

Clone the repository:

git clone [email protected]:geftimov/MapReduce.git

Go in to the folder:

cd MapReduce

Build it with Maven:

mvn clean install

Example run in each individual pattern example.

Summarization Patterns

1. Numerical Summarization ReadMe

CommentWordCount

MinMaxCount

Average

MedianStdDev (With In-Memory Map)

MedianAndStandardDeviationCommentLengthByHour (Without the Map, more efficient)

2. Inverted Index Summarization ReadMe

WikipediaExtractor

3. Counting with Counters ReadMe

CountNumUsersByState

Filtering Patterns

1. Filtering ReadMe

DistributedGrep

SimpleRandomSampling

2. Bloom Filtering ReadMe

3. Top Ten ReadMe

TopTenUsersByReputation

4. Distinct ReadMe

DistinctUser

Data Organization Patterns

1. Structured to Hierarchical ReadMe

PostCommentHierarchy

QuestionAnswerBuilder

2. Partitioning ReadMe

LastAccessDate

3. Binning ReadMe

BinningTags

4. TotalOrderSorting ReadMe

TotalOrderSortingStage

5. Shuffling ReadMe

Anonymize

Join Patterns

1. Reduce Side Join ReadMe

UserJoin

UserJoinBloomFilter

2. Replicated Join ReadMe

ReplicatedUserJoin

3. Composite Join ReadMe

CompositeUserJoin

4. Cartesian Product ReadMe

CartesianCommentComparison

About Me

Georgi Kalinov Eftimov
[email protected]