1. . This simple utf-8 word count function (it only counts). is a bit faster then the one with pregmatchall. about 10x slower then the built-in strwordcount.
  2. Right now, I write the wordcount of the main textbox in the title of the page and then count manually. My idea would be to still do the manual wordcount for the main textbox, but for a macro to then gather all the wordcounts on all the pages where I place the wordcount marker. So, I could for example mark the wordcount on the page by writing.
  3. Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for.
  1. Word Count 2 02 1
  2. Word Count 2 020
  3. Word Count 20 Minute Speech
Pages

In this post, we provide an introduction to the basics of MapReduce, along with a tutorial to create a word count app using Hadoop and Java.

The question asks how to find the count of a specific item. With this approach, the solution would require mapping the desired element to its count value as follows: scala collections do have count: list.count ( 2) I had the same problem as Sharath Prabhal, and I got another (to me clearer) solution.
Join the DZone community and get the full member experience.
Join For Free
Wordcount 2 02

In Hadoop, MapReduce is a computation that decomposes large manipulation jobs into individual tasks that can be executed in parallel across a cluster of servers. The results of tasks can be joined together to compute final results.

MapReduce consists of 2 steps:

  • Map Function – It takes a set of data and converts it into another set of data, where individual elements are broken down into tuples (Key-Value pair).

    • Example – (Map function in Word Count)

Input
Set of data
Wifispoof 3 4 8 x 5.
Bus, Car, bus, car, train, car, bus, car, train, bus, TRAIN,BUS, buS, caR, CAR, car, BUS, TRAIN
Output
Convert into another set of data
(Key,Value)
(Bus,1), (Car,1), (bus,1), (car,1), (train,1),
(car,1), (bus,1), (car,1), (train,1), (bus,1),
(TRAIN,1),(BUS,1), (buS,1), (caR,1), (CAR,1),
(car,1), (BUS,1), (TRAIN,1)
  • Reduce Function – Takes the output from Map as an input and combines those data tuples into a smaller set of tuples.

    • Example – (Reduce function in Word Count)

    Input
    (output of Map function)
    Set of Tuples

    (Bus,1), (Car,1), (bus,1), (car,1), (train,1),
    (car,1), (bus,1), (car,1), (train,1), (bus,1),
    (TRAIN,1),(BUS,1), (buS,1), (caR,1), (CAR,1),
    (car,1), (BUS,1), (TRAIN,1)
    Output
    Converts into smaller set of tuples
    (BUS,7),
    (CAR,7),
    (TRAIN,4)

    Work Flow of the Program

    Workflow of MapReduce consists of 5 steps:

    1. Splitting – The splitting parameter can be anything, e.g. splitting by space, comma, semicolon, or even by a new line (‘n’).

  • Mapping – as explained above.

  • Intermediate splitting – the entire process in parallel on different clusters. In order to group them in “Reduce Phase” the similar KEY data should be on the same cluster.

  • Reduce – it is nothing but mostly group by phase.

  • Combining – The last phase where all the data (individual result set from each cluster) is combined together to form a result.

    • Now Let’s See the Word Count Program in Java

    Fortunately, we don’t have to write all of the above steps, we only need to write the splitting parameter, Map function logic, and Reduce function logic. The rest of the remaining steps will execute automatically.

    Make sure that Hadoop is installed on your system with the Java SDK.

    Steps

    1. Start 32nd. Open Eclipse> File > New > Java Project >( Name it – MRProgramsDemo) > Finish.

    2. Right Click > New > Package ( Name it - PackageDemo) > Finish.

    3. Right Click on Package > New > Class (Name it - WordCount).

    4. Add Following Reference Libraries:

      1. Right Click on Project > Build Path> Add External

        1. /usr/lib/hadoop-0.20/hadoop-core.jar

        2. Usr/lib/hadoop-0.20/lib/Commons-cli-1.2.jar

    5. Type the following code:

    The above program consists of three classes:

    6. Make a jar file

    Right Click on Project> Export> Select export destination as Jar File > next> Finish.

    7. Take a text file and move it into HDFS format:

    To move this into Hadoop directly, open the terminal and enter the following commands:

    8. Run the jar file:

    (Hadoop jar jarfilename.jar packageName.ClassName PathToInputTextFile PathToOutputDirectry)

    Word Count 2 02 1

    9. Open the result:

    mapreduce,java,hadoop,big data,tutorial,wordcount

    Word Count 2 020

    Opinions expressed by DZone contributors are their own.

    Word Count 20 Minute Speech

    Popular on DZone