Practical Issues using Distributed Computing Environments – Apache Hadoop

Authors

  • Toma Cristian Bucharest Academy of Economic Studies, Romania

Keywords:

Apache Hadoop, HTC – High Throughput Computing/HPC – High Performance Computing, distributed computing, map-reduce, distributed file system

Abstract

The paper presents practical results obtained with sequential standard programming, RPC/RMI mechanism and Apache Hadoop distributed computing platform for a problem of computation time and power that might be used in e-mail text searching. First section is about Distributed Computing technologies and middleware introduction. In second and third section are shown few details about RPC/RMI and Apache Hadoop approaches. The fourth section presents the results of the computation for a classic problem such as word counting from large text files using standard versus remote procedure call versus map-reducing approach. In the end are shown the main advantages of the distributed systems and computing environments.

Author Biography

Toma Cristian, Bucharest Academy of Economic Studies, Romania

Faculty of Cybernetics, Statistics and Economic Informatics

Department of IT&C Technologies

References

Tom White, Hadoop – The Definitive Guide, O'Reilly Media, 528 pp, ISBN-10: 0596521979, ISBN-13: 978-0596521974, US 2009.

Chuck Lam, Hadoop in Action, Manning Publishing, 325 pp, ISBN-10: 1935182196 , ISBN-13: 978-1935182191, US 2010.

Apache Hadoop Project, http://hadoop.apache.org/

Hadoop Project, HDFS Architecture, available at http://hadoop.apache.org/docs/stable/hdfs_design.html

Vincent McBurney, So what is better, ETL or ELT?, Toolbox.com, Available at: http://it.toolbox.com/blogs/

Gopalan Suresh Raj, A Detailed Comparison of CORBA, DCOM and Java/RMI, available at: http://my.execpc.com/~gopalan/misc/compare.html

Wikipedia, Distributed computing, available at: http://en.wikipedia.org/wiki/Distributed_computing

Yahoo, Yahoo! Hadoop Tutorial, available at: http://developer.yahoo.com/hadoop/tutorial/

The Outline of Science, Vol. 1 (of 4) by J. Arthur Thomson - http://www.gutenberg.org/ebooks/20417

The Notebooks of Leonardo Da Vinci — Complete by Leonardo da Vinci - http://www.gutenberg.org/ebooks/5000

James Joyce, Ulysses, available at: http://www.gutenberg.org/ebooks/4300

Wikipedia, Cloud Computing, avaialble at: https://en.wikipedia.org/wiki/Cloud_computing

Google, Map-Reduce Programming, available at: https://developers.google.com/appengine/docs/python/dataprocessing/

Downloads

Published

2013-03-30

How to Cite

Cristian, T. (2013). Practical Issues using Distributed Computing Environments – Apache Hadoop. Journal of Mobile, Embedded and Distributed Systems, 5(1), 18-28. Retrieved from http://jmeds.eu/index.php/jmeds/article/view/92

Issue

Vol. 5 No. 1 (2013): Advances in Data Processing

Section

Articles

License

Authors who publish with this journal agree to the following terms:


    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

    1. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

    1. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).

    1. The author(s) is responsible for the correctness and legality of the paper content.

    1. Papers that are copyrighted or published will not be taken into consideration for publication in JMEDS It is the author(s) responsibility to ensure that the paper does not cause any copyright infringements and other problems.

    1. It is the responsibility of the author(s) to obtain all necessary copyright release permissions for the use of any copyrighted materials in the paper prior to the submission.

  1. The Author(s) retains the right to reuse any portion of the paper, in future works, including books, lectures and presentations in all media, with the condition that the publication by JMEDS is properly credited and referenced.
Creative Commons License
JMEDS articles by Journal of Mobile, Embedded and Distributed Systems (JMEDS) is licensed under a Creative Commons Attribution 4.0 International License.
Based on a work at http://jmeds.eu.
Permissions beyond the scope of this license may be available at http://jmeds.eu/index.php/jmeds/about/submissions#copyrightNotice.