Practical Issues using Distributed Computing Environments – Apache Hadoop
Keywords:Apache Hadoop, HTC – High Throughput Computing/HPC – High Performance Computing, distributed computing, map-reduce, distributed file system
AbstractThe paper presents practical results obtained with sequential standard programming, RPC/RMI mechanism and Apache Hadoop distributed computing platform for a problem of computation time and power that might be used in e-mail text searching. First section is about Distributed Computing technologies and middleware introduction. In second and third section are shown few details about RPC/RMI and Apache Hadoop approaches. The fourth section presents the results of the computation for a classic problem such as word counting from large text files using standard versus remote procedure call versus map-reducing approach. In the end are shown the main advantages of the distributed systems and computing environments.
Tom White, Hadoop – The Definitive Guide, O'Reilly Media, 528 pp, ISBN-10: 0596521979, ISBN-13: 978-0596521974, US 2009.
Chuck Lam, Hadoop in Action, Manning Publishing, 325 pp, ISBN-10: 1935182196 , ISBN-13: 978-1935182191, US 2010.
Apache Hadoop Project, http://hadoop.apache.org/
Hadoop Project, HDFS Architecture, available at http://hadoop.apache.org/docs/stable/hdfs_design.html
Vincent McBurney, So what is better, ETL or ELT?, Toolbox.com, Available at: http://it.toolbox.com/blogs/
Gopalan Suresh Raj, A Detailed Comparison of CORBA, DCOM and Java/RMI, available at: http://my.execpc.com/~gopalan/misc/compare.html
Wikipedia, Distributed computing, available at: http://en.wikipedia.org/wiki/Distributed_computing
Yahoo, Yahoo! Hadoop Tutorial, available at: http://developer.yahoo.com/hadoop/tutorial/
The Outline of Science, Vol. 1 (of 4) by J. Arthur Thomson - http://www.gutenberg.org/ebooks/20417
The Notebooks of Leonardo Da Vinci — Complete by Leonardo da Vinci - http://www.gutenberg.org/ebooks/5000
James Joyce, Ulysses, available at: http://www.gutenberg.org/ebooks/4300
Wikipedia, Cloud Computing, avaialble at: https://en.wikipedia.org/wiki/Cloud_computing
Google, Map-Reduce Programming, available at: https://developers.google.com/appengine/docs/python/dataprocessing/
How to Cite
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
- The author(s) is responsible for the correctness and legality of the paper content.
- Papers that are copyrighted or published will not be taken into consideration for publication in JMEDS It is the author(s) responsibility to ensure that the paper does not cause any copyright infringements and other problems.
- It is the responsibility of the author(s) to obtain all necessary copyright release permissions for the use of any copyrighted materials in the paper prior to the submission.
- The Author(s) retains the right to reuse any portion of the paper, in future works, including books, lectures and presentations in all media, with the condition that the publication by JMEDS is properly credited and referenced.
JMEDS articles by Journal of Mobile, Embedded and Distributed Systems (JMEDS) is licensed under a Creative Commons Attribution 4.0 International License.
Based on a work at http://jmeds.eu.
Permissions beyond the scope of this license may be available at http://jmeds.eu/index.php/jmeds/about/submissions#copyrightNotice.