HDP Sandbox Startup Problems?

When I installed the Hortonworks HDP 2.2 Sandbox in my VMware Player, I had a problem after connecting to the first web page indicated on the startup screen. That web page had bad links.

The quick and dirty solution is to explicitly include the port number 8888 in your URL.

If you want to see more detailed description of the problem (as I saw it) and the workaround (that worked for me), go to this blog entry:

https://kevpfowler.wordpress.com/2015/01/28/bad-links-in-hortonworks-hdp-2-2-sandbox-startup/

From the blog kevpfowler » wsu-cs by kevpfowler and used with permission of the author. All other rights reserved by the author.

Posted in Hadoop, WSU CS | Comments Off

Simple MapReduce Example

MapReduce can be very complicated to understand at first.  I found this simple example that takes you through the steps so that you can see how it is reduced.  You can find it at this site:

http://ayende.com/blog/4435/map-reduce-a-visual-explanation

From the blog ddgoddard » cs-wsu by ddgoddard and used with permission of the author. All other rights reserved by the author.

Posted in cs-wsu | Comments Off

Hadoop MapReduce

The Article on YDN on Hadoop was a great read but I found it very loaded for someone who first heard of Hadoop a couple of weeks ago. I did go hunting for a better, but simpler explanation of what MapReduce is, and how to apply it. I found this article on IBM. It uses a simple data set of cities and their temperatures to explain MapReduce. It also provides an interesting analogy into how the Early Romans may have ‘used’ MapReduce to conduct censuses.

I also found a set of HTML Slides on a Google Research Publication that goes that goes deeper into the inner workings of MapReduce, with an example of how they used it at Google in 2004. It also provides full C code for a word count example in the PDF version of the publication.

From the blog mrnganga » cs-wsu by mrnganga and used with permission of the author. All other rights reserved by the author.

Posted in Cloud Computing, cs-wsu, CS@Worcester | Comments Off

Hadoop MapReduce Clusters

The assigned MapReduce tutorial on YDN provided a detailed lesson on MapReduce execution and the basics of coding the MapClass and Reduce classes. I thought it was clearly presented and the checkpoint assignment was effective, if not too challenging. As another blogger noted, it may be necessary to reload the tutorial several times to get it to load successfully. But the persistence was worth it.

The article linked below includes a MapReduce example but is more focused on the cluster components and how the Hadoop architecture operates both to provide the HDFS and to execute MapReduce jobs. I found it helpful to understand a bit more of the bigger Hadoop picture in the context of a MapReduce job execution.

http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network/

From the blog kevpfowler » wsu-cs by kevpfowler and used with permission of the author. All other rights reserved by the author.

Posted in WSU CS | Comments Off

Hadoop MapReduce

Here are some videos I found that would explain MapReduce in detail

https://www.youtube.com/watch?v=7FcMhTTG1Cs – Part 1

https://www.youtube.com/watch?v=pDGLe4CsrhY – Part 2

https://www.youtube.com/watch?v=9h_WLsmRfFM – Part 3

https://www.youtube.com/watch?v=iiIDZTpdcuU – Part 4

https://www.youtube.com/watch?v=1aen3JsxkuM – Part 5

From the blog ani2017 » cs-wsu by anirudhu and used with permission of the author. All other rights reserved by the author.

Posted in cs-wsu | Comments Off

MapReduce

I tried to read the article provided at the Yahoo link; however, it kept telling me that there was an internal error so I could not get the webpage to open. I was able to skim through Hadoop Apache’s tutorial on MapReduce. It was extremely detailed and very good for someone who wants to get down to the nitty gritty about how to use it. It touches on various sections of MapReduce such as inputs and outputs, user interfaces, job configuration and even gives a thorough example of a WordCount program that uses many features provided by the MapReduce framework. It is found at this website:

http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Example%3A+WordCount+v2.0

The explanation of MapReduce that I liked a little bit more was found on IBM’s website. It was direct and to-the-point and quick to get one caught up on the wonders of MapReduce. It nicely describes the two tasks of MapReduce and also provides examples. The example using a city and corresponding temperatures does a good job at showing exactly how MapReduce works. This information can be found at the following link:

http://www-01.ibm.com/software/data/infosphere/hadoop/mapreduce/

From the blog wellhellooosailor » CS@Worcester by epaiz and used with permission of the author. All other rights reserved by the author.

Posted in CS@Worcester | Comments Off

CS-383 Intro Post

My name is Kevin Fowler and this is my intro post on my CS-383 blog.

I  am a software engineer in the Networking division at Dell. My experience is all in legacy languages, tools and infrastructure. I’m taking this course to learn newer technology concepts and capabilities. Not sure what I will do with it yet – just taking the first step.

From the blog kevpfowler » wsu-cs by kevpfowler and used with permission of the author. All other rights reserved by the author.

Posted in WSU CS | Comments Off