UMass Lowell at UC Berkeley
Spring 2011
UC Berkeley welcomes UMass Lowell
91.301 Organization of Programming Languages (OPL).
For help, please ask inst@eecs.berkeley.edu.
Logging onto the Icluster:
OPL students should obtain an account name and password from their
instructor at UML. The account names start with "csuml-",
as in "computer science - u mass lowell". Our 'mapreduce'
programming environment runs on the EECS Icluster. Login to
the head node from a UNIX computer with 'ssh', for example:
ssh -X csuml-aa@icluster1.eecs.berkeley.edu
The '-X' enables XWindows forwarding, which means if you type a graphical
command such as 'firefox' on the Icluster, the window will pop up on your
computer (if it is running an XWindows server). See "References"
below if you need more information about that. XWindows forwarding is
not needed to run mapreduce.
You can change your initial password by logging into "update.eecs":
ssh csuml-aa@update.eecs.berkeley.edu
but don't change your login shell (it's /bin/bash).
Here is what a typical login looks like from a UNIX system (you type the
words in bold font):
umlcomputer [42] ~ # ssh -X csuml-aa@icluster1.eecs.berkeley.edu
The authenticity of host 'icluster1.eecs.berkeley.edu (169.229.3.99)' can't be established.
RSA key fingerprint is dc:4e:4f:bf:02:7a:81:d6:7f:11:21:4a:db:47:b9:40.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'icluster1.eecs.berkeley.edu,169.229.3.99' (RSA) to the list of known hosts.
csuml-aa@icluster1.eecs.berkeley.edu's password: *******
Last login: Mon Oct 19 13:46:52 2009 from cory.eecs.berkeley.edu
Linux icluster1 2.6.18-6-amd64-perfctr-2.6.31-9 #1 SMP Tue Feb 12 10:16:40 PST 2008 x86_64 GNU/Linux
icluster1 [1] ~ #
|
You can also login from Windows computers with the ssh program called
Putty.
Testing mapreduce within scheme:
Skilled student programmers working with our CS61A instructor have
embedded mapreduce functions into a UC Berkeley version of scheme, which
is started with the UNIX command "stk-simply". Here are some sample
commands:
icluster1 [1] ~ # stk-simply
Welcome to the STk interpreter version 4.0.1-ucb1.3.5 [Linux-2.6.24-7-amd64-perfctr-2.6.34-2-x86_64]
Copyright (c) 1993-1999 Erick Gallesio - I3S - CNRS / ESSI
Modifications by UCB EECS Instructional Support Group
Questions, comments, or bug reports to .
STk> (mapreduce (lambda (input-key-value-pair)
(list (make-kv-pair `line 1))) + 0 "/gutenberg/shakespeare")
Mapreduce in progress! Your ID number is 1029551924. For progress info, see
http://icluster1:35702/jobdetails.jsp?jobid=job_200910191241_0020
Working....................
((line . 112251) . #[promise 2b5fa935fc48 (not forced)])
STk> (kv-value (stream-car
(mapreduce (lambda (input-key-value-pair)
(list (make-kv-pair `line 1))) + 0 "/gutenberg/shakespeare")))
Mapreduce in progress! Your ID number is 1410030225. For progress info, see
http://icluster1:35702/jobdetails.jsp?jobid=job_200910191241_0021
Working..................
112251
STk> (define gutenberg-wordcounts
(mapreduce
(lambda (input-key-value-pair) (list (make-kv-pair `line 1)))
+ 0 "/gutenberg/shakespeare"))
Mapreduce in progress! Your ID number is 745377083. For progress info, see
http://icluster1:35702/jobdetails.jsp?jobid=job_200910191241_0023
Working......................
gutenberg-wordcounts
STk> (match? `(* hard * night *) `(a hard days night))
#t
STk> (quit)
icluster1 [2] ~ #
|
Diagnostic information is available on the WEB sites listed in the output
of the mapreduce commands. To view those pages from your computer
in Lowell, you'll need to change the URL from "http://icluster1" to
"http://icluster1.eecs.berkeley.edu".
Finding your files in the Hadoop directory:
References:
Kevin Mullally
Feb 11, 2011