EECS Instructional Support, University of California at Berkeley
[ ISG home page ] [ who we are ] [ send us email ] [ search ] [ FAQ ]

                          College of Engineering
                     EECS Instructional Support Group



								July 24 2007

              EECS Instructional Computing - Review and Plans
	      -----------------------------------------------
		               Spring 2007


CONTENTS:
	Mission Statement
	Recent Improvements 
	Current Initiatives
	Parellel Computing Cluster
	Notable Events



Mission Statement
-----------------

  The EECS Instructional Support Group (ISG) installs and maintains networked 
  computers that are used by EECS classes.  ISG provides computer accounts for 
  instructors and students in the Instructional labs and on Instructional 
  servers.  ISG purchases, installs and maintains application software needed 
  for classes.  ISG supports instructional labs in Cory Hall, Soda Hall and 
  Hearst Field Annex.

  These are the functions in which ISG interacts with other UCB support groups:

  - we obtain enrollment lists from the Registrar (Student Information Services)
  - we synchronize our user accounts with the EECS department (IDSG)
  - we provide cardkey pre-authorization for our students to EECS Facilities
  - we bill students' voluntary printer charges to CARS
  - we coordinate our use of the EECS Network Node Bank with IDSG and CNS
  - we manage the computers in engineering labs with ESG 
  - we manage the computers in EECS conference rooms with ESG 
  - we manage the licenses for Synopsys/TCAD/HSPICE with the Device Group
  - we manage the licenses for Cadence with the BSAC group
  


Recent Improvements 
-------------------

  Due to fiscal limits, we did not make any major computer or infastructure 
  purchases in FY 20006/2007.

  - purchased software licenses for Shake on Macs in 349 Soda (for CS198)
  - purchased 50 new chairs for labs in 330 Soda and 119 Cory



Current Initiatives
-------------------

  - received grants from Google and Intel for a parallel computing cluster
  - requested AEG grant from SUN for 24 new worksatations in 275 Soda 
  - purchased 32 20" LDCs to replace old CRTs in 105 Cory (EE20N, etc)
  - purchased 32 new USB kbds for 275 Soda (CS61B, etc)
  - purchased 96 new USB mice for 271/273/275/330 Soda
  - purchased 2 new printers to replace 3 old ones in 274 Soda



Parellel Computing Cluster
--------------------------
  With generous grants from Google and Intel, we are planning to purchase this
  equipment for an Instructional Parallel Computing Cluster:

  - 26 Dell PowerEdge 1950 (each with 2 quad-core 2.33GHz Xeon, 8GB RAM)
  - a rack and a gigabit switch for the cluster internal network

  The cluster nodes will run Linux and the Nutch/HaPoop implementation of the 
  GFS distributed filesystem.   User applications will include a public-domain 
  version of the map-reduce parallel processing tool, which is implemented in 
  Java.

  We hope to have this operational by October 2007.  Classes that will use it
  include CS61A (Harvey), CS198 (Garcia) and CS194 (Yelick).


Notable Events
--------------

  See http://inst.eecs.berkeley.edu/notices.html for current events.

  ---------------------------------------------------------------------------
  Apr 6 - This server was down today
  
  	The Instructional WEB server http://inst.eecs.berkeley.edu was 
  	down today because of a disk failure.  We reinstalled it with a new 
  	disk by 3:30pm.  WEB sites that require logins were fixed by 11pm.
  	
  	The server hosts most EECS class WEB sites as well as student WEB 
  	sites and our tech support references.
  ---------------------------------------------------------------------------
  Mar 20 - EECS network was down, 2:45pm-7pm
  
  	A core router failed at about 2:45pm.  It cut off Cory and Soda Halls 
  	from each other and from the outside.  Computers in the Instructional 
  	labs could not reach most servers, which caused varying degrees of 
  	failure.  For details, please see
  	https://iris.eecs.berkeley.edu/news/2007/0320-Network_Failure_repo-1526.shtml
  
  	Some computers froze so that the users could not logout.  We logged 
  	those users out so that the contents of their accounts are safe.
  
  	Network service was restored at 7pm, and residual effects were cleared 
  	from our computers by 10pm.
  ---------------------------------------------------------------------------
  Feb 7 - Exceed (XWindows) Security Alert
  
  	X Windows server programs such as Exceed have been targets for 
  	password-sniffing and other security holes recently.  If you 
  	are running an X Windows server on your computer at home or in 
  	an office on campus, please read this excellent article for 
  	instructions about using it securely:
  
  	https://kb.berkeley.edu/jivekb/entry.jspa?externalID=1185
  ---------------------------------------------------------------------------
  Jan 2 - UNIX and WEB site logins were disabled, Jan 1 & Jan 2
  
        Logins to <a href="cgi-bin/clients?choice=13">Instructional UNIX systems</a> 
	and <a href="https://imail.eecs.berkeley.edu">email server</a> were 
        disabled from 0000 hours on Jan 1 through about 11O0 hours 
        on Jan 2.  A certificate had expired on our password server.
  ---------------------------------------------------------------------------
  Nov 24 - imail.eecs email server was down Thu & Fri
  
  	The imail.eecs email server went down sometime after 10pm on 
  	Wednesday Nov 22.  Email service was restored at about 6pm om
  	Fri Nov 24.
  
  	While imail.eecs was down, email that was sent to it was queued at 
  	the source and was delivered when imail.eecs accepted email again.
  ---------------------------------------------------------------------------
  Oct 9 - empty INBOX?  your email is on http://imail.eecs.berkeley.edu
  
  	(Oct 9 2006)  We have stopped exporting /var/mail to all computers.
  	If you use 'pine' on UNIX and find that your INBOX is empty on the
  	Instructional computers, you probably need to update the way you
  	read email.   
  
  	You can simply logon at http://imail.eecs.berkeley.edu
	</a> 
  	to read and send email via the EECS Instructional email server.
  
  	To set up spam filtering, run this command in your UNIX account:
  	/share/b/adm/bin/instmail.pl
  
  	You can also configure the email client of your choice to connect to
  	the EECS Instructional email server.
  
  	Please see http://inst.eecs.berkeley.edu/connecting.html#email for 
  	more information.
  ---------------------------------------------------------------------------
  Oct 9 - imail.eecs email server was down; cory.eecs was stalled
  
  	Sometime after 11am on Sunday Oct 8, the imail.eecs email server
  	crashed.   The problem was corrected at 8am on Monday morning.
  
  	While imail.eecs was down, email that sent to it was queued at the 
  	source and delivered when imail.eecs started accepting email again.
  
  	A side effect was that logins on cory.eecs were frozen from about 11am
  	though 11pm.  This was because an NFS filesytem from imail.eecs was 
  	mounted to cory.eecs so archaic email programs can be run there.  
  	Cory.eecs itself and the license servers there were OK.  
  ---------------------------------------------------------------------------
  July 31 - server downtimes: Samba, email, Synopsys/TCAD/Hspice licenses
  
  	We plan to have these servers down for maintenance in the next 2 weeks:
  
  	<u>mamba.cs - Tuesday Aug 1, 11am-3pm</u></b>
  	Mamba is our <u>Samba</u> server, which allows Instructional UNIX home
  	directories to be accessed from Windows accounts.
  
  	<u>cory.eecs - Friday Aug 4, 11am-3pm</u></b>
  	Cory is a general-access UNIX login server for EECS Instructional
  	accounts, and it is the <u>license server</u> for Synopsys, HSPICE 
  	and TCAD that are used by all EECS users.  While this server is down, 
  	these linceses will be unavailable.
  
  	<u>imail.eecs - Thursday Aug 10, 11am-3pm</u></b>
  	Imail is the Instructional <u>email</u> ("IMAP") server.   While this 
  	server is down, you will not be able to read on Imail or send email 
  	from the computers in the Instructional labs.  Incoming email will be 
  	queued for delivery once the server is back.
  
  July 27 - cory.eecs downtimes: July 26 (unintended), July 28 (planned)
  
  	cory.eecs shut down for unknown reasons sometime after 9:30pm on 
  	Wednesday July 26.   It was rebooted at 10:30am on July 27 and 
  	appears to be fine.
  
  	cory.eecs will be down for a scheduled systems upgrade on Friday
  	July 28 from about 1pm-6pm.
  
  	cory.eecs is a general-access UNIX login server for EECS Instructional
  	accounts, and it runs license servers for Synopsys, HSPICE and TCAD
  	that are used by all EECS users.
  ---------------------------------------------------------------------------
  July 24 - UNIX logins were broken on July 22, 4:45pm-7:45pm
  
  	On Sat July 22 for about 3 hours (4:45pm-7:45pm), login sessions on
	Instructional UNIX computers (http://inst.eecs.berkeley.edu/~inst/iesglabs.html)
	either stalled or were interrupted with error messages such as
  
  		starting common desktop environment
  		not able to find /share/b/bin/instructional.cshrc
  		OS: Undefined variable
  
  	This was caused by the crash of the "/share/b" file server, which has
  	some startup scripts (instructional.cshrc, etc) on it.  The server was
  	rebooted at 7:45pm.
  ---------------------------------------------------------------------------
  July 17 - this WEB server will be down Friday July 21 3pm-6pm
  	This WEB server (http://inst.eecs.berkeley.edu) will be down 
  	from 3pm-6pm on Friday July 21 do update the server software.
  
  	The class WEB sites will be unavailable during that time.
  	The instructors have been notified.
  	Only EE100 has a scheduled lab at that time.
  ---------------------------------------------------------------------------
  July 12 - unable to use Firefox or Mozilla on UNIX?
  
  	We are upgrading our SUN UNIX systems to Solaris 10, and the new 
  	versions of firefox and mozilla are incompatible with the older
  	configuration settings.
  
  	So if you get no response when you try to start firefox or mozilla,
  	you should delete (or rename) your ~/.mozilla/firefox directory.  It
  	will be re-created the next time you run the browser.  You can rename
  	it with this UNIX command (you can delete it later):
  
  		mv ~/.mozilla/firefox ~/.mozilla/firefox-old
  
  	You can verify that the computer you are on is running Solaris 10
  	with the UNIX command:
  	
  		uname -a
  
  	You can see a list of all of our login servers and their operating
  	systems at 
	http://inst.eecs.berkeley.edu/cgi-bin/clients.cgi?choice=servers.
  ---------------------------------------------------------------------------
  June 20 - Cory Hall will be closed overnight on Wednesday-Thursday 
  
          The electric power in Cory Hall will be shutdown 
  	from 6pm on June 21 through 6am on June 22.  
  	
  	Cory Hall will be closed to everyone during that time.
  
  	Impact on Instructional computers:
  
  	- Cory Hall labs will be down and inaccessible after noon Wednesday
  	- These servers will be down from 4pm on June 21 through 2pm on June 22:
  
            cory.eecs, pulsar.eecs, quasar.eecs, c199.eecs, iserver1.eecs (UNIX)
  	  iserver2.eecs, kramnik.eecs (WINDOWS)
  
  	NOT affected:
  
  	- Instructional email, WEB and home directory servers (they are in Soda)
  	- Instructional servers: nova.cs, star.cs, solar.cs
  	- Instructional labs in Soda Hall
  	- Instructional e-Academy site (msdnaa.eecs)
  
  ---------------------------------------------------------------------------
  June 6 - new SSH keys ("WARNING: HOST IDENTIFICATION HAS CHANGED!")
  
  	We have updated our version of SSH and installed new "host keys" on the
  	<a href='/cgi-bin/clients.cgi?choice=13'>Instructional UNIX servers</a>.
  
  	You should go ahead and accept our new server host keys.
  
  	If you are using your own SSH passphrase, you may also have to convert 
  	your public key on our servers from SSH2 format to OpenSSH format.
  
  	Here are instructions:
  
  	Host Keys
  	---------
  	When you login using SSH, your local computer stores a copy of the
  	server host key.  When the hostkey changes on the server, SSH says:
  
  		WARNING: HOST IDENTIFICATION HAS CHANGED!
  
  	If you are using "SSH Secure Shell" on Windows, you can just answer
  	"Yes" to the question "Do you want to save the new host key?"
  
  	If you are using OpenSSH (usually on MacOSX or Linux), you may have 
  	to delete your old host key manually.  In the WARNING message, look 
  	for a line such as
  	
  		Offending key in /Users/JohnDoe/.ssh/known_hosts:2
  
  	That tells you that your list of host keys is stored on your local 
  	computer in a file called /Users/JohnDoe/.ssh/known_hosts, and that
  	you should delete line 2 (the old host key) from that file.  Then 
  	run 'ssh' again and accept the new host key when it asks you.  
  
  	You can confirm that this new host key is valid by looking up 
  	that server under https://inst.eecs.berkeley.edu/~inst/hostkeys, 
	or by asking inst@eecs.berkeley.edu.
  
  	Public Keys
  	-----------
  	SSH has a feature that lets you set your own password ("passphrase"),
  	which is represented as a private key on your local computer and a
  	matching public key on the remote comuter (ie, our server).  The keys
  	are text strings that are stored in files.
  
  	Most Instructional users do not use public keys or their keys have
  	already been converted.   
  	
  	If your passphrase worked recently but fails to work now, it may be
  	because your public key is in the old "SSH2" format an needs to 
  	converted to the new OpenSSH format.  This will most often be the
  	case if you are using the "SSH Secure Shell" client on Windows, which 
  	is an SSH2 version.
  	
  	You can convert the public and private keys in your Instructional UNIX 
  	account on by ssh'ing into torus.cs.berkeley.edu (it still accepts the 
  	old SSH2 keys) and typing
  
  		/bin/ssh-keygen -X -f ./ssh2/your-ssh2.pub > .ssh/authorized_keys
  		
  		foreach id (`awk '/IdKey/ {print $2}' .ssh2/identification`)
  			/bin/ssh-keygen -X -f .ssh2/$id >> .ssh/identity
  		end
  	
  	(Replace "your-ssh2.pub" with the file name of the public key that you 
  	are using.)
  ---------------------------------------------------------------------------
  June 1 - computers down on Wed June 7, 7am-9am
  
  	Computer services in EECS may be disrupted on Wed June 7, 7am-9am
  	during maintenance on the main department servers.   Login sessions
  	on Instructional UNIX computers may stall while the /usr/sww file-
  	system is offline.  This may also stall the Instructional WEB server 
  	and email servers:
  
  	http://inst.eecs.berkeley.edu
  	http://imail.eecs.berkeley.edu
  
  	(This work was originally planned on May 31 and was postponed.)
  
  ---------------------------------------------------------------------------
  Apr 18 - empty INBOX?  your email is on cory.eecs.berkeley.edu
  Apr 16 - UNIX logins froze with "NFS server imail not responding" error
  
  	(April 2006)  We have stopped exporting /var/mail to all computers 
  	EXCEPT cory.eecs.berkeley.edu.  If you use 'pine' on UNIX and find 
  	that your INBOX is empty on other Instructional computers, login 
  	into cory.eecs (using 'ssh') and run pine there. 
  
  	The prefered way for users to read email now is directly from the 
  	email server using the IMAP protocol.   You can do that either by 
  	logging into our SquirrelMail WEB mail program 
	(https://imail.eecs.berkeley.edu) or with an IMAP email client such 
	as pine or Outlook.  
  
  	If your email is not configured for IMAP, you can run this command on 
  	an Instructional UNIX computer:  /share/b/adm/bin/instmail.pl</b>
  	It will create new configuration files and folders (but it won't 
  	destroy any existing ones) so that pine will log you into the mail 
  	server and your email folders will be accessible by SquirrelMail and
  	pine.  Please see http://inst.eecs.berkeley.edu/connecting.html#email
  	for some details.
  
  	We used to export the /var/mail filesystem from imail.eecs to all of 
  	the Instructional UNIX computers, so that users' raw email folders
  	there were accessible as UNIX files.   This was needed for older email
  	programs such as mailx and pine without IMAP, but now it is as
  	unnecessary as having a file sharing connection to a WEB server so
  	you can use an editor instead of a WEB browser to read the files.
  	
  	We are deprecating the "/var/mail" filesystem because if its server
  	goes down,  ALL of our UNIX login sessions to freeze while waiting 
  	for it to come back.   This is a bad side effect of the UNIX NFS 
  	file sharing protocol.
  
  	For example, our Instructional email server "imail.eecs.berkeley.edu" 
  	was down from about 4pm-5:30pm on April 16.  (The email server had
  	been overwhelmed by large volumes of mail which caused it to run out 
  	of virtual memory and crash.)  We were still exporting /var/mail from
  	imail.eecs to all of our UNIX systems, so during this time, all UNIX 
  	login sessions were frozen, even though most of the users were not 
  	even trying to read their email!  After that incident, we decided 
  	that it was time to remove this risk and to teach our users to use 
  	the IMAP mail programs instead.  
  	
  	We have left cory.eecs as the only computer that still has /var/mail.
  	That foes enable legacy programs that are used to notify you of new 
  	email when you login.  Those programs no longer work on our other
  	UNIX computers.
  --------------------------------------------------------------------------- 
  Apr 14 - Intermittent Network outages this week
  
  	Faulty network equipment caused intermittent outages in EECS from 
  	Apr 11 - Apr 13.  Network interruptions to the servers cause symtoms 
  	such as delayed responses to commands you type, loss of access to 
  	your home directory and possibly frozen login sessions.   For more 
  	symptoms, see below
	"Symptoms when UNIX email or home directories are missing".
  
  	The EECS netork staff reported on April 11:
  	
  	Late this afternoon our core router in Soda Hall had a hard failure
  	on one of the management switch modules. When the router tried to
  	fail over to the backup module, it also, exhibited a failure. I have
  	rebooted the first module and network service has been restored.
  	However, there may still be a problem and we are addressing this now
  	with the vendor. Further outages  may occur without warning until we
  	get this resolved. I regret the inconvenience this may cause.
  
  	This announcement, and any updates, are available at:
	https://iris.eecs.berkeley.edu/news/
  ---------------------------------------------------------------------------
  Mar 20 - Why this WEB server is slow today
  
  	We have detected an extreme number of hits on the http://inst.eecs
  	WEB server from a site in India today.  It appears to be downloading 
  	a specific course WEB site.  Although it is apparently not malicious, 
  	it has caused delays of several seconds for some WEB pages to be
  	displayed.  We will cut off access from that site if needed.
  ---------------------------------------------------------------------------
  Feb 28 - Cory & Soda Hall power failure, 8am-9am
  
  	Cory and Soda Halls lost electrical power from about 8am-9am.
  	All Instructional computers were back up by 9:45am.
  	Instructional printing was resumed at about 12:45pm.
  ---------------------------------------------------------------------------
  Spring 2006 - Printer usage quotas are now in effect
  
  	Instructional UNIX and Windows accounts now have page limits on the
  	Instructional shared printers in Soda, Cory and Hearst Field Annex.
  
  	The print quotas this semester are computed at 50 pages for each
  	course credit that the account is being used for.  In addition, 
  	students who are EE or CS undergraduate of graduate majors are 
  	given an additional 50 pages per semster on their print quotas. 
  	The print quotas are reset each semester.  Unused pages will NOT 
  	be credited to you in future semesters.
  
  	If you exceed your print quota, your next print job will be replaced
  	with a "QUOTA EXCEEDED" page.   That page explains that you can logon
  	to http://inst.eecs.berkeley.edu/webacct to view your print quota
  	allocation and to purchase additional pages.  Pages that you purchase 
  	will be billed to you, at a rate of $12 (non-refundable) for 200 pages.
  
  	Reference:
	http://inst.eecs.berkeley.edu/cgi-bin/pub.cgi?file=html/printquota-sample.html
  ---------------------------------------------------------------------------
  Spring 2006 - Forgot your password on the Instructional computers?
  
  	<u>For named</b> accounts:</u>
  
  	Login as 'newacct' (password 'newacct') again (in 199 Cory, 273 Soda or 
  	'ssh' to cory.eecs.berkeley.edu).  Enter your Student ID number.  Then 
  	select the new "p" option for resetting your password and reprinting a 
  	form.  The form will be printed the next weekday and will be available 
  	in 253 Cory after 1pm.
  
  	Named</b> accounts look like 'gbush' or 'bclinton'.
  
  	<u>For class</b> accounts:</u>
  
  	Go to the Instructional sys admin staff in 333 Soda, 378 Cory or 
  	386 Cory.  Bring your initial class account form or student ID card.
  	There is no on-line procedure for resetting the password of a class 
  	account.
  
  	Class</b> accounts look like 'cs61a-aa' or 'ee141-agore'.
  
  ---------------------------------------------------------------------------
  Fall 2004 - Security alert: Internet Explorer Active scripting is disabled
  
  	Users of Internet Explorer may be prompted repeatedly for permission 
  	to run scripts on the page.   This is because the sys admins have 
  	disabled the "Active scripting" feature of Internet Explorer on most 
  	EECS Instructional Windows computers because of a security flaw for 
  	which there is no patch yet.
  
  	Users are advised to use a different WEB browser (Mozilla, Firefox 
  	or Opera) to avoid this.
  
  	For information about the security flaw, please see 
	http://www.kb.cert.org/vuls/id/842160
  ---------------------------------------------------------------------------
  May 2002 - your LOST+FOUND directory
  
  	Mamba.cs, the Instructional UNIX home directory server, crashed on
  	April 29 and was down for 33 hours.   For an explanation of the 
  	circumstances, please see the Spring 2002 Managers' Report
	(https://inst.eecs.berkeley.edu/~inst/reports/?file=Spring_2002).
  
  	A number of files and home directories were not restored to their
  	proper names and locations.   When we couldn't determine where the 
  	files should go, we put them a directory called LOST+FOUND in your 
  	UNIX home directory.   Please see that file for an explanation of 
	how you can read these files, and ask us for help 
	(inst@eecs.berkeley.edu) if needed.
  </pre>
  
  ---------------------------------------------------------------------------
  <u>Symptoms when UNIX email or home directories are missing</u>:
  
  	- when you try to login the screen freezes 
  	- you see the error message "home directory is /" 
  	- session hangs up if you try to 'ssh' into an Instructional computer
  	- unable to read WEB pages from the http://inst.eecs.berkeley.edu 
  	- lots of annoying "NFS timeout" error messages on your screen
  	- new email deliveries will be delayed on imail.eecs 
  
  	While the server is down, you may not be able to logout in our labs
  	because you can't type any commands.  On a SunRay, even turning it
  	off doesn't log you out.  The support staff check the labs after 
  	events like this to be sure everyone gets logged out.  We also post 
  	information about the problem at http://inst.eecs.berkeley.edu to 
  	help students find out when the problem has been fixed.   So all you 
  	can really do in this case is to wait until the problem is fixed, go 
  	back to the lab (or login to the SunRay server for that lab) and log 
  	yourself out, or let us log you out.
  
  	We disable email receipt and relaying through imail.eecs when the
  	home directory server (mamba.cs.berkeley.edu) is down.  No mail is 
  	lost.  Computers that send mail queue messages that are not accepted
  	by a remote server, and they resend the messages periodically until 
  	they are received.
  ---------------------------------------------------------------------------

For additional information, please contact me:
  
  Kevin Mullally, ISG Manager
  EECS Instructional Support Group
  378 Cory Hall, (510) 643-6141
  kevinm@eecs.berkeley.edu
  http://inst.eecs.berkeley.edu/


source: ~inst/public_html/reports/managers/Spring_2007