In this segment we are going to give a brief history of computer graphics as part of our initial lecture on an overview and history.

The term computer graphics itself was coined by William Fetter of Boeing in 1960. The first graphics systems were developed in the mid 1950s, the US Air Force SAGE radar systems, and they were based in the development at MIT. I strongly encourage you, if you have an opportunity, to look at the video on the history of computer graphics, which was released by SIGGRAPH for its 25th anniversary, which although somewhat dated, has an excellent introduction to the early part of the field.

One of the interesting things I always like to do in this overview is to mention to people that text is in itself a major development in computer graphics. If you read Alan Turing's biography when he was developing the Manchester Mark I, he goes into a lot of details about the future potential of computers. Things like chess playing machines. Things like even compilers, to high level programming languages, but somehow even the notion of text escapes him.

The Manchester Mark I, in its display essentially had a bank of LEDs. And so, you're just looking directly at the memory and trying to figure out what happens. So going from that, to our actual textual displays has been a major advance. And the way in which fonts are designed and the way in which they are displayed in the scene, on the screen in order to create a smooth appearance are some of the very interesting early problems in 2D computer graphics.

Of course, textual representations haven't been the main way in which computers have been used for many decades. And the major advance has been the notion of the graphical user interface. The graphical user interface has an interesting history. It was invented at Xerox Palo Alto Research Center in the 1970's, around 1975.

But Xerox did not actually benefit the most from the development of the graphical user interface. So it was first used in the Apple Macintosh, and subsequently, has become a standard for use of any computer, on any computers anywhere.

Drawing was one of the early applications of computer graphics. Ivan Sutherland's sketch pad system, which was developed to MIT in the 1960s, and was way ahead of its time. It introduced many of the concepts for drawing in current systems. Things like pop up menus, constraint based drawing, hierarchical modeling. I've included a link there for a video that shows Sutherland's sketch pad system. I would very strongly encourage you to look at that after this lecture.

Sketchpad can be considered the first interactive computer graphics system and in fact the first PHD thesis in computer graphics. And it really set the stage for what computer graphics could do.

Following that, there was also a lot of interest in the development of paint systems. And the SuperPaint system, which was developed by Richard Shoup and Alvy Ray Smith, again, at Xerox's Palo Alto research center in the 1970s.

I've taken these images from the excellent introduction on the website which I've noted here, which Richard Shoup has put together.

And this SuperPaint system really presages modern software technologies, things like Adobe Photoshop. In fact, Adobe was one of the early start-ups in computer graphics, and nowadays of course, we don't think twice about applying general manipulations to images.

Image processing has been a very significant aspect of what we see on the screen. We rarely see images that have not been processed in some way. So you can alter images by cropping, scaling, compositing them. You can even add or remove objects. Sports broadcasts for televisions combine 2D and 3D processing whether it's showing the world record line at the Olympics, or it's showing which swimmer is in each lane. And even showing different advertisements to different regions, all of these things are possible by smart image and video processing.

That's been a quick tour of 2D computer graphics. And I've gone over it just to make a point that 2D is not something we typically study in a graphics course anymore, but has really revolutionized the world we live in. And you cannot really imagine a world today without 2D computer graphics.

But most of this course is about three-dimensional computer graphics, and I just want to give you a brief history of modeling and rendering aspects of that.

Geometric modeling is the process of creating 3D geometry, which can then be used in a computer graphics system. In the '70s and '80s, one of the major developments was the creation of spline curves and surfaces.

And these were geometric methods that enabled the Utah Teapot, as it's known, to be created. And this was developed at the University of Utah. And was at that time a very interesting model because the teapot has lots of interesting geometry. It's not a genus 0 surface. It has a handle. It has a spout and all of this could be developed with spline models.

More recently there's been a lot of interest in acquiring 3D geometry from the real world and representing it as triangle meshes that are acquired from real objects. And so I've shown you this simple 3D scan I did several years ago of this cat sculpture, and you can see the actual triangular 3D mesh elements there. Nowadays we've had scans of statues going into billions of polygons. 

A very famous effort is by Professor Levoy at Stanford and collaborators the Digital Michelangelo Project where there were scans of many of the statues of Michelangelo including a 2 billion polygon model of the David. And geometry is getting evermore complicated, and evermore interesting.

Next we come to computer graphics rendering. And so in the 1960's the challenge was handling visibility. And there were a number of techniques which were proposed. So, this is hidden line algorithms, hidden surface algorithms.

So, what I'm going to show you here is an image where I've just drawn all of the lines on the objects.

And this leads to an interesting kind of image because you see there is no real notion of which object is in front of which object. For example this is the torus and if I look at it and I look at the cube or I look at the teapot and I look at the wall, you can see that the teapot should actually be blocking the wall but I can see the line of the wall through the teapot. So I have no real way of interpreting which object is in front of each other.

That immediately leads to the problem of hidden line elimination. I want to eliminate the lines that I cannot see. And this is a challenge in art as well to create images that convey the notion of depth perception.

So this was a problem in the 1960s, and Roberts and Appel worked on it, and came up with something that looks, now, a lot more interesting because you can actually perceive depth. You can say the cube is in front of the wall, the teapot is in front of the wall. Let me just do a comparison. So here it's very hard to even say which object is in front of which one, whereas here the hidden lines have been eliminated, and you have much better depth perception.

And of course, hidden line elimination is one aspect of it, you can also extend it to hidden surface elimination. And so, this is addressing the fundamental challenge of visibility. Which object is before which object. Sutherland in 1974 wrote a famous paper that reduce visibility to sorting and compared a number of different visibility algorithms in this way.

Of course once I've done this, I can also color the objects. But so far notice that they've just colored them in a constant color. And in particular, this image still looks, not so great. Look at this cone here. And what we see in this cone is you can clearly see the different facets of the different triangles which are being used to approximate the cone. Similarly, for this sphere you can see the polygonal outlines of the, of the triangles and the geometry that's being used to approximate the sphere. So this is not ideal yet.

And therefore in the 1970s the challenge was, how can we get nice, smooth shaded image of this geometry. So I'm going to start with the image that I showed you earlier where you can clearly see the polygonal outlines. And, Henri Gouraud at the University of Utah in 1971 had a way to deal with these polygonal outlines and to interpolate the shading so that it would look smooth.

So this is what is known as Gouraud shading. It's still used in OpenGL. You know it is smooth shading and suddenly the images look a lot smoother. And in fact they look like real curved surfaces. Not our computer graphics representations of them.

However, they still looked like matte, or clay. And in fact they're just still a constant color. So the next innovation by Bui Tuong Phong was the notion of specular lighting, or Phong shading, and Phong illumination and Phong shading, models that still bear his name, are I would say still used in 99% of computer generated imagery.

What Phong did was enable the use of highlights, which now makes it appear much more realistic. And in fact gives this plastic appearance where you have both a diffuse, or body color component and the specular highlight.

Thereafter there's a lot of work. Jim Blinn and colleagues did substantial work on curved surfaces and texture. And in 1974 Ed Catmull introduced the notion of the z-buffer, or the hidden surface algorithm that is used nowadays in your graphics cards.

Those challenges having been solved in the '80s and '90s, the next challenge was what was known as global illumination.

Turner Whitted, in 1980, introduced the first recursive ray-tracing algorithm. It's what you will implement in homework 3, and you will, in fact, be able to produce images even better than this one. But transport yourself back to 1980 where you've just seen very simple computer graphic images, and some even have a bubble, which is refracting the surface, and you have reflections, refractions, all kinds of interesting visual effects. And this image from Turner Whitted is really canonical, and got many people excited about the prospect of computer graphics that it could really produce realistic imagery.

The second revolution was in the area of radiosity, wherein you could simulate the interaction between the wall and one of the boxes. So in this case the small boxes, so here and here. They are actually made of neutral gray material. So all of the color is actually reflected from the wall. So the blue wall here reflects energy onto the box, and similarly the red wall here reflects energy onto the box. And in this way they had this reddish and bluish tinge. This is a famous image known as the Cornell Box.

It's the Cornell Box and it was developed at Cornell University.

Thereafter, Kajiya in 1986 introduced something known as the rendering equation, which was a unified way to deal with many of the visual phenomena, such as the images on the left, from ray tracing or radiosity. And he produced this beautiful image, where all of the color again is coming from the caustics, from the green balls, the inter-reflections between the objects. And all of the objects, except for the green balls, are a neutral gray color.

And in this way he had a number of visual effects that people found very hard to do in a unified framework. And, indeed, much of the 80's and 90's involved ways to solve the rendering equation. And this challenge to consider global illumination of light from everywhere in the scene, remains a challenge to the current day.

Furthermore, our goal nowadays is to create photo-realistic images. That is, images that look just the way a real photograph would. And increasingly, the challenge is also to do it in real time. Real time photo-realism, and that is a frontier that now, for the first time, we are beginning to be able to reach.

Finally I've talked about the history of modeling and rendering. Of course the history of computer animation and of computer graphics in general is also very crucially important. I would highly recommend this clip, I've given the URL below, which has an excellent 10 minute clip from the video on the history of computer animation. And it covers some of the topics we've discussed. So in particular, sketch pad, animation, basic modeling and rendering. And really gives you a good synopsis of what 3D computer graphics and this course is about.