What is this class about?
Driving Questions for the Course
How did the Wildebeests charge over the ridge in The Lion King?
How did the villagers in The Hunchback of Notre Dame wave and move in crowd scenes?
Both of these are cases where Disney moved from traditional drawn cel animation to:
- Modeling the character (villagers or wildebeests)
- Creating a bunch of them
- Simulating them
That's what this class is about: How do media professionals structure their use of the computer to capture the structure of the real world and its behavior?
Part 1: Introduction to Java
We start out by learning Java, in the context of media.
Images
We'll manipulate images by creating Picture methods like this:
/**
* Method to scale the picture by a factor, and return the result
* @param scale factor to scale by (1.0 stays the same, 0.5 decreases each side by 0.5, 2.0 doubles each side)
* @return the scaled picture
*/
public Picture scale(double factor)
{
Pixel sourcePixel, targetPixel;
Picture canvas = new Picture((int) factor*this.getWidth(),(int) factor*this.getHeight());
// loop through the columns
for (double sourceX = 0, targetX=0;
sourceX < this.getWidth();
sourceX+=(1/factor), targetX++)
{
// loop through the rows
for (double sourceY=0, targetY=0;
sourceY < this.getHeight();
sourceY+=(1/factor), targetY++)
{
sourcePixel = this.getPixel((int) sourceX,(int) sourceY);
targetPixel = canvas.getPixel((int) targetX, (int) targetY);
targetPixel.setColor(sourcePixel.getColor());
}
}
return canvas;
}
and use this code in our Interactions in DrJava like this:
> Picture doll = new Picture(FileChooser.pickAFile());
> Picture bigdoll = doll.scale(2.0);
> bigdoll.show();
> bigdoll.write("bigdoll.jpg");
to create transformations like this (yes, dolls on blue screen, for later use in simulations):
We'll also use turtle graphics to do drawing with relative motion.
Sampled Sounds
We'll do sound transformations (reversals, normalizing, growing software, growing shorter) for later use in structuring rhythms.
Music
We'll use JMusic to manipulate real music with multiple voices, instruments, and parts.
Part 2: Structuring our Media
Structuring Music
The idea is to be able to group notes into phrases and parts and whole songs, in the kinds of structures that allow for musical exploration.
public class MyFirstSong {
public static void main(String [] args) {
Song songroot = new Song();
SongNode node1 = new SongNode();
SongNode riff3 = new SongNode();
riff3.setPhrase(SongPhrase.riff3());
node1.repeatNext(riff3,16);
SongNode riff1 = new SongNode();
riff1.setPhrase(SongPhrase.riff1());
node1.weave(riff1,7,1);
SongPart part1 = new SongPart(JMC.PIANO, node1);
songroot.setFirst(part1);
SongNode node2 = new SongNode();
SongNode riff4 = new SongNode();
riff4.setPhrase(SongPhrase.riff4());
node2.repeatNext(riff4,20);
node2.weave(riff1,4,5);
SongPart part2 = new SongPart(JMC.STEEL_DRUMS, node2);
songroot.setSecond(part2);
songroot.show();
}
}
In order to produce this:
Click me to hear the result: composedsong.midi
We actually see this notation on the screen:
Images
What does it mean to change "layering" in an image? We'll explain this in terms of linked lists. Then we'll build a tree of images that will be easy to animate later.
We can use the order of picture elements in a list to determine ordering (linear) on the screen, instead of representating layering. This code:
> FileChooser.setMediaPath("D:/cs1316/MediaSources/");
> PositionedSceneElement tree1 = new PositionedSceneElement(new Picture(FileChooser.getMediaPath("tree-blue.jpg")));
> PositionedSceneElement tree2 = new PositionedSceneElement(new Picture(FileChooser.getMediaPath("tree-blue.jpg")));
> PositionedSceneElement tree3 = new PositionedSceneElement(new Picture(FileChooser.getMediaPath("tree-blue.jpg")));
> PositionedSceneElement doggy = new PositionedSceneElement(new Picture(FileChooser.getMediaPath("dog-blue.jpg")));
> PositionedSceneElement house = new PositionedSceneElement(new Picture(FileChooser.getMediaPath("house-blue.jpg")));
> Picture bg = new Picture(FileChooser.getMediaPath("jungle.jpg"));
> tree1.setNext(tree2); tree2.setNext(tree3); tree3.setNext(doggy); doggy.setNext(house);
> tree1.drawFromMeOn(bg);
> bg.show();
> bg.write("D:/cs1316/first-house-scene.jpg");
Generates this image:
Tweaking the ordering slightly like this:
> tree2.setNext(doggy); doggy.setNext(tree3); tree3.setNext(house);
> bg = new Picture(FileChooser.getMediaPath("jungle.jpg"));
> tree1.drawFromMeOn(bg);
> bg.show();
> bg.write("D:/cs1316/second-house-scene.jpg");
moves the dog between trees two and three, instead of between tree3 and the house.
That then generates this image:
Sampled Sounds and Rhythms
We want to be able to create complex rhythms by organizing pieces in trees for easy manipulation and recreation.
Animations
Animations are easy at this point – just write out our images after running through the list or tree.
Part 3: Simulations
We'll start out with continuous simulations: How do we understand the ecology of plants and animals, predators and prey, and simulate it on a computer?
Here's a simulation of disease moving through a population:
We start using UML to understand our growing programs
We'll then move to discrete event simulation. If something interesting isn't happening every moment, let's skip the moments where nothing interesting is happening!
If we're generating times randomly, we still have to make sure that real events occur in real time, so we have to deal with sorting event queues and dealing with different kinds of random.
We end up simulating with media results attached – not always wildebeests and villagers, but the same ideas. Here are two example simulation results of villagers attacking a "bad guy."
