Warning! Extreme geekery up ahead!
In a recent conversation I had with a friend, the subject came up about how a computer knows how to display certain colors for certain pixels. I've dappered around with Max/MSP, and in doing so understand at least the basics of how this works. It's like this:

Every pixel can be thought of as a cell in a spreadsheet. It contains 3 values. It's x-coordinate, y-coordinate, and a number to represent color. Let's say you have a Quicktime movie at 640x480 pixels, you have a data set for every frame, for essentially 307,200 pixels. If you have a movie that's 30 seconds long at a frame rate of 30 frames a second, that's a total of 276,480,000 pixels total to keep track of. Makes sense why video files take up so much space, right? We're not even talking about audio, or how compression works here.

So essentially, all a video "file" is, is a gigantic, multidimensional spreadsheet, that get's displayed over time and when the computer reads each cell in it, it displays the correct color in the correct x and y location. Of course I am skipping over all of the details, but that is the gist of it. For more detailed info, check out the The Max/MSP and (specifically) Jitter documentation, which, whether or not you're interested in the software, does a neat job of explaining various inner workings of computer software, at a digestible level.

Max/MSP and Jitter is some interesting software that provides a graphical interface for creating programs for your computer. In general, it allows you to skip a lot of the normal technical knowledge for connecting to peripherals, or operating system services, such as MIDI, audio, video, etc, and allows you to create "patches" that resemble electronic circuitry. After you pull out some hair, you can make some really, really interesting stuff with it.

Image from Wikipedia.