This video features a number of directors talking about the difference between viewing films in widescreen vs. pan and scan. Martin Scorsese:
[Converting to pan & scan] is, technically, re-directing the movie.
Update: Thoughts from David Lynch about pan and scan taken from a 1997 interview:
I would like to see everything done letterboxed and with great sound. I’m not too interested in doing the commentary, you know, like a lot of people do. But in some strange way I kind of like pan-and-scan. Because you see things. It is a compromise, and in a couple of things you really say, “Why am I doing it?” But it’s just an interesting thing that happens- another composition. It’s not so bad, but I wish really that people could see the thing in a theater and that laserdiscs and videos didn’t exist. Because on the big screen with the sound, you become inside the film, and that’s the beauty of cinema. And it never happens on video, and it doesn’t happen on laserdisc, either.