I've appreciated this particular quote on the subject:

"My brother has his sword, and I have my mind. And a mind needs books like a sword needs a whetstone; to keep it sharp."

- Tyrion Lannister, Game of Thrones