I have that 2-volume set as well. I would agree that it is essential. Don't know if I'll ever get through all the letters, but I've read most of the major speeches, and they are AMAZING. You just can't appreciate Lincoln fully until you read his speeches, in my opinion. You really come to appreciate how phenomenal his mind really was.
Also, there's nothing like reading his early speeches to destroy the popular myth that ending slavery was not always his ultimate goal.
My only complaint about the Library of America collection is that it has no explanatory notes, nothing to give any context. For that reason, I use it as more of a reference work -- i.e., when I'm reading some other book about Lincoln, and mention is made of a particular letter or speech, I can go to the Library of America books and read the letter or speech in its entirety. But the L of A books assume that you've already got the context from some other source.