The OR and mini rulebooks just listed everything as either Good or Evil and gave no significant restrictions on building your armies from either of those big pools.
The Legions of Middle Earth ( LoME ) book separated those two sides into several smaller "lists" and then set up specific rules on how forces would be made from those lists and how you may ally lists. This got rid of a lot of cheesy forces of people just selecting best-of-the-best warriors and heroes without regard for theme, got some control around time periods mixing and made the armies that were played far closer to what you could expect from Tolkien's work (interpreted by the people that brought you 40K and WH of course
)
You don't need LoME to play, and if you and some friends or family are all playing the same off of the base rule sets and maybe some scenarios in the journey books then don't worry about it. But if you plan on playing against anyone at a local GW or independent game store, hobby club, etc. then most of them will assume you are following LoME rules when building your force.
Due to the cost and the fact that I much rather put my money to models than rule sets, I ddn't get the LoME book for a very long time, but I did borrow it from a friend for a while and then referenced it often when we got together (my notes could have almost published the book again
) and when I had a chance I picked it up and was very happy I did so.