In your old mail system, you might have organized your mail messages in folders. If you had a lot of different categories of messages, you probably nested your folders in a hierarchy. That way, you could find a message by navigating and opening your folders.
In Gmail, you can do the same thing. Only instead of putting messages in folders, you associate your messages with labels. Labels are actually more powerful than folders in that a single message can be associated with more than one label. (By contrast, we bet that in your old system you could put a message in only one folder!) As a result, in Gmail, you can find a message based on any one of its associated labels.
Now, we generally like to find our messages by searching, so we often don't bother building elaborate label hierarchies. But if you do want to make your label structure look like your old folder hierarchy, you can. That way you can still find messages via search, but you can visually navigate your label hierarchy, too.