As the book starts off, a boy is making a trip to town with his dad. The boy worries about making his dad proud. The dad worries about his boy's perception of him. The dad is also incredibly lonely...and you begin to get attached to him, but you don't want to get too attached because you know he won't be around too much longer. (My apologies if this ruins the story, but I'm pretty sure this is well known.)
As you become introduced to the rest of the characters, you realize that everyone is honorable (for the most part) and that their internal concerns & worries just comes out of their being human. For example, though the father is lonely, it's not because he has a horrible wife. In fact, he has a wonderful wife...but he is trying to wade through the complexity of relationships, internal peace and life in general. And so are all the characters.
The story progresses slowly, predictably, and you watch as the characters all act & react in various ways. Agee allows you to see the world through each character's eyes, and then how the world perceives that character. This narrative style leaves you feeling compassionate for each member of the family whereas it would normally be easy to be annoyed with or dismissive of many of the characters.
It is also worth mentioning how honest and thoughtful Agee is in showing his characters' motivations and flaws. When the little boy learns that his father has died, the loss completely goes over his head. Instead, he tries to leverage this event for popularity from the schoolchildren. "Surely they will be nice to me today," he thinks.
Later, when the mother kneels to pray, her aunt has amazing insight that the mother is praying not from her heart but rather from pride - praying what she thinks a holy woman would pray in such a circumstance. Such commentary on our actions and motivations can hit a little close to home...but I believe it must have come from his own experience. He never condemns his characters...he just lets them be fully human.
As I said in the beginning, I believe this work to be a true masterpiece. Though I was often horrified by the true despairing picture he portrayed, my mind was also deeply impressed by the truth he conveyed. And he does give hope, after his own fashion.