Forrester means that the people who are talking about his book don't have anything worthwhile to say. He seems to believe they are intellectually shallow. However, in Jamal, he sees the potential for original thinking and clever analysis.
Forrester doesn't leave his apartment, choosing to lock himself away from society. He doesn't value others thoughts and opinions because he doesn't think they have any merit. We see this with the man who brings Forrester his groceries.
