[He doesn't seem annoyed by it. Maybe a little tired, but not mad or anything.]

It's not that you're not helping. Just living here and making sure nothing comes back onto this island is already a lot, especially for how small the group is. And if you learn how to fight trolls, then eventually you could be able to clear out nearby areas yourself. But don't expect to save the entire world or anything. This world is beyond saving.