It was difficult because the majority of folks in the Colonies were farmers and were needed in the fields for their family to eat and survive the winter. Unfortunately, most fighting was done during the summer, when they were needed to plant, tend, and harvest.
Conditions also played apart since they were horrible. And with very little military bearing there was really nothing to keep Continental Soldiers at their posts.