i found this on history.com so this is not my knowledge just wanted to help
"The concept of the poorhouse originated in England during the 17th century. Municipalities were expected to care for their poor, and made a distinction between people who were old and unable to care for themselves and the able-bodied. People who were able to work were expected to do so—and could be imprisoned if they refused."
Basically they would go to prison if they refused to work.
P.S. sorry if im too late
