False. In the United States, all financial institutions are required to conduct business at a physical location only.
There are many financial institutions that conduct business online and over the phone not just at a physical location. There are also some banks that do not have a physical location at all and are only able to conduct business over the phone or online.
