There are different reasons on why Gerald Ford and Jimmy Carter each lost popularity while serving as president.
Jimmy Carter was seen as a weak and ineffective president and he was wound up during the Iran Hostage situation while Gerald Ford pardoned Nixon which was very against the will of the large part of American people.