The public's apathy/ignorance certainly lets Congress off the hook since no pressure is applied to them to make anyone accountable. Setting that aside, why doesn't Congress act on their own without strong prodding from an angry public? Quite simple. In the case of Yellen and the Federal Reserve, the bank lobby has Yellen's back and literally has purchased probably 95% of Congress as well as the Presidency. A bought-and-paid for Congress combined with an ignorant and apathetic public leaves no room for things like truth, justice, or reform.