Section 1 of Article 3 of the Constitution says:
The judicial Power of the United States, shall be vested in one supreme Court, and in such inferior Courts as the Congress may from time to time ordain and establish. The Judges, both of the supreme and inferior Courts, shall hold their Offices during good Behavior, and shall, at stated Times, receive for their Services a Compensation which shall not be diminished during their Continuance in Office.
The Constitution does leave open the possibility of impeachment and removal by the Congress. One justice was impeached, but not convicted, and another was essentially forced out under the threat of impeachment.