Jim Caldwell endured everything thrown at him during his first two seasons as the Colts' coach -- replacing a friend, rebounding from losses and fighting through injuries. Not having Peyton Manning around in 2011 was just too much. Caldwell was fired Tuesday, a little more than two weeks after the Colts' worst season in two decades.