Rebhorn, who had skin cancer, died at his home in South Orange, New Jersey, his agent Dianne Busch said.
His career spanned five decades and saw him appear in TV shows including The Good Wife and 30 Rock and films such as Meet The Parents (2000), My Cousin Vinny (1992), Carlito's Way (1993) and Basic Instinct (1992).
Diagnosed with melanoma in 1992, he continued working until last month.
His agent told The Hollywood Reporter: "He fought it all this time."
"He died [on] Friday afternoon at his home in New Jersey, where he had been receiving hospice care for a week and a half."
Rebhorn had more than 100 screen credits, one of the most recent being for Homeland, in which he played Frank Mathison, the father of Claire Danes' CIA officer Carrie Mathison.
His films included The Talented Mr Ripley (1999), in which he played a shipping magnate, plus The Game (1997) and Cold Mountain (2003).
Rebhorn often played authority figures, including the headmaster in Scent Of A Woman and the US Secretary of Defence in the 1996 blockbuster Independence Day.
The Philadelphia born actor also worked on stage, with theatre roles including Broadway productions of 12 Angry Men and Our Town.
Tributes have been paid to the prolific character actor.
Actor Jason Alexander, who played George Costanza in the long-running US sitcom Seinfeld, tweeted: "RIP James Rebhorn, a lovely man and a terrific actor."
Rebhorn played a district attorney in the show's last ever episode.
Actor Tom Arnold wrote on Twitter: "RIP. This actor was amazing. You didn't know him but trust me, you knew him."
Director Roland Emmerich, who worked with him on Independence Day, tweeted: "It's a sad day. I just learned James Rebhorn passed. Such a great talent and always a joy to work with. RIP."
Donnie Darko writer Richard Kelly tweeted: "I loved working with the great James Rebhorn on The Box. He was one of our most reliable and underrated character actors. RIP."
Actress Frances Fisher wrote: "Shocked and saddened to hear of James Rebhorn's passing. Jim was a consummate actor with whom I had the pleasure to work when we were young."
Comedian Mike Birbiglia who worked with Rebhorn on Sleepwalk With Me (2012) tweeted: "So sad to hear about the passing of James Rebhorn. He played my dad in Sleepwalk With Me. I feel so lucky to have known such a great person."
Rebhorn is survived by his wife Rebecca Linn and two daughters.
Our thoughts go out to his friends and family.