Ross added 25 points, 18 in the second half, and Celtics coach Brad Stevens said he was the difference.
"Terrence Ross changed the whole game," Stevens said. "When a guy catches fire, you've got to slow him down. We didn't get into his air space good enough and when we did, we fouled. When Ross got going it opened everything up."
Jayson Tatum missed a jumper at the buzzer that would have tied it for the Celtics, who charged back from a nine-point deficit in the final 1:43 before losing to the Magic for the second time this season.
Kyrie Irving scored 25 points for the Celtics, 10 of them from the foul line in the final 4:31. He appeared frustrated afterward with Gordon Hayward, who inbounded the ball on Boston's final possession.
"Next question," Irving said when asked about his reaction. "J.T. got a good look so let's move on."
Nikola Vucevic had 16 points and 13 rebounds for Orlando, which broke a four-game losing streak. He missed the second of two free throws with 3.2 seconds, leaving the Celtics with a chance to win or tie. Tatum took the inbounds pass and missed a 21-footer.
"Any time you have a shot to win at the end, that's all anybody focuses on," Stevens said. "That stretch at the end of the third and early fourth is what really gave them the momentum."
Boston fell behind 29-21 early in the second quarter after going 5 minutes, 45 seconds between field goals. But Jaylen Brown hit a couple of 3-pointers, and Tatum and Marcus Smart each hit one as the Celtics took command with a 23-5 run and led 51-41 at halftime.
Boston led 61-53 when Tatum went to the bench with four fouls midway through the third quarter. Vucevic and Ross scored nine points apiece in the period and Ross hit a 3-pointer in the final second to help Orlando cut the deficit to five.
"Obviously Terrence was instrumental there. When he gets going he doesn't need a lot of space," Magic coach Steve Clifford said.
Ross and Gordon then combined for 14 points in a 19-4 fourth-quarter run that regained a seven-point lead for Orlando with 5:57 remaining.
"That was huge," said Gordon, who made all 13 of his free throws. "Hopefully the next time the Celtics come to town we'll have more of a home crowd."
It was Boston's only visit of the season to Orlando.
Celtics: F-C Aron Baynes missed an 11th straight game with a fractured finger on his left hand. ... Hayward, an 88 percent free-throw shooter, missed his first three attempts. ... The Celtics missed nine free throws, seven more than Orlando. ... The Celtics lost the season series to the Magic for the first time since 2009-10.
Magic: Rookie C Mo Bamba, who missed a second game with a sore left foot, hopes to return by Wednesday. ... Khem Birch, who played 18 minutes in place of Bamba, had 10 rebounds and four blocks. ... The Magic have followed their season-high, four-game losing streaks with wins over San Antonio, Toronto and Boston.
Celtics: At Brooklyn on Monday night.
Magic: At home against Houston on Sunday night.
Roundball Rap: How NBA players are responding to the coronavirus pandemic
Also, is the season likely over?
Thunder's Danilo Gallinari helping fund COVID-19 test kits, protective equipment
Thunder forward Danilo Gallinari will help provide funding for 400 coronavirus testing kits as well as protective equipment such as masks and gloves.
Bridges leads Suns to victory in first NBA 2K game broadcast on radio
Mikal Bridges led the Phoenix Suns' NBA 2K team to a victory on Friday over Matisse Thybulle and the Philadelphia 76ers in a game that was streamed on Twitch and broadcast on the radio.