Cowan was picked by Democratic Gov. Deval Patrick to serve as interim senator until after a special election June 25. Cowan had stepped down as Patrick's top aide in December.
"This might be the shortest political career one has, but I look forward to it," he said. "It's definitely a thrill to be part of the U.S. Senate."
He is the state's second African-American senator. Republican Edward Brooke served from 1967 to 1979.
With Cowan and Sen. Tim Scott, R-S.C., there are now two African-American senators serving together for the first time. Scott, who was also a gubernatorial appointee to his seat, greeted Cowan on the Senate floor shortly after he was sworn in. Cowan said he hoped to work across the aisle with Republicans like Scott.
The primary is scheduled for April 30. Reps. Edward Markey and Stephen Lynch have declared their candidacies. Several Republicans are considering joining the race following decisions by higher profile Massachusetts Republicans to pass, including former Sen. Scott Brown, former Gov. William Weld, former state Sen. Richard Tisei and former Lt. Gov. Kerry Healey.
"It's for the people to decide, ultimately, who serves them long term in the Senate," said Cowan, who has made it clear he won't run in the special election.
Cowan grew up in North Carolina and graduated from Duke University and Northeastern University's law school. He was a partner in the prominent Boston law firm of Mintz Levin before going to work for Patrick.
Cowan has said his mother, Cynthia Cowan, who attended the swearing-in ceremonies, was a child of the segregated south who raised him and his sisters alone working as a seamstress after his father died while he was a teen.
"Days like today are what my mother spoke of when I was a kid, that if you worked hard and did the right things and you treated people well, anything could happen," Cowan said.
Asked if he would be filing any legislation, Cowan replied that it was just his first day on the job.
"I just discovered the 'Senate Only' elevator, so give me time," he joked.