At various times the Hawke, Keating and Howard governments looked done for but bounced back to win, he said.
"Those people who write the government off will be wrong," he told ABC television on Thursday.
In his last challenge against the prime minister, Mr Rudd was defeated 70 votes to 30, the biggest margin in the history of ALP contests, Mr Smith added.
"There's no vacancy, and Kevin Rudd has said that if anyone challenges - and no one is out there saying they're challenging - that he will be a human shield protecting the prime minister," he said.
The defence minister has held the seat of Perth, what he calls "the safest Labor seat in Western Australia", since 1993.
But he said he'd always treated it as marginal and won't leave a stone unturned as the coming election will be a real competition.
He's hopeful however that the true nature of the opposition leader will send votes Labor's way at the ballot box.
"My own judgment has always been that when people see the whites of Tony Abbott's eyes they'll come to the conclusion that he has neither the demeanour nor the character to be the prime minister of the country," he said.