LONDON (Reuters) - Royal Bank of Scotland (RBS) has handed instant messages sent by a former currency trader to counterparts at other banks to Britain's financial regulator as part of its probe into the foreign exchange market, a source familiar with the matter said.

Investigations into the $5 trillion-a-day market have broadened with authorities in Switzerland and Britain looking into whether traders at banks sought to manipulate benchmark foreign currency rates.

RBS sent on the instant messages to Britain's Financial Conduct Authority (FCA) after deciding they were inappropriate. The action was taken following an internal probe, the source said on Wednesday, and the trader had left the bank before the messages were uncovered. His departure was not linked to the probe, according to the source.

RBS and the FCA declined to comment.

The FCA said in June it was probing whether dealers had manipulated foreign exchange benchmarks by trading ahead of their own customers' orders, a practice known in the markets as "front running".

RBS is one of a number of institutions to have sifted through instant messages and emails and have passed on information to watchdogs as banks feel the pressure of increased scrutiny of market conduct in the wake of the Libor interest rate rigging scandal, which has seen four institutions fined around $2.7 billion and seven men charged with fraud-related crimes to date.

Another source familiar with the latest inquiry said the tone of messages between foreign exchange traders was similar to exchanges between Libor derivatives traders, whose brazen arrogance as they manipulated benchmark interest rates first stunned regulators, politicians and the public in 2012.

Anonymised messages such as: "Done … for you big boy" and "Dude, I owe you big time! Come over one day after work and I'm opening a bottle of Bollinger" were published by UK and US regulators after they reached a regulatory settlement with Barclays in June 2012 over allegations of Libor rate manipulation.

(Reporting by Matt Scuffham and Kirstin Ridley; Editing by Carmel Crimmins)