Jersey against tax schemes like K2 - chief minister
His comments came after it emerged more than 1,000 people, including comedian Jimmy Carr, were thought to be using the legal Jersey-based K2 scheme.
It is said to be sheltering £168m a year from the UK Treasury.
About 24% of all employees in Jersey were employed in the finance sector in 2011, making up about 40% of the island's economy.
'Terrible error' In a statement, Mr Gorst said: "We will continue to be clear that Jersey does not need nor does it wish to be associated with aggressive tax planning schemes of the kind to which recent publicity has been given in the UK press.
"I have every intention of ensuring to the best of my ability that this message is received, understood and acted upon by all concerned."
John Harris Jersey Financial Services CommissionThis is a UK scheme for UK nationals, sold in the UK but administered in Jersey”
"Jersey makes a living out of administering all sorts of employment schemes and this is just one of them."
Carr tweeted on Thursday that he had "made a terrible error of judgement" over using the K2 scheme and has since withdrawn from the scheme.
HM Revenue and Customs (HMRC) said the scheme was already under investigation.
A spokesman said: "We examine the implementation of avoidance schemes in detail and will not let any aspect of these cases go unchallenged."