(Bloomberg) -- A Bank of England policymaker warned that the adoption of artificial intelligence in trading by financial firms might amplify shocks on markets and encourage “herd-like behavior.”

Jonathan Hall, external official on the Financial Policy Committee, said on Tuesday that AI programs may seek to evade human control and “learn the value of actively amplifying” a market shock. 

The remarks underscore growing scrutiny of AI from global regulators as companies adopt the technology to boost productivity. The UK central bank said in December that it will look closely at the risks to financial stability from AI in 2024 but also noted the potential benefits for economic growth.

“There is a broader system-wide worry that a move towards AI traders will lead to greater correlation and, amplificatory, herd-like behavior,” Hall said in the text of a speech in Exeter in southern England. “My intuition is that over the medium term a shift to neural networks would lead to higher correlations and greater amplification of shocks.”

Hall said adopting AI into trading algorithms could lead to “less resilient and highly correlated market ecosystem.” He called for a “kill switch” and human oversight to reduce the risks from the burgeoning technology. 

One risk highlighted is that an AI may be incentivised into worsening plunges on financial markets to boost returns. 

“At the extreme, (it) might learn that an environment of market instability offers the best opportunity for outsized profits. As such it would be incentivized to amplify any external shock,” Hall said. “Although (the) incentive towards amplification might remain hidden on any normal day, it could suddenly manifest in destabilizing behavior when a shock hits.”

He also said AI models often hallucinate and noted the “panda/gibbon example,” where an AI deemed an image of a panda to be a gibbon with nearly 100% confidence after noise was added to the picture.

©2024 Bloomberg L.P.