The US president says he spoke with Vietnamese leader To Lam, who is seeking to avoid punishing 46 per cent duties.