Adler studied at Baltimore City College and the Peabody Conservatory, and won the 1927 Maryland Harmonica Championship. His first orchestral date as soloist was with the Sydney Symphony in 1939. He went on to fame, playing arrangements of classical concertos and works composed specially for him by composers such as Hindemith and Vaughan Williams, and touring Europe and Japan.
He settled in England in the 1950s, after becoming involved with the defence of those investigated for 'un-American activities'. He also wrote music for films, notably Genevieve, High Wind in Jamaica, King and Country and The Great Chase.