Provable and practical in-context policy optimization for self-improvement