It doesn’t concern Jon Batiste if some people tend to put him in a particular musical box. “That’s okay, I always break ’em,” declares the pianist, vocalist and composer who leads his Stay Human band nightly on “The Late Show with Stephen Colbert.”

Those who know Batiste solely through his television persona see a gregarious, always smiling, stylishly...