Medical professionals believe in black boxes. Medical professionals can’t directly vocalize why they believe in something, so why should we expect AI models to do the same?