OpenAI-compatible APIs vs provider abstraction: What actually transfers across LLM providers?

OpenAI-compatible APIs promise something very attractive: write one integration, change the base_url...