Managing API keys sucks. I currently have a `.env` file that looks like a CVS receipt. There's an OpenAI key, an Anthropic key, a Mistral key, a Groq key... it's a mess.
That's why I started using OpenRouter for my smaller projects. It's basically a middleman that lets you use almost any model with a single key. And the best part? They have a dedicated "Free" section that's actually pretty robust.
The "Secret" Free Menu
OpenRouter doesn't exactly hide it, but they don't scream about it either. They aggregate free endpoints from other research labs and providers. Sometimes you're running on HuggingFace's infrastructure, sometimes somewhere else.
I use this for testing my prompt engineering. Why burn my precious GPT-4 credits just to see if my system prompt handles JSON formatting correctly? I switch to a free Llama 3 endpoint via OpenRouter, iterate 50 times, and then switch back to the paid model for the final run. It saves me maybe $10 a month, but hey, that's two coffees.
Setting it up (Takes 2 mins)
1. Go to OpenRouter.ai. Clean interface, I like it.
2. Click "Sign up". You can use GitHub.
3. Go to "Keys" and hit "Create Key".
My recommendation: Set the credit limit to $0.00 if you want to be extra safe. It ensures you never accidentally call a paid model like Claude 3 Opus and wake up to a bill.
The Code
This is the best part. You don't need a special SDK.
from openai import OpenAI
# The base_url is the magic sauce
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key="sk-or-..."
)
# Look for models ending in ":free"
# My favorite right now is the Mistral 7B Instruct
completion = client.chat.completions.create(
model="mistralai/mistral-7b-instruct:free",
messages=[
{
"role": "user",
"content": "Why do programmers wear glasses? Because they don't C#.",
},
],
)
print(completion.choices[0].message.content)Is it reliable?
For a hackathon? Yes. For a production banking app? No.
Since these are free endpoints, they sometimes get congested. I've seen latency spikes go up to 5-10 seconds on bad days. But usually, for things like "ToRA Code" (a great coding model) or "Gemma 7B", it's snappy enough for a chatbot.
Give it a shot. It's one less key to manage, and that's a win in my book.