jyapayne/mnemosyne
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: increase httpx read timeout to 600s for large context SSE streams

Browse source 
With 200k+ token contexts, Anthropic can take 60+ seconds for time
to first token. The 300s timeout was too aggressive for SSE reads
during long thinking phases.
This commit is contained in:
Joey Yakimowich-Payne 2026-03-15 18:55:08 -06:00
commit 3a970efd62
1 changed files with 4 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

6
src/mnemosyne/gateway.py
View file

@ -1621,10 +1621,12 @@ def create_app(
clients.update(
{
"anthropic": httpx.Client(
base_url=anthropic_upstream, timeout=httpx.Timeout(300.0, connect=30.0)
base_url=anthropic_upstream,
timeout=httpx.Timeout(600.0, connect=30.0, read=600.0),
),
"openai": httpx.Client(
base_url=openai_upstream, timeout=httpx.Timeout(300.0, connect=30.0)
base_url=openai_upstream,
timeout=httpx.Timeout(600.0, connect=30.0, read=600.0),
),
}
)