• This site is a private, non-commercial website. As such, you're welcome here as long as you were invited. If you would like an invite, reach out to Cliff Spark

Show HN: AutoThink – Boosts local LLM performance with adaptive reasoning

  • Thread starter Thread starter codelion
  • Start date Start date
C

codelion

I built AutoThink, a technique that makes local LLMs reason more efficiently by adaptively allocating computational resources based on query complexity.
The core idea: instead of giving every query the same "thinking time," classify queries as HIGH or LOW complexity and allocate thinking tokens accordingly. Complex reasoning gets 70-90% of tokens, simple queries get 20-40%.
I also implemented steering vectors derived from Pivotal Token Search (originally from Microsoft's Phi-4 paper) that guide the model's reasoning patterns during generation. These vectors encourage behaviors like numerical accuracy, self-correction, and thorough exploration.
Results on DeepSeek-R1-Distill-Qwen-1.5B:
  • GPQA-Diamond: 31.06% vs 21.72% baseline (+43% relative improvement)
  • MMLU-Pro: 26.38% vs 25.58% baseline
  • Uses fewer tokens than baseline approaches
Works with any local reasoning model - DeepSeek, Qwen, custom fine-tuned models. No API dependencies.
The technique builds on two things I developed: an adaptive classification framework that can learn new complexity categories without retraining, and an open source implementation of Pivotal Token Search.
Technical paper: AutoThink: efficient inference for reasoning LLMs
Code and examples: https://github.com/codelion/optillm/tree/main/optillm/autoth...
PTS implementation: GitHub - codelion/pts: Pivotal Token Search
I'm curious about your thoughts on adaptive resource allocation for AI reasoning. Have you tried similar approaches with your local models?


Comments URL: Show HN: AutoThink – Boosts local LLM performance with adaptive reasoning | Hacker News

Points: 196

# Comments: 24

Continue reading...
 
Back
Top