AI
A startup claims it broke through a bottleneck that’s holding back LLMs
Miami-based AI startup Subquadratic has shared the results of an independent evaluation by Appen to validate claims about its new LLM, SubQ.
Key takeaways
- SubQ is 56 times faster than models using FlashAttention in a baseline speed test.
- SubQ scored 89.7% on LiveCodeBench, putting it in the same ballpark as other top coding models.
- SubQ sustained a 98% score on needle-in-a-haystack tests with context windows of 6 million and 12 million tokens.
- Subquadratic's architecture dynamically selects which token relationships are important on the fly.
- Subquadratic reused weights from a version of the Chinese open-source model Qwen to bootstrap SubQ.
Miami-based AI startup Subquadratic has shared the results of an independent evaluation by Appen to validate claims about its new LLM, SubQ. The model uses dynamic sparse attention instead of traditional dense attention to bypass the quadratic scaling bottleneck. Appen's tests found SubQ was 56 times faster than FlashAttention, scored 89.7% on LiveCodeBench, and achieved a 98% retrieval score on needle-in-a-haystack tests with context windows up to 12 million tokens. However, the model is not yet widely available, and Subquadratic bootstrapped SubQ by reusing weights from the Chinese open-source model Qwen.
In their words
“SubQ is either the biggest breakthrough since the Transformer ... or it’s AI Theranos.”
“I was like, ‘Wow, this could be a game changer,’ because models struggle with speed and inefficiency.”
By the numbers
- 12x
- Text processing capacity compared to most other models
- 56x
- Speed increase over FlashAttention in Appen's baseline test
- 89.7%
- SubQ score on the LiveCodeBench coding test
- $2,600
- Cost to run Anthropic's Opus 4.6 through RULER 128
- 98%
- SubQ needle-in-a-haystack retrieval score at massive scale
How it unfolded
- Google researchers publish the foundational "Attention Is All You Need" paper
- Subquadratic emerges from stealth mode with its bottleneck claims
- Subquadratic shares independent Appen evaluation results to back claims
Why this matters
-
Nobody will be building on transformers in a few years due to this efficiency breakthrough.
Turn stories like this into views
Ravenclip finds the AI news, makes the video, and posts it before attention moves on.
Common questions
- What happened with Subquadratic?
- SubQ is 56 times faster than models using FlashAttention in a baseline speed test.
- Where can I read the original report?
- Read the full report at mit_tech_review.
