AI’s Bottleneck Shifts From Compute to Memory, Data and Agent Orchestration

QuantumBit reported that Intel is addressing the KV Cache problem in large-model inference. For causal self-attention Transformers, previously processed tokens leave Key and Value results that the model reuses when generating later tokens. The cache saves computation but consumes memory. Using Qwen3-8B’s public configuration and BF16 or FP16 KV Cache at two bytes per value, QuantumBit calculated about 147KB of KV data per token, or about 147GB for 1 million tokens. The report stressed that this is a capacity projection, not a storage purchase list, and noted Qwen3-8B officially supports 32,768 tokens natively and 131,072 with YaRN.

Intel’s answer, according to QuantumBit, is layered offloading: hot cache stays in GPU HBM, warm cache can move to CPU-side DDR memory, and colder data can sink to SSD or remote storage. vLLM’s KV Offloading already supports offloading cache blocks to CPU memory and configuring secondary storage. Intel has also outlined four KV-cache directions: KV Shrink, KV Fuse, KV Cascade and KV Infinity. KV Shrink combines tiered management with hardware compression and provides hot-and-cold scheduling APIs. Intel’s QuickAssist Technology, or QAT, handles compression and decompression on dedicated accelerators, reducing general-purpose CPU use. Intel said rearranging the KV Cache storage format increased compression savings from above 10% to above 20%, with about 20% to 30% space reduction through lossless compression.

In one Intel test cited by QuantumBit, dual Xeon Gold 6554S processors, two Nvidia L20 GPUs and Qwen3-32B at an 80% cache hit rate produced up to about 5x faster time to first token, or TTFT, than a native vLLM baseline without tiered offloading. QAT hardware compression was about twice as fast as CPU software compression overall, while enabling QAT compression added less than 10% TTFT overhead compared with no compression. In a separate test by Intel and DaoCloud’s joint lab using dual Xeon Gold 6554S, eight H800 GPUs and Qwen3-32B FP8 at an 80% cache hit rate, KV Shrink reduced average TTFT from 129.81 milliseconds to 114.13 milliseconds versus LMCache, a decline of about 12.1% under single-stream load and about 4.6% at eight-way concurrency.

iFanr reported that Ant Lingbo CEO Zhu Xing, speaking at the 2026 Bund Conference, said robots still face both capability and cost problems for ordinary tasks. “Small stores are actually hard to deploy,” he said. Lingbo focuses on an embodied brain that can be reused across different robot bodies and tasks, with LingBot-World for interactive world generation, LingBot-Video as a video-generation base for embodied AI, and LingBot-VLA 2.0 for action models. At the conference, robots using Lingbo’s brain demonstrated drug picking, logistics sorting and industrial loading and unloading. The pharmacy demonstration used an 80-centimeter-wide shelf aisle, and the company said related solutions have been deployed in Guoda Pharmacy retail stores.

Zhu told iFanr that data quality and distribution matter more than sheer volume. Lingbo’s internal data funnel raised usable data from about 15% to more than 90%, and the company uses streaming delivery with suppliers rather than waiting for large batches. He said first-person, or Ego, data collection is promising but current headbands and bare hands often lack the precision needed to reconstruct hand trajectories, and he favors high-precision, portable tactile gloves. Asked whether large language model companies could quickly erase such advantages, Zhu compared the question to OpenAI or Anthropic entering autonomous driving and said the core barrier would still be data understanding and data pipelines.

QuantumBit also reported that Hangzhou-based AI protein design company Liwensuo released Lévin™ Harness to the research community, with support for Apple-silicon Macs. The product is an agent workspace that connects data, models, plug-in tools, compute and workflows. Researchers describe goals in natural language; the agent uses open large models including DeepSeek, Kimi, GLM, Qwen and Xiaomi MiMo to plan steps, then calls scientific models and tools such as AlphaFold for structure prediction, ProteinMPNN for sequence design, Pallatom for all-atom protein generation and RFantibody for antibody design. A three-dimensional molecular visualization tool lets on-screen structural information become context for the agent, so terms like “here” can refer to a specific spatial position.

Liwensuo’s Pallatom was included by Nvidia as a representative protein foundation model alongside AlphaFold 3, ESM-3 and Proteína, according to QuantumBit. It was selected as an ICML 2025 Spotlight, and Pallatom-Ligand was accepted at ICLR 2026. The company operates an industrialization platform of more than 3,000 square meters in Hangzhou, including a 1,000-square-meter clean area, with fermentation, centrifugation, homogenization, filtration and column chromatography equipment.

iFanr reported that Qualcomm is preparing a related shift for AI agent phones. Its High Bandwidth Compute, or HBC, places compute near memory to reduce data movement for bandwidth-heavy operations such as matrix multiplication, vector operations, embedding lookups and some attention and KV-cache work. The upcoming next-generation Snapdragon flagship is expected to include a Qualcomm Oryon CPU at up to 5GHz, FlexCache scalable cache architecture, Adreno Matrix Cores for AI and matrix computation, an 18MB Adreno high-speed memory, and a Hexagon NPU with up to 50% more shared memory and an Element Accelerator for Transformer workloads.

Qualcomm is also working with model makers on on-device Mixture-of-Experts architectures, iFanr reported. In one described setup, a 30B MoE model can keep tens of billions of parameters available while activating about 3 billion routed parameters per token on the NPU, reducing compute load and memory bandwidth demand. The report framed these designs as an effort to keep more data close to computation rather than treating CPU, GPU, NPU and memory as separate stages.