DeepSeek Introduces mHC Architecture to Improve Large Model Training

TLDR

• DeepSeek introduced Manifold-Constrained Hyper-Connections (mHC) to improve large-model training scalability and efficiency.
• The mHC method was tested on 3B, 9B, and 27B parameter models, showing stable performance without added computational cost.
• mHC builds on ByteDance’s 2024 hyper-connection architecture by adding a manifold constraint to reduce memory overhead.
• CEO Liang Wenfeng co-authored and uploaded the paper, reaffirming his direct involvement in DeepSeek’s technical development.
• Industry observers expect a new DeepSeek model release ahead of Spring Festival 2026, based on the company’s publication patterns.

DeepSeek has released a new AI training method, Manifold-Constrained Hyper-Connections (mHC), in a paper uploaded to arXiv by CEO Liang Wenfeng. The architecture aims to improve training scalability for large models while keeping computational costs low. Researchers tested the method on models with 3, 9, and 27 billion parameters, showing consistent training efficiency. This comes as the company is expected to launch a new model before the Spring Festival in February 2026.

DeepSeek Builds on ResNet and Hyper-Connection Foundations

According to a report by SCMP, the mHC method enhances earlier hyper-connection (HC) designs first proposed by ByteDance in 2024 as an improvement to ResNet. ResNet allows deeper neural networks by preserving signal strength across layers, but faces challenges in maintaining efficient learning at large scale. ByteDance’s HC improved signal flow but didn’t fully address memory usage in larger models.

DeepSeek introduced a manifold constraint to limit expansion and better control memory and compute costs during training. This adjustment preserved the HC benefits while making the network suitable for larger training tasks. Researchers wrote that mHC maintained performance without increasing computational overhead per unit during model training at scale.

Lead authors Zhenda Xie, Yixuan Wei, and Huanqi Cao explained that the system enables stable deep learning without collapse. They confirmed mHC works with minimal infrastructure adjustments, making it efficient for broader deployment. The architecture was tested across multiple model sizes, confirming the technique’s adaptability and reliability. DeepSeek reported that the method handled signal preservation and scalability better than previous HC-based frameworks.

Liang Wenfeng Directly Leads Technical Advancement

CEO Liang Wenfeng was listed as the final author and uploaded the paper himself, continuing his role in major DeepSeek research. He has consistently shared technical papers linked to the company’s top models, such as R1 and V3 on arXiv. Other researchers typically upload supporting studies not directly tied to product development.

His involvement in this paper signals continued leadership in the company’s core AI work. The release underscores DeepSeek’s approach of linking internal research closely with future product direction. Florian Brand, a PhD researcher at Trier University, said DeepSeek papers often indicate what models are coming next.

He noted that the R1 model followed a similar pattern of publication and then launch. Liang’s involvement has again drawn attention from analysts watching DeepSeek’s release schedule. The company has not announced a date, but its publication strategy has become predictable. DeepSeek has remained quiet on details, but research uploads suggest new systems are under development.

The post DeepSeek Introduces mHC Architecture to Improve Large Model Training appeared first on Blockonomi.