Can HBF ease the AI memory bottleneck? The dual case for lower-cost inference and tighter NAND supply
Bernstein believes high-bandwidth flash could add a capacity tier between HBM and conventional storage, lowering large-model inference costs and consuming more NAND wafer capacity. Performance limits, packaging resources and ecosystem support will determine the pace of adoption.
Original source：BernsteinFirst published on Unhidea：Original source date：
Key takeaway
If HBF can provide greater capacity at bandwidth close to HBM, it could improve AI inference economics and tighten NAND supply, but its profitability and adoption depend on pricing, manufacturing costs and ecosystem uptake.
Why does AI inference increasingly need a new memory tier?
When large models generate content, they repeatedly read model parameters and the KV cache created by earlier computations. The longer the context, the more data must remain accessible, and compute units may sit idle while waiting for data.
MoE models activate only some experts at a time, reducing the computation required for each operation, but the full set of parameters must still be stored and available for use. Once the data exceeds what HBM can economically accommodate, demand extends to other memory and storage tiers.
Retaining previous computation results also has economic value: the report says some AI labs and Google charge approximately 10% of the price of new input tokens to read cached tokens, so repeatedly using an existing context can save computation.
How does HBF lower inference costs, and what limits it?
SanDisk aims to provide 8 to 16 times the capacity while approaching HBM read bandwidth. In a tiered architecture, HBM holds active, latency-sensitive data; HBF holds less frequently used model weights and reusable cache, reducing the GPUs and HBM configured solely to accommodate the model.
SanDisk's internal simulation of a 960GB Qwen model shows that an HBF configuration can accommodate the model with one GPU, while an HBM-only configuration needs at least eight; four GPUs equipped with HBF can match the token output of eight HBM GPUs.
HBF achieves high bandwidth through parallel reads, but the latency of an individual NAND access remains higher than DRAM's; small, scattered requests may also read large amounts of unneeded data. It writes more slowly and has a limited lifespan, making it better suited to data that is read frequently but updated less often.
Why could HBF change NAND supply and demand?
Bernstein's base case estimates that HBF consumes about 6.5 times the wafer capacity of conventional TLC NAND for the same effective capacity. The calculation assumes SLC, which stores one bit per cell, and accounts for additional chip area and manufacturing yield losses; the final ratio depends on product configuration and process technology.
If HBF adoption expands, it will add AI inference demand for NAND while reducing the effective capacity produced per wafer, helping support industry prices. Other high-performance flash solutions may also add demand or occupy more capacity, with the effect depending on the technology adopted.
The HBM and GPUs needed for an individual workload may decrease, but Bernstein believes total demand could still grow if lower inference costs bring enough new applications and deployments; HBM will continue to handle active data and frequent updates.
Can higher costs translate into higher profits?
Under Bernstein's base case, HBF manufacturing cost per GB is approximately 9.5 to 11.5 times that of enterprise SSDs. Although a 16-layer stack costs more per module, fixed costs are spread over greater capacity, making its cost per GB lower than that of a 8-layer stack.
The firm assumes that HBF and HBM4 modules sell for similar prices and that HBF has 8 times the capacity. In its base case, with HBM4 priced at 38 dollars per GB, HBF sells for about 4.75 dollars per GB, corresponding to gross margins of 88.9% and 86.5% for the 16-layer and 8-layer products respectively, above SanDisk's overall fourth fiscal quarter gross margin of 84.6%.
This margin opportunity depends on actual performance, HBM prices and the memory cycle, as well as HBF chip, packaging and testing costs. Bernstein also notes that HBM suppliers may accept margins below those of ordinary DRAM for strategic value, and the same could happen with HBF.
What barriers remain before HBF can be widely adopted?
HBF requires processors and software to be redesigned together to determine how data is placed, scheduled and prefetched between HBM and HBF; it must also compete with compute chips and HBM for expensive, limited advanced packaging space.
SanDisk says its first HBF memory chip has completed tape-out, and samples of its first inference products are planned for 2027. Google, Meta and SK hynix are among those that have joined the alliance, but Bernstein notes that NVIDIA has not formally supported HBF and is promoting solutions such as CMX and Storage Next.
SanDisk's current long-term financial targets do not include HBF; Bernstein believes that if future products ramp up in volume, they could provide upside to those targets.
