The hottest AI hardware Substack posts right now

And their main takeaways
Category
Top Technology Topics

Nvidia's Groq Plot Thickens

The Chip Letter • 6334 implied HN points • 04 Mar 26
đź•ą Technology AI hardware
  1. Nvidia is quickly integrating Groq’s low-latency processor technology and team and is expected to unveil a Groq-derived inference chip at GTC.
  2. Groq’s dataflow architecture plus years of compiler work could deliver extremely fast, low-latency inference if Nvidia combines it with its wider IP and engineering.
  3. If Nvidia pulls this off it could narrow the field of inference accelerators and become a major, potentially game-changing shift in computer architecture for AI.

NVIDIA Introduces Groq LP30 and LPX Nodes

More Than Moore • 957 implied HN points • 16 Mar 26
đź•ą Technology AI hardware
  1. NVIDIA has folded Groq’s engineering and chip technology into its product line and is shipping the Groq LP30 inside LPX nodes to accelerate inference decode workloads.
  2. The LP30 offers about 1.2 PFLOP FP8 performance and ~500 MB of SRAM per chip, with 8-chip LPX units giving 4 GB and full systems scaling to 256 chips / 128 GB, prioritizing huge SRAM bandwidth for high-throughput decoding.
  3. NVIDIA will use its Dynamo orchestration to split work across Rubin, Rubin CPX and Groq LPX hardware (customers can mix up to ~25% Groq) so prefill and decode are handled by the best-suited chips to boost tokens-per-second for premium use cases.

Intel's Larabee Legacy

The Chip Letter • 7426 implied HN points • 24 Jan 26
đź•ą Technology AI hardware
  1. Larrabee was Intel's attempt to build a GPU by extending x86, but the design proved uncompetitive and the project was cancelled.
  2. The project added large new vector instructions (LRBni / 512-bit vectors) and architectural baggage that increased complexity without producing a viable graphics product.
  3. Larrabee's failure left Intel without a competitive discrete GPU, costing time and money and contributing to long-term cultural and strategic problems that weakened its position in AI and graphics markets.

TPU Mania

The Chip Letter • 18128 implied HN points • 13 Dec 25
đź•ą Technology AI hardware
  1. Google’s TPU program is the result of a long, steady effort dating back to 2013, evolving from a simple TPU v1 co‑processor into massive cloud AI supercomputers using systolic-array ideas and iterative hardware improvements up to TPU v7.
  2. Google’s control of the full stack, huge resources, and datacenter expertise give TPUs a strong practical advantage, but selling TPUs externally creates strategic trade‑offs and means customers should avoid becoming fully dependent on a single vendor.
  3. The TPU vs GPU contest is still open: architectural strengths matter, but ecosystem, software, and execution will likely decide market share, and we should expect convergence rather than one clear winner.

AWS Trainium3 Deep Dive | A Potential Challenger Approaching

SemiAnalysis • 12829 implied HN points • 04 Dec 25
đź•ą Technology AI hardware
  1. Amazon's Trainium3 chips are designed to be cost-effective and speedy, focusing on giving customers the best value. Their approach looks at everything from the hardware to the supply chain to make sure they stay competitive.
  2. AWS is working hard to make their software more accessible for developers, especially by open-sourcing critical parts of their software stack. This move aims to create a larger community of developers who can contribute and support the Trainium ecosystem.
  3. Trainium3 also features advanced networking capabilities that allow for smoother communication across chips, which is important for training large AI models efficiently. This positions Amazon to better compete with other tech giants in the rapidly evolving AI space.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Beyond the Instruction Stream

More Than Moore • 467 implied HN points • 03 Feb 26
đź•ą Technology AI hardware
  1. They use a dataflow architecture that runs the compiler's intermediate graph directly instead of a traditional instruction stream, so pipelines stay full and ALUs can execute whole loops every cycle for much higher effective throughput.
  2. Memory is handled by many small, localized MMU-like units plus runtime telemetry that adapts allocations to reduce false sharing, enabling an order-of-magnitude more outstanding memory requests and very high HBM utilization even on irregular workloads like GUPS.
  3. Their go-to-market and tooling are HPC-first while supporting common parallel models (OpenMP, CUDA, Kokkos) with a "bring your own code" approach, hardware-accelerated low-overhead kernel reconfiguration, and chiplet/RDMA-style scaling, with AI-specialized designs planned later.

Weaponized Inventory: How the U.S.–China Tech War Broke the Supply Chain

Gad’s Newsletter • 38 implied HN points • 09 Mar 26
đź•ą Technology AI hardware
  1. Sudden changes in export rules are triggering massive over-orders for AI chips that overwhelm testing, licensing, and shipping systems, so companies must add regulatory scenario planning to their demand forecasts.
  2. Most rare-earth refining and midstream processing are concentrated and slow to replicate, creating hidden Tier‑N chokepoints that require deep BOM traceability and years of investment to resolve.
  3. Complex products like humanoid robots hinge on a few hard-to-replace precision parts and long supplier‑qualification timelines, forcing a costly shift from just-in-time sourcing to resilience-focused, multi-source supply networks.

Solving The Problems of HBM-on-Logic

More Than Moore • 490 implied HN points • 17 Dec 25
đź•ą Technology AI hardware
  1. Stacking HBM directly on top of accelerators creates a severe thermal bottleneck that pushes GPU temperatures far above safe operating limits.
  2. Solving it requires many coordinated changes — removing base dice, merging/thinning stacks, adding conductive shims, and aggressive backside or double-sided cooling — and the single most effective move is halving GPU clock speed, which lowers temperatures but cuts raw compute.
  3. Those fixes bring big cost, yield, and supply-chain challenges and may only give modest net gains, so 3D HBM-on-logic looks like a research roadmap rather than a near-term commercial product, with vendors likely pursuing improved 2.5D or remote high-bandwidth memory alternatives instead.

Groq Inference Tokenomics: Speed, But At What Cost?

SemiAnalysis • 10708 implied HN points • 21 Feb 24
đź•ą Technology AI hardware
  1. Groq AI hardware showcases impressive speed and cost efficiency, outperforming other inference services while charging less.
  2. While speed is vital, supply chain diversification plays a significant role in evaluating hardware's revolutionary potential.
  3. Understanding the total cost of ownership is crucial in deploying AI software, with significant impacts from chip microarchitecture and system architecture.

Latest open artifacts (#17): NVIDIA, Arcee, Minimax, DeepSeek, Z.ai and others close an eventful year on a high note

Democratizing Automation • 150 implied HN points • 05 Jan 26
đź•ą Technology AI hardware
  1. Several major open models and updates landed at year-end — releases from NVIDIA, Arcee, LLM360, Zhipu and others noticeably pushed open-model capabilities higher.
  2. The community trend is toward bigger and Mixture-of-Experts (MoE) architectures, multi-token prediction, and openly releasing training data and checkpoints, which should speed progress and reproducibility.
  3. Important tradeoffs remain: some models excel on specific tasks like UI or coding but can be slower or weaker on very long-context workloads, and even larger, more capable variants are promised in 2026.

AMD's AI Future is Rack Scale 'Helios'

More Than Moore • 630 implied HN points • 12 Jun 25
đź•ą Technology AI hardware
  1. AMD has launched the new MI350 series of GPUs, which are designed to greatly improve AI performance, offering up to double the speed compared to the previous models.
  2. They have also introduced ROCm 7, a software update that focuses on better support for AI applications, making it easier for developers to use AMD hardware.
  3. AMD is planning for a significant shift toward rack-scale AI systems, with new products and roadmaps that aim to increase energy efficiency and performance by 2030.

What’s the future of AI hardware?

Product Identity • 138 implied HN points • 17 Jun 24
đź•ą Technology AI hardware
  1. AI hardware is still finding its identity and purpose. It's not yet clear how AI will truly enhance our devices.
  2. New gadgets often create high expectations but can lead to disappointment. Companies may hype products that aren't fully developed.
  3. Innovation in hardware often combines old ideas with new technology. It might be better to improve existing devices than to create entirely new ones.

Technically Monthly (January 2026)

Technically • 12 implied HN points • 06 Jan 26
đź•ą Technology AI hardware
  1. Try multiple vibe-coding tools by building the same thing so you learn their quirks, limits, and pricing before committing.
  2. Monitor AI with simple evals: study failures, use straightforward assertions instead of AI-judging-AI, and follow a loop of vibe check, spreadsheet, fixes, then targeted tests to cut hallucinations.
  3. Use AI thoughtfully at work by customizing prompts and iterating on workflows; learn prompt engineering or you risk being outcompeted by careless automation.

Inside Packaging And Module Technology With Saras Micro

Semiecosystem • 19 implied HN points • 24 Jun 24
đź•ą Technology AI hardware
  1. The semiconductor industry is entering a new growth cycle driven by the rise of AI tools and applications, with the next wave of growth expected to come from AI hardware.
  2. To overcome challenges in traditional chip scaling, the industry is adopting chiplet-based architectures and heterogeneously integrated packaging approaches for continued performance scaling.
  3. Advanced packaging technologies play a crucial role in supporting high-performance compute devices for AI systems, with companies like Saras exploring innovative solutions like embedded capacitive module technology for improved power delivery.

Qualcomm’s Cloud AI 100 PCIe: Now For All

More Than Moore • 93 implied HN points • 06 Jan 25
đź•ą Technology AI hardware
  1. Qualcomm's Cloud AI 100 PCIe card is now available for the wider embedded market, making it easier to use for edge AI applications. This means businesses can run AI locally without relying heavily on cloud services.
  2. There are different models of the Cloud AI 100, offering various compute powers and memory capacities to suit different business needs. This flexibility helps businesses select the right fit based on how much AI processing they require.
  3. Qualcomm is keen to support partnerships with OEMs to build appliances that use their AI technology, but they are not actively marketing it widely. Interested users are encouraged to reach out directly for collaboration opportunities.

Sparsity: The License to Skip

Software Bits Newsletter • 0 implied HN points • 07 Jan 26
đź•ą Technology AI hardware
  1. Sparsity means many weights or activations are zero so you can skip their multiplications, but random/unstructured zeros usually don’t make GPUs faster because irregular memory access and load imbalance kill performance.
  2. Hardware-friendly patterns like 2:4 sparsity and block sparsity let accelerators actually speed up computation, while pruning and ReLU-driven activation sparsity often need structure or predictive gating to become efficient.
  3. Conditional computation (Mixture of Experts) is the most powerful practical sparsity: only a few experts run per input, giving huge model capacity with much less active compute and strong empirical results.

What hardware do I need to create a LLM?

Shrek's Substack • 0 implied HN points • 18 Apr 23
đź•ą Technology AI hardware
  1. Training large language models (LLMs) needs powerful hardware, often multiple A100 GPUs with 40GiB of VRAM each. Running them is cheaper than training.
  2. Different data types like FP16 and TF32 are crucial for handling model memory. New types help manage larger numbers while saving memory.
  3. For smaller models, single hardware can work, but bigger models need a lot of VRAM or multiple systems. There's a difference between training and running models efficiently.