Samsung has a crazy idea for AI that might just work: add a processor inside RAM

2 weeks ago

(Image credit: Serve The Home)

Samsung is framing its latest foray into nan realm of processing-in-memory (PIM) and processing-near-memory (PNM) arsenic a intends to boost capacity and little nan costs of moving AI workloads.

The institution has dubbed its latest proof-of-concept technology, which it unveiled astatine Hot Chips 2023, arsenic CXL-PNM. This is simply a 512GB paper pinch up to 1.1TB/s of bandwidth, according to Serve nan Home.

It would thief to lick 1 of nan biggest costs and power sinks successful AI computing, which is nan activity of information betwixt retention and representation locations connected computing engines.

Samsung CXL-PNM

Samsung’s testing shows it’s 2.9 times much power businesslike than a azygous A-GPU, pinch a cluster of 8 CXL-PNMs 4.4 times much power businesslike than 8 A-GPUs. This is successful summation to an appliance fitted pinch nan paper emitting 2.8 times little CO2, and boasting 4.3 times much cognition ratio and biology efficiency.

It relies connected compute definitive nexus (CXL) technology, which is an unfastened modular for a high-speed processor-to-device and processor-to-memory interface that paves nan measurement for much businesslike usage of representation and accelerators pinch processors.

The patient believes this paper tin offload workloads onto PIM aliases PNM modules, which is thing it’s besides explored successful its LPDDR-PIM. It will prevention costs and powerfulness consumption, Samsung claims, arsenic good arsenic widen artillery life successful devices by preventing nan over-provisioning of representation for bandwidth.

Samsung’s LPDDR-PIM boosts capacity by 4.5 times versus in-DRAM processing and reduces power usage by utilizing nan PIM module. Despite achieving an soul bandwidth of conscionable 102.4GB/s, however, it keeps computing connected nan representation module and there’s nary request to transmit information backmost to nan CPU.

Samsung has been exploring technologies for illustration this for immoderate years, though nan CXL-PNM is nan closest it has travel to day to incorporated it into what mightiness soon go a viable product. This besides follows its 2022 HBM-PIM prototype.

Made successful collaboration pinch AMD, Samsung applied its HBM-PIM paper to large-scale AI applications. The summation of HBM-PIM boosted capacity by 2.6%, while expanding power ratio by 2.7%, against existing GPU accelerators.

The title to build nan adjacent procreation of components fresh to grip nan astir demanding AI workloads is good and genuinely underway. Companies from IBM to d-Matrix are drafting up technologies that purpose to oust the champion GPUs.

More from TechRadar Pro

The best DDR5 RAM: our apical high-performance representation picks
The best graphics cards: apical GPUs for each budgets
Samsung is astir to boost your CPU representation successful a large way

Sign up to nan TechRadar Pro newsletter to get each nan apical news, opinion, features and guidance your business needs to succeed!

Keumars Afifi-Sabet is nan Features Editor for ITPro, CloudPro and ChannelPro. He oversees nan commissioning and publication of in-depth and long-form features crossed each 3 sites, including sentiment articles and lawsuit studies. He besides occasionally contributes his thoughts to the IT Pro Podcast, and flexes his 10 years of penning acquisition successful producing contented for a assortment of publications including TechRadar Pro and TheWeek.co.uk. Keumars joined IT Pro as a unit writer successful April 2018, and has expertise successful a assortment of areas including AI, cyber security, unreality computing, and integer transformation, arsenic good arsenic nationalist argumentation and legislation.

Source Networking