Facebook Wants New and Cheaper MemoriesFor years, Intel Corp., and to a lesser extent, the systems and memory houses, dictated the roadmaps in the processor, DRAM and other segments.
For years, Intel Corp., and to a lesser extent, the systems and memory houses, dictated the roadmaps in the processor, DRAM and other segments.
Traditionally, Intel rolls out a new microprocessor and an associated chipset that supports a particular DRAM interface technology. Now, amid the explosion in social networking, there is new broker in the town. “New applications like Facebook are driving the industry,” said Sherry Garber, an analyst with Convergent Semiconductors, a market research firm.
And the industry is listening, as Facebook claims some 800 million active users and a vast number of servers in its datacenters. At the recent JEDEC Memory Server Forum in Santa Clara, Calif., Jason Taylor, director of technical operations at Facebook, described the company’s server strategy and outlined the company’s wish list — and concerns — in the processor, memory and storage arenas.
In a nutshell, Taylor wants cheaper DRAM and NAND flash. The company also wants lower power memories as well as a “DRAM alternative that is 10x cheaper with a small throughput or latency hit,” he said. At the event, JEDEC and Samsung also outlined their respective roadmaps in the DRAM server market, with DDR4 and 3D chips on the horizon.
At present, Facebook has datacenters in California, Oregon and North Carolina, with plans to build a plant in Sweden. The company configures its servers for six separate and basic types of services: Web, Memcache, Database, Hadoop, Haystack and Feed.
In April, Facebook launched the Open Compute Project, an initiative to share its technology with the industry within its datacenter in Oregon. Facebook is publishing technical specifications and mechanical CAD files for the servers, power supplies, server racks, battery backup systems and building design. AMD, Dell, HP and Intel are among the companies that co-develop technology with Facebook.
Facebook hopes to deliver a 38 percent increase in energy efficiency at 24 percent lower cost. It plans to achieve an initial power usage effectiveness (PUE) ratio of 1.07, compared with 1.5 for its current facilities.
Not surprisingly, power consumption is one of the major problems in the datacenter, where Amazon, Google, Facebook, Microsoft and others are looking for technologies to attack the problem.
Dileep Bhandarkar, distinguished engineer and chief architect for the Global Foundations Services unit at Microsoft, said traditional datacenters are sometimes 11 football fields in size and consume 20 to 50 Megawatts of power. The construction costs for a datacenter are $10 million to $15 million per Megawatt.
“In a typical datacenter, for every Watt in server power, there can be another 0.5 Watt to 1 watt consumed for power distribution losses and cooling,” he said during the JEDEC event.
Facebook’s Taylor said the temperatures within a datacenter can reach “85 degrees F in the cold (aisles) and 120 degrees F in the hot (aisles),” thereby propelling the need for new and lower-power technologies.
Another concern is processor scaling and memory failures in multicore systems within the datacenter. The “number of cores is increasing and moving from two to four processors. (We are) starting to see scaling issues at four processors related to NUMA node count,” Taylor said. NUMA, or Non-Uniform Memory Access, “is a shared memory architecture that describes the placement of main memory modules with respect to processors in a multiprocessor system,” according to Intel.
For the Web applications, there is “1 to 2GB of shared memory per Web server” within Facebook, Taylor said. “Large hits allocate 40 to 50MB.”
The trouble is that “RAM failure rates are high enough to be a significant source of service events” within Facebook, he said. Some analysts have said that the industry is getting somewhat lax on DRAM testing — or simply not even testing parts — as a means to cut IC test costs.
Taylor also outlined the company’s wish list. Interestingly, Facebook, according to Taylor, is looking for “cheaper RAM with more latency or less throughput,” without elaborating. A lower latency memory could be a possible means to reduce the power-hungry refresh cycles in a DRAM.
Like most in the industry, Facebook is also looking for a “DRAM alternative.” Several companies are developing next-generation memory types — such as memrisistors, MRAM, phase-change and RRAM — but all of these technologies are still in the early stages despite years of R&D. The ability to make and scale these newfangled devices remain a challenge.
Taylor also said that Facebook is looking for cheaper but slightly slower NAND flash, which would operate somewhere around “100k IOPS per ~1TB” or roughly “100k IOPS per ~2TB.” The Facebook technologist may have implied that the company is looking for NAND with better reliability.
On the other hand, solid-state disk (SSD) storage has “fantastic performance,” he said. “Flash has changed the game.”