Hardware overhang refers to the extent to which available hardware technology will accelerate the performance of AGI once AGI is developed.
Hardware implementing SpikingModels at human brain scale isn't the only way of achieving AGI, but probably represents the highest level of abstraction that we can be reasonably confident would be capable of achieving AGI (see LevelsOfAbstractionOfTheBrainForAGI). NeuralNetworks might offer a cheaper approach.
Hardware cost is not what is preventing AGI. What is preventing AGI is we don't yet know the algorithms and data structures used by the brain. If we knew that we could program spiking hardware accordingly.
Number of transistors manufactured in 2014: 2.5x1020; grows 10 fold every 5 years
Semiconductor industry $340b (2015)
- So $0.14 per 100M transistors
- Compare to hardware cost estimate below
- In the right ball park considering many transistors are in memory which is probably cheaper on a per transistor basis
- A more accurate cross check would include only chip fab costs and exclude chip design costs
World population: 7.4x109 (2016)
Neurons per human brain: 86x109
- 2013: transistors ever made exceeds total human neurons alive
- 2016: transistors made per year exceeds total human neurons alive
This doesn't consider the enormous speed difference between neurons and transistors, which is a factor of several million.
The IBM spiking neuron inspired TrueNorth chip has 5,400 transistors per real time spiking neuron, so:
- 2032: neuron equivalent transistors ever made exceeds total human neurons alive
- 2035: neuron equivalent transistors per year exceeds total human neurons alive
Another way of looking at things is that in 2017, enough transistors were manufactured to produce artificial spiking neurons equivalent to 2,000,000 human brains. This could be one powerful superintelligence, or many smaller ones.
This suggests to the extent that there is a hardware underhang, it isn't likely to last much longer.
Throwing together some rough numbers for IBM's TrueNorth spiking neuron chip:
- Cost per chip (excluding design cost):
- Number of chips needed for human brain scale:
86x109 neurons in a human brain
- 1,000,000 spiking neurons on a chip
- So 86,000 chips for human brain scale
- Suppose support chips, circuit board, interconnects etc. double the price
- System achieves 70% utilization
- Amortize costs over 5 years
- Human brain level system costs: $80/hr (2016)
Computer technology price-performance improves by a factor of roughly 10 every 10 years. So the cost is likely to be significantly less by the time we understand how to program spiking hardware effectively.