Kunle Olukotun is one of the pioneers of chip multiprocessor designs.
As a Stanford University professor and co-founder of SambaNova Systems, Kunle is developing new chips for next-generation artificial intelligence(AI) computing from the data center to the edge.
He also concerning the future of democracy and governance, climate change.
Met with him June 24, federated Computing Research Conference ‘ACM FCRC 2019’ at Phoenix convention center, Arizona, USA.
Q. On semiconductor for AI/ML.
I said that there was not much activity in semiconductor space because venture capitalists could find a way for money in these companies. Given the amount of you have to pump into that company, you couldn’t tell until 2years you’ve got anything out. So there was not much activity.
What changes two or three years ago intense interest in AI and Machine Learning(ML). It was google’s TPU, Nvidia making a lot of money through GPUs. And a lot of people are looking at alternative approaches. So that’s what the initial interest. I think going forward what we have seen is Moor’s law slowing down. Moor’s law drove the semi-conductor industry through 50 years, but it slowing down.
And at the same time, you need a lot of computation capability for ML to train complex models. You need a huge amount of data. It crunches on that data you need a lot of computation. And both for training, where you need to be able to analyze large data set, Tera-bite size data sets. Also for inference, where you want to be able to do inference at migrating.
I think, a combination of the two, you will be able to serve lots of people at the same time. For that reason, you’ll go to see continual interest in domain-specific approaches to computing for ML.
I think the future looks interesting. And that’s what you see a lot of companies popping up, Havana(Lab), Graphcore, Celebras.
*Graphcore, UK-based AI startup has raised more than $300 million in venture capital and is valued at $1.7 billion. Its investors include BMW and Microsoft. Custom chip known as an Intelligence Processing Unit(IPU).
*Cerebras Systems, still in stealth mode, is headed by Andrew Feldman, who founded low-energy chip startup SeaMicro and sold it to AMD.
Q. on startup’s advantage.
I think in any business, you know, what we need is innovation. And innovation is easier to do in startups. Because of it just less friction. Processes doing in big companies where you have to support existing set up for products. I think that’s the benefit. You can try to bring different approaches to startups. That would be difficult doing an existing established company.
Q. USA leading semiconductor industry. How about China?
In some areas in ML, they have advantages, because they have more data for Video and tracking, face recognition. But I think that ultimately research and advances easier done in a fully open society. How’s the AI/ML community in Korea? Is it coming up?
(- It’s Booming but small yet. )
Q. As a professor, you also leading a company.
I think doing these things work together. Researches read to innovations. And then the company finds problems that need to be researching. That’s one thing. And another thing is I work with very good people. That’s the key.
Q. proper size for the early stage semiconductor startups.
It depends on what you are trying to do. We are trying to do both Hardware and Software for new types of processors. So it’s big(more than 100).
Q. How do you think fully autonomous driving?
It can be but for a while. Level5 is very tough. I have a Tesla. I think problems on Tech and humans. I think the real problem is other human drivers. If there are all robots, all autonomous, everybody obeys the rules. But humans…
(are you also developing autonomous driving chips?)
No, we don’t do. We look at generic support for ML. It could be used in vision and speech (recognition), natural language understanding on edge and datacenter.
I think, in training, Nvidia is dominant. And Havana (labs) is trying but again they have limited software stack.
Q. It’s hard to write simply about cutting edge Tech.
(ACM FCRC)Planatary cession is for giving insight. I think the tutorials( Hardware Accelerators for Training Deep Neural Networks, UsingAItoImproveArchitecture, Non-Traditional Workload Domains, ISCA) is a little technical, too technical.
Q. About the future of AI.
The concern is about human and AI interaction. There is a new center at Stanford ‘Human-Centered AI(Stanford Institute for Human-Centered Artificial Intelligence)’. Fei-Fei Li is the Co-Director of the Stanford Institute for Human-Centered Artificial Intelligence.
(*According to Stanford HAI, Fei-Fei is considered one of the pioneering researchers in the field of artificial intelligence. She previously served as the Director of the Stanford AI Lab (SAIL) from 2013 to 2018. For her Ph.D. dissertation research, Fei-Fei looked into a combination of cognitive neuroscience and computer vision…She dedicates her research to human-centered AI and is a national leading voice for advocating diversity in STEM and AI.)
Q. How about semiconductor giants and future?
I think, they certainly one of the leaders at the moment. They can continue. I mean they can. I think underlying technology is slowing down. Intel is going to come back now and forever. Then they have to continue to innovate. That’s the only way to remain relevant.
I mean they have gotten impressive works that they have achieved so far. And they can continue. Keep that same sprite of innovation and hardworking. They can do it.
Q. Samsung is known for its limitations on SW.
Certainly, the software has to be huge stuff developing their SW and application capability. But I know in Korea, in terms of when people say to go to engineering school, is it more prestigious to be a software developer or electrical engineer? (This time might be SW engineer)
I mean I think that’s right. You need more doing software.
Q. Statistics on ICML accepting papers, Google is dominant in AI research. Google has established in the Africa AI research center in Ghana to solve its problems such as disease and food. Also, IBM research opened a center in Kenya.
I think it’s a good idea. Develop talent outside the Bay area. I came from the continent myself, Nigeria. So I think it’s a good idea. And they got lots of money, they should.
Q. Samsung has opened at Cambridge UK, Toronto Canada, etc.
Well, I think Google would have claimed that those areas already have covered. I mean, naturally, it going to start to have the most amount of research. A lot of staff in this US and EU. Google has staff there, already have places. I think, now they are going further in the field.
Q. Human had written books and made libraries for data storage. AI, outsourcing human intelligence on a computer system, is it can be kind of human?
We will see whether it can get to the human level of creativity and reasoning.
My view is that, at the moment, it’s just sophisticated tools they don’t have agency. You need to have an agency to be human. You need to have something to try to do. You need to have a soul. I don’t think they have reached the level of agency and soul.
In the far future, it’s possible but a long way to go… I would say in 50years.
Q. What happened after that?
I think it depends on the point where we get to. Does it include the democratization of capability and wealth which causes concentration? As it, what happened in the industrialization? Use to, the case is that if you in America you could graduate high school and could go into a factory for physical labor. And make a decent wage and live a decent middle-class life.
Now that’s not possible because automation and reduction in the physical labor and then the number of jobs drastically reduced. Then the amount of skill need, even the factory works going to a greater requirement for education and fewer people are able to employ these things. And so what you get is a reduction in the number of people who can actually get into the middle class by working factory-like jobs. Then you could argue that automation makes overall society more productive. But the question is did it’s an improvement for more people or just concentration on wealth. So I think the question on robotics and AI is ‘what’s the impact’. For instance, if I don’t need lawyers anymore because all these skills automated, that means humans no longer need to do the job. And the computer can do it more cheaply and doesn’t ever get tired. So then the question is what humans do.
In the future of work, how AI interacts with a human is a really important question. What happens if I have automated setup robots, and it fulfills all my needs, I don’t need to hire housekeepers, teachers. And all these jobs will potentially go way. I don’t need a driver. And then the question is ‘what people do?’ How will they become a productive member of society? I think that’s the question to ask. When we close to the singularity, before then, comes massive disruption. What do people do?
Q. Maybe this trend is already shown in booming Youtube and entertainment.
Yes, but the point is you just consuming the entertainment. There has to be produced. Who’s the production? Who is paying for the product? If I say that you are all consumers, I’m generation contents and charging for it, I’m making money. I got the process that giving economic value, you are just consumers. I don’t think it works in general. Youtube? User generating contents but who’s making the money?
Q. Problem is governance.
Yes, Exactly. Any of these problems is difficult. Humans are very short-sighted, especially, even argument could be made that this is long term problem, trying to solve them today, going to impact long term benefits. I think climate change is a key example of this. I can’t believe what’s happening in climate change. Politically America is not taken on this long term exponential challenge to the existence of the human race.
Q. Why? America is a powerful country having lots of resources.
Because there are politicians find easy benefit not telling the truth about climate change. It’s easier to say we won’t do anything about climate change, there for you to continue buying your car, use fossil fuel. I don’t know why. But, I don’t understand the point of view. It’s a really important issue people should be aware of. And you know our readership doing the hard thing. Obama tried but Trump… There are structural issues some politicians should do and the US should do the right thing.
Q. AI on Politics, Disruption of politics with AI is possible?
Do you mean replacing politicians with AI? I think you seeing what happened. The reason Trump won, you can argue that one component was what happened on Facebook. And Facebook was driven by what Russian hacking. Russian guys, they exploiting AI algorithms to promote fake news and targeted individuals. When vote benefiting Trump. It’s already played the role. Because you had an algorithm that works, that based on AI ended up forcing ideas of disruption in the election.
Q. What If AI becomes smarter than humans?
The question goes back to when AI has the agency.
Kunle has been a successful scholar and businessman, he also loves watching and playing sports and football with his 2kids.
*Prof. Kunle Olukotun
He is Cadence Design Professor of Electrical Engineering and Computer Science at Stanford University. Pioneer of Chip Multiprocessor Designs, He is a Director of the Stanford Pervasive Parallelism Lab, developing high- level language for programming accelerators and Co-leader of the Data Analytics for What’s Next (DAWN) research program.
He co-founded SambaNova Systems in November 2017. This startup has raised more than $200 million in it’s Series A and B rounds. Samsung also invested. In April 2019, the company announced a $150M Series B funding round. He founded Afara Websystems, acquired by Sun in 2002.