Industry Leaders in Signal Processing and Machine Learning: Dr. Oriol Vinyals

Dr. Oriol Vinyals is a Principal Scientist at DeepMind, and a team lead of the Deep Learning group. His work focuses on Deep Learning and Artificial Intelligence. Prior to joining DeepMind, Oriol was part of the Google Brain team. He holds a Ph.D. in EECS from the University of California, Berkeley and is a recipient of the 2016 MIT TR35 innovator award.

His research has been featured multiple times at the New York Times, Financial Times, WIRED, BBC, etc., and his articles have been cited over 100,000 times. Some of his contributions such as seq2seq, knowledge distillation, or TensorFlow are used in Google Translate, Text-To-Speech, and Speech recognition, serving billions of queries every day, and he was the lead researcher of the AlphaStar project, creating an agent that defeated a top professional at the game of StarCraft, achieving Grandmaster level, also featured as the cover of Nature. He was also a contributor to the research and papers relating to the recent AlphaFold project, an AI system providing the solution to the protein folding problem (a 50-year old grand challenge in biology).

At DeepMind he continues working on his areas of interest, which include artificial intelligence, with particular emphasis on machine learning, deep learning and reinforcement learning, and science.

We approached Oriol Vinyals with a few questions:

1. In your own words, please tell us about your background. 

Since watching "2001: A space odyssey" as a child, I became intrigued with the possibilities of AI. I am a researcher who strives to learn new things as often as possible, and as a result I immediately became enthusiastic about how machines could learn. I studied Maths and Engineering as an undergraduate at UPC (Barcelona, Spain, where I grew up), followed by the completion of my PhD at UC Berkeley which focused mostly on speech recognition research. From my studies stemmed a passion for sequences which enabled me to further my research into language and other modalities. After joining Google Brain in 2013, most of my early work involved language, sequences, and broad applications of ML (e.g., seq2seq, distillation, and large language models). In 2016, I joined DeepMind where I'm currently leading the Deep Learning team and pushing the boundaries of what ML can do, whilst widening to other research areas such as RL. My work at DeepMind played a lead role in the exciting breakthrough and publication of the AlphaStar project in 2019, an agent that beat a professional player at the game of StarCraft2, and has also helped to widen certain applications in science, for example with my involvement in the recent AlphaFold breakthrough.

2. What challenges have you had to face to get to where you are today?  

One of the early challenges I had to face is that, due to studying in Spain in a relatively less known university, getting accepted to a PhD program was fairly hard as my background was a bit unusual. Another fundamental challenge has been finding my true passion, whilst trying to be opportunistic given the opportunities provided via grants, your network of colleagues and what they can teach about, etc. This led me to navigate from speaker diarization, to speech recognition, recurrent neural networks, sequence modeling, and then more broadly machine and deep learning, which is where I found my true love and passion, although I certainly enjoyed all the steps of the way!

3. What was the most important factor in your success? 

My colleagues. They serve as a source of knowledge, inspiration, compass to my own goals and research, and they are always supportive and pushing and challenging me at the same time, which is great. It would be extremely hard to name all the amazing mentors that one way or another influenced me, which is actually an amazing feeling for which I'm grateful.

4. How does your work affect society? 

In our path towards AGI, we are building AI systems that learn how to solve problems and make foundational scientific discoveries that could open up new research possibilities. The main principle in machine learning is that it is relatively cheap to scale a system that does things for us (label images, transcribe speech, or translate text) so that everyone can benefit at little-to-no cost. Of course, and with most technology, this is a double-edged sword. AI can provide extraordinary benefits, but like all technology, it can have negative impacts unless it’s built and used responsibly. At DeepMind, we have teams working on technical safety, ethics, and public engagement who aim to ensure that we are constantly anticipating short and long-term risks, exploring ways to prevent these risks from happening, and finding ways to address them if they do. Our goal is to ensure that our impact on society is positive, whilst still pushing the boundaries of what's possible with ML. I'm quite optimistic of the future of AI in society, and hoping to be a positive contributor to the end goal to build AGI that makes the world a better place. 

5. Failures are an inevitable part of everyone’s career journey. What is the most important lesson you have learned during your career when dealing with failures?

Indeed! For me, "failures" (or perhaps more mildly put, setbacks) have given me a lot of perspective and information, oftentimes far more than "successes" have. Whether it is having been rejected from grad school (a few times!), been denied internships (at Google, actually!), or having papers rejected, has always come with a desire to improve, and the shame that comes with the setback is, in a way, fuel to keep learning and keep pushing. At the same time, these also serve a good purpose of making sure we all remain humble. No one knows exactly what they are doing, where they are going, and embracing failure is what I would recommend people do. Also, sometimes, failure may indicate that a particular choice wasn't perhaps appropriate for you, and hindsight will show you that, indeed, that was not the right path or choice. But I also realize how difficult it is and feels once you experience these setbacks. Impostor syndrome is real, and understanding your own experience with it so that you can still be the best you can has certainly proven difficult at times.

6. Although novelty and innovation is the most important factor for technology advancement, yet when a researcher, scientist or engineer has a new idea there are a lot of push backs until they prove that the new idea actually works. What is your advice on how to handle them? Especially for the readers who are in the  early stages of their career.

Interesting, I am not so sure that innovation is so critical, at least not in machine learning and deep learning. There's a feeling that most of the ideas are mostly rediscovered, and I think that is actually absolutely fine. When you work passionately on a topic, try to think carefully about what the end goal is -- don't try to innovate for the sake of innovating. In ML/DL, we have the advantage that the research we do can directly affect real world problems (e.g., protein folding), or answer profound questions about how our models work (e.g., generalization to unseen data). Whatever you are passionate about or choose, try to solve a real problem that you care about, and don't focus on innovation for the sake of innovation. Now, tell that to reviewer #2!

7. If there is one take home message you want the readers of this interview have what would it be?

That failures are part of the research journey. One of my favorite quotes is from one of the scientists that inspired me most as a child: “Failure is success in progress” A. Einstein.

8. Is there anything else you would like to add?

Nothing comes to mind. These are great questions, I especially liked the "failures" question. It's important to disclose and accept that failures are part of our journey, and every researcher or academic that I met has faced some form or another of failure. We should be more open about sharing them.

To learn more about Oriol Vinyals and for more information, visit his webpage.