Industry Leaders in Signal Processing and Machine Learning: Henrique Malvar

Dr. Henrique Malvar is a Fellow of the IEEE and a Member of the US National Academy of Engineering. He is a Distinguished Engineer at Microsoft Research, having previously served as Managing Director of the largest Microsoft Research Lab in Redmond, WA. Prior to joining Microsoft in 1997, he was Vice President of Research and Advanced Technology at PictureTel Corporation. Prior to that he was a professor at University of Brasilia. He holds a PhD in electrical engineering and computer science from the Massachusetts Institute of Technology. He has received a Claude Shannon-Harry Nyquist Technical Achievement Award from the IEEE Signal Processing Society, an Amar G. Bose Industrial Leader Award also from the SPS, and a Wavelet Pioneer Award from the SPIE.

We approached Henrique Malvar with a few questions:

1. In your own words, please tell us about your background.

I have been a Distinguished Engineer at Microsoft for almost 20 years, always as a researcher and research manager in Microsoft Research (MSR). I was the MSR Redmond lab director from 2007 to 2010. I started at MSR by creating and leading the Signal Processing group for many years. Today, I lead a research area, with a few groups looking into new input/output interfaces and new interaction modes between humans and computers, with some emphasis on accessibility.

My preferred technical topics are multirate signal processing and multimedia data compression. In multirate processing, my PhD thesis help spur the topic of lapped transforms, which I cover in reasonable detail in my book “Signal Processing with Lapped Transforms”. In applied mathematics, continuous-time extensions of extended lapped transforms are referred to as “Malvar wavelets”. Lapped transforms are used in many practical applications, such as several speech and audio compression formats, the JPEG XR image compression format, modulation schemes for data transmission over power lines, and also modulation techniques for next-generation 5G mobile networks. In data compression, I designed audio compression systems for PictureTel and Microsoft, and I contributed with components of many standard codecs, such as G.722.1, H.264, RemoteFX, and JPEG XR. I have also done research in other interesting signal processing topics, such as audio noise reduction, audio beamforming with microphone arrays, and demosaicing of single-CCD raw images.

I have served the community through committees in the National Academy of Engineering and the National Science Foundation, member of the technical program of many conferences, a perennial publications co-chair of the IEEE Data Compression Conference, plenaries co-chair for ICASSP in 2018, and member of many advisory committees at MIT, USC, EPFL, University of Washington, and others.

2. What challenges have you had to face to get to where you are today? 

I consider myself lucky, as I had to face challenges, but they were not too difficult; optimism and hard work was what it took to overcome them. First, on the personal side, moving from Brazil to the US, ensuring that the family would have a good experience with the transition. On the professional side, my first job in the US was being the manager of a small research group in a high-tech company, so to a large extent I had to “learn by doing”, as I had no experience with the work culture in the US.

An interesting constant challenge has been how to manage and conduct research that leads to real impact. Microsoft Research has an environment where that is welcome and desired, for example by researchers and engineers from the research lab collaborating closely with product teams. There’s no magic formula on how to succeed at technology transfers, so you have to approach each project with an open mind and willingness to work in many different dimensions, for example debugging code in Windows, which can be difficult. The reward is having our research ideas and technologies being used by millions, or even billions, of people.

3. What was the most important factor in your success?  

There really isn’t one most important factor, as there are so many that mattered. First, a bit of luck, as life is not as predictable as we would wish it to be. For example, being at Microsoft Research for so many years, a place where there’s significant flexibility in the pursuit of innovation. Also luck in having had many supportive managers, who trusted me with the opportunities to push my ideas forward. Then a willingness to learn, to not assume that new projects should be approached in the same way as old ones. For example, allowing team members to tackle new technical challenges outside their previous areas of expertise; many times I was impressed by people being very creative and driving innovation in areas where they had not worked before. Another dimension is to look for doing things differently not just because it’s a technically motivating challenge, but because you see the path for real impact. One example was my transform/quantization work on H.264. It started by asking the question “why should the core transform be a very close approximation to the DCT (discrete cosine transform)?”. That was a common assumption in video codec designs at the time. By relaxing that constraint, I developed a simpler transform with smaller coefficients with integer-arithmetic calculations. That ended up having the same performance as the previous DCT-like transform, but changed the decoder to be implementable in 16-bit arithmetic, instead of 32 bits, which led to significant speedup and cost reductions.

4. How does your work affect society? 

As an individual contributor, I was happy to see that my development of lapped transforms has affected the design of many kinds of modern technologies such as multimedia codecs and digital communication systems. I was one of the several dozen engineers that contributed to the H.264 video format, which today is used by billions of people. As a manager, I am happy that I was able to lead and support teams that developed new products and improvements to products such as Microsoft Windows and Office, used by billions of people. More recently, I have been very fortunate to lead a research team at Microsoft developing technologies to help people with disabilities. We developed native eye tracking interfaces for Windows (the Eye Control capability), allowing people with limited or no mobility to use a computer by controlling it with their eyes. We also developed the Soundscape app, for people with low or no vision to receive information about points of interest around them via 3D audio, allowing then to “follow the sound” instead of following screen images, to explore the world independently as they walk out and around.

5. If there is one take home message you would want the readers of this interview to take away, what would that be? 

Always be willing to learn and to revisit previous assumptions. Assumptions on how systems should be architected, on the technical background and experience necessary for team members, and many others. On learning, being on top not only of technical/scientific evolution from academic papers, but also following closely the evolution in industry, new products, new modules, as those can also be important tools to enable you to come up with innovative ideas.

6. Failures are an inevitable part of everyone’s career journey. What is the most important lesson you have learned during your career when dealing with failures?

I have been involved with research for decades, and we can say that research is a business of constant failures, as many of our ideas do not work out as expected. The most important aspect is to learn from such failures, try to understand what went wrong, as that will help in increasing your chances for success next time. Another important aspect is to own our failures, to not find blame somewhere else. If we look for external factors to explain or justify our failures, then we learn much less.making our ideas stronger, and work to overcome those doubts via theory and evidence that an idea works.

7. Although novelty and innovation is the most important factor for technology advancement, when a researcher, scientist or engineer has a new idea there are a lot of push backs until they prove that the new idea actually works. What is your advice on how to handle them? Especially for the readers who are in the  early stages of their career.

Start with being humble: most ideas we have are not really new. Do the research to find out the state of art in academia and industry around that problem. Then, make sure you convince yourself that the idea can really work, maybe through some simple test code or preliminary hardware prototype. Then, to seek support, step out of thinking about how it works, and how brilliant it may be, and look from the outside, for the perspective of who would benefit from that idea. Then, invest time on how to talk about that benefit, why that idea will make a difference once it’s implemented. Many times I actually stopped working on something when I got to that step, because I realized that it may have been a very cool idea, but it wouldn’t lead to significant impact. Also, embrace collaborations; more often than not your idea will require a team, so be ready to let go of the idea as becoming the team’s idea, so you don’t think of yourself as more important than the team.

8. Is there anything else you would like to add?

Just another thank you for inviting me to participate in this article series.

To learn more about Henrique Malvar and for more information, visit his webpage, or listen to his interview in the  UIUC Immigrant Computer Scientists Podcast series.