Many intractable problems in multimedia analysis have been substantially aided by taking context into account, such as object recognition, image search, and photo management. Multimedia is not the only field that can benefit from recognizing the importance of context. Researchers working in social computing, social signal processing, human machine interaction, robotics, computer vision, computer security, or any other field concerned with the automatic analysis of (and response to) human behavior may be greatly aided by understanding the role context plays. Indeed, some might say that understanding social context is one of the grand challenges of these fields. But how does context affect social behavior? And, further, as researchers, how can we build autonomous systems that take advantage of this contextual information? In the May issue of IEEE Signal Processing Magazine, the column article by Laurel D. Riek and Peter Robinson broadly introduce social context and discuss some of the challenges involved in building real-time systems that can process and respond to this contextual information.