Abstract: The Internet has become an integral part of everyday life dominated by multimedia applications involving various web, video, and interactive AR/VR applications. A critical performance issue in these applications is the Quality of Experience (QoE) of end-users. Despite the intense research in the past, delivering the best possible QoE for these applications is still a challenging problem because of 1) lack of clear understanding of QoE bottlenecks, 2) single-dimensional approach to resource optimization, and 3) lack of effective methodologies to combine multiple modalities of information. This dissertation develops a suit of solutions to address the above challenges including three multimedia systems: Swift, PARSEC, RoVAR. 

We first discuss how to build accurate objective QoE models using a large-scale user-study to understand and measure different QoE bottlenecks in multimedia applications. Second, we present Swift, an adaptive video streaming system using layered neural codecs. Swift addresses the challenges of compression and latency overheads of traditional layered codecs and enables a practical layered coding that is well suited for streaming under variable network conditions. Third, we present PARSEC, a viewport adaptive streaming system for panoramic videos. PARSEC enhances video quality on the client-side using a deep learning based super-resolution technique. Finally, we present RoVAR, a multi-agent tracking system for AR/VR applications. RoVAR fuses multi-modal sensor information from visual and RF tracking enabling a rich substrate for effective tracking. RoVAR brings together the complementary strengths of data-driven and algorithmic approaches to improve the tracking in terms of accuracy, robustness, and scalability across multiple agents.