# Rocksmith+ Case Study | Multi-Platform Game UX url: https://bramha.work/work/rocksmith route: /work/rocksmith format: text/plain; charset=utf-8 llms-txt: https://bramha.work/llms.txt llms-full: https://bramha.work/llms-full.txt description: UX lead case study: one scalable UI system across 5 platforms for a music learning game. Game designer and UX designer for games — accessibility, input systems, and console/mobile parity. --- Ubisoft PC Game Console Game Accessibility # Making guitar learning accessible to 1 million users Releasing a new learning game on 5 platforms with a single, scalable UI system. Role UX Lead for Mobile & Multi-platform Interactions Timeline ~2 Years Team UX Director · 2 UX Designers · 2 UI Artists · 2 UI Engineers ## Executive Summary 5 Platforms 1 UI System 01. The Problem Fragmentation: Rocksmith+ faced a massive scaling challenge: launching a PC-first MVP on Mobile and Console simultaneously. Without a unified framework, the team faced alienating users whose context would switch based on a device and burdening triple the maintenance cost for maintaining all the platforms. 02. The Solution Building a responsive UI system: I architected a technical UI system in the Snowdrop engine that scaled responsive UI across different PC monitors, TVs, tablets, and mobiles. This ensured visual parity across devices ranging from 4:3 iPads to 21:9 Ultra-wide monitors without bespoke adjustments. Unified Input Philosophy: Mapped divergent inputs (Keyboard, Gamepad, Touch) to singular behavioral outcomes, reducing cognitive load for users switching between platforms. Driving Global Collaboration: Orchestrated design collaboration between India, Japan, and the US to share user insights and create a holistic understanding of user needs across all platforms. This was the legacy Rocksmith experience — on a big screen with lots of space, with no choice except an expensive desktop or a game console. The traditional Rocksmith experience was always on a big screen Rocksmith+ gave users the option to learn guitar anywhere they wanted, on any device they wanted with 1 subscription. This unlocked a whole new segment of millions of users. The new Rocksmith+ experience ## Context I joined the team in 2020 when Rocksmith+ was in pre-launch production and Ubisoft wanted a global release on mobile, PC, and next-gen game consoles (PlayStation 5 and Xbox Series X). Very few people on the UX team had prior experience building mobile user experiences and responsive UIs and interactions for multiple platforms. The MVP for Rocksmith+ was also built only for PC. My goal was to deliver it on Mobile and Tablets in addition to Consoles to invite a whole new audience of guitar learners from around the globe. The challenges I faced at the start And drove to these outcomes ## Research The first piece of work started with extensive user testing. I requested internal and external volunteers to play a crude, ported version of the PC game on iPad without any scaling and no interactions except simple taps. My first few initial tests were to discover how people set up a portable device compared to playing on a PC or a TV to understand the context and environment in which Rocksmith+ would be experienced. The tests gleaned insights using three primary methods: - Observational studies and contextual interviews with internal and external volunteers (4–6 waves with panels sized 12–15) - Diary Studies to see if and how user needs evolved as users went from beginners to power users (10 users) - Surveys after each playtest to track key success signals via ability to learn on mobile as compared to PC, satisfaction of experience, and NPS surveys Throughout virtual and in-person testing sessions I made notes of various environments users learn guitar in: - Placing their phone on various physical contexts ranging from kitchen countertops to dedicated desks - Plugging in their guitars into a myriad of equipment — from pedals to headphones so a parent can practice without disturbing the children - Placing devices at different distances to have room to play their instruments A setup showcasing the use of a kitchen counter and an iPad Another important highlight was the amount of personalisation needed: A simple user — acoustic guitar and just the device A hardcore user — electric bass, 2+ musical accessories, and an iPad ~90% potential new users Wanted a hassle-free and wireless experience. ~5% of total responders Wanted a hardcore experience with all their gear. 77% of these were legacy Rocksmith users. We also included questions related to eyesight. I wanted to know how many users suffered from eyesight issues including near/far sightedness and colour vision deficiency. I personally suffer from near sightedness and colour vision deficiency which impassioned my desire for an accessible UI. ~63% of total responders Suffered from at least 1 eyesight issue. ~31% of total responders Suffered from either near or far sightedness. I wanted to know how many contexts the general user stories would have and how many devices a user imagined themselves playing on. 87% correlation Between size of device and complexity of setup. ~63% of responders Could see themselves using more than 2 devices, but not more than 2 setups. ~92% of new learners Said they would start with acoustic guitars and mobile phones. What I quickly realised was that the UI cannot be one size fits all for these many user segments. Users need tons of customisability to change the experience to their needs: - The ability to zoom in and zoom out on the guitar lessons based on device distance - The ability to scale the UI so that buttons are bigger if the device is farther away - Other less frequently used but important controls such as changing field of view ## Responsive UI I first began with working on a core responsive UI that would offer the base experience on all devices. It involved not only designing and validating layouts in Figma, but also working closely with UI engineers to build appropriate tools: - UI component anchoring and scaling so that UI elements know exactly where to stay on the hundreds of devices (TVs, monitors, tablets, and phones of up to 8 standard aspect ratio configurations) - Aspect ratio based scaling rules — percentage based scaling to maintain consistency without too many breakpoints - Aspect ratio break point table for bespoke rules for brackets of devices The tricky part was making 3D objects in the UI responsive. I worked very closely with the UI engineers to develop a custom system where an anchor point and multiple boundary points were projected from the 2D screen into 3D space. The camera adjusted so that the 3D point was placed perfectly where the 2D point was supposed to be on screen. It would automatically know how much to zoom in or out and how to move left and right to make sure all the 3D objects appeared at the correct places on the screen no matter the aspect ratio and screen resolution. The MVP achieved usable display results on all 5 target aspect ratios encompassing 90% of expected devices out of the box. Because the 3D element is effectively treated as a 2D element, it adjusts without any special rules when playing on phones or tablets Live test demonstrating the 3D-to-2D anchoring system, ensuring all elements remain legible and correctly positioned Fun fact — the UI system worked so well that one user got it working on their GPD Win4 handheld gaming machine ### User Testing Gallery ## Accessibility & UI Scaling Accessibility was a core tenet of the experience. One of the first accessibility features we developed was allowing users to modify the scale of their UI to fine tune their mobile and tablet experience based on how far they kept their device. The distance between user and device changed heavily based on skill level, instrument type, and physical setup. Users could customise UI scale based on their distance from the device This customisation system was easy to build as my team had already worked on a dynamic system to make our 3D UI responsive. The original system, being built on solid technical foundations, meant that users could change their configuration at run time — whenever they wanted and however many times they wanted. ## Unified Input System Since one subscription of Rocksmith+ gave access on every platform, we expected users to use the app on multiple platforms — think Netflix. We needed consistent outcomes from the same input on different platforms. From all our extensive user studies, one thing was absolutely clear — users will only have one hand free as they had a guitar in the other. Whether a mouse, a game controller, or a touch screen, the entire interface had to be usable with only 1 hand. 86% of participants Wished the interactions were similar to Netflix or Spotify on mobile. 16% of participants Used Netflix or Spotify on their console — we could learn from these apps but not rely on them for consoles. 72% of participants Found it difficult to move from mobile to console or PC due to lack of contextual input guides. 47% of participants Failed to recreate contextual outcomes (e.g., hovering for tooltips) when switching platforms. Since users were occupied with guitar chords and memorising complex techniques, it made no sense to add the cognitive load of also learning how to interact with this app on multiple platforms. The user knows what they want, but how do they recreate it on a different platform? I created a 3-dimensional interaction matrix that mapped outcome → component → action on each platform and conducted many surveys to build an exhaustive list of commonly mapped translations between interactions. For example, hovering with a mouse cursor on desktop was majorly translated as touch and hold for mobile users. A sample abstraction of the complex interaction matrix How 3 different actions consistently lead to the same outcome on any platform 80% average completion rate per user Users could consistently achieve the same activities across platforms despite changing devices. ## Platform UI Adaptations The original Rocksmith+ experience was built with only PC in mind , so many components had to undergo changes to be intuitive for mobile and console UI interactions. For example, on PC you scroll through a carousel by clicking arrows. On touch devices, you simply swipe. A large part of my role was making all these small changes to the global design system and its component libraries to improve intuitiveness on mobile, PC, and consoles. Touch focused UI rarely has arrows to scroll — the content extends slightly off-screen to hint at scrollability ~90% of participants Felt confident about the mobile UI because it looked similar to an app they already used (e.g., Netflix, Spotify). ## Console Navigation Challenge Another major issue I tackled was menu navigation for consoles. The original prototypes included expanders with nested scrolls to hide secondary information. While common on desktop and mobile, on consoles this created a major issue. An example of nested scroll — two scroll bars denoting independent scrollable containers On consoles, you cannot independently focus on a separate item — all directional input is mapped to the item in focus. You can only scroll the expanded container all the way up till you reach the header to close it. My first gut reaction was to try mapping one stick for the nested container and the other for the root container. Mapping two sticks to two containers — a solution that failed This solution had 2 core problems: - Not a natural pattern in video game menu UIs. No mainstream games use two sticks for two UI elements — not intuitive to users - One of our main goals was one-hand operation as the other hand holds a guitar. This pattern required 2 hands Having to put down the guitar to hold the controller with both hands was a no-go Going back to the drawing board, a technical designer proposed using a two-axis design pattern — vertical axis for moving between components, horizontal axis for accessing information. I created multiple quick iterations in Figma to explore commonalities across platforms. I worked very closely with a Senior Engineer on building consistent focusing in a two-axis interaction — whenever users would vertically scroll, the item closest to the current item in focus would gain focus. When you scroll up from here, what item do you expect to focus on? When we specifically asked playtesting moderator "comparing to other main menus such as your console's game library, did you have any issues with the Rocksmith+ main menu?" — 100% of results came in as "no" . This was a big success. Control scheme for consoles Control scheme for desktop ## Result Rocksmith+ launched in open beta and swiftly followed a global launch in 2022. The mobile version was able to unlock a completely new market of on-the-go learners who did not have the money, time, or privilege of learning to play the guitar on a computer or a game console. The new and improved accessibility feature set also garnered critical acclaim when Rocksmith+ won the Game's Accessibility Conference's award for Best physical / mobility accessibility . ## Learnings Working in a global team based out of San Francisco, Osaka, and Pune, I quickly learnt that project-wide impact on accessibility and a multi-device philosophy won't automatically result in a great product simply with my personal contributions. A large part of team alignment came from rooting discussion in user needs and coaching the team what a truly multi-device experience means — which included numerous coaching and workshop sessions on mobile interactions and games. This was further challenged by language and cultural barriers in the midst of COVID-19 which meant we had to take an approach of overcommunication and actively building interpersonal relationships. While I am extremely proud of the product we put out which helps millions of humans learn guitar at their own pace at an affordable price point, I am even more proud of the team culture we built that allowed people working on it to grow their careers and improve their skill sets. ## Testimonials Hiroshi Ogawa Lead UI Engineer, Ubisoft > Bramha made the foundation of Multi-platform UI, which was one of the biggest challenges in the project. His knowledge and insight always pushed our discussion forward. A clear design-based dialogue was productive and helpful in the cross-studio project in Japan and India. I respect his courage to accept the change, which is UX/UI design's most challenging part of improving the game while managing our resources. Kaiwen Young Director of User Experience, Ubisoft > Bramha's passion, knowledge and communication style contributed greatly to the quality of our project and team UX culture. Rohit Suvarna Senior Game Designer, Ubisoft > Bramha has a deep understanding of how to design user interfaces that are intuitive, effective and visually appealing. He is also an expert in user experience research and knows how to use data to drive his designs. Utkarsh Bagade Senior Engineer, Ubisoft > Bramha's approach to designing complex game systems always put UX at the highest priority. I'm very impressed by his ability to understand the tech (tools and engine) and designing features that take advantage of the current frameworks and help improve them.