Project Objectives
-
Create a delivery service robot system for proof-of-concept purposes and evaluate the company's capabilities in creating and operating a commercial robot system.
-
Operate the robot service in a real cafe environment with customers and staff for human-robot interaction (HRI) and app usability research purposes.
-
Discover technical, operation, and HRI related issues "in the wild" to guide future designs and operations.
-
Impress the public, employees, and potential competitors. Naver Labs is the R&D company of Naver, the "Google of Korea"; expectations were high.
Role
-
UX Designer and Researcher
In a team of 5 UX designers, I led the UX design of the customer interface, backend system, and offline customer experience. I also helped define the initial project scope/requirements while supporting early HRI research and the design of the staff interface.
Schedule
-
Kickoff and Initial Planning in Jun 2019
-
Research, Design, Development from July 2019
-
Release in Cafe Environment in Nov 2019
-
Additional User Tests in Dec 2019
Project Challenges
-
Taking on the unexpected role of an operation manager and project manager mid-project
-
It was the first time for the R&D organization, to develop a fully-fledged customer-facing service.
-
Poor communication and feedback with other teams and departments.
- Feature changes due to limited resources, time, and technical limitations
Process
The UX design team separated the project into two tracks, one for the robot interface and the other for the service system and customer/staff interface. I was a part of the service track, working with another UX designer who was leading the staff interface design. I was responsible for the design of the customer interface, backend system, and offline customer experience.
-
The overall process was a flexible waterfall method (common in Korea) with many of the tasks performed in parallel and iteratively.
-
Although the idea of an indoor cafe robot delivery service was already decided by management, the detailed project goals, requirements, and constraints had to be defined by the UX team.
-
The UX designers later became semi-project managers to facilitate discussion, track development progress, and prepare integration tests with the engineers.
-
Integration tests were executed during the final few months of development.
Requirements & Scenario
What do we want to learn?
With limited time and resources, the UX team decided to define the system requirements according to the key areas the company wanted to test during the pilot.
✓ Vision-based autonomous navigation in commercial spaces
✓ Human-robot interaction during movement and non-verbal robot communication
✓ Order, delivery, and robot-related experience for customers
✓ Robot order and delivery management system for staff
✓ Multi-robot delivery and operation
What do we need to build?
With limited resources and time, I proposed the following requirements to create a simple yet reliable system (MVP) and was able to help the entire project team to focus its efforts on improving the robot functionality and quality of the end-user experience to meet public expectations.
✓ Deliver one order at a time without multiple stops
→ Easier to develop order management and routing system
✓ Fix the table destinations and delivery routes
→ Easier to develop and reduce risks during delivery
✓ Tablet at each designated table to receive order
→ No order mixup or device support issues resulting from QR links
✓ Provide free beverages in return for a survey
→ No payment system or financial transaction issue
✓ Operate at a cafe open to the public in the NAVER HQ building
→ Unrestricted access to use the cafe space
Basic Storyboard
Based on the requirements I proposed, I created a storyboard and a user journey map to help internal stakeholders visualize the requirements better and use it as a vehicle for high-level discussions.
After aligning the storyboard and requirements of the system with the teams, the engineers proceeded to plan the development process and started testing different mechanisms of the robot.
The UX design team split into two tracks (2 designers each), one responsible for the robot and the other for the service system, and we started conducting preliminary research for the design of the interfaces.
Primary Research
As a part of the service track, I conducted a variety of primary research to understand the context of the cafe, understand user expectations and evaluate how existing service robots performed in real service environments.
Cafe Observation and Measurements
We observed the environment of the cafe where the service was going to take place to determine common customer routes, customer tasks, and foot traffic levels to help us choose the appropriate robot delivery routes. We also measured the hallways and furniture in the cafe to define the constraints of the robot proportions and check if any layout changes had to be made to accommodate the robot.
Cafe Manager Interviews and Observation
We interviewed the managers working at the cafe to understand their tasks, roles, and daily schedules to better address their needs for the staff interface and fast delivery. We also asked the managers about common customer complaints and sought their expert opinion on customer priorities.
We also observed and recorded the work processes of the entire team in the cafe to design a system that integrated with the existing process with minimal friction. I also documented and designed the cafe staff work process.
Cafe Staff Work Process
A physical representation of the systems used and the tasks executed by the staff
Cafe Sales Data Analysis
I analyzed the sales data from the cafe to calculate peak/low hours and see when there was a good mix of employees and visitors at the cafe to choose the operation hours of the service. I also used sales data of each menu to determine the most popular drinks and determine the frequency of different group sizes visiting the cafe.
User Survey
We conducted an online survey with 33 employees under the NAVER umbrella to understand their expectations in a robot delivery service in a cafe context. The survey was composed of questions about common user pain-points at cafes, the high-priority qualities of a satisfactory cafe experience and user expectations of a robot service at a cafe.
We used the results to determine which features or components were important for the user interface and discover if there were any new opportunities we could address.
Participant Age Groups
Firsthand Service Robot Experience
We also visited cafes/restaurants that used indoor delivery robots or some form of automation to see if we were missing anything, understand how their operations worked, and see what their limitations were. At the establishments we visited, we rigorously "tested" the robot's autonomous navigation capabilities and made multiple orders to observe the robot in various circumstances.
I also personally visited a Shinkansen (bullet train) sushi restaurant and found that it had the most identical user experience to the system we were planning on developing, using fixed routes, tablets for ordering in each table and a bullet train that delivered and returned to the kitchen.
Robot Cafe
Delivers only baked goods and customers have to order from the counter and indicate their seat location
Robot Restaurant
The ordering interface is accessed via QR code and the robot delivers to fixed locations marked with tags
Bullet Train Sushi
A separate tablet allows users to order from their table with a bullet train that delivers via a rail system
Secondary Research
I conducted additional desk research to understand current attitudes and trends relating to automation and robotics in the food & beverage industry. Research I previously conducted on delivery robots in the market was also useful when looking for points of differentiation for the robot and planning the necessary features of the pilot service.
Research Analysis
Affinity Diagrams
With my co-worker in the service track, we summarized the research results, reviewed them and laid them out through multiple affinity diagrams to help us sort our interpretation of the results. This method was especially helpful for analyzing the staff interview and user survey results, as they were both very qualitative in nature and required a lot of discussion.
We used a lot of post-its; we also made a master affinity diagram (far right)
After making various affinity diagrams, we started noticing how there were a lot of overlapping themes between the different research we conducted. To bring all the results together into one sorted diagram, we created a master affinity diagram with all the major findings laid out in a grid format by topic and research method, helping us summarize all our research in one diagram.
Key Findings
Below are some of the key findings and implementation ideas from the master affinity diagram, sorted according to each stage of the customer experience.
Stages
Research Findings
Cafe staff greet customers before taking their order
#Interview #Observations
Self-order kiosks at restaurants are highly visible with additional offline signage
#Experience
Waiting in line, holding up the line, wasting time are common painpoints in cafes
#Survey #Experience
Images of the menus are vital to users in the ordering process, not text information
#Survey #Experience
Cafe staff always reconfirm the order before payment
#Interview #Observations
Customers want a personable service experience
#Survey #Secondary
Majority of Koreans prefer using self-order kiosks for its accuracy and efficiency
#Secondary
Self-order kiosks provide clear feedback on an order being made and confirmed (e.g. receipt, order no. shown by the counter)
#Experience
Getting the right order is the most important factor in a cafe experience
#Survey
For users, the state of the beverages (spills, temp.) is way more important than speed
#Survey
Users are surprisingly not interested in detailed order/delivery statuses
#Survey
At the robot cafe and restaurant we visited, we had to stand and move to the robot to receive our order
#Experience
Staff label their drinks or read the order when a cup cover is used (can’t see inside)
#Interview #Experience
The Bullet Train sushi restaurant provided a “request help” button on their tablets
#Experience
At cafes and restaurants servers tell customers to enjoy their drinks or food
#Interview #Experience
Implementation Ideas
Set a welcoming tone in the start and onboarding screens
#Interface
Make the tablet interface immediately visible from a distance with signage
#Offline
Make the ordering experience fast and immediate, with no registration process
#Interface
Provide beverage images that are clearly discernible for fast recognition and ordering
#Interface
Clearly ask for reconfirmation of the selected menu before sending the order
#Interface
Make the tone of the interface personable and formal when necessary
#Interface
A self-order tablet would not affect the experience negatively for most Koreans
#Interface
Provide clear feedback on an order being submitted and sent, a sense of assurance, especially with mostly new users
#Interface
Ask staff to recheck the beverages by sticking labels on each drink and allow staff to request the robot to return during delivery
#Operation
Prioritize safe navigation over speed; use cup covers and don’t fill cups to the brim
#Robot #Operation
Provide simple and passive notification on each order/delivery status
#Interface
Make the robot park as closely as possible to the table and make the users sit close to where the robot parks
#Robot #Offline
Ask staff to stick a label to indicate each drink for groups
#Operation
Provide the same call help feature to address unexpected issues, especially those that are robot related
#Interface
Thank customers upon completing delivery to give a stronger sense of service
#Interface #Robot
As you can tell from some of the findings above, the service track played an important role in defining some of the requirements of the robot's interactions and movement to make it appropriate for the cafe service context. Our research also covered issues relating to the delivery routes, the robot's delivery capacity and the operation of the pilot service.
Robot Delivery Routes
Initially the team wanted to use a safer route to minimize friction with bystanders by locating the loading station (area for the staff to load the beverages in the robot) in the main passage way, requiring the staff to make nearly 1/3 of the journey.
However, with the initial mission of wanting to test the autonomous navigation capabilities of the robot, I convinced the team to challenge the robot's ability to avoid moving obstacles (aka bystanders) by proposing a more challenging route that minimized the risk of causing accidents or hindering major user tasks in the area.
Foot Traffic Map
-
Based on my observations of foot traffic levels in the cafe area, I determined that the cashier and pickup locations (blue, orange areas) had to be off limits as there is a risk of causing an accident in the pickup area and users will be standing still, ordering or picking up their beverages.
-
And for the waiting areas (green), customers were more spread out, making it less problematic to pass them as there was more room for movement and higher visibility of the robot in an open area.
Robot Delivery Routes
With the aforementioned conditions in mind, I proposed the delivery route above for the pilot service with the loading station at the far end of the cafe counter and the robot avoiding the cashier and pickup locations by accessing the main passageway through the store area.
Additionally, to test the robot's capabilities in a wide area with many more unexpected variables, I proposed having the robot deliver to both ends of the cafe (tables 1,2 and table 3). We also added a layover station in front of the store area to accommodate a two-robot operation, with a robot waiting in each of the loading and layover stations.
Robot Delivery Capacity
By analyzing the cafe sales data on the number of cups ordered per payment, I estimated the most common group sizes visiting the cafe, as it's common practice for one person to pay for a group in Korea, often through a game of rock, paper, scissors.
The calculations showed that only 3% of total orders had more than 4 cups ordered, giving us the assurance that we could serve most group sizes with a 4 cup capacity. And since most coffee carriers had a 4 cup capacity, we decided that 4 cups would be enough for most orders.
Cups per Order
Coffee Carrier
Due to the 4 cup delivery capacity, we also decided to choose tables with 4 seats (tables 1 and 2 in delivery route image) and place 4 circular cushions on benches (table 3) in order to nudge users to participate in groups of 4.
Cafe Operation Hours
By calculating the average number of cups ordered per hour, we saw that most people visited the cafe in the morning or after lunch. Wanting to minimize complaints from employees and cafe staff, we decided to operate the pilot during the low hours after 2pm for 3~4 hours until closing.
With the number of cups ordered ranging in the thousands, we decided that the low hours were more than enough demand for our pilot service to handle.
Cups Ordered Per Hour
Beverage Menu
In order to choose which menus to serve for the pilot, I calculated the most sold menus during the same service period the year before. Unlike our expectations, it turned out that iced beverages were still popular in the colder months, with iced americano being the second most popular drink even in the winter months.
By using the sales data of each menu, we were able to choose a menu that accurately reflected seasonal demand, although it didn't turn out to be very seasonal.
Menu Ordered Per Menu from Oct~Dec
High-Level Planning
Target Users
Our company's mid-term goal was to create a fully integrated robot system for the new HQ building NAVER was constructing. Because we were investing our time and resources in the pilot to learn lessons for our mid-term project, it made sense for us to target the pilot to employees under the NAVER umbrella.
However, we also wanted a more raw reaction from people who were less biased and unexperienced with robots to discover a wider variety of issues during the pilot.
With the aforementioned goals in mind, I defined our primary target group as employees and our secondary target as members of the public.
Age: 20~40s
Main Target: Employees under the NAVER umbrella
Sub Target: Non-employees
Desired Traits: People with an interest in robots, careful with the robot and proactive with feedback.
User Goals
We defined the user goals of both the customer and cafe staff members based on our research and personal experience with similar services.
Customer
⚑ Understand how to participate in the pilot quickly
⚑ Order immediately and conveniently
⚑ Enjoy beverages in their original state with minimal spill
⚑ Have fun with the robot, maybe take a picture
Staff
⚑ Review the incoming orders quickly
⚑ Efficiently execute robot orders with ease
⚑ Work within the existing system with minimal change
We also assumed that speed, convenience and minimal burden would be important traits throughout the entire experience for both user groups.
Interface Roles and Goals
Based on our understanding of the project requirements, user goals and scenarios, we proceeded to interpret the role and goals of the app interfaces and the offline components needed for the pilot.
Roles
Goals
Customer App
-
Onboard users on process
-
Inform on available menus
-
Notify on order/delivery status
-
Provide the user survey
-
Allow users to start ordering quickly
-
Visual and sufficient menu information
-
Light and timely updates
-
Minimize burden from survey and experience
Customer Offline
-
Promote the pilot service
-
Inform on process and conditions
-
Inform on designated seating
-
Mark space used by the robot
-
Sufficient info to decide participation
-
Easy to find designated seats
-
Minimal robot accidents and inefficiencies
-
Minimal disturbance to existing customers
Staff App
-
Send order notifications
-
Receive status updates from staff
-
Manage and reject orders
-
Start and cancel deliveries
-
Easy to detect and check orders
-
Minimal interaction errors
-
Sense of control for staff/managers
-
Minimal delivery mistakes
Staff Offline
-
Access to interface in workspace
-
Operation manual on system use
-
Easy and safe access to interface
-
Informative and concise
Having defined the high-priority roles of the interfaces and robot system, we started discussing all the features and conditions necessary for the system based on our research and the various benchmark services we experienced.
Through our discussions we created a more detailed list of features and components needed for the system to review with the software engineers and start designing the system. My experience with designing a food delivery app at LINE was also very helpful in defining the requirements for both the staff and customer interfaces.
System Design
Service Blueprint
After having reviewed and confirmed the initial requirements with the related parties, I created a service blueprint of the customer and backend systems as a higher-level summary of the system. By referring to this compact diagram we were able to visualize the relationship between the interfaces, features and backend components in the perspective of the user's actions.
Flowchart Diagram
After designing the overall service blueprint we proceeded to create a flowchart diagram of how all of the system components interacted with each other, including the robot, customer and staff interfaces. Through the flowchart diagram, we were able to discuss in detail with the entire team, including the engineering teams, and see if we were missing anything or check if any modifications had to be made.
In our design of the service system we focused on simplifying the sequence of tasks and exchange of information between systems, looking for areas where we could consolidate a complicated sequence and make it easier to develop within the given timeframe.
We ended up revising the flowchart diagram continuously throughout the design and development process as needed due to changes in features and specs.
App Experience Design
After designing the initial service blueprint and flowchart diagrams, I designed the UX of the customer interface in parallel with the system design, adding more details to both documents after reviewing internally and improving the designs for a better user experience.
Goals
Aside from the goals defined in the earlier stages of the design process, I added additional operational goals that addressed our internal needs as well, including operation and cost efficiency.
Interface Goals
-
Allow users to start ordering quickly
-
Sufficient menu information with visual aids
-
Light and timely updates on order
-
Minimize user burden from survey and experience
Operational Goals
-
Maximize survey response quality by attracting proactive target users
-
Shorten the beverage collection time for operational efficiency
-
Minimize the costs of uncollected/wasted drinks
-
Minimize cases of the customer ordering the wrong beverages
-
Inform people of the company's objectives and capabilities
User Journey Map
To design the experience with the emotional state of the users in mind, I created a simple user journey map and also thought about the points of divergence where users could have a split response. Thinking about the points where users can have a negative response helped me look for opportunities to alleviate user disappointment and find opportunities to magnify certain positive emotions
.png)
Some opportunities I found
-
Lure guests with the robot, or at least images of it to increase participation
-
Make the seats easy to find to minimize confusion
-
Make the onboarding and conditions pages more enjoyable/less uncomfortable
-
Notify users if there is a delay in delivery for transparency
-
Provide a guide on scanning the survey QR code to help users unknowledgeable of QR codes
-
Make the survey easy to read with minimal written answer questions
Tablet UX Research
Additionally, since this was my first time designing a larger tablet app, I conducted some light research into the UX considerations that are often made for tablet interfaces.
Some points I found relevant
-
With the bigger real estate, the primary action has to be visibly distinct and apparent.
-
Landscape mode is the most common/preferred orientation.
-
Swiping interaction is not recommended, due to the wider dimensions of the device.
-
Larger text and simple text hierarchies are recommended, especially with the distance between the mounted tablet and user for the pilot.
Navigation & Page Structure
I created a basic navigation/page structure based on the requirements and basic flow defined earlier in the design process. Because the interface was more or less a kiosk app, navigational freedom was intentionally restricted with the interface guiding the user through the entire process.
Expecting to see various customer and robot related issues, I decided to provide a request staff button for on-site support throughout the entire experience, inspired by the same feature I saw at the bullet train sushi restaurant.
Furthermore, I intentionally restricted the use of sub-pages (additional depth) to make the screen a more effective medium to push information to users, especially in the order status and survey QR pages. When an additional depth was required, I tried to use modal layers to minimize the feeling of moving between pages and still show the main experience in the background.
Core Components
Making rough sketches and wireframes of the necessary features and information, I determined the core interface components that would be needed across the app experience. Referring to the research I conducted on tablet interface design, I limited the text hierarchy to two levels and tried to clearly highlight the primary action button as well.
Base Page Composition
The experience of the app could be largely divided into 4 stages; onboarding, menu selection, order status and survey/completion. Brainstorming all the information, user actions and features needed in each of the 4 stages, I was able to design the base composition of the major pages of each stage. I made variations of these base pages to accommodate specifics pages and use cases.
Starting off the process this way helped me stick to the initial rules I set to limit the text hierarchy and highlight the primary action button, also helping maintain consistency across the pages.
Onboarding
Features
-
Inform on purpose of pilot and the process
-
Inform on conditions, including recording on-site for research
-
Guide non-participants to leave the designated seating area
With our survey results showing respondents wanting a personable experience, I wanted to make the onboarding experience more enjoyable by using soft animated illustrations to liven up the tone and align it to the casual atmosphere of the cafe. With the visual designer in charge, we fine-tuned the style of the illustration/designs and defined the requirements of the illustrations together.
Onboarding illustrations by the visual designer
Furthermore, with the slightly inquisitive and witty nature of the robot persona set by the UX designers in the robot design track, I tried to do the same for the tone of the language used in the interface, making it a bit more casual with hints of wittiness.
In the case of the onboarding pages, I made the labels for the next button reflect the response we sought from users in each onboarding page, in a casual yet affirmative way, making the experience more amusing and encouraging users to keep their word on participating in our survey.
I also added a modal layer to notify non-participating users who chose the "participate later" or the "cancel" buttons to leave the the designated seating area for participants as one of our objectives was to get many people to participate.
Onboarding Prototype
Menu Ordering
Features
-
Inform on beverage choices with images for quick recognition of each menu
-
Inform on the maximum 4 drinks limit
-
Reconfirm the order with the user to minimize the risks of users making the wrong order
With just 6 menus served, I wanted to simplify the ordering experience as much as possible, showing all the menus in one page and allowing quantity modifications within the menu page, essentially combining the features of a standard menu and cart pages into one.
I also convinced the visual designer to use photos of each menu instead of illustrations to make the images function more effectively as visual aids for users to quickly recognize each menu.
To make the ordering experience lighter and seemingly faster, I tried to reduce the feel of transitioning between pages when the user initially confirms an order and the app shows the order again for reconfirmation.
I did this by using the idea of cards reshuffling with only the selected beverages remaining in the page, reducing the need for a separate order confirmation page (refer to prototype below).
Ordering Prototype
Furthermore, I purposefully gave a "sending order" loading page after the user makes an order to provide a natural and clear feedback for the user on the order being sent to the staff system. I believed that it was important for users to be confident that their order has been sent as most users will be new to experience such a robot delivery service and immediately transitioning to the sent status could be kind of sudden.
Order Status
Features
-
Provide users a window to cancel their order immediately after ordering
-
Inform on each status of the order preparation and delivery process
-
Inform on any delays or unexpected changes in the order
-
Provide the survey QR code for users to access the survey via their phones
The composition of the page was standard, with animated illustrations representing each stage, bold text providing a description of the status and sub text showing supplementary information or informing users on unexpected changes.
I tried to make the tone of the language change appropriately to the situation and context. Overall, I made the tone a bit more formal than the onboarding stage, to show a sense of credibility and confidence to the user. However, when showing status information about the robot, I added hints of wittiness or levity in the language to be consistent with the robot's persona. And when there was a delay or cancellation by the staff, the tone was set to be very formal with an apology.
The UX designer working with me in the service track proposed the idea of making the survey seemingly a part of the order and delivery experience instead of an optional step shown at the end. Acting on this idea, I made the survey QR code appear as the last step of the experience after the user completes collecting their order.
Anticipating users to be preoccupied with their drinks or the robot, I used a notification-like sound to bring attention to the tablet when the QR code is shown after collection. I additionally provided a tool tip popup within the page in a non-intrusive form to guide users who are not familiar with QR codes, making sure to also remind them that they can request help from staff.
Order Status Prototype
Although the status updates are controlled by the staff and robot system, for the prototype a separate button was made to simulate the changes.
End and Thank You
Features
-
Express gratitude for participating in the pilot service
-
Provide additional content concerning the company and its research
-
Give access to the survey QR code for users who may need it again
-
Go back to the start page and reset
I expected some users to accidentally proceed to the end page without scanning the QR code, especially with the timer logic to automatically go to the end page after 2 minutes. So I added a "See QR again" button in the end page and locate it at the upper left corner where back navigation buttons are usually placed; once pressed the user is shown the QR code through a modal layer.
I also added a feature to show information about Naver Labs and its research because this was one of the first public service robot tests the company ever operated and it was a good opportunity to raise awareness about our research. Although I was initially looking to use a video in order to reduce user effort in interacting with a mounted tablet, I resorted to a simple slideshow format as there were no plans to create a promotional video for the project.
End and Thank You Prototype
Request Help
Features
-
Provide a request help feature to call for on-site support
-
Allow staff to end the help request
In order to make the request help feature light and minimize additional development, I made it possible for the staff to end a help request in the same tablet app through the "Cancel Request" button intended for the customer to change their mind after making a request. I was satisfied with myself for thinking of this solution(?) as it made a separate interface to receive and complete help requests unnecessary.
Furthermore, I proposed the idea of creating a LINE messenger chatbot to push help request notifications in a group chat with the members supporting the operation on-site. It not only helped us avoid making a separate interface but it was also an effective communication channel in notifying the team as all of us were using LINE messenger for work purposes.
Request Help Prototype
LINE Chat Bot
Time Out Reset
Features
-
Automatically reset or end experiences when users leave mid-way
-
Allow users to continue the experience when needed
Due to the tablet app being more of a self-order kiosk, it needed a timer logic to reset itself when a user leaves mid-way, since new customers may be confused by a page showing an ongoing experience. However, the timer logic and conditions of reseting an experience had to be made appropriate for different stages of the experience.
For the onboarding stage and the rest of the process after the survey QR is shown, the experience could be reset without notifying the user as it was safer to assume that the user actually left if there is a long delay. However, for the menu page, I believed that it was possible for users to take more time ordering and users will be disappointed if the effort they made to choose menus is reset without notice. Due to this reason, I added a separate modal layer to ask whether the user is still ordering or if the user wants to start a new experience, giving the option to continue ordering if needed.
Simulated Walkthrough for Review
We reviewed our initial app designs internally and later conducted a simulated walkthrough with the app prototypes. During the walkthrough we played the role of the customer, staff and robot, talking-out-loud to share our state of mind and opinions. Through this process, we were able to understand certain issues we hadn't thought of before and make improvements in the system flow and apps before finalizing the design.
.png)
We simulated various cases and the entire experience in the office and on-site
Before Release
Development and QA
Although the specs of the app were confirmed internally, due to unexpected difficulties in the development of the robot system, certain low-priority features had to be taken out, including the slideshow and in-app analytics. Nevertheless, we were still able to create a well designed MVP app that met the high-priorityrequirements and functioned normally during the test.
The UX designers managed much of the app QA tests and the integration tests with the robot system. During the QA process we realized how it was important to create an efficient testing environment with the ability to simulate changes from the robot, as testing became tedious and difficult due to the interdependency between the different components of the system.
Offline Information Design
For the actual operation of the pilot service, I had to plan the designs of offline notices, posters and signs for use in the cafe area. Making sure to consider the visibility and ease of understanding of the information shown, I tested different sizes and colors and decided to use the same primary action button color for high-priority information, including the signs for the designated seating and the layover station of the robot. Posters and signs were also placed with a consideration of the routes used by customers at the cafe.
User Survey Preparation
I also created the survey used for the pilot service in cooperation with the HRI design track. To reduce user burden and minimize drop-off, we composed the survey with mostly likert scale questions and just 2 short answer questions. Questions covered topics such as the initial impression and understandability of the robot's interactions, the usability of the customer app and user expectations in robot service. More detailed user tests and interviews were planned to be conducted separately after the pilot service.
Mobile survey form accessed via QR code shown on tablet
On-Site Recording
We were also planning on recording user interactions with the robot for research purposes. I had to review the implications and make the necessary adjustments with the legal team and prepare the purchase of the recording devices and the locations for recording with the designer in the service track. The video recordings later became handy in reviewing unexpected on-site issues and common user interactions with the robot and their reaction.
.jpg)
Installing cameras at each table and around the robot path for research purposes
Pilot Service Results
Although we faced a few technical bumps along the way, the pilot service was a success, gaining a lot of attention and the system functioning normally throughout the test. With the support of all the teams, we were able to address various operational and technical issues promptly on-site and improve the stability of the system as time progressed.
Customers had no difficulty understanding the use of the customer app and users rarely needed support, except on the few occasions when there were technical problems with the system. With the attention to detail made by the designers in the HRI track, the robot's interactions were seamless and in most cases intuitive enough for the users to execute the task of collecting their drinks with minimal confusion.
Full App Experience with Robot
User Response
Both employees and members of the public were excited by the presence of the robot, many uploading their experiences on instagram and some recruiting more participants by word of mouth. More than half the participants recorded video or took photos of the robot and app interface, making it a rewarding experience to see so many people enjoying the experience.
_edited.jpg)
People huddling around to take photos of the robot
.png)
Instagram posts of the robot and app interface
Few Unexpected Findings
Through the user survey results we found that users understood the robot's persona as intended by design and that their evaluation of the robot and overall service was very positive. However, through the short answer comments made by the users we realized that many users had very specific expectations that we didn't address, especially concerning the interaction methods of the robot. Furthermore, through our observational studies on-site, we not only found technical issues that we could have better addressed but also issues of robot abuse, bystander interaction and user safety during navigation.
Based on the lessons we learned through the operation of the pilot service, we made plans to continue our research into human robot interaction methods, develop solutions for some of the common faults we saw with the robots and start preparing for our mid-term goals to create variations of the robot for the services we were planning for the coming years.
Looking Back
There are still countles technical limitations that make it difficult for robots to be fully equipped for a real "wild" service environment, where timely responses to different user cases and obstacles are necessary. However, regardless of current technological restrictions, I felt that it was still important to continue pushing forward with research in the field of HRI for autonomous navigation, even in simulated or highly controlled environments. At a time where mobile robots are gaining a lot of attention, there is still a long way to go to design human-friendly navigation and context-appropriate interactions that are vital in creating a sustainable relationship with humans on roads and hallways.
Furthermore, with the limited presence of robots in our everyday lives, I felt that people were overly positive towards robots, a short term effect that will eventually erode as robots become more commonplace in our society. In the long-run creating a sense of mutual trust and respect between humans and robots would be vital for their continued development.
I believe that this experience with AROUND C not only gave me a preview to the potential of robotics and the importance of HRI, but also the realistic difficulties and priorities we need to address to really provide a satisfying robot service experience.