User Experience and conversion Rate Optimisation
Introduction
As part of my role as a Product Designer at Bestseller, I initiated and lead a usability study in a dynamic collaboration with VILA, a prominent fashion brand, aimed at optimising their online shopping platform to deliver an unparalleled user experience. From conducting user interviews to analysing data insights, every phase of the project is dedicated to enhancing usability and driving conversions.
About the brand: VILA
Since 1994 VILA has been a leading character on the feminine fashion scene focusing on individuality and a playful approach towards contemporary womenswear. At the heart of the brand is a passion for self-expression with a growing focus on mindful fashion habits, expressed in a carefully selected edit of honest design build upon authenticity and empowerment. Defined by a dynamic union of innovation and simplicity, VILA’s creative and sensual universe aims to inspire and support women navigating in an ever-changing world.
Since 1994 VILA has been a leading character on the feminine fashion scene focusing on individuality and a playful approach towards contemporary womenswear. At the heart of the brand is a passion for self-expression with a growing focus on mindful fashion habits, expressed in a carefully selected edit of honest design build upon authenticity and empowerment. Defined by a dynamic union of innovation and simplicity, VILA’s creative and sensual universe aims to inspire and support women navigating in an ever-changing world.
The Process
Embarking on a usability test journey requires careful consideration and planning. Before diving into the execution phase, it's essential to envision the entire process, from defining objectives to analysing results. This initial step sets the stage for a structured and insightful evaluation of user experience. By mapping out the process ahead of time, we ensure that each stage is well planned and executed, ultimately leading to valuable insights and actionable improvements.
Plan The Test
To ensure the usability test effectively addresses our objectives, we began by defining clear goals and objectives. This involved outlining the scope of the test and identifying key areas for focus. We developed test scenarios and tasks that align with our objectives, ensuring they provide valuable insights into user behaviour and interactions. Additionally, we decided on the methodology for the study, opting for an in-person, moderated usability study to facilitate deeper understanding and interaction with participants.
Study Goals
- Identify Usability Issues
- Identify Usability Issues
- Evaluate Navigation
- Understand User Behaviours
- Test Cart Interaction
- Evaluate Mobile Responsiveness
- Collect User Feedback
- Identify Platform-Specific Challenges
- Gather Qualitative Insights
- Explore Opportunities for Improvement
Study Scope
🔀 Navigation across categories
🔍 Product search
📑 Product listing pages
📄 Product pages
📱Mobile responsiveness
🌁 Homepage exploration
📣 User feedback
🛑️ Error rates
🔀 Navigation across categories
🔍 Product search
📑 Product listing pages
📄 Product pages
📱Mobile responsiveness
🌁 Homepage exploration
📣 User feedback
🛑️ Error rates
Study Type
In-person, Moderated Usability Study
Location: Amsterdam Office
Duration: 45 - 60 minutes
Study days
Day 1 (Friday): 3 sessions
Day 2 (Saturday): 3 sessions
Study Steps
- Introduction
- Pre-test interview
- Task
- Post-test interview
Study Task
We asked the participants to browse VILA’s website envisioning a real life scenario where they have been invited to a cocktail party and they need to find an outfit and a jacket for the occasion
Recruit Participants
As part of our recruitment process, we aimed to gather a diverse group of participants reflective of VILA's target audience. Initially, we planned to recruit 6 participants within the age range of 30 to 60+—the demographic typically associated with VILA's current user base.
Participation incentive
🎁 50 Euro Voucher from VILA
🛍️ VILA Goodie Bag
Participation incentive
🎁 50 Euro Voucher from VILA
🛍️ VILA Goodie Bag
Contacting existing VILA users through email
Initially, we sought to recruit existing VILA users through targeted email outreach. With the promise of a 50 euro voucher from the VILA website and a VILA goodie bag, we crafted personalised emails and dispatched them to a pool of at least 300 recipients. However, despite our best efforts, the response rate was disheartening, with only one participant expressing interest.
Announce the study in an Instagram post
Undeterred by this initial setback, we pivoted our approach and leveraged the power of social media to broaden our reach. Utilising our Instagram platform, we announced the Usability Study and encouraged our followers to share the post with their networks. By boosting the post and mobilising our colleagues to amplify our message, we successfully garnered at least 20 sign-ups. Here, we introduced a sign-up form allowing potential participants to specify their age range. After thorough discussions with VILA regarding their user demographics, we decided to recruit 4 participants from their primary age group and 2 from the 20 to 30 demographic, representing a target audience for future growth.
Key Learnings
This experience served as a valuable lesson in the efficacy of social media as a recruitment tool for research studies. Moving forward, we are committed to harnessing the potential of digital platforms to engage with a broader audience and facilitate meaningful participation in future endeavours.
This experience served as a valuable lesson in the efficacy of social media as a recruitment tool for research studies. Moving forward, we are committed to harnessing the potential of digital platforms to engage with a broader audience and facilitate meaningful participation in future endeavours.
Prepare Materials
During the preparation of materials step, the focus is on creating the necessary documentation and tools to facilitate a successful usability test. Here's a breakdown of the tasks involved:
Script Creation
- Develop a detailed script or test plan outlining the structure and flow of the usability test.
- Include an introduction for welcoming participants, explaining the purpose of the study, and providing instructions.
- Outline the specific tasks and scenarios that participants will be asked to complete during the test.
- Incorporate probing questions or prompts to encourage participants to provide detailed feedback and insights.
Task Development
- Create tasks that are aligned with the goals and scope of the study, focusing on key areas of interest.
- Ensure that tasks are realistic and representative of typical user interactions on the website or product.
- Sequence tasks logically to simulate a natural user journey and progression through the site.
- Balance the complexity of tasks to accommodate users of varying skill levels and familiarity with the platform.
Interview Question Preparation
- Develop a set of pre-test and post-test interview questions to gather qualitative insights from participants.
- Tailor interview questions to address specific study objectives, such as uncovering usability issues, identifying user preferences, and assessing brand perception.
- Frame questions in a conversational tone to encourage open-ended responses and deeper exploration of participant experiences.
- Include probes or follow-up questions to delve into specific areas of interest or clarify participant responses.
Consent Form Preparation
- Draft a consent form outlining the purpose of the study, participant rights, and data handling procedures.
- Ensure that the consent form complies with legal and ethical requirements for user research and data collection.
- Clearly communicate the voluntary nature of participation and the option to withdraw consent at any time.
- Obtain necessary approvals or review from legal and compliance teams to ensure adherence to privacy regulations and best practices.
Alignment with Study Goals and Scope
- Ensure that all materials, including the script, tasks, interview questions, and consent form, are closely aligned with the goals and scope of the usability study.
- Review study objectives and research questions to confirm that tasks and questions directly support the intended outcomes.
- Continuously refer back to the study goals and scope during the material preparation process to maintain focus and relevance.
- Seek input from stakeholders or team members to validate the appropriateness of tasks, questions, and consent form language in addressing study objectives.
Set Up Environment
- Reserve a dedicated space for conducting both the trial test and the actual usability test sessions.
- Ensure that the testing environment is conducive to user comfort and focus, with adequate seating, lighting, and temperature control.
- Arrange furniture and equipment to facilitate smooth user interactions and minimise distractions during testing.
- Test and optimise lighting conditions throughout the testing period to ensure optimal visibility for both participants and observers.
- Set up all necessary electronics and devices required for the usability test, including laptops, cameras, microphones, and recording equipment.
- Configure and test each electronic device to ensure proper functionality and compatibility with the testing setup.
- Allocate specific roles and responsibilities for team members involved in the usability test, such as the interviewer, observer, and technical support staff.
- Designate one laptop for the interviewer to document user interactions, take notes, and record follow-up questions during the test sessions.
- Utilise a second laptop for the observer to stream the test session live, allowing them to observe participant interactions and take detailed notes.
- Deploy multiple cameras strategically to capture different perspectives of the testing environment and participant interactions:
1- Position one camera to record the overall room setup and interactions between the participant and facilitator.
2- Place another camera to capture close-up footage of the participant's mobile screen, allowing for detailed analysis of user interactions and gestures.
3- Use a third camera as a backup recording device or additional angle to ensure comprehensive coverage of the usability test.
- Conduct thorough testing and troubleshooting of all recording equipment to verify audiovisual quality, synchronisation, and reliability.
Conduct Trial Test
- Prior to conducting the actual usability tests, it was imperative to conduct trial tests to ensure all systems and procedures were in place and functioning smoothly.
- The trial test involved creating an initial and semi-final testing environment, with all necessary materials and equipment prepared and tested.
- To recruit participants for the trial tests, an email invitation was sent to non-technical colleagues within our Amsterdam office, outlining the purpose and expectations of the trial session.
- Three participants were successfully recruited for the trial tests, representing a diverse range of perspectives and experiences within the office.
- The trial test served as a critical step in the process, allowing us to validate the effectiveness of our testing setup, task scenarios, interview questions, and post-test follow-up procedures.
- One of our designer colleagues was invited to co-facilitate one of the trial tests, providing valuable insights and feedback on the usability testing process.
- The remaining trial tests were observed by our product owner and additional team members, enabling us to gather comprehensive feedback and identify any areas for improvement.
- Through the trial tests, we were able to refine our approach and ensure that our usability testing methodology was aligned with our study goals, ultimately setting the stage for a successful execution of the actual usability tests.
By conducting thorough trial tests and leveraging the expertise of our team members, we were able to mitigate potential risks and optimise our usability testing process for maximum effectiveness and efficiency.
Conduct The Test
The usability test was conducted over two days, spanning a working day and Saturday. Participants were asked to specify their preferred study date and provide multiple time slots in the sign-up form. Based on this information, we allocated three sessions on Friday and another three on Sunday.
Location and Preparation
The tests took place in our office in Amsterdam to provide a controlled environment for the study. Participants were instructed to arrive 15 minutes prior to the test to allow them to settle comfortably and have refreshments.
Session Duration
Each session had a duration of 45 to 60 minutes to ensure a comprehensive exploration of the tasks and ample time for interviews.
Session Structure
- Introduction and Consent: Each session began with a welcome and a brief explanation of the study's purpose and what to expect. Participants were asked to sign the consent form before proceeding.
- Pre-Test Interview: We conducted a pre-test interview, asking participants questions to gather initial insights into their shopping experiences and preferences.
Task Execution: Participants were then given specific tasks to perform on the VILA website. They were instructed to think aloud during the task to provide insight into their thought process and interactions.
Note-Taking and Interaction: Throughout the task, notes were taken on participant interactions, and any questions that arose were noted for clarification.
Post-Task Interview: Following the task, we conducted a post-task interview to gather feedback on their experiences with the site and ask follow-up questions based on their interactions and shopping behaviour.
Conclusion and Appreciation
At the end of each session, I consulted with observers outside the room for any additional questions. Participants were then thanked for their participation and provided with the promised voucher. Additionally, they were given a VILA goodie bag as a token of appreciation for their time and insights.
Data Analysis
Analysing the collected data from the usability testing sessions was a comprehensive and time-intensive process. We approached this phase with meticulous attention to detail, aiming to uncover valuable insights that would inform improvements to VILA's online shopping experience. The analysis involved several key steps
Data Collection and Organization
- The first step was to transcribe all test sessions and meticulously document notes, observations, and interview responses.
- To facilitate data organization and analysis, we created an Excel spreadsheet with multiple sheets dedicated to different aspects of the analysis process.
Comprehensive Review
- Each sheet served a specific purpose, such as documenting pre-test and post-test interview responses, recording behavioural observations, and capturing insights and opportunities for improvement.
- We conducted a comprehensive review of the written documentation and meticulously studied the recordings of the test sessions to ensure that all insights were captured and understood.
Insight Prioritisation
- Once all potential insights and issues were documented, we prioritised them based on their impact on user experience and alignment with business goals.
- This prioritisation process involved careful consideration of factors such as the frequency and severity of the issues identified, as well as their potential to drive meaningful improvements to the overall user experience.
Collaborative Action
- The prioritised insights were then allocated to designated teams for further investigation and action.
- Collaborations were initiated with individuals from various teams to address the identified issues and implement solutions as needed within their respective departments.
Meaningful Recommendations
By conducting a thorough analysis of the collected data and prioritising actionable insights, we were able to provide valuable recommendations for improving the user experience and achieving business objectives.
This approach allowed us to translate the findings from our usability testing sessions into actionable recommendations that would drive meaningful improvements to VILA's online shopping experience.
Report Results
Throughout the usability study process, we maintained open communication with both our internal team and the VILA brand, providing regular updates on our progress and the steps we were taking to analyse the results. We initiated a closer collaboration with VILA following the trial test, inviting them to participate as observers in all subsequent test sessions. This involvement allowed them to play a more significant role in the study and provided valuable insights into the user experience.
Brand Collaboration
As part of our collaborative approach, VILA was given the opportunity to contribute to the development of interview questions and review the study materials. However, they expressed satisfaction with the content we had prepared, indicating a high level of trust in our expertise and methodology.
Results Presentation
After completing the analysis of the collected data, we reconvened with the VILA brand to share our findings and discuss the implications for their online shopping experience. This meeting served as an opportunity to align on next steps and prioritise actionable recommendations based on the insights uncovered during the study.
Ongoing Work
While the final study report is still in the process of being created, we remain committed to delivering comprehensive findings and actionable recommendations to support VILA's goals for enhancing their digital platform. We are dedicated to ensuring that the insights generated from the usability study are leveraged effectively to drive meaningful improvements to the user experience.
