Hybrid Concept Usability Tests
+ Moderated Interviews

IBM SCIS - SHIPMENT MILESTONE FEATURE ENHANCEMENT

The below study represents my own POV, and does not represent or indicate IBM's current or future product and business direction.

Participant details, key takeaways, revenue/metrics, and other confidential information have been redacted below for public preview.

Top-level feature research findings were provided at the top, with quantitative insights placed with participant quotes for qualitative context.
Expert SMEs from relevant industry domains were recruited internationally, with select participants highlighted to demonstrate sample breadth from operational to strategic roles.
Concept designs were presented alongside participant feedback, highlighting user value and comparative value to existing industry solutions.
Nuanced details around customs declarations, missing documentation, variations between scheduled & departure times all contributed to painpoints addressed by subsequent feature developments.
Research insights were used for both immediate product enhancements needs, as well as contributing to features being considered as part of a key new product development workstream.
"Following the study, you can tell the core team, especially PM, feels more confident making decisions for the conceptual direction of the work...
preventing hard pivots THAT CAN RENDER DESIGN MOCKUPS IRRELEVANT, WHICH WE'VE SEEN HAPPEN BEFORE."
SCIS DESIGN LEAD
"That was my very first user interview, that was cool...the feedback on the prototype was good."
SCIS PRODUCT MANAGER
"...allowed for clear & actionable insights to be drawn from the research."
SCIS jr. designer

[Role]

RESEARCHER // DESIGNER

[research team]

scis design lead

[proj. time]

2~ weeks

[PRODUCT]

ibm supply chain intelligence suite

[PUBLISHED]

2022

[WORK REQ. BY]

scis product management

[audience]

scis global pm team
scis design team

[METHODS]

EXTERNAL SME RECRUITMENT
MODERATED INTERVIEWS
concept testing
structured usability test
‍

[DELIVERABLES]

keynote & playback
as-is & to-be user experience journeys
concept validation
design + product enhancements
SCIS PERSONAS 2023

GOT QUESTIONS? LET'S CHAT!

STUDY overview:

The Shipment Details/Milestones page is part of SCIS' solution strategy to support supply chain managers in intelligently managing inbound & outbound shipments, on time, across thousands of suppliers & locations.

SCIS product management had identified challenges regarding feature adoption with our shipment milestone capabilities; and had proposed a number of feature opportunities for design consideration. These were based on internal assumptions & hypotheses about what value proposition each feature may actually offer business users.

With a short window for validation, I partnered with SCIS Design Lead to conduct a hybrid concept usability tests & moderated interviews with externally recruited SMEs, to ensure insight depth & nuance. Insights were used to resolve both short-term needs as well as contributing to active new product visioning efforts in-parallel.

Post-research delivery & design enhancements, the SCIS Shipment Milestones feature saw adoption increase by 122% (NPS), with average page time-on-task reduced to <10 seconds from previous highs.

Image from Linda Shu, SCIS Design Lead - Shipment Milestones Case Study

approach & IMPACT:

  • Crafted target screener to recruit expert domain SMEs, with final selection representing relevant roles, regions & industries for target design persona
  • Initiated & created research plan to allocate multiple researchers to effort; enabling near 24/7 research coverage across stakeholder timezones
  • Moderated interviews conducted via UserZoom one-way mirror; dynamic discussion guides & live (hidden) PM questions provoked unique field insights
  • PM education & engagement on UXR + design process; got to see firsthand interview process, observing as PM questions get translated live into dynamic conversations
  • Usability & moderated interview raw data compiled & synthesized on collaborative Mural for rapid knowledge-sharing among team
  • Role & sub-person specific insights compiled as part of SCIS Personas 2023 launch
  • Feature adoption (NPS): + 122%
  • Avg. time on tasks: <10 seconds
Research insights led to shipped product enhancements for SCIS, and saw the subsequent increase in shipment milestone feature adoption by 122% (NPS).
Research insights led to shipped product enhancements for SCIS, and saw the subsequent increase in shipment milestone feature adoption by 122% (NPS).