e-GEM Phase 1 – Usability Engineering

Summarized State of the Art

Ideas are intellectual assets in the world of business, for example, the design of new products, features, processes or brand names can be protected as patents, copyrights and trademarks. But how are creative ideas generated, converted into practical knowledge, and managed to derive customer value and business intelligence?

Traditionally, focus groups and brainstorming sessions are used to elicit ideas. The main challenges of these approaches and their variants are predicated on the complex social dynamics that are inherent to group decision making. Both focus groups and brainstorms are limited by the number of participants that can contribute within a bounded timeframe and physical space. Both are synchronous, time-consuming methods that do not scale and are relatively difficult to document.

The brainwriting technique devised by Bernd Rohrbach is a more manageable and inclusive method to facilitate the generation of ideas from a group of people. Brainwriting can be used to stimulate a group of experts with the goal of silently expressing their expertise on a precise issue (Boy 1991). These subtle improvements of the brainstorming method, even though paper-based, mitigate the negative social impact while capturing the ideas in a written format.

There is conclusive evidence that knowledge elicitation as a design process can enhanced by the interactive and component-based software technology (Boy 1997). Studies comparing group performance in physical and virtual brainstorming sessions indicate that virtual sessions generate more high quality ideas and have a higher average of creative ideas per person, as well as resulting in higher levels of satisfaction with the ideas (Chamorro-Premuzic 2015b).

However, the technology-rich approach should not overshadow the awareness of the representational model that is used in the knowledge design process (Boy 1997). Human factors principles will be adopted to ensure that the participants are capable of accomplishing the goals of the ideation process. To this end, the interactions and interfaces of e-GEM will be iteratively designed and evaluated using a set of usability inspection methods.

Problem Statement

Design teams are primarily responsible for the ideation process in research and development projects whether in corporations, governments or less formal organizations. In the competitive landscape, agile methods of knowledge elicitation that satisfy the needs of end users and stakeholders are imperative.

Historically, design teams rely on feedback from disparate user groups using research methods to elicit ideas and draw conclusions for design. However, some design ideas plagued by outdated processes and infected by unavoidable politics. Consequently, these phenomena have led to inefficient design sessions as well as products that amplify the biases of the more dominant and extroverted participants from design sessions. As a result, what was supposed to be the collaborative source of cutting-edge product development insights now becomes a string of nightmares for the design team.

The set of methods outlined in Section 2.5 will be involve the target users identified in Section 2.3 to help solve these problems by allowing them to participate at various points throughout the product life cycle. This inclusive, human-centered approach will reduce the risk of creating a misaligned prototype and optimize the potential value delivered to the target users.

Identified Scenarios, Target Users, Processes and Tasks

Two realistic scenarios for the e-GEM are proposed below. The target users are members of the design team, namely the facilitator and participants. Generally, the target users are familiar with web applications, email, computers and mobile devices.

 

 

 

 

The first scenario involved a NASA design team that was seeking to answer the design question around a solar-driven drilling tool for extracting core samples and providing geological analyses that could be remotely communicated to earth. The NASA design team comprised a mixed group of geologists, astronauts and engineers that were led by a facilitator.

The second scenario involved a more “down-to-earth” research team of graduate students who sought to redesign an optimized organizational structure for delivering university services. The scenarios were not exhaustive but covered a range of e-GEM use cases involving different levels of sophistication and different group compositions (homogeneous vs. heterogeneous).

The tasks expected in the e-GEM web application were modeled from the traditional GEM, and are outlined in the following use-case diagram, which created the basis for the storyboard and wireframes explained in Section 2.5.

 

 

Proposed System

The long-term goal of this research project is to lay a foundation for an operational “Usability and UX methods bank” that can be commercialized for industrial applications.

 

 

In the more immediate case, the problem being solved is the provision of a consolidated high-tech platform for eliciting ideas from groups and conducting participatory design more cost-effectively. The solution will be a responsive web application delivers a seamless and scalable online user experience to the end users across as many concurrent devices as possible. The solution will feature an easy-to-use PHP-based MVC architecture, lightweight libraries and robust object-oriented databases. A multimedia integration will be used to provide secure, cross-platform desktop sharing with audio and video web conferencing that is scalable in the cloud through a one-time app download.

The proposed solution will help design teams to overcome the egos of participants (and their own) by leveraging anonymity, circumvent objections from stakeholders by providing a transparent and evidence-based approach to the subjective design process, and ultimately enhance the efficacy of participatory design.

Set of Methods

The following Gantt Chart contains a timeline of the major tasks associated with the analysis, design, development, evaluation and release of the e-GEM web application.

 

 

Questions, Options, Criteria (QOC) was used to make decisions on the following design questions after careful consideration of the practical options using a set of relevant criteria:

• Which software delivery platform should be used?
• Which web framework should be used?
• How to integrate virtual collaboration using multimedia?

A Pilot Study was used to simulate and test the traditional group elicitation method (GEM) and establish a baseline. The test provided hands-on experience upon which an appropriate context could be founded for developing the enhanced GEM. The pilot study was unobtrusive and allowed for discovery of realistic limitations imposed by the traditional GEM.

User Personas were modeled for the key end users involved in the GEM i.e. the facilitator and the participant. Analysis of the personas yields a better understanding of the capabilities and limitations of the end users and help to contextualize their goals and likely expectations of the proposed solution. The user personas were based on generalizations and may not account for the characteristics of outliers from the norm.

The Scenario-based Design approach was used to test the suitability of the user personas and to construct two practical scenarios that would inform the development of the storyboard and wireframes. The two scenarios were summarized in Section 2.3.

A Storyboard of the solution and Wireframes of the user interfaces were used to mockup the system design in order to facilitate rapid prototyping and generate a visual representation of the proposed solution for a heuristic evaluation. The rationale for adopting the visual design approach is therefore obvious but required an in-depth analysis of the user tasks and prospective interactions with the proposed system. However, the design of wireframes required a high level of experience in web application design.

A Heuristic Evaluation of the prototype will be conducted with two to three expert GEM users in order to determine the compliance of the user interfaces with recognized usability principles, and discover usability problems so that they can be amended early in the design process. The heuristic evaluation should last two to three hours and will not be intrusive since the evaluators will complete the inspection independently then have their findings aggregated.

Finally, a Usability Test may be conducted at the end in order to provide an initial basis for comparing the enhanced GEM against the traditional GEM using objective metrics such as execution time and the number of ideas generated along with subjective measures of usability and satisfaction. A two-by-two factorial design multivariate analysis of variance (2×2 MANOVA) statistical method will be used to determine if there is significant difference between the traditional and enhanced methods. It is anticipated that the summative usability test may be time-consuming and resource intensive. Power analysis for a MANOVA with two levels and two dependent variables using an alpha of 0.05, a power of 0.80, and a large effect size (f = 0.40), requires a minimum sample size of 28 subjects, which may present a challenge. Additionally, a reliable subjective measures such as the System Usability Scale (SUS) or Questionnaire for User Interaction Satisfaction (QUIS) may be used to gain insights of usability but may produce results that are complex to interpret.

Future Work

• Build traction for the free use of e-GEM targeting the following scenarios:
  • Student research projects
  • University administrative decisions
  • University-affiliated industry partner projects
• Use feedback to rapidly iterate the prototype
• Widen the scope of the e-GEM to offer a more comprehensive bank of methods
• Network within the domain to build a community of Usability and UX practitioners
• Research more relevant and reliable Usability Engineering and UX Design methods
• Research licensing of the intellectual property in protected methods
• Implement selected methods through the ubiquitous and digitally-enhanced solution
• Copyright the code authored for the web platform
• Commercialize the solution through marketing