---

Methods for Analyzing Users' Needs and Behavior

Defining User Needs: Tasks, Tools, Situations

You will find many opportunities to use these methods in your professional life. Brainstorming is a first step in many organizations which need to come up with a lot of ideas quickly when offered a new opportunity. For example, a public library may hear about a grant offering and they could brainstorm about a new program they want to offer that they could propose to win the grant. Many organizations invite their customers to join focus groups to talk about services or products. The political parties even had focus groups to talk about candidates and platforms of both parties. Companies are setting up strategic intelligence departments to scan many sources of information for news that might affect profits. Software designers spend time away from their own offices, studying how real users work in their homes or offices (ethnographic studies); the goal is to improve the software by better understanding the work environment and doing a more complete task analysis. The more you practice using these methods in classes, where you can get feedback on your technique and results, the more at ease you will be when you need to use them in a more stressful work setting.
 

Types of Situations in Which User Needs Analysis Should be Done

• Designing a new system -- Here, the focus is on the purpose of the new system, the people and needs it is to meet, and the context in which it will be used. In today's rapidly changing markets, organizations are constantly rethinking their information systems to keep up with the changes in products and services. New services and products are important for survival in today's economy. As an information professional at a national brokerage firm which suddenly decides it must begin making home loans, for example, you may be called upon to help design a system which can help the firm enter a new field real estate. A public or academic library might decide to begin a fee-based consulting service for businesses. SLIS's introduction of the MIS degree is another example.

• Customizing an existing system for different groups in the organization --Here, your focus will be on users' roles to see what aspect of the system they could best make use of.  Organizations which wish to survive are always looking for ways to improve their service to customers. Yahoo created Yahooligans (their "kiddy" section) to serve younger customers. The LETRS section of the IU Main library was developed to cater to the special needs of humanities scholars. A brokerage firm which had many customers with personal computers might look into to tailoring their quote system so that those users could dial in and look at stock prices during the day, from their homes or offices. The Monroe County Public Library created a special homework desk to better support the work of local teachers and their pupils. SLIS introduced distance courses for those students who have difficulty in commuting to classes on the Bloomington campus.

• Fine tuning an existing system -- Here the focus is on differences between individuals.  Information systems almost always need continuous fine tuning because, even if clients' needs do not change, there is almost always room for improving the way information is delivered. For example, a brokerage firm discovered over time that families often had several accounts (parents, retirement, investments for children). Each account received its own monthly statement. This meant multiple envelopes, multiple inserts, wasted time and resources. So they fine tuned the system to "bunch" the statements all going to one address into one envelope. A public library might set up a phone number or web site through which patrons could call to renew books or videos, thus saving them the time and effort of traveling to the library, only to find that the materials they seek is not available. SLIS augmented its web site to better suit MIS students.

Top-down  vs. Bottom-up Approach to Needs Analysis

Analysis of user needs can be done top-down or bottom-up.  In the top-down approach, you start with the users' decision-making and work down to information required for problem-solving.  For example, to determine students' information needs, first identify their goals (e.g., registering for classes or planning their career), then identify the specific tasks, settings, and tools needed to reach these goals.

The top-down approach is particularly useful in those situations where you are designing a new system or customizing an existing system, but can also be used when customizing or fine-tuning a system.  This is the approach described in the class devoted to the participatory design process.

In the bottom-up approach, you start with the information used, asking users how useful it is for the decisions they have to make.  For example, SLIS administrators could study how students interact with different kinds of information (e.g., the SLIS Web site, printed materials, information from alumni or current students) and build up a picture or model of the goals students are pursuing with these materials (e.g., deciding to enroll at SLIS or choosing which courses to take).

The bottom-up approach is particularly useful in those situations where you are fine-tuning a new system, but can also be used when customizing or fine-tuning a system.

Methods of User Needs Analysis

• an increase in the generalizability of your results from the people you study to the groups they represent
• an increase in the degree to which your results can be independently replicated by others
• an increase in the degree to which your results are independent of the individual bias of the person doing the study
• an increase in the control over extraneous factors

• a loss of information about the context of behavior and thus the ability to interpret answers people give or the behaviors they display. For example, in a survey you may know how many people gave answer x to question y, but you do not know how respondents interpreted the question (were they all really answering the "same" question?) or why they answered the way they did.
• a loss of flexibility in where and when the study is done
• an increase in the degree to which normal work is disturbed by the study;
• a decrease in the ability to draw practical conclusions from the results due to the narrow focus necessary to control outside factors
• an increase in the cost and time necessary to do the study
• a narrowing of the pool of people who can plan and carry out the study (possibly down to only professional researchers) and thus distancing of professional service providers from the process (they only get the information as filtered through "experts" who may not fully understand the work environment in question).

• based on researchers common-sense understanding of the situation
• no rigorous testing, no control groups
• based on researcher's preconceived ideas and biases which can influence results
• produce results that are NOT generalizable to other groups even in same population
• produce results that can NOT be used to predict future actions
• produce results that are NOT replicable by other researchers
• can NOT be taught to others since they are "gut" reactions of the researcher
• can be used at any time without prior preparation
• cost little in money, time, or resources
• are qualitative

Examples:  use of researcher's intuitions about situations s/he observes

involve more rigorous testing than intuitive methods, but are still not theory-based like scientific methods

the bias of the analyst can influence results

data collection is done more systematically

some thought is given to purpose of the study when collecting information

are qualitative

often collect information which does not directly apply to the current problem

results must be explained and summarized

 Examples: brainstorming, focus groups (= structured brainstorming), scanning

• employ more controls than impressionistic methods
• results are more replicable and more generalizable than impressionistic methods
• methods can be taught to other researchers
• planning before data collection is focused on the purpose of the study
• results must be interpreted and summarized
• involve qualitative and quantitative methods


Examples: observing/ethnographic studies/shadowing, asking/structured interviews, less rigorous surveys, comparative case studies/focus groups which employ some structured questioning, sorting, and other less impressionistic techniques

• produce results which are replicable
• produce results which are generalizable to broader populations
• produce results that can be used to predict future actions
• test theory-based hypotheses
• control for outside factors which might influence the results
• can be taught to others so they can reproduce the process
• include qualitative and quantitative methods
• people may lie or misunderstand the question and give incorrect information
• people may want to please the interviewer and give incorrect information


Examples: surveys, questionnaires, content analysis, rating scales, personality tests, scales for accessing quality (of a product, of communications) or satisfaction

Who Needs to Be Studied

Stakeholders may include people at all levels of the organization. For example, a public library's information system should serve the needs of patrons, staff, members of the Board of Directors, donors, volunteers, etc.  Because of time limitations, you will not have time to study ALL stakeholders relevant to your client's tasks. It will probably not be possible for you to study the needs of those outside the organization, for example. However, you should aim at studying as many of the members of the group as possible. For example, a site being developed for a social service organization should involve staff at all levels of the organization. Do not rely on the opinions of only the managers, for example, or only of staff. Involve a cross-section of the user group in your study.

Needs analysis should involve real users, not people who just happen to be handy. Only real users can give you the understanding of their needs that you require for system development.  Remember that you are only going to be able to talk with a relatively small number of people. It is therefore important that you choose people who will give you results that you can generalize to the whole group (McGrath, p. 155). This means that you don't want to pick subjects who are idiosyncratic in many ways from the user group. You will also want to be as systematic as possible in your use of methods.

Triangulation

Choosing Your Methods

You will need to make decisions about when to use which methods, and for what purposes. Here is some information that will help you make these decisions:

Scanning

reading through materials about your client, produced for the client, produced by the client -- email, websites, annual reports, press clippings, government studies, statements by competition

scan for impressions -- what is the general nature of the group, of their needs?

tour facilities used by the group

emphasis is not on taking notes but on gathering impressions

information gathered by scanning can be used as the basis for questions in interviews or focus groups, for items in surveys

inexpensive to do in many situations

not replicable or generalizable

can be done over time

can be used by corporate intelligence units to keep track of customers wants,  competitor activities,  government regulation, and potential products

• can involve observations (e.g., shadowing) plus informal questioning in the user's natural habitat
• help you understand the nature of the user's work -- the tasks they do, the environment in which they live and work, the skills they possess, the tools they use, the information they make use of, the information they create, the people who are involved and how they interact with one another
• occur in the user's natural setting -- their workplace, home, school -- instead of an artificial environment such a laboratory
• take a long time to do thoroughly with many users especially if their environment is complex
• can be done "quick and dirty" if you realize that you won't find out everything but only want relevant information to make you aware of potential problems in the system being developed
• does not rely on people's memory or ability to verbalize often complex thoughts or behavioral processes
• can provide a "reality check" on what people say in interviews or focus groups

• range from simple brainstorming (except the moderator should have some preplanned main and probing questions) to group discussions combined with sorting, surveying and other tasks to elicit individual opinions in a more controlled manner
• produce a variety of ideas especially if participants interact and develop each other's ideas
• more diverse groups may produce more and better ideas, although some varieties of diversity (such as when workers and managers are combined) may inhibit some participants
• the researcher has to be able to synthesize participants' ideas into system suggestions
• results not generalizable, not replicable (unless more systematic methods are included)
• give you a chance to hear the words and phrases used by members of your user group. Use of this language when designing the system and when introducing it to users will improve usability.
• the moderator has to be aware of group dynamics (keep everyone involved; don't let one or two people monopolize conversation; watch for people just agreeing with previous statements, facilitate the expression of opposing views, ask for details)

Structured Interviews

• can be used to corroborate more impressionistic methods
• enable you to go into depth about a particular subject (for example, you may have a user describe the most important goals and problems they face, then, for each, probe about how they go about achieving their goals and overcoming their problems, gradually getting "down" to the specific level of particular types of information needed)
• may suffer from the difficulty people have in verbalizing what they do, and why
• rely on people's memory, which is often unreliable
• require some preliminary knowledge about the user's environment and tasks so that questions can be designed which will tell you something new about their information needs
• give you a chance to hear the stories and metaphors that the subjects use as they describe their tasks and environments. You can use these metaphors in your system design to increase system acceptance and understandability
• avoid the problem (in focus groups) of having people intimidated by the presence of other users

Timeline for Methods

Bibliography

Hale, Martha L. (1986) Administrators and information: A review of methodologies used for diagnosing information use. Advances in Librarianship 14: 75-99. (New York: Academic Press.)

Hughes, John et al. (1995) The role of ethnography in interactive systems design. Interactions April: 57-65.

Kuhlthau, Carol C. (1993) Methodology for studying the constructive process of information seeking, ch. 6 in Seeking Meaning. A Process Approach to Library and Information Services, pp. 79-107. Norwood, NJ: Ablex.

McGrath, Joseph E. (1994) "Methodology Matters: Doing Research in the Behavioral and Social Sciences" in Baecker, Ronald et. al. (1995) Human Computer Interaction: Toward the Year 2000. San Francisco: Stanley Kaufmann Publishers, Inc., p. 152-169.

Van House, Nancy A. (1995) User needs assessment and evaluation for the UC Berkeley Electronic  Environmental Library Project: A preliminary report. Digital Libraries '95: 2nd Annual  Conference on the Theory and Practice of Digital Libraries.   http://csdl.tamu.edu/DL95/papers/vanhouse/vanhouse.html