Google Analytics

Friday, February 09, 2007

Questions, Questions and More Questions

To answer the question at ASTD’s Learning Circuits Blog, I am going to try to stick to Tony’s initial thought of specific questions and dive very deep into the technique I use when consulting for a needs assessment or evaluation.

I use a technique I call the 360 Needs Assessment. This process involves asking questions at several levels within an organization. The idea is to separate the facts from the opinions. If I can triangulate information, I know that I have a fact, if I can’s substantiate it…well then more investigation is required. Or I know that the information is a minority opinion.

So, I first ask questions of the stakeholders:
  • Why do you think this is a problem?
  • Have you every tried to solve it before? How? What was the result?
  • Why do you think training is the answer?
  • If the problem is solved, describe the ideal state, the results, the outcome, your expectations.
  • If the problem isn’t solved, what are the consequences?
  • Did the employees ever know how to do this in the past?
  • Do you think internal or external forces are causing the change.

Then I move to the employees impacted by the problem/challenge/potential training, I want to know if the stakeholders (usually management) know what is going on in the trenches (hint, they rarely do).I use similar questions but not always phrased as below.
  • What do you think is causing the problem?
  • What do you think is the solution to the problem?
  • Are their incentives that drive you away from the desired goal?
  • Is the environment conducive to your obtaining the desired outcome?
  • Would training in XYZ solve this problem for you?
  • Do you feel that you know how to perform the tasks that are required?
  • What would you do to solve this problem?
  • Have you ever seen this problem before? What was the attempt to solve it?

Then I move to managers of the employees:
  • How long has this been a problem?
  • Is it as "bad" or "good" as indicated by the stakeholders?
  • Why aren’t employees performing (solving the problem?)
  • What is the result on the customers or downstream employees?
  • Is their another way (besides training) to solve this problem?
  • Do you have the right people in place?
  • What external or internal factors are contributing this is problem?

Next, I talk to downstream employees
  • What problems do you encounter as a result of the performance upstream?
  • What outputs would you like to see but are not getting from the upstream employees?
  • Why do you think you are not getting what you need?
  • How would you solve the problem?
  • How often do you meet with employees upstream? (usually never)
  • How do you have to compensate for the employees upstream?

Finally, I talk to the ultimate customers; usually they are outside of the organization. This is not always possible for many reasons but can be extremely insightful.
  • Please rate the product/service/attention provided by organizations XYZ?
  • What would you like to see improved?
  • What is fantastic and that you don’t want to see change?
  • How has performance changed over time?
  • Is the organization responsive to your needs?
  • What are your needs?
  • How does organization XYZ understand what your needs are?
These are some of the tactical questions I ask. I don’t always get access to all these people but I certainly try. I also like to get people who have been with the organization for a long time, short time and intermediate. The goal is to gather data from a variety of places. Sometimes when you learn something at one level, you need to go back to another level and ask more questions.

It does take a while with intense interviews, however, you can cram a number of interviews into a day or week and really get a feel for the answers after doing a half-a-dozen or so.

Recommended Games and Gadgets
Recommended Books
Content Guide


Dave Lee said...

Hi Karl:
What a great system of questions, thanks for sharing.

It's probably out of the spirit of The Big Question, but I just had a few questions about your implementation.

You seem to suggest a preference for the order in which these groups of questions are carried out (ie, stakeholders, then employees, then managers of the employees, the downstream employees, then customers). Do you see this sequence as vital or just preferred? In other words, if you were able to schedule interviews with say managers and downstream employees before you could schedule time with stakeholders, would that work?

My second question, is in an ideal situation, how many interviews in each of the five groups would you like to have?

Karl Kapp said...


Thanks for the question. I do like speak with the stakeholders first because I want to know what is on their mind and try to verify (or not) their perception of the current situation or problem. If they frame it a certain way then I can probe to see if they are in touch or not.

However, that is not always the order that the client allows (mostly do to trying to schedule time from stakeholders). Sometimes I have to mix in stakeholder interviews with other interviews, however, I don't like to wait until the end to interview stakeholders because then I can't bounce what they tell me off of the rank and file, if that is the case, I don't get a trianglation on the information.

Naturally, the order listed is preferred but I have done it in other orders, ultimate customers first, then employees, then stakeholders..etc. However, I think the order is helpful in getting to the heart of the matter.

Many times I start with trainers or employees and then work up and then back down.

In an ideal situation, in terms of number of interviews, I think between 6-8 on each level except stakeholders because usually their are only one or two major stakeholders...for them, you want to interivew all of them.

I find at about 6 interviews or so the information starts to be repeated.

Thanks for the question.