Accessible Search Form           Advanced Search

Session 12- Use Evaluation To Track Your Progress (Especially for Promotores)

Manual Contents

Session 12
Use Evaluation To Track Your Progress (Especially for Promotores)

Page Contents

Objectives

By the end of this session, promotores will learn how to:

  • Create a vision for a Your Heart, Your Life community project.
  • Learn the basics of evaluation.
  • Choose strategies that can be used to implement the Your Heart, Your Life project in the community.
  • Learn the role of a community health worker in the evaluation process.
  • Collect data to show the results of the project.
  • Participate as a team member in the project's evaluation process.
  • Create an evaluation workplan for the project's activities.

Back up to Page Contents

Materials and Supplies

To conduct this session, you will need:

Back up to Page Contents

Handouts

Give these handouts to each group member during the session:

Back up to Page Contents

Introducing the Session

  1. Welcome
  2. About This Session
  1. Welcome
    Welcome group members to the session.
  2. About the Session
    • Say:
      You have learned a lot of information during this training. You have gained new skills and shown great progress and motivation. Now, you are ready to put the Your Heart, Your Life training into action in your community. Congratulations!
    • Say:
      This session is especially for promotores. It will teach you how to participate as a team member to evaluate your project.

Back up to Page Contents

Conducting the Session

  1. Develop a Vision for the Your Heart, Your Life Community Project
  2. Basic Information on Evaluation
  3. Your Heart, Your Life: Three Strategies To Offer in Your Community
  4. Data Collection
  5. The Role of Promotores in the Evaluation Process
  6. How To Work as a Member of the Evaluation Team
  7. Group Activity: Develop an Evaluation Workplan for Your Project
  1. Develop a Vision for the Your Heart, Your Life Community Project
    • Say:
      During this session, you will develop a vision for a Your Heart, Your Life community project.
    • Ask:
      Can someone tell me what a vision is?
      Note: Allow 2 to 3 minutes for group members to respond. Write their answers on the blackboard or a large piece of paper taped to the wall.
    • Add the following answers if they are not mentioned:
      • A vision is your dream, an image, or a picture of what you want to do and what you imagine your community project will do for the people you serve.
      • It is the direction or goal of your project.
      • It is what inspires, motivates, and engages people to take action.
    • Say:
      A vision can be written as a statement or expressed as a drawing.
    • Some examples of vision statements are:
      • A heart healthy and stroke-free community
      • An active, healthy, and informed community
    • Say:
      Now it's time for you to create a vision for a community project. Think of this question when you create your vision: What would you like your community to be like after implementing your heart health project?
      Note: Divide the participants into small groups.
    • Say:
      Choose one person from your group to share your group's vision.
      Note: Give each small group a set of color markers and a piece of poster board. Allow 20 minutes for groups to come up with their visions.
    • Say:
      Now, a member of each group will present each vision to the rest of the group members.
      Note: Allow 2 to 3 minutes for each group to share its vision. Congratulate the promotores for their enthusiasm and a job well done.
  2. Basic Information on Evaluation
    • Say:
      When you offer the Your Heart, Your Life classes in your community, you will want to know if your project helped you to achieve your vision. Evaluation can help you do this. Now, let's talk about what evaluation is and some important steps you need to know when you evaluate a project.
    • Say:
      Evaluation is a well-thought-out process to assess the value of your project.
    • Ask:
      What are some benefits of evaluation?
      Note: Allow about 2 minutes for group members to answer. Write their answers on the blackboard or on a large piece of paper taped to the wall.
    • Add the following answers if they are not mentioned.
      Evaluation offers you a way to:
      • Find out if your project is working or is successful.
      • Learn if you are meeting the goals of the project.
      • Find out which project areas need to be changed or improved.
      • Gather information about your project that you can share with the community and those who fund your project.
      • Learn if your project activities are helping people make healthy choices.
    • Give each group member the “Examples of Project Evaluation” handout.
    • Say:
      Now we will look at how evaluation is used in a few projects.
    • Ask volunteers to read aloud each example and tip.
      Examples of Project Evaluation

      Example 1

      During the month of May, doctors from a community clinic referred 45 patients to the heart health sessions conducted by promotores. Thirty participants attended all of the heart health sessions. The other 15 participants attended only a few sessions.

      After the classes ended, the promotores conducted followup visits. These visits revealed that the 30 participants who attended all the classes were using the project's heart healthy recipes, participating in physical activities, and taking their medicines as the doctor told them. The other 15 participants who did not attend the classes were not using the recipes, most were not doing any physical activity, and several were taking their medicines only when they remembered.

      The promotores saw that the project had a greater positive impact on participants who took part in all of the heart healthy sessions.

      Tip: Plan to track participants at every stage of your project – (1) referrals, (2) class attendance, and (3) followup after the classes.

      Example 2

      A promotora is a member of the health promotion team working on a project to increase the physical activity of community members. Participants attended a series of heart health sessions. The promotora reviewed the results of the project and found that 15 out of 20 participants who went to at least 6 of the training sessions were now walking 30 minutes or more per day.

      The promotora learned that people who attended the heart healthy sessions increased their physical activity.

      Tip: Learn your project goals; review them throughout your project. Make sure that project activities make sense and are helping you to reach the goals of the project.

      Example 3

      A group of promotores posted flyers in the community about an upcoming cholesterol-screening event. They held the screening, but only a few people showed up. After the poor turnout, the promotores thought about different ways to get the word out. They met with community leaders, got their input, and developed a new strategy. The new strategy was to go door to door to talk about the importance of cholesterol screening and to ask people to share this information with friends and family. The promotores held a second screening and had a much better turnout.

      By finding out what didn't work and getting input from the community, the promotores were able to make changes in the way they recruited participants.

      Tip: Don't focus only on the positive results of the project. You can learn a great deal by looking at what went wrong and what did not work.

      Example 4

      A promotora conducted several heart healthy sessions for community members. One participant shared her high blood pressure story. She described how her doctor had told her she had high blood pressure and about all the healthy changes she made. After 3 months, she had lost 10 pounds and her blood pressure was under control.

      The promotora asked this participant to share her story at a community gathering. More community members are now interested in taking part in the heart healthy sessions.

      Tip: Be creative. Project evaluation is about more than just numbers. Participants' stories, pictures, and journals can be very powerful to describe how your project has affected them.

    • Say:
      You have learned examples of project evaluation. Let's go over the seven steps to create a plan for implementing and evaluating your project.
      1. Choose the activities. Work with your agency to choose the activities you want to do. For example, your project may offer to train promotores using the “Your Heart, Your Life” manual.
      2. Implement project activities. Perform the activities of your project as planned. For example, you can recruit 20 promotores from the community and conduct the Your Heart, Your Life training.
      3. Collect data. Collect data to show if your project is helping promotores. For example, you can use questionnaires to find out how promotores used the information, what they learned in the sessions, and if they made heart healthy changes in their lives.
      4. Enter data. Enter information from the completed questionnaires into a database. This task can be done by trained promotores or trained program staff.
      5. Analyze the data. An evaluator can analyze the data and summarize the findings. For example, an evaluator may find that promotores who walk for
        60 minutes daily have lower blood pressures and have lost weight.
      6. Write a report about the results. A report can show how the community has changed as a result of the project. For example, the evaluator may describe how promotores' eating habits and physical activity patterns have changed as a result of their participation in the training.
      7. Share the results. Promotores can share results with community members. Sharing results can increase community members' interest in the project and motivate them to take personal action to improve their health.
    • Say:
      You have seen how evaluation can help you. Now, let's take a look at two types of evaluation: process evaluation and outcome evaluation.
    • Give each group member the “Types of Evaluation” handout.
    • Ask volunteers to read aloud the types of evaluation and
      each example.

      Types of Evaluation

      Process Evaluation

      Process evaluation tells you about the content of project activities. You can learn if you are doing the activities as they were planned. It also tells you who is participating in your project activities. You can track the specifics of how you carry out your project, such as the time spent on activities and how many participants attended the activities. The results of process evaluation help you know which activities are more successful than others. It also gives you the feedback you need to improve your project.

      Example: You can collect information about the number of sessions you taught from the “Your Heart, Your Life” manual and how much time you spent on activities during the training sessions.

      Outcome evaluation

      Outcome evaluation describes the effect your project had on participants. You can learn how the participants changed or are changing after completing the course. You can track how participants' knowledge, feelings (attitudes), or actions (behaviors) have changed after taking part in the project. You can also track the changes in the clinical values. (For example, you can check to see if participants' blood pressure or weight has decreased.)

      Example: A questionnaire is given to participants before the first class. The same questionnaire is given after the last class. The results of the two questionnaires are compared. This will tell you how much participants learned.

      Other Evaluation Methods

      You can use other methods to evaluate your project. You can ask participants for their stories (testimonials) about how the course has affected them, and collect the stories as the project evolves. They can submit photographs and journals about the changes they have made during the project.

  3. Your Heart, Your Life: Three Strategies To Offer in Your Community
    • Note: Before the session, read the charts that serve as a guide. These charts list the three strategies in more detail. Each strategy includes: goals, description of activities, setting, and target audience.
    • Say:
      You can offer the Your Heart, Your Life training in your community using three strategies. These strategies are:
      1. Train the Trainer. This strategy consists of using the “Your Heart, Your Life” manual to train a group of promotores, who then go back to their communities and train other promotores.
      2. Community Education. In this strategy, trained promotores use the manual and picture cards to teach community members by using one of the following options:
        • Teaching all lessons from the manual to community members.
        • Teaching all lessons from the manual and screening community members. Screenings can include: the height, weight, and waist measures of participants. Blood pressure, blood cholesterol, and blood glucose screenings also can be conducted with the help of other agencies and health professionals. If needed, promotores can refer project participants with elevated values to a medical setting for followup.
          • Ask:
            When you do screenings as part of your community education, why do you think it is necessary to partner with other agencies such as a clinic?
            Note: Allow about 2 minutes for group members to respond.
            Note: Add the following answers if they are not said.
            • To secure trained health care staff to screen participants for high blood pressure, high blood cholesterol, or blood glucose. This person usually works in a clinic or a hospital.
            • To get an authorized agency to obtain permission forms from participants before they are screened for high blood pressure, high blood cholesterol, or blood glucose.
            • To refer participants with elevated numbers to a health care professional at the clinic to confirm that their levels are high and to get health information.
      3. Lifestyle and Clinical Management. In this strategy, promotores function as part of a health care team. Trained promotores: (1) teach sessions of the manual to patients, (2) monitor patients' clinical measures (blood pressure, blood cholesterol, body mass index (BMI), waist measures, and blood glucose levels) in collaboration with a health care provider, and (3) follow up with patients to offer support and encouragement in addition to making sure that patients are following their treatment.
        Ask:
        Does anyone have questions about the strategies?
        Note: Allow 2 minutes for group members to respond.
  4. Data Collection
    Note: Before the session, read the chart that serves as a guide. The outcome evaluation chart describes each strategy, the forms needed to collect outcome data for each strategy, when the forms should be used, and the type of information you can collect to evaluate your project.
    • Say:
      Once you have figured out what strategy or strategies you want to offer in your community, then you need to collect data for each strategy. For the “Your Heart, Your Life” manual, data collection forms for the three strategies are included at the end of this session.
      Note: Give group members copies of the data collection forms, and describe each form. The data collection forms are:
    • Say:
      Let's look at the data collection forms for the three strategies. We will start with the Train the Trainer strategy. This strategy uses the “Promotores Train the Trainer Pretest and Posttest” forms. The pretest form was used at the beginning of the training, and you will use the posttest form at the end of training.
      The changes in the responses from pretest to posttest will help you find out if you learned new information or skills. After the training, you will also complete the “Feedback Form – What Did You Think About the Training?” questionnaire. This form provides information on your likes and dislikes of the training, how confident you feel about training others, and suggestions for improving the training.
    • Say:
      For the second strategy – Community Education – you can use the “My Health Habits Pretest and Posttest” forms to find out if participants are changing their health habits. If you add screenings to your community education strategy, you can use the “Screening Form” to record the clinical values of project participants.
    • Say:
      Using the third strategy – Lifestyle and Clinical Management – you can use the “My Health Habits Pretest and Posttest” to find out how patients are changing their health habits. You can also use the “Clinical Measures and Followup Form” to record the patients' changes in clinical values.
      The “Promotores Activities Form” can be used to keep track of the followup activities that help patients stay on their treatment plan.
    • Say:
      The “Clinical Measures and Followup Form” is useful for community health workers who work in a clinical setting.
    • Ask:
      How many of you work in a clinic, hospital, or other medical setting?
      Note: Allow 2 minutes for group members to respond.
    • Ask each group member to look at the “Promotores Activities Form” handout again.
    • Say:
      When you work with patients, it is very important to keep track of the followup activities to help patients stay on the medicines prescribed by their doctors. You should also keep track of the followup activities you do to help patients make lifestyle changes.
      Note: Since you have already reviewed the “Promotores Activities Form” handout, ask volunteers if they have any questions or comments about the activities listed on the handout. Allow 2 minutes for group members to respond.
    • Say:
      Collecting information on followup activities can highlight the important role that community health workers have in helping patients stay on their treatment plans and control their risk factors.
    • Say:
      The last form that we will talk about is the “Recording Log.” This log tells you which form to use for each strategy and tells you what type of information to gather for each strategy. The recording log focuses on process evaluation activities.
    • Say:
      Let's review the “Recording Log” handout. The information you will gather for the Train the Trainer strategy is:
      • Number of participants trained
      • Number of sessions taught
      • Number and percentage of participants who rate the training “good” or higher
      • Number and percentage of participants who report that they “will” or “most likely will” change their health habits
      • Number of participants using the “Your Heart, Your Life” manual 30 days after the training
      • Types of activities that participants are completing 30 days after the training
    • Ask a volunteer to read aloud the information for the community education strategy.
    • Ask a volunteer to read aloud the information for the lifestyle and clinical management strategy.
    • Say:
      Does anyone have questions?
    • Say:
      The “Recording Log” form is important, because the results can help you know which activities are being implemented. It also gives you the feedback you need to improve your project.
    • Say:
      Before using these data collection forms, we recommend that you review the “More Information” box below. This box contains tips for gathering information for your community project.

    More Information

    Tips for gathering information for your community project:

    Before you offer a questionnaire to others:

    • Check with your agency. Your agency may need to approve the questionnaire before you use it. Privacy and consent of project participants are important when doing evaluation. You will want to follow the policies of your organization and your funding agency.
    • Make sure you understand how to fill out the questionnaire before you hand it out. Know the form well, so you can answer any questions participants may have.

    The day of filling out the survey:

    • Bring plenty of forms and pens with you.
    • Ask participants to answer each question completely. Missing information will make it hard for you to analyze the data.
    • If the budget allows, give a small prize to people who sign up for the course or fill out the form.
    • Always remember to thank participants.

    Note: During the pretest and posttest, the promotores or volunteers may ask the questions aloud if people need help with their forms. Questions may be read aloud but the answers to the questions should not be given.

  5. The Role of Promotores in the Evaluation Process
    • Ask:
      Has anyone already worked on the evaluation of a project? Can you share what you did?
      Note: Allow 3 minutes for group members to respond.
    • Say:
      Some of you may be thinking that evaluation is too complicated and that there is no role in it for you. But that is not true. Here are some examples of the roles that community health workers can have in the evaluation process.
    • Promotores can:
      • Help to decide which activities should be conducted.
      • Hand out questionnaires and other instruments to participants.
      • Collect the forms.
      • Enter data into a computer.
      • Provide feedback on what worked and what didn't work and how the activities could be improved.
      • Share results of the evaluation with the community.
      • Attend training meetings about project evaluation.
    • Say:
      Remember that your involvement in the evaluation process can help show the value of having community health workers on the project team. The actions of community health workers are key in tracking project activities in the community.
    • Ask:
      If you have not been part of an evaluation team, would you be interested in having a role in the evaluation process of a Your Heart, Your Life project?
      Note: Allow 5 minutes for group members to respond.
    • Say:
      Thank you for sharing your experiences about being part of an evaluation team. With time and practice, your confidence level will increase. Just as you have been trained in how to conduct the sessions of the “Your Heart, Your Life” manual, you will be trained in how to conduct evaluation activities for your project. In less time than you think, you will be making great contributions to your evaluation team.
  6. How To Work as a Member of the Evaluation Team
    • Say:
      People who evaluate projects need to work as a team. Promotores can be an important part of this team.
    • Say:
    • A community project team may include:
      • A project manager
      • An evaluator
      • Promotores
      • Health professionals
    • Ask:
      Do you have an evaluator on your project team or in your agency?
      Note: Allow 2 minutes for group members to respond.
    • Ask:
      What are some things you can do to get involved with project evaluation when you return to your agency?
      Note: Allow 2 to 3 minutes for group members to respond. Write their responses on the blackboard or a large piece of paper taped to the wall.
    • Add any of these answers if they are not said.
    • Promotores can:
      • Find out who does evaluation in their agencies.
      • Share this session on evaluation with supervisors.
      • Ask their supervisors how they can be involved in evaluating the project.
      • Ask their supervisors if they can attend trainings on project evaluation.
    • Say:
      If your agency does not have an evaluator, you can try to link with an outside evaluator who supports community health worker projects.
    • Say:
    • An evaluator can help to:
      • Develop an evaluation plan.
      • Make sure that the project is carried out as intended.
      • Conduct an analysis.
      • Write a report on the data collected.

      More Information

      Your agency can get the services of an outside evaluator by contacting:

      • Other promotores programs
      • The National Promotores Network
      • A university that has a school of public health or public health project
      • A foundation that has community health projects and can refer you to an evaluation consultant
      • Your State health department
  7. Group Activity: Develop an Evaluation Workplan for Your Project
    • Say:
      The purpose of this activity is to develop an evaluation workplan for a community project.
    • Divide group members into the same three groups that they were in when they developed the vision for their community projects. Give each group a different project from the “Develop an Evaluation Workplan for Your Project” handout.
      Note: Group 1 will work on project 1, group 2 will work on project 2, and group 3 will work on project 3.
    • Say:
      This handout on evaluation includes the project descriptions and a blank chart with questions to help you develop a plan on how to evaluate your activities. The questions are:
      1. Who will be your target audience?
      2. What strategy will you use?
      3. Which forms will you use to collect data for your project?
      4. Describe the activities to carry out the strategy:
        • How will you recruit participants?
        • When will you schedule the classes?
        • Who will teach the classes?
      5. Decide who will manage the data:
        • Who will collect the data?
        • Who will enter the data?
        • Who will analyze the data?
      6. Who will write the evaluation report for the project?
        Note: Ask one person from each group to present the group's evaluation plan to the entire group. Allow about 5 minutes for each presentation.
        Say:
        Thank you for doing such a wonderful job! Note: Ask one person from each group to present the group's evaluation plan to the entire group. Allow about 5 minutes for each presentation.
    • Say:
      Thank you for doing such a wonderful job!

    Community Projects

    Note: Discuss your project and the evaluation workplan using “Develop an Evaluation Workplan for Your Project” as a guide.

    Project 1: Training Promotores To Teach the Your Heart, Your Life Manual in the Community

    • Your agency has agreed to conduct the Your Heart, Your Life Train the Trainer workshop for several agencies in the community. Your agency will train 25 promotores who have asked to participate in the workshop.
    • Your supervisor asks you and two other promotores to plan the Train the Trainer workshop and to develop the evaluation plan.

    Project 2: Helping Community Members Make Heart Healthy Lifestyle Changes

    • Your community-based organization is in Ajo, NM.
    • Your supervisor asks you to help develop the evaluation plan for the Your Heart, Your Life community education project.
    • The course is for adults aged 30 to 60.
    • The goal is to recruit, teach, and increase the knowledge, positive attitudes, and skills of the community members attending your classes.

    Project 3: Helping Patients Manage Their Risk Factors for Heart Disease and Make Lifestyle Changes

    • Your clinic is on the border between the United States and Mexico.
    • Your clinic has decided to get involved in a project on heart health to help patients change their behaviors and lower their clinical measures.
    • You and two other promotores attended a promotores conference to receive the Your Heart, Your Life training.
    • Now, your supervisor wants your group to develop activities and an evaluation plan for the project.

    Note: Allow about 30 minutes for each group to come up with their evaluation plans.

Back up to Page Contents

Review of Today's Key Points

Say: Let's review what we learned today.

  • What is evaluation?
  • Evaluation is a well-thought-out process for assessing the value of your project. It helps you check to see if you are meeting the goals of the project.
  • What are the benefits of evaluation?
    • Finding out if your project is working as planned, or is successful.
    • Learning if you are meeting the goals of the project.
    • Finding out which project areas need to be changed or improved.
    • Gathering information about your project that you can share with the community and those who fund your project.
    • Learning if your project activities are helping people make healthy choices.
  • What is the role of promotores in the evaluation process?
    • Helping to decide which activities should be carried out.
    • Administering questionnaires and other instruments.
    • Collecting the instruments.
    • Entering data into computers.
    • Providing feedback on the ways the activities are done and how they can be improved.
    • Sharing results of the evaluation with the community.
    • Attending training meetings about project evaluation.
  • What strategies can you evaluate using the “Your Heart, Your Life” manual?
    • You can evaluate the training of promotores to teach heart health in the community.
    • You can evaluate what community members have learned and which changes they have made to have heart healthy lifestyles.
    • You can evaluate how patients have been helped in managing their risk factors for heart disease and making changes in their lifestyles.
  • Who forms the community project team for evaluation?
    • A project manager
    • An evaluator
    • Promotores
    • Health professionals

Back up to Page Contents

Closing

  • Say:
    Thank you for coming today. What did you think of today's session?
    Note: Allow a moment for group members to respond.
  • Say:
    Promotores play an important role in the evaluation of a community project. We hope this session helps you achieve positive results in your work with the community. I wish you success in implementing the Your Heart, Your Life project in your community.

Go to Session 11

Go to the Appendix


Information on this page is taken from the English print version of “Your Heart, Your Life, A Community Health Worker's Manual.” U.S. Department of Health and Human Services, National Institutes of Health, National Heart Lung and Blood Institute, NIH Publication No. 08-3674, Originally Printed 1999, Revised May 2008.

Last Updated March 2012

Twitter iconTwitterimage of external icon Facebook iconFacebookimage of external icon YouTube iconYouTubeimage of external icon Google+ iconGoogle+image of external icon