With Every Heartbeat Is Life:  A Community Health Worker's Manual for African Americans

Session 12
Use Evaluation To Track Your Progress (Especially for Community Health Workers)

Contents

• Introduction
• Session 1 — Knowledge Is Power: Know Your Risk for Heart Disease
• Session 2 — Act in Time to Heart Attack Signs
• Session 3 — Get Energized! Say YES to Physical Activity
• Session 4 — Help Your Heart: Control Your High Blood Pressure
• Session 5 — Be Heart Smart: Keep Your Cholesterol in Check
• Session 6 — Embrace Your Health! Aim for a Healthy Weight
• Session 7 — Protect Your Heart: Take Good Care of Your Diabetes for Life
• Session 8 — Make Heart Healthy Eating an Everyday Family Reunion
• Session 9 — Eat in a Heart Healthy Way—Even When Time or Money Is Tight
• Session 10 — Take Control of Your Health: Enjoy Living Smoke Free
• Session 11 — Review and Graduation
• Session 12 — Use Evaluation To Track Your Progress (Especially for Community Health Workers)
• Appendix — Activities for Training Community Health Workers

Objectives

By the end of this session, community health workers will learn how to:

• Create a vision for a With Every Heartbeat Is Life project for your community
• Learn the basics of evaluation.
• Choose strategies that can be used to implement the With Every Heartbeat Is Life project in your community
• Learn the role of a community health worker in the evaluation process.
• Collect data to show the results of the project.
• Participate as a team member in the project's evaluation process.
• Create an evaluation workplan for the project's activities.

Materials and Supplies

To conduct this session, you will need:

• "With Every Heartbeat Is Life" manual
• Blackboard and chalk or large pieces of paper, markers, and tape
• Color markers and sheets of poster board

Handouts

Give these handouts to each group member during the session:

• "Examples of Project Evaluation"
• "Types of Evaluation"
• "With Every Heartbeat Is Life: Three Strategies To Offer in Your Community"
• "Outcome Evaluation Form"
• "Promotores Train the Trainer Pretest and Posttest"
• "Feedback Form—What Did You Think About the Training? (for Strategy 1)"
• "My Health Habits Pretest and Posttest"
• "Screening Form"
• "Clinical Measures and Followup Form"
• "Community Health Worker Activities Form"
• "Promotores Activities Form"
• "Recording Log (for Strategies 1, 2a, 2b, and 3)"
• "Develop an Evaluation Workplan for Your Project"

Session Outline

Introducing the Session

1. Welcome
2. About the Session

Conducting the Session

1. Develop a Vision for the With Every Heartbeat Is Life Community Project
2. Basic Information on Evaluation
3. With Every Heartbeat Is Life: Three Strategies To Offer in Your Community
4. Data Collection
5. The Role of Community Health Workers in the Evaluation Process
6. How To Work as a Member of the Evaluation Team
7. Group Activity: Develop an Evaluation Workplan for Your Project

Review of Today's Key Points

Closing

Introducing the Session

1. Welcome
• Welcome group members to the session.
2. About the Session
1. Say:
   You have learned a lot of information during this training. You have gained new skills and shown great progress and motivation. Now, you are ready to put the With Every Heartbeat Is Life training into action in your community. Congratulations!
2. Say:
   This session is especially for community health workers. It will teach you how to participate as a team member to evaluate your project.

Conducting the Session

1. Develop a Vision for the With Every Heartbeat Is Life Project for Your Community
• Say:
  During this session, you will develop a vision for a With Every Heartbeat Is Life project for your community.
• Ask:
  Can someone tell me what a vision is?
• Note: Allow 2 to 3 minutes for group members to respond. Write their answers on the blackboard or a large piece of paper taped to the wall.
• Add the following answers if they are not mentioned:
  • A vision is your dream, an image, or a picture of what you want to do and what you imagine your community project will do for the people you serve.
  • It is the direction or goal of your project.
  • It is what inspires, motivates, and engages people to take action.
• Say:
  A vision can be written as a statement or expressed as a drawing.
• Some examples of vision statements are:
  • A heart healthy and stroke-free community
  • An active, healthy, and informed community
• Say:
  Now it's time for you to create a vision for a community project. Think of this question when you create your vision: What would you like your community to be like after implementing your heart health project?
• Note: Divide the participants into small groups.
• Say:
  Choose one person from your group to share your group's vision.
• Note: Give each small group a set of color markers and a piece of poster board. Allow 20 minutes for groups to come up with their visions.
• Say:
  Now, a member of each group will present each vision to the rest of the group members.
• Note: Allow 2 to 3 minutes for each group to share its vision. Congratulate the community health workers for their enthusiasm and a job well done.
2. Basic Information on Evaluation
• Say:
  When you offer the With Every Heartbeat Is Life project in your community, you will want to know if your project helped you to achieve your vision. Evaluation can help you do this. Now let's talk about what evaluation is and some important steps you need to know when you evaluate a project.
• Say:
  Evaluation is a well-thought-out process to assess the value of your project.
• Ask:
  What are some benefits of evaluation?
• Note: Allow about 2 minutes for group members to answer. Write their answers on the blackboard or on a large piece of paper taped to the wall.
• Add the following answers if they are not mentioned.
• Evaluation offers you a way to:
  • Find out if your project is working or is successful.
  • Learn if you are meeting the goals of the project.
  • Find out which project areas need to be changed or improved.
  • Gather information about your project that you can share with the community and those who fund your project.
  • Learn if your project activities are helping people make healthy choices.
• Give each group member the "Examples of Project Evaluation" handout.
• Say:
  Now we will look at how evaluation is used in a few projects.
• Ask volunteers to read aloud each example and tip.
• Examples of Project Evaluation

Example 1

During the month of May, doctors from a community clinic referred 45 patients to the heart health sessions conducted by community health workers. Thirty participants attended all of the heart health sessions. The other 15 participants only attended a few sessions.

After the classes ended, the community health workers conducted followup visits. These visits revealed that the 30 participants who attended the classes were using the project's heart healthy recipes, participating in physical activities, and taking their medicines as the doctor told them. The other 15 participants who did not attend the classes were not using the recipes, most were not doing any physical activity, and several were taking their medicines only when they remembered.

The community health workers saw that the project had a greater positive impact on participants who took part in all of the heart health sessions.

Tip: Plan to track participants at every stage of your project—(1) referrals, (2) class attendance, and (3) followup after the classes.

Example 2

A community health worker is a member of the health promotion team working on a project to increase the physical activity of community members. Participants attended a series of heart health sessions. The community health worker reviewed the results of the project. She found that 15 out of 20 participants who went to at least 6 of the training sessions were now walking for 30 minutes or more per day.

The community health worker learned that people who attended the heart health sessions increased their physical activity.

Tip: Learn your project goals; review them throughout your project. Make sure that project activities make sense and are helping you to reach the goals of the project.

Example 3

A group of community health workers posted flyers in the community about an upcoming cholesterol screening event. They held the screening, but only a few people showed up. After the poor turnout, the community health workers thought about different ways to get the word out. They met with community leaders, got their input, and developed a new strategy. The new strategy was to go door to door to talk about the importance of cholesterol screening and to ask people to share the information they learned with friends and family. Then, the community health workers held a second screening and had a much better turnout.

By finding out what didn't work and getting input from the community, the community health workers were able to make changes in the way they recruited participants.

Tip: Don't focus only on the positive results of the project. You can learn a great deal by looking at what went wrong and what did not work.

Example 4

Community health workers conducted several heart health sessions for community members. One participant shared her high blood pressure story. She described how her doctor told her that she had high blood pressure and about all the healthy changes she made. After 3 months, she had lost 10 pounds, and her blood pressure was under control.

The community health workers asked this participant to share her story at a community gathering. More community members are now interested in taking part in the heart health sessions.

Tip: Be creative. Project evaluation is about more than just numbers. Participants' stories, pictures, and journals can be very powerful tools that describe how your project has affected them.

• Say:
  You have learned examples of project evaluation. Now, let's go over the seven steps to create a plan for implementing and evaluating your project.
  1. Choose the activities. Work with your agency to choose the activities that you want to do. For example, your project may offer the sessions from the "With Every Heartbeat Is Life" manual.
  2. Implement project activities. Perform the activities of your project as planned. For example, you can recruit 20 parents from a local school and conduct the With Every Heartbeat Is Life project at the school.
  3. Collect data. Collect data to show if your project is helping parents. For example, you can use questionnaires to find out how parents used the information, what they learned in the sessions, and if they made heart healthy lifestyle changes.
  4. Enter data. Enter information from the completed questionnaires into a database. This task can be done by trained community health workers or trained staff.
  5. Analyze the data. An evaluator can analyze the data and summarize the findings. For example, an evaluator may find that parents who walk for 60 minutes daily have lower blood pressures and have lost weight.
  6. Write a report about the results. A report can show how the community has changed as a result of the project. For example, the evaluator may describe how parents' eating habits and physical activity patterns have changed as a result of their participation in the project.
  7. Share the results. Community health workers can share results with community members. For example, sharing results can increase community members' interest in the project and motivate them to take personal action to improve their health.
• Say:
  You have seen how evaluation can help you. Now, let's take a look at the types of evaluation: process, outcome, and other evaluation methods.
• Give each group member the "Types of Evaluation" handout.
• Ask volunteers to read aloud the types of evaluation and each example.

Types of Evaluation

Process Evaluation

Process evaluation tells you about the content of project activities. You can learn if you are doing the activities as they were planned. It also tells you who is participating in your project activities. You can track the specifics on how you carry out your project, such as the time spent on activities and how many participants attended the activities. The results of process evaluation help you to know which activities are more successful than others. It also gives you the feedback you need to improve your project.

Example: You can collect information about the number of sessions that you taught from the "With Every Heartbeat Is Life" manual and how much time you spent on activities during the training sessions.

Outcome evaluation

Outcome evaluation describes the effect your project had on your participants. You can learn how the participants changed or are changing after completing the course. You can track how participants' knowledge, feelings (attitudes), or actions (behaviors) have changed after taking part in the project. You also can track the changes in clinical values. (For example, you can check to see if participants' blood pressure or weight has decreased.)

Example: A questionnaire is given to participants before the first class. The same questionnaire is given after the last class. The results of the two questionnaires are compared, which will tell you how much participants learned.

Other Evaluation Methods

You can use other methods to evaluate your project. You can ask participants for their stories (testimonials) about how the project has affected them, and you can collect the stories as the project evolves. Participants also can submit photographs and journals about the changes they have made during the project.

3. With Every Heartbeat Is Life: Three Strategies To Offer in Your Community
• Note: Before the session, read the charts that serve as a guide. These charts list the three strategies in more detail. Each strategy includes: goals, description of activities, setting, and target audience.
• Say:
  You can offer the With Every Heartbeat Is Life project in your community using three strategies. These strategies are:
  1. Train the Trainer. This strategy consists of using the "With Every Heartbeat Is Life" manual to train a group of community health workers, who then go back to their communities and train other community health workers.
  2. Community Education. In this strategy, trained community health workers use the manual and picture cards to teach community members by using one of the following options:
     • Teaching all lessons from the manual to community members.
     • Teach all lessons from the manual and screen community members. Screenings can include the height, weight, and waist measurements of participants. Blood pressure, blood cholesterol, and blood glucose screenings also can be conducted with the help of other agencies and health professionals. If needed, community health workers can refer participants with elevated values to a medical setting for followup.
  • Ask:
    When you do screenings as part of your community education, why do you think it is necessary to partner with other agencies such as a clinic?
  • Note: Allow about 2 minutes for group members to respond.
  • Add the following answers if they are not said.
    • To secure trained health care staff to screen participants for high blood pressure, high blood cholesterol, or blood glucose.
    • To get an authorized agency to obtain permission forms from participants before they are screened for high blood pressure, high blood cholesterol, or blood glucose.
    • To refer participants with elevated numbers to a health care professional at the clinic to confirm if their levels are high and to get health information.
  3. Lifestyle and Clinical Management. In this strategy, community health workers work in a clinic as part of a health care team. Trained community health workers: (1) teach the manual to patients, (2) monitor patients' clinical measures (blood pressure, blood cholesterol, body mass index (BMI), waist measurements, and blood glucose levels) in collaboration with a health care provider, and (3) follow up with patients to offer support and encouragement in addition to making sure that patients are following their treatment.
• Note: Some community health workers may be working in clinical settings, and they may teach patients about conditions such as HIV/AIDS, substance abuse, or prenatal care. Community health workers can use these skills to work with patients who have risk factors for heart disease, such as high blood pressure, high blood cholesterol, or diabetes.
• Give each group member the "With Every Heartbeat Is Life: Three Strategies To Offer in Your Community" handout on pages 395—397.
• Say:
  First, let's review the three strategies to plan how you could offer them in your community.
• Ask:
  Does anyone have questions about the strategies?
• Note: Allow 2 minutes for group members to respond.
4. Data Collection
• Note: Before the session, read the chart that serves as a guide. The outcome evaluation chart describes each strategy, the forms needed to collect outcome data for each strategy, when the forms should be used, and the type of information you can collect to evaluate your project.
• Say:
  Once you have figured out which strategy or strategies you want to offer in your community, then you need to collect data for each strategy.
• Give each group member the "Outcome Evaluation Form" handout.
• Note: Give group members copies of the data collection forms, and describe each form. The data collection forms are:
  • "Community Health Worker Train the Trainer Pretest and Posttest"
  • "Feedback Form—What Did You Think About the Training?"
  • "My Health Habits Pretest and Posttest"
  • "Screening Form"
  • "Clinical Measures and Followup Form"
  • "Community Health Worker Activities Form"
  • "Recording Log (for Strategies 1, 2a, 2b, and 3)"
• Say:
  Let's look at the data collection forms for the three strategies. We will start with the Train the Trainer strategy. This strategy uses the "Community Health Worker Train the Trainer Pretest and Posttest" forms. The pretest form was used at the beginning of the training, and you will use the posttest form at the end of training.
  The changes in the responses from pretest to posttest will help you find out if you learned new information or skills. After the training, you will also complete the "Feedback Form—What Did You Think About the Training?" questionnaire. This form provides information on your likes and dislikes of the training, how confident you feel about training others, and suggestions for improving the training.
• Say:
  For the second strategy—Community Education—you can use the "My Health Habits Pretest and Posttest" forms to find out if participants are changing their health habits. If you add screenings to your community education strategy, you can use the "Screening Form" to record the clinical values of project participants.
• Say:
  Using the third strategy—Lifestyle and Clinical Management—you can use the "My Health Habits Pretest and Posttest" to record how patients are changing their health habits. You can also use the "Clinical Measures and Followup Form" to record the patients' changes in clinical values.
• The "Community Health Worker Activities Form" can be used to keep track of the followup activities that help patients stay on their treatment plan.
• Say:
  The "Clinical Measures and Followup Form" is useful for community health workers who work in a clinical setting.
• Ask:
  How many of you work in a clinic, hospital, or other medical setting?
• Note: Allow 2 minutes for group members to respond.
• Ask each group member to look at the "Community Health Worker Activities Form" handout again.
• Say:
  When you work with patients, it is very important to keep track of your followup activities to help patients stay on the medicines prescribed by their doctors. You should also keep track of the followup activities you do to help patients make lifestyle changes.
• Note: Since you have already reviewed the "Community Health Worker Activities Form" handout, ask volunteers if they have any questions or comments about the activities listed on the handout. Allow 2 minutes for group members to respond.
• Say:
  Collecting information on followup activities can highlight the important role that community health workers have in helping patients stay on their treatment plans and control their risk factors.
• Say:
  The last form that we will talk about is the "Recording Log." This log tells you which form to use for each strategy and what type of information to gather for each strategy. The recording log focuses on process evaluation activities.
• Say:
  Let's review the "Recording Log" handout. The information you will gather for the Train the Trainer strategy is:
  • Number of participants trained
  • Number of sessions taught
  • Number and percentage of participants who rate the training "good" or higher
  • Number and percentage of participants who report that they "will" or "most likely will" change their health habits
  • Number and percentage of participants using the "With Every Heartbeat Is Life" manual 30 days after the training
  • Types of activities that participants are completing 30 days after the training
• Ask a volunteer to read aloud the information for the community education strategy.
• Ask a volunteer to read aloud the information for the lifestyle and clinical management strategy.
• Ask:
  Does anyone have questions?
• Say:
  The "Recording Log" form is important because the results can help you to know which activities are being implemented and how. It also gives you the feedback you need to improve your project.
• Say:
  Before using these data collection forms, we recommend that you review the "More Information" box below. This box contains tips for gathering information for your community project.

More Information

Here are some tips for gathering information for your community project.

Before you offer a questionnaire to others:

• Check with your agency. Your agency may need to approve the questionnaire before you use it. Privacy and consent of project participants are important when doing evaluation. You will want to follow the policies of your organization and your funding agency.
• Make sure you understand how to fill out the questionnaire before you hand it out. Know the form well, so you can answer any questions participants may have.

The day you administer the questionnaire:

• Bring plenty of forms and pens with you.
• Ask participants to answer each question completely. Missing information will make it hard for you to analyze the data.
• If the budget allows, give a small prize to people who sign up for the program or fill out your form.
• Always remember to thank participants.

Note:During the pretest and posttest, the community health workers or volunteers may ask the questions aloud if people need help with their forms. Questions may be read aloud, but the answers to the questions should not be given.

5. The Role of Community Health Workers in the Evaluation Process
• Say:
  Some of you may be thinking that evaluation is too complicated and that there is no role in it for you. But that is not true.
• Ask:
  Has anyone already worked on the evaluation of a project? Can you share what you did?
• Note: Allow 3 minutes for group members to respond.
• Say:
  Here are some examples of the role that community health workers can have in the evaluation process.
• Community health workers can:
  • Help to decide which activities should be conducted.
  • Hand out questionnaires and other instruments to participants.
  • Collect the forms.
  • Enter data into a computer.
  • Provide feedback on what worked and what didn't work and how the activities could be improved.
  • Share results of the evaluation with the community.
  • Attend training meetings about project evaluation.
• Say:
  Remember that your involvement in the evaluation process can help show the value of having community health workers on the project team. The actions of community health workers are key in tracking project activities in the community.
• Ask:
  If you have not been a part of an evaluation team, would you be interested in having a role in the evaluation process of the With Every Heartbeat Is Life project?
• Note: Allow 5 minutes for group members to respond.
• Say:
  Thank you for sharing your experiences about being part of an evaluation team. With time and practice, your confidence level will increase. Just as you have been trained in how to conduct the sessions of the "With Every Heartbeat Is Life" manual, you will be trained in how to conduct evaluation activities for your project. In less time than you think, you will be making great contributions to your evaluation team.
6. How To Work as a Member of the Evaluation Team
• Say:
  People who evaluate projects need to work as a team. Community health workers can be an important part of this team.
• Say:
• A community project team may include:
  • A project manager
  • An evaluator
  • Community health workers
  • Health professionals
• Ask:
  Do you have an evaluator on your project team or in your agency?
• Note: Allow 2 minutes for group members to respond.
• Ask:
  What are some things you can do to get involved with project evaluation when you return to your agency?
• Note: Allow 2 to 3 minutes for group members to respond. Write their responses on the blackboard or a large piece of paper taped to the wall.
• Add any of these answers if they are not said.
• Community health workers can:
  • Find out who does evaluation in their agencies.
  • Share this session on evaluation with supervisors.
  • Ask their supervisors how they can be involved in evaluating the project.
  • Ask their supervisors if they can attend trainings on project evaluation.
• Say:
  If your agency does not have an evaluator, you can try to link with an outside evaluator who supports community health worker projects.
• Say:
• An evaluator can help to:
  • Develop an evaluation plan for the project.
  • Make sure that the project is carried out as intended.
  • Conduct an analysis.
  • Write a report on the data collected.

More Information

Your agency can get the services of an outside evaluator by contacting:

• Other community health worker projects
• A university that has a school of public health
• A foundation that has community health projects and can refer you to an evaluation consultant
• Your state health department

7. Group Activity: Develop an Evaluation Workplan for Your Project
• Say:
  The purpose of this activity is to develop an evaluation workplan for a community project.
• Divide group members into the same three groups that they were in when they developed the vision for their community projects. Give each group a different project from the "Develop an Evaluation Workplan for Your Project" handout.
• Note: Group 1 will work on project 1, group 2 will work on project 2, and group 3 will work on project 3.
• Say:
  This handout on evaluation includes the project descriptions and a blank chart with questions to help you develop a plan on how to evaluate your activities. The questions are:
  1. Who will be your target audience?
  2. What strategy will you use?
  3. Which forms will you use to collect data?
  4. Describe the activities you will use to conduct the strategy.
     • How will you recruit participants?
     • When will you schedule the classes?
     • Who will teach the classes?
  5. Decide who will manage the data:
     • Who will collect the data?
     • Who will enter the data?
     • Who will analyze the data?
  6. Determine who will write the evaluation report.

Community Projects

Note: Discuss your project and the evaluation workplan using "Develop an Evaluation Workplan for Your Project" as a guide.

Project 1: Training Community Health Workers To Promote the With Every Heartbeat Is Life Project in Their Communities

• Your agency has agreed to conduct the With Every Heartbeat Is Life Train the Trainer workshop for 25 community health workers from community agencies that have requested it.
• Your supervisor asks you and two other community health workers to plan the Train the Trainer workshop and to develop the education program.

Note: Discuss your project and the evaluation workplan using "Develop an Evaluation Workplan for Your Project" as a guide.

Project 2: Helping Community Members Make Heart Healthy Lifestyle Changes

• Your community-based organization is in Baltimore, MD.
• Your supervisor asks you to help develop the evaluation plan for the With Every Heartbeat Is Life community education project.
• The course is for adults aged 30 to 60.
• The goal is to recruit, teach, and increase the knowledge, positive attitudes, and skills of the community members attending your classes.

Note: Discuss your project and the evaluation workplan using "Develop an Evaluation Workplan for Your Project" as a guide.

Project 3: Helping Patients Manage Their Risk Factors for Heart Disease and Make Lifestyle Changes

• Your clinic is located in southeast Washington, DC.
• Your clinic has decided to get involved in a project on heart health to help patients change their behaviors and lower their clinical measures.
• You and two other community health workers attended a community health workers conference to receive the With Every Heartbeat Is Life training.
• Now, your supervisor wants your group to develop activities and an evaluation plan for the project.

Note: Allow about 30 minutes for each group to come up with their evaluation plans.

• Note: Ask one person from each group to present the group's evaluation plan to the entire group. Allow about 5 minutes for each presentation.
• Say:
  Thank you for doing such a wonderful job!

Review of Today's Key Points

• Say:
  Let's review what we learned today.

  • What is evaluation?

  • Evaluation is a well-thought-out process for assessing the value of your project. It helps you check to see if you are meeting the goals of the project.

  • What are the benefits of evaluation?

    • Finding out if your project is working as planned, or is successful.
    • Learning if you are meeting the goals of the project.
    • Finding out which project areas need to be changed or improved.
    • Gathering information about your project that you can share with the community and those who fund your project.
    • Learning if your project activities are helping people make healthy choices.

  • What is the role of community health workers in the evaluation process?

    • Helping to decide which activities should be carried out.
    • Administering questionnaires and other instruments.
    • Collecting the instruments.
    • Entering data into computers.
    • Providing feedback on the ways the activities are done and how they can be improved.
    • Sharing results of the evaluation with the community.
    • Attending training meetings about project evaluation.

  • What strategies can you evaluate using the "With Every Heartbeat Is Life" manual?

    • You can evaluate the training of community health workers to promote heart health in the community.
    • You can evaluate what community members have learned and which changes they have made to have heart healthy lifestyles.
    • You can evaluate how patients have been helped in managing their risk factors for heart disease and making changes in their lifestyles.

  • Who forms the community project team for evaluation?

    • A project manager
    • An evaluator
    • Community health workers
    • Health professionals

Closing

• Say:
  Thank you for coming today. What did you think of today's session?
• Note: Allow a moment for group members to respond.
• Say:
  Community health workers play an important role in the evaluation of a community project. We hope this session helps you achieve positive results in your work with the community. I wish you success in implementing the With Every Heartbeat Is Life project in your community.

Handouts

Last Updated December 2010