|
|
|
In this section, you will find the following:
What Questions Will our Club’s Survey Results Help Us Answer?
The Getting Results section describes three survey reports that the Tool Kit can generate:
These reports, in graph or table formats, can help Clubs answer four types of questions:
- How are our Club members doing now in each of the outcome indicators?
- In what types of Club activities are youth participating? What are the participation patterns across the three grade groups (elementary, middle and high school)?
- Are the results more positive for youth who have been Club members for several years?
- How do our results change over time?
To look at each of these questions in more detail, go to How Is our Club Doing?.
Back to top
How Is our Club Doing?
TOOL KIT SCORES: INDICATORS OF CLUB STRENGTHS AND NEEDS
The Tool Kit scores describe the attitudes, beliefs and perceptions of youth who take the surveys. When your Club’s youth answer the survey questions, they receive scores that place them in one of three categories, based on how the 2003 national field-test group scored:
- Doing Great: Indicator scores at this level are in the upper range of the field-test group. These scores suggest the percentage of Club youth who have achieved the outcome described. This group of youth is likely to be able to provide leadership and peer mentoring to Club members in the indicator area.
- Doing Fine: Indicator scores at this level fall in the middle range of the field-test group. These scores show the percent of Club members with a generally positive status on each outcome. Positive support from Club programs should continue to benefit this group of youth.
- Room to Grow: Indicator scores at this level fall in the bottom range of the field-test group. These scores show the percent of youth who have a need for additional support on each outcome.
Each young person’s scores are grouped with those of other survey takers from your Club during the same survey administration period. These cumulative results then are reported in graphs or tables created by the Tool Kit.
THE “75/25 PERCENT” RULE: USE IT TO INTERPRET RESULTS
The 75/25 Percent Rule suggests:
- A reasonable goal for Clubs to achieve on the Tool Kit surveys is for 75 percent of youth to respond in the “Doing Great” or “Doing Fine” categories on as many outcome indicators as possible.
- If more than 25 percent of youth surveyed have “Room to Grow” in one or more indicators, Club staff should explore ways to provide additional support to youth in these outcome categories.
Use the 75/25 Percent Rule to look across your Club’s data to identify indicators that are both strong and weak. Consider what outcome categories could benefit from increased focus and action.
Back to top
QUESTIONS FOR CLUB LEADERS AND STAFF TO CONSIDER
- How are all of our Club members doing now in each major outcome category?
- In which outcome indicators are our youth strongest (i.e., the largest percentage of youth are “doing great”)?
- Why might our members be doing especially well in this area?
- Are there things we are doing in the Club that contribute to these positive results?
- What can we learn from these strong indicators to apply to the weaker ones?
- In which outcome indicators do our members need more support (i.e., more than 25 percent have “room to grow”)?
- Do we currently offer programs or other opportunities that support youth in these indicator categories?
- How could we improve or add programs or other opportunities that better support members in these areas?
- How can we learn from our strengths as we consider improving other programming?
- In what types of Club activities are youth participating? How do participation patterns differ across the three grade groups (elementary, middle and high school)?
- Which activities are most popular with youth?
- Are current activity participation rates helping to accomplish the Club’s goals?
- Are there some activities in which youth are not participating very often? Why is that?
- What changes can we make to encourage youth to participate in the full range of available activities?
- Think about the outcome results specific to each grade group (elementary, middle and high school). Are some groups participating less than others in activities that are important for them?
- Are there enough activities in this area for this age group?
- Why aren’t members at this age taking advantage of existing opportunities? How should we address this?
- Should we add age-specific activities?
- Should we change the way we lead activities to make them more appealing to a particular age group?
- Are the results more positive for youth who have been Club members for several years?
- How do the outcome indicators differ for youth who have been members for a long time?
- Does longer-term membership in our Club seem to lead to more youth Doing Fine or Doing Great in some outcomes but not others? If so, why might that be?
- If there are no differences in results for long-time members, why not?
- Focus on the Club Benefits or Club Connections outcome indicators.
- Is long-term membership related to higher outcomes? If not, what does that mean for our Club?
- What changes within the Club might increase both benefits to youth and long-term membership?
Back to top
Looking at Outcome Indicator Categories and Specific Indicators
The Tool Kit provides Clubs with information related to the achievement of two key youth development outcome goals of the 2002-2006 Strategic Plan that BGCA has established for the entire Boys & Girls Club Movement:
(make these phrases bookmarks/hyperlinks to respective #3 subheads below)
DEEPENING IMPACT
The results for three outcome indicators provide direct feedback to Clubs about their impact on youth. These indicators address the extent to which youth participate in important activities and opportunities the Club offers.
Here are some questions Clubs may want to ask if results are weak for one these indicators (i.e., at least 25 percent of youth are in the “room to grow” category).
- Club Benefits – If these results are weak overall or for a particular age group, think about the breadth and depth of activities and opportunities the Club provides in each indicator area. Think about questions such as:
- Are we providing enough opportunities for youth to experience benefits in each area included in this indicator?
- Are we providing depth as well as breadth across the eight types of activities listed?
- Are we providing skill-building activities, in addition to recreation opportunities, in these areas?
- Are our programs set up so that youth can progress to more challenging activities or higher skill levels?
- Are youth participating in these activities often enough to experience the benefits?
- Is the physical environment colorful, bright and first class?
- Are members regularly recognized via bulletin boards, announcements and posters?
- Are programs and activities fun?
- Club Connections – If these results are weak, think about how Club activities are staffed and what kinds of opportunities the Club provides for youth to bond with each other.
- Are we providing enough opportunities for youth to have meaningful interactions with adults (i.e., not just supervision)?
- Are we retaining staff long enough to foster supportive relationships between adults and youth?
- Are we retaining youth for long enough that they develop a sense of connection and community?
- Do we provide enough opportunities for youth to work together over a period of time toward a common goal to bond and develop a group identity?
- Is our Youth Development Strategy evident throughout the Club? Are members greeted by name? Do staff members regularly remind youth that the facility is their Club?
- Are informal guidance conversations with members taking place? Are members’ suggestions incorporated in program activities? Are staff members constantly looking for opportunities to acknowledge young people for a job well done?
Back to top
STRENGTHENING CHARACTER
Eight outcome indicators reflect the general well-being of Club youth and their emerging development as caring, responsible and productive individuals. These indicators can be influenced by youth experiences at the Club, at home and in their community.
Here are some questions Clubs may want to ask if results are weak for one these indicators (i.e., at least 25 percent of youth are in the “room to grow” category).
- Caring Behavior
- Tolerance for Others
- How is our Club encouraging its members to think about and support one another?
- What kinds of activities do we organize to increase young people’s understanding of themselves and others?
- How are we encouraging youth to know and become friends with peers who are different from them?
- Risk Avoidance
- What percentage of our youth has room to grow in this area?
- What do we know about the possibilities of youth involvement with drugs, alcohol or other dangerous attachments?
- Are we thinking about and addressing the risky behaviors that the surveys address (e.g., alcohol or other drug abuse, alienation from school, etc.)? If so, how?
- How can we strengthen our connections with young people who might be most at risk?
- Are there other community resources that can help us?
- Moral Compass
- Is this an area where many of our youth indicate room to grow?
- How do we integrate the idea of “doing what’s right even when no one is looking” into our everyday activities?
- What activities can our Club implement to facilitate the development of peer leaders who model good character for other youth and encourage them to value “doing what’s right”?
- What are we as staff doing to model good character for youth on an everyday basis?
- Leadership
- Is this an area where many of our youth indicate room to grow?
- What kinds of meaningful leadership opportunities are available to youth of all ages at our Club?
- Are there enough opportunities for all youth to discover their leadership potential?
- Are there opportunities for youth to work as leaders in the Club and community?
- Educational Commitment
- Is this an area where many of our youth indicate room to grow?
- Are our education activities as strong and interesting as they might be?
- Are we encouraging youth to think about their school performance, talk with our staff and work toward achieving high standards and ambitious goals?
- How active are youth in our Power Hour program? Is the program implemented by a stable, valued adult who encourages youth to participate regularly?
- Are youth participating in high-yield learning activities throughout the Club that are fun and engaging yet reinforce the skills and knowledge they are learning in school?
- Technological Skills
- Are 75 percent or more of our youth reporting they are doing fine or great in this outcome?
- Do we have enough up-to-date equipment and software to serve all of our youth?
- Are knowledgeable adults routinely available to encourage youth to use computers and other technology for fun, creativity and learning?
- Are youth teaching youth and working in groups or pairs so they are learning and teaching each other?
- How many youth have fully mastered the available software at our Club? How do we encourage youth to use their knowledge within our Club?
- High-School Youth Employment Competence
- Is this an area where many of our high-school youth indicate room to grow?
- Do we place enough emphasis on career and post-secondary school opportunities with our high-school members?
- Are there practical ways high-school youth can become involved in thinking about and planning for their futures?
- Are we using the career development programs offered by BGCA?
- Do youth come to know adults in the working world and have a sense of how their high-school experiences connect with their future adult lives?
Back to top
Staff Exercise: Analyze Graphs and Discuss Results
Here is an exercise that you can use with your Club staff to practice analyzing Tool Kit survey results. Gather a group of your Club colleagues to examine and discuss the results from a recent Tool Kit survey administration period. Working in groups of two or more is most likely to optimize the thinking about results and help you develop a useful, effective plan of action.
In advance of your meeting, be sure to go to the Generating Reports section of Getting Results to have the Tool Kit produce reports using your Club’s survey data. Alternatively, if your Club does not have any recent survey data in the Tool Kit, you can conduct the exercise using the sample reports for Boys & Girls Clubs of Anywhere featured in the Interpreting Results section of Getting Results.
Remember, the Tool Kit can provide you with the following reports (the first two reports are formatted as bar graphs; the third is formatted as a table):
- Report 1: Overall Results for All Club Outcome Indicators
- Report 2: Rates of Participation in Eight Categories of Club Activities by Grade Level
- Report 3: Deepening Impact: Outcome Indicator Scores by Length of Club Membership
Follow the steps below to conduct your group exercise.
REPORT 1: OVERALL RESULTS FOR ALL CLUB OUTCOME INDICATORS
- On a large sheet of newsprint or flip chart paper, make two columns.
- Write the strongest outcome indicators, those on which 75 percent or more of the youth scored in the “Doing Fine” and Doing Great” categories, in the left column on the paper.
- Write the weakest outcome indicators, those on which 25 percent or more of youth had “Room to Grow,” in the right column on the paper.
- Discuss the possible reasons for these strengths and weaknesses.
- Look at Report 2: Rates of Participation in Eight Categories of Club Activities by Grade Level to explore whether youth in one or more grade levels (elementary, middle and high school) have lower participation levels in areas related to the weaker outcome indicators.
- Brainstorm and discuss programming changes that would provide more or improved support to members in these areas.
REPORT 2: RATES OF PARTICIPATION IN EIGHT CATEGORIES OF CLUB ACTIVITIES BY GRADE LEVEL
- List the most popular activities for all grade levels (elementary, middle and high school), then for each grade level. Brainstorm to identify the possible reasons for the high participation rates in these activities.
- List the least popular activities overall and for each grade level. Discuss whether the participation rates for these activities are too low or whether they seem appropriate based on what your staff members think is the right mix of activities to meet your Club’s goals.
- If participation levels seem too low in some areas, discuss possible reasons and ways to encourage more participation in these areas.
- Go back and look at Report 1: Overall Results for All Club Outcome Indicators. Consider and discuss whether the activity participation results can help explain the overall outcome results.
REPORT 3: DEEPENING IMPACT: OUTCOME INDICATOR SCORES BY LENGTH OF CLUB MEMBERSHIP
- Identify indicators in which long-time members are doing better than those who have been members for a shorter time.
- If long-time members do not seem to be doing better than relatively new members, brainstorm possible reasons for this.
- Brainstorm and discuss possible strategies for improving retention of members and increasing their frequency of attendance and level of participation in activities and programs.
Back to top
Interpreting Changes in Results over Time
The first time a Club administers the Tool Kit surveys, the data that is gathered suggest how the Club’s youth are doing at that time. These results become the Club’s starting point, or baseline, for comparison against outcome measurement data gathered periodically over time.
After the first year of using the Tool Kit, a Club can then re-administer the surveys on a regular basis. The surveys should be re-administered on a set schedule every year that works within the Club’s program year, for example, every fall and spring or every winter. This allows Clubs to be more certain the information they are collecting is consistent with the previous year’s data.
The results of the follow-up surveys describe how Club members are doing over time. Using these later results, Club staff can look back and ask questions about the changes between the first set of results and the new data, such as:
- Do the results from the follow-up surveys show a smaller percentage of youth with “Room to Grow” in areas identified for improvement after the first round of surveys?
- Are the outcome indicators with the strongest results (those for which the largest percentage of youth are “Doing Great”) different from those that were strongest last year? Are these areas the Club targeted for improvement?
- Which outcome indicator areas need more focus this year? Did we lose ground anywhere? What are some possible reasons (e.g., different set of youth with different needs, de-emphasized these areas to free up resources for improvements in other areas, etc.)?
- Do we need to re-adjust the focus of Club programs to address new needs? How do we do this while maintaining our strengths and sustaining improvements?
Back to top
|