pool to school

Evaluating School Commute Programs

Data can tell the story of how activities​ ​and events can encourage walking,​ ​bicycling, carpooling, and riding​ ​the bus.​ This guidebook suggests strategies for gathering information to evaluate program effectiveness. Download the entire Guidebook here.

  • intro Students report how they got to school as part of Walk to School Day in Alameda County.

    Introduction

    Data can tell the story of how activities and events can encourage walking, bicycling, carpooling, and riding the bus. Statistics about how many students are participating in Walk and Roll to School Day or how many emissions the third grade saved over the year can educate parents, school administrators, city or county staff, elected officials, and students themselves about how programs and infrastructure improvements benefit their lives and the community. This information helps guide policy and funding decisions in order to maximize impacts and help provide funding equitably.

    This guidebook suggests strategies for gathering information to evaluate the effectiveness of Safe Routes to Schools (SRTS) and other programs that educate and encourage students to walk, bike, carpool, take transit, or ride the school bus. Parent surveys and student hand tallies are standard methods of evaluating program impacts and are often required as part of grant awards. Programs can also track participation rates, evaluate teachers and lessons, and present the data to their communities.

    Evaluation Requirements

    This guidebook suggests ways of collecting and evaluating data quickly and easily. For information about MTC’s requirements for Regional SRTS programs, contact Craig Goldblatt. For state and federally-funded programs, see the Technical Assistance Resource Center (TARC). More information is available on our website at: www.sparetheairyouth.org/evaluation-reports

  • hand tallies The student hand tally is a simple five minute exercise that helps track program progress over time and is often required for programs receiving SRTS grant funding.

    Data Collection

    Developing relationships with school and district staff and agreeing about the importance of data collection will enable evaluation of programs to run smoothly.

    • Make participation in data collection a clear requirement of receiving services.
    • Clearly define evaluation requirements and expectations.
    • Ask for permission from the principal or the school district. Note whether there are any rules about collecting information from students and parents.

    Tip: Check with the school’s administrative office to see what data they currently collect. The school may already be collecting data that would be useful in program evaluation

  • hand tallies The student hand tally is a simple five minute exercise that helps track program progress over time and is often required for programs receiving SRTS grant funding.

    Student Hand Tallies

    Student hand tallies are a quick and effective way of gathering data about students’ transportation mode. Hand tallies are often required for Safe Routes to School (SRTS) funding. Teachers, program staff, and/or volunteers simply go to classrooms at participating schools and ask students how they get to/from school.

    • Tallies collected over multiple years provide a time series of how transportation behaviors have changed. Hand tallies are considered the most accurate method of collecting information about the school commute.
    • The National Center for SRTS has developed a standard tally sheet for use, or MTC has provided a simplified template that can be downloaded from the web.
    Data Collection

    Developing relationships with school and district staff and agreeing about the importance of data collection will enable evaluation of programs to run smoothly.

    • Make participation in data collection a clear requirement of receiving services
    • Clearly define evaluation requirements and expectations
    • Ask for permission from the principal or the school district. Note whether there are any rules about collecting information from students and parents.

    Tip: Check with the school’s administrative office to see what data they currently collect. The school may already be collecting data that would be useful in program evaluation.

    Timing

    The student tally should be conducted in the spring and the fall.

    • Collect data in September or October, after school has been underway at least a few weeks, but before the weather changes, and again in April or May, before testing or other end-of-the-year events.
    • Conduct the tallies on a Tuesday, Wednesday, or Thursday so that students can be asked about their travel on the previous day as well as how they plan to get home.
    • Avoid Mondays, Fridays, and short Wednesdays if only one day is being tallied, since they tend to have different travel patterns than other days of the week. Also avoid weeks with abnormal transportation patterns, such as the week of Walk and Roll to School Day, testing, or all-school or all-grade field trip days, etc.
    • Work with the school principal or administrator to select a date.
    • Let school staff know about the tallies through a simple school administrator letter, which includes information about timing and instructions for administering and returning the survey.
    Collecting the Data

    Teachers or parent volunteers can minimize disruption to the classroom. Outside volunteers (or paid SRTS program staff, if available) can administer the surveys, although this requires a greater level of coordination with the school office.

    • Send a teacher letter with the tally form to help administer the tally. Give a hard deadline or specific date for completing the survey.
    • Have completed forms delivered to the school office. A program organizer can pick them up, an administrator can scan and email them, or they can be mailed. Program staff can provide self-addressed stamped envelopes to avoid two trips.
    • If volunteers or teachers are collecting the data, it can be helpful to show them the tally form and explain why the information is being gathered.
    • Ask teachers of special education classrooms to provide information on how their students get to and from school in order to minimize disruption.

    Suggestions for getting good information from student hand tallies:

    • Don’t identify yourself as with the Safe Routes to School program, as that may affect students’ responses.
    • Be respectful of teachers’ and classrooms’ time; while the survey will only take about five minutes, you may need to offer to come back at a better time if necessary.
    • Let students know that you are interested in how they plan to get home at the end of the day, rather than how they got to their after school activities.
    • Bring pictures of walking, biking, carpooling, etc to help kids understand the questions.
    • Ask students first how they got to and plan to get home from school on the day you are tallying, then ask students about the previous day, and finally ask about the next day.
    • Define carpooling to younger students, including alternatives like getting a ride or driving with students from another family
    • For kindergartners, ask very specific questions. It may be easier to ask them individually how they got to and from school. Pictures work well. Students can stand in a group by mode to avoid double-counting.
    Entering the Data

    Most tally data should be compatible with the National Center for SRTS’s Data Collection system. There are two options for forms to collect this data:

    • The National Center’s standard tally sheet can be mailed directly to the National Center for data input or input by program staff. However, the form can be challenging to use, and it can take several weeks or months to have the data input into the system. Tip: Be sure to copy your surveys before putting them in the mail in case they get lost.
    • Use an alternative form, which collects the same data but is more user-friendly. Program staff must enter the data into the National Center manually, using their online interface. TIP: The National Center has an online Navigating the SRTS Data System Tutorial with additional instructions. Staff can be contacted at info@saferoutesinfo.org
    How Much Data to Collect

    The National Center recommends collecting data for at least 30 trips for each time of day (30 in the morning as well as 30 in the afternoon) per grade. Therefore two days of tallies at a single classroom with at least 24 students provides sufficient data. However, it is recommended that you collect data from at least two classrooms per grade, ideally more.

    If the school uses aptitude assessments or another non-random means of assigning students to classrooms, it is even more important to capture trip patterns from at least two classrooms per grade.

    • Schools may also want to count bicycles parked on bike racks to gain a more accurate estimate of bicycling use as a comparison to the hand tally data.
    Analyzing the Data

    The National Center for SRTS’s Data Collection System is online at: www.saferoutesdata.org/. Most programs in the San Francisco Bay Area have an existing login. New users must request access to individual schools, which allows the user to input new data and download the existing data. Because hand tally data captures trips taken by every student in a classroom, it is considered more accurate than the self-reported parent survey. It is recommended that hand tally data is used to track progress toward program goals over time. Other analysis, such as effectiveness of specific program activities, can be cross-tabulated against hand tally data.

    Understanding Teen Perspectives

    Teen perspectives are often different from parents' and younger children's. Surveying teen participants can keep program content fresh and engaging.

    Understanding how students perceive their environment can provide a great deal of insight into what activities to focus on.

    Similar to parent surveys, teen surveys are more robust than a hand tally. Questions can cover a variety of topic areas including:

    • Mode choice and reasons for that choice
    • Perception of parental sentiment
    • Built environment and potential barriers
    • Personal behavior
    • Personal safety and crime
  • Which parent survey to use?

    The National Center for SRTS parent survey is an established survey form and methodology. Results can be sent or entered into the Data Collection System, which generates reports by school and program-wide, comparing among time periods. However, some programs may wish to collect additional information or change the questions. Potential modifications to the parent survey:

    • Additional questions about trip sharing (carpool, transit, and busing)
    • Questions about participation in and effectiveness of specific activities and events
    • Using a 5-point scale (strongly agree, agree, neutral, disagree, strongly disagree) for parents’ concerns allowing their students to walk or bike. Make sure you have a way of analyzing the information before asking this
    • Including contact information and a website
    • Removing demographic questions, which may reduce parental participation among some communities
    • Asking parents about their willingness to volunteer for particular activities

    Parent Surveys

    Parent surveys help programs understand parents' concerns and perceptions of walking and bicycling. Because they collect information about transportation mode choice and how far from school the family lives, they provide valuable insight into the potential for shifting to active or shared modes of transportation. Surveys help identify which activities a new program should offer, or which activities are working well in a mature program.

    Administering Parent Surveys

    Parent surveys can be distributed in hard copy or online, via an online survey such as SurveyMonkey. The National Center’s parent survey is available in a variety of languages. Spanish surveys should be hard copy due to low online response rates.

    • Pass out hard copy surveys to students at the time of the student tally or ask students to bring the survey home to their parents as part of ‘backpack mail.’
    • Have students return the form to their teachers by the end of the week, and teachers turn their forms into the school office. The office can have a pre-stamped mailer to return the surveys in, or they can scan them and send to the program manager or implementer for data entry.
    • Conduct the parent survey less frequently than hand tallies. Sending the parent survey every spring and fall can reduce response rates. Recommendation: Survey parents every two or three years, or as funding guidelines require.
  • Six Step Process for Basic Evaluation

    • Before
      • Plan the program / Collect information
      • Write objectives
      • Decide what, how and when to measure
    • During
      • Conduct the program
    • After
      • Collect information and interpret findings
      • Use results
    Source: National Center for Safe Routes to School, evaluation

    Tracking Activities & Event Evaluation

    Keeping track of SRTS activities and evaluating events at a school can help program administrators focus on the most effective activities and events.

    How to Track Activities

    SRTS programs should track what activities and events are being held at each school. Most providers maintain an excel spreadsheet (or many) that has all the schools in the program and all the activities offered through the program.

    • If multiple people administer programs at schools, use a shared Google Doc or set up a Google Drive folder to share materials.
    • For activities held more than once a year, keep track of how often they are provided (i.e. did the school participate in Walking Wednesdays monthly or weekly?).
    • Provide explicit instructions for filling out the spreadsheet to ensure consistent records.
    Tracking Participation

    If possible, recording how many people participated in a specific event or activity can indicate the level of interest students and parents have in an event, as well as tracking the reach of the activity.

    • Indicate who was invited to participate in the activity (e.g. all fourth graders, or two classrooms of second graders).
    • When possible, keep track of how many individuals participated in an event. For school-wide events, you can keep track of how many stickers/pencils/ chocolates you hand out to students, assign a volunteer to keep a tally, or have students place stickers on a poster to show how they got to school.
    • Track the number of classes offered and total students taught for each educational offering.
    • Make it clear whether the school was able to participate in the activity. Some SRTS activities (i.e. bike rodeos) do not have funding for every school to participate.
    parent surveys Evaluating the success of an event allows a program to focus on those activities and events that will truly create mode shift toward walking and bicycling.
    Evaluation of Events

    Evaluate events to determine if they affected students’ travel habits or interest in walking or bicycling. Ask participants to complete a survey after the event is finished. Online surveys are easy to administer and can be taken at the parent’s convenience, but may have lower response rates.

    Ask how participants heard about the event, typical travel modes, ways to improve the event, and attitudes about walking/biking before and after the event.

    Program implementers should also record information such as costs, promotion, and general opinions throughout the planning and implementation of the event.

    • Offer an incentive or raffle drawing for people taking your survey to encourage participation.
    • Ask for open-ended responses that can be used as supportive quotes.
    • Have school staff encourage and remind parents to take surveys.
    • Ask school staff or volunteers to hold a discussion of participants’ feedback at an event.
    • Use existing communication channels such as parent newsletters, list serves, weekly folders in students' backpacks, etc., to remind parents to take surveys.
    Teacher Evaluation

    Teachers provide insight into how programs are being administered at schools and how effective they are. These perspectives can be useful in determining students’ attitudes towards the programs and improving how the program reaches students.

    Survey teachers a few weeks after kickoff to ask to identify program strengths and potential improvements. Send a second survey to teachers six months or longer after kickoff to evaluate the effectiveness of the program, focusing on how well the program reaches students.

  • Intro Discuss participants' observations after the walk audit using a map to facilitate dialogue and to capture input about specific locations.

    Walking & Bicycling Audits & Observations

    Walking and biking audits may be conducted to examine both the physical environment as well as pedestrian and bicyclist behaviors around schools. This information can supplement data collected using other methods, including the student hand tallies and parent surveys.

  • Who to Invite

    Audits are a great opportunity to bring the school community and other stakeholder together to discuss issues and opportunities for future collaboration. Inviting a variety of stakeholders to participate in the walk audit brings together many different perspectives.

    Successful walk audits engage key stakeholders, including:

    • Parents
    • School Staff
    • School district officials
    • City staff from the Public Works and/or Traffic Department
    • Local engineers and planners
    • Law enforcement officials
    • Safe Routes to School Task Force or Safety Committee
    • Children
    • Neighbors
  • Audit Schedule

    Observation of drop-off behaviors at a school that starts at 8:00 a.m. might have the following schedule:

    7:30 - Convene observers in an office or at the school library. Coffee and pastries are always appreciated!

    7:40 - Have observers separate and walk to key locations to watch drop-off patterns and behaviors.

    8:00 - Gather at school office for the bell. Introductions.

    8:10 - Walk around the immediate vicinity of the school and discuss specific observations, issues, and solutions.

    8:40 - Meet indoors to look at a map and discuss intersections further from the school, as well as other actions the group can take to address concerns.

  • While you have a group of interested parents convened for the walk audit, also talk to the group about:

    • Sharing volunteer opportunities with other parents and teachers.
    • Providing parents with traffic safety information and expectations, such as through a police letter or principal reminder in the school newsletter.
    • Gathering information to share during back-to-school night.

    Data to Collect

    In addition to observations from the walk audit, collecting additional data can help identify issues and appropriate solutions, as well as supporting grant applications and funding requests.

    The data collected should include:

    • Infrastructure: Traffic control devices within the school area (i.e. signals, signage and pavement markings), presence and conditions of sidewalks, curb ramps, paths and bicycle facilities, bicycle parking, crossing guards. This data is often available from the jurisdiction or GoogleMaps
    • School circulation: Drop-off and pickup movement and policies, walking or bicycling routes.
    • Conditions at intersections: Crossing distance, traffic light timing, size of the staging area, behavior of cars in relation to pedestrians.
    • Programs that educate children on walking and biking, as well as those designed to encourage students to do so.
    • Behavior of students and parents walking, bicycling and driving near school.
    • Automobile traffic counts and turning movement counts help identify appropriate traffic control devices and locations for crossing guards.

    Sample forms to collect a variety of data can be found in the Resources chapter.

  • Analyzing the Data

    The data collected during the walking and biking audits may be analyzed by program implementers and used in conjunction with tally and/or parent survey results to develop a school report. The data can be used to identify engineering and programmatic improvements with local leaders.

    • Encourage participants to focus on the issues rather than solutions. Participants are likely to offer specific solutions, but it is more useful to get at the source of problems.
    • Publish common concerns or observations in the school newsletter and discuss how the school and SRTS community is working to address them.
    • Work with your local jurisdiction and/ or the SRTS implementation team to develop an improvement map to identify the priority infrastructure needs, which can be the basis of a grant application.

    Data collected from a walk audit can be used to apply for future infrastructure funding.

    Intro
  • Additional Resources

    Sample Evaluation Reports
    Sample Walk Audit Guides
    Tools For Walk Audit Next Steps