BioTeach

 

 

 

 

 

 

 

BioTeach Program of the MassBioEd Foundation

 

Program Evaluation Report for 2005-2006 Cohort

August 2006

 

Polly Hubbard

Jim Hammerman

Karen Mutch-Jones

 

With work from additional team members

Jana Marcotte

Deborah Munson

Gillian Puttick

 

 

Cambridge, Massachusetts


Table of Contents

 

 

Executive Summary .......................................................................

Program Overview .......................................................................... 3

Evaluation Overview ..................................................................... 4

Data collection .................................................................... 5

Participants ...................................................................................... 7

2005 Summer Training Program .................................................. 9

Follow-up training .............................................................. 13

Supplies ........................................................................................... 13

Supplies purchased .............................................................. 14

Kits purchased ...................................................................... 15                 

Supplies use .......................................................................... 15

Impact of supplies on teaching ............................................ 17                      Dissemination to non-attending teachers               18

Integration of Labs .......................................................................... 20

Lab use ................................................................................. 20

Month of use ........................................................................ 22

Reason for use ...................................................................... 22

Supports ............................................................................... 24

Effect on Science Department ...................................................... 25

Participant Ideas for Support ........................................................ 26

Impact of Program on Students ..................................................... 27

Life Science Career Awareness .................................................... 28

Career awareness student activities .................................... 28

Teacher/counselor communication ................................... 29

Life science career awareness pilot ..................................... 31

Awareness of industry .......................................................... 32

Collaborations between counselors and others ................. 33

Leadership teams .................................................................. 34

Issues to Note .................................................................................. 34

Recommendations .......................................................................... 36


Program Overview

 

BioTeach is envisioned as an ambitious multi-faceted program of MassBioEd, the Massachusetts Biotechnology Council’s non profit education foundation. The program’s founding mission is to enable every public high school in Massachusetts to teach biotechnology by the year 2010.  By integrating biotechnology instruction and programs on career opportunities in the life sciences into the core biology program of every high school in Massachusetts, students will be motivated to pursue opportunities in life sciences education and careers after high school. BioTeach’s goals are to outfit school science labs with lab supplies to teach biotechnology; provide professional training for biology teachers in biotechnological science; help teachers access and use biotechnology curricula, expose students and teachers to career awareness activities in the life sciences, and partner with schools, government and local businesses to support scientific curiosity, increase student participation in science and for some, train for a career in the life sciences.  BioTeach’s first source of funding was a $1.4 million grant from the United States Department of Labor, followed by private resources primarily contributed by the Massachusetts biotechnology industry and $1.45 million from the Massachusetts legislature.

 

BioTeach offers competitive grants to schools for purchase of approved supplies such as equipment and continuing financial support for three years to purchase consumables. Schools designated as high needs supported by the Department of Labor grant receive up to $11,000 for supplies and up to $3,000 in consumables during Year One. They can receive up to $2,500 annually in matching funds for supplies and maintenance for Years Two, Three and Four. The remaining schools receive up to $7,800 for supplies and up to $1,000 in consumables during Year One and up to $1,200 annually in matching funds for consumables and maintenance for Years Two, Three and Four.

 

The program provides consultants to train a minimum of two teachers from each awarded school to promote use of equipment. Training consists of a three-day summer institute developed and run by staff at the Museum of Science and the Boston University CityLab, and a fourth day developed and run by the community lab at Biogen Idec. Subsequent free training of one day every other year is available for Years Three, Five and Seven. In addition, a subset of schools also received a series of three shorter workshop training sessions developed in collaboration with the Metro South/West Regional Employment Board and hosted by biotechnology companies for teams of teachers, guidance staff and administrators. These  “life science career awareness workshops” (LSCA) piloted some introductory presentations and action plan activities designed to build upon teacher and guidance capacity to influence student direction in careers in the life sciences and to strengthen the connection between classroom learning and the workplace.

 

 

 

 

 

All of these training activities were situated within a broader conceptual web that included:

  • building a strategic partnership with the Department of Education so that pre-service teachers would be introduced to biotechnology and so that BioTeach can contribute to the STEM (science technology, engineering and math) pipeline, and
  • building a speakers’ board of scientists who would volunteer to work in school settings with teachers and classes, and
  • remaining closely attuned to individual biotechnology company activities in schools to spread the growing knowledge base about what challenges and supports schools need to integrate biotechnology into the core curriculum.

 

This report describes and explores comparative outcome data about those programmatic aspects-school grants and workshops and career awareness workshops enacted since July 2005, focusing on the period January 2006 to June 2006. Where helpful, survey data from August-December 2005 collected by MassBio staff has been provided to explain or augment the end of year findings. Formative feedback has been regularly provided to program staff since March 2006. While this report is not meant to consolidate all formative findings previously shared, quarterly reports in the accompanying folder may be reviewed for many recommendations and additional notes used during oral formative feedback sessions from January to June.

Evaluation Overview

 

102 of 143 possible participants (71%) chose to participate in the program evaluation. The evaluation was voluntary although the memorandum of understanding signed by principals and science chairs required high schools to participate in three evaluation surveys. With no budget for compensating participants only 22 teachers, 16 department chairs, and 18 career awareness team members completed both of their pre and post surveys. Many more individuals submitted fewer than the required number of surveys. Thus pre and post responses are reported in the aggregate to take advantage of the significantly higher number of respondents who completed at least one survey. Evaluation responses cannot be said to be representative of the entire group of program participants due to a lack of data about those who chose not to participate. However there is enough cross checking of the data to feel confident that these responses point to some significant issues to consider. The five overarching evaluation questions developed by TERC in conjunction with the BioTeach program manager and director for this report are:

        1: To what extent do various constituencies participate in BioTeach?

        2: What do teacher and guidance counselor participants learn about biotechnology from their BioTeach professional development?

        3: To what extent and how do various constituencies implement new learning and change their behaviors as hoped for by the BioTeach program?

        4: How do students’ experiences in class change as a result of their teachers’ and schools’ participation in BioTeach?

        5: To what extent do these changes result in long-term changes in attitudes and knowledge in biotechnology for students?

 

Two additional evaluation questions were developed and are included in the design to identify preliminary data to support a planned longitudinal evaluation. These are:

 

        6: To what extent do schools that participated in BioTeach sustain biotechnology classroom and school efforts over time?

 

        7: To what extent does BioTeach stimulate changes in non-school arenas for the teaching and learning of biotechnology?

 

Question six was partially addressed this year due to a lack of participant data and will be readdressed during Year Two (2006-2007). Question seven will be addressed as part of the longitudinal evaluation to begin in Fall 2007.

 

Data Collection

 

TERC Evaluation began in October 2005. Evaluators worked with MassBioEd staff to design an evaluation that matched the scope and goals of the BioTeach program. TERC data collection began in January 2006 soon after the design was finalized. Given the late start, data could not be collected early in the school year and pre-and post- workshop surveys developed prior to the award were not coordinated with the evaluation design.  While this prevented a true pre/post comparison of the effects of the training on participants, the evaluation team attempted to gather some data retrospectively by asking participants to think back and assess such things as supply use and lab work prior to their participation in the program. Refer to Chart One for a summary of data collection instruments and total number of respondents.


Chart One: Summary of Data Collection Instruments and Respondents

 

Instrument

Number of Evaluation Participants

Number of Respondents

Percent of Evaluation Participants

Month of Data Collection

Pre training survey

Data collected pre consent

108

--

July-August 05

Post training survey

Data collected pre consent

107

--

July-August 05

Early Implementation Teacher survey (Fall)

64

41

64%

February 06

Late Implementation Teacher survey (Spring)

64

41

64%

June 06

Observation Follow-up Day One

Day Two

Day Three

 

11

18

16

 

11

18

16

 

 

100%

                   January 06              February 06                    February 06                  

Early Implementation Department Chair survey

24

24

100% *

February 06

Late Implementation Department Chair survey

24

23

96%*

June 06

Purchasing orders

34

34

100%

April 06

Pre LSCA survey

30

20

67%**

February 06

Post LSCA survey

30

18

60%**

April 06

Observation of LSCA

                      Day One

                       Day Two

                       Day Three

 

 

30

 

10 of 17

14 of 23

17 of 23

 

 

64%

 

February

April

June

NOTE: There are different subgroups being evaluated so there are different levels of participation in the evaluation

* The percents for department chairs are calculated out of 24 chairs although 34 were required to participate per the MOU generated prior to contract with TERC. TERC requires that evaluations be voluntary. Only 24  chairs signed informed consent documents.

** The percents are calculated of 30 LSCA participants.

 

Teacher survey data were collected four times. Pre- and Post- Training Survey s developed by the BioTeach staff prior to the evaluation contract were administered in the summer of 2005 to evaluate the immediate impact of the training. Online Early Implementation and Late Implementation Surveys were administered in January and June 2006 to evaluate classroom implementation. In addition, the three follow-up training days were observed by evaluators and instructors were surveyed and informally interviewed about teacher responses to follow-up training. Sometimes N’s vary as teachers did not complete every item on a survey.

 

School data were collected four times. Original Applications developed prior to the evaluation contract were completed in March 2005. Online Early and Late Implementation Surveys were administered to Science Department Chairs in January and June 2006. School supplie s Purchasing Orders were collected and reviewed in April 2006. Sometimes N’s vary as department chairs did not complete every item on a survey.

 

(Pilot) Life Science Career Awareness Program  survey data were collected twice. Five schools in cohort one sent teams to the Life Science Career Awareness (LSCA) workshops. One school that was not part of the cohort was invited to have a team participate in the LSCA work as well.  Individuals completed a Pre-workshop survey in February 2006 and a Final survey in May.

 

LSCA observation data were collected by members of the evaluation team during the three training events for pilot team members.

 

Participants

 

The program participants can be divided into two groups: those in training workshops (108 summer, 61 follow-up training) and those who attended the Life Science Career Awareness seminars (30). At least two science teachers from every school were required to register for one of six summer training workshops.

 

Data obtained from evaluation participants provide more detail about teacher prior experience and training ( Chart Two ). The majority of summer training attendees who responded were teachers (83%) with 9% department chairs who also teach, and 7% non-teaching administrators. While most of the 108 entered the BioTeach workshops having taught general, advanced placement, honors, or other biology courses, 11 (about 10%) had not and listed courses such as anatomy, chemistry, and physics as their area of expertise.

 

Chart Two: Participant Characteristics

Characteristics

Number

Percent

N

                   Role

 

 

108

Teachers

90

84%

 

Department Chairs, teaching

10

9%

 

Administrators, non-teaching

8

7%

 

 

 

 

 

                  Prior Teaching Experience

 

 

100

General Biology

62

62%

 

AP/Honors Biology

41

41%

 

Biotechnology

4

4%

 

Vocational/Technical

3

3%

 

Non Biology

11

11%

 

NOTE: Teachers may teach more than one course so prior teaching experience will not sum to 100.

 

By December, thirty-seven teachers (34%) reported teaching one or more science or vocational science classes with 75% of respondents teaching either 2 or 4 courses during the year (1 or 2 courses per semester). By June 2006, evaluation respondents reported teaching Introductory or General Biology courses most frequently followed by other non biology courses such as chemistry, AP/ Honors biology, biology-related Vocational courses and just a few individuals teaching Integrated Science, Second Year Biology, or Biotechnology ( Chart Three ).

 

Chart Three: Courses Taught September 2005-June 2006

Courses

Term

Number

Introductory Biology

Fall

20

Spring

19

Any second year Biology

Fall

3

Spring

3

AP/ Honors Biology

Fall

8

Spring

10

Integrated Science including Biology

Fall

2

Spring

2

Biotechnology

Fall

2

Spring

2

Vocational/Technical Biology

Fall

5

Spring

4

Other non-Biology

Fall

14

Spring

15

NOTE: From 41 respondents to Spring survey

 

 

 

All these details support the conclusion that most of the teachers who attended were able and likely to present biotechnology to students based on the classes they taught. The eleven who were not biology teachers had no easy opportunity to integrate the summer training with students. If the goal remains to focus on student exposure to biotechnology, summer training should continue to be focused on those who will teach biology topics in the coming academic year and/or who support biology teachers until programmatic connections can be made to other science domains like chemistry .

 

We were interested in the biotechnology training that these teachers received prior to and during BioTeach. Thirty-five teachers of the original 108 participants had received some prior training in biotechnology during the previous five years. This “above beginner-level” of experience for 32% of the group was not identified early enough for differentiated instruction to occur. By December 2005, 10 of 41 teachers reported receiving additional biotechnology training from providers other than MassBioEd. Of the 9 who reported the number of hours of training they had received, the median was 4-8 hours, but three people received more than 18 hours. While we wondered whether these teachers with more experience or knowledge about biotechnology would get the same thing out of the BioTeach program as their less experienced colleagues, we found this did not seem to affect the groups’ overall highly positive response to the training.   Nor did it seem to influence frequent or earlier implementation.

 

Massachusetts does not mandate class period lengths, making it challenging to design a lesson suitable for all schools in the state. Of 32 schools reporting, 18 schools (56%) reported their typical science class as between 45 and 60 minutes while 10 (31%) reported longer periods of 61-90 minutes. Lab periods are often longer than regular science class periods to allow for more in-depth investigations. Typical lab periods were 61-90 minutes for half the schools although 9 (29%) reported 45-60 minutes and two (6%) reported less than 45 minutes being available for labs ( Chart Four ).

 

Chart Four: Class Length by Regular Science Classes and Lab Classes, 2005-06

 

Regular Classes

Lab Classes

Class Length

Number of Schools

Proportion of Schools

Number of Schools

Proportion of Schools

< 45 minutes

4

12%

2

6%

45 to 60 minutes

18

56%

9

29%

61 to 90 minutes

10

31%

16

50%

TOTAL

32

100%

27

84%

 

This variability in classroom contexts may influence teacher’s ability to integrate the BioTeach labs into their curriculum. A period with 45 total minutes may not provide enough time to set up and clean up any wet lab, biotechnology-focused or not. The program may want to consider focusing their resources on those schools that have at least one 60 minute period every week or 10 days to devote to wet labs. Additionally the program might want to review the labs to be sure the number and pacing of lab segments (e.g. wait an hour, sit overnight, etc) are realistic for high school schedules.

 

Training program

 

The summer training consisted of three sequential days of training of approximately six hours each followed by one full day of training in late January or early February. For summer training, teachers could select one of six possible dates in either of two locations: South Boston (Boston University CityLab) or Cambridge (Museum of Science). Approximately 19 people attended each session.  Staff from both organizations delivered the trainings although a common format was used ( Chart Five) .

 

Chart Five: Format of Summer Training Session

Time

Activity

Day One

 

8:30

Registration. Introduction, Pre Workshop Survey

10:00

Safety and Basic Lab Skills

11:00

Transformation lab content lecture

12:00

Lunch

1:00

Prep materials, begin lab

3:00

Complete lab and practice analysis

4:00

Adjourn

Day Two

 

9:00

Data Interpretation and transformation efficiency

10:00

Mini Prep lab content lecture

11:00

Mini Prep lab

12:00

Lunch

1:00

DNA Fingerprinting (Restriction enzyme) content

2:00

Restriction digest and cast gel

3:00

Load and run gel, stain gel

4:00

Adjourn

Day Three

 

9:00

PCR content lecture, acquire sample

10:00

Continue lab

11:00

PCR paper-based activity and animation

12:00

Lunch

1:00

Make gels, prep and load PCR, run gels

2:00

Alu insertion content and pedagogical model

3:00

Stain and analyze gels

3:30

Post workshop survey

4:00

Adjourn

 

Teacher needs directed the pace of some sessions but, in general, the sessions were broadly similar in content, goals and time on task. The focus of the training was helping teachers learn how to use specialized supplies such as electrophoresis boxes associated with biotechnology techniques. Four different wet labs were presented in three days to give teachers exposure and practice and to provide them with something they could use with their students. The four labs chosen were: DNA Fingerprinting, Isolation of Plasmid DNA through Mini Prep, Polymerase Chain Reaction , and Transformation of E.coli Cells. Learning objectives were outlined in the teacher materials ( Chart Six) .

 

Chart Six: Learning Objectives for Labs

Lab

Learning Objectives

DNA Fingerprinting

Investigate restriction enzymes as “DNA scissors” and understand their use in research and criminal justice.

 

Use restriction enzymes to cut DNA.

 

Perform the process of gel electrophoresis. To differentiate DNA samples by size.

 

Learn how to analyze gels, use them to infer information about cut DNA. 

 

 

Isolation of Plasmid DNA through Mini Prep

Brainstorm reasons why researchers or pharmaceutical companies would want to isolate plasmid DNA.

 

Isolate and purify a cloned plasmid by performing a mini-prep procedure.

 

 

Polymerase Chain Reaction

Determine human ALU genotypes and genotypes of control samples via PCR and gel electrophoresis.

 

Isolate human DNA from cheek cells.

 

Use PCR to amplify the ALU region of human DNA.

 

Use agarose gel electrophoresis to analyze the amplified ALU region.

 

Understand the process of DNA amplification by PCR and its many uses.

 

 

Transformation of E.coli cells

Investigate transformation as a mechanism of genetic exchange.

 

Create competent cells by chemically and thermally treating E. coli cells.

 

Insert a plasmid containing luminescent genes into competent E. coli cells.

 

Determine if transformation was successful by selection on ampicillin media.

 

Calculate the efficiency of the transformation reaction.

It was not expected that participants would teach only these labs, or indeed required that they teach any of them. However when asked at the conclusion of their training how likely it was that they would include any of the labs in their curriculum, teachers responded and, on a scale of 0 (“Definitely will not”) to 4 (“Definitely will”) the average across all four labs was 2.4 (“50-50 chance”), with the lab most likely to be taught as DNA Fingerprinting ( Chart Seven ).

 

Chart Seven: Likelihood of Teaching Lab as of Summer 2005

 

Likelihood of Teaching Lab in 2005-2006

 

 

Labs

0 Definitely Not

1  Probably Not

2       50/50 Chance

3   Probably Will

4 Definitely Will

MEAN

N [1]

Transformation

3

11

24

25

24

2.6

87

Mini prep

6

18

21

22

14

2.2

81

Fingerprinting

1

2

21

34

33

3.1

91

PCR

6

18

21

22

14

1.7

81

TOTAL

16

49

87

103

85

2.4

 

 

Based upon post-workshop survey data, sessions were very well received by participants. Of 107 respondents, fifty percent (54) were “very confident” of their ability to incorporate some of the lab activities into their curriculum and 44% (47) were “somewhat confident”. Only 3% (3) were “not at all confident”. When asked at the end of the training if they would teach biology or biotechnology courses or content in the coming academic year, 82% (88) responded “yes”. 

 

When asked how well the training met their expectations, 25% (27) of 108 participants did not respond.  However, 63% (51) of those that did respond (81) agreed the training had met all of their often numerous expectations, and 13 of those described the training as exceeding their expectations ( Chart Eight ).

 

Chart Eight: Teacher Expectations

Response Types

Number

Percent

N

No Response

27

25%

108

Exceeded expectations

13

16%

81

Met expectations

51

63%

81

Did not meet expectations

13

16%

81

Non codeable

4

5%

 

 

Comments illustrating examples of expectations exceeded follow:

 

“The workshop was very reinforcing and exceeded my expectations.  You all provided motivation to me in bringing more activities and labs into the curriculum.”

 

“I got a lot more than that!  I learned a lot about biotechnology and I learned various ways to get my students to understand the material through great classroom activities!”

 

“Labs were excellent-well organized, explained well and the protocols were the best I have ever used at a workshop like this.”

 

Only 12% (13) expressed unmet expectations. A sample of these remarks follows.

 

“The workshop was strong in biotech technique, but application or link to classroom curriculum would be helpful.”

 

“Though I now feel comfortable using the equip. I still feel unsure about how I would use it.”

 

“I thought the background information exceeded my expectations, however, we needed more assistance and explanations on the grant (costs, when equip. would arrive, etc.)”

Teachers were still feeling only somewhat to moderately prepared to teach all labs 11 months after the training. When 40 teachers responded to a question in June 2006 to rate how prepared they feel to teach each of the four labs on a scale from 0 to 3 (Not at all; Somewhat prepared; Moderately; and Extensively prepared) they felt most prepared to teach DNA and least prepared to teach PCR ( Chart Nine ). Some comments identified that lack of expensive equipment to complete PCR experiments was the chief reason they scored this at “Not at all”.

Chart Nine: Preparedness to Teach Each Lab as of May 2006

Labs

0             Not at all

1    Somewhat

2   Moderately

3    Extensively

Mean

DNA

12% (5)

22% (9)

35% (14)

30% (12)

1.83

 

 

 

 

 

 

PCR

22% (9)

40% (16)

22% (9)

15% (6)

1.30

 

 

 

 

 

 

Mini Prep

20% (8)

42% (17)

25% (10)

12% (5)

1.30

 

 

 

 

 

 

Transformation

18% (7)

35% (14)

30% (12)

18% (7)

1.48

Statistical tests on the differences between these means reveals responding teachers feel significantly more prepared to teach the DNA lab than any of the others.

When asked to describe the overall impact of the summer training on their teaching of biology or biotechnology on a scale of 0 to 3 (Not at all, Somewhat, Moderately, Extensively), on average teachers rated the summer training (1.51) as having between “somewhat” and “moderate” impact.

 

Follow-up Training

Follow-up training ( Chart Ten ) was delivered by the community lab manager at Biogen Idec and staff from the Museum of Science and CityLab. The focus was to introduce teachers to the biotechnology industry in Massachusetts, help them develop an initial awareness of career paths in the industry and fill gaps left from the initial training.  These topics were developed in response to the most frequent suggestions on post workshop surveys.  Teachers indicated that they wanted time to practice labs and help identifying where in their curriculum these labs could be inserted.

 

Chart Ten : Follow-up Sessions Overview

 

Date

 

Topics For all dates

Evaluation Participants Attendance

(% of summer workshop attendees)

1/25/06

2/2/06

2/14/06

•Small group activity mimicking process of drug development

•Lecture on biotech industry and the drug discovery process

•Tour of research labs

•Career Awareness lunch with members of   Biogen Idec

•Break-out sessions

a)        Review DNA fingerprinting and Transformation labs

b)        Curriculum design to integrate labs

 

10 (9%)

14 (13%)

17 (16%)

TOTAL:

41 of 61 overall attendees

38%

 

Staff provided excerpts from the state science standards and provided mini-lectures about concepts and themes in the labs that could be emphasized to address core science learning goals. Teachers who chose to go to these sessions worked in small groups with their textbooks to plan ways to introduce the labs during the following months.

 

Attendance of 38% of the original attendees was problematic for evaluating this component of the program. More than half of those attending the lab review sessions had not attended the summer training. This made it difficult for instructors to give high ratings to participants learning the lab . It is clear teachers and schools did not understand that this was a required part of the program .

 

From the follow-up sessions, staff reported learning about participant’s needs and the challenges of placing the lab work into a conceptual framework . They reported using this information plus formative data from early implementation surveys (not reported here) to redesign the summer training for 2006.

 

Supplies

Prior to BioTeach, MassBioEd supported biotechnology education efforts by providing needed supplies that schools often couldn't purchase on their own.  At times, MassBioEd was disappointed to learn that schools couldn't use the supplies as frequently or fully as they had envisioned.  Therefore, with the inception of BioTeach, MassBioEd planned to continue fulfilling supplies needs while also incorporating training to help teachers become more familiar with the apparatus via specific lab applications.  This section of the report describes this new effort, including the purchasing, use, impact, and sharing of supplies for 2005-06 BioTeach schools. Data for this analysis were collected through early and late implementation surveys given to teachers and to department chairs.  Supplies orders were reviewed and observation notes from follow-up training sessions provided additional information.

 

Supplies Purchased

In September/October 2005, participating schools generated purchasing orders for supplies.  A few schools waited until later in the academic year to place their orders.   Once approved by the BioTeach Program Manager, each school's supplies were ordered and delivered to them.

 

A review of all the purchasing orders for supplies submitted between September 2005 and April 2006 provides profiles of how schools are using supplies funds provided by the grant. Most schools' order lists contained supplies that are specific to biotechnology such as ELISA plate readers, electrophoresis apparatus and power sources that support molecular/cellular biology investigations. Others ordered supplies such as micropipettes, spectrophotometers, and centrifuges that are likely to be used in more advanced biology labs and indicate that biotechnology is most likely being carried out.  The aforementioned supplies could be used in the four labs that teachers worked on during the summer training as well as in other, similar biotechnology labs. A few schools that ordered these kinds of biotechnology supplies augmented their order by purchasing some more general apparatus and supplies at the same time.  Some schools ordered supplies such as balances, vortexes, hot plates, or incubators.  While these are used more broadly across biology or chemistry labs and not in biotechnology per se, this equipment, if not already owned by the school, is still essential for running biotechnology labs.

 

While most supplies ordered have clear biotechnology applications, there do seem to be some exceptions. For instance, five high-need schools placed orders for microscopes or microscope supplies.  While these may be used for some biotechnology labs, microscopes were not used in the summer training and are not used commonly in biotechnology labs.  Three schools ordered a number of items that are unlikely to be used in biotechnology labs and are apt to be employed in a chemistry class such as a fume hood or a distiller. Additionally some orders were placed for anatomy/physiology and ecology supplies. Overall though, the supplies lists from most schools seem consistent with biotechnology requirements and provide a foundation for teaching biotechnology labs .

 

The ordering process was too slow for some teachers who needed the supplies at the start of the school year —this was often the chief complaint found on spring and fall implementation surveys. However, by December 2005, 30 of the 35 teachers (86%) who responded to a survey question about supplies purchases said they had received most of the supplies they had ordered with the grant (excluding kits and consumables).

 

Open-ended survey comments from teachers and department chairs indicated that once received, many teachers were pleased with and eager to use the new supplies. As one teacher explained, "The lab equipment enables us to map out a future course in biotechnology."  Teachers and Department Chairs appreciated the fact that the supply orders were tailored to their school's needs, a few thanked MassBioEd for their generosity, and several were enthusiastic about using "quality equipment" actually used in the field. 

Kits Purchased                                                                                                              While schools were expected to allocate funds for consumables needed in biotechnology labs, teachers were able to purchase three kits produced by Ward Natural Science with BioTeach funds. Each kit contained some consumables along with other materials needed to teach either the DNA, Transformation, or Mini Prep labs.  A review of all individual school orders in May indicates that twenty-four schools purchased 104 kits for $13,696, an average of  $571 per school. However, over half the survey respondents didn't try to order any of the kits and many of those who tried discovered that they were not always available (of those who tried to order a kit 65% could not get DNA, 50% could not get Transformation, and 44% could not get Mini Prep).  These results are from the spring survey and we can not tell if teachers eventually ordered and received the kits for use at a later time, ordered from a different supplier, or did not order. This will be followed up during year two if funds permit tracking cohort one again. Those who were able to order and use the kits rated them highly, saying that kits were priced fairly, of good to high quality, a time-saver, and something that they might order again.  Thus, if the ordering challenges can be eliminated, supporting schools' purchase of kits might enhance the ability to teach the labs.

 

Supplies Use

When surveyed about first semester use of the supplies ordered via BioTeach, department chairs from participating schools indicated limited use, with over half stating that the supplies hadn’t been used at all.  Many chairs noted that the supplies hadn't arrived until late in the fall semester, and thus, teachers would wait until the appropriate point in their spring semester to use them.  Such predictions appear to be accurate for when surveyed again in May 2006 many more supplies were being used —only 3 of 22 department chair survey respondents indicated supplies weren’t being used in any course [2] . Overall, 11 department chairs reported equipment use in one type of course in their schools and 9 reported equipment use in two or three types of courses. 

Schools used the supplies in a fairly wide range of classrooms.  As shown in Chart 11 , these same department chairs reported use in a range of courses with use occurring most frequently in advanced placement or Honors level courses. The 7 “other” courses include 3 forensic science courses, 1 genetics honors biology course, 1 biotechnology unit in a 9th grade general science course, 1 chemistry course, and 1 after-school program. However, it is important to remember that classes are not equally balanced within a school context—there are usually many more introductory level biology classes and sections in a school than there are AP/Honors biology classes. This commonly accepted statement is born out in looking at the participants. Teachers report teaching more general or introductory biology classes (62) than advanced placement and honors classes (41 combined). Thus it is likely that the supplies are being incorporated into higher-level courses in greater proportions.

Chart Eleven: Use of Grant-Funded Supplies in Courses

Department chairs were also asked to select three types of supplies purchased with grant funds that were used most frequently with students during the spring. Chairs could select multiple responses but were directed to select the top three. As shown in Chart , 22 department chairs replied that micropipettes were used most (rated in the top 3 in all but two schools who completed the survey) followed by electrophoresis supplies, temperature control apparatus, and Ward lab kits.

Twenty-two Department chairs rated which three types of supplies that would be most important to purchase for their biology labs next year and these numbers also appear in Chart 12 .


Chart Twelve: Supplies Used Most Frequently with Students

Supplies

Used this year

Important to buy next year

Micropipette

18

4

Electrophoresis supplies

11

6

Temp Control apparatus

10

3

Lab kits

10

17

Centrifuge

6

4

Light Apparatus

2

2

Autoclave

1

6

Other

1

2

Thermocycler

0

10

 

Similar to reports by department chairs, we see increases in student supplies use during the spring semester. On the May 2006 survey, teachers’ responses about students' use of supplies January-June 2006 showed statistically significant increases over the January to June 2005 period for use of a micropipette, serological pipette, vortex, centrifuge , and electrophoresis box.  No statistically significant changes occurred for use of a balance , practice washing glassware , and practice of sterile techniques .  However, teachers were already teaching the use of the balance to more than 2 classes and about washing glassware to between 1 and 2 classes in the prior year so there might have been fewer additional classes to whom these techniques could be taught.

 

Impact of Supplies on Teaching

Teachers were asked to rate the impact of new supplies purchased via BioTeach on their teaching; their responses are presented in Chart Thirteen . By the end of the academic year, 64% of responding teachers indicated that the equipment and supplies were having an impact with 28% of the group noting extensive impact.  The reasons behind the 14 'no impact' responses (36%) were not entirely clear.  A few in this group did not attend the summer workshop and thus did not take advantage of learning to use new equipment. Some in this group had few if any teaching responsibilities (e.g., administrators) making impact on classroom practice less likely.  For others, the reasons remain unknown.

 

Chart Thirteen: Impact of Equipment on Teaching

 

 

Some

 

Moderate

 

Extensive

 

Total

 

 

No Impact

By Spring

(N=39 teachers)

15% (6)

21% (8)

28% (11)

64% (25)

 

36% (14)

 

Twenty-two (22) teachers responded to the survey item about impact of equipment at both the early and late implementation phases, and their responses at these two points in time were compared.  As indicated by the striped bars in Chart Fourteen , 91% of teachers stated that the equipment was having some type of impact on their teaching during the early implementation phase with the largest percentages noting some and moderate levels of impact.  At late implementation, 82% of this group (indicated by the solid bars) still reported impact of equipment on their teaching with the largest percentages noting moderate and extensive levels of impact.  There is a slight increase in the “no impact” rating at late implementation, however, there are only a total of 4 teachers in this category which are not enough to draw many conclusions.

 

Chart Fourteen: Impact of Equipment on Teaching at Two Points in Time

 

Dissemination of Supplies Information to Non-Attending Teachers

On the May survey, department chairs reported that a large proportion of teachers who did not attend the summer workshop but who had participating colleagues in BioTeach were made aware of the new supplies.  Half of these department chairs indicated that their entire department was informed and all of the other department chair respondents, except one, reported that half or more of their staff was aware that new supplies had been purchased. Numerically, this translates into a total of about 140 non-attending teachers, an average of 80% of the teachers in these science departments, who were made aware of new supplies.

While levels of awareness are high, opportunities for learning how to use the new supplies have been more limited. According to department chairs, about 25% of non-attending teachers in their departments have been taught how to use some of the supplies and about 15% of non-attending teachers have been taught how to use all of the supplies. There are a few departments in which between 10% and 50% of non-attending teachers have used some of the supplies and/or have done some of the summer activities with students (one to two more people per department).  Still, over half of the reporting department chairs stated that none of their non-attending teachers have used the supplies yet. Size of the department seemed to have no impact on the additional numbers of teachers who used some or all of the supplies or activities--on average one or two additional teachers did so no matter how large the department. It may be useful to think about how BioTeach can support larger departments to increase the spread of the actual use of BioTeach supplies and activities since they're currently doing so in roughly the same numbers, but at a lower proportional rate than are smaller departments. This relationship can continue to be tracked in future years.

 

Survey responses from teachers echo findings from department chair surveys.  Teachers report (on both December 2005 and May 2006 surveys) that they shared some of their BioTeach learning experiences, but this was often done informally with a small number of colleagues ( Chart Fifteen ).

 

Chart Fifteen: Sharing BioTeach Training Across Science Departments

 

How Training Was Shared

 

1

Not at all

2

Limited to one teacher

3

A few teachers

4

Some teachers

5

Many teachers

Informally discussed attendance at workshop with science teachers at my school

 

22% (9)

 

18% (7)

 

30% (12)

 

25% (10)

 

5% (2)

Demonstrated how to use new lab supplies to teachers

65% (26)

15% (6)

8% (3)

10% (4)

2% (1)

Other

98% (39)

2% (1)

0% (0)

0% (0)

0% (0)

 

While some teachers actually demonstrated how to use lab supplies for other teachers, the majority did not; in fact, 61% (20) of 35 respondents did not demonstrate in the early implementation phase and as seen in Chart Fifteen , 65% (26) of 40 respondents did not demonstrate during the late implementation phase. So while awareness of new biotechnology supplies has spread, emphasis on dissemination that can support additional teachers to use the supplies appears to be needed to increase the number of teachers who can benefit from their colleagues’ work in the BioTeach program .

 

While there is still work to be done to increase supplies use, particularly with non-attending teachers, data reveal that schools are using supplies more frequently and fully than reported by MassBioEd staff in past years. Overall, schools are ordering supplies that will provide a foundation for biotechnology lab work and trained teachers are increasingly using it in a range of their classes, even though the range isn't yet proportionate to schools' course offerings.  Furthermore, the new supplies are having some impact on teaching and holds promise for new opportunities and rigorous science experiences for students , as described in the following teacher comment:

 

The experience at the [BioTeach] program was EXCELLENT.  There is not any way science teachers could get that experience and the possibility of the supplies to open doors to students.  Going on a field trip to complete a lab on electrophoresis was excellent for my students in the past.  But having the supplies at our school and thus opening the exposure to all of our students is an outstanding privilege.  I am hoping that some of our advanced placement bio students will try comparative DNA analysis of organism in different phyla to study the cladistic changes with biochemical evidence.  Please continue to offer teachers this experience to make science class state of the art.

 

 

Integration of Labs

 

Forty-one teachers and 23 department chairs responded to survey questions about the ways in which they used the four labs and accompanying resources introduced in the summer training. On the spring survey only 16 different teachers reported teaching any of the four labs and only 21 reported using anything: information, labs or resources from the training. Reviewing the data about lab use can help the project staff understand what supports and hinders integrating biotechnology in high school classrooms.

 

Lab Use

Differences in which labs teachers used and how many classes they integrated them into are outlined in Chart Sixteen. The DNA Fingerprinting lab was most commonly used (15 teachers used at least a part of it) followed by the Transformation lab (9), PCR (4), and MiniPrep labs (3). The DNA lab had the most accompanying context and a problem to be solved which might have made it a more popular lab. Also the role of restriction enzymes on DNA is considered a more fundamental concept for science students and may have mapped more readily onto existing curriculum .

Of teachers who used each lab, 60% used it with one class and 40% with more than one class. Teachers who had completed at least one lab were asked to select one they had completed with students and share more detailed information about in the class in which it was used [3] . The 16 teachers who responded reported splitting their use of the DNA lab between AP/ Honors and General/ Integrated courses. They were more likely to use the Transformation lab in the AP/ Honors courses, but the numbers on these are very small, so these differences are not statistically significant and could just represent chance variation in responses.

Chart Sixteen: Integration of Labs into Science Classes

Lab

AP/ Honors

General/ Integrated

Vocational

Non-College

N

DNA

5 (50%)

4 (40%)

1 (10%)

 

10

Transformation

3 (60%)

1 (20%)

 

1 (20%)

5

PCR

 

 

1 (100%)

 

1

Given that there are more general biology than advanced biology teachers responding to the survey, this initial distribution of implementation at higher student levels may indicate a fragile willingness to use biotechnology with all students, in all classes. This perception exists for some teachers as is shown by some teacher comments in early and late implementation surveys:

“Integrating biotechnology labs in AP biology is much easier than in intro biology.”

“My students are either very entry, low level ESL or they are Special Ed and ESL. I can't convey the concepts or the relevance of these activities to 15 year olds. ..”

“…Hard to bring 9th gr. biology students to the point where they are ready to work with the concepts required. ”

“In my general biology class, going beyond the electrophoresis protocol is too much.”

“The materials were not available for me or my students; resigned to advanced classes only.”

“The materials were only available to those teaching at the AP level due to the size of our school. I did use some of the DNA demonstrations to my students ie. bonding between nitrogen bases.”

“I look forward to attempting these labs next year (as long as I don't have 30 students with behavior and learning disabilities…)”

The perception of biotechnology being only for the high level students should be programmatically addressed as a negative stereotype so that all students receive some aspect of the training delivered to the teachers .

Teachers were asked if they used most of the labs, a part of a lab, or none of the lab during the second semester of 2006. This was one way to identify whether the lab was considered a “plug-in” or not. Plug-ins were assumed by program staff to be easier to apply than something that required more massaging to match classroom goals. For those who used each lab, teachers were roughly split as to whether they used most of the lab or only a part ( Chart Seventeen) .

Chart Seventeen: Amount of Each Lab Used

Lab

Not Used

Used a Part

Used Most

N

DNA

55% (22)

20% (8)

25% (10)

40

PCR

90% (36)

5% (2)

5% (2)

40

Mini Prep

92% (37)

5% (2)

2% (1)

40

Transformation

70% (28)

12% (5)

18% (7)

40

Teachers who had completed at least one lab with students were asked to rate the lab on a variety of criteria. Ratings for all labs identified are in Chart Eighteen . The DNA lab was rated more highly than the Transformation lab on all criteria and these differences were mostly statistically significant. The Transformation lab was rated especially low on its links to future curriculum. Only one teacher picked the PCR lab and rated it moderately on all 5 criteria.


Chart Eighteen: Lab Ratings

 

Criteria

 

Lab

 

N

0

Not at All

1

To Some degree

2

Moderately

3

Extensively

 

Mean

 

Matched your primary goals for student learning

DNA

Transform

PCR

10

5

1

0

1

0

0

0

0

4

3

1

6

1

0

2.6

1.8

2

Was interesting for most of your students

DNA

Transform

PCR

10

5

1

0

1

0

0

0

0

3

3

1

7

1

0

2.7

1.8

2

Was at the appropriate level of difficulty for most of your students

DNA

Transform

PCR

10

5

1

0

1

0

1

0

0

3

3

1

6

1

0

2.5

1.8

2

Built upon other lessons/concepts this semester

DNA

Transform

PCR

10

5

1

0

1

0

0

0

0

2

4

1

8

0

0

2.8

1.6

2

Will link to future curriculum this semester

DNA

Transform

PCR

10

5

1

1

2

0

0

3

0

5

0

1

4

0

0

2.2

0.6

2

 

Month of Use

Teachers were asked when they had first used information, ideas or any of the four labs from the summer training. Of the 41 who responded by the end of the year, 20 said they did not use the information at all. Of the remaining 21 teachers, half were using something (e.g., a reference sheet, a lab, a piece of supplies) from the training by November 2005 and a quarter started using the information during the late spring (April-June 2006). This late implementation was confirmed by questions asked of the groups at the follow-up training days concluding in mid February. However the follow-up days were developed to assist teachers who might need more support or training so it was expected that we would see teachers who had not yet implemented labs in greater proportion attending these sessions.

 

Reason for Use

While teachers chose curriculum for many different reasons, it appears that the program has the best chance of being implemented if it helps students prepare for or pass the statewide mandated test due in 2008 . As one teacher put it:

 

The curriculum frameworks for biology are very extensive. In fact, they were recently expanded to include human physiology. Biotechnology, per se, is not a part of the frameworks, so time spent on that is time taken away from material that will be tested.”

 

It is likely that the science curricula and instruction will change as mathematics and English language arts curricula and instruction changed with the advent of the statewide test. Of the teachers responding to a question about the primary learning goal of a biotechnology lab, “learn how science is used in the real world” (47%) was the most frequent, followed by “learning lab techniques” (30%) and “learning science concepts” (14%). It is likely that, if teachers felt the labs could teach core science concepts well, they would be more likely to integrate the labs more frequently as a way to teach students fundamental knowledge . Students do not yet get tested on their knowledge of how science is used in the real world. However understanding real world applications has an influence on student career paths, among other positive effects.

 

The most frequently cited reason for not integrating the labs into the curriculum during the school year was the perception of “appropriateness.” In June 2006, 10 (24%) of 41 respondents to a question about appropriateness reported that at least one of all four labs was “not appropriate” for the content of the classes they teach. Many teachers felt the PCR lab was too difficult for their students and that they did not have the right supplies or supplies to teach it and the MiniPrep lab. In contrast, almost no teachers reported that they had better labs on the topics and were using those instead ( Chart Nineteen ). Interviews with teachers planned for the spring would have explored this response in order to explain it reliably.  However, teachers did not volunteer to be interviewed in high enough numbers, so interviews were postponed to cohort two when stipends will be higher.

 

Chart Nineteen: Reasons for Not Integrating Labs, June 2006

 

Labs

Reasons for Non Use

DNA

PCR

Mini Prep

Transformation

N/A-- I taught this lab

42% (17)

12% (5)

12% (5)

28% (11)

Not yet at that point in my curriculum

18% (7)

15% (6)

15% (6)

18% (7)

Not appropriate for the content of the classes I teach this year

30% (12)

35% (14)

40% (16)

35% (14)

Too difficult for my students

10% (4)

22% (9)

15% (6)

12% (5)

We don't have the right supplies or supplies

12% (5)

22% (9)

22% (9)

15% (6)

Did not find labs useful enough

5% (2)

10% (4)

10% (4)

5% (2)

Have other labs on these topics that I like better

2% (1)

0% (0)

0% (0)

0% (0)

NOTE: Respondents could select all that applied so these will not sum to 41.

 

 

 

 

 

During group conversations on the follow-up days, teachers described challenges in relation to integrating biotechnology in general and/or the 4 core labs in particular.

 

General themes of challenges to integration they described are:

  • Matching lab length to classroom period length (both long and short).
  • Time to prep/set up/ clean up is too much (e.g. fingerprinting cannot be done in 4 sessions, teachers adding sessions). Finding labs for one session or one day would be very helpful
  • Labs too procedurally complicated for basic biology or year one students
  • Presence of challenging students (e.g. beginning English language speakers, large class sizes 30+,  and low attendance levels of urban students)
  • Lack of teacher content knowledge
  • Labs center around same theme (genetics). Teachers cover it at the same time. Is there a way to do different labs or spread these out across the year or semester to illustrate other science concepts?

 

Supports

The program tried to anticipate the supports teachers would need to integrate the training and staff worked hard to provide resources such as reference materials, PowerPoint presentations, “ready to go” kits, and follow-up training.

 

Teachers were asked how many, if any, additional materials they used to teach the lab or prepare students for each lab. Results are shown in Chart Twenty . The data on this are of limited value for the PCR and MiniPrep labs since so few (N=5) teachers taught either of these. Still, on average, for each of the four labs, teachers claim that they need “Some” additional resources (medians all 2 on a range from 0 to 4) with a range for both the DNA and Transformation labs from “None” to “Many Others”. In particular, of those teachers who taught the labs, substantial numbers (>20%) used a Textbook or a Kit manual when it was available, and some teachers teaching the DNA lab used a Website. Between 20%-40% of those who taught each lab used no additional resources.

 

Chart Twenty: Teachers Reporting Any Use of Additional Supports

Supports

DNA

Mini Prep

PCR

Transformation

Did not teach

50% (20)

80% (32)

80% (32)

65% (26)

Textbook

25% (10)

8% (3)

8% (3)

10% (4)

Website

12% (5)

2% (1)

5% (2)

2% (1)

Kit Manual

20% (8)

8% (3)

2% (1)

10% (4)

Other teacher

5% (2)

2% (1)

0% (0)

2% (1)

Periodical article

5% (2)

2% (1)

2% (1)

2% (1)

Other

8% (3)

0% (0)

2% (1)

2% (1)

 

The PowerPoint presentations about techniques were used by seven of 16 teachers presenting the labs. Nine teachers used the DNA fingerprinting PowerPoint (CD and print outs) to prepare themselves or as guides for student use. Fewer instances of PowerPoint use were reported for the Transformation lab (6) and the Mini Prep lab (3) (note: use could have been different teachers or from the same pool of 9).   Between 20 and 40% of those who taught the labs reported using no additional supports.

 

Of those who taught the labs, the resources were used by at least half of the teachers. This is neither good nor bad---but it does mean the assumption that teachers MUST have a       “plug-in” may not be accurate. Some may and some may not. But for some, the package of resources is not yet a complete stand-alone resource.

 

Effect on Science Department

 

A key program goal is to have biotechnology lessons and concepts present in each school. This can best be observed by looking into the degree of spread from teachers who attended the training to teachers who did not attend the training. While the spread of information about supplies has been reviewed in the supplies section, here we report dissemination of lessons and concepts.

Teachers who participated in the summer trainings shared their experiences of the labs by sharing lab protocols. Details are in Chart Twenty-One . However 20% of teachers (8 of 40) did not share with colleagues in these ways.

Chart Twenty-One: Sharing the Summer Training with Science Departments

 

How Training Was Shared

 

1

Not at all

2

Limited to one teacher

3

A few teachers

4

Some teachers

5

Many teachers

Informally discussed attendance at workshop with science teachers at my school

 

22% (9)

 

18% (7)

 

30% (12)

 

25% (10)

 

5% (2)

Shared lab protocols from the summer training with teachers

38% (15)

25% (10)

18% (7)

18% (7)

2% (1)

Presented a training or workshop to teachers

92% (37)

0% (0)

0% (0)

5% (2)

2% (1)

e. Other

98% (39)

2% (1)

0% (0)

0% (0)

0% (0)

 

There did seem to be a small increase in both general and advanced biology classes in the number of “Wet Laboratory” activities in the schools since the project began. Of 22 department chairs answering the question, 11 respondents reported that 80% or more of their advanced biology science students engaged in wet lab activities in 2005. 15 reported that in 2006, an increase of 4. Similar gains were reported for basic biology classes: seven reported that 80% or more of basic biology students engaged in wet lab activities during 2005 and that increased to 11 by 2006. Additionally the number of general biology classes where no students engaged in wet lab activities dropped from 4 in 2005 to zero in 2006. For advanced biology students the number dropped from 7 to 5. This may indicate a positive trend that should be followed in year two.

Another measure of school impact was the extent to which schools were able to provide their own funds to support the purchase of consumable materials like solutions. The Department chairs described the amount of school funds, not grant funds, used to purchase consumables or supplies for biology classes this year. The mean expenditure was $2820 with the middle half of departments spending between $1160 and $3800. The maximum amount spent was $8000 (in two high need school districts) and the minimum spent was just $100 (also at a high need school). There is no significant correlation between amount spent and department size.

Evaluators noticed varying degrees of attention paid by department chairs to the requirements of the grant as laid out in memoranda of understanding (MOU). The MOU stated “To receive the funding and professional support, MassBioEd requires the High School demonstrate a commitment to: a) introducing and/or strengthening their biology program through the inclusion of biotech wet labs; and b) helping MassBioEd ensure the success of its BioTeach program.  This commitment from your school requires: Participation in three, short evaluative surveys, including a field visit by MassBioEd staff, in order to assess the efficacy and impact of the BioTeach .”   Of 34 schools in cohort one, only 24 department chairs or their equivalents sent responses to the survey. It may not be clear to principals and department chairs that they have a small duty to inform the grantee twice a year of the effects of the grant. They may also not have seen the MOU.

 

Participant Ideas for Support

 

Teachers and department chairs were asked to report their ideas for ways to increase the teaching of biotechnology in their schools.

 

Seventy-three percent (16) of the 22 department chairs reporting felt their teachers could benefit from more curriculum design training for integrating biotechnology into existing programs. Other topics appear in Chart Twenty-Two . The requests for the same labs and different labs being nearly equal may indicate a need for offering choice in training to address department needs.

 

Chart Twenty-Two: Department Chair requests for Follow Up Training

Follow Up Training Topics

Percent

Number

More curriculum design training for integrating biotechnology into existing biology curricula

73%

16

Exactly these labs and their associated equipment

41%

9

Different labs and different equipment

36%

8

More program design training for developing a stand-alone biotechnology class

23%

5

Other (please specify)

5%

1

 

Thirty-nine teachers reported their interest in receiving more training in a variety of topics. All topics in Chart Twenty-Three referred to developing student abilities or experiences. The topic with the highest average response was “Developing ways to share information about careers in the sciences” (1.82), and the topic with the most individuals “very interested” was “Developing out-of-school biotechnology field experiences” (12). This is promising for soliciting greater participation in life science career awareness activities. The topic with the most responses overall appeared at the “interested” level: “Refocusing biotechnology labs to emphasize core science concepts required in my school” (17).  This finding seems to indicate that teachers are trying to find supports for integrating biotechnology into more basic biology classes . As it is similar to the Department chairs focus the providers of follow up training should take this into account.

 

Chart Twenty-Three : Teacher Interest in Further Training as of May 2006

 

Training Topics

0

Not interested

1

Somewhat interested

2

Interested

3

Very interested

 

Mean

Developing ways to share information about careers in the sciences

 

10% (4)

 

23% (9)

 

41% (16)

 

26% (10)

 

1.82

Developing biotechnology labs for use with general biology students

 

15% (6)

 

23% (9)

 

38% (15)

 

23% (9)

 

1.69

Developing out-of-school biotechnology field experiences

 

18% (7)

 

28% (11)

 

23% (9)

 

31% (12)

 

1.67

Refocusing biotechnology labs to emphasize core science concepts required in my school

 

15% (6)

 

26% (10)

 

44% (17)

 

15% (6)

 

1.59

Developing biotechnology lab activities in one 50 minute class

 

23% (9)

 

21% (8)

 

33% (13)

 

23% (9)

 

1.56

Developing ability to interpret data from biotechnology labs

 

23% (9)

 

28% (11)

 

28% (11)

 

21% (8)

 

1.46

Teaching the use of additional science laboratory apparatus

 

23% (9)

 

33% (13)

 

26% (10)

 

18% (7)

 

1.38

 

Impact of Program on Students

 

Estimating the number of students affected in any way by the BioTeach program as outlined to this point involves some extrapolation. A total of 124 teachers participated in the BioTeach program through the summer institutes and follow-up days. In addition, we estimate that another 140 non-participating teachers learned something about the program from their colleagues or department chairs. We know from surveys that teachers participating in the evaluation reached a rough average of 34 students per semester in the courses they teach that might be appropriate for biotechnology content. (For this calculation, we assume that students taking an appropriate course did so both semesters.) Thus, we can say that the teachers of approximately 4,200 students (124 x 34 = 4216) participated directly in the BioTeach program, and that the teachers of approximately 9,000 students (264 x 34 = 8976) were affected in some way by the BioTeach program. In year two classroom visits should provide more meaningful data to expand upon this finding.

Life Science Career Awareness 

In concert with BioTeach's vision of promoting and supporting student science literacy in the classroom, the mission of the Project's Life Science Career Awareness pilot program is:

To provide technical assistance and professional development opportunities for school counselors to support their implementation of the Massachusetts Model for Comprehensive School Counseling Programs through the BioTeach initiative by designing and delivering a scope and sequence of career development education (CDE) experiences for students interested in pursuing a career in biotechnology, bioengineering and/or the life sciences.

 

During the 2005-06 academic year, career awareness professional development focused on work with six pilot schools (a description of this work and outcomes is below).  However, since increasing student participation in science courses and biotechnology careers was a publicized Project Goal, staff hoped that teachers from across the cohort who weren't part of the pilot might begin to share information with their guidance colleagues. Therefore, a few survey questions were asked of teachers and department chairs to measure any changes that may have taken place.

 

Career Awareness Student Activities

To establish a baseline, teachers and science department chairs were asked to quantify and describe the kinds of career awareness activities they or the department implemented during the fall prior to their participation in the BioTeach Project.  Of the 26 teachers who responded to the Teacher survey, 16 implemented one activity for students, seven implemented 2 activities, two implemented 3 activities, and one implemented 4 activities.  Types of activities implemented most frequently included field trips to a science museum or the Boston University CityLab, hosting guest speakers in biotechnology, and presenting continuing education information (e.g., college) in biotechnology. Department chairs concurred but, in addition to the aforementioned activities, identified class discussions, videos, and science fairs as vehicles used in their schools for increasing career awareness. These activities were primarily targeted at college-bound students, and only secondarily targeted vocational and non-college-bound students.

In the spring of 2006 after completing the summer BioTeach workshops and a follow-up session, neither the amount nor type of career awareness activities had changed very much. Of the 39 teachers responding to the final survey, 12 (31%) said they had implemented none of the options (which included an “Other” category). Experiences that garnered over 20% of responses included the same activities identified at baseline (a trip to a science museum or the BU CityLab, a guest speaker on biotechnology, and a presentation about continuing education/ college in biotechnology).  More intensive experiences for students as interns or job shadowing scientists, or going to a biotech company were identified less frequently, although 8 teachers (21%) said they had done one of these.  A breakdown of activities and responses is listed in Chart Twenty-Four.

 

Chart Twenty-Four: Career Awareness Activities

Career Awareness Activities

Response Percent

Response Total

A field trip to a science museum or the BU CityLab

35.9%

14

A presentation about continuing education (i.e. college) in biotechnology

30.8%

12

None

30.8%

12

A guest speaker for a class or assembly in biotechnology

23.1%

9

Other (please specify)

10.3%

4

A job shadow experience between students and lab scientists

7.7%

3

An internship for students with a science company

7.7%

3

A field trip to a biotechnology company

5.1%

2

 

Once again, Department Chairs identified other activities beyond those that teachers reported as shown in Chart Twenty-Five . Some activities were rare (e.g., a Biotechnology Club and Internships at biotech companies) while most schools continued to have field trips and guest speakers in classes or assemblies. The majority of these were designed either for all students or all science students. However, students in high level science courses seemed to have more field trips as part of classes (7) than did students in low level science courses (1).

 

Chart 25: Career Activities Reported by Department Chairs

 

Number of Department Chairs who identified each activity for each student group

 

Activity

Not offered at our school

Offered to all students school-wide

Offered to all science students

Offered to high level science students

Offered to low level science students

Biotechnology Club

18

2

-

-

-

Field trips as part of classes

5

2

6

7

1

Guest speaker in classes

6

3

7

2

2

Guest speakers in assemblies

15

2

3

-

-

Job shadow day

15

4

1

1

 

Internships at biotech companies

18

-

-

2

1

Other

18

-

1

Science Career Track

-

1

CityLab intern

 

Science Teacher-Guidance Counselor Communication

At baseline, both teachers and science department chairs reported little to no communication with guidance counselors about biotechnology. The overall amount of communication about biotechnology education or careers for students post high school appeared to change little following their work in BioTeach, with 64% of 39 respondents saying they hadn’t communicated at all with guidance counselors about these topics during the spring, 2006 ( Chart Twenty-Six ).  Similarly, 19% of department chairs maintained there was no communication during the same time period with the majority indicating that science and guidance engaged in “a little” or “some” communication.

 

However, if teachers and guidance counselors did communicate about these biotechnology issues, they’re likely to communicate several times with 13% saying they communicated twice, and 18% saying they communicated 3 or more times. For 72% of teachers, levels of communication are about the same as in previous years, but 23% say there’s either more or a lot more communication with guidance counselors than in the past.

 

Chart Twenty-Six: Communication Between Science Teachers and Guidance Counselors

Communication between teachers and counselors about Biotechnology education/work experiences post high school

Percent

Number

Not at all

64 %

25

Once

5%

2

Twice

13%

5

Three or more times

18%

7

 

 

 

Communication between Science and Guidance Departments about student plans for science careers

 

 

None

19%

4

A little

29%

6

Some

43%

9

A moderate amount

5%

1

A great deal

5%

1

Difference in communication between teacher and the guidance department this year

 

 

A lot less

3%

1

Less

3%

1

About the same

72%

28

More

10%

4

A lot more communication

13%

5

 

This change is communication between teachers and the guidance department is statistically significantly (p <.05) so we can conclude that, on average, communication about biotechnology is increasing, based on teachers’ reports. Finally there is a correlation between the level of communication and the positive change in level of communication (p<.001) suggesting that in schools where teachers already talk with guidance counselors about biotechnology career issues, they’re likely to be talking even more lately.

Given that no formal structure was in place to facilitate science teacher communication with guidance counselors nor were specific supports provided to encourage career awareness student activities for the entire cohort , it isn't surprising that only a few small changes were evident. And it is worth noting the positive communication changes for a small segment of the cohort .  Furthermore, this year one data might be useful in making future comparisons as BioTeach staff integrate biotechnology career information and activities in follow-up events for future cohorts.

 

Life Science Career Awareness Pilot –Six Leadership Teams

The BioTeach Life Science Career Awareness (LSCA) efforts were developed for a pilot project with thirty participants. The goals of the pilot were to:

 

        Partner school counselors with teachers in professional development to promote the coordination of efforts to increase life science literacy and career development education.

        Build the capacity of school counselors to lead a systemic change initiative through a sustainable leadership team model.

        Increase the number of career development experiences for students, in particular, workplace learning experiences that expand students’ awareness of career options and their understanding of job/career expectations and rewards.

 

Leadership teams with guidance counselors, science teachers, and occasionally department chairs or principals from six different high schools were established.  Each team had between 3 and 7 members and five of these six teams included members who attended the summer training. Three LSCA workshop sessions were offered to all teams between late February and early June 2006, and each team was supported by Workforce Investment Board (WIB) representatives who assisted them in collaborating and designing and, in some cases, implementing a career development action plan. While the LSCA workshop agendas scheduled work periods for action planning, there was never sufficient time for this to occur.  Therefore, almost all teamwork occurred outside the workshop sessions. Two of the teams received additional funding for planning.

 

In addition to learning more about the field of biotechnology, the state of the biotechnology industry in Massachusetts, and the work of two specific biotechnology companies, teams also had an opportunity to network and learn from each other. At the third session, a MA guidance/career development model that provides the framework for team planning was explained further and four of the six teams presented their project plans. More information about the LSCA workshop sessions and team attendance is detailed in Chart Twenty-Seven .

 

 

 

 

 

 

 

 

 

 

 

Chart Twenty-Seven: LSCA Workshop Sessions Overview

 

Session #

(Length)

 

Date

 

Host

 

Topics

Attendance

(of 6 possible teams and 30 possible participants)

1

(3.5 hrs)

2/28/06

AstraZeneca

•Company tour

•Lecture on biotech industry and the drug discovery process

•Intro to AstraZeneca and its role in the local biotech industry

•Networking dinner

6 teams

28 participants

2

(3.5 hrs)

4/27/06

Biogen Idec

•Intro to Biogen Idec, its role in the biotech industry and the range of employee positions and work

•Panel discussion with Biogen Idec employees—focus on preparation for and wide range of work in the company

•Presentations by faculty from high schools and community college regarding biotech courses and programs

•Networking dinner

6 teams

21 participants

3

(2 hrs)

6/8/06

MassBioEd

•Overview of Guidance/Career Development Model

•Presentation of Project Plans and Implementation by School Leadership Teams (4 of 6)

4 teams

18 participants

 

Team members from the six pilot schools were surveyed prior to the first session and after the final session to gather their perceptions about the type and extent of their learning. Evaluators also collected observation data during the three sessions. There were 20 respondents to the baseline survey and 18 respondents to the final survey.  In both cases, guidance counselors, teachers, and administrators were represented with slightly more guidance respondents in each data set.  It is important to note that 17 of the 18 respondents on the final survey were from the four teams who followed through with developing their action plans and presenting them at the last session.  Their interest and commitment may have been greater than the two teams who did not complete the final requirement, and thus, they may not precisely reflect the larger population of guidance counselors, teachers, and administrators in the cohort.  Still, this data is important in terms of helping those developing the LSCA program to understand the impact of the sessions on this group.

Awareness of the Industry and Contributions of LSCA Sessions                                 Prior to the LSCA workshop sessions, members from the pilot teams completed a survey and rated their awareness of seven items related to the biotechnology industry on a 5 point scale that ranged from “Not at all” aware to “Very aware.” Respondents reported a moderate awareness overall (mean 2.98) about the biotechnology industry. They had relatively more awareness (mean response ≥ 3.0) of educational requirements for biotech, the work of biotech in medicine, and employment opportunities in biotech. They had relatively less awareness (mean response < 3.0) of the work of biotech in environmental protection, agriculture, ethics/ law, and particularly defense.

Following the three LSCA workshop sessions, team members responded to a modified question rating their awareness of ten issues related to the biotechnology industry on the same 5-point scale. Two of these items were the same as on the pre-test: awareness of employment opportunities in the biotech industry, and educational requirements needed to work in the biotech industry and the rest were different. [4] Their mean rating for all items on the post-test increased to 3.99, indicating a more solid awareness. Comparing just the two questions that were the same on both the pre- and post-test—the mean rating increased from 3.23 on the pre-test to 4.17 on the post-test.  This positive increase is statistically significant. Thus, whether directly comparing awareness on the same two items, or more indirectly on the full sets of items (which differed pre- and post-), team members have a higher level of awareness at the conclusion of the pilot.

Final survey responses to individual items regarding awareness showed positive change in all areas, not just in the composite as illustrated above. Following the LSCA work, team members indicated they had a moderate level of awareness regarding information technology use in biotechnology, biotechnology lab applications in science classes , and the drug development process and high to very high awareness levels of biotechnology educational requirements, the range of biotechnology jobs and employment opportunities, opportunities for 2-year and 4-year college graduates, and life sciences industry needs for students trained in math and science .

 

The differences between areas with moderate as compared with higher levels of awareness makes sense in the context of a pilot focusing on career awareness; we might expect that respondents would become more aware of job and career opportunities than they are of the content of the work and its use in classrooms.

 

Collaboration Between Guidance Counselors and Others

Collaboration between guidance counselors and teachers existed, but at a minimal level at baseline. In June 2006 following the LSCA work, collaboration was rated again.  This time, guidance counselors and teachers from the pilot teams indicated an increase in the amount of collaboration to more than moderate levels. While collaboration increased in all individual areas, those with the most notable gains included: school to work counseling, internships in the life sciences, science mentors for students, and career or occupational advising. 

Changes in the amount and type of collaboration between guidance counselors and teachers/chairs were highlighted during the team action plan presentations at the final LSCA workshop session. Members from all four presenting teams described how few opportunities they previously had to communicate—several said this was the first time they truly collaborated.  Several guidance counselors described how they were often on the periphery, working individually with students in their offices and not integrating a career component into curricular areas.  All teams expressed pleasure at developing new and/or stronger collegial relationships and departmental connections.  And one team said that because of their collaboration around biotechnology generally and developing the action plan more specifically, they have begun to work on other curricular issues.  They felt that their team's work was having a positive impact on their school's culture.

Through the team collaboration, several boundaries were crossed and the territories of guidance and teaching were sometimes blended.  For instance, teachers on the teams became involved with practice SAT sessions, "power lunches" where students ate and conversed with biotechnology company representatives, and planning for internship programs which had generally been under the purview of their guidance colleagues.  Similarly, guidance counselors took an active role in key decision-making meetings about biotechnology course development, student-mentor tutoring programs in science, and the development of science skill logs, all of which had been the domain of science teachers. All four teams described having interdependent roles where science teachers would introduce biotechnology topics and guidance counselors would enhance student understanding through career activities.

Collaboration between guidance counselors and partners such as Connecting Activities staff, career specialists and tech prep coordinators was also examined. At baseline, team members rated this type of communication at a moderate level—slightly higher than they had rated collaboration between science teachers and guidance. In June 2006, following the LSCA workshop collaboration increased slightly to somewhat more than a moderate amount. Once again, the most notable areas of change in terms of the amount of collaboration were: school to work counseling, internships in the life sciences, science mentors for students, and career or occupational advising. 

Given these similarities, it is not surprising that further analysis yielded no significant differences in what guidance counselors and teachers talk about and what guidance counselors and external partners talk about.  Also, while there appeared to be somewhat greater change in the amount of communication between guidance and science teachers in comparison with guidance counselors and external partners, this difference was not statistically significant.  Thus, the amount of collaboration didn't depend on who was communicating or the topic of their communication.  Little additional information about collaboration between guidance and external partners can be gleaned from the observation of team presentations since only one team included a connecting activities staff member.  While it is possible that other teams relied on an external partner to support the teams in general ways, no team made this explicit in their presentations.

 

Leadership Teams for LSCA

Members of each of the four teams responding to the final survey provided information on their interaction processes. Email was an occasional mode of communication with 3 respondents saying they communicated via email “not at all,” 10 survey respondents saying they communicated and planned via email “once or twice,” and 5 respondents saying they communicated via email “a moderate amount." In general, team meetings were of limited duration, with half of the meetings lasting less than one hour.  However, 39% of meetings lasted 1.5 hours or more. There was little consistency among team members regarding their perceptions of meeting length, so no connections can be made between the amount of time working on action plans and the scope or quality of those plans. 

 

In addition to their accomplishments of building collaborative relationships and reducing barriers between the science and guidance departments, four of the six teams completed School Action Plans for Career Awareness.  Two of the four implemented a large portion of planned activities, one team implemented a few activities, and one team implemented none of their activities.  All have clear "next steps" in place for the fall of 2006 when they will resume implementation.  The following list is a sample of activities that have been planned by the teams.  Those with asterisks have already been implemented.

  • *Middle school outreach to educator and encourage enrollment in biotechnology program
  • *Transition plan for grade 8 to 9 in science
  • *Develop practice SAT sessions
  • Produce a team-taught biotech course
  • *Got training on the Real Game and tweaked it to emphasize biotech opportunities
  • *Linked students with internships
  • *Established a student mentor tutoring program in major subject areas—including science
  • *Organized and offered a Biotech Career Panel presentation to students, which included Biogen Idec, Cubist Pharm, Organon Research Center
  • *Conducted various pre and post surveys about level of student awareness of the meaning of biotechnology and biotech careers
  • *Extended power lunches to include biotechnology--rising seniors at all academic levels came to a lunchtime presentation by a representative from Middlesex CC (team met her at the 2 nd Career Awareness event) and the president of a local biotech company 
  • Scheduled and planned for a CityLab mobil visit for general biology sophomores
  • Scheduled video for after CityLab called Bioscience: Real Jobs, Real People and a post-assessment after CityLab
  • Developing ways to expose 500 freshman and sophomore biology students to biotechnology and, at the end of the course, create a capstone unit where students would apply some earlier learning via biotech labs. 
  • Creating skill logs where students will record new skills they learned and document levels of mastery.

 

 

Issues to Note

 

Across the course of the year the evaluation team noted some challenges that appear likely to affect school engagement with the program and ultimately program success. The frequency and consistency of communication between the program and its’ consultants was often inadequate for timely and complete planning, development, reflection, and revision. Mass Bio Staff, while clearly professional and very capable, were markedly challenged to be proactive in responding to evaluation data and despite their best intentions were often unable to coordinate the multiple cycles of communication necessary with so many consultants on the project. More staff would help the program “divide and conquer” and to coordinate the many components of the project at this critical phase of development.

 

A review of methods used to undertake research on human subjects was conducted by the TERC Internal Review Board prior to data collection.  All teachers participated voluntarily and provided written consent post program induction, and they were informed about the purpose and use of evaluation data and their rights. To maintain confidentiality, only evaluators and the program team had access to raw data and all individual, school/college, and district information remained anonymous to others.

 

Evaluators entered the project after the summer training workshop, and thus, had to gain participant consent retroactively. Since participants didn't know the evaluators or the goals of this program evaluation, many were understandably hesitant to participate. They hadn't allocated time for evaluation tasks and, without a stipend, they were not motivated to find time during a busy school year. As a result participation rates were lower than expected making it difficult in some instances to draw conclusions about the program.

 

BioTeach project staffing changed considerably during the 2006-2007 time period. The director and program manager both left the program in April 2007. Communication, goal-setting, design and implementation challenges that existed in this multi-faceted and complex project were exacerbated by these personnel changes, and the addition of partners. Time was needed for new staff to both understand and rectify difficulties.

 

On several items in surveys, the range of choices was not broad enough for those who had selected the higher levels. E.g., Contact with guidance counselors had a high value of 3 or more times; number of teachers to share information with had a high value of fifteen teachers.  Therefore in a few cases, high frequencies are not reported with as much precision as low frequencies which tended to be more discrete. E.g. no teachers, one teacher.

 

 

 


Recommendations

 

Data was shared with Program staff throughout the year often leading to discussion and mid course recommendations.  Thus several recommendations have already been reviewed during planning for cohort 2 (*). The program could benefit from considering the following to increase implementation and spread:

 

Communication

Clarify the message sent to all participants about required levels of participation. More consistent participation is needed from individuals to ascertain if “the program” as devised is getting to the target audience. Some effort must be made to communicate required participation to individual participants including department chairs, teachers, principals*, guidance counselors, and guidance administrators prior to the summer training.

 

Emphasize the expectation that the labs be presented to all students regardless of ability , to broaden the range of students exposed to biotechnology and to support teachers in classrooms with high proportions of low-level students. Consider admitting only teachers who teach general biology.

 

Focus on biology teachers . If the goal remains to focus on student exposure to biotechnology, summer training should continue to be focused on those who will teach biology topics in the coming academic year and/or who support biology teachers. If programmatic connections can be made to other science domains like chemistry, then expanding the training to those teachers will be productive.*

 

Training

Continue to develop the methods of lab instruction and accompanying materials so that they demonstrate firm connections to core science concepts across the general biology curriculum. *

 

Create some activities programmatically that would help spread use from trained teachers to non-trained teachers, especially in larger science departments. Examples of this could include asking department chairs to commit one hour of a staff meeting in the fall to a demonstration of the training. Offer graduate credit or PDPs for running a class post summer on an aspect of the training. At the very least, create the expectation that these teachers are expected to share what they have learned with others.

 

Consider focusing resources on those schools that have labs periods longer than 45 minutes . Review the labs to be sure the number and pacing of lab segments (e.g. wait an hour, sit overnight, etc.) are realistic for high school schedules.

 

 

 

 

Supplies

Restructure the ordering of supplies so that teachers could have materials as soon as October. *

 

Management

Support the program manager so she can devote time to collaborative planning with key consultants and act as the chief conduit between groups as they develop new activities and goals for their components. As the grant period progresses the manager will also have to monitor school compliance and “commitment” outlined in the MOU. This communication is critical to the program’s success and activities with schools should be reduced if time or additional staff cannot be found to help her manage this important job.

 

Encourage program spread at the school level by asking for a school-site coordinator for the grant. This will help schools deliver teachers to the training, guidance counselors to the effort, department chairs to support ordering appropriate supplies, and data on grant impact. This should also ease the managers’ job.

 

Career Awareness

Formal, required activities are needed to help activate and support relationships between science and guidance departments if communication and collaboration between departments across the cohort is an important goal.  Activities such as those devised for the pilot school teams may be appropriate.

 

Work with industry to develop introductory out of school field experiences in biotechnology for voc-tech students and other high school students in their first three years of high school.

 

Determine how to engage and support the two pilot schools that did not complete action plan requirements while making program expectations about level and type of participation clear.

 

 

 


[1] MassBio staff collected this pre and post workshop data, prior to hiring evaluators and seeking informed consent so the N is higher than that of the evaluation.

[2] Remember that despite an N of 24 department chairs, not all questions were answered by each chair. Oversights and other such lack of responses on individual questions accounts for the small shifts question by question in the total N.

[3] In the accompanying Chart Sixteen no teacher described using the mini prep lab so it was not included.

[4] These ten items were: 1) Educational requirements or training needed to work in the biotechnology industry; 2) Employment opportunities in biotechnology; 3) The drug development process; 4) The extent of information technology use in biotechnology; 5) The range of jobs in biotechnology industries; 6) The opportunities for graduates of 2 year colleges; 7) The opportunities for graduates of 4 year colleges; 8) Biotechnology lab applications in the science classroom; 9) Life sciences industry need for students trained in math and science; 10) A model for guidance/career development in biotechnology.