Embedding opportunities for participation and feedback in large mathematics lectures via audience response systems

Christopher C. Tisdell

doi:10.3934/steme.2021006

Article Contents

2021, Volume 1, Issue 2: 75-91. Doi: 10.3934/steme.2021006 open access

This issue Previous Article Next Article Good practices of delivery and teaching leadership for online educators in technical disciplines: A perspective

Embedding opportunities for participation and feedback in large mathematics lectures via audience response systems

Christopher C. Tisdell^1,2, ,

1.
School of Mathematics and Statistics, The University of New South Wales, Sydney NSW 2052, Australia
2.
Institute for Teaching and Learning Innovation (ITaLI), University of Queensland, Brisbane Qld 4072, Australia

* Correspondence: cct@unsw.edu.au; Tel: +61-2-93856792
* Correspondence: cct@unsw.edu.au; Tel: +61-2-93856792

Received: February 28, 2021

Revised: March 31, 2021

Published: May 2021

Abstract Full Text(HTML) Figure(6) / Table(11) Related Papers Cited by

Abstract

The purpose of this work is to interpret the experiences of students when audience response systems (ARS) were implemented as a strategy for teaching large mathematics lecture groups at university. Our paper makes several contributions to the literature. Firstly, we furnish a basic model of how ARS can form a teaching and learning strategy. Secondly, we examine the impact of this strategy on student attitudes of their experiences, focusing on the ability of ARS to: assess understanding; identify strengths and weaknesses; furnish feedback; support learning; and to encourage participation. Our findings support the position that there is a place for ARS as part of a strategy for teaching and learning mathematics in large groups.

Keywords:

Mathematics Subject Classification: Research article.

Citation:

FullText(HTML)

Figure 1. Sample Question from Quiz: Q1ii)

Download: Full-size image PowerPoint slide

Figure 2. Sample Question from Quiz: Q2i)

Download: Full-size image PowerPoint slide

Figure 3. Sample Question from Quiz: Q2ii)

Download: Full-size image PowerPoint slide

Figure 4. Class Responses to Question 1ii) of Quiz

Download: Full-size image PowerPoint slide

Figure 5. Class Responses to Question 2i) of Quiz

Download: Full-size image PowerPoint slide

Figure 6. Class Responses to Question 2ii) of Quiz

Download: Full-size image PowerPoint slide

Table 1. Groups of Interest

Group	Details
Target:	Undergraduate students in large mathematics classes
Sample:	Students in Lecture Group 1 of MATH1131 during the algebra lectures where the intervention took place
Comparison:	Students in Lecture Group 1 of MATH1131 during the calculus lectures where no intervention took place

| Show Table

DownLoad: CSV

Table 2. Themes and Relevant Components of Audience Response Systems

Theme	Relevant Components
ARS:	Lecture (re)design and (re)deliveryEmbedding of discussion, assessment and feedback
Digital Education:	Use of mobile devices (laptops, phones, tabletsCreation of YouTube videos
Open Educational Resources:	Use of Google Forms

| Show Table

DownLoad: CSV

Table 3. Summary of Intervention: Framework for Delivery

Activity	Technology
Deliver material at start of lecture	None necessarily required
Assess material via short, formative quiz	Accessed via Google Forms / responses via m-devices on wifi or network
Feedback to class on the results of quiz	Discuss results via graphs from Google Forms
Discussion and thoughts on how to improve (if needed)	None necessarily required

| Show Table

DownLoad: CSV

Table 4. Evaluation Overview

Evaluation Approach	Timing/Sample/Analysis	Evaluation Focus
Attitude Data 1: (Sample Group)	Post intervention. Attitudinal data collected from bespoke survey of 348 students within component of intervention (algebra lectures). Six-point Likert scale employed; mean values calculated, including 95% confidence intervals. Comments collected and coded.	Impact on students' attitudes towards their learning experience.
Attitude Data 2: (Sample Group)	3 weeks after Survey 1. Attitudinal data collected from survey of ~180 students within component of intervention (algebra lectures). This is a subset of the 348 students from previous survey. Six-point Likert scale employed; mean values calculated, including 95% confidence intervals. Comments collected and coded. Sample Group and Control Group compared via statistical tests.	Impact on students' attitudes towards their learning experience.
Attitude Data 3: (Control Group)	3 weeks after Survey 1. Attitudinal data collected from survey of 102 students within component where no intervention took place (calculus lectures). This is a subset of the 348 students from previous survey. Six-point Likert scale employed; mean values calculated, including 95% confidence intervals. Sample Group and Control Group compared via statistical tests.	Impact on students' attitudes towards their learning experience.

| Show Table

DownLoad: CSV

Table 5. Statements in Survey 1 (A-F) and Survey 2 (G-H)

Item	Statement
A	The quizzes provided a valuable opportunity to test my understanding of basic ideas
B	The quizzes helped to identify specific strengths and weaknesses of my understanding
C	It was valuable to have immediate feedback and discussion of the results
D	The quizzes encouraged and supported my learning
E	Overall, I was satisfied that these quizzes were a valuable learning tool
F	I would like to have these kinds of quizzes available to support my learning in future lectures
G	This lecturer provided feedback to help me learn
H	This lecturer encouraged student input and participation during classes

| Show Table

DownLoad: CSV

Table 6. Responses of Sample Group to Survey 1 and 2

Statement	Strongly Disagree	Disagree	MildlyDisagree	Mildly Agree	Agree	StronglyAgree	n
A	3	1	1	16	149	178	348
B	3	2	7	44	157	135	348
C	3	0	2	18	115	210	348
D	3	2	1	46	165	131	348
E	3	2	1	24	163	155	348
F	3	0	1	15	136	193	348
G	1	4	2	24	66	81	178
H	0	1	1	4	44	131	181

| Show Table

DownLoad: CSV

Table 7. Responses of Control Group to Survey 1 and 2

Statement	Strongly Disagree	Disagree	MildlyDisagree	Mildly Agree	Agree	StronglyAgree	n
G	2	10	17	37	26	10	102
H	5	11	22	37	19	9	103

| Show Table

DownLoad: CSV

Table 8. Analysis of Responses of Sample Group to Survey 1 and 2

Statement	Mean Score/ 6	ConfidenceInterval 95%	% OverallAgree*	StandardDeviation ofMean	n
A	5.42	±0.08	99	0.75	348
B	5.17	±0.09	97	0.82	348
C	5.51	±0.08	99	0.75	348
D	5.19	±0.09	98	0.82	348
E	5.31	±0.08	98	0.82	348
F	5.47	±0.08	99	0.72	348
G	5.21	±0.14	96	0.94	178
H	5.66	±0.10	99	0.70	181
*Overall Agreement is defined as those responses of: Mildy Agree; Agree; or Strongly Agree.

| Show Table

DownLoad: CSV

Table 9. Analysis of Responses of Control Group to Survey 2

Statement	Mean Score/ 6	ConfidenceInterval 95%	% OverallAgree*	StandardDeviation ofMean	n
G	4.03	±0.23	72	1.18	102
H	3.79	±0.24	63	1.25	104
*Overall Agreement is defined as those responses of: Mildy Agree; Agree; or Strongly Agree.

| Show Table

DownLoad: CSV

Table 10. Themed Comments from Sample Group in Survey 1

Theme	Number
Efficacy	32
Appreciation	29
Constructive Suggestions	16

| Show Table

DownLoad: CSV

Table 11. Significance Tests Between Sample Group and Control Group to Survey 2

Statement	Student's t-testp < 0.05?	Mann-Whitney U-test p < 0.05?	Effect Size (Cohen's d)
G	Yes	Yes	1.10
H	Yes	Yes	1.84

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	I.E. Allen and C.A. Seaman, Likert scales and data analyses, Quality Progress, 40 (2007), p. 64-65.
[2]	Archer, M.S., Bhaskar, R., Collier, A., Lawson, T., Norrie, A. Critical Realism: Essential Readings. 2009, London, UK: Routledge.
[3]	Bagley, S.F., Improving student success in calculus: a comparison of four college calculus classes[dissertation]. 2014, San Diego State University: San Diego, USA.
[4]	Baker, J.W., The 'classroom flip': using web course management tools to become the guide by the side, in Selected Papers from the 11th International Conference on College Teaching and Learning. 2001, Florida Community College at Jacksonville: Jacksonville (FL), p. 9–17.
[5]	Banks, D.A., Audience Response Systems in Higher Education: Applications and Cases. 2006, Hershey, PA, USA: Information Science Publishing.
[6]	Berends, M., Survey methods in educational research, in Handbook of Complementary Methods in Education Research, J.L. Green, G. Camilli, P.B. Elmore, Ed. 2006, Lawrence Erlbaum Associates. p. 623-640. Retrieved from http://psycnet.apa.org/record/2006-05382-038
[7]	Bligh, D.A., What's the Use of Lectures? 1972, Harmondsworth, UK: Penguin Books.
[8]	Bonwell, C.C., Eison, J.A., Active Learning: Creating Excitement in the Classroom. 2005, San Francisco, USA: Jossey-Bass.
[9]	Box, G.E., Robustness in the Strategy of Scientific Model Building. 1979, Ft. Belvoir: Defense Technical Information Center. Retrieved from http://www.dtic.mil/docs/citations/ADA070213
[10]	S. Chen, S.J. Yang and C. Hsiao, Exploring student perceptions, learning outcome and gender differences in a flipped mathematics course, British Journal of Educational Technology, 47 (2016), p. 1096-1112. doi: 10.1111/bjet.12278.
[11]	Codecogs. Retrieved from https://www.codecogs.com/latex/eqneditor.php?latex=D
[12]	Coe, R., Waring, M., Hedges, L.V., Arthur, J., Research Methods and Methodologies in Education. 2017, Los Angeles, CA: SAGE.
[13]	Cohen, J., Statistical Power Analysis for the Behavioral Sciences. 1988, Abingdon-on-Thames, UK: Routledge.
[14]	J. Cohen, Things I have learned (so far), Am Psychol, (1990), p. 1304-1312.
[15]	Cohen, L., Manion, L., Morrison, K., Research Methods in Education. 2018, London: Routledge.
[16]	Creswell, J.W., Qualitative Inquiry and Research Design: Choosing among Five Approaches. 2007, Thousand Oaks, CA: Sage.
[17]	Cronhjort, M., Filipsson, L., Weurlander, M., Improved engagement and learning in flipped-classroom calculus. Teaching Mathematics and its Applications: An International Journal of the IMA, 2018. 37(3): p. 113–121. doi: 10.1093/teamat/hrx007.
[18]	Day A.L., Case study research, in Research Methods & Methodologies in Education, 2nd ed. R. Coe, M. Waring, L.V. Hedges, J. Arthur, Ed. 2017, Los Angeles, CA: SAGE. p. 114-121.
[19]	Duncan, D., Clickers in the Classroom: How to Enhance Science Teaching Using Classroom Response Systems. 2005, San Francisco, CA: Pearson Education.
[20]	P.K. Dunn, A. Richardson, C. McDonald and F. Oprescu, Instructor perceptions of using a mobile-phone-based free classroom response system in first-year statistics undergraduate courses, International Journal of Mathematical Education in Science and Technology, 43 (2012), p. 1041-1056. doi: 10.1080/0020739x.2012.678896.
[21]	P.K. Dunn, A. Richardson, F. Oprescu and C. McDonald, Mobile-phone-based classroom response systems: Students' perceptions of engagement and learning in a large undergraduate course, International Journal of Mathematical Education in Science and Technology, 44 (2013), p. 1160-1174. doi: 10.1080/0020739x.2012.756548.
[22]	Google, Create forms. 2017. Retrieved from https://www.google.com.au/forms/about/
[23]	Higher Education Research & Development Society of Australasia, HERDSA Fellowship Scheme Handbook. 2014, Milperra, NSW: HERSDSA. Retrieved from https://www.herdsa.org.au/sites/default/files/Fellowship\%20Handbook_6_5_2014.pdf
[24]	B.M. Johnston, Implementing a flipped classroom approach in a university numerical methods mathematics course, International Journal of Mathematical Education in Science and Technology, 48 (2017), p. 485-498. doi: 10.1080/0020739x.2016.1259516.
[25]	V. Jungic, H. Kaur, J. Mulholland and C. Xin, On flipping the classroom in large first year calculus courses, International Journal of Mathematical Education in Science and Technology, 46 (2015), p. 508-520. doi: 10.1080/0020739x.2014.990529.
[26]	S.O. King and C.L. Robinson, 'Pretty lights' and maths! Increasing student engagement and enhancing learning through the use of electronic voting systems, Computers & Education, 53 (2009), p. 189-199. doi: 10.1016/j.compedu.2009.01.012.
[27]	Kline, R.B., Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research. 2004, Washington, DC: American Psychological Association, p. 95.
[28]	K. Larkin and N. Calder, Mathematics education and mobile technologies, Math Ed Res J, 28 (2016), p. 1-7. doi: 10.1007/s13394-015-0167-6.
[29]	Lomen, D.O., Robinson, M.K., Using ConcepTests in single and multivariable calculus, in Electronic Proceedings of the Sixteenth Annual International Conference on Technology in Collegiate Mathematics. 2005. Retrieved October 17, 2017 from http://archives.math.utk.edu/ICTCM/i/16/S107.html
[30]	B. Love, A. Hodge, N. Grandgenett and A.W. Swift, Student learning and perceptions in a flipped linear algebra course, International Journal of Mathematical Education in Science and Technology, 45 (2014), p. 317-324. doi: 10.1080/0020739x.2013.822582.
[31]	W. Maciejewski, Flipping the calculus classroom: an evaluative study, Teaching Mathematics and its Applications, 35 (2016), p. 187-201. doi: 10.1093/teamat/hrv019.
[32]	Markie, P., Rationalism vs. Empiricism. 2017. Retrieved from https://plato.stanford.edu/entries/rationalism-empiricism/
[33]	J. Murphy, J. Chang and K. Suaray, Student performance and attitudes in a collaborative and flipped linear algebra course, International Journal of Mathematical Education in Science and Technology, 47 (2015), p. 653-673. doi: 10.1080/0020739x.2015.1102979.
[34]	E. Naccarato and G. Karakok, Expectations and implementations of the flipped classroom model in undergraduate mathematics courses, International Journal of Mathematical Education in Science and Technology, 46 (2015), p. 968-978. doi: 10.1080/0020739x.2015.1071440.
[35]	von Neumann, J., The mathematician, in Works of the Mind, R.B., Haywood Ed. 1947, Chicago: University of Chicago Press. p. 180-196.
[36]	Novak, J., Kensington-Miller, B., Evans, T., Flip or flop? Students' perspectives of a flipped lecture in mathematics. International Journal of Mathematical Education in Science and Technology, 2017. 48(5): p. 647-658. doi: 10.1080/0020739x.2016.1267810.
[37]	Center for Educational Research and Innovation, Giving Knowledge for Free: The Emergence of Open Educational Resources. Retrieved from http://www.oecd.org/edu/ceri/38654317.pdf
[38]	J. Petrillo, On flipping first-semester calculus: a case study, International Journal of Mathematical Education in Science and Technology, 47 (2016), p. 573-582. doi: 10.1080/0020739x.2015.1106014.
[39]	Robson, L., Guide to Evaluating the Effectiveness of Strategies for Preventing Work Injuries: How to Show Whether a Safety Intervention Really Works. 2001, Cincinnati, OH: DHHS.
[40]	R. Salzer, Smartphones as audience response systems for lectures and seminars, Analytical and Bioanalytical Chemistry, 410 (2018), p. 1609-1613. doi: 10.1007/s00216-017-0794-8.
[41]	S. Sawilowsky, New effect size rules of thumb, Journal of Modern Applied Statistical Methods, 8 (2009), p. 467-474. doi: 10.22237/jmasm/1257035100.
[42]	Shadish, W.R., Cook, T.D., Campbell, D.T., Experimental and Quasi-experimental Designs for Generalized Causal Inference. 2001, Belmont, CA: Wadsworth Cengage Learning.
[43]	G. M. Sullivan and Jr. A.R. Artino, Analyzing and interpreting data from Likert-type scales, Journal of Graduate Medical Education, 5 (2013), p. 541-542.
[44]	C.C. Tisdell, Critical perspectives of pedagogical approaches to reversing the order of integration in double integrals, International Journal of Mathematical Education in Science and Technology, 48 (2017), p. 1285-1292. doi: 10.1080/0020739X.2017.1329559.
[45]	C.C. Tisdell, Pedagogical alternatives for triple integrals: moving towards more inclusive and personalized learning, International Journal of Mathematical Education in Science and Technology, 49 (2018), p. 792-801. doi: 10.1080/0020739X.2017.1408150.
[46]	C.C. Tisdell, On Picard's iteration method to solve differential equations and a pedagogical space for otherness, International Journal of Mathematical Education in Science and Technology, 50 (2019), p. 788-799. doi: 10.1080/0020739X.2018.1507051.
[47]	C.C. Tisdell, Schoenfeld's problem-solving models viewed through the lens of exemplification, For the Learning of Mathematics, 39 (2019), p. 24-26.
[48]	Trochim, W.M.K., The Research Methods Knowledge Base. 2006. Retrieved from https://www.socialresearchmethods.net/kb/positvsm.php
[49]	University of Edinburgh. What is digital education? 2019. Retrieved from https://www.ed.ac.uk/institute-academic-development/learning-teaching/staff/digital-ed/what-is-digital-education
[50]	Wang, V.C., Handbook of Research on e-learning Applications for Career and Technical Education: Technologies for Vocational Training. 2009, Hershey, PA: IGI Global.
[51]	Wasserman, N., Norris, S., Carr, T., Comparing a "flipped" instructional model in an undergraduate Calculus Ⅲ course, in Proceedings of the 16th Annual Conference on Research in Undergraduate Mathematics Education; S. Brown, G. Karakok, K.H. Roh, M. Oehrtman Ed..2013, Denver, CO.
[52]	Yin, R.K., Case Study Research: Design and Methods. 4th ed. 2009, Thousand Oaks, CA: Sage.