Question Generation Capabilities of “Small" Large Language Models
- verfasst von
- Joshua Berger, Jonathan Koß, Markos Stamatakis, Anett Hoppe, Ralph Ewerth, Christian Wartena
- Abstract
Questions are an integral part of test formats in education. Also online learning platforms like Coursera or Udemy use questions to check learners’ understanding. However, the manual creation of questions can be very time-intensive. This problem can be mitigated through automatic question generation. In this paper, we present a comparison of fine-tuned text-generating transformers for question generation. Our methods include (i) a comparison of multiple fine-tuned transformers to identify differences in the generated output, (ii) a comparison of multiple token search strategies evaluated on each model to find differences in generated questions across different strategies and (iii) a newly developed manual evaluation metric that evaluates generated questions regarding aspects of naturalness and suitability. Our experiments show a difference in question length, structure and quality depending on the used transformer architecture, which indicates a correlation between transformer architecture and question structure. Furthermore, different search strategies for the same model architecture do not greatly impact structure or quality.
- Organisationseinheit(en)
-
Forschungszentrum L3S
- Externe Organisation(en)
-
Hochschule Hannover (HsH)
Technische Informationsbibliothek (TIB) Leibniz-Informationszentrum Technik und Naturwissenschaften und Universitätsbibliothek
- Typ
- Aufsatz in Konferenzband
- Seiten
- 183-194
- Anzahl der Seiten
- 12
- Publikationsdatum
- 20.09.2024
- Publikationsstatus
- Veröffentlicht
- Peer-reviewed
- Ja
- ASJC Scopus Sachgebiete
- Theoretische Informatik, Allgemeine Computerwissenschaft
- Elektronische Version(en)
-
https://doi.org/10.1007/978-3-031-70242-6_18 (Zugang:
Geschlossen)
