Journal of Research and Educational Research Evaluation

Validity and Reliability Analysis of Reading Comprehension Skill Test Using Islamic English Texts

Asa Bani Putria — 2023-08-30

One of the requirements for graduation in Islamic universities is the ability to master English. Students encounter obstacles in fulfilling these requirements, especially non-English study program students. This study aims to prove the validity and estimate the reliability of the reading comprehension skills test instrument using Islamic English texts. The method used in this study was the research and development method proposed by Mardapi. Proof of content validity used the Aiken's V formula while estimation of reliability between raters used a two-way ANOVA test which was then calculated using Hoyt's formula. The research data shows that the Aiken V coefficient value for each statement item is more than 0,3. It can be said that all aspects are valid. The estimated reliability of the two-way ANOVA test yields a value of 0,94. It can be said that this instrument is quite reliable. Thus, it can be concluded that the test instrument for reading comprehension skills of Islamic English texts has high validity and reliability among raters is quite strong and is expected to be used as a reference for the test instrument makers.

Bibliometric Analysis: Research Trends in the Development of the TPACK Instrument in the 2019–2023 Period

Muhlis Muhlis — 2023-08-30

Kajian ini bertujuan untuk mengkaji perkembangan penelitian pengembangan instrumen penilaian TPACK dalam 5 tahun terakhir, dari tahun 2019 hingga 2023, dengan kajian bibliometrik. Kajian analisis bibliometrik dalam penelitian ini mengumpulkan 120 dokumen artikel dari database Scopus pada topik TPACK, yang kemudian dianalisis menjadi 59 dokumen artikel. Proses analisis data menggunakan software Mendeley dan Vosviewer. Hasil analisis dokumen menunjukkan bahwa 1) Tingkat sitasi tertinggi terdapat pada artikel yang ditulis oleh Habibi, dkk dengan jumlah sitasi 84 kali dengan rasio 28,00 per tahun; 2) Sumber jurnal “Journal of Physics: Conference Series” tempat terbit terbanyak untuk artikel dengan total 10 dokumen artikel; 3) Ada dua jenis artikel yang diterbitkan,yaitu artikel jurnal dan artikel prosiding; 4) Jenis instrumen yang dikembangkan meliputi 86% berupa instrumen nontes dan 14% berupa instrumen tes; dan 5) Tren publikasi dengan melihat Network Visualization, Overlay Visualization, Density Visualization hingga perkembangan publikasi artikel per tahun dimana tahun 2020 merupakan tahun yang banyak menerbitkan dokumen artikel. Temuan dari analisis bibliometrik dapat membantu peneliti dalam meninjau kemajuan beberapa instrumen pengembangan studi TPACK sebelumnya untuk dijadikan referensi dan menciptakan kebaruan dalam studi selanjutnya. Kepadatan Visualisasi terhadap perkembangan publikasi artikel per tahun dimana tahun 2020 merupakan tahun yang banyak menerbitkan dokumen artikel.Temuan dari analisis bibliometrik dapat membantu peneliti dalam meninjau kemajuan beberapa instrumen pengembangan studi TPACK sebelumnya untuk dijadikan referensi dan menciptakan kebaruan dalam studi selanjutnya. Kepadatan Visualisasi terhadap perkembangan publikasi artikel per tahun dimana tahun 2020 merupakan tahun yang banyak menerbitkan dokumen artikel. Temuan dari analisis bibliometrik dapat membantu peneliti dalam meninjau kemajuan beberapa instrumen pengembangan studi TPACK sebelumnya untuk dijadikan referensi dan menciptakan kebaruan dalam studi selanjutnya.

Mr. Private University Barriers to World-Class Education: The Case of Indonesia

Ferdi Widiputera — 2023-08-30

This study provides a comprehensive analysis of the barriers that prevent private universities from achieving world-class status. Six private universities with national A, B, and C accreditations were examined using qualitative methods including interviews, focus groups, and university documents, as well as the Times Higher Education (THE) assessment criteria in 2021. The research highlights a stark contrast between private universities in developed countries, which have more flexibility in achieving World-Class University (WCU) status, and those in Indonesia, which face significant challenges. Obstacles include the youth of the institutions, limited student enrollment, faculty lacking doctoral degrees, inadequate adoption of international teaching methods, insufficient research output, and weak research linkages. There's also a lack of language skills among faculty, a shortage of international staff, and limited collaboration with recognized institutions. The paper recommends that private universities improve the quality of education with government support through incentives, funding for domestic and international doctoral training, modern research facilities, and other relevant initiatives.

The Application of Aiken's V Method for Evaluating the Content Validity of Instruments that Measure the Implementation of Formative Assessments

Siti Nurjanah — 2023-08-30

This study aims to analyze the content validity of the measuring instrument used for formative assessment implementation by utilizing the Aiken method. The selection of formative assessment was based on the consensus within experts that it offers significant enhancements to the progression of classroom learning. Nevertheless, there is currently no valid instrument available to measure it. The instrument under study encompasses three key dimensions: perceptions, practices, and challenges. The instrument consists of a closed-ended questionnaire with 37 items and an open-ended questionnaire with 2 items. The content validity was evaluated by three raters who are highly experienced in their respective disciplines. After doing a quantitative study of the Aiken's V value, it was determined that all items of the instrument for implementing formative assessment were deemed valid. According to the qualitative analysis, raters provided some modest revisions for certain items. Overall, the instrument used to assess the implementation of formative assessment is deemed valid through content validity analysis relying on Aiken's method. Instructors, researchers, and policy makers can utilize the findings of this study to obtain information regarding the implementation of formative assessment through the utilization of the measuring instrument employed in this study.

Development of An Evaluation Instrument Model for The Counseling Guidance Service Program for Student Discipline

Muhammad Imanul Adam — 2023-08-30

This research aimed to develop disciplinary instruments and student rules using a questionnaire. To achieve this objective, a Research and Development was employed by using an assessment instrument in the form of a questionnaire (non-test). As the subject of this research, small and large scales with 250 students were used by ten development steps. Validation and reliability were simplified into three stages: preliminary, development, and evaluation. The preliminary stage involved determining instrument specifications, writing the instrument, and determining scoring. The development stage involved counseling guidance, trialing, analysis, and revision. The evaluation stage involved assembling and refining the instrument. Validity analysis and content reliability were performed using Aiken's V formula and Crombach's Alpha. Field tests showed that all items on the student discipline and rules instrument had a value of > 0.3, indicating validity and reliability. Whereas if the reliability test is >0.7, the instrument is valid and reliable, so the instrument is suitable for use. The results were in the valid and reliable category with a Likert scale of 4,3, and 2, indicating good rules and discipline among SMA AZ Semarang City students. However, efforts are needed to improve violations of rules and discipline, even if they are minor.

Content Validity and Inter-Rater Reliability in Developing Minimum Competency Assessment Test Instruments (AKM)

Zalika Umami — 2023-08-30

This research aims to describe the results of validation by experts regarding the development of AKM test instruments in class XI geography subjects. This research is development research (R&D) with the research model used, namely the 4-D model (design, define, develop, disseminate). The product in this research is the AKM test instrument in class XI Geography. This expert validation involved 4 validators including 1 evaluation expert lecturer, 1 geography education expert lecturer, and 2 geography subject expert teachers. The tests carried out by experts are the AKM test instrument assessment test and the validity and reliability test results of the AKM test instrument. The results of this research are the AKM test instrument with expert assessment results based on language of 87% which is in the very good category, 87% of the construction aspect is in the very good category and 89% of the material aspect is in the very good category. Furthermore, the validity assessment by experts related to the development of the AKM test instrument obtained a v'Aikens index test result of 0.8125 which was included in the very valid criteria. The results of the reliability test showed that Avage Measures was 0.686, which is included in the ICC criteria for good reliability. In this way, the overall development of the AKM test instrument is valid and can also be used by educators as a reference in developing the AKM test instrument to support students' abilities.