The implications of systematic training for the validity of online ability testing
Fluid intelligence is critical for a wide variety of cognitive tasks, and it is considered one of the most important factors in learning. Hence, fluid intelligence is a vital construct in aptitude testing. There is a long history of the assumption that fluid intelligence must be simply genetically determined. However, recent studies (see Jeaggi, Buschkuehl, Jonides & Perrig, 2008) have indicated that fluid intelligence can be improved with training. If training improves fluid intelligence, it needs a measure for current psychometric fitness level to determine the true potential of a candidate in aptitude testing. Cognitive training data of participants (N=12.548) of a long-term online study is used to identify cognitive aspects that are subject to improvement by training and to aptitude testing. The results indicate that taking into account the cognitive training level is a vital element of test fairness for aptitude testing in general and for online pre-screening in particular.

Adaptive personality assessment for unsupervised pre-screening
Conventional big-5 orientated personality inventories (normative and ipsative) have proven their effectiveness for applicant screening under supervised conditions. However, these types of instruments are very prone to targeted biased responses which make them less effective in unsupervised conditions. Furthermore, in order to establish sufficient reliability these instruments are very time-consuming for participants which causes problems in acceptance under various circumstances. With adalloc^TM a new method is presented which enables an adaptive personality assessment online. It is demonstrated how the adalloc^TM method allows bias-robust assessment in a fraction of the test time of conventional methods. Results from an international deployment of the adalloc^TM method for unsupervised online pre-screening of undergraduates (N=95.314) are presented along with results from in vitro studies on the effect of targeted biased responses.

Tamper-resistant aptitude pre-screening by real-time item generation
The Internet has offered new ways for deploying psychometrics for applicant pre-screening. In pre-screening settings online assessment is typically and most effectively administered in an unsupervised mode, i.e. no test administrator is present while a candidate takes the online assessment. The online and unsupervised test administration settings call for quality criterion and technical solutions that go beyond those of classical testing. The extended quality criterions and technical requirements will be explained. Implementations and evaluations of this extended standard will be shown exemplarily that are in action for graduate selection (N= 35.325), for selection of technicians (N=7.963) and for selection of apprentices (N=192.431). Re-test results (supervised on-site) show that security issues, candidate authentication and tamper attempts of unsupervised test administrations can be successfully managed by real-time item generation.

The impact of instant feedback during unsupervised online aptitude testing
New technology allows completely new methods and structures of cognitive ability tests. Unsupervised aptitude testing requires quality aspects that go beyond classical psychometric criteria. In particular the instruction sequence has to be constructed in a way that takes into account individual prerequisites of participants to grant fair testing. However, beliefs of self-efficacy have a significant impact on performance in aptitude testing. Self-efficacy beliefs are not only influenced by the feedback to example tasks in the instruction sequence but during a test by the perceived difficulties of tasks and the individual assumptions about correct and incorrect responses. In order to analyse the impact of instant feedback during unsupervised online aptitude testing participants of a broader validation study (N=602) were randomly allocated to different feedback versions of an online logical reasoning test. The results indicate a significant impact of feedback variants on performance and the reliability of the test itself.

If you are interested in more details about any of the above topice - please contact us.

The authors, Dr. Achim Preuß und Maike Wehrmaker, will be attending the conference in person and would love to see you there.