This work began because in the teaching department in which Greg lectures (at a University of Technology), they needed to have some way of predicting whether or not a student had the potential for a career in Mechanical Engineering. The use of existing tests was subject to a contract and royalty payments, so Greg started from the beginning and developed his own bank of tests. He did not believe that conventional tests of ability in mathematics and physics were sufficient to predict engineering aptitude.

He began by listing all the separate types of thinking that might need to be measured, with a view to applying the test and establishing whether or not any of the sub-tests was predictive of engineering performance. For five semesters they correlated each student’s result for each test with that student’s mark for the subject ‘Engineering Mechanics 1’.

Greg's initial bank of tests included all of the following:

  1. Knowledge of mechanical consequences (part of this test is reproduced below, as an example)
  2. Freehand drawing dexterity
  3. Mental arithmetic
  4. Ability to define the meaning of given common words
  5. Ability to describe the steps in getting a car started
  6. Ability to estimate angles drawn on paper
  7. Ability to estimate lengths of lines drawn on paper as well as of real objects
  8. Manual dexterity in hammering panel pins into wood and marking out and cutting a hexagon from thick cardboard
  9. Ability to recognize and name common mechanical objects presented to candidates
  10. Ability to recognize and name the materials of which common objects are made
  11. Ability to estimate the mass of given objects that can be lifted by the candidate
  12. Ability to estimate the volumes of each a set of varied containers of different sizes
  13. Short term memory applying to numbers, words, pictures and items displayed on a tray
  14. Dexterity when drawing with a pencil and ruler
  15. Knowledge of engineering history, processes, products and companies
  16. Ability to classify given sets of words, e.g. ‘fish peas, peanuts, beer’: these are ‘foodstuffs’
  17. Ability to de-code simple codes
  18. Vocabulary applying to geometry
  19. Vocabulary applying to physics
  20. Common sense evaluation of the logistics of undertaking simple projects, and estimation of the time required for certain tasks
  21. Use of prepositions
  22. Number of books read under different headings, in the candidate’s life (not a test, but a self-report)
  23. Types and frequency of the use of tools in the candidate’s own experience (also a self-report)
  24. Mathematical thinking applied to conventional mathematical problems as encountered in high school
  25. Knowledge of common mathematical formulae, e.g. the area of a circle, the volume of a sphere
  26. Knowledge of some of the basic properties of materials, e.g. given a list of five materials, name the hardest one among them
  27. A survey of the depth of interest that the candidate showed for a variety of subjects (also a self-report). The idea here was to see if the candidate expressed an interest in topics related to engineering, when these topics were imbedded in a long list of many other types of topic, covering, for example, sport, the arts, entertainment and politics.

In due course, all the tests were tested for their predictive value, by correlating test scores with the candidates’ subsequent grades in the subject we call ‘Engineering Mechanics 1’. Nearly all of the tests in the bank showed a positive correlation with the subsequent grades achieved by the students. Only four of the tests showed virtually no correlation: these were those that investigated the amount and type of reading that the candidates had done in their lives, and the number and range of topics that the candidates claimed to be interested in.

For the remaining 24 tests, the correlation coefficients ranged from +0.27 to + 0.48. Of itself, this would not appear to be a strong enough correlation to predict the suitability of any individual. More reliability was obtained later, by taking an aggregate score of the 14 most predictive tests, and the correlation of this aggregate with the mark obtained in ‘Engineering Mechanics 1’ was found to be +0.61. At this level of correlation, one could definitely say that the better the score on the entrance test, the more likely a candidate was to do well in Mechanics 1. However, at this time the level of predictability was still not high enough to be able to claim that any given candidate would or would not make it in Mechanics 1. In particular, there was the problem that a significant number of students who scored high on this bank of tests actually failed the subject. The researchers realized that one cannot say that potential will always be realized: there are always capable students who put in too little effort, for a variety of reasons.

The merit of the test became apparent after it had been in use for a few successive semesters, when it was observed that candidates who had scored less than 28% overall for the bank of tests had not managed to pass Mechanics 1, even on their third attempt. We concluded that at the very least, this empirical measure gave us a lower cut-off point. Candidates scoring 29% or more were found to pass the subject on their second or third attempt. This result gave us confidence that we could identify candidates worthy of giving extra attention, who might eventually benefit from extended training. And we could predict with virtually complete certainty that anyone scoring less than the empirical 28% would not be a suitable candidate for studying engineering.

After considerable experience, we narrowed down the size of the bank of tests undertaken by candidates to 12, as a practical necessity, for the sake of logistics.

If anyone is interested in the development of this research project, or in using the tests, they are welcome to e-mail the author at greg@gregorypastoll.co.za


TEST OF MECHANICAL CONSEQUENCES © Gregory Pastoll 2005
(This is an extract from one of a bank of approx 30 different tests)

Circle the appropriate letter in each case, e.g. a b c d e

  1. Two pendulums: each has the same mass, but the string lengths are different. Which one will swing faster?

    a..... b..... c..... (both the same)



  2. A drum with two sections of different diameters is free to turn on an axle. Attached to the drum are ropes woundin opposite directions. One person pulls on the rope at ‘a ‘, and another at ‘b ‘.Which one will find it easier to pull?

    a..... b..... c..... (no difference)



  3. A string is fastened to a solid beam at ‘a’.
    A heavy collar is allowed to fall from ‘a ‘ to ‘b ‘,
    where it strikes a bar tied to the string.

    If the string breaks, where is it most likely to break?



    a..... b..... c..... (somewhere in the middle)

  4. You have five small stones. All five are approximately round and have the same density.

    Stone a b c d e
    Diameter(mm) 10 20 30 40 50
    Mass(g) 1.5 12 40 94 185

    Which one could you throw the furthest? a..... b..... c..... d..... e

    Which one could you throw the least far? a..... b..... c..... d..... e
 
 
 

© Gregory Pastoll, 2010

All Rights Reserved. All Intellectual Property is the property of Gregory Pastoll.