Auckland OLY1 chat - archive

[greenglacier]

This study (from the Auckland med school) might have something to do with it:

https://www.springerlink.com/content/h16121u1p0k58411/

Fhttps://www.springerlink.com/content/h16121u1p0k58411/rom the abstract: Prior academic achievement remains the best measure of subsequent student achievement within a medical programme. Interview scores have little predictive value. Future directions include longer term studies of what UMAT predicts, and of novel ways to combine selection tools to achieve the optimum student cohort.

 

[frootloop]

Have they given any clues as to what the selection criteria will be if the interview goes? A grades:UMAT ratio of 67:33 like Otago, or more UMAT weighted like Australia? I'd imagine that, without the interview, you'd end up having to use percentage marks, rather than the 0-9 GPA scale to be able to distinguish between candidates.

 

[4everAlone]

Have they given any clues as to what the selection criteria will be if the interview goes? A grades:UMAT ratio of 67:33 like Otago, or more UMAT weighted like Australia? I'd imagine that, without the interview, you'd end up having to use percentage marks, rather than the 0-9 GPA scale to be able to distinguish between candidates.

They haven't. All we've been told is that they were going to remove it, there was massive outcry from student representatives at the meeting, enough to get them to delay making any decision. This was a few months ago and we haven't heard anything since.

I'm interested to read that article greenglacier and will try to find a free copy online. As I've said before, I'm hugely against the removal of the interview here, but that's for obviously biased reasons

 

[greenglacier]

I had a quick read of that above study (very recently published) a couple of days ago. The discussion is rather long but basically as I understand it, the main author (who, although from the Auckland med school, is but one person there) reckons that, based on the available data (and he keeps referring to a paper covering both Auckland and Otago that is due for publication very soon - so we need to wait for that before drawing any conclusions), prior academic performance performs moderately well at predicting success in med school, the interview is useless (because it fails to predict success in a composite grade at 4th year that accounts for performance in written exams, clinical exams, communication skills tests, and consultant grading based on observations of students on placement), and UMAT may or may not be useless as further investigation (and possible refining of the test) needs to be done. He seemed to be suggesting a system that either treated UMAT and/or the interview as thresholds (then ranked solely on grades, with special categories to ensure equal access), or one where everyone was ranked solely on grades, then UMAT used to further inform the selection decision for those near the cutoff.

The point is, the article seems quite negative about the interview, and given that it comes from Auckland, this may account for Auckland looking at dumping the interview.

 

[Matt]

You always have to be a little bit suspicious about universities dumping interviews because they're very costly and there's a bit of an ulterior motive there.

The next thing you have to ask is how good is a medical school grade (composite or otherwise) as an outcome predictor. Now, I think these studies are very useful to have and I'm glad more medical schools including my own are doing them, but I don't think a composite grade is the right outcome for an interview. I think written exams are a good outcome for OLY1 grades (although, intuitively it'd make sense that they are predictive). I'll have to read this article though, it's a shame we didn't pick up on it earlier, I think it's in MSO's interest to be aware of and pick apart these studies.

 

[juna]

If a decision is made, either to remove interviews, or to keep it, the university should let us know before we accept our place in health sciences or biomed. Because some people will, like me, have chosen Auckland based on the fact that UMAT has relatively low weighting in Auckland, and is confident in interviews. I'm just frustrated that the university hasn't said anything although it's already September. The lady that I spoke to in Student Center said that those concerned will find out throughout the year, but I think that will be highly unfair...

 

[juna]

I actually emailed the Dean about this issue, and I will share what he says once he emails be back (that is if he emails back).

 

[frootloop]

If a decision is made, either to remove interviews, or to keep it, the university should let us know before we accept our place in health sciences or biomed. Because some people will, like me, have chosen Auckland based on the fact that UMAT has relatively low weighting in Auckland, and is confident in interviews. I'm just frustrated that the university hasn't said anything although it's already September. The lady that I spoke to in Student Center said that those concerned will find out throughout the year, but I think that will be highly unfair...

How will it make a difference? Your only other option in NZ is Otago, which does not use interviews, therefore if Auckland told you right now they were going to drop the interview, both schools become the same in terms of getting in. How would that be any different than if they told you in the middle of next year? And remember that although you may feel that dropping the interview would disadvantage you personally, there are others who it would advantage, so it balances. The school has like 3 or 4 applicants for every spot in undergrad med, they're spoilt for choice, therefore they're the ones who make the rules. That's just how it goes...

 

[Matt]

On reading through that study more thoroughly I really feel like it is a very poor measure of how the interview performs as a selection tool and that I would be wondering as to the motives of Auckland were they to dispense with it.

 

[juna]

I can only speak for myself, but the main reason that I chose Auckland over Otago, despite half of my family being there, is because of the interview, and less weighting on UMAT. So I will be really really annoyed if I find out half way throughout the year that they dropped interviews because obviously then, I would have gone to Otago. So if "Auckland told me right now they were going to drop the interview" then I will be going to Otago next year. So you see, that makes a lot of difference.

 

[frootloop]

I can only speak for myself, but the main reason that I chose Auckland over Otago, despite half of my family being there, is because of the interview, and less weighting on UMAT. So I will be really really annoyed if I find out half way throughout the year that they dropped interviews because obviously then, I would have gone to Otago. So if "Auckland told me right now they were going to drop the interview" then I will be going to Otago next year. So you see, that makes a lot of difference.

If Auckland and Otago end up with very similar/the same entry criteria, it will mean that your chances of med are the same at both schools. So overall, it would make no difference to your chances if they drop it now, or in the middle of the year (assuming they do it at all).

 

[juna]

But the whole point is that we don't know whether they are going to drop the interviews, frootloop. Of course if Auckland decide to drop interviews, then Otago and Auckland will be similar. However at the moment,WE DON'T KNOW and I would rather know now, before I sign up for Auckland, and find out later, when I could have gone down to Otago, with all my school friends, and be flatting with my sister.

 

[juna]

So what I am trying to say is that the chances of me getting into med in Auckland, if they drop interviews, will be similar, but that can be debatable because the papers are different in Auckland. But it will impact other parts of my stay, like financial. So that is why, personally, I would like to know this year, not half way through next year.

 

[lordgarlic]

Personally I don't think the interview stands as a tool to select someone who is "suitable for medicine". As pointed out in the aforementioned article, the predictive value of interviews is rather poor. At the end of the day I think interviews act more as a social selector more than anything

 

[Matt]

Personally I don't think the interview stands as a tool to select someone who is "suitable for medicine". As pointed out in the aforementioned article, the predictive value of interviews is rather poor. At the end of the day I think interviews act more as a social selector more than anything

The article is poor because it basically follows the line of reasoning below:

1. It shows no correlation between the three selection tools: UMAT/interview/grades and recognises that this is likely to mean they test different things.
2. It then correlates the three selction school with grades in medical school.
3. Big surprise grades correlate with grades and the other two don't.
4. Therefore the interview is no good..... uh, sorry, wtf?

I agree that to a certain extent interviews are akin to social engineering. I don't see this as a problem.

 

[lordgarlic]

Then the next thing we can discuss with the worthiness of using medical school grades as a tool of measuring achievement and success. It goes back to the beginning question that we always ask and that is "What makes a good doctor?"

Grades are only a small element of what makes up a doctor at the end of the day. A lot of attributes are very difficult to measure, if not impossible to measure

 

[Matt]

Then the next thing we can discuss with the worthiness of using medical school grades as a tool of measuring achievement and success. It goes back to the beginning question that we always ask and that is "What makes a good doctor?"

Grades are only a small element of what makes up a doctor at the end of the day. A lot of attributes are very difficult to measure, if not impossible to measure

Sure grades aren't perfect but that's the problem with these things, people trying to find perfect predictors... we never will, that was a given from the start. That said grades are a fair measure of knowledge and hard work (most people won't dispute that) and those are desirable from both students and doctors. The fact that it's not a perfect measure should be obvious.

Communication skills are a massive deal in medicine. It makes sense to test for them and if I were a Dean it's what I'd want my graduates to be good at. The interview hasn't got any evidence behind it but I'm yet to see it be tested for what it's actually trying to achieve - that is communication skills.

That said, I think it's intuitive that an interview does test communication skills and therefore it makes sense to use it. Especially when we now have MMIs that dispense with the issue of reliability.

 

[juna]

Does anybody know when UoA will be announcing whether they will drop interviews or not?

 

[greenglacier]

[MENTION=814]Matt[/MENTION] - FYI, I'm intending to start a discussion on this matter once the NZ-wide study hinted at in that article gets published (it's currently in press). I think it's probably best to wait until we have the full picture. Once it's released I'll start a thread that also links to the UWA study released earlier this year (which seems to have gone un-noticed here), because it provides an interesting comparison, where this (very interesting) discussion can be fully fleshed out.

 

[juna]

Hey guys, as promised, this is what Professor Alan Merry, Head of Department said: " The interview will remain in place for admission in 2012. However, we are considering the admission system and it is possible that an alternative admission process will be in place for 2013. The process or selection tools are not yet certain, but GPA will continue to play a significant role in selection." I think that they are seriously looking at other ways to select people for med in Auckland from 2013 onward. I have no idea what they will replace it with though.