How to fix olympiads biggest problem. (Or at least my biggest problem)

[AlterNSO]

In another thread on here someone mentioned that Illinois had a test bank for regional tournaments that ES's could draw from. I wonder if this would help a lot of issues with ES's. Test writing for Science Olympiad is never easy and I've written quite a few. The high level of mastery of many students and the need to differentiate them makes it a challenge(It can also be challenging to differentiate students at the lower level of mastery). I think this is a big deterrent to people to volunteering and also to people making mistakes on the rules. For many ES's in an event like optics most of their effort probably goes into making the test and then they look at the other aspects of the event as an afterthought. Anyway it would be a very large undertaking to making a nationwide test bank for regionals and state tournements, it could solve many of the issues around ES's.

On a side note I found a Maryland has a good test writing guide I found, something like that posted to the national site might help some of the test writing issues as well. Here is a link.
https://marylandscienceolympiad.org/ind ... temid=1127

[Unome]

On a side note I found a Maryland has a good test writing guide I found, something like that posted to the national site might help some of the test writing issues as well. Here is a link.
https://marylandscienceolympiad.org/ind ... temid=1127

This looks pretty good in most places, although the part on Participation is a little strange (I have some minor disagreements on style, but nothing important).

[drcubbin]

That's a well-thought out document. I'll pass it along to our coaches. Thanks!

[19alekb]

I made myself have a 24 hour cooling off period after our state competition before I posted this. My first attempt was a flame fest and not constructive, and I hope this can actually generate ideas to help fix the issue.

I think Olympiad is an incredibly program, That's why I volunteer hundreds of hours a year and thousands of dollars to support it. That does not mean that it is perfect, and we should always be assessing ourselves and looking for ways to improve.

My biggest issue I have with Scioly, and the biggest weakness I see in the organization is STATE LEVEL event supervisors having absolutely no clue about the rules of there event. Second is that there doesn't seem to be any push to fix this.

For example, the optics competition at state was a complete joke. The rules clearly state the size of the mirrors to be used (5-8 cm). My kids and multiple others said they were way bigger than that. This killed our teams ability to compete because my kids had developed a system that would work great with any legal mirror, and they consistently got within 2 cm of the target, but when you gave them mirrors that much outside the rules our system wasn't useable.

Second I heard from my kids and 2 other teams that the event supervisor knocked over the mirror as they took off the cover, and then put the mirror back "in the general area" of where it was. Despite the rules explicitly stating they were not to touch the covers. what's even worse is that the event supervisor had illegal (though for a different reason) mirrors at the regional competition, and we called him on it and provided legal mirrors.

I'm a little shocked that this type of incompetence hasn't resulted in a fist fight. Maybe it has and I just haven't heard the story.

The real issue is it just totally deflates the kids. we all know the time it takes to be competitive, and when the people running it can't even be bothered to read the rules it just sucks the will to work out of them, or makes them think that all of Olympiad is run by morons.

So, what are some things that can be done to prevent this from happening or to correct it when it does and make sure it doesn't happen again?

This happened at regionals to me(I'm in optics), we still got first because the proctor let us put the mirror back in place, but he didn't have to.

[Raleway]

Incompetent ES's won't go away... there just simply are not enough volunteers most of the time. Even when they are well-intentioned, most of them just have not competed before. The easiest and most simple solution to this is to allow alumni to proctor as many events as possible (albeit I understand many might not want to, have the time, or ability to cover all the expenses being a college student). It's easy to be angered as a competitor, but we need to realize that manpower is limited and that many ES think they are giving back and donating time to the science community (of which they definitely are). Their understanding is to promote science (not results), which can lead to some conflicting ideas. If an ES does mess up, as it can happen to anyone, the better question is what procedure happens next. That might be a better issue to broach.

Regarding tests, I think we've been too spoiled. What constitutes a "good" test? My dad, a PhD in Chemistry, saw all of my Science Olympiad chemistry tests as really easy, which is very reasonable. If I asked him to write a test regarding the same extraordinarily open-ended topics that SOINC lists, the curve would probably balloon at 10%. Science is just that broad. Tests are inherently flawed in this system because no one test can cover all the topics listed in just 50 minutes while also differentiating skill levels. If we really wanted to test pure skill, each test would be speed based to see who knows their stuff the best. That being said, some tests simply are too easy. However, if a test was that easy, then people should all get a full score on them. I've had tests where I have seen all the information, done practice, but still got a 97% and thus 4th place. That's on me, not the test. The only time I feel a test really is too easy is if two or more teams got 100% on the test. That means that the test was so easy that multiple teams were able to completely solve all the problems. In the end, each team is subject to the same test- thus it is fair game. Just my two cents. There are more important issues to think about such as team distribution nationally and scheduling and venues and the whole lot.

[knottingpurple]

Incompetent ES's won't go away... there just simply are not enough volunteers most of the time. Even when they are well-intentioned, most of them just have not competed before. The easiest and most simple solution to this is to allow alumni to proctor as many events as possible (albeit I understand many might not want to, have the time, or ability to cover all the expenses being a college student). It's easy to be angered as a competitor, but we need to realize that manpower is limited and that many ES think they are giving back and donating time to the science community (of which they definitely are). Their understanding is to promote science (not results), which can lead to some conflicting ideas. If an ES does mess up, as it can happen to anyone, the better question is what procedure happens next. That might be a better issue to broach.

Regarding tests, I think we've been too spoiled. What constitutes a "good" test? My dad, a PhD in Chemistry, saw all of my Science Olympiad chemistry tests as really easy, which is very reasonable. If I asked him to write a test regarding the same extraordinarily open-ended topics that SOINC lists, the curve would probably balloon at 10%. Science is just that broad. Tests are inherently flawed in this system because no one test can cover all the topics listed in just 50 minutes while also differentiating skill levels. If we really wanted to test pure skill, each test would be speed based to see who knows their stuff the best. That being said, some tests simply are too easy. However, if a test was that easy, then people should all get a full score on them. I've had tests where I have seen all the information, done practice, but still got a 97% and thus 4th place. That's on me, not the test. The only time I feel a test really is too easy is if two or more teams got 100% on the test. That means that the test was so easy that multiple teams were able to completely solve all the problems. In the end, each team is subject to the same test- thus it is fair game. Just my two cents. There are more important issues to think about such as team distribution nationally and scheduling and venues and the whole lot.

I do feel which tests which deviate from the rules are unfair and should be prevented if possible, because it's entirely possible that some competitors have had a reason to study information outside the extent of the rules while others have not, making a test which should be about competitors' working hard according to the rules instead a test about some unrelated activity or class or interest students may or may not have.

[Skink]

This is just my opinion, but nobody should be getting "A" grades on standard Science Olympiad events. If they are, either they were super over-prepared (unlikely for C but possible for B, I guess) or the event didn't challenge them to think critically or tackle new problems using old information. I'm not unpacking this more now because I don't want to end up writing an ES manual...just that I see too many high-scoring events and wish that would change. I've seen the 97% for fourth place problem, and it renders that entire ranking (and all involved participants' work) nigh meaningless.

[knightmoves]

If we really wanted to test pure skill, each test would be speed based to see who knows their stuff the best.

That's not what speed tests. If you can answer 150 simple MC questions in the time it takes the next person to answer 145, maybe you just read faster, or have better-trained test technique.

Speed on difficult questions starts to test ability rather than nonsense. Besides, difficult questions are more interesting.

[knottingpurple]

If we really wanted to test pure skill, each test would be speed based to see who knows their stuff the best.

That's not what speed tests. If you can answer 150 simple MC questions in the time it takes the next person to answer 145, maybe you just read faster, or have better-trained test technique.

Speed on difficult questions starts to test ability rather than nonsense. Besides, difficult questions are more interesting.

Speed being a tiebreaker benefits people who have practiced taking tests over people who have studied. If you want to see who knows stuff best, very general open ended explanation questions are much better, so you can see not only who can regurgitate but who understands enough to explain well.

[Raleway]

If we really wanted to test pure skill, each test would be speed based to see who knows their stuff the best.

That's not what speed tests. If you can answer 150 simple MC questions in the time it takes the next person to answer 145, maybe you just read faster, or have better-trained test technique.

Speed on difficult questions starts to test ability rather than nonsense. Besides, difficult questions are more interesting.

Speed being a tiebreaker benefits people who have practiced taking tests over people who have studied. If you want to see who knows stuff best, very general open ended explanation questions are much better, so you can see not only who can regurgitate but who understands enough to explain well.

It's impossible to score general open-ended tests properly when most test graders are not those who have written the test or have knowledge of the event. It is also hard to grade on a holistic scale since someone might obviously have the better answer but all of them satisfy the parameters set by the answer key. Those questions are inherently flawed.