All right. I'll admit it. I wasn't overly excited at this year's State Junior Director's meeting when we were informed that the state qualifying 3P Junior Olympics must be scored with Orion. This gave match sponsors two options: 1. hold the match and send the targets to CMP for their staff to score and post-- thus creating a substantial delay in the ability to hold a timely award's ceremony or 2. purchase the system.
Faced with purchasing the system, again, I felt that there were a number of negatives.
1. Expense. Even with the less expensive scanner (thus SLOWER), the bundle of software, targets and licensing came in at nearly $700.
2. Continued expense. Each year a new licensing fee would be paid and the targets used for the system are much more expensive than non-Orion targets.
3. Lack of NRA support. The NRA has not approved the Orion system for scoring and, thus, it means that as a match sponsor, I can only use the system for National Standard and USA Shooting matches. I can't use it for any of my NRA Sectionals or NRA registered or sanctioned matches.
With the assurance that progress was being made with NRA compliance and that the Orion system was going to remain the standard for qualifying JO's, I took the plunge and purchased the system.
Honestly, if I had had the opportunity to actually play with the thing before I bought it, I probably would've opted not to purchase it. In fact, my confidence in it has also deteriorated and I'm not sure I even want my matches scored on it.
There are a number of areas in which I find the product to be cumbersome.
1. Rather than dovetailing the scanning program with the Orion scoring program, the two remain separate. This means that scoring becomes a several step process by which you must first open the scanner program and scan each target, then close that program and open the Orion program, associate the targets you scanned and then score them.
2. Although there are a number of categories in which you can enter a competitor, printing the results of those categories requires one to print separate pages for each selected category rather than the simple excel program I've used in the past in which I can have a single bulletin with results and list a final column with awards.
3. The times given for scanning varied GREATLY from what I experienced. Perhaps this is due to the speed of my laptop, but that variable was not addressed in the original literature.
But the worst problem I had with my experience thus far has revolved around the National Standards Rules for Visual Image Electronic Scores and the actual scoring algorithms.
According to rule 8.5.4 (pg 37 of the 2010-2012 Rulebook), "Scores determined by approved VIS systems are final unless protested in accordance with Rules 8.2 and 8.5.5. During VIS scoring, the Statistical Officer, or Scorer, my only correct obvious scoring errors, such as failure to locate a shot, multiple shots on a bull, paper tears, etc.)
Rule 8.5.5 tells us that "The score of an individual shot may be protested if the competitor believes the VIS system socre the shot incorrectly. When a scoring protest is submitted in accordance with Rule 8.2, the Statistical Officer or Scorer will evaluate the protested shot. If there is an obvious error (see Rule 8.5.4) the Competition Director, Statistical Officer or Scorer will make a manual intervention to identify the shot location and return any protest fee paid by the competitor. The Statistical Officer or Scorer may not make a manual intervention if the VIS system has located the shot hole. In this case, the original scan of that shot will be rescored using the protest algorithm in the VIS system. The protest will be upheld and the protest fee returned to the competitor only if the whole number value of that shot goes up. If it remains the same, a two-point penalty will be deducted from the score for that shot. Except for obvious errors, scores in final rounds may not be protested. The score outcome of the protest scoring algorithm is final and may not be appealed."
So, how did this play out at my recent match? Terribly.
Although the speed in which I and my match staff improved over the 4 relays we held, the bottom line is that while I can generally score and post scores for a stage of 8 shooters within 25 minutes, we elongated that time by 10 minutes each stage.
When we went to figure out who won the categories, we had to print 10 sheets of paper (overall sporter, overall precision, J1 sporter, J1 precision, J2 sporter, J2 precision, J3 sporter, J3 precision and Tyro sporter, Tyro precision rather than the single sheet I have used with excel for the past decade or so.
But, the worst part of the day came when the top 3 precision scores came in with a tie for 1st and 3rd place was a scant two points behind that.
Looking at the targets, one of the shooters in the tie for 1st felt that from a visual perspective, it looked as if she had shot two points higher than she was credited (both given 9.9's by the scanner). Reading Rule 8.5.3, however, she was afraid to trust her eyes as the rule states "Scoring rings printed on paper targets used in a VIS system are for the shooter's reference only. They may not be used to determine the value of the shot." This, coupled with the odious warning of Rule 8.5.5, promising a 2 point penalty if her challenge was denied, she was bullied by the rulebook into accepting scores she was not sure of in order to at least maintain a 2nd place finish, rather than being penalized into 3rd.
THIS IS A PROBLEM. Rules should not be written to intimidate shooters from challenging, especially junior shooters, who are supposed to be learning their sport. A two point penalty on top of the monetary penalty is excessive.
USING TWO DIFFERENT ALGORITHMS IS A PROBLEM. If I were to score paper targets, I would be remiss if I did not plug each shot that was close enough to make me wonder. But, essentially, having a less accurate, more time expediant algorithm for standard scoring and reserving the more accurate algorithm for challenges is like eyeing the scoring until someone challenges and then inserting the plug. Worse than that, it is taunting the shooter to "go ahead, make my day" by telling him that the plug will be used, but if they're wrong-- they'll pay a heavy penalty of two points. "Do you feel LUCKY? Well... DO YOU?"
Oh, so, what happened? Well, the kid who was afraid to challenge ate it. She relingquished her opportunity to question the system and was awarded 2nd place. When the match ended, however, I ran the protest algorithm and the score jumped from 9.9 to 10.1. Then, I told the system to score the target as if for the first time, and it went back to 9.9.
I haven't had time to run my next experiment, but I have a mind to challenge every 9.9, 8.9, etc for all three of the top scores. I wonder if after using the "better" algorithm if I would've had a tie at all? Or, would the top 3 places have changed? (I will run this experiment to include with my report to the National Council for them to consider.)
I've played with this system for about two months now, though this was my first match experience with it. I used the system because I wanted the shooters to get accustomed to it, as well as the match staff, before we actually employed it for registered competition. I was disappointed.
I could trade off timeliness for accuracy and still walk away satisfied if I felt that the match winner was, beyond a doubt, the person who fired the higher score. When I put in 12 hours running 4 relays, I generally go home tired but satisfied. I'm satisfied that I've run a quality match that was fair and well-run. This past weekend, I just felt deflated. The kid who came in 2nd felt robbed and the kid that won couldn't feel good about his personal best because he wasn't really sure he had earned the title.
I felt cheated because under the National Standard Rulebook I was impotent to do much to correct anything-- perhaps an administrative challenge? Generally, when I'm scoring by hand, I allow the competitor to see his target with the plug and, if necessary, the aidr of a magnifying glass. Everyone goes home with the knowledge that they were credited with what they fired.
So, why am I sharing all this?
Basically, I'm looking for some feedback from some of you before I write to the National Council about my concerns. I'd like to itemize the many areas that need addressing and have the benefit of varied feedback.
If you have experience with the Orion system, I'd like to hear your thoughts. If you have opinions about the newly revised rules as pertain to the Visual Image Scoring, let me hear those, too.
Personally, I have always believed that the rules should be written for the protection of the shooter, not the convenience of the match staff-- though I have been both.
Lamentations on Orion and VIS rules
Moderators: pilkguns, Marcus, m1963
-
JHMARTIN as GUEST
I've run a few matches now with ORION ... from large (150-230 shooters) matches as well as our small state 4-H (26 shooters).
I have access to the simple DR-3010C scanner (use for small matches) to the production DR-5010C scanner. Personally I prefer the 5010C's speed, but image quality on eiter performs well.
Yes, you do have to use two programs, the one for the scanner and then ORION itself, but I do not mind as that allows flexibility in the scanners you can use.
You can keep both programs open at the same time and just flip back and forth... the scanner program dumps directly into the ORION match directory (or you should have it set that way)
The manual does state that best performance will come from a dula processor machine with decent memory installed .... I use a dual processor laptop and have no issues there. Your biggest gain in match processing speed will be to get a fast laser printer to print out the target results for challenges.
Also some hints: (note we're now using Ver 1.25)
1) Get your shooter info EARLY. Ask folks here in NM, I HOUND the coaches to get me shooter info to create the match before (read weeks) the match.
2) Make sure you break the shooters out into relays and keep that current. This will make target assignment much easier as you will not have to search thru all the shooters to assign each target .... Erik ... can we get a bigger target assignment window so we can see more than 8 shooters?
3) I also use the extra classification windows for DAY-1, DAY-2, etc .... ESPECIALLY if you are using rule 10.1.6 to shoot a final at the end of each day.
4) If you have program suggestions, do not send them to CMP, send them directly to Shooters Tech ... Erik has a list of upgrades and requests he is working on.
==========================
All that said to compliment the system (and I do prefer it over having 10-15 volunteers, some who have not ever scored a target before or have not since last year, or the questions over "which plug do we use... etc) I have a few gripes too.
1) Kruger prints targets that are accurate and well within the ISSF, USAS, CMP, (and yes NRA) standards. The shooter keys off the black bull when shooting, as does ORION when scoring, the lines are printed with enough accuracy that plugging should be allowed ... even on ESTs like Megalink a protest is not scored electronically, but measured off the paper or rubber ... while that's a real pain, on the ORION targets it's wasy .... my belief is that we should be allowed to plug.
2) This is pretty critical when you have a variety of guns that shoot a wide variety of speeds .... and you have a wide variety of target support (cardboard, etc behind the target) .... and you have a wide variety of temperatures and humidity conditions to deal with ... all in the same match .... Orion can have some issues here. For 95% of the time ORION is pretty darned spot on the scoring .... and most of the time when it is off the shots are in the .0 to .8 range and it's never off by more than a tenth, so in the qualification part of the match the exact tenths do not matter.
I do look at the .9's on all targets ... if the hole is ragged, ORION may not center the scoring ring on the hole ... I have no trouble moving it so that it is centered, to me that's an obvious error... 9 out of 10 times the integer score does not change. I just don't bother on the obvious errors if it's going to change a .5 to a .6. I do closely study all the holes on FINALs targets. I also believe that ORION's scoring algorithm has changed a bit to be more friendly to the shooter. (I've run the card scanner at the OTC and one of the "games" we play is to see how good we can score to the tenth before we run a card thru ... believe it or not you can get REALLY good at scoring to tenths by eye) ORION quite often gives a shooter a tenth that I would not have by eye. If ORION gives the shooter the benefit, bless it's heart, I don't change those ... usually the circle is dead on the hole then too.
3) If ORION's program was static I would be kinda bummed at the yearly renewal, but they keep updating the program, making it better, and also making it easier to use. I look at the cost there and compare it with lunches I'd have to buy volunteer scorers across all the matches we do, and ORION eats cheaper.
4) I think that the CMP decision to require ORION for the 3-P JO Qualifiers is pure B.S. I agree about sending it off, that totally screws up the awards process, but let me give you a hint. You can score those targets by hand, even write on the targets, and as long as your writing does not look about the same size as a hole, ORION ignores that. Also ORION uses noting above on the top of the target, I have the shooters whire their names and positions in big black letters at the top (P1-P2, S1, etc) ... I hate looking at those little fill in the dots, don't use them ... also, since the targets are A-4 size they will go thru a printer and you can print your own headers on them.
I don't want to risk a printed sticker coming off inside the scanner ....
5) I also do not agree with the council's determination that ORION is an EST to be treated like a fully electronic unit like the MegaLinks. On a FULL EST the shooter has instantaneous feedback, on ORION they do not ... they have to wait until after they shoot ... just like normal paper targets. Applying the 2 point penalty like for a FULL EST to me is overboard ... let us charge $10, let's say, per bull and let us plug it. This will keep (hopefully) shooters from wasting time and challenging all the bulls on a target ... the 2 points is a deterrent, but I'm not 100% convinced that ORION is that consistent across all the conditions I noted in #2 above.
==================
Now all the plus's & minus's noted above .... I'd still prefer ORION (scoring more consistently) than a host of folks scoring by hand. It ain't perfect, but neither are the MegaLinks and Sius-Ascors.
IF YOU GET THE SHOOTER INFO BEFOREHAND, you can make you match run very fast ..... practice with your daily practices in running the system.
I usually print up more targets than I need for the match and they are used up in practices in the following weeks.
RE: NRA .... yeah, well the behemoth moves slowly .... looks like they will FINALLY be ready to let us use ORION in 2012.
Rats forgot to login, so whoops ... JHMARTIN
I have access to the simple DR-3010C scanner (use for small matches) to the production DR-5010C scanner. Personally I prefer the 5010C's speed, but image quality on eiter performs well.
Yes, you do have to use two programs, the one for the scanner and then ORION itself, but I do not mind as that allows flexibility in the scanners you can use.
You can keep both programs open at the same time and just flip back and forth... the scanner program dumps directly into the ORION match directory (or you should have it set that way)
The manual does state that best performance will come from a dula processor machine with decent memory installed .... I use a dual processor laptop and have no issues there. Your biggest gain in match processing speed will be to get a fast laser printer to print out the target results for challenges.
Also some hints: (note we're now using Ver 1.25)
1) Get your shooter info EARLY. Ask folks here in NM, I HOUND the coaches to get me shooter info to create the match before (read weeks) the match.
2) Make sure you break the shooters out into relays and keep that current. This will make target assignment much easier as you will not have to search thru all the shooters to assign each target .... Erik ... can we get a bigger target assignment window so we can see more than 8 shooters?
3) I also use the extra classification windows for DAY-1, DAY-2, etc .... ESPECIALLY if you are using rule 10.1.6 to shoot a final at the end of each day.
4) If you have program suggestions, do not send them to CMP, send them directly to Shooters Tech ... Erik has a list of upgrades and requests he is working on.
==========================
All that said to compliment the system (and I do prefer it over having 10-15 volunteers, some who have not ever scored a target before or have not since last year, or the questions over "which plug do we use... etc) I have a few gripes too.
1) Kruger prints targets that are accurate and well within the ISSF, USAS, CMP, (and yes NRA) standards. The shooter keys off the black bull when shooting, as does ORION when scoring, the lines are printed with enough accuracy that plugging should be allowed ... even on ESTs like Megalink a protest is not scored electronically, but measured off the paper or rubber ... while that's a real pain, on the ORION targets it's wasy .... my belief is that we should be allowed to plug.
2) This is pretty critical when you have a variety of guns that shoot a wide variety of speeds .... and you have a wide variety of target support (cardboard, etc behind the target) .... and you have a wide variety of temperatures and humidity conditions to deal with ... all in the same match .... Orion can have some issues here. For 95% of the time ORION is pretty darned spot on the scoring .... and most of the time when it is off the shots are in the .0 to .8 range and it's never off by more than a tenth, so in the qualification part of the match the exact tenths do not matter.
I do look at the .9's on all targets ... if the hole is ragged, ORION may not center the scoring ring on the hole ... I have no trouble moving it so that it is centered, to me that's an obvious error... 9 out of 10 times the integer score does not change. I just don't bother on the obvious errors if it's going to change a .5 to a .6. I do closely study all the holes on FINALs targets. I also believe that ORION's scoring algorithm has changed a bit to be more friendly to the shooter. (I've run the card scanner at the OTC and one of the "games" we play is to see how good we can score to the tenth before we run a card thru ... believe it or not you can get REALLY good at scoring to tenths by eye) ORION quite often gives a shooter a tenth that I would not have by eye. If ORION gives the shooter the benefit, bless it's heart, I don't change those ... usually the circle is dead on the hole then too.
3) If ORION's program was static I would be kinda bummed at the yearly renewal, but they keep updating the program, making it better, and also making it easier to use. I look at the cost there and compare it with lunches I'd have to buy volunteer scorers across all the matches we do, and ORION eats cheaper.
4) I think that the CMP decision to require ORION for the 3-P JO Qualifiers is pure B.S. I agree about sending it off, that totally screws up the awards process, but let me give you a hint. You can score those targets by hand, even write on the targets, and as long as your writing does not look about the same size as a hole, ORION ignores that. Also ORION uses noting above on the top of the target, I have the shooters whire their names and positions in big black letters at the top (P1-P2, S1, etc) ... I hate looking at those little fill in the dots, don't use them ... also, since the targets are A-4 size they will go thru a printer and you can print your own headers on them.
I don't want to risk a printed sticker coming off inside the scanner .... 5) I also do not agree with the council's determination that ORION is an EST to be treated like a fully electronic unit like the MegaLinks. On a FULL EST the shooter has instantaneous feedback, on ORION they do not ... they have to wait until after they shoot ... just like normal paper targets. Applying the 2 point penalty like for a FULL EST to me is overboard ... let us charge $10, let's say, per bull and let us plug it. This will keep (hopefully) shooters from wasting time and challenging all the bulls on a target ... the 2 points is a deterrent, but I'm not 100% convinced that ORION is that consistent across all the conditions I noted in #2 above.
==================
Now all the plus's & minus's noted above .... I'd still prefer ORION (scoring more consistently) than a host of folks scoring by hand. It ain't perfect, but neither are the MegaLinks and Sius-Ascors.
IF YOU GET THE SHOOTER INFO BEFOREHAND, you can make you match run very fast ..... practice with your daily practices in running the system.
I usually print up more targets than I need for the match and they are used up in practices in the following weeks.
RE: NRA .... yeah, well the behemoth moves slowly .... looks like they will FINALLY be ready to let us use ORION in 2012.
Rats forgot to login, so whoops ... JHMARTIN
More of an explanation to #5 above:JHMARTIN as GUEST wrote:5) I also do not agree with the council's determination that ORION is an EST to be treated like a fully electronic unit like the MegaLinks. On a FULL EST the shooter has instantaneous feedback, on ORION they do not ... they have to wait until after they shoot ... just like normal paper targets. Applying the 2 point penalty like for a FULL EST to me is overboard ... let us charge $10, let's say, per bull and let us plug it. This will keep (hopefully) shooters from wasting time and challenging all the bulls on a target ... the 2 points is a deterrent, but I'm not 100% convinced that ORION is that consistent across all the conditions I noted in #2 above.
One of the reasons that folks feel that ESTs are more shooter friendly is in my opinion due to the fact that the shooters gets INSTANT feedback as to where the target system determines where 10.9 is in regards to the shot. It may NOT really be in the exact center of the black, but the shooter can adjust where their shot hits using their sights ... this feedback is going on between the shooter and the scoring system, not only in sighters, but also in record shots. There are no rings on EST's so the shooter MUST use the EST for feedback.
The reason I do not feel that ORION is, and feel it should not be treated as a full up EST, is that there is absolutely NO feedback between the scoring system and the shooter. The shooter only learns where ORION felt the 10.9 spot was long after they have cased their guns. During sighters and records the ONLY feedback that the shooter has are the printed rings on the targets ... the printed rings are used .... this is the other reason that I feel we should be able to plug ORION targets on challenge.
===============
Again.... I prefer ORION to hand scoring though in 99.99% of the shots
-
bellyshooter
- Posts: 23
- Joined: Fri Oct 05, 2007 1:32 pm
- Location: Rhode Island
I've done much of what you suggested regarding prior set-up, etc.
So far as the idea of scoring the targets myself and then sending them in for official scoring, it kind of defeats the purpose as I would not be allowed to plug close shots so in a close match the awards placement would come down to no more than a guess.
You did not mention anything regarding the two algorithms for scoring. How is this fair? It seems to me that it is interjecting a greater margin of error since an Orion scored tie (without the prostest algorithm) could actually be several points different with a rescoring. Similarly, if the margin for error between the first scoring attempt of 9.9 and the protested scoring attempt of 10.1 is to be seen as a sample, then any ties in center-shots are also suspect.
Because we are not allowed to plug the Orion targets (since those rings are only for the shooter's reference... it makes me wonder how useful that reference is. Suppose you're shooting prone and call a shot high. You look through your scope and clearly discern that the dot has been lightly nipped at 12 o'clock. Since you are used to plugging shots and your eye is calibrated for this, you opt NOT to adjust the sights b/c you know that there is enough space for the outer gauge not to break the line and this shot, though high, is still a '10' as you called it. Then, when Orion scores it, it turns out that it is scored a 9.9. Perhaps this has cost you a subsequent shot as well, since you didn't take an insurance click.
So far as the idea of scoring the targets myself and then sending them in for official scoring, it kind of defeats the purpose as I would not be allowed to plug close shots so in a close match the awards placement would come down to no more than a guess.
You did not mention anything regarding the two algorithms for scoring. How is this fair? It seems to me that it is interjecting a greater margin of error since an Orion scored tie (without the prostest algorithm) could actually be several points different with a rescoring. Similarly, if the margin for error between the first scoring attempt of 9.9 and the protested scoring attempt of 10.1 is to be seen as a sample, then any ties in center-shots are also suspect.
Because we are not allowed to plug the Orion targets (since those rings are only for the shooter's reference... it makes me wonder how useful that reference is. Suppose you're shooting prone and call a shot high. You look through your scope and clearly discern that the dot has been lightly nipped at 12 o'clock. Since you are used to plugging shots and your eye is calibrated for this, you opt NOT to adjust the sights b/c you know that there is enough space for the outer gauge not to break the line and this shot, though high, is still a '10' as you called it. Then, when Orion scores it, it turns out that it is scored a 9.9. Perhaps this has cost you a subsequent shot as well, since you didn't take an insurance click.
-
bellyshooter
- Posts: 23
- Joined: Fri Oct 05, 2007 1:32 pm
- Location: Rhode Island
I've done much of what you suggested regarding prior set-up, etc.
So far as the idea of scoring the targets myself and then sending them in for official scoring, it kind of defeats the purpose as I would not be allowed to plug close shots so in a close match the awards placement would come down to no more than a guess.
You did not mention anything regarding the two algorithms for scoring. How is this fair? It seems to me that it is interjecting a greater margin of error since an Orion scored tie (without the prostest algorithm) could actually be several points different with a rescoring. Similarly, if the margin for error between the first scoring attempt of 9.9 and the protested scoring attempt of 10.1 is to be seen as a sample, then any ties in center-shots are also suspect.
Because we are not allowed to plug the Orion targets (since those rings are only for the shooter's reference... it makes me wonder how useful that reference is. Suppose you're shooting prone and call a shot high. You look through your scope and clearly discern that the dot has been lightly nipped at 12 o'clock. Since you are used to plugging shots and your eye is calibrated for this, you opt NOT to adjust the sights b/c you know that there is enough space for the outer gauge not to break the line and this shot, though high, is still a '10' as you called it. Then, when Orion scores it, it turns out that it is scored a 9.9. Perhaps this has cost you a subsequent shot as well, since you didn't take an insurance click.
So far as the idea of scoring the targets myself and then sending them in for official scoring, it kind of defeats the purpose as I would not be allowed to plug close shots so in a close match the awards placement would come down to no more than a guess.
You did not mention anything regarding the two algorithms for scoring. How is this fair? It seems to me that it is interjecting a greater margin of error since an Orion scored tie (without the prostest algorithm) could actually be several points different with a rescoring. Similarly, if the margin for error between the first scoring attempt of 9.9 and the protested scoring attempt of 10.1 is to be seen as a sample, then any ties in center-shots are also suspect.
Because we are not allowed to plug the Orion targets (since those rings are only for the shooter's reference... it makes me wonder how useful that reference is. Suppose you're shooting prone and call a shot high. You look through your scope and clearly discern that the dot has been lightly nipped at 12 o'clock. Since you are used to plugging shots and your eye is calibrated for this, you opt NOT to adjust the sights b/c you know that there is enough space for the outer gauge not to break the line and this shot, though high, is still a '10' as you called it. Then, when Orion scores it, it turns out that it is scored a 9.9. Perhaps this has cost you a subsequent shot as well, since you didn't take an insurance click.
Let me start off stating again that I do not believe ORION is perfect, but neither is hand scoring and neither are ESTs. I do believe ORION is better than hand scoring overall.
You use different "algorithms" when you hand score and you consider that fair, right? One is by eye, and the other uses mechanical gauges and your ability/experience in using that gauge.
In a numerical calculation such as ORION uses, you have to change the algorithm else you feed the same numbers into the same calculation .... you should expect to see the same result if you do that.
I cannot comment too much other than guesses on the difference in the (I think) three algorithms that ORION uses.
1) Normal scoring of entire target
2) Quick scoring of entire target (for lots of ratty holes) (I've never used this, if it's an obvious error I move the scoring ring)
3) Protest scoring ... which I think uses a single bull variant of the V1.18 & below whole target protest scoring ...
On the whole, I think ORION does a good job of determining the hole in the target and scoring that hole.
There are times it is a "smidge" off, but in almost all cases I've studied the "smidge" off is to the benefit of the shooter. In those times the visible/scanned hole is SMALLER than the .177 scoring ring and ORION lays the edge of the scoring ring on the outside of the edge it detects, moving the scoring ring closer to the center, which also has the effect of moving the inner edge of the scoring ring closer to the center .... basically a (.177 scoring diameter) - (detected hole diameter) TIMES 2 advantage to the shooter.
Also if you look you will see some shots that tear paper fiber off the top layer of the target, outside the edge of the hole, taking with it some of the black ink ... it appears the hole has a gouge that takes part of the line when the hole does not really extend that far.
And then there are the occasional "obvious errors" where ORION is way off the placement of the scoring ring ... just manually fix those.
While there are times I want to say "HEY that's way off!!" in scoring, if you really want to be fair, go and look at each and every other hole that ORION scores for that shooter and I think you'll find, on the whole, and overall, the shooter benefits.
I've spent a lot of time looking at how this system scores the targets, I went after it to show why we should not use it. I found could not support that argument. Again I think that over all the bulls, the shooter benefits .... AND IS AT LEAST AS ACCURATE and even more important, MORE CONSISTENT than hand scoring.
Your point about nipping lines and not adjusting the sights is the same argument I have about the lack of feedback in a supposedly EST ... I believe we should be able to plug a contested shot.
=======================
Some examples of what I'm talking about. Download this file (it's about 1MB) and enlarge to about 200%
http://www.vc4hss.com/_2011_Info/ORION_Scoring.jpg
Middle target .... a good ORION scoring ring placement
Left Target ... if ORION is off, it's probably like this ... the scoring ring is to the benefit of the shooter
Right Target ..... obvious error, even at about 50% ... here I'll leave it alone as even if I fix it the integer value is the same
You use different "algorithms" when you hand score and you consider that fair, right? One is by eye, and the other uses mechanical gauges and your ability/experience in using that gauge.
In a numerical calculation such as ORION uses, you have to change the algorithm else you feed the same numbers into the same calculation .... you should expect to see the same result if you do that.
I cannot comment too much other than guesses on the difference in the (I think) three algorithms that ORION uses.
1) Normal scoring of entire target
2) Quick scoring of entire target (for lots of ratty holes) (I've never used this, if it's an obvious error I move the scoring ring)
3) Protest scoring ... which I think uses a single bull variant of the V1.18 & below whole target protest scoring ...
On the whole, I think ORION does a good job of determining the hole in the target and scoring that hole.
There are times it is a "smidge" off, but in almost all cases I've studied the "smidge" off is to the benefit of the shooter. In those times the visible/scanned hole is SMALLER than the .177 scoring ring and ORION lays the edge of the scoring ring on the outside of the edge it detects, moving the scoring ring closer to the center, which also has the effect of moving the inner edge of the scoring ring closer to the center .... basically a (.177 scoring diameter) - (detected hole diameter) TIMES 2 advantage to the shooter.
Also if you look you will see some shots that tear paper fiber off the top layer of the target, outside the edge of the hole, taking with it some of the black ink ... it appears the hole has a gouge that takes part of the line when the hole does not really extend that far.
And then there are the occasional "obvious errors" where ORION is way off the placement of the scoring ring ... just manually fix those.
While there are times I want to say "HEY that's way off!!" in scoring, if you really want to be fair, go and look at each and every other hole that ORION scores for that shooter and I think you'll find, on the whole, and overall, the shooter benefits.
I've spent a lot of time looking at how this system scores the targets, I went after it to show why we should not use it. I found could not support that argument. Again I think that over all the bulls, the shooter benefits .... AND IS AT LEAST AS ACCURATE and even more important, MORE CONSISTENT than hand scoring.
Your point about nipping lines and not adjusting the sights is the same argument I have about the lack of feedback in a supposedly EST ... I believe we should be able to plug a contested shot.
=======================
Some examples of what I'm talking about. Download this file (it's about 1MB) and enlarge to about 200%
http://www.vc4hss.com/_2011_Info/ORION_Scoring.jpg
Middle target .... a good ORION scoring ring placement
Left Target ... if ORION is off, it's probably like this ... the scoring ring is to the benefit of the shooter
Right Target ..... obvious error, even at about 50% ... here I'll leave it alone as even if I fix it the integer value is the same
-
bellyshooter
- Posts: 23
- Joined: Fri Oct 05, 2007 1:32 pm
- Location: Rhode Island
jhmartin wrote:Let me start off stating again that I do not believe ORION is perfect, but neither is hand scoring and neither are ESTs. I do believe ORION is better than hand scoring overall.
I agree, I have seen several instances of problematic scoring with all three systems. Still, I believe that hand scoring (with competent scorers) has yielded the best results. There is something infinately satisfying in actually viewing a plugged target and having the opportunity to see what the scorers saw.
You use different "algorithms" when you hand score and you consider that fair, right? One is by eye, and the other uses mechanical gauges and your ability/experience in using that gauge.
Actually, when I use a mechanical gauge, I use the SAME gauge for every shot on every target so, at least for those shooters who are shooting shoulder-to-shoulder that day, there will be no variation. As for the calibration of the eye, I would say that since it is aided by magnification and an additional set or two of eyes so that 2 scorers agree, I trust that system more. Ironically, the Orion system allows for a scorer to change the 'gross errors' by eye.
In a numerical calculation such as ORION uses, you have to change the algorithm else you feed the same numbers into the same calculation .... you should expect to see the same result if you do that.
My issue is, the Orion system uses a more accurate, though less time expedient, algorithm for challenges than it does for its standard scoring. Why aren't we using the best algorithm available from the get-go? Since you have used Orion in your matches, I would challenge you to go back, open one up and go through and rescore your top finishers on every close shot using the protest algorithm. You will find that the scores fluctuate. That means that you could well have given awards to the wrong people.
I cannot comment too much other than guesses on the difference in the (I think) three algorithms that ORION uses.
1) Normal scoring of entire target
2) Quick scoring of entire target (for lots of ratty holes) (I've never used this, if it's an obvious error I move the scoring ring)
3) Protest scoring ... which I think uses a single bull variant of the V1.18 & below whole target protest scoring ...
On the whole, I think ORION does a good job of determining the hole in the target and scoring that hole.
For a sport that demands precision, "on the whole" and doing a "good" job seems inadequate to me.
There are times it is a "smidge" off, but in almost all cases I've studied the "smidge" off is to the benefit of the shooter. In those times the visible/scanned hole is SMALLER than the .177 scoring ring and ORION lays the edge of the scoring ring on the outside of the edge it detects, moving the scoring ring closer to the center, which also has the effect of moving the inner edge of the scoring ring closer to the center .... basically a (.177 scoring diameter) - (detected hole diameter) TIMES 2 advantage to the shooter.
Also if you look you will see some shots that tear paper fiber off the top layer of the target, outside the edge of the hole, taking with it some of the black ink ... it appears the hole has a gouge that takes part of the line when the hole does not really extend that far.
The shots I experienced problems with in my match were precise and clean.
And then there are the occasional "obvious errors" where ORION is way off the placement of the scoring ring ... just manually fix those.
Agreed. And, happily, I haven't really had those.
While there are times I want to say "HEY that's way off!!" in scoring, if you really want to be fair, go and look at each and every other hole that ORION scores for that shooter and I think you'll find, on the whole, and overall, the shooter benefits.
That is what I've done. And, when you do that, there can be a variance of .2 from the original scoring scan to the protest algorithm. Think about it. If out of 60 shots there are as few as 2 shots that the protest algorithm would score differently, then that would mean that with the top two or three shooters, there is a potential of 4-6 shots with a .2 difference. So, if those shooters are tied in both score and center shots, someone was at a disadvantage. And, under the current rules, if that kid wants to challenge one of those close ones, he/she risks dropping two penalty points. Since center-shots are used for the first tie-breaking rule, if Orion was .2 off, that may not effect the whole number but would have a profound effect on the decimal determination of a center shot.
I've spent a lot of time looking at how this system scores the targets, I went after it to show why we should not use it. I found could not support that argument. Again I think that over all the bulls, the shooter benefits .... AND IS AT LEAST AS ACCURATE and even more important, MORE CONSISTENT than hand scoring.
Perhaps your scorers are, as you mentioned in your first post, new to the game and learning on the job. In that case, you're probably better off with Orion. But, if you have an experienced and trained staff, I would put that staff up against Orion for speed and accuracy. Best of all, I would say that it would be more satisfying for the shooter to actually be able to see the plugged shot and go home feeling like, "damn, that was close-- but, it WASN'T in... they were right."
Your point about nipping lines and not adjusting the sights is the same argument I have about the lack of feedback in a supposedly EST ... I believe we should be able to plug a contested shot.
I'm hoping that this rule will be revisited by the National Council.
=======================
Some examples of what I'm talking about. Download this file (it's about 1MB) and enlarge to about 200%
http://www.vc4hss.com/_2011_Info/ORION_Scoring.jpg
Middle target .... a good ORION scoring ring placement
Left Target ... if ORION is off, it's probably like this ... the scoring ring is to the benefit of the shooter
Right Target ..... obvious error, even at about 50% ... here I'll leave it alone as even if I fix it the integer value is the same
Whew ... long replies & all .... boiling it all down:
1) I like ORION
2) I think we should be able to plug challenged holes
3) It's BS that CMP is requiring ORION to be used for 3-P JO Qualifiers. It's as bad as NRA not allowing a target because it lacks the NRA chicken on it. What .... next year they'll require megalinks like at Perry or Anniston??? .... or have to travel there to shoot the qualifier??? ... again BS
1) I like ORION
2) I think we should be able to plug challenged holes
3) It's BS that CMP is requiring ORION to be used for 3-P JO Qualifiers. It's as bad as NRA not allowing a target because it lacks the NRA chicken on it. What .... next year they'll require megalinks like at Perry or Anniston??? .... or have to travel there to shoot the qualifier??? ... again BS
-
bellyshooter
- Posts: 23
- Joined: Fri Oct 05, 2007 1:32 pm
- Location: Rhode Island
More BS... but, perhaps, progress...
I spoke with Vicki at CMP this morning and expressed some of my concerns regarding Orion, chief among them the 2 point penalty in rule 8.5.5 for scoring protests that are not upheld.
I was able to give her something to consider that apparently went unnoticed when the rule was written as a measure of combating excessive challenge issues.
As it is written, a competitor has no real means to challenge a center shot. Why is this a problem? Because center shots are the first means of breaking a tie. So, if you have two shooters with identical scores in both points and center shots, but one feels that he/she has actually earned a center shot for which he/she wasn't credited, that shooter is pretty much screwed.
According to the rule, any challenged shot value that does not result in a whole number change is assessed a two point penalty. So, if the shooter challenged and the challenged target went up .1 or .2 and that resulted in an additional center shot which would then put him/her in 1st place, according to the rules, that shooter would be rewarded with a 2 point penalty because the change in score was not a whole number. Hmmm... I guess that would pretty much take care of the tie-breaker :(
Anyhow, at least I've got them thinking a bit...
I was able to give her something to consider that apparently went unnoticed when the rule was written as a measure of combating excessive challenge issues.
As it is written, a competitor has no real means to challenge a center shot. Why is this a problem? Because center shots are the first means of breaking a tie. So, if you have two shooters with identical scores in both points and center shots, but one feels that he/she has actually earned a center shot for which he/she wasn't credited, that shooter is pretty much screwed.
According to the rule, any challenged shot value that does not result in a whole number change is assessed a two point penalty. So, if the shooter challenged and the challenged target went up .1 or .2 and that resulted in an additional center shot which would then put him/her in 1st place, according to the rules, that shooter would be rewarded with a 2 point penalty because the change in score was not a whole number. Hmmm... I guess that would pretty much take care of the tie-breaker :(
Anyhow, at least I've got them thinking a bit...
Agree that that is an oversight in the rules.
But I would argue that the intent of the rule would allow the match jury to allow the 2 point penalty be over ridden and the shooter be given the same numerical score with an additional center shot.
Write that up in the match report and let CMP come back and tell you you erred ... my bet is they would let the jury decision stand. If the CMP nipped ya, I'd run that all the way to the 3-P Council jury of appeal.
However, if the center shot protest failed, then I'd go with the 2 point penalty as again, that is the intent of the rule.
All the above .... my $0.02
Very good bellyshooter, a great catch of a hole in the rule.
But I would argue that the intent of the rule would allow the match jury to allow the 2 point penalty be over ridden and the shooter be given the same numerical score with an additional center shot.
Write that up in the match report and let CMP come back and tell you you erred ... my bet is they would let the jury decision stand. If the CMP nipped ya, I'd run that all the way to the 3-P Council jury of appeal.
However, if the center shot protest failed, then I'd go with the 2 point penalty as again, that is the intent of the rule.
All the above .... my $0.02
Very good bellyshooter, a great catch of a hole in the rule.
-
Guest
And that matters how? He saw an opportunity and developed a product.Anonymous wrote:I have never seen any discussion about the fact that the Orion system was developed by, and is marketed by, Gary Anderson's son Erik.
It's a good product, performs well, and if nothing else, it got better target paper into the 3-P competitions.
If you're a muckity-muck with the NRA, then yeah, I think it mattered and that's why the feet dragging has taken place to get it approved for comps there.
How does a shooter challenge the target when it has to be mailed off to be scored? In our state looks like only one Orion system in the state and our side of the state has to mail it off to the other side to get scored, or how about the ones that have to be sent in to CMP to be scored? So not only do they need to wait to find out the score, placement etc, don't have the opportunity to challenge even.
I do have to laugh .... "Trust 'em?"cmj wrote:How does a shooter challenge the target when it has to be mailed off to be scored? In our state looks like only one Orion system in the state and our side of the state has to mail it off to the other side to get scored, or how about the ones that have to be sent in to CMP to be scored? So not only do they need to wait to find out the score, placement etc, don't have the opportunity to challenge even.
Supposedly the prelim results are posted 14May and are final 21May. But they would have to post not only the ORION printouts, but the target scans too ... I scan at no compression and the targets at about 3.5mB each
This was not a very well thought out methodology.
Link to the CMP match Director Guidelines is here:
http://www.odcmp.com/3P/JO.htm
-
Old Dog as Guest
File size
I have found that file size is critical to Orion performance. JHMARTIN, what scanner are you using to get 3.5 MB files? The most I have been able to get our of a 3010 is 1.5MB. I wish I had a 4010 or 5010 but can't afford them.
Even more important than file size is the quality of the target you use for calibration. I made a special calibration target that is punched with a leather punch and very lightly probed with a .177 plug. Using this target for calibration solved about 90% of my problems. Follow the instructions in the version 25 software.
Even more important than file size is the quality of the target you use for calibration. I made a special calibration target that is punched with a leather punch and very lightly probed with a .177 plug. Using this target for calibration solved about 90% of my problems. Follow the instructions in the version 25 software.
Orion Scoring
I purchased an Orion System with the 3010C scanner. We have been using it at practices for both smallbore and air rifle using it twice a week on average. We've carted the system to to other clubs in the area to run test matches, both in Air and smallbore.
I have found that most errors (the few that there are) are within the specs of the system. Orion has missed a shot in the black once, but that was due to a paper tear.
We've tested the system with ripped targets, wet targets (yes, we are in WA state) and anything else we could think of. The system has scored consistently in all of these tests.
I have been disappointed with the reporting system, but the latest update helps solve most that problem. I am using the comprehensive data dump, import it into Excel. I have set up worksheets to glean the data I want, and run pivot tables against that data to generate match results. The data dump does change a bit from match type and how you do classifications, so I have a different spreadsheet for each type of match that I will run.
The shooter data is a bit of an issue as well. Entering data before the match is a must. I've started creating bogus matches for each club, and put all of that club's shooter information into that match. When I need to create a new match, I import the data from each club, and eliminate the non-participants. That makes it a bit easier to manage match setup.
I have found that most errors (the few that there are) are within the specs of the system. Orion has missed a shot in the black once, but that was due to a paper tear.
We've tested the system with ripped targets, wet targets (yes, we are in WA state) and anything else we could think of. The system has scored consistently in all of these tests.
I have been disappointed with the reporting system, but the latest update helps solve most that problem. I am using the comprehensive data dump, import it into Excel. I have set up worksheets to glean the data I want, and run pivot tables against that data to generate match results. The data dump does change a bit from match type and how you do classifications, so I have a different spreadsheet for each type of match that I will run.
The shooter data is a bit of an issue as well. Entering data before the match is a must. I've started creating bogus matches for each club, and put all of that club's shooter information into that match. When I need to create a new match, I import the data from each club, and eliminate the non-participants. That makes it a bit easier to manage match setup.
Re: File size
DR5010-C .... the school system has 2 of these ... for the State 4-H match I use a DR3010-COld Dog as Guest wrote:what scanner are you using to get 3.5 MB files?
Erik has told us that Version 1.1 should be out soon that has some databasing functions .... no other details, but I keep checking the update site.....jwmatter wrote:I have been disappointed with the reporting system, but the latest update helps solve most that problem. I am using the comprehensive data dump, import it into Excel. I have set up worksheets to glean the data I want, and run pivot tables against that data to generate match results. The data dump does change a bit from match type and how you do classifications, so I have a different spreadsheet for each type of match that I will run.
The shooter data is a bit of an issue as well. Entering data before the match is a must. I've started creating bogus matches for each club, and put all of that club's shooter information into that match. When I need to create a new match, I import the data from each club, and eliminate the non-participants. That makes it a bit easier to manage match setup.
Also ... good idea on the bogus club matches
-
Guest
Re: File size
Really looking forward to the database function. Last time I talked to Erik, I think he said in January.[/quote]Erik has told us that Version 1.1 should be out soon that has some databasing functions .... no other details, but I keep checking the update site.....
Also ... good idea on the bogus club matches