Wider camera requirements for online quiz bowl

Old college threads.
Locked
User avatar
Snom
Lulu
Posts: 53
Joined: Tue Oct 18, 2016 5:52 pm

Wider camera requirements for online quiz bowl

Post by Snom »

I originally made a few posts on the Discord about this last week, but in the end decided that the consequences outweighed the drawbacks. After more incidents this past week and encouragement from some Discord posters, I have decided to propose this on the forums to gather feedback.

The proposal:

All collegiate, regular season, all/most subjects, online, tournaments require participants to provide a video feed displaying at least their computer screen(s) and their hands during all "playing time" (tossups where they can still buzz, their team's bonuses, the other team's bonuses if the tournament allows bouncebacks, and any time they are on the clock for answering a question). If for any reason a player is required to remove their hands from the frame or remove the webcam, the player is temporarily disqualified for the duration of the tossup or bonus cycle.

How players can meet these requirements
These requirements can be satisfied with either an external webcam, or a smartphone with wireless local area network (WLAN) access. With an external webcam the setup should be obvious. This picture shows how we can accomplish it with a smartphone (my iPhone 7+) by placing it around a foot and a half to the side of my laptop. The phone can be propped up on any object that is taller than the phone is wide. Either the computer or phone should be placed at an angle so that the screen is visible. The phone should be placed close enough to the computer or an outlet for a charging cable to reach.

Image

The video stream provided by the above setup can be seen here:
https://youtu.be/T8ldNbOQccg

As you can see, it is clear when I am using Discord and playing the game and when I switch windows to my browser. Even if I were cheating by going to a different Discord server, the screen change is noticeable enough. During "playing time" when defined above, there is no reason any player should ever be doing anything on their screen that isn't viewing the game channel. Additionally, there is no way that I could use my phone without being caught or removing my hands from the frame. It is also clear when I am typing -- typing buzz is only four letters, if a player is typing more than that on a tossup something is up.

If table space is limited, players may place their webcam somewhere behind them and to the side so the camera can see over their shoulder. As long as their hands and screen are visible, any set up is fine. This video shows another acceptable camera angle:
https://youtu.be/Ypl957f5PMI

Getting the camera into Discord / Zoom:
If using an external webcam, simply change your video source in the application to the webcam.

If using the phone setup, there are two options. If the phone is close to you as in the first example, or you are using Zoom, simply download the Discord/Zoom app on your phone and join the call using your phone.

For Zoom tournaments, you can simply join the call on both your computer and phone without issue. You can use your computer's speakers and mic. Discord will not allow you to join the same call on two devices without using two accounts, so your computer will not be in the call (you can still chat as normal). If you cannot hear well from the phone's speaker due to the phone's distance, consider using wireless earbuds if available.

If you are using Discord, and the phone is far away as in my second setup, an app will need to be used to pipe the video from the phone to the computer. This will make your phone appear as a webcam to Discord. I have had great success with https://iriun.com/ , but https://www.kinoni.com/ may also work for you. Setting up either app takes a couple of minutes at most. Simply download and run the app on the phone, and run the corresponding app on your computer. A new virtual webcam will appear with your phone camera feed. Even if you are using the close setup, this solution may be preferable because you can unmute/mute yourself more easily to confer on bonuses/tossups.

The rationale:

For the second time in a month, the quiz bowl discourse has been completely centered on cheating accusation against a player at a tournament who did not use a webcam. If you are unable to provide a camera, and are under any valid suspicion of cheating, you should not be allowed to play quiz bowl. Quiz bowl is a competitive activity -- cheating allegations at every tournament with a decent field size is simply unacceptable. While some have argued that we should simply forget about it and accept that cheating will occur, a quick survey of the Discord server two weeks ago and today shows that people will care about cheating when it occurs, as we should. I would like if we could put all the vague speculation and arguing over stat increases to rest.

However, relying on a vague criterion of "valid suspicion" is a bit shaky in my opinion, and we should instead simply require webcams for all participants. This should hopefully prevent even the slightest concern about "in game biking" from arising. The "Andy Watkins" method of acquiring packets in advance can obviously not be solved by this, but that hasn't been an issue yet online.

Requiring a webcam that shows only the face of the participant (if even that) is simply not enough. Last night, after the cheating accusations, I tried to "simulate biking" on the same bonuses that I played earlier that day against the accused team with a webcam on. While upon close inspection I was able to tell when I was moving my eyes around a bit, without the screen and my hands being visible, I could not differentiate when I was and was not googling only from the webcam. I could at best have around 75% confidence, but we should not be vilifying people for on a hunch!
Even then, I had to focus on one video stream the whole time. In a game with 8 players, it would be pretty hard to be confident at all. Observing the cameras of participants at LIT yesterday also showed that some people just don't like staring at one place for all that long and naturally move their eyes around, even when they are clearly not cheating.

tl;dr the benefits of this set up are simply massive. it is nearly IMPOSSIBLE to cheat with these measures in place. Catching cheaters with current rules is hard. The proposed rules will make any and all cheating very obvious to moderators or any other players paying the slightest attention to the video feed.

Possible concerns and limitations:

1. Tournaments are fraught with technical difficulties already, requiring a more complicated setup will lead to even more!

I don't really have a good answer for this, other than that I have not really found any issues using my phone and Iriun for two tournaments in the past month.

2. This is too expensive and will make playing online quiz bowl less accessible!

96% of American adults aged 18-26 own a smartphone. I suspect when only considering college students the percentage is even higher. If you are in the small population that does not have a phone but still wishes to play quiz bowl, consider buying a $30 webcam off amazon. The cost of in person quiz bowl is much, much higher anyway even considering this one time cost.

I am tentatively limiting this proposal to only collegiate events, as I don't think the above justification is as readily applied to high schoolers.

3. I don't have the bandwidth to play

You can try reducing your bandwidth usage on Discord by disabling the video of everyone else in the call as a player. I've tested this and seen that works in reducing download bandwidth at least. As for others with poor internet, I believe the upside of ELIMINATING in-game-cheating and protecting the integrity of the game is important enough to outweigh the cost of a few players being unable to participate. Your opinion may be different. I'm making this thread to gather opinions first, before implementing it in any tournament.

4. The setup is too complicated!

We are used to driving 2, 3, 4, even 6 hours to play in person quiz bowl. Sparing 15 minutes maximum before a tournament to set up a webcam is NOT a prohibitively burdensome measure, given the benefits.

If these rules were implemented, I would encourage college teams to set up all the things well in advance during practices, so all players know what to do when tournament day comes.

5. My phone is overheating!

Try using an external webcam.
Albert Zhang
Columbia '24
State College '16-'20
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Wider camera requirements for online quiz bowl

Post by Mike Bentley »

I'm in general in favor of an approach like this for "serious" tournaments. It's a relatively low barrier to entry and will make a lot of cheating more difficult. Of course, in practice it's going to be hard to get through to people that you need to do this setup and stick with it through the tournament. Even the very basic norm of "moderators must have headsets" that I've been hammering home for 5+ years isn't widely adopted.

There is a downside of the tournament being a bit less personable than a camera pointed at someone's face as all you get to see is the back of their head.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Wider camera requirements for online quiz bowl

Post by Mike Bentley »

I'll also note that I personally find the USB Extension Cord + Web Cam much easier to set up than a separate phone rig with multiple accounts. It's a one-time thing you do and then it basically works forever if you're playing tournaments from the same room.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Sean
Wakka
Posts: 133
Joined: Tue Jun 30, 2020 10:30 am

Re: Wider camera requirements for online quiz bowl

Post by Sean »

Mike Bentley wrote: Sun Sep 27, 2020 4:17 pm I'm in general in favor of an approach like this for "serious" tournaments. It's a relatively low barrier to entry and will make a lot of cheating more difficult. Of course, in practice it's going to be hard to get through to people that you need to do this setup and stick with it through the tournament. Even the very basic norm of "moderators must have headsets" that I've been hammering home for 5+ years isn't widely adopted.
Taking a more hopeful stance, the demand for online tournaments is going to be much higher this year than in previous years so I would imagine that more high-quality online tournaments are going to happen which may hopefully (🤞) solidify the "moderators must have headsets" rule somewhat. On the flip side, it's very possible tournaments will transition to being in-person with precautions instead of being all online before any lasting norms for online tournaments can be established just because of the frustration of all the kinks and allegations. Extensive anti-cheating measures in novice-level outreach tournaments would hinder the outreach aspect, but perhaps it would be possible for AQBL to put a measure like this in their more competitive tournaments since they will be coordinating a lot of online mirrors.
Sean Farrell
Mandarin High '20
North Florida '23
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Wider camera requirements for online quiz bowl

Post by Mike Bentley »

Sean wrote: Sun Sep 27, 2020 4:45 pm
Mike Bentley wrote: Sun Sep 27, 2020 4:17 pm I'm in general in favor of an approach like this for "serious" tournaments. It's a relatively low barrier to entry and will make a lot of cheating more difficult. Of course, in practice it's going to be hard to get through to people that you need to do this setup and stick with it through the tournament. Even the very basic norm of "moderators must have headsets" that I've been hammering home for 5+ years isn't widely adopted.

There is a downside of the tournament being a bit less personable than a camera pointed at someone's face as all you get to see is the back of their head.
Taking a more hopeful stance, the demand for online tournaments is going to be much higher this year than in previous years so I would imagine that more high-quality online tournaments are going to happen which may hopefully (🤞) solidify the "moderators must have headsets" rule somewhat. On the flip side, it's very possible tournaments will transition to being in-person with precautions instead of being all online before any lasting norms for online tournaments can be established just because of the frustration of all the kinks and allegations. Extensive anti-cheating measures in novice-level outreach tournaments would hinder the outreach aspect, but perhaps it would be possible for AQBL to put a measure like this in their more competitive tournaments since they will be coordinating a lot of online mirrors.
I hadn't been thinking about novice high school events. I think there's an argument to be made there that the more anti-cheating measures you have in place the more burdensome it will be. Although it's also the case that people new to quizbowl don't really have expectations of what quizbowl is and are likely more receptive to "this is what you have to do to play."

If a tournament is going to enforce this policy (which, again, I think any serious collegiate event should at least consider), then we need to come up with ways to make sure it actually gets enforced. It's been my observation that people are terrible at adhering to "requirements" at online events. And you as a TD may not even know about it until someone gets to your room and, say, the camera is off.

One idea is to require that all players do an equipment test before playing. I'd go further and say that an equipment test should be done prior to the day of the tournament. In the online tournaments I've hosted, I've done this with moderators. Scaling up to all players in a tournament, especially for larger events, will be a burden. But I think it's worth doing, at least until this norm becomes more established.

And at almost every event with a camera policy, some people have not been able to have their video on. Are they not going to be allowed to play? So far, no TD I know has been this stringent. But it's worth considering. Especially since "camera off" tends to be contagious. Once people see that it's not really required they start turning theirs off.

Is there anyone out there who has played a Zoom tournament where their connection is good enough to hear the moderator but not good enough to broadcast their video? I've heard of this on Discord but not on Zoom.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Snom
Lulu
Posts: 53
Joined: Tue Oct 18, 2016 5:52 pm

Re: Wider camera requirements for online quiz bowl

Post by Snom »

Mike Bentley wrote: Sun Sep 27, 2020 5:10 pm And at almost every event with a camera policy, some people have not been able to have their video on. Are they not going to be allowed to play? So far, no TD I know has been this stringent. But it's worth considering. Especially since "camera off" tends to be contagious. Once people see that it's not really required they start turning theirs off.

Is there anyone out there who has played a Zoom tournament where their connection is good enough to hear the moderator but not good enough to broadcast their video? I've heard of this on Discord but not on Zoom.
I think it's reasonable to say that if you cannot meet the requirements needed to prevent cheating, you should not be allowed to play.
Albert Zhang
Columbia '24
State College '16-'20
User avatar
Sean
Wakka
Posts: 133
Joined: Tue Jun 30, 2020 10:30 am

Re: Wider camera requirements for online quiz bowl

Post by Sean »

So far cheating allegations have been more of an issue at the collegiate level, so that's definitely where the most attention needs to be--I only referred to AQBL because that was an organization that would be conducting a lot of online mirrors that I could name off the top of my head, though perhaps ACF is in a similar situation of potential influence for this matter(?). Applying a policy like this, or any policy really, as a blanket requirement for every main season college tournament is difficult though due to the level of independence a host has when carrying out a tournament. I think the aforementioned technology check is doable because it's less involved, but it would still be hard to enforce hosts to do. The writers for a given set may be in a position to enforce this at all mirrors of that set, but they would be doing so at their own expense.
Sean Farrell
Mandarin High '20
North Florida '23
User avatar
Red Panda Cub
Wakka
Posts: 211
Joined: Thu Dec 22, 2011 9:59 pm

Re: Wider camera requirements for online quiz bowl

Post by Red Panda Cub »

I wonder if there is much marginal gain from this system over the easy to implement system I prefer. Namely: just require players to have at least one hand visible on their webcam during live questions. It's pretty hard to google stuff quickly typing with one hand, and most players can show a hand without any undue faff or setup. Just hold a mug or rest your chin on your hand or whatever. I did this all day at IKEA yesterday and it was not at all inconvenient.
Joey Goldman
Oxford '17
City, University of London '19
rdc20
Lulu
Posts: 48
Joined: Fri Jan 19, 2018 5:42 pm

Re: Wider camera requirements for online quiz bowl

Post by rdc20 »

This thread has been quiet for a few months, but I'd like to add a few thoughts and respond to a some of the points raised earlier.

First of all, iriun, the webcam app that Albert talked about in his original post, is super easy to use. Between installing the app on my phone and installing the desktop app, the entire process took less than five minutes. People should not be put off by the extra device involved; this is really easy to set up.
Mike Bentley wrote: Sun Sep 27, 2020 4:17 pm There is a downside of the tournament being a bit less personable than a camera pointed at someone's face as all you get to see is the back of their head.
Secondly, there is a moderately straightforward way to address this point that Mike raised. With about 10 minutes of poking around OBS, I was able to set up a "virtual camera" that lets me combine video feeds from my laptop's webcam and a second camera, such as a USB webcam or an external cell phone camera connected using one of the methods that Albert discussed in the original post. After setting up the OBS scene to show both video feeds concurrently (I put the video feed showing my hands in the bottom corner of the screen so that my face is still the most prominent thing displayed), you can click "Start Virtual Camera," and you will be able to set this new OBS virtual camera as your video input on Discord, Zoom, or Google Meet for as long as you keep OBS running in the background. This will let moderators and competitors view your hands, computer, and face. This solution sometimes requires a bit of debugging, but setting camera resolutions to 720p and restarting OBS takes care of these problems in my experience. This may also be more demanding for computers with poor graphics processing or other limitations, so I'd welcome input on people's experience using this setup in live calls with other people. OBS gives the option to reduce the frame rate of each video source, so this is an easy way to reduce the load on your computer preemptively.
Joey Goldman wrote: Mon Sep 28, 2020 6:54 am I wonder if there is much marginal gain from this system over the easy to implement system I prefer. Namely: just require players to have at least one hand visible on their webcam during live questions. It's pretty hard to google stuff quickly typing with one hand, and most players can show a hand without any undue faff or setup...
Third, I'd like to talk about Joey's recommendation that players be required to keep one hand visible at all times. My understanding from conversations in discord is that one of the players who was caught cheating at NHBB nationals could easily be seen typing, in part because of the requirement that players keep one hand visible on screen. This is certainly an improvement over how cheating accusations were weighed months ago, but this situation exposed a few shortcomings. First, convincing the right people that this person was in fact cheating seemed to rely on the existence of a recording of a match by another competitor. A more competent organization like PACE, ACF, or NAQT would probably do a better job at handling a similar situation without needing to rely on a recording of the match, but I think that there is still a possibility that cheaters exhibiting similar visual signs of googling answers could slip through the cracks and get away with cheating absent blatant cheating being visible live. My second point is that requirements to keep one hand visible are difficult to enforce and annoying for players to abide by. The moderators at NHBB nationals supposedly did a poor job of enforcing the rule, which I think is understandable given the need for near constant monitoring and the awkwardness that an opponent would face were they to ask the moderator to enforce the rule against a particular player. Again, different hosts and organizations may be better at enforcing this rule. But this rule is honestly unprofessional enough that I would be shocked if the default recommendation of PACE, ACF, or NAQT was that players keep one hand raised at face level for the duration of the competition.

I'd like to make a final point about why I think camera setups that show players' hands and computer screens should be the expectation at regular season events and the requirement at nationals events. Like Albert and others have said, our goal should be to eliminate cheating from online quizbowl, not just to catch people after the fact. Setups that show players' hands and computer screens are more conducive to this goal for a couple of reasons. Firstly, having your hands and computer screen visible makes the very act of cheating online as difficult as it would be at an in-person event. And secondly, rules of this type are much easier to enforce than rules requiring one hand to be visible at all times. All that needs to be done is for the moderator or scorekeeper to confirm that everyone is in compliance with the camera rules at the beginning of each round. There is no need for continuous monitoring, barring any effort during a round on the part of a player to significantly alter what is visible on their camera.

Implementing these measures can protect the integrity of online quizbowl by preventing the cheating that has persisted for months as well as enhance the professionalism and enjoyability of the online quizbowl experience by allowing every player's face to be visible. While I'm disappointed that ACF has decided that Regionals will not serve as a nationals-qualification event, since these measures could raise the integrity of online Regionals to a level sufficient for nationals qualification, I'm sure there are other tournaments at the high school and college level where stringent camera requirements can be implemented. State competitions, PACE NSC, and NASAT will be largely online this year, so I hope the organizations behind these events will consider implementing some of the ideas in this thread for their events. Again, I welcome responses to my points, since I'm sure there is stuff that I missed.
Robert Condron
Strake Jesuit 2020
University of Chicago 2024
User avatar
ValenciaQBowl
Auron
Posts: 2558
Joined: Thu Feb 05, 2004 2:25 pm
Location: Orlando, Florida

Re: Wider camera requirements for online quiz bowl

Post by ValenciaQBowl »

My second point is that requirements to keep one hand visible are difficult to enforce and annoying for players to abide by.
I'm not sure that enforcing a rule that players have to have one hand visible is any harder to enforce than the tech set ups being described above. I'm obviously not going to be playing any collegiate tournaments, but I'd hate to have to try to set all that up (I'm old, it's taken me a while to figure out Discord and Zoom competitions, but this is probably not as onerous to the ute). And I'm not excited about having to monitor all that if and when I read for a NAQT nationals competition. I suppose it could be the job of a dedicated scorekeeper?
Chris Borglum
Valencia College Grand Poobah
User avatar
nickdai
Wakka
Posts: 113
Joined: Fri Mar 13, 2020 10:36 pm

Re: Wider camera requirements for online quiz bowl

Post by nickdai »

ValenciaQBowl wrote: Tue Dec 15, 2020 8:00 pm
My second point is that requirements to keep one hand visible are difficult to enforce and annoying for players to abide by.
I'm not sure that enforcing a rule that players have to have one hand visible is any harder to enforce than the tech set ups being described above. I'm obviously not going to be playing any collegiate tournaments, but I'd hate to have to try to set all that up (I'm old, it's taken me a while to figure out Discord and Zoom competitions, but this is probably not as onerous to the ute). And I'm not excited about having to monitor all that if and when I read for a NAQT nationals competition. I suppose it could be the job of a dedicated scorekeeper?
I’d argue that it is much easier to enforce a camera pointed at a screen than a raised hand. I’ve heard stories about suspected cheaters that only raise their hand in the beginning and then put it down in the middle cause they know no one will check. And even if someone does check they just say they forgot, and that’s probably a very legitimate excuse. My brain stops functioning in the middle of a match and that leads to me also often forgetting to raise my hand for a few tossups, but I don’t cheat. Meanwhile, if someone deliberately turns off their camera in the middle, they can’t just make excuses that they forgot or anything like that.

Having a dedicated scorekeeper check cameras is a good idea, which is what Florida Open did. However, we had the luxury of an abundance of staffers. At most regional tournaments I’ve played this year, half the rooms were staffed by solo mods, and those mods can’t just check cameras live. They can check between tossups but that’s not as effective with regards to the hand raising. In a time where most circuits are having a lack of staffers and can’t double staff rooms, this is probably not a good idea.

The difficulty of the tech setup is a legitimate concern, but I could probably make a step by step video guide and after 1 or 2 tournaments it’ll be much easier. In reality, it’s not as complicated as it sounds, you just have to make sure to download everything beforehand.
Nicholas "Nicc" Dai
戴若涵

Westview High School '21
University of California, San Diego '25
User avatar
LeoLaw
Lulu
Posts: 42
Joined: Sun Feb 12, 2017 10:16 pm

Re: Wider camera requirements for online quiz bowl

Post by LeoLaw »

nickdai wrote: Tue Dec 15, 2020 9:32 pm I’d argue that it is much easier to enforce a camera pointed at a screen than a raised hand.
This is very difficult for people who only use laptop(s) like me. I suppose you can join the zoom/discord call with a phone then point its camera at a screen. I would also argue that in the case where the camera is not stuck on top of a screen it is better to point the camera at your hands, which I saw at least one player doing when observing games at Flopen .

Edit: clearly I didn't read the op when I wrote this. But I still think that having just both hands visible without the screen is good enough and easier.
Leo
New College of Florida '19
University of Florida ‘25
Locked