Guidelines
Note TWO spans in the article that you consider well-supported and unproblematic. To the extent possible, focus on takeaways that are related to the core themes of the article, rather than more tangential details. When marking spans, select a minimal, self-contained segment. For each span, briefly describe in your own words what is the main factual takeaway from it.
Note ALL points in the article that prompt doubt, discomfort, or that require critical reflection to discern their true implications. These are texts where a surface-level reading leads to one takeaway, but a more critical reading would not cause one to not believe the original takeaway with the same level of certainty.
This may include issues such as misleading word choices, logical fallacies, misleading framing, missing but important context, unanswered questions that might have bearing on the interpretation of information in the article, situations where critical information is omitted etc.
For each point, you will annotate:
Span: "The university president said, 'A new scholarship program will begin on Monday.'"
Factual takeaway: A new scholarship program will begin on Monday.
Why? The university president is a credible and relevant authority on this matter. The statement is presented as reliable information about the start of the program, so the takeaway should focus on the program beginning.
Span: "The famous chef said, 'I think traditional cookbooks are becoming obsolete.'"
Factual takeaway: The famous chef said that traditional cookbooks are becoming obsolete.
Why? The relevant fact is that the chef expressed this view, not whether cookbooks are actually becoming obsolete.
Factual Takeaways
Example 1
Relevant span of text from the article
It could be the clearest sign of life that we’ve ever found on Mars, which is incredibly exciting,” acting NASA administrator Sean P. Duffy said.
Takeaway
NASA: rocks found could be clearest sign yet of possible life on Mars
Example 2
Relevant span of text from the article
The study found that people who drank coffee in the morning had a lower risk of dying from cardiovascular disease and had a lower mortality risk than all-day coffee consumers
Takeaway
Drinking coffee in the morning may be better for heart health than drinking it at other times of the day.
Points of Concern
Example 1
Relevant span of text from the article
Shake Shack’s spokesperson added that many of the items can fit into eating patterns often followed by people using GLP-1 medications such as Ozempic, Wegovy and Zepbound, with an emphasis on higher-protein, more balanced meals that do not compromise on flavor.
Potential takeaway
The new products in Shake Shack’s menu are appropriate for people using GLP-1 drugs.
Question to ask
Is this claim supported by experts, or is it just the company’s perspective?
Severity4I mostly do not believe the original takeaway.
Example 2
Relevant span of text from the article
Approximately 7,000 cases were identified across dozens of U.S. states in 2025, according to the CDC, and it has reportedly been identified in at least 60 countries.
Potential takeaway
This fungus has infected many people.
Question to ask
Is this a lot? How does it compare to other infections?
Severity3I am uncertain whether to believe the original takeaway.
Example 3
Relevant span of text from the article
Tate asks a person off-screen to hand him two boxes of pizza... The boxes had Romanian type on the sides, potentially letting officials know he was in the country.
Potential takeaway
Tate was arrested because his exchange with Greta Thunberg revealed his location in Romania.
Question to ask
What evidence does the article provide that authorities identified his location through this video, rather than already knowing he was in Romania?
Severity3I am uncertain whether to believe the original takeaway.
Example 4
Relevant span of text from the article
Entire article
Potential takeaway
"Human Passport" is a necessary solution for ensuring accountability in AI agents.
Question to ask
Why is "Human Passport" mentioned so prominently throughout the article, and does the article provide balanced coverage of alternative approaches?
Severity3I am uncertain whether to believe the original takeaway.
Platform walkthrough
Watch the short video walkthrough first, or use the written steps below as a reference while you complete the task.
Video guide
The video shows the same process described in the written instructions below.
Step 1
Choose the article that is more likely to raise doubts or questions. To make your selection, click “Click to choose this article” at the top of the article.
Step 2
At the top of the page, type your ID into the Participant ID field before you start annotating.
Step 3
The article appears on the right. For both factual takeaways and points of concern, select the exact sentences or spans you want to annotate.
Step 4
The selected text appears on the left. Choose Factual Takeaway or Point of Concern, complete the required fields, and click Save annotation.
Step 4
When you are done annotating the current article, scroll to the bottom and click Finish article and continue to move to the next article. After you finish every article, the final submission button becomes available so you can submit all of your annotations.
Article choice
Select one of the three articles below. Use your professional judgment to choose the article that appears more likely to raise doubts or questions. The article you choose will be opened for annotation.
Article text will appear here when the task loads.
No annotations yet.
Complete
Guidelines
Platform walkthrough
Before you begin
Edit annotation