ID,Starting URL,Category,Difficulty,Task,HITL_Operator_ID,operator_hitl_eval,operator_hitl_reasoning,operator_hitl_link
2,https://www.acehardware.com,READ,easy,"Search for ""LED light bulbs"" on AceHardware.com and provide the titles and sale prices of the first 5 products that appear.
Only use http://acehardware.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-1,Success,Navigates and finds correct items via search,https://www.loom.com/share/85352b1277d84363b16101550ee27766
19,https://www.airbnb.com,DELETE,easy,"Log in to your Airbnb account, save a Guest Favorite property to your wishlist, and then go to your wishlist and remove the property you previously added.
Only use http://airbnb.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-4,Success,"AI Agent successfully selected guest favorite property listing, saved it to wishlist, and deleted it with confirmation from human.",https://www.loom.com/share/d72f7868dc2e4172bbc020fbfe17cbdc?sid=06ba2a9b-89c1-4b29-982d-afd857147870
20,https://www.airbnb.com,DELETE,hard,"Log in to Airbnb, send a query to a host of a New York City property, and then delete your message confirming that the message does not appear on your profile.
Only use http://airbnb.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-3,Failure,"AI Agent failed to delete. could not find delete option. Instead chose to archive the whole message, failing task. ",https://www.loom.com/share/60832434f452439c8e1d9fc12ab3ea4c
22,https://www.airbnb.com,READ,easy,"Browse the “Play” section in Paris and extract the titles and brief descriptions of the top five properties.
Only use http://airbnb.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-2,Success,Agent succesfully goes and finds options and clicks in to extract info from each listing,https://www.loom.com/share/93627d5d5cd543edbc98f6372f711dcc
33,https://www.alibaba.com,READ,easy,"Search for ""smartphones"" on Alibaba and note how many listings are available.
Only use http://alibaba.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-7,Success,AI Agent successfully locates a number for the amount of listings of smartphones on Alibaba.,https://www.loom.com/share/bb3f8b6626124e35a42b862c35bf2e02
35,https://www.alibaba.com,READ,hard,"Search for ""RFID tags"", filter by a minimum order quantity of less than 1000, and list the first 3 supplier names.
Only use http://alibaba.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-5,Success,"AI Agent successfully completed the task. Interesting to note, Agent first searched by category instead of going to search bar, but corrected. Additionally, Agent answer included the same supplier twice, as it was listed with a different product. Answer is technically correct. ",https://www.loom.com/share/c64d0336840e41f4bee5f5b690a059e7
51,https://www.allrecipes.com,CREATE,easy,"Log in, write a detailed review for the ""Lemon Chicken Orzo Soup"" recipe including a 5-star rating and your comments on taste and presentation.
Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-8,Failure,"Hits ""Signal - Not Acceptable"" error when navigating into the webpage for the recipe. Tries navigating to incognito and via search but both paths fail",https://www.loom.com/share/e2d32b9e72d34c0788a85fadd95a9718
56,https://www.allrecipes.com,CREATE,easy,"Log in, create a new recipe collection titled ""Quick Weeknight Dinners"" and populate it with 5 chosen recipes.
Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-11,Failure,"Agent created a new collection titled correctly. Agent checked back with Favorites Tab to check if it had properly favorited an item before continuing to search for recipes. Agent added 4 recipes, and noticed that there was only four recipes in the collection. Agent misidentified the missing recipe, and attempted to add that recipe again. The agent checked for that specific recipe to be in the collection, yet ignored checking that there were 5 recipes in the collection overall. Thus, it misidentified the number of recipes, and populated a collection with 4 recipes instead of 5. ",https://www.loom.com/share/39cbb83e3c90456781acae3726f934f7
63,https://www.allrecipes.com,READ,easy,"Search for a seasonal, fall-appropriate recipe and note the recommended serving size mentioned.
Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-10,Success,"Agent successfully found a serving size for a recipe designated for the fall time of the year. Interestingly, the Agent identified Halloween as a Fall time of year as opposed to searching in the search bar for ""Fall"".  ",https://www.loom.com/share/4cd22d81732c4d69be2ca3f4ffe7c581
67,https://www.allrecipes.com,READ,easy,"Locate a ""chocolate chip cookie"" recipe page and summarize the calorie count from its nutritional information.
Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-9,Success,"Agent successfully navigated and found the calorie count. Notably, Agent could have clicked ""Jump to nutritional facts"" but scrolled instead. ",https://www.loom.com/share/b1be71530e1043ce98cf0c5c1b80c7a9
73,https://www.alltrails.com,DELETE,easy,"Log in to your account, add the ""Canyon Loop"" trail to your saved trails, and then delete it—confirm that it no longer appears in your list.
Only use http://alltrails.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-12,Success,"Agent saved the trail and successfully deleted it. Agent needed human assistance to log in, as well as navigate to browser settings to allow third-party cookies for log-in portal. Agent also needed human assistance to confirm the save and delete functions.",https://www.loom.com/share/da0184a08aa347938290fc0f21e9b25d
80,https://www.alltrails.com,UPDATE,easy,"Log in to your account and update your adventure profile by changing your favorite activity from ""hiking"" to ""mountain biking.""
Only use http://alltrails.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-13,Success,"Agent selected ""hiking,"" as it wasn't previously saved as a favorite. Then it changed its response to ""mountain biking,"" constituting a success.",https://www.loom.com/share/bd57ff75e59f4e20a4045c1b3a4d81f1
93,https://www.apa.org,READ,hard,"Use the website’s search function to find articles published in 2022 related to ""clinical psychology"" and output the titles of the first four matches.
Only use http://apa.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-15,Success,"Agent filters for all the necessary criteria, and lists the desired output. Agent first searched for the titles within the year 2022, and then narrowed the search by filtering for articles alone.",https://www.loom.com/share/138e4d0f4d9c4ce4828cbdef5d7ee6d8
102,https://apkpure.com,READ,easy,"Search for the app ""Spotify"" on APKPure and list the latest version number along with its release date as shown on the version history section.
Only use http://apkpure.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-18,Success,Agent was able to find the necessary details for Spotify's version release date and latest version number. Agent needed human assistance to complete CAPTCHA verification.,https://www.loom.com/share/79b47ee0d25846e5b28a6386d73dc134
104,https://apnews.com,READ,easy,"Use the website’s search bar to look for articles containing ""climate change"" and list the titles of the first five articles returned.
Only use http://apnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-19,Success,"Agent was able to successfully return the top article names for ""climate change"" search.",https://www.loom.com/share/84350e0bf560465682535f1acf37abbe
113,https://archive.org,READ,easy,"Use advanced search for ”Space images"" on archive.org and output the capture dates and titles of the first 10 images listed.
Only use http://archive.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-20,Success,"Agent successfully navigated to the Advanced Search function and input ""space"" as the description field, and media type as ""image"". Then it returned the capture date and titles of the first ten images. Agent did require two searches and approval from human to proceed with searches.",https://www.loom.com/share/3948c0065c4f4d9295d73f72d90d1800
125,http://www.barnesandnoble.com,DELETE,easy,"Log in, add the physical book ""To Kill a Mockingbird"" to your wishlist, then remove it and verify its deletion.
Only use http://barnesandnoble.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-23,Success,"Agent requested human assistance to proceed with security risk, and also log in. It identified the proper book, asked for confirmation to add it to the wishlist, and checked to make sure it was in the wishlist. Agent struggled to add the book to the wishlist on its first 3 attempts(figuring out which functions add the books to the wishlist), and had to re-search the book a few times. It then properly identified the method to add books to the wishlist and with confirmation, proceeded. Then Agent asked for confirmation to delete it before completing. ",https://www.loom.com/share/28d123708a074a6c8d788714df11e925
129,http://www.barnesandnoble.com,READ,easy,"Use the store locator by entering ZIP code 90210, then display the address, store hours, and contact details of the nearest location.
Only use http://barnesandnoble.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-22,Success,Agent successfully identified the closest B&N to zip code 90210 when presented with multiple options.,https://www.loom.com/share/a5512e8360ba4b99bdcaf99c86a3b745
139,https://www.bbb.org,READ,easy,"Search for any companies on bbb.org and output the BBB rating of the first business listed in the results.
Only use http://bbb.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-24,Success,"Agent did successfuly find the first restaurant listed and identified where the rating would be, but this specific restaurant did not have a rating. It asked if it should keep looking, and after browsing more, decided to filter for restaurants with ratings. The drawback of this method/question is that the BBB rating filter only allows you to select one rating at a time, so the Agent was only able to find restaurants with the given rating that it selected in the filter. ",https://www.loom.com/share/e73525e3052a440597389d10a88c6d5d
140,https://www.bbb.org,READ,hard,"Search for any businesses in Los Angeles with a BBB rating of A+ and list the names of the first five businesses displayed.
Only use http://bbb.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-25,Success,Agent was able to successfully identify the first 5 businesses listed with an A+ rating,https://www.loom.com/share/0d63cc5a6aaf4e5c819d8d5e57ff3ebe
149,https://www.bestbuy.com,CREATE,easy,"Log into your Best Buy account and create a new wishlist titled ""Holiday Shopping,"" then add the ""Sony 65-inch 4K Ultra HD TV"" to that wishlist.
Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-29,Success,Agent was able to create a wishlist in the account and add the desired TV to the wishlist. Agent needed human assistance to log in several times and would frequently get logged out. Agent also misclicked multiple times before successfully saving the desired TV.,https://www.loom.com/share/88f9901daf4e4431b40ac59835212350
151,https://www.bestbuy.com,CREATE,easy,"On the ""Sony 75-inch 4K TV"" product page, post a new question in the Q&A section asking, ""Does this TV support VESA wall mounting?""
Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-26,Failure,Gateway error,https://www.loom.com/share/52c28e65062149fa90e974d254b6dfa1
159,https://www.bestbuy.com,READ,easy,"Locate the guide titled ""How to Choose the Best 4K TV: The Essential Guide"" and summarize the top 3 tips provided.
Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-27,Failure,"Agent did not source the guide directly from BestBuy; rather, Agent looked up the name of the guide in a separate tab on Bing. Agent also tried to identify tips by searching for tips as a key word, and then tried to locate a Help or How-To section. Lastly, it summarized 3 succinct tips for selecting a TV.",https://www.loom.com/share/2367be1048104d73a0c10d5cd6de7741
162,https://www.bestbuy.com,UPDATE,hard,"Log into your account, add an item to your cart, then switch the delivery option from standard shipping to in-store pickup and confirm that the order details update accordingly.
Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-28,Success,Agent successfully added the item to the cart for delivery and changed it to pickup at the nearest address.,https://www.loom.com/share/a5d861d524c9459fad82b3c562182286
172,https://www.booking.com,READ,hard,"Search for 8 review score or above hotels in Manhattan, New York for next weekend with free cancellation, free Wi-Fi, and breakfast included. Compare at least four options and pick the one closest to the Empire State Building.
Only use http://booking.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-30,Success,"Agent was able to find two hotels that meet the desired criteria. However, Agent's method was not the most efficient, as it filtered for hotels with a balcony at one point, among other inefficient search criteria. ",https://www.loom.com/share/8b64cc7fc3794d7093cb0151cf7a161d
180,https://www.buzzfeed.com,CREATE,easy,"Using the Community submission portal, draft a BuzzFeed listicle titled “Top 10 Internet Memes of 2023” that includes five list items, each with a short description.
Only use http://buzzfeed.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-33,Success,"Agent created and saved the post successfully. Agent asked to include thumbnail, which human told agent to continue without. Agent also needed assistance to log in.",https://www.loom.com/share/3dc0b5de3c6546f399fbf63748e19dcb
183,https://www.buzzfeed.com,DELETE,hard,"Log in, post a comment on one of the front page news articles on BuzzFeed, then locate this previously posted comment and delete the comment, ensuring it is removed from the article page.
Only use http://buzzfeed.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-34,Success,"Agent successfully commented and deleted its comment, and requested approval from human before commenting and deleting.",https://www.loom.com/share/1e5957bb1324419e9ca9d85b61902ea5
198,https://www.canada.ca,READ,easy,"Navigate to the COVID-19 information section on Canada.ca and note three public health guidelines currently recommended.
Only use http://canada.ca to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-36,Failure,"Agent noted two guidelines rather than outlining three distinct ones. One could argue that the agent noted more than three, but in either case, agent failed to outline three distinct guidelines.",https://www.loom.com/share/a86c6c1e8f184fabb0e33bda6cc392f7
206,https://www.caranddriver.com,DELETE,hard,"Log in to your Car and Drive account, save and article to your bookmarks, then delete the previously bookmarked article from your ""My Bookmarks"" list, and confirm that the article is no longer stored.
Only use http://caranddriver.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-37,Success,"Agent was able to identify an article, bookmark it, and then unbookmark it and confirm. However, it needed human assistance to verify all of these actions. ",https://www.loom.com/share/1262049f0dc8478f95a3c78934473e7e
209,https://www.caranddriver.com,READ,easy,"Navigate to Car and Driver’s homepage and list the titles of the top three featured car reviews.
Only use http://caranddriver.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-39,Success,Agent was able to find 3 featured reviews on the front page.,https://www.loom.com/share/4f73dbc16010432390ac460edd4386c1
213,https://www.cars.com,READ,easy,"In the ""New Cars"" section, check the pricing for 2024 BMW 3 Series 330e and list at least two pricing components (e.g., MSRP, incentives) that are displayed.
Only use http://cars.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-40,Success,Agent was able to find multiple pricing/incentive components for the BMW series 3 330e.,https://www.loom.com/share/49099a8e9d944d52ade7f37246b3af13
220,https://www.cbr.com,READ,easy,"Navigate to the CBR homepage and list the titles of the top five most recent articles in the news section.
Only use http://cbr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-42,Success,Agent was able to locate and report 5 top articles from the news section.,https://www.loom.com/share/38c7e87875204c9ba2e37a897d813fbf
221,https://www.cbr.com,READ,easy,"Navigate to an article featuring an embedded movie trailer and extract its title, description, and the link to the full trailer (if available).
Only use http://cbr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-41,Failure,Agent consistently tried to play video when it should have collected the link to the videos instead.,https://www.loom.com/share/67f242fa08154ddf85941e5f1dc3f582
222,https://www.cbr.com,READ,easy,"Use the site’s search function to look up ""Star Wars"" and list the first five article titles with a brief note on each article’s focus.
Only use http://cbr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-43,Success,Agent successfully found Star Wars-related articles and summarized them efficiently.,https://www.loom.com/share/0da9b1fea9854df9a52b7480e53f1deb
228,https://www.cbssports.com,READ,easy,"Check the live stream schedule and list the next two sports events along with their start times and the channels on which they will be broadcast.
Only use http://cbssports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-44,Success,Agent successfully navigated through CBS Sports and found the next two events streaming.,https://www.loom.com/share/81b96de835d246d6bf13ad15083c416c
231,https://www.cdc.gov,READ,easy,"Find the latest measles advisory issued by the CDC, then note the publication date along with a brief summary of the advisory.
Only use http://cdc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-45,Success,Agent successfully identified the most recent advisory and summarized it succinctly.,https://www.loom.com/share/91dc8d0139ee482086b1519694a3598d
236,https://my.clevelandclinic.org,READ,easy,"Navigate to the Health Education section and list the top 5 articles on heart disease available on the Cleveland Clinic website.
Only use http://clevelandclinic.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-46,Success,Agent successfully identified the top 5 most relevant articles on heart disease as prompted.,https://www.loom.com/share/c509ec8303b6495680a001a68298f94b
247,https://www.cnet.com,READ,easy,"Search for a CNET column on artificial intelligence and list any recommended products that the article highlights.
Only use http://cnet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-50,Failure,"Agent failed to access the site, and human could access on same browser. ",https://www.loom.com/share/a9ecb32027af465bb06317193e2ff5bd
249,https://www.cnet.com,READ,easy,"Browse any Deals or Offers section available and list the details of at least one affiliate deal mentioned.
Only use http://cnet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-47,Failure,"Agent failed to access the site, and human could access on same browser. ",https://www.loom.com/share/46abbfeba2da45e885a1809d1366f8a4
256,https://collider.com,DELETE,hard,"Log into your Collider newsletter subscription settings, ensure that communications have been subscribed to, and unsubscribe from all communications, then verify that a confirmation message is displayed.
Only use http://collider.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-52,Success,Agent managed to unsubscribe to all notifications successfully.,https://www.loom.com/share/54916f41fbf5427ebdcdccf57ff132bb
261,https://collider.com,READ,easy,"Search for Collider’s coverage on a war film and provide a summary of the review highlights.
Only use http://collider.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-51,Failure,"Agent used the browser search function rather than the site search function, and also left the site to search bing before returning to the website.",https://www.loom.com/share/2ae30e7b3e914f2a9cf875446f02c1d0
264,https://www.collinsdictionary.com,READ,easy,"Look up the word ""onomatopoeia"" and extract the main definition and one usage example from the Collins Corpus.
Only use http://collinsdictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-53,Success,Agent managed to find a definition and usage of the word onomatopoeia.,https://www.loom.com/share/6e506344b31a49d591eda6fe9267500b
266,https://www.columbia.edu,READ,hard,"Use the search functionality to locate pages detailing tuition and fees, then extract the published tuition fee information for undergraduate programs.
Only use http://columbia.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-54,Success,"Agent managed to find a multitude of information on Columbia's tuition pricing, and provided a link as well. ",https://www.loom.com/share/fb921d5142914eeb8734f426dbfde0b1
267,https://www.columbia.edu,READ,easy,"Identify and list the dates for the upcoming Columbia University campus events from the events calendar.
Only use http://columbia.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-55,Success,Agent identified a plethora of upcoming events listed on the Columbia website.,https://www.loom.com/share/8196ada8a6fb474480e74aa90f8e6e67
269,https://www.commonsensemedia.org,READ,easy,"Use the search feature to find the expert review of the animated film “Frozen” and extract its recommended age range and one key critique.
Only use http://commonsensemedia.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-56,Success,Agent identified a review include age range for the movie Frozen as well as a key critique.,https://www.loom.com/share/9168436354d74afebaa4eb104eb20cfe
292,https://newyork.craigslist.org,CREATE,easy,"Log in to Craigslist, search for a baby stroller in your area in the ""for sale"" section, and save it to your favorites.
Only use http://craigslist.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-57,Success,Agent was able to identify a stroller in human's area and save it to favorites and confirm.,https://www.loom.com/share/d0f26c072dd7457c9318af3bd7013aa9
299,https://www.crunchbase.com,CREATE,hard,"Log in to Crunchbase and create a new list titled ""Tech Unicorns,"" then add the companies Uber, Airbnb, and Stripe to the list.
Only use http://crunchbase.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-65,Failure,Agent got blocked from accessing crunchbase.,https://www.loom.com/share/02ecc12a4a604f75999a8f2a503ba2fc
306,https://www.crunchbase.com,CREATE,easy,"Add ""Tesla Inc."" to your followed companies list so you receive future updates on funding and corporate changes.
Only use http://crunchbase.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-63,Failure,Agent got blocked from accessing crunchbase.,https://www.loom.com/share/e09d1209d3014b00a3bba87b5942efd1
308,https://www.crunchbase.com,CREATE,easy,"Create a search for companies in Los Angeles and filter for companies in the E-commerce sector and save the updated query.
Only use http://crunchbase.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-62,Success,Agent was able to identify Los Angeles companies in the e-commerce sector. Agent notably passed CAPTCHA verification without human assistance.,https://www.loom.com/share/92bc9f263e624e3baf9d0baa9ff530e2
323,https://www.crunchyroll.com,CREATE,hard,"Access your account and set up a watchlist called ""Classic Anime"" by adding at least four classic anime series from the catalog.
Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-71,Success,Agent successfully created list with 4 Anime shows that it located.,https://www.loom.com/share/500982124b3648569659f01dba50fa33
328,https://www.crunchyroll.com,DELETE,easy,"Sign in to your profile, create a new watchlist titled ""Action-packed Series,"" then remove the watchlist titled ""Action-packed Series,"" confirming its deletion from your account.
Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-67,Success,"Agent was able to create the list and delete it, both requiring human approval to do so. ",https://www.loom.com/share/b02c3a4e547e4d15afe52c31f4a3d54b
331,https://www.crunchyroll.com,READ,easy,"Visit the news or blog section and summarize the details of an upcoming Crunchyroll event.
Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-68,Success,"Agent successfully found and reported an upcoming Crunchyroll event including dates,",https://www.loom.com/share/00b0b0259f204bf38d6de46be226428a
332,https://www.crunchyroll.com,READ,easy,"Navigate to the details page of the anime ""One Piece"" and extract the available language and subtitle options.
Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-66,Success,Agent was able to successfully locate and report the possible language options.,https://www.loom.com/share/449c22aa6baa472cbd696e42a163e550
334,https://www.crunchyroll.com,UPDATE,easy,"Log in to your account, create a ""Weekend Binge"" watchlist and add ""Attack on Titan."" Then open the ""Weekend Binge"" watchlist, and update it by adding the anime ""Hunter x Hunter"" to the list.
Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-69,Success,"Agent successfully created the playlist, and then added both tv shows to update it.",https://www.loom.com/share/4e1ca68f414b4c81bccc694349c67996
364,https://www.dickssportinggoods.com,CREATE,easy,"Register a new user account on Dick’s Sporting Goods by filling out the required personal information and confirm the registration via the on-screen confirmation message.
Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-74,Failure,Agent was not able to proceed with mock information within a reasonable time - user could have completed the task much quicker with minimal wait.,https://www.loom.com/share/1db5725203ee4b3cbe476f78abba1179
368,https://www.dickssportinggoods.com,CREATE,easy,"Log in, write a detailed product review for a Wilson baseball glove including a star rating, a title, and your experience with the product, then submit the review.
Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-77,Success,Agent was able to add a review and submit with all qualifications outlined in the prompt.,https://www.loom.com/share/6f5fd9e56446493b8261871c22475fa8
375,https://www.dickssportinggoods.com,UPDATE,easy,"Add a pair of Nike running shoes to your cart, then go to your cart and modify the quantity of the Nike running shoes from 1 to 2.
Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-76,Success,"Agent was able to add a random shoe, and then up the quantity to 2 in the cart.",https://www.loom.com/share/311482f6e0fd4d8cb60fa14942d836a7
376,https://www.dickssportinggoods.com,UPDATE,easy,"Log in to your profile and update your address by adding your new apartment details.
Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-75,Success,"Agent was able to update users address, with much help from human input",https://www.loom.com/share/e2b9b25e9c744b0889bcb842c75ad6cc
377,https://www.dictionary.com,READ,easy,"Look up the word ""pronunciation,"" play its audio sample, and note the phonetic transcription displayed.
Only use http://dictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-79,Success,Agent was able to play the audio source and display the phonetic transcription.,https://www.loom.com/share/bb555d03e16b488786aae1166305f5a2
378,https://www.dictionary.com,READ,easy,"Look up the word ""quintessential"" and record both its pronunciation guide and the available audio sample.
Only use http://dictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-78,Failure,Agent was unable to record or download the audio file for the pronounciation of the word.,https://www.loom.com/share/beb87dbf0a3f4d808d132cb23ce9a87f
391,https://www.dw.com,READ,easy,"Visit the DW homepage and list the headline and publication time of the top news article featured in the main section.
Only use http://dw.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-80,Success,Agent was able to navigate to the lead article and return the title.,https://www.loom.com/share/2964071f21e0484baca64e7b16e870c7
397,https://www.eater.com,READ,easy,"Locate a feature article on international dining scenes and list two featured cities mentioned in the piece.
Only use http://eater.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-81,Success,Agent was able to find an article that mentioned multiple eating destinations.,https://www.loom.com/share/1f941216787e40ee812c2534f8bd9583
403,https://www.ebay.com,READ,hard,"Find three different sellers offering the same ""Blue Tooth Speaker"" item. Compare the total cost (including shipping and taxes), seller ratings, return policies, and estimated delivery times. Create a table to compare the various speakers and recommend the best purchase option if I'm purchasing from New York City, NY.
Only use http://ebay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-82,Success,Agent was able to successfully find and compare the same speaker sold by three different vendors with all qualities required.,https://www.loom.com/share/757118633b2d490e94c3d01363499b1c
412,https://economictimes.indiatimes.com,READ,easy,"Navigate to the Markets section and record the current BSE and NSE index values as displayed on the dashboard.
Only use http://economictimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-83,Success,Agent was able to successfully navigate and find the correct index values as tasked.,https://www.loom.com/share/5245862c0a2f41d7ba984ba73f68f9f4
417,https://www.elle.com,READ,easy,"Use the site’s search function to find articles about “Sustainable Fashion” and record the publication dates of the top 3 results.
Only use http://elle.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-85,Success,Agent was able to find three articles that were relevant and pertinent when searching for sustainable fashion.,https://www.loom.com/share/bae3d8c7c9c6430499d438e28d75678f
418,https://www.elle.com,READ,easy,"Identify the digital magazine section and record the publication date of the most recent issue available.
Only use http://elle.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-84,Failure,"Agent timed out - 10 minutes to find a publication was more than enough time to navigate the website, and users would likely search themselves instead of tasking agent. ",https://www.loom.com/share/65565384a21a4e4786284617cdf2256f
420,https://en.wikipedia.org/wiki/Main_Page,READ,easy,"What does the 'Climate Change' article say is driving the current rise in global temperatures?
Only use http://en.wikipedia.org/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-86,Success,Agent was able to navigate Wikipedia and find a proper explanation for the question posed.,https://www.loom.com/share/b33585301d194478b855cf802bf8662b
421,https://en.wikipedia.org/wiki/Main_Page,READ,easy,"Select the 'Page information' option under the 'Tools' dropdown to see how many page views the 'COVID-19' article has in the past 30 days.
Only use http://en.wikipedia.org/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-87,Success,Agent was able to find the correct number of page views as instructed.,https://www.loom.com/share/49d16910832a45968204d07aa43f68f4
424,https://www.encyclopedia.com,READ,easy,"Use the search bar to find and list the title along with the publisher of the encyclopedia entry on ""Climate Change.""
Only use http://encyclopedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-88,Success,Agent was able to find and identify the publisher of the page on climate change.,https://www.loom.com/share/cb915cc84f2247e4910d34b66f7b1b7a
425,https://www.encyclopedia.com,READ,easy,"Search for ""Artificial Intelligence"" and list any related news or magazine articles or media referenced on the entry.
Only use http://encyclopedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-89,Success,"Agent was able to identify the resources section, although it took the agent several tries to identify where on the page it was. It was also able to distinguish between news articles and books.",https://www.loom.com/share/a0912cb6c0b746158fb8ad6011f78f04
430,https://www.eonline.com,READ,easy,"Search for articles mentioning ""Beyoncé"" and provide the headlines and publication dates for the first three results.
Only use http://eonline.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-90,Success,Agent was able to find and list 3 articles related to Beyonce.,https://www.loom.com/share/4727f42511514081a3f8553a4b57c304
432,https://www.epa.gov,READ,easy,"Find the latest update on air quality data for Los Angeles on the EPA’s Air Quality System page and record the reported index value.
Only use http://epa.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-92,Failure,"Agent struggled to locate Los Angeles, and scrolled too far to the point where ""select a city"" was off the screen, and it started to try and select a county. This was inconvenient for the user, so designated this task as a ""fail.""",https://www.loom.com/share/4ee8fe23b9e04c379aec8472e2c5e4b0
433,https://www.epa.gov,READ,easy,"Locate the SmartWay program page and list two benefits that it offers for businesses.
Only use http://epa.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-91,Success,Agent was able to scan the page and list two relevant benefits for businesses.,https://www.loom.com/share/097e21b69661497ca51b11b0f12c34d0
445,https://www.esquire.com,DELETE,hard,"Create a comment on a recent article as described in a previous task, then delete that same comment and verify that it is no longer visible.
Only use http://esquire.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-98,Failure,"Agent got logged out twice, and struggled to post the comment multiple times. Agent notably input a username longer than 15 letters, which was the maximum character limit, and struggled to reduce the count to only 15 characters.",https://www.loom.com/share/e824bbbe7ae04027a8fb45e016d47c56
446,https://www.esquire.com,READ,hard,"Navigate through the archive by decade and note the publication year of one highlighted article from each decade (e.g., 1980s, 1990s, 2000s, etc.).
Only use http://esquire.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-95,Success,Agent found an article from each of the decades required.,https://www.loom.com/share/cf4d5c1cacf64dc2a86c9142ad7403d4
449,https://www.esquire.com,READ,easy,"Navigate to the Esquire homepage and list the headlines of the top 3 featured articles.
Only use http://esquire.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-93,Success,Agent managed to find the top 3 features articles on the website.,https://www.loom.com/share/25a2efd933534b9c82b7ec7498954bf5
453,https://www.etsy.com,READ,easy,"Sign in to your Etsy account using your email and password.
Only use http://etsy.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-99,Success,"Agent was able to log in, with human guidance on which account to use. ",https://www.loom.com/share/26f183a04728493795e08fd0e21e727d
460,https://www.eventbrite.com,READ,easy,"Access the Eventbrite Help Center to locate guidelines on organizing virtual events; then list the key steps and recommendations provided.
Only use http://eventbrite.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-100,Success,Agent was able to access the help center and find the necessary guidelines as tasked.,https://www.loom.com/share/ba891de7228f41d6bc4b47a9a2cc4899
481,https://www.expedia.com/,READ,easy,"Search for flights from New York’s JFK to Los Angeles’ LAX on June 15th with a return on June 20th, and list the top three cheapest fares.
Only use http://expedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-103,Success,Agent was able to find and list three flights as desired.,https://www.loom.com/share/29d80c80a38a4018ada37ec7896651fb
488,https://www.facebook.com,CREATE,hard,"Log in and set up a new Facebook Page for a fictitious business called “Eco-Groceries” complete with a short business description.
Only use http://facebook.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-108,Failure,Agent was unable to create the page after an error of multiple failed attempts to create a page.,https://www.loom.com/share/399e3a4c5bad44469cf74499a5656fb3
495,https://www.facebook.com,CREATE,easy,"Log in, go to the Videos section to see Reels, and save the first reel that will appear on the page.
Only use http://facebook.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-111,Success,Agent was able to navigate to reels and save the first reel it encountered.,https://www.loom.com/share/a595266e0e9b4b6982e24534afcc0206
505,https://www.fandom.com,READ,hard,"Find an image posted to the Stranger Things Wiki forum and identify the dimensions of the largest downloadable option
Only use http://fandom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-112,Success,Agent was able to locate an image and provide a link to its dimensions. Agent needed human confirmation to move to the next image after the first image did not provide its dimensions.,https://www.loom.com/share/747a723f645f4190ba5140e84872b072
508,https://www.fda.gov,READ,easy,"Locate the section dedicated to FDA-regulated foods and extract the risk factors listed in the Food Safety sub‐section.
Only use http://fda.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-113,Failure,Human timed the agent out due to inconvenience of time taken by agent to figure out task.,https://www.loom.com/share/0b2a0d95155c4fabbb7273ba20d945aa
517,https://www.firstcry.com,READ,easy,"Search for baby strollers on FirstCry and list the prices, key features, and customer ratings of the top 3 models displayed.
Only use http://firstcry.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-115,Success,Agent stalled for a bit but then was quickly able to identify three strollers as tasked.,https://www.loom.com/share/c615374eded1429493b67b77fbb662d9
519,https://www.firstcry.com,READ,easy,"Browse the maternity care products section and extract details (price, customer rating, and availability) for the ""Maternity Support Belt.""
Only use http://firstcry.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-117,Success,"Agent was able to find a ""maternity support belt"" and extract pricing and additional details.",https://www.loom.com/share/9e70f36e9647421aad3501fc7764fd4e
521,https://www.firstcry.com,READ,easy,"Check the ""FirstCry Prime"" membership page and note down the benefits offered, including free shipping and early sale access details.
Only use http://firstcry.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-114,Success,"Agent was able to identify that Firstcry Prime had changed to Firstcry Plus, and provided benefits on it.",https://www.loom.com/share/fac08f80b2664778b77715b54bf00cca
537,https://flickr.com,READ,easy,"Search Flickr for photos tagged ""sunset"" and list the titles and usernames of the first 5 results.
Only use http://flickr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-120,Success,"Agent was able to return the names of the first few titles labelled ""Sunset.""",https://www.loom.com/share/edf3b136bfe148dca626537dd56e78d6
538,https://flickr.com,READ,easy,"Open the details of a selected photo and extract its metadata details (title, description, and tags) as displayed on its page.
Only use http://flickr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-121,Success,Agent was able to extract metadata details of the photo it selected.,https://www.loom.com/share/18dd9b1455e3439795564e4e2609f51d
543,https://www.flipkart.com,READ,hard,"In the ""Laptops"" section, apply the filter for ""Dell"" and extract the average discount percentage on the first 3 Dell laptops displayed.
Only use http://flipkart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-124,Success,"Agent was able to aggregate the total of the three discount it saw, and averaged it out. ",https://www.loom.com/share/0e9958e2a67a44dd96a951f42b11c7f7
545,https://www.flipkart.com/,READ,hard,"Navigate to the ""Mobiles"" category, search for ""iPhone"", and list the product names, prices, and discount details of the top 5 search results.
Only use http://flipkart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-123,Success,Agent was able to locate and report top 5 results including price and other details as asked.,https://www.loom.com/share/ce2bf908e90c4c939adfac9137198b09
552,https://food52.com,READ,hard,"Log in, use the community forum search on Food52 to find posts tagged with “grilling” and list the titles of the first three posts along with the respective usernames of the posters.
Only use http://food52.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-127,Success,"Agent found the first three post titles and usernames related to ""grilling.""",https://www.loom.com/share/9c629e4960074aa2a2a26ed050947e86
553,https://food52.com,READ,easy,"Browse the “A Few of Our Faves” section and list three recipes featured this week, including their titles.
Only use http://food52.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-125,Failure,"Agent was not able to find the ""A few of our faves"" section. Human timed it out, because agent was headed down an incorrect path that was unlikely to succeed.",https://www.loom.com/share/953b2059719c459880db4f0080aa1d11
554,https://food52.com,READ,easy,"Use the Food52 search function to find a recipe that includes both lemon and thyme in the ingredients, then list the step-by-step cooking instructions of the first matching result.
Only use http://food52.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-126,Success,Agent found a recipe with lemon and thyme ingredients and listed out the steps to make it.,https://www.loom.com/share/8e5452cae0da45fcb9c17dc0352b4fc3
564,https://www.ford.com,CREATE,hard,"Establish a new saved configuration for a Ford Mustang Mach-E with added options such as an extended battery and panoramic sunroof, then note the generated build reference number.
Only use http://ford.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-130,Success,"Agent was able to create the custom build and save it to the collection, and return the build reference as a url.",https://www.loom.com/share/9f42fa69206c4df49514eb2e08dbfcf4
565,https://www.ford.com,CREATE,hard,"Utilize the financing calculator tool to simulate a Ford Credit financing option for a Ford Fusion, and output the estimated monthly payment along with the term details.
Only use http://ford.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-129,Success,Agent was able to report out a credit financing option for a Ford Fusion including monthly payment and term details.,https://www.loom.com/share/83139f880a1641e4a8197440377c29c3
567,https://www.ford.com,DELETE,hard,"Login, create a new saved vehicle configuration, then access your saved vehicle configurations and delete it, and confirm that it is no longer present in your saved builds list.
Only use http://ford.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-128,Success,"Agent successfully designed and saved a build, and proceeded to delete it afterwards. Agent asked for human to submit zip code and log in.",https://www.loom.com/share/0645b4da79f64544bf4ba0bff6aa9e09
574,https://fortune.com,READ,easy,"Navigate to the Fortune homepage and list the titles of the top 5 featured business articles.
Only use http://fortune.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-132,Success,Agent was able to find and return the titles of the top 5 features business articles.,https://www.loom.com/share/819fe591f0a8460199c7e89b9ed01ccc
577,https://www.foxnews.com,READ,hard,"Use the website’s search function to find an article related to the ""US Economy"" and extract the main points from the introduction.
Only use http://foxnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-134,Success,Agent was able to find an article related to US economy and summarize the main points from the introduction.,https://www.loom.com/share/486e862425cc49e4ad39f823560dd2f9
578,https://www.foxnews.com,READ,easy,"Retrieve an archived article on ""immigration policy"" from the site’s archive; summarize its publication date and primary focus.
Only use http://foxnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-133,Success,"Agent was able to find an archived article about ""immigration policy.""",https://www.loom.com/share/2f259f6f60f446ee920aa0da0171f4a2
582,https://www.foxsports.com,READ,easy,"Browse the video highlights section and list the titles of the five most recent NBA highlight videos.
Only use http://foxsports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-135,Success,"Agent was able to find the nba highlights, but had to have human instruct it to navigate to the NBA channel before it successfully identified the highlights, as it was stuck beforehand.",https://www.loom.com/share/efeacefe21494adc9e4888955fd1da45
591,https://www.freepik.com,READ,easy,"Navigate to the Reimagine tool page and note the supported input file formats along with any usage requirements mentioned.
Only use http://freepik.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-136,Failure,"Agent failed to locate any details on file format availability, even after human prompting in the right direction.",https://www.loom.com/share/37cd1870ec1641a3a8cd85b90ccd2b6d
604,https://gamerant.com,READ,easy,"Use the search function to find articles about ""Cyberpunk 2077"" and provide the headlines of the first five results.
Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-137,Success,"Agent was able to find and return 5 articles related to ""Cyberpunk 2077.""",https://www.loom.com/share/57a04d8a25764f0ea9979bf60d4dfcf9
608,https://gamerant.com,READ,easy,"Search for latest PS5 review on Game Rant and extract the publication date along with a brief summary of the main criticisms.
Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-139,Failure,"Agent got stuck and then navigated to bing to search, which is out of bounds for the task.",https://www.loom.com/share/a5f12c5975384a8196fd2729ae0fbd91
609,https://gamerant.com,UPDATE,hard,"Log in to your account, post a comment on ""Monster Hunter Wilds Review."" Then navigate to your comment and update it with additional details.
Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-138,Failure,"Agent was able to create and post a comment on the desired article, but GameRant has to approve the comment before it can be updated, which takes ~24 hours. Due to the fact that the comment could not be updated, task is failed.",https://www.loom.com/share/e87f32e12dbd44f68ff2754626d46e79
612,https://gamerant.com,UPDATE,easy,"Change your display name in your account settings to include a gaming tag ""rockstar36458"" and confirm that the update is applied.
Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-140,Success,Agent was able to change the display name to the desired name as tasked.,https://www.loom.com/share/d9cb3628ad184695871b8d053468ecc0
615,https://www.gamespot.com,READ,easy,"Select the review article for ""The Legend of Zelda: Tears of the Kingdom"" and extract the review score along with three key highlights mentioned in the review.
Only use http://gamespot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-141,Success,Agent was able to locate the score of the review and list three key elements of the review as well.,https://www.loom.com/share/0beadeff71c44d84839bfddf7d6dd011
622,https://www.gamesradar.com,READ,easy,"Use the website’s search function to look for articles on ""comics"" and list the titles of the first three results.
Only use http://gamesradar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-142,Success,"Agent returned the top three relevant articles on ""comics"" as tasked,",https://www.loom.com/share/168a6d1d9e53440f9be47031b35b2e0f
652,https://github.com,CREATE,easy,"Fork the repository ""microsoft/vscode"" to your GitHub account and verify that the fork appears in your profile repository list.
Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-144,Success,Agent was able to fork the repository to the account as tasked.,https://www.loom.com/share/a6478d018af24dd68aaf215bef6e6d56
660,https://github.com,DELETE,easy,"Log in, create a repository called ""OldProject"" on your Github account then delete the repository named ""OldProject"" from your GitHub account and verify its no longer there. 
Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-146,Success,"Agent created the repository and then deleted it. Notably, agent asked for confirmation 3/4 times, each time prompted by the website to do so,. ",https://www.loom.com/share/40dbbae0e02c42c2baccd5637e9ecd92
661,https://github.com,DELETE,hard,"Log in, create a repository called ""sample-repo"" with a new branch ""old-feature"" then delete the ""old-feature"" branch using GitHub’s branch management options.
Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-145,Failure,"Agent was not able to locate branch options within a reasonable timeframe, so user timed out the agent.",https://www.loom.com/share/0cb1f053ca034e9cab31aff0d783b22b
672,https://github.com,UPDATE,hard,"Create a repository named ""sample-repo-2"" then change the branch protection rules for the ""main"" branch in your repository to require at least one approving review before merging any changes.
Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-147,Success,Agent created the repository and required one review before merging changes.,https://www.loom.com/share/67ba45175b1c4e15b00bdddabb47fef2
680,https://www.glassdoor.com/index.htm,CREATE,easy,"Post a new discussion in a Glassdoor community Bowl about work-life balance at tech startups, providing your insights and suggestions.
Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-150,Success,Agent was successfully able to create a post about work-life balance on glassdoor,https://www.loom.com/share/61816c93c75a40e5a8adba39fdf642fe
687,https://www.glassdoor.com/index.htm,DELETE,hard,"Log in to your Glassdoor account, post a new comment in a community discussion about job search tips, then delete your comment and verify that it is no longer displayed.
Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-151,Success,"Agent was successfully able to post a comment and then delete it later as well, surprising thing is the delete operation was so fast that i was not able to notice it.",https://www.loom.com/share/335ba4a0fa954959a95b684f362f3d67
688,https://www.glassdoor.com/index.htm,READ,easy,"Navigate to the New York City Bowl and display the number of members as well as the bio of the bowl.
Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-149,Failure,"Agent used the Bing search tab, rather than attempting to navigate the website.",https://www.loom.com/share/96ec82fe51374df38abc166dc1ac5ab0
689,https://www.glassdoor.com/index.htm,READ,hard,"Navigate to the ""Interviews"" section, search for interview experiences at ""Tesla"", and report the most frequently mentioned interview question.
Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-148,Success,"Agent looked at multiple interview accounts to aggregate interview question data, and returned the most common question asked by these accounts.",https://www.loom.com/share/b5bf8a992a1841429d4ff4ee64149b97
695,https://www.goal.com/en-us,READ,easy,"Find the match analysis of the last Champions League game and write down the main tactical observation discussed in the article.
Only use http://goal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-152,Failure,"Agent failed to look for summary or insights or key movements of the match, it only focused on find the key words which weren't directly available ",https://www.loom.com/share/6794d3e02fb04525ba483cd019a8f6d2
703,https://www.goodhousekeeping.com,DELETE,hard,"Log in to your GoodHousekeeping.com account, add one recipe to ""Weekend Cooking"" list, and then remove one recipe from your ""Weekend Cooking"" list, and verify that it no longer appears in your saved items.
Only use http://goodhousekeeping.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-153,Success,"Agent successfully added a receipe and then deleted it as well, however there was no option available to create a personalized list",https://www.loom.com/share/7c43978aa58f4f1290ed7fcabd1edf8e
710,https://www.goodhousekeeping.com,READ,easy,"In the Beauty category, filter for articles discussing skincare routines and list the publication dates of the first three posts.
Only use http://goodhousekeeping.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-156,Success,Agent was successfully able to extract required information,https://www.loom.com/share/a187daae33434e05bc1d2df78c553270
715,https://www.goodreads.com,READ,easy,"Search for the book ""To Kill a Mockingbird"" on Goodreads, view its ratings, and list the top 3 user reviews.
Only use http://goodreads.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-157,Success,Agent was successfully able to extract top three user reviews from the given book,https://www.loom.com/share/55b2719fa11c460fbd89e98d991e1e1f
746,https://www.grubhub.com,READ,easy,"Navigate to Catering, and summarize 5 of the FAQs, as well as the associated answers. 
Only use http://grubhub.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-159,Success,Agent was successfully able to find and summarize 5 FAQ's from the catering section,https://www.loom.com/share/2e378793790c4e78aa5c68eb5917958e
751,https://www.harpersbazaar.com,READ,easy,"Search for the article on ""Spring Fashion Trends"" and extract the names of the five luxury brands mentioned in it.
Only use http://harpersbazaar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-161,Success,Agent successfully found the article and it did extract the required information,https://www.loom.com/share/90298b7270d946b380a489e52bcf82e7
756,https://www.healthline.com,READ,easy,"Use the search tool to look for “diabetes management” and output the titles of the first five articles displayed.
Only use http://healthline.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-163,Success,Agent successfully extracted the information requested to it,https://www.loom.com/share/46c6b7729b1847b89cff7890971ed76f
758,https://www.healthline.com,READ,easy,"Navigate to Healthline’s homepage and list the titles of the top five featured health articles.
Only use http://healthline.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-164,Success,Agent did pull the 5 articles from featured section not directly from the homepage but it is still somewhat correct,https://www.loom.com/share/245febbd34d94cf9afabcf4ac05ca51e
765,https://www2.hm.com/en_us/index.html,READ,easy,"View the ""Sale"" section and record the discount percentage displayed on the first product.
Only use http://hm.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-165,Success,Agent was successfully able to find the sale section and pull the discount percentage on the first product,https://www.loom.com/share/1c8bc36e7d5a46c9bedfadf2b58d4b9b
771,https://www.homedepot.com,CREATE,easy,"Log in to your Home Depot account and create a new project list titled “Spring Garden,” then add the product “Husky Outdoor Planter Box” to the list.
Only use http://homedepot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-166,Failure,Agent failed to find the exact product mentioned in the task,https://www.loom.com/share/55ffa3c0146842a394abb279a153ecc2
775,https://www.homedepot.com,CREATE,easy,"Submit a “Contact Us” inquiry regarding delivery options for a “Samsung Family Hub Refrigerator,” including your contact details and a brief question.
Only use http://homedepot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-167,Success,Agent surprisingly had a live chat with an live agent on the other side by first providing the contact information and then asking the question with a brief description.,https://www.loom.com/share/a40b11fbe8554963914064c4bf09710c
782,https://www.homedepot.com,READ,easy,"Search for “porch lumber” on Home Depot and extract the pricing details for the 2x4 pressure-treated lumber available for delivery.
Only use http://homedepot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-168,Success,Agent did filter the luber according to the requirements and at the end gave an convincing answer,https://www.loom.com/share/5481befa7b4947ba907891426fc46a41
793,https://www.hotels.com,READ,hard,"Filter search results for properties in Paris available next month that offer spa amenities and bars, and list the amenities of the first three hotels.
Only use http://hotels.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-169,Success,Agent successfully filtered hotels with required amenities and extracted top 3 hotels info,https://www.loom.com/share/c3729852497f47e2b529b03d5a94a417
799,https://www.housebeautiful.com,CREATE,easy,"Sign up for notifications on new articles by entering your email and opting in for alerts on interior design trends.
Only use http://housebeautiful.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-170,Success,Agent successfully found the signup section for receiving new articles and signed up for it,https://www.loom.com/share/849ba26d01154bc5ac3d7c31df1fcab3
804,https://www.housebeautiful.com,READ,easy,"Navigate to the homepage and list the titles of the top 3 featured articles on modern interior design.
Only use http://housebeautiful.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-172,Failure,Agent failed to open the webpage,https://www.loom.com/share/d6b85a1b2a4b4a9ab72cb7d4547e4694
819,https://www.ign.com,READ,easy,"Navigate to the Guides section, Locate IGN’s GTA V guide, read the cheat codes section, and provide a summary of at least three cheat codes mentioned.
Only use http://ign.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-176,Failure,"Agent failed to access the website directly due to an error ""Site Unavailable""",https://www.loom.com/share/294a578b2cf946d9b85887eddda150a4
827,https://www.ikea.com,CREATE,easy,"Log in and post a review for the ""BILLY bookcase"" emphasizing its ease of assembly, then share your review on the product page.
Only use http://ikea.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-178,Success,Agent successfully found the product page and submitted a positive review as mentioned,https://www.loom.com/share/0bc81cc284914ee3b3074e3a1606815d
839,https://www.ikea.com,UPDATE,easy,"Access your profile, navigate to your saved addresses, and update your default delivery address to include specific instructions for building access.
Only use http://ikea.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-177,Success,Agent succcessfully navigated to the account settings and updated the instructions in saved address,https://www.loom.com/share/35d20a7ec5794273b0544dc5122cb4ce
850,http://indeed.com,READ,easy,"Check if a ""Delivery Driver"" job in Orlando provides 401k and paid time off. If it does, add it to my save list.
Only use http://indeed.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-180,Success,Agent successfully found the job as mentioned and added to saved list,https://www.loom.com/share/984904cca7f84df9bc088cf5ad36a8b3
854,https://www.independent.co.uk/us,READ,easy,"Navigate to the ""World"" news section on Independent.co.uk and list the headlines of the top 3 articles.
Only use http://independent.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-181,Success,Agent successfully finds the title of the articles,https://www.loom.com/share/f3cbc4362f954493b55d605e27d44f4f
856,https://www.india.com,READ,easy,"Locate the article covering the latest economic policy announcement in the Business section and note its publication date and headline.
Only use http://india.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-182,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/71f0e766d2ed49ccbc74d94784284774
860,https://www.indiamart.com,READ,easy,"Look up ""LED lighting systems"" and note any indicators of supplier verification and lead response times that appear on the product pages.
Only use http://indiamart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-183,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/ba1f8594e18540bc8c3c0608430235da
861,https://www.indiamart.com,READ,easy,"Use the search function to find ""organic tea leaves"" and list the top 5 supplier names along with their location details.
Only use http://indiamart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-184,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/0f04c3cd677740f088ed0feb10bf6d3f
873,https://www.instacart.com,CREATE,easy,"Search for ""Peanut Butter and Jelly"", add it to your cart, and then go to your cart to check the estimated delivery time for the order.
Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-186,Success,Agent successfully found a item which did contain peanut butter and jelly and added it to the cart to display the estimated delivery time,https://www.loom.com/share/ef977079aac6451c8da3598b5f1d1ac9
875,https://www.instacart.com,CREATE,easy,"Log in to Instacart and add “Organic Carrots” to your shopping cart for immediate checkout.
Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-185,Success,Agent successfully found the item requested and added it to the shopping cart,https://www.loom.com/share/e2109f74db3644998bbf71c6412fbe0b
883,https://www.instacart.com,READ,hard,"Search for organic bananas on Instacart and list the top 3 prices along with their retailer names.
Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-187,Success,Agent successfully found the organic bananas from three different stores and retured there prices respectively,https://www.loom.com/share/07ae754082344723a4f90db27488013d
885,https://www.instacart.com,UPDATE,easy,"Log in to your Instacart account, add 6 bunches of organic bananas to your cart, and then access your cart and update the quantity of your organic bananas from 6 to 10 bunches.
Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-188,Success,Agent successfully updated the organic bananas bunches from 6 to 10,https://www.loom.com/share/b57cc255ea1149bf82799e03edef7f62
890,https://www.instructables.com,READ,easy,"Filter projects tagged with ""woodworking"" and list the titles and authors of the first 5 Instructables.
Only use http://instructables.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-189,Success,Agent successfully found the 5 projects tagged with woodworking,https://www.loom.com/share/dcdb3ec864ed4f1ba70f723901cc59f2
895,https://www.investopedia.com,READ,easy,"Go to the market analysis section, locate the latest update on US economic indicators, and extract the names of at least three key indicators mentioned.
Only use http://investopedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-190,Success,Agent successfully found the article realted to economic damage and summed it up in 3 key indicators,https://www.loom.com/share/d825089ef52d45719057803977e9a1da
897,https://www.irs.gov,READ,easy,"Go to the Help page on IRS.gov that explains Direct Pay options for individuals and list the available payment methods mentioned.
Only use http://irs.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-191,Success,Agent successfully found the payment options for Direct Pay,https://www.loom.com/share/5e8a28ddfa734b39a60d3ccf29da7384
905,https://www.istockphoto.com,READ,easy,"Filter search results for ""business meeting"" by horizontal orientation, then list the first 5 image descriptions or titles displayed.
Only use http://istockphoto.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-192,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/0f1f50c3b2844eba879d47e824b57ea4
913,https://www.jagranjosh.com,READ,easy,"Search for the ""Previous Year Question Papers"" section and report the number of downloadable papers available for the UPSC exam.
Only use http://jagranjosh.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-194,Failure,Agent failed to return the number of papers it found as it was trying to search for more papers and it kept going even after I asked it to stop,https://www.loom.com/share/1b61261b0127485ab6ed17075e156b30
916,https://www.jagranjosh.com,READ,easy,"Check the academic calendar section for upcoming exam dates and list the next three major exams along with their schedules.
Only use http://jagranjosh.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-193,Success,Agent successfully found the upcoming exams from three different sections,https://www.loom.com/share/487078d29f4e45ffb969cc786ded1b3f
919,https://www.jstor.org,READ,easy,"Search for an article on ""Innovation in Urban Design"" and list the keywords associated with it from the metadata.
Only use http://jstor.org/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-195,Failure,Agent failed to find the article and so it was not able to find the keywords as well,https://www.loom.com/share/7345b55a14ce440a95169ef78b0c4949
927,https://www.justdial.com,READ,easy,"Locate the “24x7 Emergency Medical Clinic” in Bangalore and display its operating hours along with a brief description of services.
Only use http://justdial.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-196,Failure,Agent failed to access the information of operating hours of the medical clinic,https://www.loom.com/share/8b463ace45b0459fa85e2503b70f7f87
943,https://www.khanacademy.org,READ,easy,"Search for ""Algebra"" courses on Khan Academy and list the titles and short descriptions of the first five results.
Only use http://khanacademy.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-199,Failure,Agent failed to find the first five courses related to algebra as it only returned the top 4,https://www.loom.com/share/363fc6ee19dd4777b443dd6b681b4fe6
945,https://www.khanacademy.org,READ,easy,"On the Khan Academy homepage, list all available interface languages and note the total number of languages offered.
Only use http://khanacademy.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-197,Success,Agent successfully found the number of languages offered,https://www.loom.com/share/78714670fd4842a1907cbb69ebc3eb02
948,https://www.khanacademy.org,UPDATE,easy,"Log in to your student account and update your monthly learning goal on the progress dashboard to reflect a new target for completed exercises.
Only use http://khanacademy.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-198,Failure,Agent failed find an option to update the monthly learning goal in the available sections. The account seems to have no progress or goals set yet.,https://www.loom.com/share/197c4e4f2f694983adf7c61031880e2e
953,https://kidshealth.org,READ,easy,"Navigate to the ""Parents"" section and locate an article about managing screen time for children; then, provide a brief summary of the article title and its main tips.
Only use http://kidshealth.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-200,Failure,Agent stopped abruptly with no reason mentioned and even after retrying it was not able to contiue working on the task,https://www.loom.com/share/481db56ebd554f77b578ec10ccaeec57
958,https://www.kmart.com.au,CREATE,hard,"Log in to your Kmart account, create a new wishlist titled ""Holiday Gifts,"" and add 3 items representing gifts for him, for her, and for kids.
Only use http://kmart.com.au/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-201,Success,Agent successfully added the items to a shopping wishlist,https://www.loom.com/share/6766580910f648259831b5654e147bfc
960,https://www.kmart.com.au,DELETE,easy,"Log in to your Kmart account, add a ""Camera"" to your shopping cart, then remove the ""Camera"" item from your shopping cart, and confirm its deletion from the cart.
Only use http://kmart.com.au/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-202,Success,Agent successfully added an item then removed it as suggested,https://www.loom.com/share/51653ffc4d7d4c05ad87ad3bc35cb3ed
977,https://www.latimes.com,READ,easy,"Locate a feature article with a photo gallery and list the captions of the photos presented within the gallery.
Only use http://latimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-204,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/80cae66f97e1433e8e020c9fbb9eaf0d
980,https://www.latimes.com,READ,easy,"Use the search function to locate articles on “Los Angeles” and “immigration”, then provide the titles of the first three results.
Only use http://latimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-205,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/f3cdc6edfd114710acb99454d23cfc28
981,https://www.latimes.com,READ,easy,"Browse the entertainment section and list the top three celebrity news headlines along with their publication dates.
Only use http://latimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-206,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/bdd2650c636b47e8af9d0539034be088
992,https://letterboxd.com,CREATE,hard,"Compose a detailed film review diary entry for ""The Grand Budapest Hotel,"" focusing on its cinematography, and rate it 5 stars.
Only use http://letterboxd.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-207,Success,Agent successfully found the film and posted a review as suggested,https://www.loom.com/share/673c45ce024443ac95dcfedf73cf1b35
995,https://letterboxd.com,CREATE,easy,"Update your profile with a public update listing your top 3 favorite films of all time.
Only use http://letterboxd.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-208,Success,Agent successfully added 3 films to the profile in Favorite section,https://www.loom.com/share/0d9baa35e54d463b86f1dff9c0af03d2
1015,https://www.linkedin.com,CREATE,easy,"Add a job history entry to your profile titled Analyst at LabRite, from June 2021 - September 2023.
Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-213,Success,Agent successfully added the experience as suggested,https://www.loom.com/share/333a5756433d45339ad0009a6285ac91
1017,https://www.linkedin.com,CREATE,easy,"Comment on a recent article posted by Google, ensuring that your comment has a positive and professional connotation.
Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-210,Success,Agent was successfully able to post a comment on a google's recent post,https://www.loom.com/share/54cb8853a8ab4ebabe518b30458261b2
1021,https://www.linkedin.com,CREATE,easy,"Compose a text-based post asking your network for advice on upskilling in data analytics and mention two trending skills you wish to learn.
Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-209,Success,Agent successfully composed a post on linkedIn,https://www.loom.com/share/9d13551e90e446e19796ea70d628dc53
1033,https://www.linkedin.com,UPDATE,easy,"Make a post thanking several colleagues. Then edit the post to add an additional relevant hashtag before resubmitting it.
Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-211,Success,Agent successfully added a new post and later edited it to update the hashtags,https://www.loom.com/share/3936ddd915434ccaad7076c4b701fc80
1041,https://www.livejournal.com,DELETE,easy,"Log in, create a temporary journal entry titled ""Temporary Post,"" then delete this entry and confirm that it no longer appears on your journal.
Only use http://livejournal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-215,Success,Agent successfully added a temporary journal and then deleted it,https://www.loom.com/share/bb03606157d4498599208795ab21c735
1043,https://www.livejournal.com,DELETE,hard,"Login, create a friends list and add 3 individuals, then from your friends list, remove a specific friend by selecting their profile and confirming their deletion from your contact list.
Only use http://livejournal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-214,Success,Agent successfully added three friends and removed one at the end,https://www.loom.com/share/a877a6d12fac42f79f650e904f893d64
1050,https://www.livejournal.com,UPDATE,hard,"Log in and create a journal entry and schedule send it for 12 hours later. Then modify a scheduled journal entry by changing its posting time to an earlier hour on the same day as originally planned.
Only use http://livejournal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-216,Failure,Agent failed to update the timings of the post it already posted some time ago,https://www.loom.com/share/6a0dd71a149348c8832890b4232e13c8
1052,https://www.livescience.com,READ,easy,"Use the LiveScience search bar to look up ""volcano research"" and list the titles of the relevant articles that appear.
Only use http://livescience.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-217,Success,Agent successfully found the information as requested,https://www.loom.com/share/26e467a2863b4abb8525e59240a6777d
1053,https://www.livescience.com,READ,easy,"Navigate to the ""Space"" category and extract the title, author (if available), and publication date of the first article.
Only use http://livescience.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-218,Success,Agent successfully found the information as requested,https://www.loom.com/share/1ccc5308df4849249c6c76c3b77e6922
1057,https://www.loc.gov,FILE_MANIPULATION,easy,"Download a digitized manuscript related to early American literature from the catalog and verify that the preview includes the word ""manuscript.""
Only use http://loc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-219,Success,Agent successfully found a manuscript and was able to download it as well,https://www.loom.com/share/fa15a26c1aea4e3b9a47bcceef48098d
1063,https://www.lonelyplanet.com,READ,easy,"Use the search function to find articles on “Nepal trekking” and extract the titles of at least three related articles.
Only use http://lonelyplanet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-220,Success,Agent successfully found the title of top three articles as requested,https://www.loom.com/share/1fcb4e5d27c64eb5963ac45decf4099a
1084,https://www.macys.com,READ,easy,"Find the product description and reviews for a specific item.
Only use http://macys.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-225,Success,Agent successfully found the information about a dress with its price and reviews,https://www.loom.com/share/4a906832ef2f4e09b4f046f3998d2e45
1087,https://www.made-in-china.com,READ,hard,"Navigate to the suppliers profiles section, select a verified supplier offering ""electronic components,"" and extract the certification details provided on their profile.
Only use http://made-in-china.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-226,Failure,"Agent failed to access the website due to an error ""No Internet""",https://www.loom.com/share/1ff0ecb867a849cd953149c08f34b25b
1094,https://www.marksandspencer.com,READ,easy,"Navigate to the ""Food"" section and list the names and prices of the first five ready-meal products.
Only use http://marksandspencer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-227,Failure,"Agent failed to access the website due to an error ""No Internet""",https://www.loom.com/share/3400ff8530194203ab619ee969eea485
1096,https://www.marksandspencer.com,READ,easy,"Go to the ""Help"" or ""Customer Service"" section and list the top five frequently asked questions (FAQs) displayed.
Only use http://marksandspencer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-228,Failure,"Agent failed to access the website due to an error ""No Internet""",https://www.loom.com/share/ea3fb3918a36463b851c5cf6148e7415
1106,https://www.maryland.gov/Pages/default.aspx,READ,easy,"Use the instant search feature to look up ""public records retention policy"" and provide the URL of the first matching result.
Only use http://maryland.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-230,Success,Agent successfully found the info as requested,https://www.loom.com/share/397201cdce494fa98c258f5e15c387a0
1107,https://www.maryland.gov/Pages/default.aspx,READ,easy,"Explore the ""Feedback"" section and summarize the process given for submitting citizen input.
Only use http://maryland.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-229,Failure,"Agent failed to access the website due to an error ""No Internet""",https://www.loom.com/share/8d039cf44a2a4c6d975b4d44794b1373
1109,https://mashable.com,READ,easy,"Find an article that includes a section on upcoming events or festivals, and list the event names mentioned within that section.
Only use http://mashable.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-231,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/38f68cd13611440db5c8e9b282d96216
1110,https://mashable.com,READ,easy,"Browse the Mashable homepage and list the top three trending headlines along with their publication dates.
Only use http://mashable.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-232,Failure,"Agent failed to access the website due to an error ""Site Unavailable""",https://www.loom.com/share/502ec5f7ab164a5682f94570bc4e4b76
1126,https://www.medscape.com,READ,easy,"Use the search feature to find articles on ""COVID-19 long-haulers"" and list the first five article titles that appear.
Only use http://medscape.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-233,Success,Agent successfully found the top 5 articles as requested,https://www.loom.com/share/9e4bb4d0353f4ce0956dbd9dcdec3e1d
1134,https://www.mercari.com,READ,hard,"Browse the electronics category and calculate the average price of used smartphones shown in the listings.
Only use http://mercari.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-234,Success,Agent successfully found the average pricing of the used smartphones as requested,https://www.loom.com/share/eaa30d4d25cd43b78c8dccf1bf5d3c41
1144,https://www.metacritic.com,READ,easy,"Use the site search to find reviews for “The Shawshank Redemption” and extract both the critic consensus and the overall user score.
Only use http://metacritic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-236,Success,Agent successfully found the information from an article as requested,https://www.loom.com/share/a8707a343aec4115b21f2c186991e234
1147,https://www.metacritic.com,READ,easy,"Use the search bar to locate reviews for the film “Titanic” and find the publication date of one of the critic reviews.
Only use http://metacritic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-235,Success,Agent successfully found the article,https://www.loom.com/share/0eff7fb79e12408ca68077595a3b6a83
1154,https://www.michigan.gov/som,READ,easy,"Locate the most recent press release on state economic development on Michigan.gov and extract the release date along with the main headline.
Only use http://michigan.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-237,Success,Agent successfully found the ecomonic department and from that found the latest article as requested,https://www.loom.com/share/d22558965f504499abe058d817cabb31
1166,https://www.mountsinai.org,READ,easy,"Search for information on virtual urgent care service and outline the steps a patient should follow to access a virtual consultation.
Only use http://mountsinai.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-238,Success,Agent successfully found the information in virtual urgent care steps,https://www.loom.com/share/05b48a9158b7482994f02beb68013156
1197,https://www.ndtv.com,READ,easy,"Locate the regional news segment for Rajasthan and list the top three headlines along with their brief introductions.
Only use http://ndtv.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-240,Failure,"Agent failed to find news from Rajasthan directly from the website so it tried to search it on another domain, which is not allowed",https://www.loom.com/share/d079531976a440538f97f2b549212a15
1198,https://www.ndtv.com,READ,easy,"Navigate to the Business section in the footer and list the titles of the five most recent articles along with their publication dates.
Only use http://ndtv.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-239,Success,Agent successfully found the top-5 articles from the business section,https://www.loom.com/share/9ff022ceaa954e5d93e273aa8ec480d5
1210,https://www.newegg.com,READ,easy,"Search for ""external SSD"" in the storage category, apply a capacity filter for 1TB, and list the top three product models with their prices.
Only use http://newegg.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-242,Success,Agent successfully found the top 3 ssd as requested,https://www.loom.com/share/6ae46e333a1643629f0e395ffba48f80
1212,https://www.newegg.com,READ,easy,"Visit the Newegg Premier membership page, read through the listed benefits, and summarize at least five key membership perks.
Only use http://newegg.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-243,Success,Agent successfully found the information on premium membership,https://www.loom.com/share/7edb7cff50b143f4b2b62d75bf8ca029
1224,https://www.nj.com,READ,easy,"Locate the ""Food & Dining"" guide and extract the names of three recommended local restaurants highlighted in the guide.
Only use http://nj.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-244,Success,Agent successfully found the recommendation on 3 restaurant from an article in Food & Dining section,https://www.loom.com/share/bd51787434e645919c64a87e7b0ff293
1230,https://www.nordstrom.com,DELETE,easy,"Log in, add the ""Ray-Ban Aviator Sunglasses"" to your wish list, then remove that item and confirm its deletion from your account.
Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-245,Failure,Agent failed to login even after multiple help from my side,https://www.loom.com/share/04d3dfff320640639a3f67370e1dcd06
1231,https://www.nordstrom.com,DELETE,easy,"Log in, create a wish list titled ""Past Purchases"", and then permanently delete that wish list.
Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-246,Failure,Agent failed to login even after multiple help from my side,https://www.loom.com/share/9516c4e3ed17496f9d7bfe093a57aa7a
1233,https://www.nordstrom.com,READ,easy,"Use the search bar to locate BOSS Menswear Suit and record the product name, price, and available sizes displayed.
Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-247,Success,Agent successfully found the suit with its price and available sizes,https://www.loom.com/share/ae1fa80395ce4e5e873bd986f2f81d75
1236,https://www.nordstrom.com,READ,easy,"Navigate to the ""Under $100"" section in the Women's Handbags category and record the names, prices, and availability of the first three items.
Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-248,Success,Agent successfully found the information on handbags from the mentioned section,https://www.loom.com/share/f931815495a14faeb5dc6ae247b43551
1240,https://www.nps.gov/index.htm,READ,easy,"Navigate to the homepage and search find a park by state in Wyoming on the first page
Only use http://nps.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-249,Success,Agent successfully found all the parks mentioned in the given location,https://www.loom.com/share/eec83e432f2a46d1aec4627e9abae55b
1241,https://www.ny.gov,READ,easy,"Search for official guidelines regarding the renewal of a New York State driver’s license, and note the contact information provided.
Only use http://ny.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-250,Success,Agent successfully found the contact information for renewal of a license,https://www.loom.com/share/8638e6a0e10449f4a04394c30c02bf85
1245,https://www.nyc.gov,READ,easy,"Search for the FAQ page on recycling and waste management services and extract two common questions along with their answers.
Only use http://nyc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-252,Success,Agent successfully found two questions on waste management as requested,https://www.loom.com/share/c06b7df265ba404a9b12039c6ad268c4
1246,https://www.nyc.gov,READ,easy,"Navigate to the ""Benefits & Support"" section and list the names of three programs available to NYC residents.
Only use http://nyc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-251,Success,Agent successfully found the information on programs in NYC as requested,https://www.loom.com/share/11e5ea1be38d48ae925eb31334cc8e0e
1254,https://nypost.com,READ,easy,"Browse the News section and summarize the leading article by noting its headline, a brief excerpt, and the publication time.
Only use http://nypost.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-255,Success,Agent successfully found the information on a recent article,https://www.loom.com/share/f5fa0ae2e8f249648084269594efc66a
1257,https://nypost.com,READ,easy,"Go to the Business section and extract the headline of the article discussing the latest market trends.
Only use http://nypost.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-254,Success,Agent successfully found the article from buisness section,https://www.loom.com/share/5435771d5b224c90ae240a622bcd8847
1259,https://nypost.com,UPDATE,hard,"Log in to the NY Post comment section, leave a comment on an article from the front page, then locate this comment, and edit it to add more information.
Only use http://nypost.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-253,Success,Agent successfully wrote a comment on a recent article and edited it later,https://www.loom.com/share/6f00c56e88c64ad292f1dbe394cb8ff1
1270,https://www.oed.com/?tl=true,READ,hard,"Filter search results to show only entries for words first used from 1500 to 1600 and list the headwords of the first 10 results.
Only use http://oed.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-256,Success,Agent successfully filtered words and gave the top 10 as requested,https://www.loom.com/share/c8f96832553f4a2db45e3e8ef34d5396
1307,https://www.oregon.gov/Pages/index.aspx,READ,easy,"Identify the page for the Oregon agency responsible for environmental protection and list the contact details provided on that page.
Only use http://oregon.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-262,Success,Agent successfully found the environmental quality page and contact details mentioned in there,https://www.loom.com/share/485c6f09a91a41ba97be3f5c604c6b49
1321,https://www.pcgamer.com,DELETE,hard,"Log in to your pcgamer.com account, post a comment on one of the front page articles, then locate your recently posted comment and delete it.
Only use http://pcgamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-266,Success,Agent successfully wrote a comment on a recent article and deleted it later,https://www.loom.com/share/a96709ac8a8d46dfa2303c374550afbd
1324,https://www.pcgamer.com,READ,easy,"Navigate to the ""Reviews"" section and list the titles of the three most recent PC hardware review articles.
Only use http://pcgamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-267,Success,Agent successfully found the top-3 articles on hardware reviews,https://www.loom.com/share/14cc95ec8a104b8c9555a3d97821a795
1356,https://pixabay.com,READ,hard,"Go to the ""Videos"" section and output the file sizes from the download section and resolutions for the top 3 trending videos.
Only use http://pixabay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-269,Success,Agent successfully found the top-3 trending videos along with their download sizes,https://www.loom.com/share/a6484b32718840698ee376f4bfb5e533
1359,https://pixabay.com,READ,easy,"Search for ""vintage"" imagery and list the publication dates (if provided) and contributor ids (if provided) for the top 5 results.
Only use http://pixabay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-270,Success,Agent successfully found the information on top-5 results as requested,https://www.loom.com/share/0f66394869894683ac3d621d3984a523
1360,https://pixabay.com,READ,easy,"Search for ""abstract background"" in the vectors/illustrations category and output the contributor usernames and at least three tags from each of the first 5 results.
Only use http://pixabay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-268,Success,Agent successfully found the top-5 results and information related to it,https://www.loom.com/share/db0b0eba3cac4798a0c22d4908fa1bff
1364,https://www.playstation.com/en-us,CREATE,easy,"Subscribe to the newsletters by entering your email to receive monthly updates on PlayStation news, deals, and events.
Only use http://playstation.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-271,Failure,"Agent couldn't find the newsletter subscription page, so it tried to perform a Microsoft Bing search to retrieve it",https://www.loom.com/share/f8b00cbdbd3c4580a29f41396a052ea9
1365,https://www.playstation.com/en-us,CREATE,easy,"In the support section, submit a formal inquiry about issues with remote downloads, specifying your console model and error details.
Only use http://playstation.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-272,Success,"Agent initiated a conversation with a helper bot, thus submitting a formal inquiry. ",https://www.loom.com/share/830fb1cc90c540498792a41261950f2b
1369,https://www.playstation.com/en-us,READ,easy,"Browse the homepage and identify the main featured release; provide its headline and a brief summary.
Only use http://playstation.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-273,Success,"Agent was able to browse the homepage and locate the main article release, returning its headline and summary. ",https://www.loom.com/share/b647d674a63a4b9691b7dafb47be3997
1377,https://www.polygon.com,READ,easy,"Find any top 19 list of recommended PC games and pull out the first three game titles from that list.
Only use http://polygon.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-274,Success,Agent found three game titles from a recommended games list.,https://www.loom.com/share/7ce851a7c0df4dc6bfa0dac0dc116ea6
1392,https://www.purdue.edu,READ,easy,"Locate the section on study abroad programs and list two destination countries where Purdue students can study.
Only use http://purdue.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-275,Success,Agent was able to find and return two locations for Purdue students to study abroad.,https://www.loom.com/share/6be02622f0414fa6a7570531c78e2159
1395,https://quizlet.com,READ,easy,"Log in, visit the Quizlet Help Center, navigate to the FAQ section on creating your questions, and summarize the first three steps listed.
Only use http://quizlet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-276,Success,"Agent was able to navigate to the FAQ section on creating your questions, and summarize the first three steps listed.",https://www.loom.com/share/3c5f6915ba0d4746bda46ddade3b72c5
1400,https://www.radiotimes.com,CREATE,easy,"Use the social media sharing buttons on a Radiotimes article to generate a shareable link and create a custom message endorsing the article.
Only use http://radiotimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-278,Success,Agent was able to create a shareable link and embed it within its generated message.,https://www.loom.com/share/264fd2c005bd407db10fe4f428c592b0
1407,https://www.radiotimes.com,READ,hard,"Locate tonight's featured TV schedule on Radiotimes, and list the titles of shows airing on both BBC and ITV.
Only use http://radiotimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-279,Failure,Agent timed out - human determined that 10 minutes was inefficient for the task.,https://www.loom.com/share/d33ff5659aae427a803e2bb5a248f7e4
1413,https://www.realsimple.com,READ,easy,"Visit the Travel & Lifestyle section under Shopping and record the author’s name and publication date for the leading article.
Only use http://realsimple.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-280,Success,Agent was able to visit the Shopping section and record the author’s name and publication date for the leading article.,https://www.loom.com/share/d745ea240f334e7c959f33d576395048
1416,https://www.realtor.com,READ,easy,"Filter apartments for rent in New York City with a monthly rent under $2,500, and list the first three listings’ addresses, rental prices, and highlighted amenities.
Only use http://realtor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-281,Failure,Agent timed out - human determined that 10 minutes was inefficient for the task.,https://www.loom.com/share/d7350e00b4cc49b49fc29b885a8e9464
1417,https://www.realtor.com,READ,easy,"Go to the “Find a Realtor” section, enter criteria for a buyer in Phoenix, AZ, and list the names and contact details of the first five matching real estate professionals.
Only use http://realtor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-282,Success,Agent was able to input criteria and find 5 agents as well as their contact information as tasked.,https://www.loom.com/share/1bd7b6ee39484efe808b22b3ccc0f4b0
1428,https://www.redbubble.com,READ,easy,"Look up “Floral Patterns” on Redbubble and list the product titles that appear in the search results.
Only use http://redbubble.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-283,Success,Agent was able to find 20 product titles for a search of floral pattern.,https://www.loom.com/share/51324eb381cf40c99b0ab32f104cad5f
1438,https://www.rei.com,READ,easy,"Search for ""hiking boots"" on REI.com and list the names, prices, and ratings of the top three results.
Only use http://rei.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-284,Success,Agent was able to find the three highest rated hiking boots and return them.,https://www.loom.com/share/00e1e801f97e4782b1bd8b1492e193ca
1448,https://restaurantguru.com,READ,easy,"Search for Italian restaurants in New York City and list the top 3 restaurants by user rating.
Only use http://restaurantguru.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-285,Success,Agent was able to find the three highest rated Italian restaurants in NYC.,https://www.loom.com/share/21fa843105bc49e29b5ff388ee70672d
1456,https://resy.com,CREATE,easy,"Create a reservation request for a large group of eight at a restaurant in ""NoMad Diner"", including a note requesting a private dining area.
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-292,Success,Agent was able to reserve an 8 person table at the desired restaurant.,https://www.loom.com/share/cee0abe0c39a4c50b47693ffd5775ac5
1464,https://resy.com,CREATE,easy,"Create a personalized list titled ""Romantic Dinners"" on Resy and add the restaurants ""Le Bernardin,"" ""Jeju Noodle Bar,,"" and ""Eleven Madison Park"" to it.
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-290,Success,Agent was successfully able to add all of the restaurants to a saved list.,https://www.loom.com/share/3ab447ddcff04c90878fff8b6d605f25
1466,https://resy.com,DELETE,easy,"Login, add ""Passerine"" as favoriated in your account. then access your favorites list on Resy and remove it from the list.
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-291,Success,Agent was able to add the restaurant to the hit list and then remove it.,https://www.loom.com/share/faa8b6702bd445fb9cd9c7e2d8ba9640
1468,https://resy.com,READ,easy,"How many $'s does the Carbone in Greenwich Village, New York have?
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-293,Success,Agent was able to identify that Carbone has a 4 $ rating.,https://www.loom.com/share/a1d96a35c6804f3999e66e54cd8956c0
1469,https://resy.com,READ,hard,"Search for available reservation times for a French restaurant in New York City on an upcoming Friday evening.
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-286,Success,Agent was able to find available reservation times for French restaurants in New York City tonight.,https://www.loom.com/share/a985dbf577804361b0b32f0d4f0b3641
1470,https://resy.com,READ,easy,"Find Arden, in New York. List the available dining times for tonight. 
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-294,Success,Agent was able to find reservation times for Arden in NYC.,https://www.loom.com/share/b864f98aad8141d19f04806932a4112c
1472,https://resy.com,UPDATE,easy,"Make a reservation at ""Tacombi - Williamsburg"" for 7:00 PM on a given night. Modify your ""Tacombi - Williamsburg"" reservation by changing the dinner time from 7:00 PM to 8:00 PM.
Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-288,Failure,"Agent was unable to alter the reservation directly, but attempted to cancel the reservation and then create a new one. ",https://www.loom.com/share/55ce718465024a4d98555ba006043ffe
1478,https://www.reverso.net/text-translation,READ,easy,"Use the collaborative dictionary to translate 'proud' from English to Arabic and return the result
Only use http://reverso.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-295,Success,Agent was able to find the Arabic translation for the word proud.,https://www.loom.com/share/339e91e6305a44f29d68b655e73b5c1d
1491,https://www.samsung.com/us,READ,easy,"Navigate to the Samsung Galaxy S23 product page and list its key technical specifications such as processor, battery capacity, and display resolution.
Only use http://samsung.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-296,Failure,Human timed agent out - it couldn't find the specs for the Galaxy s23 after multiple attempts.,https://www.loom.com/share/f49f2cba8ff1445593e097f66d89ff19
1503,https://screenrant.com,READ,easy,"What is the description of the game 'Marvel Rivals'?
Only use http://screenrant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-299,Failure,"Agent couldn't find a Marvel Rivals synopsis, so it tried to perform a Microsoft Bing search to retrieve it",https://www.loom.com/share/ff30845bcd4e4733829821ac939bcd19
1504,https://screenrant.com,READ,easy,"Find the article '10 TV Shows That Went Downhill After A Major Cliffhanger' and summarize why 'The Flash' is included.
Only use http://screenrant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-298,Success,"Agent was able to find the article and explained why ""The Flash"" was included in the article.",https://www.loom.com/share/f5c685cf4fb24487869db2635ca900fe
1522,https://us.shein.com/?ref=www&rep=dir&ret=us,UPDATE,easy,"While logged in, add a ""gold hoop earrings"" product to your cart, update its quantity to 2 units, and verify that the total price is adjusted accordingly.
Only use http://shein.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-301,Failure,"Agent was unable to add the item to the cart, so this task was a failure.",https://www.loom.com/share/1a5013c1ea74468ab53dd84a0c687d1a
1520,https://us.shein.com/?ref=www&rep=dir&ret=us,READ,hard,"Browse the ""New In"" section and list the product names, prices, and available colors of the top 5 most popular items.
Only use http://shein.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-300,Failure,Agent was unable to find the most popular items and list the respective details.,https://www.loom.com/share/928f7200485743ba8841c34c20a732cc
1525,https://www.si.com,READ,easy,"Locate an SI.com feature on college sports and provide a brief summary outlining its key discussion points.
Only use http://si.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-302,Success,Agent found an article and returned a short summary as well.,https://www.loom.com/share/a1f2648de3a64e6689376fa3e9c94e40
1528,https://www.si.edu,READ,easy,"Navigate the homepage to locate the ""Smithsonian Open Access"" section and list the titles of the first five featured digital assets.
Only use http://si.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-303,Success,Agent found the first five titles of the Smithsonian digital assets.,https://www.loom.com/share/e7b149504f994800a153b6778127d892
1539,https://www.simplyrecipes.com,UPDATE,hard,"Login, post a comment praising a Lentil Soup recipe, then edit your previously submitted comment with additional recommendations for pairings with this dish. 
Only use http://simplyrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-304,Success,Agent was able to comment and then edit and save its comment. Human had to log in and confirm both actions.,https://www.loom.com/share/f51eaff07d374cfe88c92d30b32f354f
1545,https://www.skysports.com,READ,easy,"Identify and read the latest article under the rugby section covering match recaps, and list the names of the two teams that played.
Only use http://skysports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-306,Success,Agent found the most recent match recap and summarized it for the human.,https://www.loom.com/share/3e29ef3d27c244f2992495027c3d9e0a
1546,https://www.skysports.com,READ,easy,"Browse the homepage for any breaking news related to the ""Football"" and provide a short text summary of the update.
Only use http://skysports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-305,Success,Agent was able to find relevant news related to Football and provide a summary.,https://www.loom.com/share/bc43d8b0b5e04d0c90ce3f5c6ecb4162
1550,https://www.slideshare.net,READ,easy,"Search for presentations on ""digital marketing trends 2023"" and list the titles and authors of the first five results.
Only use http://slideshare.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-308,Success,Agent was able to list the titles and authors of the first five results.,https://www.loom.com/share/f9a501a33b794e1cbe1c1d08836c7278
1551,https://www.slideshare.net,READ,easy,"Browse the ""Business"" category and extract the title of the most-viewed presentation.
Only use http://slideshare.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-307,Success,Agent successfully found the title of the most recent presentation.,https://www.loom.com/share/2ae5a74a414b4c83a24116390d8cd2c0
1553,https://www.slideshare.net,READ,easy,"Identify three presentations on ""sustainable energy"" sorted by popularity and list their titles.
Only use http://slideshare.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-309,Success,Agent was able to find and return three presentations on sustainable energy.,https://www.loom.com/share/4b8f7349ae15476f87ba9cacbe266191
1555,https://www.smithsonianmag.com,READ,hard,"Browse the History category and record the title and publication date of the most popular article (based on visible engagement or share counts).
Only use http://smithsonianmag.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-310,Success,Agent identified an article based on its mention in a 2024 most popular article.,https://www.loom.com/share/16d834dc65be4a5ba208fc6b0adb71d1
1566,https://www.sofascore.com,READ,hard,"Explore the heatmap visualization on a live football game page and specify the zone where the highest concentration of shots occurred.
Only use http://sofascore.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-312,Success,Agent identified the portion of the pitch that received the most attention by looking at the heat map.,https://www.loom.com/share/957445d3027046f985113c2b4c172bd5
1573,https://www.sportskeeda.com,CREATE,easy,"Log in to your Sportskeeda account and post a comment on the latest ""UEFA Champions League"" article sharing your opinion on the match outcome.
Only use http://sportskeeda.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-314,Success,"Agent was able to post a relevant comment that it generated, as tasked.",https://www.loom.com/share/b3fc2bee55c4494f926830bc6e2b6184
1586,https://www.sportskeeda.com,UPDATE,easy,"Access your account settings on Sportskeeda and update your bio to better reflect your interests in both traditional sports and esports.
Only use http://sportskeeda.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-313,Success,Agent successfully updated the bio to reflect the desired changes and saved it.,https://www.loom.com/share/3c42863cfeb84a19baf5ddda3145106b
1591,https://open.spotify.com,CREATE,easy,"Create a podcast playlist by starting a new list and adding one episode from a podcast in the ""Tech Trends"" category.
Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-315,Success,Agent created a playlist with the first episode from a tech podcast.,https://www.loom.com/share/1eba6becd83c4c6a82a50498836297be
1601,https://open.spotify.com,DELETE,easy,"Create a temporary playlist titled ""Test Playlist,"" add two tracks, then delete the playlist and confirm its removal from your library.
Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-317,Success,"Agent was able to create the playlist, add two tracks, and delete it successfully.",https://www.loom.com/share/ceff1d6bbe04424e8e72037cd23836cb
1605,https://open.spotify.com,FILE_MANIPULATION,hard,"Login and find out how to request extended streaming data, and follow the steps to request this data. 
Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-316,Failure,Interface failed to respond multiple times when human took control of browser to log in.,https://www.loom.com/share/c1d4746287004768b7f5d725948eec1c
1609,https://open.spotify.com,READ,easy,"Browse the Spotify home page and list the names of the top 5 trending playlists currently featured.
Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-318,Success,Agent identified the 5 top trending playlists on Spotify.,https://www.loom.com/share/9463ed7671f94dae96979bbe0ab57b50
1625,https://www.springer.com/us,READ,hard,"Find the ""Philosophy"" articles page and summarize the abstract of the most recent article posted.
Only use http://springer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-320,Success,Agent identified and summarized a recent philosophy article with a scientific twist as well.,https://www.loom.com/share/d5ae495541a04de182591ad3395108eb
1635,https://stackexchange.com,CREATE,easy,"Log in, navigate to the Data Science community, and post an answer on a question about ""feature engineering techniques"" including at least two bullet points.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-325,Success,"Agent was able to log in and post an answer to a question about ""feature engineering techniques."" ",https://www.loom.com/share/b354f8d8c30841b88e7bf1b3e04591bd
1640,https://stackexchange.com,CREATE,hard,"Post an answer in the Super User community to a question regarding ""SSD vs HDD performance"" and include relevant benchmark numbers.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-331,Success,"Agent posted a response to the question regarding ""SSD vs HDD performance"" and included data as well.",https://www.loom.com/share/f9cf7b71abbb42c1a5960b0b94f9d790
1642,https://stackexchange.com,CREATE,easy,"Log in and post a new question in the Academia community titled ""How to balance research and teaching responsibilities?"" with contextual examples from your experience.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-328,Success,Agent was able to create the post successfully.,https://www.loom.com/share/f08de55d6b704fe492ee6be07a7fbdc0
1644,https://stackexchange.com,CREATE,hard,"Using the Markdown editor, log in and create a comprehensive guide titled ""How to use StackExchange effectively"" on the Meta site.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-326,Success,"Agent was able to post a guide titled ""How to use StackExchange effectively"" on the Meta site. ",https://www.loom.com/share/7924cdabcf1a4f7db85f18df5c2692af
1658,https://stackexchange.com,DELETE,easy,"In the Super User community, post a temporary answer labeled ""Test answer for deletion"" to an existing question and then delete it, ensuring it no longer appears in your contributions.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-322,Failure,"After CAPTCHA popped up 4 times after human solved it again and again, it became clear there was a connection issue between the browser and the agent. ",https://www.loom.com/share/7f523d79a2784f38abb3f206068ff728
1661,https://stackexchange.com,READ,easy,"Visit the Meta site of StackExchange and list the titles of the 3 most recent community discussion topics.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-330,Success,Agent was able to find three most recent community discussion topics.,https://www.loom.com/share/2e9ca161cabc4774b4a5c87ac1baf3f9
1662,https://stackexchange.com,READ,easy,"Use the main search bar to find questions tagged with ""python"" in the Stack Overflow community and output the titles of the first 5 results.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-323,Failure,CAPTCHA didn't process no matter how many times human solved it.,https://www.loom.com/share/1a1b44bd39b94e089e36efaf322cde6f
1669,https://stackexchange.com,UPDATE,easy,"Log in and post a question in the Cross Validated community titled ""Statistical Significance in A/B Testing."" Then update the title of this question to ""Understanding Statistical Significance in A/B Testing.""
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-329,Success,"Agent was able to post the question, and then revise and update the title of the question.",https://www.loom.com/share/b31278e480a141f4b47e90c7f0365648
1671,https://stackexchange.com,UPDATE,easy,"Log in to the Web Applications community and answer a question ""increasing website engagement."" Then update your answer with additional examples and an emoji.
Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-327,Failure,"Agent was not able to find the question, so this task is a fail. ",https://www.loom.com/share/b387f365140e493db24bec79f2ba678d
1676,https://stackoverflow.com/questions,CREATE,hard,"Provide an answer to a question regarding best practices for REST API development in Ruby on Rails, including a sample code implementation.
Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-335,Failure,"Agent found the answer it needed to provide, and returned it instead of commenting on the question. ",https://www.loom.com/share/b9f2ab60e2974cf5a1aa9ce361ef728d
1690,https://stackoverflow.com/questions,DELETE,easy,"Log in, post a question about Node.js with the tag ""node"" and then go back to delete the tag from your posted question on Node.js, ensuring that the removal is confirmed.
Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-332,Success,"Agent was able to post about Node.js with the tag ""node"" and then go back to delete it.",https://www.loom.com/share/0630fede78604a67b23aa532140f599f
1692,https://stackoverflow.com/questions,READ,easy,"Browse through the ""Help Center"" to locate the page on editing posts and list the top three tips mentioned.
Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-333,Success,Agent identified three tips from the Help Center page and returned them.,https://www.loom.com/share/24b1f5545eae49ebb97037cdbb3714e7
1698,https://stackoverflow.com/questions,UPDATE,hard,"Log in to your account, and post a question regarding web security, then update the tags on your question about web security to also include ""sql-injection""
Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-334,Failure,"Instead of creating a new post and editing that, it updated a previously posted post. ",https://www.loom.com/share/73a77d54805b4fc4b31d7142efdd71b1
1704,https://stardewvalleywiki.com/Stardew_Valley_Wiki,READ,easy,"Analyze which villagers have birthdays in Summer using the calendar data
Only use http://stardewvalleywiki.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-337,Success,Agent was able to identify which villagers have summer birthdays successfully.,https://www.loom.com/share/283932ed63fa4d109ac36aeef9252be5
1705,https://stardewvalleywiki.com/Stardew_Valley_Wiki,READ,easy,"Find and list all the gifts that Sebastian ""loves"" according to his gift preferences page.
Only use http://stardewvalleywiki.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-336,Success,"Agent identified a list of Sebastian's ""loved"" gifts.",https://www.loom.com/share/fc1aa4d991b248c8a0b01aee44a47f77
1713,https://www.statista.com,READ,easy,"Filter the available reports by the sports industry and list the titles of the first three annual reports presented.
Only use http://statista.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-338,Success,Agent was able to filter by the sports industry and list the top three reports mentioned.,https://www.loom.com/share/e719c65807a8478fb145fa99d95af9fc
1723,https://store.steampowered.com,READ,hard,"Go to the Top Sellers section and list all games currently discounted at more than 50%.
Only use http://steampowered.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-340,Success,"Agent was able to browse the list of top sellers and provide a list of 50% or more discounted games. Human asked the agent to stop at some point because it was scrolling, which was both exhaustive but redundant.",https://www.loom.com/share/0de4035fa39743968e020b42cacb72c1
1729,https://www.studocu.com,CREATE,easy,"Create a new account and then create a course name of  ""Introduction to Economics."" Ensure that a confirmation message appears.
Only use http://studocu.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-341,Failure,"Agent could not find a way to name the specific course ""Introduction to Economics.""",https://www.loom.com/share/75aff4cdb4ca42f5addc4ad03eeb420a
1732,https://www.studocu.com,CREATE,hard,"Assemble a new study plan by creating a ""Revision Pack"" StudyList that curates documents on key course topics and then use the site’s sharing feature to share it with a friend.
Only use http://studocu.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-342,Success,Agent was able to create the study list and generate a link to be able to share.,https://www.loom.com/share/3308ade497a342e6986bcbb8ba3ab067
1745,https://www.target.com,READ,easy,"Check how long shipping takes for an ipad Air to be delivered to the New York City 10003 Zip Code
Only use http://target.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-343,Success,"Agent updated the zip code to 10003 and found the time that it would take to ship, as well as the shipping cost.",https://www.loom.com/share/d9ff2cfb7d81444aaef88fbd4c3ed0da
1750,https://www.techradar.com,READ,easy,"Search for articles related to “energy efficient computers” and extract a recommendation mentioned in one of the articles.
Only use http://techradar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-344,Success,Agent found a relevant suggestion about power saving for a vehicle.,https://www.loom.com/share/ef83fbea84c64c66b9731d0ebc17f1d0
1761,https://www.telegraph.co.uk/us,DELETE,easy,"Log in to your Telegraph account, find an article on the front page and save it to your reading list, then access your reading list, delete the saved articles, and verify that it no longer appears.
Only use http://telegraph.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-346,Success,"Agent was able to add an article from the front page, save it, and then delete it from the saved list. ",https://www.loom.com/share/e3484f134eb94f99a9a2f4cc1b81bb05
1765,https://www.telegraph.co.uk/us,READ,easy,"Use the search function to find an article about a “Royal Ceremony” and provide the headline and publication date of the first result.
Only use http://telegraph.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-345,Failure,"Agent found an article for a red carpet walk, not a royal ceremony, which would imply British Royalty/wedding.",https://www.loom.com/share/4ce3c3bd6502402cbf26bd11b387046d
1768,https://www.telegraph.co.uk/us,UPDATE,easy,"Log in to your Telegraph account, navigate to your profile settings, and change your newsletter subscription preference from daily updates to weekly updates.
Only use http://telegraph.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-347,Failure,"Agent subscribed to a daily newsletter, and when prompted by human to change the preference to a weekly cadence, the agent tried to subscribe to a daily newsletter rather than changing the existing preference.",https://www.loom.com/share/3ccc1849528b471ba9a65bff0f6a302c
1769,https://testbook.com,READ,easy,"Search the articles section for ""SSC exam pattern changes"" and extract the key updates mentioned.
Only use http://testbook.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-349,Success,Agent was able to find and provide a sufficient amount of key exam updates.,https://www.loom.com/share/0bc6633f51ff4e56a57074db149ae4c0
1774,https://www.texas.gov,READ,easy,"In the ""Find Services"" section, locate the official guidelines on vehicle registration and extract the checklist of submission requirements shown.
Only use http://texas.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-350,Success,"Agent was able to find the ""Find Services"" section and provide all the registration details demanded.",https://www.loom.com/share/f31fa648c64b46d7b7f9f8ed533f845c
1778,https://www.theatlantic.com,READ,easy,"Navigate to The Atlantic homepage and list the titles of the top three featured articles displayed.
Only use http://theatlantic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-352,Success,Agent found the top three featured articles displayed and returned them.,https://www.loom.com/share/7dd0b06515b54d259bdf2dadea67bf90
1780,https://www.theatlantic.com,READ,easy,"Visit the subscription page and identify the differences in benefits offered between the digital, digital and print subscriptions.
Only use http://theatlantic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-351,Success,Agent quickly identified the differences between each subscription model.,https://www.loom.com/share/d55ef8b0e09d48d7aad9f92471d4c225
1784,https://www.thegamer.com,READ,easy,"Search for reviews  and list the headlines of the top five game review articles currently featured.
Only use http://thegamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-353,Success,Agent successfully found and returned the top five featured articles on game reviews.,https://www.loom.com/share/3379ab3407fd4143a66ebcd5dbad0692
1785,https://www.thegamer.com,READ,hard,"Locate an Opinion or Cultural Commentary article discussing modern gaming culture and summarize its central argument in one or two sentences.
Only use http://thegamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-354,Failure,"Agent found an article and summarized the synopsis, but failed to summarize the argument. ",https://www.loom.com/share/70ea4cce4329430e8dc54bfb6eba3989
1792,https://www.thehindu.com,READ,easy,"Search the Health section for COVID-19 related news, and output the title and publication time of the first article that appears.
Only use http://thehindu.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-355,Success,Agent found the first article related to COVID-19.,https://www.loom.com/share/9ab426bd8e894276a180e4c2eaf6e66c
1798,https://www.themoviedb.org,READ,easy,"Browse the ""Trending"" section on themoviedb.org and extract the titles of the top 5 trending movies at the moment.
Only use http://themoviedb.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-356,Success,Agent found the top 5 trending movies on the website.,https://www.loom.com/share/06ffe60589af45858c49584180b81caf
1801,https://www.thepioneerwoman.com,CREATE,easy,"Log in to the site and save your favorite recipe from the ""Comfort Food"" category to your saved recipes collection.
Only use http://thepioneerwoman.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-358,Success,Agent was able to find a recipe in the comfort food section and save it.,https://www.loom.com/share/8a5f9a820e34432486b570749858a4a6
1813,https://www.thepioneerwoman.com,READ,easy,"Locate a tutorial featuring Ree Drummond’s signature dish.
Only use http://thepioneerwoman.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-357,Success,"Agent found a tutorial recipe with Ree Drummonds signature cookies, and provided a link as well.",https://www.loom.com/share/b4360731f56a44d6935f8e751ffdea1f
1816,https://www.thesaurus.com,READ,easy,"Look up the synonyms for ""rapid"" and list the first five synonyms that appear.
Only use http://thesaurus.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-359,Success,"Agent found the first 5 synonyms for ""rapid.""",https://www.loom.com/share/239c8aa13b4d4b3f8a76bc35a0cd5356
1819,https://www.thespruce.com,READ,easy,"Search for ""DIY kitchen backsplash ideas"" on TheSpruce and list the titles of the first five articles.
Only use http://thespruce.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-360,Success,"Agent found the first five articles that came up when it searched for ""DIY Kitchen Backsplash Ideas.""",https://www.loom.com/share/0dcc2ec986ac44318a7a29dca2b4ce22
1828,https://www.thoughtco.com,READ,easy,"Browse the ThoughtCo homepage and list the titles of the three newest articles displayed on the landing page.
Only use http://thoughtco.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-361,Success,Agent found and returned the first three articles on the homepage.,https://www.loom.com/share/440863d345144980b3edaa6eee82fabc
1829,https://www.threads.net,READ,hard,"Use the search function to find threads mentioning “sustainability” and summarize the main discussion points from the first three results.
Only use http://threads.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-362,Success,Agent was able to find 3 threads and summarize them as tasked.,https://www.loom.com/share/d92d6951a60e4a13a4d5b4173809702c
1832,https://www.ticketmaster.com,READ,hard,"Visit the New York Giants' page, and find an available game to find tickets for. 
Only use http://ticketmaster.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-364,Success,"Agent was technically correct in the sense that no tickets are available for NFL games because as of April 29th, 2025, the NFL schedule has not yet been released.",https://www.loom.com/share/eada76b1d70f49cd931efb28b4860e6f
1836,https://www.tiktok.com/explore,READ,easy,"Browse the For You feed and list the usernames and view counts of the first 5 videos displayed.
Only use http://tiktok.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-365,Failure,"Agent provided the like count, but not the view count. ",https://www.loom.com/share/4a06f6f93707479790ed02c4428fd1f9
1843,https://time.com,READ,easy,"Navigate to the ""Science"" section and summarize the key findings of the leading article in 2–3 sentences.
Only use http://time.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-366,Success,Agent was able to find the first article and provide a 2-3 sentence summary effectively.,https://www.loom.com/share/5a21fd851cdd4eb98aaf4bb32f4d13ea
1848,https://www.timeanddate.com,READ,easy,"Browse the API section to retrieve the Date Calculator API Pricing details including the package prices and number of credits
Only use http://timeanddate.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-367,Success,Agent was able to find pricing details and return them for Time and Date.,https://www.loom.com/share/e91057e3b75b4338bee8d57ed1e60601
1853,https://www.today.com,CREATE,easy,"Save an article about summer recipes to your reading list and add a brief note explaining why you found it interesting.
Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-371,Failure,"Agent was able to save the recipe, but couldn't access the bookmarks folder and din't provide a note as to why it liked the recipe. ",https://www.loom.com/share/17ce764a1d834ed7859ed6b9c9d3e06f
1858,https://www.today.com,READ,hard,"Scroll through the infinite content feed in the News category and record how many new articles load after an extended scroll.
Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-370,Success,Agent was able to scroll until it couldn't anymore and recorded 120 different articles.,https://www.loom.com/share/c636b561c7cd4dd6946ac1b6bb171494
1860,https://www.today.com,READ,easy,"Search for articles on “healthy breakfast recipes” and display the title and summary of the top result.
Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-369,Success,"Agent was able to return the title and summary of the top article when searching for “healthy breakfast recipes.""",https://www.loom.com/share/dbf78b03714a46d4aae535a50265884e
1861,https://www.today.com,READ,easy,"Navigate to the homepage and locate the schedule for the live Today show broadcast, then record the start time of today’s episode.
Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-372,Success,Agent was able to find the schedule and return the time of today's episode.,https://www.loom.com/share/265886ec04224d089e729129f9d21886
1868,https://www.tomsguide.com,FILE_MANIPULATION,easy,"Locate the downloadable PDF version of the ""How to Enable 2FA"" guide on Tom's Guide, download it, and verify that the filename contains the text ""2FA"".
Only use http://tomsguide.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-373,Failure,Agent was unable to locate the guide with a downloadable pdf.,https://www.loom.com/share/c77b6f9390c34dffa93e47e71695e8cb
1873,https://www.tomsguide.com,READ,easy,"In the Reviews section, filter for gaming review and note down the headline of the top-listed review article.
Only use http://tomsguide.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-374,Success,Agent was able to filter for reviews and listed the top review headline.,https://www.loom.com/share/f3719ce945f44dcd98290a56bc3c3b41
1883,https://www.travelweekly.com,READ,easy,"Browse the homepage and list the top five featured travel industry news headlines.
Only use http://travelweekly.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-375,Success,Agent was able to navigate to the homepage and list the top five featured travel industry news headlines.,https://www.loom.com/share/1dadc99a54c14323893106588b3e13a0
1885,https://www.travelweekly.com,READ,easy,"Use the search tool to look for upcoming webinar events on travel technology and output the titles of the events.
Only use http://travelweekly.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-376,Failure,Agent wasn't able to find any webinars related to travel technology.,https://www.loom.com/share/4d2604581b404397a7370915155ba012
1888,https://us.trip.com/?locale=en-us,READ,easy,"Browse the Trip homepage and list the top 5 trending featured properties
Only use http://trip.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-377,Success,Agent was able to browse the homepage and list the top 5 trending properties.,https://www.loom.com/share/6cca7c0d64554d4eb4168cd7ead7097b
1890,https://us.trip.com/?locale=en-us,READ,easy,"Search for tours available in Rome tomorrow and list the top three tours sorted by ""Top Rated"".
Only use http://trip.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-378,Success,"Agent was able to search for tours available in Rome tomorrow and list the top three tours sorted by ""Top Rated"".",https://www.loom.com/share/ccb756ffbdf54e02a8de3c71468f729b
1892,https://www.tripadvisor.com,READ,hard,"Identify the top-rated hotel in Paris, verify if it offers free cancellation, and analyze at least three recent guest reviews to see if they mention staff helpfulness.
Only use http://tripadvisor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-379,Success,"Agent was able to filter by the necessary constraints, locate the highest rated hotel, and provide multiple anecdotes of the hotel's service. ",https://www.loom.com/share/c4ac4e017b24491a8032bae7c60096bb
1897,https://www.trustpilot.com,READ,hard,"Use Trustpilot’s search function to filter  HR & Recruiting located in ""London"", then list the review summaries for the top three highest‑rated companies.
Only use http://trustpilot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-380,Success,"Agent filtered by HR & Recruiting located in ""London"", then listed the review summaries for the top three highest‑rated companies.",https://www.loom.com/share/9de36385ab79453ab0f274add78374b7
1911,https://www.twitch.tv,UPDATE,easy,"Modify your alert settings by updating the text that appears when someone donates Bits, ensuring your channel name is featured prominently in the alert.
Only use http://twitch.tv to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-381,Success,"Agent was able to create a bit donation message, and then update it to include a name, which human provided. Name was able to be different than account name. ",https://www.loom.com/share/855c3569496444eb81a9de7f67a060e5
1914,https://www.ucdavis.edu,READ,easy,"Navigate to the Admissions section and extract the key deadline dates for Fall 2025 undergraduate applications.
Only use http://ucdavis.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-382,Success,Agent was able to screen the Admissions section and extract the key deadline dates for Fall 2025 undergraduate applications.,https://www.loom.com/share/09ad9976c8af4334bd50bd1a7554f53e
1915,https://www.ucdavis.edu,READ,easy,"Search for the UC Davis library page and retrieve the opening hours and contact information for the main library.
Only use http://ucdavis.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-383,Success,Agent was able to identify hours and contact information at the UC Davis main library.,https://www.loom.com/share/84bfda4fbd9145a5a2db75f6bf1ec4c0
1917,https://www.ucdavis.edu,READ,easy,"Go to the Financial Aid and Scholarships page and extract the main eligibility criteria and deadlines for applying for aid.
Only use http://ucdavis.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-384,Success,Agent was able to extract the main eligibility criteria and deadlines for applying for aid.,https://www.loom.com/share/30d15d920aa34ec18ba6103fa5927a50
1924,https://www.ucla.edu,READ,easy,"Search for information on UCLA’s athletic programs or sports teams and list the sports teams mentioned on the page.
Only use http://ucla.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-385,Success,Agent was able to identify all of the sports teams at UCLA.,https://www.loom.com/share/d3271a21051f49518bf9c23f3fbc487d
1930,https://www.ufl.edu,READ,easy,"Use the site’s search bar to find information on UF research computing facilities and note two primary services offered.
Only use http://ufl.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-386,Success,Agent looked at UF research computing facilities and noted two primary services offered.,https://www.loom.com/share/f45244fe07934c09975f4556b58405cf
1931,https://www.ufl.edu,READ,easy,"Browse the Academic Departments listing and extract key details on the Computer & Information Science & Engineering department’s research focus areas.
Only use http://ufl.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-387,Success,Agent was able to scan the Computer Science department and identify research domains.,https://www.loom.com/share/81c1d23815894d9cab853bdb774ee881
1938,https://www.ultimate-guitar.com,READ,easy,"Find the guitar tabs for ""Hotel California"" by the Eagles.
Only use http://ultimate-guitar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-388,Failure,Agent was unable to access the site on the browser.,https://www.loom.com/share/480a5947a89146ac95a74bc6e8dce1f0
1939,https://umich.edu,READ,easy,"Browse the University of Michigan homepage and list the top three headlines featured in the News section.
Only use http://umich.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-389,Success,Agent was able to identify the top headlines on the front page.,https://www.loom.com/share/36139c1a70bb4846af8046461601d115
1942,https://umich.edu,READ,easy,"Explore the “Michigan Online” section and list two online courses or learning platforms mentioned on that page.
Only use http://umich.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-390,Success,"Agent listed two online courses or learning platforms mentioned on the ""Michigan Online"" page.",https://www.loom.com/share/79a08f6474f84015bdae6b111bf55eb5
1951,https://www.uniqlo.com/us/en,DELETE,hard,"Log in to your Uniqlo account, add the BlockTech Shirt to a wishlist, then remove it from the wishlist, confirming that the product no longer appears on the list.
Only use http://uniqlo.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-391,Failure,"Agent was unable to find the BlockTech shirt, so this was a failure. ",https://www.loom.com/share/6d3a572b9c074f6e9001140f575a245c
1966,https://www.upenn.edu,READ,easy,"Search the website for information on interdisciplinary research centers and list two names or brief descriptions provided on the page.
Only use http://upenn.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-392,Success,Agent was able to find information on interdisciplinary research centers and list two names provided.,https://www.loom.com/share/2b1df8ae1dc54e71a9f3c61f4b15f364
1968,https://www.urbandictionary.com,READ,easy,"Find the definition for “woke” and list any descriptive tags or keywords that appear alongside the definition.
Only use http://urbandictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-393,Success,"Agent was able to find the definition of ""woke"" and listed ""cultural marxism"" as keywords. ",https://www.loom.com/share/88c744c8639b4ee4b9ac54c779dd9a73
1969,https://www.urbandictionary.com,READ,hard,"Use the search bar to look up “stan” and compare the top two definitions by summarizing the contrasting perspectives or nuances in their descriptions.
Only use http://urbandictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-394,Success,"Agent was able to find the definition for ""stan,"" and briefly summarized the differences for the top two definitions. ",https://www.loom.com/share/e0336d8b997743f1b0ca0528cbbf8ad9
1974,https://www.usa.gov,FILE_MANIPULATION,easy,"Find and download the most recent government guide on disaster preparedness from USA.gov, then verify that the file is in PDF format.
Only use http://usa.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-395,Success,"Agent found the pdf, downloaded it, confirmed it was in pdf form, and offered a link to access the pdf. ",https://www.loom.com/share/3b6f8a412b2b42098883c826c3b49c82
2021,https://www.usnews.com,READ,easy,"Search the site for ""economic trends 2023"" and list the titles and publication dates of the first five matching articles.
Only use http://usnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.",HITL-Operator-400,Success,"Agent listed the titles and publication dates of the first five matching articles for ""economic trends 2023.""",https://www.loom.com/share/87a65e2ad36b463cac7fd32f72e5b95c