mirror of
https://github.com/browseros-ai/BrowserOS.git
synced 2026-05-20 04:21:23 +00:00
- Add hover_at, type_at, drag_at coordinate tools to server - Add hoverAt, typeAt, dragAt methods to Browser class - Export server internals (browser, tool-loop, registry) for eval imports - Copy eval app from enterprise repo with agents, graders, runner, dashboard - Nest eval-targets inside apps/eval - Adapt sessionExecutionDir → workingDir for current server API - Add biome ignore for dashboard HTML to prevent lint breaking onclick handlers
158 KiB
Vendored
158 KiB
Vendored
| 1 | ID | Starting URL | Category | Difficulty | Task | HITL_Convergence_ID | convergence_hitl_eval | convergence_hitl_reasoning | convergence_hitl_link |
|---|---|---|---|---|---|---|---|---|---|
| 2 | 2 | https://www.acehardware.com | READ | easy | Search for "LED light bulbs" on AceHardware.com and provide the titles and sale prices of the first 5 products that appear. Only use http://acehardware.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-1 | Success | Succesfully finds the items after searching | https://www.loom.com/share/7b15c03fbcac47e88ded1bfb5d71bbff |
| 3 | 19 | https://www.airbnb.com | DELETE | easy | Log in to your Airbnb account, save a Guest Favorite property to your wishlist, and then go to your wishlist and remove the property you previously added. Only use http://airbnb.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-4 | Failure | The AI Agent could not open AirBnb due to proxy issues. And it returned "Airbnb is currently experiencing technical difficulties" | https://www.loom.com/share/ae2265ccd64440d9846d87603c4e7bed |
| 4 | 20 | https://www.airbnb.com | DELETE | hard | Log in to Airbnb, send a query to a host of a New York City property, and then delete your message confirming that the message does not appear on your profile. Only use http://airbnb.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-3 | Failure | The AI Agent could not open AirBnb due to proxy issues. And it returned "Airbnb is currently experiencing technical difficulties" | https://www.loom.com/share/0a7f91936bf84da3aae0277195dc5fc1 |
| 5 | 22 | https://www.airbnb.com | READ | easy | Browse the “Play” section in Paris and extract the titles and brief descriptions of the top five properties. Only use http://airbnb.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-2 | Failure | The AI Agent could not open AirBnb due to proxy issues. Had to leave the website in order finish task. | https://www.loom.com/share/e18b25f9453d41b484484264020108ef |
| 6 | 33 | https://www.alibaba.com | READ | easy | Search for "smartphones" on Alibaba and note how many listings are available. Only use http://alibaba.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-7 | Success | Succesfully finds the number of items listed | https://www.loom.com/share/0ad69855d4b94aceac4f78fb4b025321 |
| 7 | 35 | https://www.alibaba.com | READ | hard | Search for "RFID tags", filter by a minimum order quantity of less than 1000, and list the first 3 supplier names. Only use http://alibaba.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-5 | Success | Succesfully finds the items after searching | https://www.loom.com/share/13569940b6a44df29a18548dbe5abf73 |
| 8 | 51 | https://www.allrecipes.com | CREATE | easy | Log in, write a detailed review for the "Lemon Chicken Orzo Soup" recipe including a 5-star rating and your comments on taste and presentation. Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-8 | Success | Succesfully logs in, and pasts review for the Lemon Orzo Chicken! | https://www.loom.com/share/9fb301de82e24beaa2a0d41cedeb146d |
| 9 | 56 | https://www.allrecipes.com | CREATE | easy | Log in, create a new recipe collection titled "Quick Weeknight Dinners" and populate it with 5 chosen recipes. Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-11 | Failure | The AI agent got stuck in a loop of finding the place to save recipes | https://www.loom.com/share/51f9ca7e60e44865a15478a2550a27e3 |
| 10 | 63 | https://www.allrecipes.com | READ | easy | Search for a seasonal, fall-appropriate recipe and note the recommended serving size mentioned. Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-10 | Failure | The AI agent could not search for the reciepe as it ended up on a "Not Found" page | https://www.loom.com/share/ed63ca8a7c9b4c34b56fa81867de3967 |
| 11 | 67 | https://www.allrecipes.com | READ | easy | Locate a "chocolate chip cookie" recipe page and summarize the calorie count from its nutritional information. Only use http://allrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-9 | Failure | The AI agent could not search for the reciepe as it ended up on a "Not Found" page | https://www.loom.com/share/e87cf0e40fd947889cf810d61b2f5f8a |
| 12 | 73 | https://www.alltrails.com | DELETE | easy | Log in to your account, add the "Canyon Loop" trail to your saved trails, and then delete it—confirm that it no longer appears in your list. Only use http://alltrails.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-12 | Failure | The AI agent was not able to login via google | https://www.loom.com/share/54769f20cd9c4b669b7ad66ea9ea882b |
| 13 | 80 | https://www.alltrails.com | UPDATE | easy | Log in to your account and update your adventure profile by changing your favorite activity from "hiking" to "mountain biking." Only use http://alltrails.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-13 | Failure | The AI agent was not able to login via google | https://www.loom.com/share/2b2373e6764a457a952e5e5503640ba5 |
| 14 | 93 | https://www.apa.org | READ | hard | Use the website’s search function to find articles published in 2022 related to "clinical psychology" and output the titles of the first four matches. Only use http://apa.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-15 | Success | Succesfully finds the items after searching and filtering | https://www.loom.com/share/fbb1e33028024bb38f8ae439f965b5f1 |
| 15 | 102 | https://apkpure.com | READ | easy | Search for the app "Spotify" on APKPure and list the latest version number along with its release date as shown on the version history section. Only use http://apkpure.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-18 | Failure | The AI agent was not able to clear verification even after I had to intervene | https://www.loom.com/share/c5e9215f28da4a69a50dc9d2994516ae |
| 16 | 104 | https://apnews.com | READ | easy | Use the website’s search bar to look for articles containing "climate change" and list the titles of the first five articles returned. Only use http://apnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-19 | Success | Successfully finds the items after searching | https://www.loom.com/share/dbf7e1fb3a1940c7880dfd4160ede581 |
| 17 | 113 | https://archive.org | READ | easy | Use advanced search for ”Space images" on archive.org and output the capture dates and titles of the first 10 images listed. Only use http://archive.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-20 | Failure | The AI agent got random 10 images names not the first 10 as requested | https://www.loom.com/share/35bf8df54bea4470a7a8a2c20c39d440 |
| 18 | 125 | http://www.barnesandnoble.com | DELETE | easy | Log in, add the physical book "To Kill a Mockingbird" to your wishlist, then remove it and verify its deletion. Only use http://barnesandnoble.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-23 | Success | The AI agent was succesfully able to login and execute the deletion task | https://www.loom.com/share/93fba45ce67948e3825e09d3c1bf4f11 |
| 19 | 129 | http://www.barnesandnoble.com | READ | easy | Use the store locator by entering ZIP code 90210, then display the address, store hours, and contact details of the nearest location. Only use http://barnesandnoble.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-22 | Failure | The AI agent was not able to find the store locator even after trying multiple times | https://www.loom.com/share/79be1a58d3d44039bfcc21be2d8b1015 |
| 20 | 139 | https://www.bbb.org | READ | easy | Search for any companies on bbb.org and output the BBB rating of the first business listed in the results. Only use http://bbb.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-24 | Failure | The AI agent got stuck at the captcha even after I had to intervene | https://www.loom.com/share/b72888f2d2a54952a619158a0aa03dc8 |
| 21 | 140 | https://www.bbb.org | READ | hard | Search for any businesses in Los Angeles with a BBB rating of A+ and list the names of the first five businesses displayed. Only use http://bbb.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-25 | Failure | The AI agent got stuck at the captcha even after I had to intervene | https://www.loom.com/share/3102371cfdce4c4bb5400006ba960fae |
| 22 | 149 | https://www.bestbuy.com | CREATE | easy | Log into your Best Buy account and create a new wishlist titled "Holiday Shopping," then add the "Sony 65-inch 4K Ultra HD TV" to that wishlist. Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-29 | Failure | The AI agent got stuck at the login process | https://www.loom.com/share/b83f2c61e3e84ca4b58dde62bae83824 |
| 23 | 151 | https://www.bestbuy.com | CREATE | easy | On the "Sony 75-inch 4K TV" product page, post a new question in the Q&A section asking, "Does this TV support VESA wall mounting?" Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-26 | Success | The AI agent got a little stuck when I gave it a nickname that did not follow the format and then I intervened and it completed the task successfully | https://www.loom.com/share/e7df8ad4b5f94157bc7365491ebd1d95 |
| 24 | 159 | https://www.bestbuy.com | READ | easy | Locate the guide titled "How to Choose the Best 4K TV: The Essential Guide" and summarize the top 3 tips provided. Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-27 | Failure | The AI agent got maxed out in its steps and was not able to find the tips from the guide | https://www.loom.com/share/354a6054f7364accb955b42376618cc1 |
| 25 | 162 | https://www.bestbuy.com | UPDATE | hard | Log into your account, add an item to your cart, then switch the delivery option from standard shipping to in-store pickup and confirm that the order details update accordingly. Only use http://bestbuy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-28 | Failure | The AI agent got stuck at the login process | https://www.loom.com/share/985f6da81543474abddbfd45cd166f82 |
| 26 | 172 | https://www.booking.com | READ | hard | Search for 8 review score or above hotels in Manhattan, New York for next weekend with free cancellation, free Wi-Fi, and breakfast included. Compare at least four options and pick the one closest to the Empire State Building. Only use http://booking.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-30 | Failure | The AI agent did not apply any filters and it also did not compare four results as asked, so it gave wrong output at the end | https://www.loom.com/share/333f0c663c2247a08be79f0505634122 |
| 27 | 180 | https://www.buzzfeed.com | CREATE | easy | Using the Community submission portal, draft a BuzzFeed listicle titled “Top 10 Internet Memes of 2023” that includes five list items, each with a short description. Only use http://buzzfeed.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-33 | Failure | The AI agent was not able to complete the task and it timmed out | https://www.loom.com/share/bf6d961912eb416db76c233d7e3bd1c8 |
| 28 | 183 | https://www.buzzfeed.com | DELETE | hard | Log in, post a comment on one of the front page news articles on BuzzFeed, then locate this previously posted comment and delete the comment, ensuring it is removed from the article page. Only use http://buzzfeed.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-34 | Failure | The AI agent was not able to login via google | https://www.loom.com/share/7ba21c353cb343cdac168639fe593d03 |
| 29 | 198 | https://www.canada.ca | READ | easy | Navigate to the COVID-19 information section on Canada.ca and note three public health guidelines currently recommended. Only use http://canada.ca to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-36 | Failure | The AI agent was not able to complete the task as it was not able to open the site | https://www.loom.com/share/0f0913bcb5184ee1b73f0aee6e9b77ad |
| 30 | 206 | https://www.caranddriver.com | DELETE | hard | Log in to your Car and Drive account, save and article to your bookmarks, then delete the previously bookmarked article from your "My Bookmarks" list, and confirm that the article is no longer stored. Only use http://caranddriver.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-37 | Failure | The AI agent was not able to complete the task, as it was not able to save the article | https://www.loom.com/share/e1f0ec662db34a3e8334deaa15451ef6 |
| 31 | 209 | https://www.caranddriver.com | READ | easy | Navigate to Car and Driver’s homepage and list the titles of the top three featured car reviews. Only use http://caranddriver.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-39 | Success | The AI agent was successfully able to access the top three featured car reviews | https://www.loom.com/share/990dbca0fd9e4faba2661c08a3dde3bd |
| 32 | 213 | https://www.cars.com | READ | easy | In the "New Cars" section, check the pricing for 2024 BMW 3 Series 330e and list at least two pricing components (e.g., MSRP, incentives) that are displayed. Only use http://cars.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-40 | Failure | The AI agent got timmed out and it was not able to search for the exact model in the selection box on the top, instead it was trying to look for a search bar | https://www.loom.com/share/d3910e073e884ff9962b8b20302b575e |
| 33 | 220 | https://www.cbr.com | READ | easy | Navigate to the CBR homepage and list the titles of the top five most recent articles in the news section. Only use http://cbr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-42 | Success | The AI agent was successfully able to extract the top 5 news articles | https://www.loom.com/share/e187f60aecb34e7382e578ff81e41c95 |
| 34 | 221 | https://www.cbr.com | READ | easy | Navigate to an article featuring an embedded movie trailer and extract its title, description, and the link to the full trailer (if available). Only use http://cbr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-41 | Success | The AI agent was successfully able to extract the info from the embedded movie | https://www.loom.com/share/32ef212ed8a44de8ad44b29b1469d387 |
| 35 | 222 | https://www.cbr.com | READ | easy | Use the site’s search function to look up "Star Wars" and list the first five article titles with a brief note on each article’s focus. Only use http://cbr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-43 | Success | The AI agent was successfully able to gather 5 articles related to Star Wars and their focus | https://www.loom.com/share/56f3da44df734ada9dcda8b7efdc568e |
| 36 | 228 | https://www.cbssports.com | READ | easy | Check the live stream schedule and list the next two sports events along with their start times and the channels on which they will be broadcast. Only use http://cbssports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-44 | Success | The AI agent was able to successfully find the live stream schedule of Men's Final Four event | https://www.loom.com/share/b5fc8a4d7c3146c1836f198ddc9616a4 |
| 37 | 231 | https://www.cdc.gov | READ | easy | Find the latest measles advisory issued by the CDC, then note the publication date along with a brief summary of the advisory. Only use http://cdc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-45 | Failure | The AI agent was not able to load the website page | https://www.loom.com/share/e327d611f6df48c8a2cb5790a7e634a7 |
| 38 | 236 | https://my.clevelandclinic.org | READ | easy | Navigate to the Health Education section and list the top 5 articles on heart disease available on the Cleveland Clinic website. Only use http://clevelandclinic.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-46 | Failure | The AI agent failed to load proper UX of the website and it was also not able to find any articles | https://www.loom.com/share/2daf669b26a54151a82b834184bac2bf |
| 39 | 247 | https://www.cnet.com | READ | easy | Search for a CNET column on artificial intelligence and list any recommended products that the article highlights. Only use http://cnet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-50 | Success | The AI agent was successfully able to find the phones suggested in AI section's one article | https://www.loom.com/share/e5f8adcb6bae4e67a5d4d5c3b76ce091 |
| 40 | 249 | https://www.cnet.com | READ | easy | Browse any Deals or Offers section available and list the details of at least one affiliate deal mentioned. Only use http://cnet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-47 | Success | The AI agent was successfully able to find a affiliated deal and list its details | https://www.loom.com/share/ccbc3969bacb4b2ea1428e88a9621308 |
| 41 | 256 | https://collider.com | DELETE | hard | Log into your Collider newsletter subscription settings, ensure that communications have been subscribed to, and unsubscribe from all communications, then verify that a confirmation message is displayed. Only use http://collider.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-52 | Failure | The AI agent was not able to login via google | https://www.loom.com/share/67e8ed9781af419aa35650faac1ba459 |
| 42 | 261 | https://collider.com | READ | easy | Search for Collider’s coverage on a war film and provide a summary of the review highlights. Only use http://collider.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-51 | Success | The AI agent was successfully able to find a war movie review and provide a summary on it | https://www.loom.com/share/5d8ccb2af9ec43c68d7b296206950464 |
| 43 | 264 | https://www.collinsdictionary.com | READ | easy | Look up the word "onomatopoeia" and extract the main definition and one usage example from the Collins Corpus. Only use http://collinsdictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-53 | Failure | The AI agent could not get past the captcha page even after my intervention | https://www.loom.com/share/5fe9efc476a346cda5aa9682202fb514 |
| 44 | 266 | https://www.columbia.edu | READ | hard | Use the search functionality to locate pages detailing tuition and fees, then extract the published tuition fee information for undergraduate programs. Only use http://columbia.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-54 | Success | The AI agent was successfully able to extract tution fees info from the website | https://www.loom.com/share/b2ffc7bbab1d40a898fd9c2ef001fd7d |
| 45 | 267 | https://www.columbia.edu | READ | easy | Identify and list the dates for the upcoming Columbia University campus events from the events calendar. Only use http://columbia.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-55 | Failure | The AI agent could not get past the captcha page even after my intervention | https://www.loom.com/share/a2c149eda871458590d42365c231b74d |
| 46 | 269 | https://www.commonsensemedia.org | READ | easy | Use the search feature to find the expert review of the animated film “Frozen” and extract its recommended age range and one key critique. Only use http://commonsensemedia.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-56 | Success | The AI agent was successfully able to access the movie and extract info out of it | https://www.loom.com/share/18cf0e6241e04088ace949e10f5fd005 |
| 47 | 292 | https://newyork.craigslist.org | CREATE | easy | Log in to Craigslist, search for a baby stroller in your area in the "for sale" section, and save it to your favorites. Only use http://craigslist.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-57 | Success | The AI agent was successfully able to login and save the item in my favorites section | https://www.loom.com/share/5fc54e89a75849ff9569077aacccb598 |
| 48 | 299 | https://www.crunchbase.com | CREATE | hard | Log in to Crunchbase and create a new list titled "Tech Unicorns," then add the companies Uber, Airbnb, and Stripe to the list. Only use http://crunchbase.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-65 | Failure | The AI agent got blocked by Cloudflare due to security issues | https://www.loom.com/share/b8b10d79e00b48309c781e3ceecc8010 |
| 49 | 306 | https://www.crunchbase.com | CREATE | easy | Add "Tesla Inc." to your followed companies list so you receive future updates on funding and corporate changes. Only use http://crunchbase.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-63 | Failure | The AI agent got blocked by Cloudflare due to security issues | https://www.loom.com/share/50187b23a19943d3acf6a399186a0479 |
| 50 | 308 | https://www.crunchbase.com | CREATE | easy | Create a search for companies in Los Angeles and filter for companies in the E-commerce sector and save the updated query. Only use http://crunchbase.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-62 | Failure | The AI agent got blocked by the security features of the website so it could not move forward | https://www.loom.com/share/4d347e197d074e1c93e2ab5b33894db9 |
| 51 | 323 | https://www.crunchyroll.com | CREATE | hard | Access your account and set up a watchlist called "Classic Anime" by adding at least four classic anime series from the catalog. Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-71 | Failure | The AI agent failed to access the webpage with a message "Website is not accessible via this address." | https://www.loom.com/share/3684a729f39944cf88fc1af312f06c0f |
| 52 | 328 | https://www.crunchyroll.com | DELETE | easy | Sign in to your profile, create a new watchlist titled "Action-packed Series," then remove the watchlist titled "Action-packed Series," confirming its deletion from your account. Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-67 | Failure | The AI agent could not get past the captcha page even after my intervention | https://www.loom.com/share/29be046e48b848f992ca74c4bcbe61f3 |
| 53 | 331 | https://www.crunchyroll.com | READ | easy | Visit the news or blog section and summarize the details of an upcoming Crunchyroll event. Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-68 | Success | The AI agent was successfully able to access upcoming event info | https://www.loom.com/share/dd9de6280cd04dae8fce04dc3fed8834 |
| 54 | 332 | https://www.crunchyroll.com | READ | easy | Navigate to the details page of the anime "One Piece" and extract the available language and subtitle options. Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-66 | Success | The AI agent was successfully able to extract movie's language and subtitle options | https://www.loom.com/share/90cf8dada4564dd1bffb5031377d2ae3 |
| 55 | 334 | https://www.crunchyroll.com | UPDATE | easy | Log in to your account, create a "Weekend Binge" watchlist and add "Attack on Titan." Then open the "Weekend Binge" watchlist, and update it by adding the anime "Hunter x Hunter" to the list. Only use http://crunchyroll.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-69 | Failure | The AI agent failed to access the webpage with a message "Website is not accessible via this address." | https://www.loom.com/share/36e9c65055ba40309cbdfc31da490a13 |
| 56 | 364 | https://www.dickssportinggoods.com | CREATE | easy | Register a new user account on Dick’s Sporting Goods by filling out the required personal information and confirm the registration via the on-screen confirmation message. Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-74 | Failure | The AI agent was not able to access the webpage returning a message "It seems that the Dick's Sporting Goods website is currently unavailable in your region due to GDPR regulations". | https://www.loom.com/share/81ed828eb7f34fe1a27a1810830b537a |
| 57 | 368 | https://www.dickssportinggoods.com | CREATE | easy | Log in, write a detailed product review for a Wilson baseball glove including a star rating, a title, and your experience with the product, then submit the review. Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-77 | Failure | The AI agent was not able to access the webpage returning a message "It seems that the Dick's Sporting Goods website is currently unavailable in your region due to GDPR regulations". | https://www.loom.com/share/07458cf09cb447df84b97c5dd88a87d9 |
| 58 | 375 | https://www.dickssportinggoods.com | UPDATE | easy | Add a pair of Nike running shoes to your cart, then go to your cart and modify the quantity of the Nike running shoes from 1 to 2. Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-76 | Failure | The AI agent was not able to access the webpage returning a message "It seems that the Dick's Sporting Goods website is currently unavailable in your region due to GDPR regulations". | https://www.loom.com/share/1f3578ab63494c91bbda8d2ec297963d |
| 59 | 376 | https://www.dickssportinggoods.com | UPDATE | easy | Log in to your profile and update your address by adding your new apartment details. Only use http://dickssportinggoods.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-75 | Failure | The AI agent was not able to access the webpage returning a message "It seems that the Dick's Sporting Goods website is currently unavailable in your region due to GDPR regulations". | https://www.loom.com/share/192d282cd745486f87695f1a023ecf06 |
| 60 | 377 | https://www.dictionary.com | READ | easy | Look up the word "pronunciation," play its audio sample, and note the phonetic transcription displayed. Only use http://dictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-79 | Success | The AI agent was successfully able to extract the phonetic transcription for the given word | https://www.loom.com/share/2b8b8d5a413b460aaf5104d6d419c04a |
| 61 | 378 | https://www.dictionary.com | READ | easy | Look up the word "quintessential" and record both its pronunciation guide and the available audio sample. Only use http://dictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-78 | Success | The AI agent was successfully able to extract the audio sample and pronunciation of the given word | https://www.loom.com/share/597342a89dd24a798a34beaeb5228adb |
| 62 | 391 | https://www.dw.com | READ | easy | Visit the DW homepage and list the headline and publication time of the top news article featured in the main section. Only use http://dw.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-80 | Success | The AI agent was successfully able to find the main article and its publication time | https://www.loom.com/share/a49b9b22387a44d48a85babbbb530070 |
| 63 | 397 | https://www.eater.com | READ | easy | Locate a feature article on international dining scenes and list two featured cities mentioned in the piece. Only use http://eater.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-81 | Success | The AI agent was successfully able to extract two cities names from an featured article | https://www.loom.com/share/e8ce2c2b9bb9427883429d527f53524c |
| 64 | 403 | https://www.ebay.com | READ | hard | Find three different sellers offering the same "Blue Tooth Speaker" item. Compare the total cost (including shipping and taxes), seller ratings, return policies, and estimated delivery times. Create a table to compare the various speakers and recommend the best purchase option if I'm purchasing from New York City, NY. Only use http://ebay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-82 | Success | The AI agent was successfully able to provide a list of three sellers and a suggesstion on what should someone buy | https://www.loom.com/share/a95cb6c1a2b5499a965ed61ad43df447 |
| 65 | 412 | https://economictimes.indiatimes.com | READ | easy | Navigate to the Markets section and record the current BSE and NSE index values as displayed on the dashboard. Only use http://economictimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-83 | Success | The AI agent was successfully able to get the BSE and NSE current index values | https://www.loom.com/share/7f70c98b9e754b249e2407a1a4304906 |
| 66 | 417 | https://www.elle.com | READ | easy | Use the site’s search function to find articles about “Sustainable Fashion” and record the publication dates of the top 3 results. Only use http://elle.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-85 | Failure | The AI agent was not able to access the first 3 articles as it went with the 2nd, 3rd and 4th one | https://www.loom.com/share/904feb9f76f04945aa427f6ec9402bdd |
| 67 | 418 | https://www.elle.com | READ | easy | Identify the digital magazine section and record the publication date of the most recent issue available. Only use http://elle.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-84 | Failure | The AI agent was not able to extract info from the latest article it picked up an old article | https://www.loom.com/share/9e8b64dacca64fef9e6034017335fe5e |
| 68 | 420 | https://en.wikipedia.org/wiki/Main_Page | READ | easy | What does the 'Climate Change' article say is driving the current rise in global temperatures? Only use http://en.wikipedia.org/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-86 | Success | The AI agent was successfully able to find the insights on the question being asked | https://www.loom.com/share/2cdc4386214b447aaeafad09bb851f91 |
| 69 | 421 | https://en.wikipedia.org/wiki/Main_Page | READ | easy | Select the 'Page information' option under the 'Tools' dropdown to see how many page views the 'COVID-19' article has in the past 30 days. Only use http://en.wikipedia.org/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-87 | Success | The AI agent was successfully able to extract no. of viewers from the last 30 days on the given article | https://www.loom.com/share/d844111c5e754f1d88b683949166643b |
| 70 | 424 | https://www.encyclopedia.com | READ | easy | Use the search bar to find and list the title along with the publisher of the encyclopedia entry on "Climate Change." Only use http://encyclopedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-88 | Success | The AI agent was successfully able to extract info of the author from the article | https://www.loom.com/share/f5065819fbb145f9b226729bd7ef2f51 |
| 71 | 425 | https://www.encyclopedia.com | READ | easy | Search for "Artificial Intelligence" and list any related news or magazine articles or media referenced on the entry. Only use http://encyclopedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-89 | Failure | The AI agent was not able to find any references from the article | https://www.loom.com/share/e5d460ce24cb44dc865a6971cc958008 |
| 72 | 430 | https://www.eonline.com | READ | easy | Search for articles mentioning "Beyoncé" and provide the headlines and publication dates for the first three results. Only use http://eonline.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-90 | Success | The AI agent was successfully able to find the first three articles mentioning Beyonce | https://www.loom.com/share/7e47bec4acd44f38af97944ae0af71ff |
| 73 | 432 | https://www.epa.gov | READ | easy | Find the latest update on air quality data for Los Angeles on the EPA’s Air Quality System page and record the reported index value. Only use http://epa.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-92 | Failure | The AI agent failed to access the webpage directly and was trying to get in via google search but it failed to get in that way as well although we specifically mentioned not to open any other website | https://www.loom.com/share/e289aeea532a48ff8456a711ed2b2b38 |
| 74 | 433 | https://www.epa.gov | READ | easy | Locate the SmartWay program page and list two benefits that it offers for businesses. Only use http://epa.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-91 | Failure | The AI agent failed to access the webpage | https://www.loom.com/share/16a3c0eeb408476bb90b4f9be5dc46a1 |
| 75 | 445 | https://www.esquire.com | DELETE | hard | Create a comment on a recent article as described in a previous task, then delete that same comment and verify that it is no longer visible. Only use http://esquire.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-98 | Failure | The AI agent failed to access the comment section from the featured article and it also tried to access the article later through google which is not allowed as per our task | https://www.loom.com/share/1c6a0d81bd714e32b914e350229b20ac |
| 76 | 446 | https://www.esquire.com | READ | hard | Navigate through the archive by decade and note the publication year of one highlighted article from each decade (e.g., 1980s, 1990s, 2000s, etc.). Only use http://esquire.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-95 | Failure | The AI agent completed this task partially as it pulled articles publication years from 1980-2000s, however it missed the previous decades (1930-1970s) | https://www.loom.com/share/2e18c1ae23d94abc8db34bf0b1b371fe |
| 77 | 449 | https://www.esquire.com | READ | easy | Navigate to the Esquire homepage and list the headlines of the top 3 featured articles. Only use http://esquire.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-93 | Success | The AI agent was successfully able to extract the top three articles info from the homepage | https://www.loom.com/share/c2aa0ccf970741368b752e6da82152d3 |
| 78 | 453 | https://www.etsy.com | READ | easy | Sign in to your Etsy account using your email and password. Only use http://etsy.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-99 | Failure | The AI agent failed to solve the captcha while logging in, even after I had to intervene | https://www.loom.com/share/6ce4eabfdcb244099d77ce9c1b74e176 |
| 79 | 460 | https://www.eventbrite.com | READ | easy | Access the Eventbrite Help Center to locate guidelines on organizing virtual events; then list the key steps and recommendations provided. Only use http://eventbrite.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-100 | Failure | The AI agent failed to access the website directly and so it tried to search it on google and it is not allowed as per the task | https://www.loom.com/share/9da46221f036419f8467cebecd4c68a9 |
| 80 | 481 | https://www.expedia.com/ | READ | easy | Search for flights from New York’s JFK to Los Angeles’ LAX on June 15th with a return on June 20th, and list the top three cheapest fares. Only use http://expedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-103 | Failure | The AI agent failed to solve the captcha while logging in, even after I had to intervene | https://www.loom.com/share/ee6e1040dba1495bb084e5e5f86dbc51 |
| 81 | 488 | https://www.facebook.com | CREATE | hard | Log in and set up a new Facebook Page for a fictitious business called “Eco-Groceries” complete with a short business description. Only use http://facebook.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-108 | Failure | The AI agent failed to create a page on facebook as suggested because it ran out of attempts | https://www.loom.com/share/7dfd56ad84e04d888f6b28e9d6f8d1c2 |
| 82 | 495 | https://www.facebook.com | CREATE | easy | Log in, go to the Videos section to see Reels, and save the first reel that will appear on the page. Only use http://facebook.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-111 | Failure | The AI agent failed to save the reel in the platform itself so it returns the link to that reel | https://www.loom.com/share/1d8a10c83b3c4219be59aed6105051e2 |
| 83 | 505 | https://www.fandom.com | READ | hard | Find an image posted to the Stranger Things Wiki forum and identify the dimensions of the largest downloadable option Only use http://fandom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-112 | Failure | The AI agent was not able to find downloadable image with explicit dimensions on the Stranger Things Wiki forum and so it was not able to return the largest one | https://www.loom.com/share/be451fd872264f1c972c90f30937c380 |
| 84 | 508 | https://www.fda.gov | READ | easy | Locate the section dedicated to FDA-regulated foods and extract the risk factors listed in the Food Safety sub‐section. Only use http://fda.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-113 | Failure | The AI agent failed this task as it tried to access the website through google and not directly | https://www.loom.com/share/14bf8e0851d54d959f844f9ae7efed6c |
| 85 | 517 | https://www.firstcry.com | READ | easy | Search for baby strollers on FirstCry and list the prices, key features, and customer ratings of the top 3 models displayed. Only use http://firstcry.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-115 | Failure | The AI agent failed to pull out info of the top 3 strollers, however it did end up getting info on 3 random strollers | https://www.loom.com/share/5f851c60bdfb4f20acf4e68e0ffa34c3 |
| 86 | 519 | https://www.firstcry.com | READ | easy | Browse the maternity care products section and extract details (price, customer rating, and availability) for the "Maternity Support Belt." Only use http://firstcry.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-117 | Success | The AI agent was successfully able to find info on maternity belts as asked | https://www.loom.com/share/499c908683ae483baed6bafbce2c4b3d |
| 87 | 521 | https://www.firstcry.com | READ | easy | Check the "FirstCry Prime" membership page and note down the benefits offered, including free shipping and early sale access details. Only use http://firstcry.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-114 | Failure | The AI agent failed to find the benifits of prime membership as it was trying to look for them via the search bar located on homepage | https://www.loom.com/share/6f4cc65076014d459fccee67a99fa25b |
| 88 | 537 | https://flickr.com | READ | easy | Search Flickr for photos tagged "sunset" and list the titles and usernames of the first 5 results. Only use http://flickr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-120 | Failure | The AI agent failed the task due to running out of attempts and it was also facing problems accessing the search bar | https://www.loom.com/share/c1e16c32ca014000b21e5d300876f277 |
| 89 | 538 | https://flickr.com | READ | easy | Open the details of a selected photo and extract its metadata details (title, description, and tags) as displayed on its page. Only use http://flickr.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-121 | Success | The AI agent successfully found the info on a particular image | https://www.loom.com/share/44a2fe9854674e84af929136565dc003 |
| 90 | 543 | https://www.flipkart.com | READ | hard | In the "Laptops" section, apply the filter for "Dell" and extract the average discount percentage on the first 3 Dell laptops displayed. Only use http://flipkart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-124 | Failure | The AI agent failed to apply "Dell" filter in the laptops section and so it tried to directly search for dell in the search bar and ended up extracting other dell products with one laptop, so it gave wrong average of the discount percentage on top 3 dell laptops | https://www.loom.com/share/206e7fe22a244543adb36925f23c2934 |
| 91 | 545 | https://www.flipkart.com/ | READ | hard | Navigate to the "Mobiles" category, search for "iPhone", and list the product names, prices, and discount details of the top 5 search results. Only use http://flipkart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-123 | Failure | The AI agent failed to find top-5 products instead it only returned the first product with a poor UI experience of website being accessed | https://www.loom.com/share/2efdc69e89a64d4c9e939071be172b8c |
| 92 | 552 | https://food52.com | READ | hard | Log in, use the community forum search on Food52 to find posts tagged with “grilling” and list the titles of the first three posts along with the respective usernames of the posters. Only use http://food52.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-127 | Failure | The AI agent failed to get past the login page | https://www.loom.com/share/a7fbc52e51d1406284d640138ab79357 |
| 93 | 553 | https://food52.com | READ | easy | Browse the “A Few of Our Faves” section and list three recipes featured this week, including their titles. Only use http://food52.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-125 | Failure | The AI agent was not able to find the desired section as it never scrolled to the position where it was located and pivoted in between by going into bread section and providing the 3 featured recpies | https://www.loom.com/share/f5a3e071ffda42999eeab081e3a5f602 |
| 94 | 554 | https://food52.com | READ | easy | Use the Food52 search function to find a recipe that includes both lemon and thyme in the ingredients, then list the step-by-step cooking instructions of the first matching result. Only use http://food52.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-126 | Success | The AI agent was successfully able to find the required reciepe(steps to perform) with given ingredients | https://www.loom.com/share/a723c8e00d5f41cb840afec5aa9dd9d4 |
| 95 | 564 | https://www.ford.com | CREATE | hard | Establish a new saved configuration for a Ford Mustang Mach-E with added options such as an extended battery and panoramic sunroof, then note the generated build reference number. Only use http://ford.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-130 | Failure | The AI agent failed as the site returned a 'Forbidden' error, preventing access to the configuration page. | https://www.loom.com/share/70be73d2673c4bf39fcfea399a2e24d2 |
| 96 | 565 | https://www.ford.com | CREATE | hard | Utilize the financing calculator tool to simulate a Ford Credit financing option for a Ford Fusion, and output the estimated monthly payment along with the term details. Only use http://ford.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-129 | Failure | The AI agent failed to select a specific model in the price calculator as it was always ending up on a 404 not found page | https://www.loom.com/share/13f65fa22b8a4532ab848d9d5ffde56a |
| 97 | 567 | https://www.ford.com | DELETE | hard | Login, create a new saved vehicle configuration, then access your saved vehicle configurations and delete it, and confirm that it is no longer present in your saved builds list. Only use http://ford.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-128 | Failure | The AI agent failed to save the configuration of a particular vehicle | https://www.loom.com/share/25420774a12b436da600ad2ab6a7062b |
| 98 | 574 | https://fortune.com | READ | easy | Navigate to the Fortune homepage and list the titles of the top 5 featured business articles. Only use http://fortune.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-132 | Success | The AI agent successfully did extract the required information | https://www.loom.com/share/d6ca014b1bef4a90bc7fda6815a1c5ef |
| 99 | 577 | https://www.foxnews.com | READ | hard | Use the website’s search function to find an article related to the "US Economy" and extract the main points from the introduction. Only use http://foxnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-134 | Success | The AI agent successfully did extract the required information | https://www.loom.com/share/283440f9e97742388f28f4c99fbbd4c2 |
| 100 | 578 | https://www.foxnews.com | READ | easy | Retrieve an archived article on "immigration policy" from the site’s archive; summarize its publication date and primary focus. Only use http://foxnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-133 | Success | The AI agent successfully did extract the required information | https://www.loom.com/share/24f50ec7f4d84b96a70bb486d73dae37 |
| 101 | 582 | https://www.foxsports.com | READ | easy | Browse the video highlights section and list the titles of the five most recent NBA highlight videos. Only use http://foxsports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-135 | Failure | The AI agent failed to access the webpage directly and was trying to get in via google search but it failed to get in that way as well although we specifically mentioned not to open any other website | https://www.loom.com/share/e99247b2452d4207b6386df0582ee4ac |
| 102 | 591 | https://www.freepik.com | READ | easy | Navigate to the Reimagine tool page and note the supported input file formats along with any usage requirements mentioned. Only use http://freepik.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-136 | Failure | The Ai agent failed to extract the supported image upload formats and instead gave the steps to use the reimagine tool | https://www.loom.com/share/495394ecbc31410a83001c2044be6ef2 |
| 103 | 604 | https://gamerant.com | READ | easy | Use the search function to find articles about "Cyberpunk 2077" and provide the headlines of the first five results. Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-137 | Success | The AI agent successfully did extract the required information | https://www.loom.com/share/f39cc34d31844d5ca7ff5e0a0f77071d |
| 104 | 608 | https://gamerant.com | READ | easy | Search for latest PS5 review on Game Rant and extract the publication date along with a brief summary of the main criticisms. Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-139 | Success | The AI agent successfully did extract the required information | https://www.loom.com/share/bd97473f8a694c28bb96454f66d06096 |
| 105 | 609 | https://gamerant.com | UPDATE | hard | Log in to your account, post a comment on "Monster Hunter Wilds Review." Then navigate to your comment and update it with additional details. Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-138 | Failure | The AI agent was not able to login via google | https://www.loom.com/share/eb84921827ef411787bb693db1c6f001 |
| 106 | 612 | https://gamerant.com | UPDATE | easy | Change your display name in your account settings to include a gaming tag "rockstar36458" and confirm that the update is applied. Only use http://gamerant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-140 | Failure | The AI agent was not able to login via google | https://www.loom.com/share/16979098671940abaaaebc1fb65c55e9 |
| 107 | 615 | https://www.gamespot.com | READ | easy | Select the review article for "The Legend of Zelda: Tears of the Kingdom" and extract the review score along with three key highlights mentioned in the review. Only use http://gamespot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-141 | Failure | The AI agent was not able to extract review score from the article we requested and it did fidn the article but was not able to find the review score and tried to search it on google and that is not allowed as mentioned in our task. | https://www.loom.com/share/0d125f64fc234a1cbc97ab3a8e468545 |
| 108 | 622 | https://www.gamesradar.com | READ | easy | Use the website’s search function to look for articles on "comics" and list the titles of the first three results. Only use http://gamesradar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-142 | Success | The AI agent successfully did extract the required information | https://www.loom.com/share/950403e6293447d294b0b5eaa3f3cd9b |
| 109 | 652 | https://github.com | CREATE | easy | Fork the repository "microsoft/vscode" to your GitHub account and verify that the fork appears in your profile repository list. Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-144 | Success | The AI agent was successfully able to fork the suggested repo in the user's github account and confirm it | https://www.loom.com/share/d79781a32be54240893668791dc15c84 |
| 110 | 660 | https://github.com | DELETE | easy | Log in, create a repository called "OldProject" on your Github account then delete the repository named "OldProject" from your GitHub account and verify its no longer there. Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-146 | Failure | The AI agent was not able to get past the login page even after multiple attempts | https://www.loom.com/share/e22222d98f38448bb03a2f5c848ae9e5 |
| 111 | 661 | https://github.com | DELETE | hard | Log in, create a repository called "sample-repo" with a new branch "old-feature" then delete the "old-feature" branch using GitHub’s branch management options. Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-145 | Failure | The AI agent was not able to find options to create a branch on the newly generated repo | https://www.loom.com/share/e9fd8e816d9e46da964c0ff56dba4893 |
| 112 | 672 | https://github.com | UPDATE | hard | Create a repository named "sample-repo-2" then change the branch protection rules for the "main" branch in your repository to require at least one approving review before merging any changes. Only use http://github.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-147 | Failure | The AI agent was not able to get past the login page even after multiple attempts | https://www.loom.com/share/77c0a97303ae43dd8e59b0e2acbe4e04 |
| 113 | 680 | https://www.glassdoor.com/index.htm | CREATE | easy | Post a new discussion in a Glassdoor community Bowl about work-life balance at tech startups, providing your insights and suggestions. Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-150 | Failure | The AI agent was not able to access the website due to verification issues and did end up searching on google | https://www.loom.com/share/35deb214ef6e41eb9f0b135632233559 |
| 114 | 687 | https://www.glassdoor.com/index.htm | DELETE | hard | Log in to your Glassdoor account, post a new comment in a community discussion about job search tips, then delete your comment and verify that it is no longer displayed. Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-151 | Failure | The AI agent failed to solve the captcha while logging in, even after I had to intervene | https://www.loom.com/share/e49a800aee7e43d8a5e95692c2c5ee02 |
| 115 | 688 | https://www.glassdoor.com/index.htm | READ | easy | Navigate to the New York City Bowl and display the number of members as well as the bio of the bowl. Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-149 | Failure | The AI agent was not able to access the website due to verification issues | https://www.loom.com/share/95e6aa2bb844406095eb0bf4adb9023f |
| 116 | 689 | https://www.glassdoor.com/index.htm | READ | hard | Navigate to the "Interviews" section, search for interview experiences at "Tesla", and report the most frequently mentioned interview question. Only use http://glassdoor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-148 | Failure | The AI agent was not able to access the website due to verification issues | https://www.loom.com/share/a85d93bf78bd486a9db8dcc8e8030b53 |
| 117 | 695 | https://www.goal.com/en-us | READ | easy | Find the match analysis of the last Champions League game and write down the main tactical observation discussed in the article. Only use http://goal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-152 | Failure | The agent was able to return an analysis of a champions league match, but it failed to identify the most recent match played. | https://www.loom.com/share/33f63f7309944aafb35e4465d966e5dd |
| 118 | 703 | https://www.goodhousekeeping.com | DELETE | hard | Log in to your GoodHousekeeping.com account, add one recipe to "Weekend Cooking" list, and then remove one recipe from your "Weekend Cooking" list, and verify that it no longer appears in your saved items. Only use http://goodhousekeeping.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-153 | Success | The agent was able to add the recipe, delete it, and return an executive summary of the task process. | https://www.loom.com/share/2dd0b3f185c7458c99b3736b9e3f248e |
| 119 | 710 | https://www.goodhousekeeping.com | READ | easy | In the Beauty category, filter for articles discussing skincare routines and list the publication dates of the first three posts. Only use http://goodhousekeeping.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-156 | Success | Agent was able to locate the three publication dates of the articles as tasked. | https://www.loom.com/share/ab1af3cb38a44946b46f32868a294828 |
| 120 | 715 | https://www.goodreads.com | READ | easy | Search for the book "To Kill a Mockingbird" on Goodreads, view its ratings, and list the top 3 user reviews. Only use http://goodreads.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-157 | Success | Agent was able to find the review for the book and three top reviews and succinctly report it. | https://www.loom.com/share/3376003a4599447f88434c4958a7dd68 |
| 121 | 746 | https://www.grubhub.com | READ | easy | Navigate to Catering, and summarize 5 of the FAQs, as well as the associated answers. Only use http://grubhub.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-159 | Failure | Agent got stuck in a loop of the two same pages, and could not locate or navigate to the FAQ page, | https://www.loom.com/share/23e31f1a067144c4a7ea391c975157f8 |
| 122 | 751 | https://www.harpersbazaar.com | READ | easy | Search for the article on "Spring Fashion Trends" and extract the names of the five luxury brands mentioned in it. Only use http://harpersbazaar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-161 | Success | Agent successfully identified 5 brands from the chosen article as tasked. | https://www.loom.com/share/f9aa0c0f25e24234adda3d38349b7cad |
| 123 | 756 | https://www.healthline.com | READ | easy | Use the search tool to look for “diabetes management” and output the titles of the first five articles displayed. Only use http://healthline.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-163 | Failure | Agent was unable to find the search bar, and could not navigate the surrounding sections. | https://www.loom.com/share/eb7c03f2aca94a86b26efbe8198662ae |
| 124 | 758 | https://www.healthline.com | READ | easy | Navigate to Healthline’s homepage and list the titles of the top five featured health articles. Only use http://healthline.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-164 | Success | Agent was able to locate and return 5 features articles on the home page. | https://www.loom.com/share/fbb450067e3b43a182c5a4bd5602504e |
| 125 | 765 | https://www2.hm.com/en_us/index.html | READ | easy | View the "Sale" section and record the discount percentage displayed on the first product. Only use http://hm.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-165 | Failure | Human timed agent out - agent got stuck in a loop and could not navigate to or locate the sale section of the website. | https://www.loom.com/share/2fe10bb3888442faa760b40f012714ef |
| 126 | 771 | https://www.homedepot.com | CREATE | easy | Log in to your Home Depot account and create a new project list titled “Spring Garden,” then add the product “Husky Outdoor Planter Box” to the list. Only use http://homedepot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-166 | Failure | Issues logging in - agent and human tried multiple times, but there was a disconnect between the website and convergence. | https://www.loom.com/share/4ee6b5ae35054a549f9331550aad310a |
| 127 | 775 | https://www.homedepot.com | CREATE | easy | Submit a “Contact Us” inquiry regarding delivery options for a “Samsung Family Hub Refrigerator,” including your contact details and a brief question. Only use http://homedepot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-167 | Failure | Agent was unable to connect with the website server and got denied access. | https://www.loom.com/share/60b09d619f9a4842a0bfd908a412ab06 |
| 128 | 782 | https://www.homedepot.com | READ | easy | Search for “porch lumber” on Home Depot and extract the pricing details for the 2x4 pressure-treated lumber available for delivery. Only use http://homedepot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-168 | Failure | Agent ran into "access denied" issues when trying to access the information on the Home Depot site. | https://www.loom.com/share/ae8c1ba546ba42a585fd39b5ae63fc82 |
| 129 | 793 | https://www.hotels.com | READ | hard | Filter search results for properties in Paris available next month that offer spa amenities and bars, and list the amenities of the first three hotels. Only use http://hotels.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-169 | Failure | Agent attempted to solve the CAPTCHA requirement before asking human to, which resulted in the website blocking due to a fail in the CAPTCHA. | https://www.loom.com/share/8faadb3cff824973b4dbc3b83501da20 |
| 130 | 799 | https://www.housebeautiful.com | CREATE | easy | Sign up for notifications on new articles by entering your email and opting in for alerts on interior design trends. Only use http://housebeautiful.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-170 | Failure | Agent could not find the unpaid newsletter option, so it went to look it up on Google, which is out of bounds for the task. | https://www.loom.com/share/aa6a0b33812946c3a73fdf61ae1fbbdc |
| 131 | 804 | https://www.housebeautiful.com | READ | easy | Navigate to the homepage and list the titles of the top 3 featured articles on modern interior design. Only use http://housebeautiful.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-172 | Failure | Agent returned an article, but did not return three. Thus, task is a failure, | https://www.loom.com/share/d99640af62bc47d5a67ea39dfdc9e72e |
| 132 | 819 | https://www.ign.com | READ | easy | Navigate to the Guides section, Locate IGN’s GTA V guide, read the cheat codes section, and provide a summary of at least three cheat codes mentioned. Only use http://ign.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-176 | Success | Agent was able to find and summarize three GTA V cheat codes as tasked. | https://www.loom.com/share/16203c5183734e35a358a29594c307b7 |
| 133 | 827 | https://www.ikea.com | CREATE | easy | Log in and post a review for the "BILLY bookcase" emphasizing its ease of assembly, then share your review on the product page. Only use http://ikea.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-178 | Failure | Agent got stuck at the login phase again, this time misreading the user@example.com email as the email input filled out, which it had to fill out with user email. Agent did not ask for the user's email, so it got stuck and was a fail. | https://www.loom.com/share/3438c4f82d5943e9a78ce2130f2f7dc4 |
| 134 | 839 | https://www.ikea.com | UPDATE | easy | Access your profile, navigate to your saved addresses, and update your default delivery address to include specific instructions for building access. Only use http://ikea.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-177 | Failure | IKEA and agent could not log in, even when human provided credentials. | https://www.loom.com/share/2e6ed0a8d72a4559bdfdad3524ef5594 |
| 135 | 850 | http://indeed.com | READ | easy | Check if a "Delivery Driver" job in Orlando provides 401k and paid time off. If it does, add it to my save list. Only use http://indeed.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-180 | Failure | CAPTCHA task was unable to be solved by agent or by human. | https://www.loom.com/share/2839c1f0c5c446868f0f5670de3bf81b |
| 136 | 854 | https://www.independent.co.uk/us | READ | easy | Navigate to the "World" news section on Independent.co.uk and list the headlines of the top 3 articles. Only use http://independent.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-181 | Success | Agent was able to navigate to the World section and find the headlines of the top 3 articles. | https://www.loom.com/share/ca0f7c54cc9444ec9fa52ed453a53b0e |
| 137 | 856 | https://www.india.com | READ | easy | Locate the article covering the latest economic policy announcement in the Business section and note its publication date and headline. Only use http://india.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-182 | Failure | Agent was not able to find and return an article related to economic policy. | https://www.loom.com/share/73733ac885a441638e1712a5ff5e8b16 |
| 138 | 860 | https://www.indiamart.com | READ | easy | Look up "LED lighting systems" and note any indicators of supplier verification and lead response times that appear on the product pages. Only use http://indiamart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-183 | Success | Agent was able to identify supplier verification for multiple pages, and also reported that it could not identify any lead times listed on the pages, | https://www.loom.com/share/a770ce748ea94f809a1e5181290fa886 |
| 139 | 861 | https://www.indiamart.com | READ | easy | Use the search function to find "organic tea leaves" and list the top 5 supplier names along with their location details. Only use http://indiamart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-184 | Success | Agent was able to identify supplier details for tea leaves as tasked. | https://www.loom.com/share/fcdf01e5b30346ebadd90b265b94929f |
| 140 | 873 | https://www.instacart.com | CREATE | easy | Search for "Peanut Butter and Jelly", add it to your cart, and then go to your cart to check the estimated delivery time for the order. Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-186 | Failure | Agent's website wasn't loading so it decided to perform a Google search which is out of bounds. | https://www.loom.com/share/9c56b0d5f1e74c1996d0de189fe291b0 |
| 141 | 875 | https://www.instacart.com | CREATE | easy | Log in to Instacart and add “Organic Carrots” to your shopping cart for immediate checkout. Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-185 | Failure | Login access was blocked after "request was invalid" - tried to sign in with Google and was blocked. | https://www.loom.com/share/7860684a9cbd417d9aa76a532d5a9623 |
| 142 | 883 | https://www.instacart.com | READ | hard | Search for organic bananas on Instacart and list the top 3 prices along with their retailer names. Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-187 | Failure | Agent faced persistent technical issues from the instacart website that made it difficult to accomplish the task. | https://www.loom.com/share/cb227c736e034c259dd9203b3e165fc8 |
| 143 | 885 | https://www.instacart.com | UPDATE | easy | Log in to your Instacart account, add 6 bunches of organic bananas to your cart, and then access your cart and update the quantity of your organic bananas from 6 to 10 bunches. Only use http://instacart.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-188 | Failure | Login access was blocked after "request was invalid" - tried to sign in with Google and was blocked. | https://www.loom.com/share/cc5754b85b75409e980785a89b3b1bb6 |
| 144 | 890 | https://www.instructables.com | READ | easy | Filter projects tagged with "woodworking" and list the titles and authors of the first 5 Instructables. Only use http://instructables.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-189 | Success | Agent was able to find and return the first 5 woodworking instructables. | https://www.loom.com/share/10b032badee344189b708e2a0e6eb7c3 |
| 145 | 895 | https://www.investopedia.com | READ | easy | Go to the market analysis section, locate the latest update on US economic indicators, and extract the names of at least three key indicators mentioned. Only use http://investopedia.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-190 | Success | Agent was not only able to extract three indicators, but ten. | https://www.loom.com/share/32b875482cf24d0bbd2bb62128af4380 |
| 146 | 897 | https://www.irs.gov | READ | easy | Go to the Help page on IRS.gov that explains Direct Pay options for individuals and list the available payment methods mentioned. Only use http://irs.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-191 | Failure | Agent's website wasn't loading so it decided to perform a Google search which is out of bounds. | https://www.loom.com/share/93d84568bc3c498d99bf289e3dd099a0 |
| 147 | 905 | https://www.istockphoto.com | READ | easy | Filter search results for "business meeting" by horizontal orientation, then list the first 5 image descriptions or titles displayed. Only use http://istockphoto.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-192 | Success | Agent was able to return the description and title of the first 5 photos listed. | https://www.loom.com/share/de4f3dfe1841494aad0d8967f873b71b |
| 148 | 913 | https://www.jagranjosh.com | READ | easy | Search for the "Previous Year Question Papers" section and report the number of downloadable papers available for the UPSC exam. Only use http://jagranjosh.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-194 | Success | Agent found many previous years' papers and returned the names and ability to unlock them. | https://www.loom.com/share/51a421176b8243d288392bdc3cbfb780 |
| 149 | 916 | https://www.jagranjosh.com | READ | easy | Check the academic calendar section for upcoming exam dates and list the next three major exams along with their schedules. Only use http://jagranjosh.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-193 | Failure | Agent was able to locate the exams on the academic calendar, but only returned two instead of three, constituting a failure. | https://www.loom.com/share/e9b2af63fe044ce993ceead038578d7c |
| 150 | 919 | https://www.jstor.org | READ | easy | Search for an article on "Innovation in Urban Design" and list the keywords associated with it from the metadata. Only use http://jstor.org/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-195 | Failure | Agent struggled to format the date correctly and got stuck. It was inefficient with its methodology and thus timed out. | https://www.loom.com/share/5bf5c971fc004d0ca8c3299971e55772 |
| 151 | 927 | https://www.justdial.com | READ | easy | Locate the “24x7 Emergency Medical Clinic” in Bangalore and display its operating hours along with a brief description of services. Only use http://justdial.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-196 | Failure | Human timed agent out - agent got stuck in a loop and could not navigate to or locate the correct section of the website. | https://www.loom.com/share/8a81b2e1888841f0a324ec6b36b5dcf2 |
| 152 | 943 | https://www.khanacademy.org | READ | easy | Search for "Algebra" courses on Khan Academy and list the titles and short descriptions of the first five results. Only use http://khanacademy.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-199 | Success | Agent was able to locate and return the top 5 results when "algebra" was searched. | https://www.loom.com/share/2ab06c892c364788a65a21fcc965f30b |
| 153 | 945 | https://www.khanacademy.org | READ | easy | On the Khan Academy homepage, list all available interface languages and note the total number of languages offered. Only use http://khanacademy.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-197 | Failure | Human timed agent out - agent couldn't perform a search on the homepage, which human could complete relatively quickly. | https://www.loom.com/share/22db6c99513f4b5e8eaffc5c2d472ad8 |
| 154 | 948 | https://www.khanacademy.org | UPDATE | easy | Log in to your student account and update your monthly learning goal on the progress dashboard to reflect a new target for completed exercises. Only use http://khanacademy.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-198 | Failure | Login access was blocked after "request was invalid" - tried to sign in with Google and was blocked. | https://www.loom.com/share/f6a9d9d4a9044040a90f2fe655047037 |
| 155 | 953 | https://kidshealth.org | READ | easy | Navigate to the "Parents" section and locate an article about managing screen time for children; then, provide a brief summary of the article title and its main tips. Only use http://kidshealth.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-200 | Success | Agent was able to locate an article on screen time management and return an accurate summary. | https://www.loom.com/share/efdbfdee073545d192c3e8b09140c677 |
| 156 | 958 | https://www.kmart.com.au | CREATE | hard | Log in to your Kmart account, create a new wishlist titled "Holiday Gifts," and add 3 items representing gifts for him, for her, and for kids. Only use http://kmart.com.au/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-201 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/5005577ca2914348baf60c230538c35e |
| 157 | 960 | https://www.kmart.com.au | DELETE | easy | Log in to your Kmart account, add a "Camera" to your shopping cart, then remove the "Camera" item from your shopping cart, and confirm its deletion from the cart. Only use http://kmart.com.au/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-202 | Failure | The AI agent failed to login as the webpage did not had an option to login using google | https://www.loom.com/share/eeb8fdfb194e4c048d41bd3c1a6c09ac |
| 158 | 977 | https://www.latimes.com | READ | easy | Locate a feature article with a photo gallery and list the captions of the photos presented within the gallery. Only use http://latimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-204 | Success | The Ai agent successfully found the captions from an image present in a featured article | https://www.loom.com/share/6dfa460e39604be3bf50612e90af5fc1 |
| 159 | 980 | https://www.latimes.com | READ | easy | Use the search function to locate articles on “Los Angeles” and “immigration”, then provide the titles of the first three results. Only use http://latimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-205 | Success | The AI agent successfully found articles related to the tags described in the task | https://www.loom.com/share/ee4e4bc8a0fa417dad951c9122f0753f |
| 160 | 981 | https://www.latimes.com | READ | easy | Browse the entertainment section and list the top three celebrity news headlines along with their publication dates. Only use http://latimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-206 | Success | The AI agent was successfully able to find information as requested | https://www.loom.com/share/53a43028d9374c5cba1906d8f5cfc091 |
| 161 | 992 | https://letterboxd.com | CREATE | hard | Compose a detailed film review diary entry for "The Grand Budapest Hotel," focusing on its cinematography, and rate it 5 stars. Only use http://letterboxd.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-207 | Failure | The AI agent failed to find the info directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/af03f005dd0e4e3e9ca2f1e4955a3f55 |
| 162 | 995 | https://letterboxd.com | CREATE | easy | Update your profile with a public update listing your top 3 favorite films of all time. Only use http://letterboxd.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-208 | Failure | The AI agent failed to add three films to favorite films list as it ended up adding only one | https://www.loom.com/share/52272e611d8745a2ab8aba4794bc035c |
| 163 | 1015 | https://www.linkedin.com | CREATE | easy | Add a job history entry to your profile titled Analyst at LabRite, from June 2021 - September 2023. Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-213 | Success | The AI agent successfully added the experience as requested | https://www.loom.com/share/107bf4e3d54e49588fbd56545cddbedc |
| 164 | 1017 | https://www.linkedin.com | CREATE | easy | Comment on a recent article posted by Google, ensuring that your comment has a positive and professional connotation. Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-210 | Failure | The AI agent failed to post a positive comment on google's recent post as it just wrote it but never submitted it | https://www.loom.com/share/2334575f20bd4d8387699c74e664d78a |
| 165 | 1021 | https://www.linkedin.com | CREATE | easy | Compose a text-based post asking your network for advice on upskilling in data analytics and mention two trending skills you wish to learn. Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-209 | Success | The AI agent successfully composed a post about two skills it wants to learn and returned that as an final output | https://www.loom.com/share/a37bc47133f14955bb142e6ce1f30f16 |
| 166 | 1033 | https://www.linkedin.com | UPDATE | easy | Make a post thanking several colleagues. Then edit the post to add an additional relevant hashtag before resubmitting it. Only use http://linkedin.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-211 | Failure | The AI agent did create the post and edited it as well but never posted it again. | https://www.loom.com/share/36ac54ea6dd9404182e4dab356a3f035 |
| 167 | 1041 | https://www.livejournal.com | DELETE | easy | Log in, create a temporary journal entry titled "Temporary Post," then delete this entry and confirm that it no longer appears on your journal. Only use http://livejournal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-215 | Success | The AI agent successfully created a temporary post and deleted it as well | https://www.loom.com/share/a283fbf86bd24ee399f6325376baaed4 |
| 168 | 1043 | https://www.livejournal.com | DELETE | hard | Login, create a friends list and add 3 individuals, then from your friends list, remove a specific friend by selecting their profile and confirming their deletion from your contact list. Only use http://livejournal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-214 | Failure | The AI agent failed to add friends into the list and was trying to submit a support request for issues with their text field | https://www.loom.com/share/fcb35eea80334ab1bd60bcb664bcd8e0 |
| 169 | 1050 | https://www.livejournal.com | UPDATE | hard | Log in and create a journal entry and schedule send it for 12 hours later. Then modify a scheduled journal entry by changing its posting time to an earlier hour on the same day as originally planned. Only use http://livejournal.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-216 | Failure | The AI agent was not able to access the Publish Time option and schedule a post at a particular time | https://www.loom.com/share/738aeb1714064ddcb389dc1089fe3db0 |
| 170 | 1052 | https://www.livescience.com | READ | easy | Use the LiveScience search bar to look up "volcano research" and list the titles of the relevant articles that appear. Only use http://livescience.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-217 | Success | The AI agent successfully ofund the required info on articles | https://www.loom.com/share/4ab6228b88e843db81e4adeda59c225f |
| 171 | 1053 | https://www.livescience.com | READ | easy | Navigate to the "Space" category and extract the title, author (if available), and publication date of the first article. Only use http://livescience.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-218 | Success | The AI agent successfully ofund the required info on a featured article | https://www.loom.com/share/438665a91f0a4c3e8eab556d0a21dfc9 |
| 172 | 1057 | https://www.loc.gov | FILE_MANIPULATION | easy | Download a digitized manuscript related to early American literature from the catalog and verify that the preview includes the word "manuscript." Only use http://loc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-219 | Failure | The AI agent failed to access website url with an error message displayed "The requested URL could not be retrieved" | https://www.loom.com/share/b0930e5622a34620a3215a26c6f8245a |
| 173 | 1063 | https://www.lonelyplanet.com | READ | easy | Use the search function to find articles on “Nepal trekking” and extract the titles of at least three related articles. Only use http://lonelyplanet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-220 | Failure | The AI agent failed to find 3 relevant articles as it was trying to look for the exact same words "Nepal trekking" in those tiles and so it took a lot of time to find those kind of articles | https://www.loom.com/share/c468e84f25324c90971b727c5b419123 |
| 174 | 1084 | https://www.macys.com | READ | easy | Find the product description and reviews for a specific item. Only use http://macys.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-225 | Success | The AI agent successfully found the required information | https://www.loom.com/share/82d6b3168cc8457a9b6a96e4066a52d0 |
| 175 | 1087 | https://www.made-in-china.com | READ | hard | Navigate to the suppliers profiles section, select a verified supplier offering "electronic components," and extract the certification details provided on their profile. Only use http://made-in-china.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-226 | Success | The AI agent successfully found the certification information of a verified supplier | https://www.loom.com/share/6681c346e7674811a8760dbd2d1e0aae |
| 176 | 1094 | https://www.marksandspencer.com | READ | easy | Navigate to the "Food" section and list the names and prices of the first five ready-meal products. Only use http://marksandspencer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-227 | Success | The AI agent was able to find the information from Food section as requested | https://www.loom.com/share/dc33774c884e4265b6349c7dc8763463 |
| 177 | 1096 | https://www.marksandspencer.com | READ | easy | Go to the "Help" or "Customer Service" section and list the top five frequently asked questions (FAQs) displayed. Only use http://marksandspencer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-228 | Success | The AI agent successfully found 5 FAQ's as requested | https://www.loom.com/share/2b2b16d139de40de9a96afdc8c59ad27 |
| 178 | 1106 | https://www.maryland.gov/Pages/default.aspx | READ | easy | Use the instant search feature to look up "public records retention policy" and provide the URL of the first matching result. Only use http://maryland.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-230 | Failure | The AI agent was not able to access the webpage | https://www.loom.com/share/1de392d8787f4691bddf876f597c77d3 |
| 179 | 1107 | https://www.maryland.gov/Pages/default.aspx | READ | easy | Explore the "Feedback" section and summarize the process given for submitting citizen input. Only use http://maryland.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-229 | Failure | The AI agent was not able to access the webpage | https://www.loom.com/share/e08a092d308741e0b79b4ebd3fda3057 |
| 180 | 1109 | https://mashable.com | READ | easy | Find an article that includes a section on upcoming events or festivals, and list the event names mentioned within that section. Only use http://mashable.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-231 | Success | The AI agent was successfully able to find the events from a particular article section | https://www.loom.com/share/b9a4090f0de24914a61c0a79fb66289a |
| 181 | 1110 | https://mashable.com | READ | easy | Browse the Mashable homepage and list the top three trending headlines along with their publication dates. Only use http://mashable.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-232 | Success | The AI agent was successfully able to find the articles and their publication dates | https://www.loom.com/share/ca9173faffdd45738885886332f2ed9c |
| 182 | 1126 | https://www.medscape.com | READ | easy | Use the search feature to find articles on "COVID-19 long-haulers" and list the first five article titles that appear. Only use http://medscape.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-233 | Success | The AI agent successfully found the information as requested | https://www.loom.com/share/b87d92bd11724f0dbe7ced05067f1771 |
| 183 | 1134 | https://www.mercari.com | READ | hard | Browse the electronics category and calculate the average price of used smartphones shown in the listings. Only use http://mercari.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-234 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/9227bef7559e4eb686f5abadc6f15ee0 |
| 184 | 1144 | https://www.metacritic.com | READ | easy | Use the site search to find reviews for “The Shawshank Redemption” and extract both the critic consensus and the overall user score. Only use http://metacritic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-236 | Failure | The AI agent was not able to access the search bar | https://www.loom.com/share/a27fe9415c8c4e3b8899a7a351da5c50 |
| 185 | 1147 | https://www.metacritic.com | READ | easy | Use the search bar to locate reviews for the film “Titanic” and find the publication date of one of the critic reviews. Only use http://metacritic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-235 | Failure | The AI agent was not able to access the search bar | https://www.loom.com/share/9b4df13a83354a4c9b74f0d18e8c9e90 |
| 186 | 1154 | https://www.michigan.gov/som | READ | easy | Locate the most recent press release on state economic development on Michigan.gov and extract the release date along with the main headline. Only use http://michigan.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-237 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/a1654884b9e5447489f78bef1fd058d7 |
| 187 | 1166 | https://www.mountsinai.org | READ | easy | Search for information on virtual urgent care service and outline the steps a patient should follow to access a virtual consultation. Only use http://mountsinai.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-238 | Success | The AI agent successfully found the relevant information as asked in the task | https://www.loom.com/share/6749cf662c62444da10ecb3c558fda7d |
| 188 | 1197 | https://www.ndtv.com | READ | easy | Locate the regional news segment for Rajasthan and list the top three headlines along with their brief introductions. Only use http://ndtv.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-240 | Success | The AI agent successfully found the information from Rajasthan section as requested | https://www.loom.com/share/ec6a6475ce91431ca8e8fd15f4dcd9e4 |
| 189 | 1198 | https://www.ndtv.com | READ | easy | Navigate to the Business section in the footer and list the titles of the five most recent articles along with their publication dates. Only use http://ndtv.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-239 | Failure | The AI agent failed to get the titles and dates of the articles directly from the website as it was trying to search them on google after a while | https://www.loom.com/share/86a64181c4424d33ad0b5609a240132a |
| 190 | 1210 | https://www.newegg.com | READ | easy | Search for "external SSD" in the storage category, apply a capacity filter for 1TB, and list the top three product models with their prices. Only use http://newegg.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-242 | Failure | The AI agent was not able to access the website due to unusual traffic detection | https://www.loom.com/share/1a8ded24e3b5439894247c63f0c11e6b |
| 191 | 1212 | https://www.newegg.com | READ | easy | Visit the Newegg Premier membership page, read through the listed benefits, and summarize at least five key membership perks. Only use http://newegg.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-243 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/75a191fa7e2545d7bec3c644a9991cc9 |
| 192 | 1224 | https://www.nj.com | READ | easy | Locate the "Food & Dining" guide and extract the names of three recommended local restaurants highlighted in the guide. Only use http://nj.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-244 | Success | The Ai agent successfully found the dining places names from the food & dining section | https://www.loom.com/share/b131f6650f2a4119b5fabb6bc89b7849 |
| 193 | 1230 | https://www.nordstrom.com | DELETE | easy | Log in, add the "Ray-Ban Aviator Sunglasses" to your wish list, then remove that item and confirm its deletion from your account. Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-245 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/f9b5b69665b6429380d350f44714e9fd |
| 194 | 1231 | https://www.nordstrom.com | DELETE | easy | Log in, create a wish list titled "Past Purchases", and then permanently delete that wish list. Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-246 | Failure | The AI agent failed to get past the login page | https://www.loom.com/share/418d71a77f4a4825892e9c6d941c129a |
| 195 | 1233 | https://www.nordstrom.com | READ | easy | Use the search bar to locate BOSS Menswear Suit and record the product name, price, and available sizes displayed. Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-247 | Failure | The AI agent failed to open the website directly | https://www.loom.com/share/2b9f1aa0523f4df6add88164221d0b04 |
| 196 | 1236 | https://www.nordstrom.com | READ | easy | Navigate to the "Under $100" section in the Women's Handbags category and record the names, prices, and availability of the first three items. Only use http://nordstrom.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-248 | Failure | The AI agent did find the links for three handbags as asked but it was then blocked to get more information on price and availability | https://www.loom.com/share/08f15c880a55460f9fba742bd3dfa5ac |
| 197 | 1240 | https://www.nps.gov/index.htm | READ | easy | Navigate to the homepage and search find a park by state in Wyoming on the first page Only use http://nps.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-249 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/81acc3b30a534b5393e7a126d53c9d3a |
| 198 | 1241 | https://www.ny.gov | READ | easy | Search for official guidelines regarding the renewal of a New York State driver’s license, and note the contact information provided. Only use http://ny.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-250 | Failure | The AI agent failed to open the website directly | https://www.loom.com/share/b268a4df029e492d957aa519f541ad30 |
| 199 | 1245 | https://www.nyc.gov | READ | easy | Search for the FAQ page on recycling and waste management services and extract two common questions along with their answers. Only use http://nyc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-252 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/cd7bfdd2383c4171a86984994566690d |
| 200 | 1246 | https://www.nyc.gov | READ | easy | Navigate to the "Benefits & Support" section and list the names of three programs available to NYC residents. Only use http://nyc.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-251 | Failure | The AI agent failed as proxy was not able to finish the task | https://www.loom.com/share/18a4d964e9534edda0546b52e8f35246 |
| 201 | 1254 | https://nypost.com | READ | easy | Browse the News section and summarize the leading article by noting its headline, a brief excerpt, and the publication time. Only use http://nypost.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-255 | Success | The AI agent successfully found the information | https://www.loom.com/share/d117d187807a4d89a1d849afae99a33a |
| 202 | 1257 | https://nypost.com | READ | easy | Go to the Business section and extract the headline of the article discussing the latest market trends. Only use http://nypost.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-254 | Success | The AI agent successfully found the information | https://www.loom.com/share/6e187876d9934456b38c3975270c6c58 |
| 203 | 1259 | https://nypost.com | UPDATE | hard | Log in to the NY Post comment section, leave a comment on an article from the front page, then locate this comment, and edit it to add more information. Only use http://nypost.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-253 | Failure | The AI agent opened Twitter to comment on a post which we asked to actually do it on the article itself. | https://www.loom.com/share/b893dbc36712462385a74003e236ca3c |
| 204 | 1270 | https://www.oed.com/?tl=true | READ | hard | Filter search results to show only entries for words first used from 1500 to 1600 and list the headwords of the first 10 results. Only use http://oed.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-256 | Failure | The AI agent was not able to filter the search on the website directly so it tried to search on google, which is not allowed as mentioned in the task | https://www.loom.com/share/e75a0e7faf524b3bab2d571b43e5606b |
| 205 | 1307 | https://www.oregon.gov/Pages/index.aspx | READ | easy | Identify the page for the Oregon agency responsible for environmental protection and list the contact details provided on that page. Only use http://oregon.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-262 | Failure | The AI agent failed to access the webpage | https://www.loom.com/share/32045d4ac55644c09fb936e68389913c |
| 206 | 1321 | https://www.pcgamer.com | DELETE | hard | Log in to your pcgamer.com account, post a comment on one of the front page articles, then locate your recently posted comment and delete it. Only use http://pcgamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-266 | Success | The AI agent successfully posted a comment on a recent article and deleted it later | https://www.loom.com/share/aef6ec1523d84452872c06714e18d658 |
| 207 | 1324 | https://www.pcgamer.com | READ | easy | Navigate to the "Reviews" section and list the titles of the three most recent PC hardware review articles. Only use http://pcgamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-267 | Failure | The AI agent failed to give 3 titles of the articles from the reviews section, it gave those from a different section | https://www.loom.com/share/95c46b58df1040f49d8389b46e1c3fb2 |
| 208 | 1356 | https://pixabay.com | READ | hard | Go to the "Videos" section and output the file sizes from the download section and resolutions for the top 3 trending videos. Only use http://pixabay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-269 | Failure | The AI agent failed to find information on two of the trending videos | https://www.loom.com/share/1da56d91fedb42aabd1602bcd1359f6b |
| 209 | 1359 | https://pixabay.com | READ | easy | Search for "vintage" imagery and list the publication dates (if provided) and contributor ids (if provided) for the top 5 results. Only use http://pixabay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-270 | Failure | The AI agent failed to get past the verification page | https://www.loom.com/share/c92957d48a23478ca0b671f90d59dbfc |
| 210 | 1360 | https://pixabay.com | READ | easy | Search for "abstract background" in the vectors/illustrations category and output the contributor usernames and at least three tags from each of the first 5 results. Only use http://pixabay.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-268 | Failure | The AI agent failed to get past the verification page | https://www.loom.com/share/551170a61ebd403598ccbdfa30fe5afd |
| 211 | 1364 | https://www.playstation.com/en-us | CREATE | easy | Subscribe to the newsletters by entering your email to receive monthly updates on PlayStation news, deals, and events. Only use http://playstation.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-271 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/649b1c33009f49a7b6f226384d4ffecf |
| 212 | 1365 | https://www.playstation.com/en-us | CREATE | easy | In the support section, submit a formal inquiry about issues with remote downloads, specifying your console model and error details. Only use http://playstation.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-272 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/b389bca7878e46839b46edcc39cc4f6f |
| 213 | 1369 | https://www.playstation.com/en-us | READ | easy | Browse the homepage and identify the main featured release; provide its headline and a brief summary. Only use http://playstation.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-273 | Failure | The AI agent failed to access the webpage | https://www.loom.com/share/357cf36f6ee44934af57be181ab77ff4 |
| 214 | 1377 | https://www.polygon.com | READ | easy | Find any top 19 list of recommended PC games and pull out the first three game titles from that list. Only use http://polygon.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-274 | Failure | The AI agent failed to access the top 3 game titles from the webpage so it tried to search on google | https://www.loom.com/share/6c1a724995fc4b35ac425187bfd693ba |
| 215 | 1392 | https://www.purdue.edu | READ | easy | Locate the section on study abroad programs and list two destination countries where Purdue students can study. Only use http://purdue.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-275 | Failure | The AI agent was only able to find one country where they send their students to study abroad | https://www.loom.com/share/16a79e92006c40e1aa55ad2b646317dc |
| 216 | 1395 | https://quizlet.com | READ | easy | Log in, visit the Quizlet Help Center, navigate to the FAQ section on creating your questions, and summarize the first three steps listed. Only use http://quizlet.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-276 | Failure | The AI agent was not able to get past the CAPTCHA page | https://www.loom.com/share/9be1a54aeb9b4623969473215032e4cd |
| 217 | 1400 | https://www.radiotimes.com | CREATE | easy | Use the social media sharing buttons on a Radiotimes article to generate a shareable link and create a custom message endorsing the article. Only use http://radiotimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-278 | Success | The AI agent successfully found from where can we share the aricle but it was getting blocked on twitter and it still created a endorsement message with the article and gave a line to share the article as well | https://www.loom.com/share/157a74148a424f5bb5e7cc284ef967d7 |
| 218 | 1407 | https://www.radiotimes.com | READ | hard | Locate tonight's featured TV schedule on Radiotimes, and list the titles of shows airing on both BBC and ITV. Only use http://radiotimes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-279 | Success | The AI agent successfully found the shows airing today on BBC and ITV | https://www.loom.com/share/3fdbc22178dc442dbc60bc536eae9a17 |
| 219 | 1413 | https://www.realsimple.com | READ | easy | Visit the Travel & Lifestyle section under Shopping and record the author’s name and publication date for the leading article. Only use http://realsimple.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-280 | Failure | The AI agent was able to find the Shopping section correctly but it never went to the Travel and Lifestyle reviews section to get the leading article instead it got the article from the shopping section itself. | https://www.loom.com/share/693a4838f4ed4639b31aad5251e5ea4a |
| 220 | 1416 | https://www.realtor.com | READ | easy | Filter apartments for rent in New York City with a monthly rent under $2,500, and list the first three listings’ addresses, rental prices, and highlighted amenities. Only use http://realtor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-281 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/12a4bb5408cf4a158c653ae0162f248c |
| 221 | 1417 | https://www.realtor.com | READ | easy | Go to the “Find a Realtor” section, enter criteria for a buyer in Phoenix, AZ, and list the names and contact details of the first five matching real estate professionals. Only use http://realtor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-282 | Failure | The AI agent failed to open the website directly so it tried to access the webpage through google which is not allowed as mentioned in the task itself | https://www.loom.com/share/d97aa7c83d9b4cd2b1355768757c4e2f |
| 222 | 1428 | https://www.redbubble.com | READ | easy | Look up “Floral Patterns” on Redbubble and list the product titles that appear in the search results. Only use http://redbubble.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-283 | Success | The AI agent successfully gave the list of the items displayed on the website | https://www.loom.com/share/073f5872916248728e22213f3d917a59 |
| 223 | 1438 | https://www.rei.com | READ | easy | Search for "hiking boots" on REI.com and list the names, prices, and ratings of the top three results. Only use http://rei.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-284 | Success | The AI agent successfully found the top 3 results on hiking shoes as requested | https://www.loom.com/share/1a1fb0f6ab7d4ed8b54cc47a2a7c07d1 |
| 224 | 1448 | https://restaurantguru.com | READ | easy | Search for Italian restaurants in New York City and list the top 3 restaurants by user rating. Only use http://restaurantguru.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-285 | Failure | The AI agent failed to find the top 3 restaurants from new york city as it did try to search for the location but never eneded up selcting "New York City" | https://www.loom.com/share/c36c6034d6d643e485fde6f167a92991 |
| 225 | 1456 | https://resy.com | CREATE | easy | Create a reservation request for a large group of eight at a restaurant in "NoMad Diner", including a note requesting a private dining area. Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-292 | Failure | The AI agent did try to book the table for 2 not for 8 as requested in the task | https://www.loom.com/share/5d87bb67bae94c0f86d7d13ddd923aa1 |
| 226 | 1464 | https://resy.com | CREATE | easy | Create a personalized list titled "Romantic Dinners" on Resy and add the restaurants "Le Bernardin," "Jeju Noodle Bar,," and "Eleven Madison Park" to it. Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-290 | Failure | The AI agent was not able to load the webpage let alone the login page | https://www.loom.com/share/ef054bbba4f944ac8b5a41609cda5a47 |
| 227 | 1466 | https://resy.com | DELETE | easy | Login, add "Passerine" as favoriated in your account. then access your favorites list on Resy and remove it from the list. Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-291 | Failure | The AI agent was not able to get past the login page | https://www.loom.com/share/43452e44c2a14b25be6d0b6fd11e2db4 |
| 228 | 1468 | https://resy.com | READ | easy | How many $'s does the Carbone in Greenwich Village, New York have? Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-293 | Failure | The AI agent failed to complete the task as it encountered difficulties accessing the search functionality | https://www.loom.com/share/0543750538a84c24b65009457c41b09e |
| 229 | 1469 | https://resy.com | READ | hard | Search for available reservation times for a French restaurant in New York City on an upcoming Friday evening. Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-286 | Failure | The AI agent was not able to get past the login page | https://www.loom.com/share/6dadd4969ea84198b9cff1c4d6cbeaa9 |
| 230 | 1470 | https://resy.com | READ | easy | Find Arden, in New York. List the available dining times for tonight. Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-294 | Failure | The AI agent failed to find the exact restaurant mentioned in the task resulting "Page Not Found" errors | https://www.loom.com/share/f84f6b8767e44c888b354cec23250606 |
| 231 | 1472 | https://resy.com | UPDATE | easy | Make a reservation at "Tacombi - Williamsburg" for 7:00 PM on a given night. Modify your "Tacombi - Williamsburg" reservation by changing the dinner time from 7:00 PM to 8:00 PM. Only use http://resy.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-288 | Failure | The AI agent failed to complete the task, however it was never visible what it was trying to do as the webpage was never fully loaded | https://www.loom.com/share/7d333e833d134e0487d3b4cc6c2c4fe8 |
| 232 | 1478 | https://www.reverso.net/text-translation | READ | easy | Use the collaborative dictionary to translate 'proud' from English to Arabic and return the result Only use http://reverso.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-295 | Failure | The AI agent was not able to get past the captcha page | https://www.loom.com/share/cd6d3617a26f461eb538ba9efafdbb7f |
| 233 | 1491 | https://www.samsung.com/us | READ | easy | Navigate to the Samsung Galaxy S23 product page and list its key technical specifications such as processor, battery capacity, and display resolution. Only use http://samsung.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-296 | Success | The AI agent successfully found the information on the model as requested | https://www.loom.com/share/e7e7f09c18f74b619fb7b092d0809cd4 |
| 234 | 1503 | https://screenrant.com | READ | easy | What is the description of the game 'Marvel Rivals'? Only use http://screenrant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-299 | Success | The AI agent successfully found the information on the game as requested from multiple articles | https://www.loom.com/share/9875b9a391af46908b61592ca55fd552 |
| 235 | 1504 | https://screenrant.com | READ | easy | Find the article '10 TV Shows That Went Downhill After A Major Cliffhanger' and summarize why 'The Flash' is included. Only use http://screenrant.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-298 | Success | The AI agent successfully found the info from the mentioned article after explicitly directing it towards search bar | https://www.loom.com/share/0ab661352015409492dd1e426e3cc968 |
| 236 | 1522 | https://us.shein.com/?ref=www&rep=dir&ret=us | UPDATE | easy | While logged in, add a "gold hoop earrings" product to your cart, update its quantity to 2 units, and verify that the total price is adjusted accordingly. Only use http://shein.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-301 | Failure | Agent was unable to login - login email field was populated with "user@example.com" so the agent thought this was human's email. | https://www.loom.com/share/0c2f3f08d8c24ecdb963f11c76e92584 |
| 237 | 1520 | https://us.shein.com/?ref=www&rep=dir&ret=us | READ | hard | Browse the "New In" section and list the product names, prices, and available colors of the top 5 most popular items. Only use http://shein.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-300 | Failure | The AI agent was not able to get past the CAPTCHA page | https://www.loom.com/share/f2c69a87ef4a4bbdaaa34e401929df26 |
| 238 | 1525 | https://www.si.com | READ | easy | Locate an SI.com feature on college sports and provide a brief summary outlining its key discussion points. Only use http://si.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-302 | Success | Agent was able to find a story about Travis Hunter and summarize key points. | https://www.loom.com/share/b99e979c45764a139bd2bb0d79bf5cbd |
| 239 | 1528 | https://www.si.edu | READ | easy | Navigate the homepage to locate the "Smithsonian Open Access" section and list the titles of the first five featured digital assets. Only use http://si.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-303 | Success | Agent was able to list five titles of digital assets on the open access page. | https://www.loom.com/share/d152de951021445b9bee02f56292911f |
| 240 | 1539 | https://www.simplyrecipes.com | UPDATE | hard | Login, post a comment praising a Lentil Soup recipe, then edit your previously submitted comment with additional recommendations for pairings with this dish. Only use http://simplyrecipes.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-304 | Failure | Agent could not find the comment section on the specific post, and did not login nor navigate to a different post to submit a comment. | https://www.loom.com/share/06c1bebcb3734fe7bab0bd952e1b66dc |
| 241 | 1545 | https://www.skysports.com | READ | easy | Identify and read the latest article under the rugby section covering match recaps, and list the names of the two teams that played. Only use http://skysports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-306 | Success | The agent named the two teams that played in the last recap article and linked the article as well. | https://www.loom.com/share/3de73cc979ad423d914c0762521c9f87 |
| 242 | 1546 | https://www.skysports.com | READ | easy | Browse the homepage for any breaking news related to the "Football" and provide a short text summary of the update. Only use http://skysports.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-305 | Failure | Agent collected many article headlines and summarized them, but as it was clear that it was successful in doing so but taking more time than was sufficient, human instructed the agent to return what it had already compiled. Agent ignored the human because it was the specific proxy agent designed to research articles, highlighting the downside of having multiple proxy agents rather than Operator's model of just one flow. Human also attempted to repeat the same process with the Orchestrator agent, but this didn't work either. | https://www.loom.com/share/d73bfccd043a458a8b53eefa6c8d42cd |
| 243 | 1550 | https://www.slideshare.net | READ | easy | Search for presentations on "digital marketing trends 2023" and list the titles and authors of the first five results. Only use http://slideshare.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-308 | Success | Agent was successful in returning 5 titles and authors for the desired search. | https://www.loom.com/share/9176671d932a41cb9a688b037d495485 |
| 244 | 1551 | https://www.slideshare.net | READ | easy | Browse the "Business" category and extract the title of the most-viewed presentation. Only use http://slideshare.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-307 | Success | Agent was able to find the most popular file and return a downloadable link to it. | https://www.loom.com/share/8193719bc7c342e9ab1c1ff504bbcc80 |
| 245 | 1553 | https://www.slideshare.net | READ | easy | Identify three presentations on "sustainable energy" sorted by popularity and list their titles. Only use http://slideshare.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-309 | Failure | Agent returned that it could not find any results for the desired search. | https://www.loom.com/share/98a77186d4584dbd90452fd3defedade |
| 246 | 1555 | https://www.smithsonianmag.com | READ | hard | Browse the History category and record the title and publication date of the most popular article (based on visible engagement or share counts). Only use http://smithsonianmag.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-310 | Success | Agent was able to return a table of the top articles with the engagement results, and had one with results, so this is technically a successful task. | https://www.loom.com/share/d92f95b7df9a4f35b22b6631838a6d3f |
| 247 | 1566 | https://www.sofascore.com | READ | hard | Explore the heatmap visualization on a live football game page and specify the zone where the highest concentration of shots occurred. Only use http://sofascore.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-312 | Failure | Initial link did not work for agent so it attempted to search Google, which is out of bounds. | https://www.loom.com/share/7bc2c78f88504cc788b652f944fce290 |
| 248 | 1573 | https://www.sportskeeda.com | CREATE | easy | Log in to your Sportskeeda account and post a comment on the latest "UEFA Champions League" article sharing your opinion on the match outcome. Only use http://sportskeeda.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-314 | Failure | Agent asked for CAPTCHA to be solved by human, but no CAPTCHA was present in browser. | https://www.loom.com/share/1ad69d3992fd4cda98df4476e247d9a8 |
| 249 | 1586 | https://www.sportskeeda.com | UPDATE | easy | Access your account settings on Sportskeeda and update your bio to better reflect your interests in both traditional sports and esports. Only use http://sportskeeda.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-313 | Success | Agent was able to update the bio, first inputting a generated bio and then updating it by tweaking the sentences. | https://www.loom.com/share/9733864c8b824457923f48d0b77381ca |
| 250 | 1591 | https://open.spotify.com | CREATE | easy | Create a podcast playlist by starting a new list and adding one episode from a podcast in the "Tech Trends" category. Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-315 | Failure | Agent ran into an issue accessing the website, and attempted to perform a Google search to locate the website which is out of bounds. | https://www.loom.com/share/155105b4a7d94727bbf54c60faa25966 |
| 251 | 1601 | https://open.spotify.com | DELETE | easy | Create a temporary playlist titled "Test Playlist," add two tracks, then delete the playlist and confirm its removal from your library. Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-317 | Failure | Agent ran into a connectivity issue and was unable to complete the task. | https://www.loom.com/share/2a2ce32836844e02bea117ee62c817be |
| 252 | 1605 | https://open.spotify.com | FILE_MANIPULATION | hard | Login and find out how to request extended streaming data, and follow the steps to request this data. Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-316 | Failure | Agent ran into a connectivity issue and tried to search for connectivity problems on DownDetector which is not the spotify website. | https://www.loom.com/share/313d6420a6ac4c198d4d5fcde6eee204 |
| 253 | 1609 | https://open.spotify.com | READ | easy | Browse the Spotify home page and list the names of the top 5 trending playlists currently featured. Only use http://spotify.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-318 | Failure | Agent ran into an issue accessing the website, and attempted to perform a Google search to locate the website which is out of bounds. | https://www.loom.com/share/ba7c9b7c799a4718b10facc9ebaa3c54 |
| 254 | 1625 | https://www.springer.com/us | READ | hard | Find the "Philosophy" articles page and summarize the abstract of the most recent article posted. Only use http://springer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-320 | Success | Agent was able to find the most recent article about philosophy and return a summary of the abstract. | https://www.loom.com/share/821f4fb715d940bbaa48e9ca44e38676 |
| 255 | 1635 | https://stackexchange.com | CREATE | easy | Log in, navigate to the Data Science community, and post an answer on a question about "feature engineering techniques" including at least two bullet points. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-325 | Failure | Agent stalled out, human stopped due to inconvenient length of time waiting for next step to load. | https://www.loom.com/share/31b6f91357934518b2c01b3d30b90124 |
| 256 | 1640 | https://stackexchange.com | CREATE | hard | Post an answer in the Super User community to a question regarding "SSD vs HDD performance" and include relevant benchmark numbers. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-331 | Failure | Agent got stuck in a loop searching for the super user page, and could nto locate it. Thus, this task is a fail. | https://www.loom.com/share/cf1051def9d94c899ca081422fad8969 |
| 257 | 1642 | https://stackexchange.com | CREATE | easy | Log in and post a new question in the Academia community titled "How to balance research and teaching responsibilities?" with contextual examples from your experience. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-328 | Failure | Agent was unable to identify that the website wasn't letting it post the question because there was an erroneous field which the agent had to delete. It couldn't recognize this and never was able to post, so this is a failure. | https://www.loom.com/share/ec22f669115f453e9759291e62fb04a9 |
| 258 | 1644 | https://stackexchange.com | CREATE | hard | Using the Markdown editor, log in and create a comprehensive guide titled "How to use StackExchange effectively" on the Meta site. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-326 | Failure | Agent was unable to post a comment, so this attempt was a fail. | https://www.loom.com/share/0695e3934ad7426cbab593928ae1ae8f |
| 259 | 1658 | https://stackexchange.com | DELETE | easy | In the Super User community, post a temporary answer labeled "Test answer for deletion" to an existing question and then delete it, ensuring it no longer appears in your contributions. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-322 | Failure | Agent received the following message: "It seems there was an issue submitting your question on Puzzling Stack Exchange. The error message received was: "Your question couldn't be submitted. Resolve 2 issues before posting." | https://www.loom.com/share/03c33415b20d44078866b76bd8756ff9 |
| 260 | 1661 | https://stackexchange.com | READ | easy | Visit the Meta site of StackExchange and list the titles of the 3 most recent community discussion topics. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-330 | Success | Agent was able to find and return the 3 most recent community discussions | https://www.loom.com/share/e2bca74b701a4b9dab9902d2fd57b16d |
| 261 | 1662 | https://stackexchange.com | READ | easy | Use the main search bar to find questions tagged with "python" in the Stack Overflow community and output the titles of the first 5 results. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-323 | Success | Agent was able to output the top 5 results and provided links as well. | https://www.loom.com/share/c2cdad6343c2419ba604adee37b95517 |
| 262 | 1669 | https://stackexchange.com | UPDATE | easy | Log in and post a question in the Cross Validated community titled "Statistical Significance in A/B Testing." Then update the title of this question to "Understanding Statistical Significance in A/B Testing." Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-329 | Failure | Agent couldn't locate the login page, so was not able to accomplish this task. | https://www.loom.com/share/421235d6981b42088268cea4981bec45 |
| 263 | 1671 | https://stackexchange.com | UPDATE | easy | Log in to the Web Applications community and answer a question "increasing website engagement." Then update your answer with additional examples and an emoji. Only use http://stackexchange.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-327 | Failure | Agent struggled mightily with the CAPTCHA, and even after human logged in and solved CAPTCHA, this did not register with the browser, and the agent kept on trying to solve puzzles. | https://www.loom.com/share/ad1388ffe5ff46e4b1de0b7c2f544992 |
| 264 | 1676 | https://stackoverflow.com/questions | CREATE | hard | Provide an answer to a question regarding best practices for REST API development in Ruby on Rails, including a sample code implementation. Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-335 | Failure | Agent struggled mightily with the CAPTCHA, and even after human logged in and solved CAPTCHA, this did not register with the browser, and the agent kept on trying to solve puzzles. | https://www.loom.com/share/ab68c0e52379426ebf32724cec2e5508 |
| 265 | 1690 | https://stackoverflow.com/questions | DELETE | easy | Log in, post a question about Node.js with the tag "node" and then go back to delete the tag from your posted question on Node.js, ensuring that the removal is confirmed. Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-332 | Failure | Human timed agent out - agent was being inefficient and was taking long to generate post fields. | https://www.loom.com/share/837399d52ae44b4e8fdbfce05aec4420 |
| 266 | 1692 | https://stackoverflow.com/questions | READ | easy | Browse through the "Help Center" to locate the page on editing posts and list the top three tips mentioned. Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-333 | Success | Agent was able to locate the help page and returned the top three tips. | https://www.loom.com/share/db89cda03de34f86bb19a59f294c36a6 |
| 267 | 1698 | https://stackoverflow.com/questions | UPDATE | hard | Log in to your account, and post a question regarding web security, then update the tags on your question about web security to also include "sql-injection" Only use http://stackoverflow.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-334 | Failure | Agent was unable to submit the question after trying multiple times. Proxy reached maximum amount of steps. | https://www.loom.com/share/c8ac299e02444b91a1ffd70c694d5b82 |
| 268 | 1704 | https://stardewvalleywiki.com/Stardew_Valley_Wiki | READ | easy | Analyze which villagers have birthdays in Summer using the calendar data Only use http://stardewvalleywiki.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-337 | Failure | Agent had collected the birthday seasons for each character, but then chose to do more analysis after already possessing the answer. Human tried to chat with the agent to get it to return everything it had collected so far, but the agent ignored human. | https://www.loom.com/share/7679e791165749588a449bdf8b48e3de |
| 269 | 1705 | https://stardewvalleywiki.com/Stardew_Valley_Wiki | READ | easy | Find and list all the gifts that Sebastian "loves" according to his gift preferences page. Only use http://stardewvalleywiki.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-336 | Success | Agent was able to locate Sebastian's profile and find the gifts that he "loves." | https://www.loom.com/share/7d7c59a1199d4d4a812885bd39c330cf |
| 270 | 1713 | https://www.statista.com | READ | easy | Filter the available reports by the sports industry and list the titles of the first three annual reports presented. Only use http://statista.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-338 | Failure | Agent was unable to isolate report names, so this task is a fail. | https://www.loom.com/share/45389b0dba5a4970835961b5b50fd79e |
| 271 | 1723 | https://store.steampowered.com | READ | hard | Go to the Top Sellers section and list all games currently discounted at more than 50%. Only use http://steampowered.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-340 | Success | Agent was able to find multiple games discounted and returned them. | https://www.loom.com/share/bb3d5e33f0df41adaf15f8fafbeb76ea |
| 272 | 1729 | https://www.studocu.com | CREATE | easy | Create a new account and then create a course name of "Introduction to Economics." Ensure that a confirmation message appears. Only use http://studocu.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-341 | Failure | Agent was unable to access the website on the browser. | https://www.loom.com/share/92454b2594a94cfc9b9c20b929e2c3d8 |
| 273 | 1732 | https://www.studocu.com | CREATE | hard | Assemble a new study plan by creating a "Revision Pack" StudyList that curates documents on key course topics and then use the site’s sharing feature to share it with a friend. Only use http://studocu.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-342 | Failure | Human timed agent out - agent was being inefficient and was taking long to generate post fields. | https://www.loom.com/share/6f913111aa084cbdb8f53a6f1c573d94 |
| 274 | 1745 | https://www.target.com | READ | easy | Check how long shipping takes for an ipad Air to be delivered to the New York City 10003 Zip Code Only use http://target.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-343 | Failure | Agent was unable to locate ipad Air, and got stuck in a loop of a non-loading screen trying to figure out what to do. | https://www.loom.com/share/867a1abc49ed44098fadfbe1ae619639 |
| 275 | 1750 | https://www.techradar.com | READ | easy | Search for articles related to “energy efficient computers” and extract a recommendation mentioned in one of the articles. Only use http://techradar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-344 | Failure | Agent stalled out, human stopped due to inconvenient length of time waiting for next step to load. | https://www.loom.com/share/19e2ccc82b904371afc9e23db05afb91 |
| 276 | 1761 | https://www.telegraph.co.uk/us | DELETE | easy | Log in to your Telegraph account, find an article on the front page and save it to your reading list, then access your reading list, delete the saved articles, and verify that it no longer appears. Only use http://telegraph.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-346 | Failure | Website access to Google was blocked, so agent couldn't access website. | https://www.loom.com/share/92f13cc7678b40aeb3f4c720170f3959 |
| 277 | 1765 | https://www.telegraph.co.uk/us | READ | easy | Use the search function to find an article about a “Royal Ceremony” and provide the headline and publication date of the first result. Only use http://telegraph.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-345 | Success | Agent was able to find an article about British monarchal proceedings in Australia. | https://www.loom.com/share/f7d6a09b181b447db657c55f78d11bc0 |
| 278 | 1768 | https://www.telegraph.co.uk/us | UPDATE | easy | Log in to your Telegraph account, navigate to your profile settings, and change your newsletter subscription preference from daily updates to weekly updates. Only use http://telegraph.co.uk to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-347 | Failure | Agent wasn't able to process log in, so human timed it out. Was inefficient with login process anyways, so wouldn't be suitable for consumer. | https://www.loom.com/share/4b36bc38a55f4799bf54c25bacfedec6 |
| 279 | 1769 | https://testbook.com | READ | easy | Search the articles section for "SSC exam pattern changes" and extract the key updates mentioned. Only use http://testbook.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-349 | Success | Agent reported out the details of the original test and the updates made since. | https://www.loom.com/share/a3dc43c56b16432aacd949252ac8584a |
| 280 | 1774 | https://www.texas.gov | READ | easy | In the "Find Services" section, locate the official guidelines on vehicle registration and extract the checklist of submission requirements shown. Only use http://texas.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-350 | Failure | Agent claimed to have completed the task, but returned nothing. | https://www.loom.com/share/7cb1f4cb59764d87844c380005e12fed |
| 281 | 1778 | https://www.theatlantic.com | READ | easy | Navigate to The Atlantic homepage and list the titles of the top three featured articles displayed. Only use http://theatlantic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-352 | Success | Agent was able to return and identify the top three featured articles displayed. | https://www.loom.com/share/da973d15ea7548d8b732bb5291f141c6 |
| 282 | 1780 | https://www.theatlantic.com | READ | easy | Visit the subscription page and identify the differences in benefits offered between the digital, digital and print subscriptions. Only use http://theatlantic.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-351 | Success | Agent was able to return and distinguish between the multiple subscription options. | https://www.loom.com/share/f88a613530df404ab7da68e96637682f |
| 283 | 1784 | https://www.thegamer.com | READ | easy | Search for reviews and list the headlines of the top five game review articles currently featured. Only use http://thegamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-353 | Failure | Agent only returned two articles because it said that only two were featured on the homepage. Agent could have found 3 more on different pages to retrieve the total 5. | https://www.loom.com/share/042e5a07100740fb9e927ebc50ba9a45 |
| 284 | 1785 | https://www.thegamer.com | READ | hard | Locate an Opinion or Cultural Commentary article discussing modern gaming culture and summarize its central argument in one or two sentences. Only use http://thegamer.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-354 | Success | Agent was able to find a relevant article and summarize its argument in three sentences. | https://www.loom.com/share/3cc3d83a97254bcab92c1e61dce08b05 |
| 285 | 1792 | https://www.thehindu.com | READ | easy | Search the Health section for COVID-19 related news, and output the title and publication time of the first article that appears. Only use http://thehindu.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-355 | Success | Agent was able to identify the most recent article posted and find its publication date and time. | https://www.loom.com/share/e3f5a9267d5747c091933b86cd44f563 |
| 286 | 1798 | https://www.themoviedb.org | READ | easy | Browse the "Trending" section on themoviedb.org and extract the titles of the top 5 trending movies at the moment. Only use http://themoviedb.org to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-356 | Success | Agent was able to identify and return the top 5 featured movies. | https://www.loom.com/share/5b2ce5c97da040ada3635117b45b03cd |
| 287 | 1801 | https://www.thepioneerwoman.com | CREATE | easy | Log in to the site and save your favorite recipe from the "Comfort Food" category to your saved recipes collection. Only use http://thepioneerwoman.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-358 | Success | Agent was able to log in and successfully save the recipe to its bookmark section. | https://www.loom.com/share/39dd5fe316ec476aa36b1a8f2b6c47be |
| 288 | 1813 | https://www.thepioneerwoman.com | READ | easy | Locate a tutorial featuring Ree Drummond’s signature dish. Only use http://thepioneerwoman.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-357 | Success | Agent returned the tutorial link, as well as a list of ingredients to make the recipe. | https://www.loom.com/share/6bad73e1420a45a0a7c48f12e48d2726 |
| 289 | 1816 | https://www.thesaurus.com | READ | easy | Look up the synonyms for "rapid" and list the first five synonyms that appear. Only use http://thesaurus.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-359 | Success | Agent was able to find and return the first five synonyms for "rapid." | https://www.loom.com/share/49ee794894204f7db2510f16dc8b5cb8 |
| 290 | 1819 | https://www.thespruce.com | READ | easy | Search for "DIY kitchen backsplash ideas" on TheSpruce and list the titles of the first five articles. Only use http://thespruce.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-360 | Success | Agent returned the first five article titles that it found, as asked. | https://www.loom.com/share/0f813823db7b45e3a8ad6785511ec7fa |
| 291 | 1828 | https://www.thoughtco.com | READ | easy | Browse the ThoughtCo homepage and list the titles of the three newest articles displayed on the landing page. Only use http://thoughtco.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-361 | Success | Agent was able to find and return the three featured articles on the homepage. | https://www.loom.com/share/9e7b98470e364c7bb27b8ed07b298a30 |
| 292 | 1829 | https://www.threads.net | READ | hard | Use the search function to find threads mentioning “sustainability” and summarize the main discussion points from the first three results. Only use http://threads.net to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-362 | Failure | Agent navigated to Instagram, the parent company behind Threads, but didn't have to log in to complete the task. | https://www.loom.com/share/0c8943cd2590468da7c0582fffa7e02f |
| 293 | 1832 | https://www.ticketmaster.com | READ | hard | Visit the New York Giants' page, and find an available game to find tickets for. Only use http://ticketmaster.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-364 | Failure | Agent couldn't access the site, so it reverted to searching via Google, which is off-limits. | https://www.loom.com/share/1a809b3b47524e2dbe81a959f383efc4 |
| 294 | 1836 | https://www.tiktok.com/explore | READ | easy | Browse the For You feed and list the usernames and view counts of the first 5 videos displayed. Only use http://tiktok.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-365 | Failure | Human tried to solve CAPTCHA manually, but was unable to interact with the browser. | https://www.loom.com/share/df5dc172b7584568a5fc1ada759c601c |
| 295 | 1843 | https://time.com | READ | easy | Navigate to the "Science" section and summarize the key findings of the leading article in 2–3 sentences. Only use http://time.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-366 | Success | Agent was able to locate an article and summarize it in 2-3 sentences. | https://www.loom.com/share/7f51fa7fb0a94262b9120c85e0d662e3 |
| 296 | 1848 | https://www.timeanddate.com | READ | easy | Browse the API section to retrieve the Date Calculator API Pricing details including the package prices and number of credits Only use http://timeanddate.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-367 | Success | Agent was able to retreive pricing details and returned them in a succinct table. | https://www.loom.com/share/4b8afe70334a4e6986efec092a971172 |
| 297 | 1853 | https://www.today.com | CREATE | easy | Save an article about summer recipes to your reading list and add a brief note explaining why you found it interesting. Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-371 | Failure | Agent was unable to save the article because "the save button would not respond." | https://www.loom.com/share/5be029176b524a3486a861f3058f091a |
| 298 | 1858 | https://www.today.com | READ | hard | Scroll through the infinite content feed in the News category and record how many new articles load after an extended scroll. Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-370 | Success | Agent was able to identify that if you scroll you are able to generate 10 more articles. | https://www.loom.com/share/142ff9500ad94659902516cd1db938e4 |
| 299 | 1860 | https://www.today.com | READ | easy | Search for articles on “healthy breakfast recipes” and display the title and summary of the top result. Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-369 | Failure | Agent was inefficient, clicking into the top result and then ignoring the fact that it had found the top result to go find another one. There was no need for a second search, so the agent failed this task due to lack of efficiency. | https://www.loom.com/share/840d9e9ce457486eb7d196810f558af6 |
| 300 | 1861 | https://www.today.com | READ | easy | Navigate to the homepage and locate the schedule for the live Today show broadcast, then record the start time of today’s episode. Only use http://today.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-372 | Failure | Agent tried the search bar instead of searching the homepage for options to see the tv show times. | https://www.loom.com/share/b1b4bb01045b4b5286cc34cc10c5a5fe |
| 301 | 1868 | https://www.tomsguide.com | FILE_MANIPULATION | easy | Locate the downloadable PDF version of the "How to Enable 2FA" guide on Tom's Guide, download it, and verify that the filename contains the text "2FA". Only use http://tomsguide.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-373 | Failure | Agent timed out - was looking for a lengthy period of time and clearly was not going to locate the file. It found multiple articles related to the topic, but couldn't find one with a downloadable guide. | https://www.loom.com/share/67b22dedb2544e14a35f348cac7bc388 |
| 302 | 1873 | https://www.tomsguide.com | READ | easy | In the Reviews section, filter for gaming review and note down the headline of the top-listed review article. Only use http://tomsguide.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-374 | Failure | Agent failed to locate the reviews page from the home page. | https://www.loom.com/share/bab1bd51f08941c9a0ddb0b8dfb9c5c0 |
| 303 | 1883 | https://www.travelweekly.com | READ | easy | Browse the homepage and list the top five featured travel industry news headlines. Only use http://travelweekly.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-375 | Success | Agent was able to list the top five featured travel industry news headlines. | https://www.loom.com/share/b7ab7aa4d1d746dcb1fb1f29bc4d2ddb |
| 304 | 1885 | https://www.travelweekly.com | READ | easy | Use the search tool to look for upcoming webinar events on travel technology and output the titles of the events. Only use http://travelweekly.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-376 | Success | Agent was able to find three relevant webinars and provided links to the events. | https://www.loom.com/share/af51258aa067414984f33936e98317ab |
| 305 | 1888 | https://us.trip.com/?locale=en-us | READ | easy | Browse the Trip homepage and list the top 5 trending featured properties Only use http://trip.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-377 | Success | Agent provided a table to 5 trending properties as well as price, rating and reviews. | https://www.loom.com/share/5bab6356e476484fa2802ee3244338eb |
| 306 | 1890 | https://us.trip.com/?locale=en-us | READ | easy | Search for tours available in Rome tomorrow and list the top three tours sorted by "Top Rated". Only use http://trip.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-378 | Success | Agent was able to return three top rated tours in Rome available for tomorrow, along with price. | https://www.loom.com/share/727183d5ad12495b8355b91f04f32f02 |
| 307 | 1892 | https://www.tripadvisor.com | READ | hard | Identify the top-rated hotel in Paris, verify if it offers free cancellation, and analyze at least three recent guest reviews to see if they mention staff helpfulness. Only use http://tripadvisor.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-379 | Failure | Human timed the agent out due to inefficiency - agent was taking a long time to make progress. | https://www.loom.com/share/5d95c92fd2bc4d8c9a36c630f7b69d17 |
| 308 | 1897 | https://www.trustpilot.com | READ | hard | Use Trustpilot’s search function to filter HR & Recruiting located in "London", then list the review summaries for the top three highest‑rated companies. Only use http://trustpilot.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-380 | Success | Agent was able to come up with a list of three companies succinctly ranked by rating. | https://www.loom.com/share/b2a171d3ba7b4d968ae70b7898f46abf |
| 309 | 1911 | https://www.twitch.tv | UPDATE | easy | Modify your alert settings by updating the text that appears when someone donates Bits, ensuring your channel name is featured prominently in the alert. Only use http://twitch.tv to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-381 | Failure | Agent ran into a technical issue, and was unable to complete the task. | https://www.loom.com/share/b68f3dc3a05d4e1cace61aeb2488b8a4 |
| 310 | 1914 | https://www.ucdavis.edu | READ | easy | Navigate to the Admissions section and extract the key deadline dates for Fall 2025 undergraduate applications. Only use http://ucdavis.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-382 | Success | Agent was able to identify the total schedule including key dates for fall applicants. | https://www.loom.com/share/b9f225c132fb4e7db5a24c7b5d41910e |
| 311 | 1915 | https://www.ucdavis.edu | READ | easy | Search for the UC Davis library page and retrieve the opening hours and contact information for the main library. Only use http://ucdavis.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-383 | Success | Agent was able to find and return an abundance of library hours and contact information. | https://www.loom.com/share/69b9ba1abd834690bee8bc431413fb7a |
| 312 | 1917 | https://www.ucdavis.edu | READ | easy | Go to the Financial Aid and Scholarships page and extract the main eligibility criteria and deadlines for applying for aid. Only use http://ucdavis.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-384 | Success | Agent was able to find the guidelines and key dates for financial aid applications. | https://www.loom.com/share/7ea0fc5ca0b14fb898b866564b49689f |
| 313 | 1924 | https://www.ucla.edu | READ | easy | Search for information on UCLA’s athletic programs or sports teams and list the sports teams mentioned on the page. Only use http://ucla.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-385 | Success | Agent listed all the different programs and teams that were listed on the page. | https://www.loom.com/share/c13bd73c371d443c902ab44b55b4fb2c |
| 314 | 1930 | https://www.ufl.edu | READ | easy | Use the site’s search bar to find information on UF research computing facilities and note two primary services offered. Only use http://ufl.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-386 | Success | Agent was able to scan the website and list two primary services offered. | https://www.loom.com/share/7faf425c890640cd866b129b27670e74 |
| 315 | 1931 | https://www.ufl.edu | READ | easy | Browse the Academic Departments listing and extract key details on the Computer & Information Science & Engineering department’s research focus areas. Only use http://ufl.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-387 | Failure | Human timed agent out - agent was being inefficient and was taking long to locate data. | https://www.loom.com/share/ed0b3b1070d54df1a42a27f0d3da8164 |
| 316 | 1938 | https://www.ultimate-guitar.com | READ | easy | Find the guitar tabs for "Hotel California" by the Eagles. Only use http://ultimate-guitar.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-388 | Success | Agent was able to find multiple tabs on guitar chords for "Hotel California." | https://www.loom.com/share/a4af29e21e464fe99c53b17275585cb6 |
| 317 | 1939 | https://umich.edu | READ | easy | Browse the University of Michigan homepage and list the top three headlines featured in the News section. Only use http://umich.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-389 | Success | Agent was able to find the three top headlines from the UM website. | https://www.loom.com/share/3cd00614e78742b3b19c20f14e3ad2a2 |
| 318 | 1942 | https://umich.edu | READ | easy | Explore the “Michigan Online” section and list two online courses or learning platforms mentioned on that page. Only use http://umich.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-390 | Success | Agent was able to list two online courses from the "Michigan online" page. | https://www.loom.com/share/8faecb36e9cd4488b965e9e5a1536eb3 |
| 319 | 1951 | https://www.uniqlo.com/us/en | DELETE | hard | Log in to your Uniqlo account, add the BlockTech Shirt to a wishlist, then remove it from the wishlist, confirming that the product no longer appears on the list. Only use http://uniqlo.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-391 | Failure | Human timed the agent out due to inefficiency - agent was taking a long time to make progress. | https://www.loom.com/share/7c7e03dabd2b41e6be914e39536f1059 |
| 320 | 1966 | https://www.upenn.edu | READ | easy | Search the website for information on interdisciplinary research centers and list two names or brief descriptions provided on the page. Only use http://upenn.edu to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-392 | Success | Agent was able to list two names of interdisciplinary research centers. | https://www.loom.com/share/99d74cdf7e5d46b48326e0e49ecd5a8f |
| 321 | 1968 | https://www.urbandictionary.com | READ | easy | Find the definition for “woke” and list any descriptive tags or keywords that appear alongside the definition. Only use http://urbandictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-393 | Failure | Human timed the agent out due to inefficiency - agent was taking a long time to make progress. | https://www.loom.com/share/05cdf32d4c594733b97d74b14f0ae034 |
| 322 | 1969 | https://www.urbandictionary.com | READ | hard | Use the search bar to look up “stan” and compare the top two definitions by summarizing the contrasting perspectives or nuances in their descriptions. Only use http://urbandictionary.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-394 | Success | Agent was able to find two definitions for "stan," and compare and contrast them. | https://www.loom.com/share/a9010c37a7fe44a49efd2fffd8277782 |
| 323 | 1974 | https://www.usa.gov | FILE_MANIPULATION | easy | Find and download the most recent government guide on disaster preparedness from USA.gov, then verify that the file is in PDF format. Only use http://usa.gov to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-395 | Failure | Agent experienced connectivity issues, and could not manage to connect to the website via the browser. | https://www.loom.com/share/cf32215825c84c0191ffea6990fabcb9 |
| 324 | 2021 | https://www.usnews.com | READ | easy | Search the site for "economic trends 2023" and list the titles and publication dates of the first five matching articles. Only use http://usnews.com to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site. | HITL-Convergence-400 | Success | Agent was able to search for "economic trends 2023" and list the titles and publication dates of the first five matching articles. | https://www.loom.com/share/13102495dd314e50a84d4ce28da37c01 |