Search Results for author:
Found 9 papers, 4 papers with code
LITE: Intent-based Task Representation Learning Using Weak Supervision
Users write to-dos as personal notes to themselves, about things they need to complete, remember or organize.
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
Finally, we conduct a robustness analysis by testing M3A against a range of task variations on a representative subset of tasks, demonstrating that variations in task parameters can significantly alter the complexity of a task and therefore an agent's performance, highlighting the importance of testing agents under diverse conditions.
Latent State Estimation Helps UI Agents to Reason
In the context of autonomous UI agents we then show that LLMs used in this manner are more than $76\%$ accurate at inferring various aspects of latent state, such as performed (vs. commanded) actions and task progression.
UINav: A Practical Approach to Train On-Device Automation Agents
Automation systems that can autonomously drive application user interfaces to complete user tasks are of great benefit, especially when users are situationally or permanently impaired.
Android in the Wild: A Large-Scale Dataset for Android Device Control
The dataset contains human demonstrations of device interactions, including the screens and actions, and corresponding natural language instructions.
Lexi: Self-Supervised Learning of the UI Language
Along with text, these resources include visual content such as UI screenshots and images of application icons referenced in the text.
Inducing a hierarchy for multi-class classification problems
In applications where categorical labels follow a natural hierarchy, classification methods that exploit the label structure often outperform those that do not.
Bew: Towards Answering Business-Entity-Related Web Questions
We present a practical approach, called BewQA, that can answer Bew queries by mining a template of the business-related webpages and using the template to guide the search.
FLIN: A Flexible Natural Language Interface for Web Navigation
AI assistants can now carry out tasks for users by directly interacting with website UIs.
