Commit Graph

302 Commits

Author SHA1 Message Date
Omkar Bansod
e0db74be2a Improve Chat Mode UX with better action guidance and fixed examples (#95)
* update chatmode to say use Agent mode for perform actions

* fixed default examples

* clean up
2025-09-16 08:34:26 -07:00
Omkar Bansod
18ada46905 feat: Add auto-submit functionality for chat providers (#94)
- Extend ProviderSchema with autoSubmit, openIn, submitKey, focusBeforeSubmit flags
- Update ChatGPT and Claude providers with auto-submit configuration
- Centralize auto-submit logic in executeProviderAction using chrome.browserOS.sendKeys
- Simplify CommandInput by removing duplicate provider handling
- Enables easy scaling for future providers (Perplexity, Grok, etc.)

Fixes: ChatGPT and Claude now automatically send Enter after query is pasted
Follows CEO specification: get tab ID + send Enter via chrome.browserOS API
2025-09-15 10:10:25 -07:00
Omkar Bansod
96c7eee994 fix: resolve scroll misalignment on side panel home page (#93) 2025-09-15 09:44:20 -07:00
Omkar Bansod
ff7123c130 feat(Major): Sidepanel UI Redesign with Grouped Agent Activity (#90)
* feat: implement grouped message sections for thinking, planning, and execution

- Add GroupedThinkingSection with collapsible thinking messages
- Add GroupedPlanningSection for plan-related content
- Add GroupedExecutionSection with step execution tracking
- Update MessageList to use new grouped message rendering
- Improve message organization and visual hierarchy

* feat: add supporting components for message grouping

- Add ExecutionSection for step-based execution display
- Add ThinkingSection for thinking message rendering
- Add animations library for smooth UI transitions
- Add utility functions for message processing

* feat: integrate grouped message rendering in core components

- Update MessageList to use new grouped sections
- Refactor MessageItem to remove TaskSteps dependencies
- Implement message grouping logic for thinking/planning/execution
- Optimize message rendering timing and reduce delays

* feat: simplify typing indicator to minimalistic thinking animation

* feat: update UI components and styling for redesigned sidepanel

- Update Chat component for new message grouping
- Enhance ChatInput with improved styling
- Refactor CollapsibleThoughts for better integration
- Update Header component styling
- Add new CSS classes and Tailwind config for gradients

* update loding animation

* feat/ Parent Collapsible wrapper for agent activity

* feat/skeleton animation for laoding

* fix(chore):coloring of lines

* update chatpannel

* skeleton update

* fix errros rendering issue

* centralise skeleton calling

* code cleanup

* single thinking component

* auto colsapse after results appears

* fix/scroll position issue
2025-09-15 07:47:02 -07:00
Nikhil Sonti
6fcb3e4ad2 add tool usage frequency metrics 2025-09-15 07:43:40 -07:00
Nikhil Sonti
8606191290 update feedback service to use /api/agent-feedback API 2025-09-13 13:46:14 -07:00
Nikhil Sonti
a9e271d98b enable feature flag for new-agent with new APIs 2025-09-13 12:59:55 -07:00
Nikhil Sonti
82d0bd2369 clean-up: remove observe tool 2025-09-13 12:33:36 -07:00
Nikhil Sonti
2a7a91115d clean-up: remove visual observe 2025-09-13 12:22:55 -07:00
Nikhil Sonti
325053b231 minor: langchain variable renames 2025-09-13 12:18:33 -07:00
Nikhil Sonti
e5c73be44b getState has image size, executor takes medium image size 2025-09-13 12:17:02 -07:00
Nikhil
0c8a81021a NewAgent updates (#92)
* Newagent: Add missing tools - MCP, group tabs, get seelected tabs

* deleting: PoCAgent and ReactLoopImpl

* Newagent; Adding MCP prompt

* NewAgent: glow animation added

* newagent: show pointer anitmation for element click, type and clear too

* pointer timeout 3s

* wip: new predefined mode

* CLEAN-UP: moved back to singleton architecture

* fix: prevent sending PROHIBITED tags

* NewAgent: predefined planner implemented

* Minor: renaming for better readability

* Fix: Newtab pre-defined plan execution

* predefined planner, added MCP to prompt

* Fix manifest.json version

* predefined planner also gets reasoning history; commented TODO for executor

* minor: rename

* Browsercontext: enable snapshot cache again with 1s TTL

* Langchain remove: llm cache

* langchain provider: properly read vision

* supportsVision moved to executionContext and using that in new agent
2025-09-13 11:18:17 -07:00
Nikhil
e25d463d95 NewAgent (#91)
* PocAgent + refactor (#77)

* Screenshot tool fixed

* ReactAgent loop

ReactAgent loop

v0.2

* Trim to max tokens implemented correctly

trim max tokens

* JSON Parse fix

Fixed json.parse

* Minor fix -- add system message always at position 0

* minor fix

* Added support for passing screenshot size to captureScreenshot

backup

* Make react agent use screenshot tool

* Refactor backend and execution (#75)

* wip: new exection class and manager

* wip: new pubsub channels

* wip: new background handlers

* new execution logic

* removed execution status

* handle workflow status for processing in sidepanel

* mcp server fix

* sending pause message

* better portName parsing, sidepanel sends tabId, storing tabId too

* 49.0.0.26 release

* docs: OmkarBansod02 signed the CLA in browseros-ai/BrowserOS-agent#$pullRequestNo

* Refactor backend and execution (#75)

* wip: new exection class and manager

* wip: new pubsub channels

* wip: new background handlers

* new execution logic

* removed execution status

* handle workflow status for processing in sidepanel

* mcp server fix

* sending pause message

* better portName parsing, sidepanel sends tabId, storing tabId too

* Moved react agent into POCAgent

* Revert changes of ReacStrategy from BrowserAgent

* Minor fix

* fix: execution class abort issue

* clean-up: removed un-used MessageTypes

* clean-up: execution-manager simplified

* better abort handling

---------

Co-authored-by: Felarof <nithin.sonti@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Screenshot failure handling

* fixes to custom agent -> sidepanel

* fix: portName stable

* fix: pause/reset + newtab connector

* New agent: wip-1

* fix: JSONify message content in MessageManagerReadyOnly

* disable cacheLLM and fixing image type detection

* new agent: wip-2

* rename: <BrowserState> as <browser-state> and <SystemReminder> as <system-reminder>

* new-agent: wip-3

---------

Co-authored-by: Felarof <nithin.sonti@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-09-12 11:12:45 -07:00
Omkar Bansod
400d20fdf8 Feature/Feedback-System (#85)
* feat(feedback):Introduce a feedback feature with firebase service

* fix firebasconfig issue to test the UI

* fix response button alignment

* feat/add firebase config

* update firebase output

* fix(chore)

* add userQuery

* add Enter button to submit feedback

---------

Co-authored-by: Felarof <nithin.sonti@gmail.com>
2025-09-11 13:33:56 -07:00
gbsierra
1bbf89dbd5 Minor change (#88) 2025-09-11 10:44:58 -07:00
Anuj Choudhary
2b85f0fb67 Fix the location of the CONTRIBTUIND.md (#89) 2025-09-11 10:44:31 -07:00
Felarof
4ce3f48f83 Add chat agent back to Nxtscape.ts and fix the error 2025-09-10 14:13:19 -07:00
github-actions[bot]
3a0a79d824 docs: gbsierra signed the CLA in browseros-ai/BrowserOS-agent#$pullRequestNo 2025-09-09 01:55:45 +00:00
Felarof
1abbee638a Braintrust basic evals (#87)
* implement validator eval

* implement online eval foundation

* further implementing online evals

* enhance evaluation data logging

* implement LLM scoring, remove redundant EventEnricher

* cleanup

* fix build errs from merging, extend LLM scorer context

* settled evaluation framework

* update evals documentation

* fix evals screenshots

* fix typos

* Evals config moved to env variables and tested

* test

* Update manifest to 49.1

* Removed duplciate + button

* Just use previous way of registering tools as that is not required for evals

* Add claude commands for research, plan and implement

* evals2 research and plan

implementation plan

new implementation plan

* Evals2 implementation

test test

* Removed old eval hooks

Remove old evals hooks

* evals 2 added to env

* Eval2 enhancement plan

backup

* Make Braintrust project configurable

Make Braintrust project configurable

* Enhanced scorer -- using Gemini 2.5 pro for evaluation

backup v0.1

enhancement v0.2

v0.2

backup v0.3

backup v0.4

* Deleted old evals directory

* Clean up old evals code

* Bunch of fixes and improvements

backup

fixes 0.1

more fixes

fixes

more elaborate prompts

braintrust logger fix

* Renamed files

backup
2025-09-05 18:04:07 -07:00
Felarof
5def144110 Update README.md 2025-09-05 11:45:49 -07:00
Felarof
ffbd9852c6 Add claude commands for research, plan and implement 2025-09-04 14:22:08 -07:00
Felarof
003d37d25c CLA fix (temporarily storing CLA in browserOS-agent until we create it as a submodule) 2025-09-03 12:57:22 -07:00
Felarof
fef9db598f Merge pull request #73 from OmkarBansod02/feature/interactive-plan-editor
Feature/Interactive Task Plan Editor
2025-09-01 18:17:40 -07:00
Felarof
3efa238765 Merge branch 'main' into pr/73 2025-09-01 18:16:59 -07:00
Omkar Bansod
0eb4df7369 Delete task if content is empty 2025-09-02 00:04:21 +05:30
Felarof
83ebce05c8 Merge pull request #81 from felarof99/alpha -- Adding all minor fixes and PoC agent -- except execution context
Adding all minor fixes and PoC agent -- except execution context
2025-09-01 08:17:00 -07:00
Omkar Bansod
24d44d618d fix:editable steps after task execution issue 2025-08-30 23:33:04 +05:30
Felarof
2c38eb40ab Revert "Refactor backend and execution (#75)"
This reverts commit a8432d1da2.
2025-08-29 18:19:04 -07:00
Felarof
2884b49d4d Revert "fix: ExecutionContext fixes"
This reverts commit f1c408978c.
2025-08-29 18:18:59 -07:00
Nikhil Sonti
11950de442 fix: ExecutionContext fixes
better abort handling

clean-up: execution-manager simplified

fix: execution class abort issue
2025-08-29 18:18:55 -07:00
Felarof
a5874f22e7 Merge pull request #76 -- separated react agent loop into POCAgent and reverted react loop changes in BrowserAgent
Separated react agent loop into POCAgent
2025-08-29 08:32:23 -07:00
Felarof
21a22df227 Minor fix 2025-08-29 08:28:10 -07:00
Felarof
b613776d4d Revert changes of ReacStrategy from BrowserAgent 2025-08-29 08:25:10 -07:00
Felarof
423b2e4699 Moved react agent into POCAgent 2025-08-29 08:23:23 -07:00
Nikhil
9aab165c99 Refactor backend and execution (#75)
* wip: new exection class and manager

* wip: new pubsub channels

* wip: new background handlers

* new execution logic

* removed execution status

* handle workflow status for processing in sidepanel

* mcp server fix

* sending pause message

* better portName parsing, sidepanel sends tabId, storing tabId too
2025-08-29 08:23:23 -07:00
github-actions[bot]
3055c35483 docs: OmkarBansod02 signed the CLA in browseros-ai/BrowserOS-agent#$pullRequestNo 2025-08-29 08:23:23 -07:00
Nikhil Sonti
f5b5f7883b 49.0.0.26 release 2025-08-29 08:23:23 -07:00
Nikhil
dba4956c30 Refactor backend and execution (#75)
* wip: new exection class and manager

* wip: new pubsub channels

* wip: new background handlers

* new execution logic

* removed execution status

* handle workflow status for processing in sidepanel

* mcp server fix

* sending pause message

* better portName parsing, sidepanel sends tabId, storing tabId too
2025-08-29 07:54:01 -07:00
Felarof
d3280ed7e4 Make react agent use screenshot tool 2025-08-28 17:17:26 -07:00
Felarof
72a0087920 Added support for passing screenshot size to captureScreenshot
backup
2025-08-28 17:17:26 -07:00
Felarof
02b77f85d1 minor fix 2025-08-28 16:34:42 -07:00
Felarof
361e224216 Merge pull request #74 from felarof99/aug27-react-agent-loop
Aug27 react agent loop
2025-08-28 13:47:43 -07:00
github-actions[bot]
f60c6aa062 docs: OmkarBansod02 signed the CLA in browseros-ai/BrowserOS-agent#$pullRequestNo 2025-08-28 20:24:05 +00:00
Nikhil Sonti
9b0e33c033 49.0.0.26 release 2025-08-28 12:58:54 -07:00
Felarof
4c793bef23 Minor fix -- add system message always at position 0 2025-08-28 11:54:44 -07:00
Felarof
0463262058 JSON Parse fix
Fixed json.parse
2025-08-28 11:51:59 -07:00
Felarof
74c0bdfc4f Trim to max tokens implemented correctly
trim max tokens
2025-08-28 11:33:11 -07:00
Felarof
253f5e446c ReactAgent loop
ReactAgent loop

v0.2
2025-08-28 10:53:01 -07:00
Felarof
e3a19d9a3c Screenshot tool fixed 2025-08-28 10:53:01 -07:00
Omkar Bansod
e28a0789de code cleanup and add comments 2025-08-28 18:09:22 +05:30