Jev maps a spoken go back to a click in about 300ms
Browser agents- Cost
- $0.0002 per decision
- Latency
- ~300ms
- Result
- A spoken go back can finish before the sentence ends
Ask Jev which browser intent the live voice transcript already commits to, then which on-screen target to use.
Questions
Primitive
Choice
Ask a multiple choice question
- Name
- intent
- Instructions
- Which browser action does the user ask for in `transcript`? Judge the words said so far. If the sentence is unfinished, pick the action the words already commit to; if no action is recognizable pick none. `page` and `elements` describe what is currently on screen.
- Criteria
- navigate_url
Open a specific website or URL by name (go to / open / visit / take me to <site>) Not for: Searching for a topic; clicking something already on the page. Examples: go to wikipedia, open youtube, take me to github.com, visit example dot com.
- search_web
Search for a topic or phrase (search for / look up / google / find <query>), on the web or on a named site Not for: Typing into a specific named field without searching; opening a site's homepage. Examples: search for alan turing, look up typesafe jev, google cheap flights, search wikipedia for cats.
- click_element
Click / press / open / select / choose a link, button, tab, result or item that is on the current page Not for: Opening a website by name; typing text. Examples: click the first result, click sign in, open the second link, press the more information link.
- type_into_field
Type or enter specific text into an input box, search box or text field on the page Not for: Running a search on a search engine (that is search_web); pressing enter alone. Examples: type hello world into the search box, enter my email, write good morning in the comment box.
- select_option
Choose an option from a dropdown / select menu Not for: Clicking a link or button. Examples: select english from the language dropdown, choose the large size.
- press_enter
Press the Enter / Return key, or submit what was typed Not for: Typing text; clicking a named button. Examples: press enter, hit enter, submit.
- scroll_down
Scroll / move down the page Not for: Scrolling up; navigating. Examples: scroll down, scroll down a bit, go to the bottom, page down.
- scroll_up
Scroll / move up the page Not for: Scrolling down. Examples: scroll up, back to the top, page up.
- go_back
Go back to the previous page in history (back / go back / undo that / previous page) Not for: Scrolling up; closing a tab. Examples: go back, undo, back, previous page.
- go_forward
Go forward in history Not for: Scrolling down. Examples: go forward, forward.
- reload
Reload / refresh the current page Not for: Navigating elsewhere. Examples: reload, refresh the page.
- open_new_tab
Open a new empty tab Not for: Opening a website by name in the current tab. Examples: open a new tab, new tab.
- close_tab
Close the current tab Not for: Going back. Examples: close this tab, close tab.
- switch_tab
Switch to another / the next / the previous tab Not for: Opening or closing tabs. Examples: next tab, switch tab, go to the other tab.
- confirm
Approve a pending action the browser asked to confirm (yes / confirm / do it / go ahead) Not for: New commands. Examples: confirm, yes do it, go ahead.
- cancel
Cancel / never mind / stop the pending action Not for: Going back in history. Examples: cancel, never mind, stop.
- none
Not a browser command, or nothing recognizable yet (fragment, chit-chat, silence, filler) Not for: Anything that clearly matches another option. Examples: um, okay so, what do you think, the weather is nice.
- navigate_url