This page documents the interactive element detection system within the DOM Processing Engine. It explains the logic within ClickableElementDetector that identifies elements users can interact with, how these elements are catalogued in the selector_map, and how this information flows through the serialization pipeline to enable agent actions.
For information about how the DOM tree is initially constructed, see 5.1 DOM Tree Construction For details on the complete serialization pipeline that uses these interactive elements, see 5.2 DOM Serialization Pipeline
Sources: browser_use/dom/serializer/serializer.py1-40 browser_use/dom/views.py17-82
Interactive element detection is the process of identifying which DOM elements are meaningful for agent interaction. The detection logic filters the raw DOM tree into a prioritized list of actionable nodes. This includes:
<button>, <a>, <input>, <select>, <textarea>.<iframe> elements or div containers with overflow content.The detection produces a selector map (DOMSelectorMap) that maps stable IDs to EnhancedDOMTreeNode objects, enabling the Agent to reference and interact with specific elements via an index.
Sources: browser_use/dom/serializer/clickable_elements.py4-7 browser_use/dom/views.py218-232 browser_use/dom/serializer/serializer.py69-71 browser_use/dom/views.py270-275
The following diagram illustrates the flow from a raw EnhancedDOMTreeNode to its inclusion in the selector_map based on interactivity heuristics.
Sources: browser_use/dom/serializer/clickable_elements.py4-203 browser_use/dom/serializer/serializer.py100-148 browser_use/dom/views.py218-224
The ClickableElementDetector class in browser_use/dom/serializer/clickable_elements.py provides the static method is_interactive() which evaluates nodes against several tiers of heuristics.
The detector maintains a set of interactive_tags that are inherently clickable:
button, input, select, textarea, a, details, summary, option, optgroup.label is excluded from the direct tag check to prevent double-activation of the associated input via the for attribute browser_use/dom/serializer/clickable_elements.py59-62 Labels are instead handled by checking for nested form controls.Sources: browser_use/dom/serializer/clickable_elements.py139-152 browser_use/dom/serializer/clickable_elements.py59-62
The system handles modern frameworks (React, Vue, Angular) by checking the has_js_click_listener property browser_use/dom/serializer/clickable_elements.py39-42 This property is populated during DomService processing using CDP's getEventListeners to detect elements with click, mousedown, or mouseup handlers without needing to mutate the DOM.
Sources: browser_use/dom/serializer/clickable_elements.py39-42 browser_use/dom/service.py431-450
Elements are considered interactive if they possess specific ARIA roles or state properties:
button, link, menuitem, option, radio, checkbox, tab, textbox, combobox, slider, spinbutton, search, searchbox, row, cell, gridcell browser_use/dom/serializer/clickable_elements.py182-202checked, expanded, pressed, selected, focusable, editable, settable browser_use/dom/serializer/clickable_elements.py118-124aria-disabled="true" or aria-hidden="true" are explicitly excluded browser_use/dom/serializer/clickable_elements.py105-115Sources: browser_use/dom/serializer/clickable_elements.py105-135 browser_use/dom/serializer/clickable_elements.py182-202
A specialized helper has_form_control_descendant detects nested inputs within wrappers (like label or span). It searches up to a depth of 2 levels to identify patterns like label > span > input browser_use/dom/serializer/clickable_elements.py9-25
Sources: browser_use/dom/serializer/clickable_elements.py9-25 browser_use/dom/serializer/clickable_elements.py58-72
The DOMTreeSerializer maintains a _clickable_cache to avoid redundant calls to the detector during its multi-pass serialization process (simplification, optimization, and indexing).
Sources: browser_use/dom/serializer/serializer.py74-75 browser_use/dom/serializer/serializer.py417-433
The following diagram maps the code entities involved in converting a detected interactive element into an LLM-addressable index and subsequently back into a browser action.
Sources: browser_use/dom/service.py35-62 browser_use/dom/serializer/serializer.py41-82 browser_use/dom/views.py913-914 browser_use/agent/service.py17-20
Iframes are marked as interactive if they are larger than 100x100 pixels browser_use/dom/serializer/clickable_elements.py50-52 This allows the Agent to target the iframe itself for scrolling or focus management. Small iframes are treated as decorative. The DomService also collects hidden interactive elements in iframes to provide scroll hints to the LLM browser_use/dom/service.py70-126
Sources: browser_use/dom/serializer/clickable_elements.py44-52 browser_use/dom/service.py70-126
The detector performs a keyword search on attributes (class, id, data-*) for indicators like search, magnify, glass, query, or lookup browser_use/dom/serializer/clickable_elements.py75-103 This ensures search icons that are technically just div or span elements are captured.
Sources: browser_use/dom/serializer/clickable_elements.py75-103
For complex controls like <input type="range">, the serializer can add virtual "compound components" to provide more granular information to the LLM, such as valuemin and valuemax from the AX tree browser_use/dom/serializer/serializer.py187-198 Note that date/time inputs are excluded from compound components to avoid confusing the LLM with sub-fields, as they require ISO format values browser_use/dom/serializer/serializer.py179-186
Sources: browser_use/dom/serializer/serializer.py150-198
During the final stage of serialization, the _assign_interactive_indices_and_mark_new_nodes function iterates through the filtered tree. If a node is interactive and meets visibility requirements, it is assigned a sequential integer and added to the selector_map.
This map is then used by the Tools service to resolve an integer index back to a target_id and backend_node_id for CDP execution.
Sources: browser_use/dom/serializer/serializer.py617-727 browser_use/tools/service.py91-125
Refresh this wiki