Observe

observe

Capture a screenshot of the current simulator screen with set-of-marks annotations, plus a structured element list.

observe(
  annotate=True,          # draw numbered marks on screenshot (default true)
  include_offscreen=False # include partially-offscreen elements
)

Returns:

{
  "screenshot": "<base64 PNG>",
  "marks_count": 12,
  "elements": [
    {
      "mark_id": 1,
      "type": "Button",
      "text": "Sign In",
      "identifier": "signInButton",
      "frame": { "x": 120, "y": 580, "w": 240, "h": 48 }
    }
  ],
  "redactions": [ /* SecureTextField frames blacked out */ ]
}

See Concepts → Observe for the model.