raft/webdriver.org

Webdriver implementation in Chicken Scheme

• Dependencies
• Error Conditions
• WebDriver
  • Geckodriver
• WebDriver API
  • Communication
  • Session management
  • API Access Methods
  • Timeouts
    • Setting and getting timeouts
  • Elements
    • Element Class
    • Finding Elements
      • Location Strategies
      • Accessor Methods
    • Working with Elements
• About This Egg
  • Source
  • Author
  • Version History
  • License

Dependencies

Dependency	Description
srfi-34	Exception Handling
srfi-35	Exception Types
base64	decoding screenshot data
http-client	API interaction
intarweb	Supporting HTTP functionality
uri-common	Supporting HTTP functionality
coops	Object system
alist-lib	Handling alists from JSON objects
medea	JSON handling

Error Conditions

(define-condition-type &wd-exception &error wd-exception?
  (stacktrace wd-stacktrace)
  (data wd-data))

Every API error code (key "error" in the returned JSON data) gets its own condition type, prefixed by &. They all inherit from &wd-exception.

Name	API Error Code
detached-shadow-root	detached shadow root
element-click-intercepted	element click intercepted
element-not-interactable	element not interactable
insecure-certificate	insecure certificate
invalid-argument	invalid argument
invalid-cookie-domain	invalid cookie domain
invalid-element-state	invalid element state
invalid-selector	invalid selector
invalid-session-id	invalid session id
javascript-error	javascript error
move-target-out-of-bounds	move target out of bounds
no-such-alert	no such alert
no-such-cookie	no such cookie
no-such-element	no such element
no-such-frame	no such frame
no-such-shadow-root	no such shadow root
no-such-window	no such window
script-timeout	script timeout
session-not-created	session not created
stale-element-reference	stale element reference
timeout	timeout
unable-to-capture-screen	unable to capture screen
unable-to-set-cookie	unable to set cookie
unexpected-alert-open	unexpected alert open
unknown-command	unknown command
unknown-error	unknown error
unknown-method	unknown method
unsupported-operation	unsupported operation

WebDriver

The core element of the library is the <WebDriver> class and its subclasses. The class has the following fields:

(define-class <WebDriver> ()
  ((browser #f)
   (active? #f)
   (browser-pid #f)
   (server #f)
   (port #f)
   (session-id #f)
   (prefs #f)
   (capabilities #f)))

The parent class provides a handful of methods, but does not implement all of them; some are the sole responsibility of the subclass. The launch method, on the other hand, bears shared responsibility. It sets a finalizer to ensure termination of the web driver process in case the class is disposed of with a still-open driver.

(define-method (launch #:after (instance <WebDriver>) options)
  (set-finalizer! instance (lambda (obj)
                             (when (slot-value instance 'active?)
                               (terminate instance)))))

(define-method (terminate (instance <WebDriver>))
  (terminate-session instance)
  (process-signal (slot-value instance 'browser-pid))
  (set! (slot-value instance 'browser-pid) #f)
  (set! (slot-value instance 'active?) #f))

(define-method (construct-capabilities (instance <WebDriver>) #!optional caps)
  (raise 'subclass-responsibility))

(define-method (postprocess-result (instance <WebDriver>) result)
  result)

Main initialization is done by calling the new-WebDriver procedure with the respective class name and optionally an alist of options.

(define (new-WebDriver browser #!optional options)
  (let ((instance (make browser)))
    (launch instance options)
    (sleep 1)
    instance))

Geckodriver

The Geckodriver is used to control Firefox.

(define-class <Gecko> (<WebDriver>)
  ((browser #:firefox)
   (server "127.0.0.1")
   (port 4444)))

(define-method (launch (instance <Gecko>) options)
  (let ((pid (process-run "geckodriver > /dev/null 2>&1")))
    (set! (slot-value instance 'browser-pid) pid)
    (set! (slot-value instance 'active?) #t)
    (set! (slot-value instance 'capabilities) options)))

The capabilities object for Geckodriver is of the form {"capabilities": {...}}. For more information on capabilities, see https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities.

(define-method (construct-capabilities (instance <Gecko>))
  (let ((caps (or (slot-value instance 'capabilities) (list))))
    `((capabilities . ,caps))))

Sometimes, Geckodriver returns the results of a command in a JSON object with the sole key "value". We have to correct that before returning the data to the user.

(define-method (postprocess-result (instance <Gecko>) result)
  (alist-ref/default result 'value result))

WebDriver API

Communication

Data is sent to the API via a central class method. For convenience, there is a send-with-session variant that automatically adds the session id.

(define-method (send (instance <WebDriver>) data uri method)
  (let* ((remote (string-append "http://" (slot-value instance 'server) ":" (->string (slot-value instance 'port)) "/"))
         (result (postprocess-result instance
                                     (with-input-from-request
                                      (make-request #:method method
                                                    #:uri (uri-reference (string-append remote uri))
                                                    #:headers (headers `((content-type application/json))))
                                      (if data (json->string data) "")
                                      read-json))))
    (if (and (list? result) (alist-ref/default result 'error #f))
        (wd-throw result)
        result)))

(define-method (send-with-session (instance <WebDriver>) data uri method)
  (send instance data (string-append "session/" (slot-value instance 'session-id) "/" uri) method))

Session management

Session management is very simple. There is just one method to initialize a new session. Everything else is handled automatically.

(define-method (initialize-session (instance <WebDriver>))
 (let ((result (send instance (construct-capabilities instance) "session" 'POST)))
   (set! (slot-value instance 'session-id) (alist-ref result 'sessionId))))

(define-method (terminate-session (instance <WebDriver>))
  (when (slot-value instance 'session-id)
    (send instance #f (string-append "session/" (slot-value instance 'session-id)) 'DELETE))
  (set! (slot-value instance 'session-id) #f))

-- testing session -----------------------------------------------------------
Initial state ........................................................ [ PASS]
Session id check ..................................................... [ PASS]
Session id after termination ......................................... [ PASS]
3 tests completed in 3.788 seconds.
3 out of 3 (100%) tests passed.
-- done testing session ------------------------------------------------------

API Access Methods

(define-method (set-url (instance <WebDriver>) url)
  (send-with-session instance `((url . ,url)) "url" 'POST))

(define-method (url (instance <WebDriver>))
  (send-with-session instance #f "url" 'GET))

-- testing url ---------------------------------------------------------------
Initial state ........................................................ [ PASS]
Navigating to the first website ...................................... [ PASS]
2 tests completed in 5.247 seconds.
2 out of 2 (100%) tests passed.
-- done testing url ----------------------------------------------------------

(define-method (back (instance <WebDriver>))
  (send-with-session instance #f "back" 'POST))

(define-method (forward (instance <WebDriver>))
  (send-with-session instance #f "forward" 'POST))

(define-method (refresh (instance <WebDriver>))
  (send-with-session instance #f "refresh" 'POST))

(define-method (title (instance <WebDriver>))
  (send-with-session instance #f "title" 'GET))

(define-method (status (instance <WebDriver>))
  (send-with-session instance #f "status" 'GET))

(define-method (source (instance <WebDriver>))
  (send-with-session instance #f "source" 'GET))

(define-method (screenshot (instance <WebDriver>))
  (base64-decode (send-with-session instance #f "screenshot" 'GET)))

(define-method (print-page (instance <WebDriver>))
  (send-with-session instance #f "print" 'POST))

(define-method (execute-async (instance <WebDriver>) script args)
  (send-with-session instance `((script . ,script) (args . ,args)) "execute/async" 'POST))

(define-method (execute-sync (instance <WebDriver>) script args)
  (send-with-session instance `((script . ,script) (args . ,args)) "execute/sync" 'POST))

Timeouts

The following timeouts are defined:

• script: defaults to 30'000, specifies when to interrupt a script that is being evaluated. A nil value implies that scripts should never be interrupted, but instead run indefinitely.
• pageLoad: defaults to 300'000, provides the timeout limit used to interrupt an explicit navigation attempt.
• implicit: defaults to 0, specifies a time to wait for the element location strategy to complete when locating an element.

(define-class <WDTimeouts> ()
  ((script 30000)
   (pageLoad 300000)
   (implicit 0)))

(define-method (extract (instance <WDTimeouts>))
  `((script . ,(slot-value instance 'script))
    (pageLoad . ,(slot-value instance 'pageLoad))
    (implicit . ,(slot-value instance 'implicit))))

Setting and getting timeouts

(define-method (set-timeouts (instance <WebDriver>) (timeouts <WDTimeouts>))
  (send-with-session instance (extract timeouts) "timeouts" 'POST))

(define-method (timeouts (instance <WebDriver>))
  (let ((result (send-with-session instance #f "timeouts" 'GET)))
    (make <WDTimeouts>
      'script (alist-ref result 'script)
      'pageLoad (alist-ref result 'pageLoad)
      'implicit (alist-ref result 'implicit))))

Elements

Element Class

(define-class <WDElement> ()
  ((driver #f)
   (element #f)))

(define-method (send-with-session (instance <WDElement>) data uri method)
  (send-with-session (slot-value instance 'driver) data
                     (string-append "element/" (slot-value instance 'element) "/" uri)
                     method))

Finding Elements

Location Strategies

(define css-selector "css selector")
(define link-text "link text")
(define partial-link-text "partial link text")
(define tag-name "tag name")
(define xpath "xpath")

Accessor Methods

(define-method (find-element (instance <WebDriver>) strategy selector)
  (let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "element" 'POST)))
    (make <WDElement> 'driver instance 'element (car (alist-values result)))
    element))

(define-method (find-elements (instance <WebDriver>) strategy selector)
  (let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "elements" 'POST)))
    (map
     (lambda (elem)
       (make <WDElement> 'driver instance 'element (car (alist-values elem))))
     result)))

(define-method (find-element (instance <WDElement>) strategy selector)
  (let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "element" 'POST)))
    (make <WDElement> 'driver (slot-value instance 'driver) 'element (car (alist-values result)))
    element))

(define-method (find-elements (instance <WDElement>) strategy selector)
  (let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "elements" 'POST)))
    (map
     (lambda (elem)
       (make <WDElement> 'driver (slot-value instance 'driver) 'element (car (alist-values elem))))
     result)))

Working with Elements

(define-method (attribute (instance <WDElement>) attribute)
  (let ((result (send-with-session instance #f
                                   (string-append "attribute/" attribute)
                                   'GET)))
    (if (equal? "true" result)
        #t
        result)))

(define-method (property (instance <WDElement>) property)
  (send-with-session instance #f (string-append "property/" property) 'GET))

(define-method (clear (instance <WDElement>))
  (send-with-session instance #f "clear" 'POST))

(define-method (click (instance <WDElement>))
  (send-with-session instance #f "click" 'POST))

(define-method (computed-label (instance <WDElement>))
  (send-with-session instance #f "computedlabel" 'GET))

(define-method (computed-role (instance <WDElement>))
  (send-with-session instance #f "computedrole" 'GET))

(define-method (enabled? (instance <WDElement>))
  (send-with-session instance #f "enabled" 'GET))

(define-method (selected? (instance <WDElement>))
  (send-with-session instance #f "selected" 'GET))

(define-method (name (instance <WDElement>))
  (send-with-session instance #f "name" 'GET))

(define-method (rect (instance <WDElement>))
  (send-with-session instance #f "rect" 'GET))

(define-method (screenshot (instance <WDElement>))
  (base64-decode (send-with-session instance #f "screenshot" 'GET)))

(define-method (text (instance <WDElement>))
  (send-with-session instance #f "text" 'GET))

(define-method (set-value (instance <WDElement>) value)
  (send-with-session instance `((text . ,value)) "value" 'POST))

About This Egg

Source

The source is available at https://fossil.lyrion.ch/chicken-webdriver.

Author

Daniel Ziltener

Version History

0.5	Initial Release

License

Copyright (C) 2023 Daniel Ziltener

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the <organization> nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.