Chicken/raft
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
raft/raft.org

710 lines
24 KiB
Org Mode
Raw Blame History

#+title: Raft - Webdriver implementation in Chicken Scheme
#+author: Daniel Ziltener
#+property: header-args:scheme :session *chicken* :comments both
* Helpers :noexport:
:PROPERTIES:
:header-args:scheme: :prologue "(import (chicken string))"
:END:
** Strip garbage from test results
#+name: test-post
#+begin_src scheme :var input='() :results output
(for-each (lambda (str)
(or (substring=? str ";")
(substring=? str "Note")
(print str)))
(string-split input "\n"))
#+end_src
** Prepare in-line testing
#+name: prep-test
#+begin_src scheme :noweb yes :tangle tests/run.scm :results silent
(import r7rs
test
(chicken base)
(chicken string)
(chicken process)
(chicken gc)
<<dependencies-for-imports()>>
)
#+end_src
* Dependencies
#+name: dependencies
| Dependency | Description |
|-------------+-----------------------------------|
| alist-lib | Handling alists from JSON objects |
| base64 | decoding screenshot data |
| coops | Object system |
| http-client | API interaction |
| intarweb | Supporting HTTP functionality |
| medea | JSON handling |
| srfi-34 | Exception Handling |
| srfi-35 | Exception Types |
| uri-common | Supporting HTTP functionality |
#+name: dependencies-for-egg
#+begin_src emacs-lisp :var tbl=dependencies :colnames yes :results raw :exports none
(mapconcat (lambda (row) (car row)) tbl " ")
#+end_src
#+name: dependencies-for-imports
#+begin_src emacs-lisp :var tbl=dependencies :colnames yes :results raw :exports none
(mapconcat (lambda (row) (concat (car row) "\t\t;;" (cadr row)))
tbl "\n")
#+end_src
#+begin_src scheme :noweb yes :tangle raft.scm :exports none
(import r7rs)
(define-library (raft)
(import (scheme base))
(export <Raft>
<Gecko>
make-Raft
terminate
initialize-session
terminate-session
set-url
url
back
forward
refresh
title
status
source
screenshot
print-page
execute-async
execute-sync
<RaftTimeouts>
set-timeouts
timeouts
<RaftElement>
css-selector
link-text
partial-link-text
tag-name
xpath
find-element
find-elements
attribute
property
clear
click
computed-label
computed-role
enabled?
selected?
name
rect
screenshot
text
set-value
&raft-exception
raft-exception?
raft-stacktrace
raft-data
<<export-conditions()>>
)
(include "raft-impl.scm"))
#+end_src
#+begin_src scheme :noweb yes :tangle raft-impl.scm :exports none
(import r7rs
(chicken base)
(chicken string)
(chicken process)
(chicken gc)
<<dependencies-for-imports()>>
)
#+end_src
#+begin_src scheme :tangle tests/run.scm :exports none :mkdirp yes
(include-relative "../raft-impl.scm")
#+end_src
* Error Conditions
#+name: raft-exception
#+begin_src scheme :tangle raft-impl.scm
(define-condition-type &raft-exception &error raft-exception?
(stacktrace raft-stacktrace)
(data raft-data))
#+end_src
Every API error code (key "error" in the returned JSON data) gets its own condition type, prefixed by ~&~. They all inherit from ~&raft-exception~.
#+name: error-code-table
| Name | API Error Code |
|---------------------------+---------------------------|
| detached-shadow-root | detached shadow root |
| element-click-intercepted | element click intercepted |
| element-not-interactable | element not interactable |
| insecure-certificate | insecure certificate |
| invalid-argument | invalid argument |
| invalid-cookie-domain | invalid cookie domain |
| invalid-element-state | invalid element state |
| invalid-selector | invalid selector |
| invalid-session-id | invalid session id |
| javascript-error | javascript error |
| move-target-out-of-bounds | move target out of bounds |
| no-such-alert | no such alert |
| no-such-cookie | no such cookie |
| no-such-element | no such element |
| no-such-frame | no such frame |
| no-such-shadow-root | no such shadow root |
| no-such-window | no such window |
| script-timeout | script timeout |
| session-not-created | session not created |
| stale-element-reference | stale element reference |
| timeout | timeout |
| unable-to-capture-screen | unable to capture screen |
| unable-to-set-cookie | unable to set cookie |
| unexpected-alert-open | unexpected alert open |
| unknown-command | unknown command |
| unknown-error | unknown error |
| unknown-method | unknown method |
| unsupported-operation | unsupported operation |
#+name: export-conditions
#+begin_src emacs-lisp :var src=error-code-table :exports none :results raw
(mapconcat
(lambda (row)
(let ((replace `((?n . ,(cl-first row)))))
(format-spec "&%n %n?" replace)))
src "\n")
#+end_src
#+name: define-conditions
#+begin_src emacs-lisp :var src=error-code-table :exports none :results raw
(mapconcat
(lambda (row)
(let ((replace `((?n . ,(cl-first row))
(?c . ,(cl-second row)))))
(format-spec "(define-condition-type &%n &raft-exception %n?)" replace)))
src "\n")
#+end_src
#+name: define-condition-thrower
#+begin_src emacs-lisp :var src=error-code-table :exports none :results raw
(concat "(define (raft-throw data)\n"
" (case (alist-ref data 'error)\n"
(mapconcat
(lambda (row)
(let ((replace `((?n . ,(cl-first row))
(?c . ,(cl-second row)))))
(format-spec " ((\"%c\") (raise (make-condition &%n (alist-ref data 'stacktrace) data)))" replace)))
src "\n")
"\n"
" (else (raise (make-condition &raft-exception (alist-ref data 'stacktrace) data)))\n"
" )\n"
")")
#+end_src
#+name: conditions
#+begin_src scheme :tangle raft-impl.scm :noweb yes :exports none
<<define-conditions()>>
<<define-condition-thrower()>>
#+end_src
* WebDriver
The core element of the library is the ~<Raft>~ class and its subclasses. The class has the following fields:
#+name: webdriver-class
#+begin_src scheme :tangle raft-impl.scm
(define-class <Raft> ()
((browser #f)
(active? #f)
(browser-pid #f)
(server #f)
(port #f)
(session-id #f)
(prefs #f)
(capabilities #f)))
#+end_src
The parent class provides a handful of methods, but does not implement all of them; some are the sole responsibility of the subclass. The ~launch~ method, on the other hand, bears shared responsibility. It sets a finalizer to ensure termination of the web driver process in case the class is disposed of with a still-open driver.
#+name: webdriver-basics
#+begin_src scheme :tangle raft-impl.scm
(define-method (launch #:after (instance <Raft>) options)
(set-finalizer! instance (lambda (obj)
(when (slot-value instance 'active?)
(terminate instance)))))
(define-method (terminate (instance <Raft>))
(terminate-session instance)
(process-signal (slot-value instance 'browser-pid))
(set! (slot-value instance 'browser-pid) #f)
(set! (slot-value instance 'active?) #f))
(define-method (construct-capabilities (instance <Raft>) #!optional caps)
(raise 'subclass-responsibility))
(define-method (postprocess-result (instance <Raft>) result)
result)
#+end_src
Main initialization is done by calling the ~make-Raft~ procedure with the respective class name and optionally an alist of options.
#+name: webdriver-init
#+begin_src scheme :tangle raft-impl.scm
(define (make-Raft browser #!optional options)
(let ((instance (make browser)))
(launch instance options)
(sleep 1)
instance))
#+end_src
** Geckodriver
The Geckodriver is used to control Firefox.
#+name: geckodriver-basic
#+begin_src scheme :tangle raft-impl.scm
(define-class <Gecko> (<Raft>)
((browser #:firefox)
(server "127.0.0.1")
(port 4444)))
(define-method (launch (instance <Gecko>) options)
(let ((pid (process-run "geckodriver > /dev/null 2>&1")))
(set! (slot-value instance 'browser-pid) pid)
(set! (slot-value instance 'active?) #t)
(set! (slot-value instance 'capabilities) options)))
#+end_src
The capabilities object for Geckodriver is of the form ={"capabilities": {...}}=.
For more information on capabilities, see https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities.
#+name: geckodriver-capabilities
#+begin_src scheme :tangle raft-impl.scm
(define-method (construct-capabilities (instance <Gecko>))
(let ((caps (or (slot-value instance 'capabilities) (list))))
`((capabilities . ,caps))))
#+end_src
Sometimes, Geckodriver returns the results of a command in a JSON object with the sole key ="value"=. We have to correct that before returning the data to the user.
#+name: geckodriver-postprocess
#+begin_src scheme :tangle raft-impl.scm
(define-method (postprocess-result (instance <Gecko>) result)
(alist-ref/default result 'value result))
#+end_src
#+name: prep-geckodriver-test
#+begin_src scheme :noweb strip-tangle :exports none :post test-post(input=*this*) :results output
<<prep-test>>
<<raft-exception>>
<<conditions>>
<<webdriver-class>>
<<webdriver-basics>>
<<webdriver-init>>
<<geckodriver-basic>>
<<geckodriver-capabilities>>
<<geckodriver-postprocess>>
<<raft-send>>
<<raft-init-session>>
<<raft-term-session>>
#+end_src
* WebDriver API
** Communication
Data is sent to the API via a central class method. For convenience, there is a ~send-with-session~ variant that automatically adds the session id.
#+name: raft-send
#+begin_src scheme :tangle raft-impl.scm
(define-method (send (instance <Raft>) data uri method)
(let* ((remote (string-append "http://" (slot-value instance 'server) ":" (->string (slot-value instance 'port)) "/"))
(result (postprocess-result instance
(with-input-from-request
(make-request #:method method
#:uri (uri-reference (string-append remote uri))
#:headers (headers `((content-type application/json))))
(if data (json->string data) "")
read-json))))
(if (and (list? result) (alist-ref/default result 'error #f))
(raft-throw result)
result)))
(define-method (send-with-session (instance <Raft>) data uri method)
(send instance data (string-append "session/" (slot-value instance 'session-id) "/" uri) method))
#+end_src
** Session management
Session management is very simple. There is just one method to initialize a new session. Everything else is handled automatically.
#+name: raft-init-session
#+begin_src scheme :tangle raft-impl.scm
(define-method (initialize-session (instance <Raft>))
(let ((result (send instance (construct-capabilities instance) "session" 'POST)))
(set! (slot-value instance 'session-id) (alist-ref result 'sessionId))))
#+end_src
#+name: raft-term-session
#+begin_src scheme :tangle raft-impl.scm
(define-method (terminate-session (instance <Raft>))
(when (slot-value instance 'session-id)
(send instance #f (string-append "session/" (slot-value instance 'session-id)) 'DELETE))
(set! (slot-value instance 'session-id) #f))
#+end_src
#+name: raft-session-test
#+begin_src scheme :tangle tests/run.scm :noweb strip-tangle :exports results :post test-post(input=*this*) :results output
<<prep-geckodriver-test>>
(test-group "session"
(let ((browser (make-Raft <Gecko>)))
(test "Initial state" #f (slot-value browser 'session-id))
(test-assert "Session id check" (string? (begin (initialize-session browser) (slot-value browser 'session-id))))
(test-assert "Session id after termination" (eq? #f (begin (terminate-session browser) (slot-value browser 'session-id))))
(terminate browser)))
#+end_src
#+RESULTS: raft-session-test
: -- testing session -----------------------------------------------------------
: Initial state ........................................................ [ PASS]
: Session id check ..................................................... [ PASS]
: Session id after termination ......................................... [ PASS]
: 3 tests completed in 3.788 seconds.
: 3 out of 3 (100%) tests passed.
: -- done testing session ------------------------------------------------------
** API Access Methods
#+name: raft-url
#+begin_src scheme :tangle raft-impl.scm
(define-method (set-url (instance <Raft>) url)
(send-with-session instance `((url . ,url)) "url" 'POST))
(define-method (url (instance <Raft>))
(send-with-session instance #f "url" 'GET))
#+end_src
#+name: raft-url-test
#+begin_src scheme :tangle tests/run.scm :noweb strip-tangle :exports results :post test-post(input=*this*) :results output
<<prep-geckodriver-test>>
<<raft-url>>
(test-group "url"
(let ((browser (make-Raft <Gecko>)))
(test "Initial state" #f (slot-value browser 'session-id))
(test "Navigating to the first website" "http://info.cern.ch/hypertext/WWW/TheProject.html"
(begin (initialize-session browser)
(set-url browser "http://info.cern.ch/hypertext/WWW/TheProject.html")
(url browser)))
(terminate browser)))
#+end_src
#+RESULTS: raft-url-test
: -- testing url ---------------------------------------------------------------
: Initial state ........................................................ [ PASS]
: Navigating to the first website ...................................... [ PASS]
: 2 tests completed in 5.247 seconds.
: 2 out of 2 (100%) tests passed.
: -- done testing url ----------------------------------------------------------
#+begin_src scheme :tangle raft-impl.scm
(define-method (back (instance <Raft>))
(send-with-session instance #f "back" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (forward (instance <Raft>))
(send-with-session instance #f "forward" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (refresh (instance <Raft>))
(send-with-session instance #f "refresh" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (title (instance <Raft>))
(send-with-session instance #f "title" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (status (instance <Raft>))
(send-with-session instance #f "status" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (source (instance <Raft>))
(send-with-session instance #f "source" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (screenshot (instance <Raft>))
(base64-decode (send-with-session instance #f "screenshot" 'GET)))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (print-page (instance <Raft>))
(send-with-session instance #f "print" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (execute-async (instance <Raft>) script args)
(send-with-session instance `((script . ,script) (args . ,args)) "execute/async" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (execute-sync (instance <Raft>) script args)
(send-with-session instance `((script . ,script) (args . ,args)) "execute/sync" 'POST))
#+end_src
** Timeouts
The following timeouts are defined:
- =script=: defaults to 30'000, specifies when to interrupt a script that is being evaluated. A nil value implies that scripts should never be interrupted, but instead run indefinitely.
- =pageLoad=: defaults to 300'000, provides the timeout limit used to interrupt an explicit navigation attempt.
- =implicit=: defaults to 0, specifies a time to wait for the element location strategy to complete when locating an element.
#+begin_src scheme :tangle raft-impl.scm
(define-class <RaftTimeouts> ()
((script 30000)
(pageLoad 300000)
(implicit 0)))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (extract (instance <RaftTimeouts>))
`((script . ,(slot-value instance 'script))
(pageLoad . ,(slot-value instance 'pageLoad))
(implicit . ,(slot-value instance 'implicit))))
#+end_src
*** Setting and getting timeouts
#+begin_src scheme :tangle raft-impl.scm
(define-method (set-timeouts (instance <Raft>) (timeouts <RaftTimeouts>))
(send-with-session instance (extract timeouts) "timeouts" 'POST))
(define-method (timeouts (instance <Raft>))
(let ((result (send-with-session instance #f "timeouts" 'GET)))
(make <RaftTimeouts>
'script (alist-ref result 'script)
'pageLoad (alist-ref result 'pageLoad)
'implicit (alist-ref result 'implicit))))
#+end_src
** Elements
*** Element Class
#+begin_src scheme :tangle raft-impl.scm
(define-class <RaftElement> ()
((driver #f)
(element #f)))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (send-with-session (instance <RaftElement>) data uri method)
(send-with-session (slot-value instance 'driver) data
(string-append "element/" (slot-value instance 'element) "/" uri)
method))
#+end_src
*** Finding Elements
**** Location Strategies
#+begin_src scheme :tangle raft-impl.scm
(define css-selector "css selector")
(define link-text "link text")
(define partial-link-text "partial link text")
(define tag-name "tag name")
(define xpath "xpath")
#+end_src
**** Accessor Methods
#+begin_src scheme :tangle raft-impl.scm
(define-method (find-element (instance <Raft>) strategy selector)
(let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "element" 'POST)))
(make <RaftElement> 'driver instance 'element (car (alist-values result)))))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (find-elements (instance <Raft>) strategy selector)
(let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "elements" 'POST)))
(map
(lambda (elem)
(make <RaftElement> 'driver instance 'element (car (alist-values elem))))
result)))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (find-element (instance <RaftElement>) strategy selector)
(let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "element" 'POST)))
(make <RaftElement> 'driver (slot-value instance 'driver) 'element (car (alist-values result)))))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (find-elements (instance <RaftElement>) strategy selector)
(let ((result (send-with-session instance `((using . ,strategy) (value . ,selector)) "elements" 'POST)))
(map
(lambda (elem)
(make <RaftElement> 'driver (slot-value instance 'driver) 'element (car (alist-values elem))))
result)))
#+end_src
*** Working with Elements
#+begin_src scheme :tangle raft-impl.scm
(define-method (attribute (instance <RaftElement>) attribute)
(let ((result (send-with-session instance #f
(string-append "attribute/" attribute)
'GET)))
(if (equal? "true" result)
#t
result)))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (property (instance <RaftElement>) property)
(send-with-session instance #f (string-append "property/" property) 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (clear (instance <RaftElement>))
(send-with-session instance #f "clear" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (click (instance <RaftElement>))
(send-with-session instance #f "click" 'POST))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (computed-label (instance <RaftElement>))
(send-with-session instance #f "computedlabel" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (computed-role (instance <RaftElement>))
(send-with-session instance #f "computedrole" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (enabled? (instance <RaftElement>))
(send-with-session instance #f "enabled" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (selected? (instance <RaftElement>))
(send-with-session instance #f "selected" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (name (instance <RaftElement>))
(send-with-session instance #f "name" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (rect (instance <RaftElement>))
(send-with-session instance #f "rect" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (screenshot (instance <RaftElement>))
(base64-decode (send-with-session instance #f "screenshot" 'GET)))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (text (instance <RaftElement>))
(send-with-session instance #f "text" 'GET))
#+end_src
#+begin_src scheme :tangle raft-impl.scm
(define-method (set-value (instance <RaftElement>) value)
(send-with-session instance `((text . ,value)) "value" 'POST))
#+end_src
* About This Egg
#+begin_src scheme :noweb yes :tangle raft.egg :exports none
;; -*- scheme -*-
((author "Daniel Ziltener")
(synopsis "A WebDriver API implementation for Chicken")
(category web)
(license "BSD")
(version <<latest-release()>>)
(dependencies r7rs <<dependencies-for-egg()>>)
(test-dependencies test)
(components
(extension webdriver
(csc-options "-X" "r7rs" "-R" "r7rs" "-sJ"))))
#+end_src
#+begin_src scheme :tangle tests/run.scm :exports none
(test-exit)
#+end_src
** Source
The source is available at [[https://gitea.lyrion.ch/Chicken/raft]].
** Author
Daniel Ziltener
** Version History
#+name: version-history
| 0.5 | Initial Release |
#+name: gen-releases
#+begin_src emacs-lisp :var vers=version-history :results raw :exports none
(mapconcat (lambda (row) (concat "(release \"" (number-to-string (car row)) "\") ;; " (cadr row)))
vers "\n")
#+end_src
#+name: latest-release
#+begin_src emacs-lisp :var vers=version-history :exports none :results code
(number-to-string (caar vers))
#+end_src
#+begin_src scheme :noweb yes :tangle raft.release-info :exports none
;; -*- scheme -*-
(repo git "https://gitea.lyrion.ch/Chicken/raft")
(uri targz "https://gitea.lyrion.ch/Chicken/raft/archive/{egg-release}.tar.gz")
<<gen-releases()>>
#+end_src
** License
#+begin_src fundamental :tangle LICENSE
Copyright (C) 2023 Daniel Ziltener
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the <organization> nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#+end_src
Reference in a new issue View git blame Copy permalink