Parsers
Parsers are at the core of what Pathom does. This section explains how Pathom parsers accomplish this responsibility and describes how they work.
The parser’s job
Every EQL transaction is a vector, which means by definition, an EQL transaction is a collection of things. The parser’s job is to walk this collection, figuring out the value of each requested entry and return a map with them.
In case of Pathom parsers, they also have a distinction between read
and mutate
, when
the parser is built those are provided separately.
It may look like a simple task but it can get quite complicated depending on how you intend to coordinate that processing. Let’s take a look at the built-in parsers from Pathom:
Serial parser
This is the simplest parser available as it doesn’t do anything more than a reduce, calling the reader for each entry in the query.
For the sake of understanding, let’s see how we can implement a simple naive serial parser:
; this a simple parser that reads entries from the env based on user query
(defn reduce-parser [env eql]
(reduce
(fn [out entry]
(assoc out entry (get env entry)))
{}
eql))
; process EQL!
(reduce-parser {:foo "bar" :answer 42} [:answer])
; => {:answer 42}
Now let’s push a different query to our parser, adding a parameter to our entry:
(reduce-parser {:foo "bar" :answer 42} ['(:answer {:with "param"})])
; => {(:answer {:with "param"}) nil}
So, that didn’t go so well; we got now a key out that includes the list and no value.
We could deal with this manually, checking for the list; but EQL has enough syntax that this turns in trouble quickly. To fix this, instead of dealing with the EQL directly, let’s convert it to the AST format and process that:
(defn reduce-parser [env eql]
; convert to ast
(let [ast (eql/query->ast eql)]
(reduce
(fn [out {:keys [key]}]
(assoc out key (get env key)))
{}
; process children
(:children ast))))
; now that we can get specific parts of entry, syntax details fade away
(reduce-parser {:foo "bar" :answer 42} [:answer])
; => {:answer 42}
(reduce-parser {:foo "bar" :answer 42} ['(:answer {:with "param"})])
; => {:answer 42}
You may be thinking: what about nested queries?
And the answer is: the parser has nothing to do with that; let’s understand why.
Let’s compare these two queries:
[:a
:b]
[:a
{:b [:c]}]
How many elements does each query have? The answer is two, for both. It just
happens that in the second query, the second element is a join; but from the
parser’s point of view, in the first case it gets :a
and :b
while in
the second case it gets an :a
and {:b [:c]}
.
That’s the reason why it’s not valid to do multiple joins in the same map in EQL, the map itself is considered one entry because the parser sees it as one element.
Pathom processes sub-queries at the reader level. The reader can look at the
AST and see that element has :children
, then it calls the parser again with the
children (and usually with some modifications to the environment) to process that
sub-query.
To illustrate how this processing works, let’s use the Pathom parser again:
(def count-parser
(p/parser {::p/env {::p/reader #(swap! (::counter %) inc)}}))
This time we created a count parser
, this parser will just increase a counter
and return its value, no matter what you ask, you can try it:
Notice that if you do queries with joins, they will just be ignored since the reader returns the value immediately. If we’d like to increase the counter only on the leaves, we can leverage Pathom reader composition and put something before just to walk the joins.
(defn join-walk-reader
[{:keys [query] :as env}]
(if query
(p/join env)
::p/continue))
(def count-parser
(p/parser {::p/env {::p/reader [join-walk-reader
#(swap! (::counter %) inc)]}}))
For our purposes |
You are encouraged to play around with the following example and understand the ordering in which Pathom processes the elements:
Through the previous description you may have realized, since readers process sub
queries, the result is a depth first pre-order traversal. To illustrate, think of the query:
|
These are the basics of the serial parser; and here is a list of things the Pathom parser does on top of was discussed in the previous section:
-
Fill environment with
:ast
,:parser
(itself) and:query
-
Support plugins
-
Provide
::p/path
to allow for path tracking -
Support output renaming via
:pathom/as
parameter -
Add parser related tracing events