std.url

URLs (RFC 3986), written in heff: a URL read from its text into its parts (parse), written back (render), a reference resolved against the URL it is in (resolve, RFC 3986 §5.2), and the escapes of its parts and of a query’s keys and values. Where the RFC leaves a choice, Go’s net/url is the model. The parts are kept as they are written, escapes and all: unescape reads one.

import std.url as URL

-- | >>> next "https://example.com/docs/a/index.html" "../b/"
-- | Just ("https://example.com/docs/b/")
next : String -> String -> Maybe String
next page link = case URL.parse page of
  Nothing -> Nothing
  Just base ->
    maybe Nothing (\ref -> Just (URL.render (URL.resolve base ref))) (URL.parse link)

URL

type URL =
  { scheme : String
  , user : Maybe String
  , host : String
  , port : Maybe Int
  , path : String
  , query : Maybe String
  , fragment : Maybe String
  }

A URL, or a reference relative to one, in its parts as they are written (escaped): the scheme ("" in a relative reference), the user information before an @, the host (an IPv6 address without its brackets), the port, the path, and the query and the fragment, without their ? and #.

escapeQuery

escapeQuery : String -> String

A key or a value of a query escaped, as Go’s QueryEscape: a space +, a letter, a digit, -, ., _ and ~ as they are, every other byte %XX.

>>> escapeQuery "a b&c=d/é"
"a+b%26c%3Dd%2F%C3%A9"
>>> escapeQuery "-_.~+"
"-_.~%2B"

escapePath

escapePath : String -> String

A segment of a path escaped, as Go’s PathEscape escapes it: a letter, a digit, -, ., _, ~ and $&+:=@ as they are, every other byte %XX — a space %20, a / %2F, a ; %3B.

>>> escapePath "a b/c?d#e"
"a%20b%2Fc%3Fd%23e"
>>> escapePath "x=1;y+z@h:p!$&'()*,~"
"x=1%3By+z@h:p%21$&%27%28%29%2A%2C~"

unescape

unescape : String -> Maybe String

Each escape %XX (in either case) the byte it is, a + kept; nothing if a % is not followed by two hex digits.

>>> unescape "a%20b+c%2fd%C3%A9"
Just ("a b+c/dé")
>>> unescape "100%"
Nothing
>>> unescape "%zz"
Nothing

unescapeQuery

unescapeQuery : String -> Maybe String

A key or a value of a query read: each escape %XX the byte it is, each + a space; nothing if a % is not followed by two hex digits.

>>> unescapeQuery "a+b%2Bc"
Just ("a b+c")
>>> unescapeQuery "%4"
Nothing

parse

parse : String -> Maybe URL

The URL, or the relative reference, s writes, if it writes one: a scheme (a letter, then letters, digits, +, - and .) and :, then perhaps // and an authority (user information and @, a host — an IPv6 address in brackets — and : and a port, its digits), a path, perhaps ? and a query, and perhaps # and a fragment. Nothing if the scheme or the port is not one, if a % is not followed by two hex digits, or if a byte is a control character. An empty port is none.

>>> parse "https://ann:pw@example.com:8080/a/b%2Fc?x=1&y=2#top"
Just ({ scheme = "https", user = Just ("ann:pw"), host = "example.com", port = Just (8080), path = "/a/b%2Fc", query = Just ("x=1&y=2"), fragment = Just ("top") })
>>> parse "http://[::1]:80/"
Just ({ scheme = "http", user = Nothing, host = "::1", port = Just (80), path = "/", query = Nothing, fragment = Nothing })
>>> parse "mailto:ann@example.com"
Just ({ scheme = "mailto", user = Nothing, host = "", port = Nothing, path = "ann@example.com", query = Nothing, fragment = Nothing })
>>> parse "../a?q#"
Just ({ scheme = "", user = Nothing, host = "", port = Nothing, path = "../a", query = Just ("q"), fragment = Just ("") })
>>> parse "//example.com"
Just ({ scheme = "", user = Nothing, host = "example.com", port = Nothing, path = "", query = Nothing, fragment = Nothing })
>>> (List.map (\s -> maybe True (\_ -> False) (parse s)) ["http://a:8x/", "1http://a", ":x", "a b:c", "http://a/%zz", "http://a/%2", "http://a/\n", "http://[::1", "http://[::1]x", "http://[a]/", "http://a]/"] : List Bool)
[True, True, True, True, True, True, True, True, True, True, True]

render

render : URL -> String

The text of u: parse’s inverse on what it reads, but that a URL with a scheme and a path from the root has an authority written, if an empty one, as Go writes it (file:/a is file:///a), and that an empty port is not written. An IPv6 host (one with a :) is written in brackets; a path is written from the root after an authority, and after ./ where its first segment would be read as a scheme.

>>> List.map (\s -> maybe "" render (parse s)) ["https://ann:pw@example.com:8080/a/b%2Fc?x=1&y=2#top", "http://[::1]:80/", "mailto:ann@example.com", "../a?q#", "//example.com", "file:///etc/hosts", "http:////x", "?", "#f", ""]
["https://ann:pw@example.com:8080/a/b%2Fc?x=1&y=2#top", "http://[::1]:80/", "mailto:ann@example.com", "../a?q#", "//example.com", "file:///etc/hosts", "http:////x", "?", "#f", ""]
>>> List.map (\s -> maybe "" render (parse s)) ["file:/etc/hosts", "http://a:/"]
["file:///etc/hosts", "http://a/"]
>>> render { scheme = "http", user = Nothing, host = "fe80::1", port = Just 8080, path = "x", query = Nothing, fragment = Nothing }
"http://[fe80::1]:8080/x"
>>> render { scheme = "", user = Nothing, host = "", port = Nothing, path = "a:b/c", query = Nothing, fragment = Nothing }
"./a:b/c"

resolve

resolve : URL -> URL -> URL

The URL the reference ref names, read where base is, as RFC 3986 §5.2 resolves one (strictly: a scheme in ref is its own, even base’s): ref’s parts from the first it has on, base’s before them, a relative path merged with base’s, and the . and .. segments of the path taken away.

>>> ((let resolved : String -> String = \r -> maybe "" (\b -> maybe "" (\x -> render (resolve b x)) (parse r)) (parse "http://a/b/c/d;p?q") in List.map resolved ["g:h", "g", "./g", "g/", "/g", "//g", "?y", "g?y"]) : List String)
["g:h", "http://a/b/c/g", "http://a/b/c/g", "http://a/b/c/g/", "http://a/g", "http://g", "http://a/b/c/d;p?y", "http://a/b/c/g?y"]
>>> ((let resolved : String -> String = \r -> maybe "" (\b -> maybe "" (\x -> render (resolve b x)) (parse r)) (parse "http://a/b/c/d;p?q") in List.map resolved ["#s", "g#s", "g?y#s", ";x", "g;x", "g;x?y#s", ""]) : List String)
["http://a/b/c/d;p?q#s", "http://a/b/c/g#s", "http://a/b/c/g?y#s", "http://a/b/c/;x", "http://a/b/c/g;x", "http://a/b/c/g;x?y#s", "http://a/b/c/d;p?q"]
>>> ((let resolved : String -> String = \r -> maybe "" (\b -> maybe "" (\x -> render (resolve b x)) (parse r)) (parse "http://a/b/c/d;p?q") in List.map resolved [".", "./", "..", "../", "../g", "../..", "../../", "../../g"]) : List String)
["http://a/b/c/", "http://a/b/c/", "http://a/b/", "http://a/b/", "http://a/b/g", "http://a/", "http://a/", "http://a/g"]
>>> ((let resolved : String -> String = \r -> maybe "" (\b -> maybe "" (\x -> render (resolve b x)) (parse r)) (parse "http://a/b/c/d;p?q") in List.map resolved ["../../../g", "../../../../g", "/./g", "/../g", "g.", ".g", "g..", "..g"]) : List String)
["http://a/g", "http://a/g", "http://a/g", "http://a/g", "http://a/b/c/g.", "http://a/b/c/.g", "http://a/b/c/g..", "http://a/b/c/..g"]
>>> ((let resolved : String -> String = \r -> maybe "" (\b -> maybe "" (\x -> render (resolve b x)) (parse r)) (parse "http://a/b/c/d;p?q") in List.map resolved ["./../g", "./g/.", "g/./h", "g/../h", "g;x=1/./y", "g;x=1/../y", "g?y/./x", "g#s/../x", "http:g"]) : List String)
["http://a/b/g", "http://a/b/c/g/", "http://a/b/c/g/h", "http://a/b/c/h", "http://a/b/c/g;x=1/y", "http://a/b/c/y", "http://a/b/c/g?y/./x", "http://a/b/c/g#s/../x", "http:g"]
>>> maybe "" (\b -> maybe "" (\x -> render (resolve b x)) (parse "a/./b")) (parse "http://h")
"http://h/a/b"

parseQuery

parseQuery : String -> Maybe (List (String, String))

A query’s keys and values, in order: its pieces between &s (an empty one left out), each a key, and = and a value ("" if it has none), both read by unescapeQuery; nothing if one does not read. Only & separates two (a ; is no separator, but text).

>>> parseQuery "a=1&b=x+y%21&a=3&flag&&=v&k=a=b"
Just ([MkPair ("a") ("1"), MkPair ("b") ("x y!"), MkPair ("a") ("3"), MkPair ("flag") (""), MkPair ("") ("v"), MkPair ("k") ("a=b")])
>>> parseQuery ""
Just ([])
>>> parseQuery "a=%zz"
Nothing

renderQuery

renderQuery : List (String, String) -> String

A query of the keys and values, in the order given, each key=value escaped by escapeQuery, joined by &.

>>> renderQuery [MkPair "q" "heff lang", MkPair "page" "2", MkPair "x&y" ""]
"q=heff+lang&page=2&x%26y="
>>> parseQuery (renderQuery [MkPair "a b" "c=d&e", MkPair "" "+"])
Just ([MkPair ("a b") ("c=d&e"), MkPair ("") ("+")])