I was trying to do something like this a while ago…
Need all high-resolution images from a fashion collection on Vogue’s often-paywalled slideshow viewer.
deps.edn
{:paths ["script"]
:deps {etaoin/etaoin {:mvn/version "1.0.40"}}}
script/dl.clj
(ns dl
(:require [etaoin.api :as e]))
(defn grab
[url]
(e/with-safari d
(e/go d url)
(e/wait-exists
d {:css "ul li picture.responsive-image > source"})
(loop
[links []]
(let [img (e/get-element-attr
d {:css "ul li picture.responsive-image img"}
"src")]
(if (= "disabled" (e/get-element-attr
d {:css "div[data-testid='RunwayGalleryControlNext'] > svg"}
"arrowdisabled"))
(conj links img)
(do
(when (e/visible?
d {:css ".persistent-bottom button[aria-label='Collapse']"})
(e/click
d {:css ".persistent-bottom button[aria-label='Collapse']"})
(e/wait d 1))
(e/click
d {:css "div[data-testid='RunwayGalleryControlNext']"})
(e/wait d 1)
(recur (conj links img))))))))
Then
(def links
(dl/grab "https://www.vogue.com/fashion-shows/spring-1993-ready-to-wear/john-galliano/slideshow/collection"))
(doseq [link links]
(spit "ss-93-rtw.txt" (format "%s\n" link) :append true))
while read p; do curl "${p}" -so "ss-93-rtw/${p##*/}"; done < ss-93-rtw.txt
Sorry Vogue..