fetch_url#

pypillometry.fetch_url(url, cache_file=None, cache_dir=None, force_download=False, session=None, cache_layout='flat')[source]#

Download an HTTP(S) or SSH URL to a local file, with caching.

HTTP(S) URLs are downloaded with download() (OSF client URLs such as https://osf.io/<id> are converted to download URLs).

SSH URLs have the form ssh://[user@]host[:port]/path and are downloaded with the system’s scp, which uses ~/.ssh/config (host aliases, users, keys). Only key-based login is supported: password prompts are disabled (BatchMode=yes). The path is absolute on the server; use ssh://host/~/path for a path relative to the home directory. An SSH URL ending in / downloads the whole folder.

Parameters:

  • url (str) – HTTP(S) or SSH URL

  • cache_file (Optional[str]) – local filename to store the download in

  • cache_dir (Optional[str]) – directory to store the download in (used if cache_file is None); see cache_layout. If both are None, a new temporary directory is used. The location is logged.

  • force_download (bool) – if False (default), an existing local copy is reused; if True, the file is downloaded again

  • session (Optional[Session]) – requests session for HTTP(S) downloads (e.g., from osf_authenticate() for private OSF projects)

  • cache_layout (str) – where in cache_dir the download is stored: "flat" (default) uses the file name only (cache_dir/<file>), "mirror" the full remote path (cache_dir/<host>/<path>). With "flat", files with the same name from different URLs share one local copy, so the first one would be reused; use "mirror" (or force_download) in that case.

Return type:

str

Returns:

local path of the downloaded file (or folder)