Globus¶

globus ¶

Globus integration: read NUANCE data straight from a Globus collection.

Three ways in, all recorded in provenance (collection id, path, checksum):

nuscope.load("globus://<collection-id>/path/to/file.dm4"): the file is streamed over HTTPS from a Globus Connect Server v5 collection into a local cache and opened. A file that is already cached with the same size is reused;
GlobusSession().ls(collection, path), .read_bytes(...) and .download(...) to browse and fetch;
GlobusSession().transfer(src, src_path, dst, dst_path) for large or many files: a managed, checksum-verified Globus Transfer between collections (e.g. an instrument PC collection to Quest or your laptop's Globus Connect Personal).

Authentication uses the Globus SDK's native-app login (a browser link, then a code pasted back), with refresh tokens stored under ~/.globus/app. It needs a Globus native app client id registered for NUScope at https://app.globus.org/settings/developers, given as client_id= or the NUSCOPE_GLOBUS_CLIENT_ID environment variable.

GlobusSession ¶

GlobusSession(client_id: Optional[str] = None, *, app=None, transfer_client=None, http=None, auth_header: Optional[Callable[[str], str]] = None, app_name: str = 'NUScope')

Talks to Globus Transfer (listing, transfers) and to collections over HTTPS (reads).

transfer_client, http and auth_header can be injected (tests, service accounts); by default they come from a Globus UserApp login.

require ¶

require(*collections: str) -> None

Ask for HTTPS access to these collections at the next login.

find ¶

find(collection: str, path: str = '/', pattern: str = '*', max_depth: int = 4) -> Iterator[RemoteFile]

Walk a directory tree (breadth first) yielding files that match a glob pattern.

download ¶

download(collection: str, path: str, dest: str | Path, chunk: int = 1 << 22) -> tuple[Path, str, int]

Stream a file to dest; returns (path, sha256, size).

transfer_to ¶

transfer_to(src: str, src_path: str, dst: str, dst_path: str, *, label: str = 'NUScope transfer', recursive: bool = False, sync_level: str = 'checksum') -> str

Submit a checksum-verified Globus Transfer; returns the task id.

parse_url ¶

parse_url(url: str) -> tuple[str, str]

globus:/// -> (collection id, absolute path).

fetch ¶

fetch(url: str, session: Optional[GlobusSession] = None, cache_dir: Optional[Path] = None, refresh: bool = False) -> tuple[Path, Entity]

Download a globus:// URL into the cache; return (local path, provenance entity).