Skip to content

Changelog

Unreleased

4.0.1

Sign-in by pasted code needs Crucible API 4.0.1.

Added

  • crucible login --paste and client.login(method="paste"): sign in in any browser and paste the code its page shows.
  • Press Enter while a browser sign-in waits to paste a code instead, when the browser cannot reach the terminal.
  • When the API answers from a new address for good (301 or 308), the client warns once and names the crucible config set api_url command, which also moves a stored sign-in to the new address.

Changed

  • Signing in and renewing a session follow an API that moved to a new path on the same host; a move to another host or from https to http is still refused.
  • samples.create() and datasets.create() accept the record's fields as a dict as well as a model, like the other create() methods.

Fixed

  • datasets.create() given something other than a Dataset or a dict raises a TypeError instead of an AttributeError.

4.0.0

Requires Crucible API 4.0 and Python 3.11 or later. Upgrading to 4.0 lists what to change in existing scripts.

Added

  • Sign in with a browser: crucible login (--device for SSH and headless machines), crucible logout, and client.login()/logout(); tokens go in the OS keyring, or a private file where none is available (auth_store).
  • Actions that need a recent sign-in, such as creating an API key, offer a browser sign-in in the CLI and retry.
  • whoami and status show whether the client uses an API key or a sign-in, and from where.
  • Named API keys: account.create_api_key(), list_api_keys(), rename_api_key(), and revoke_api_key(), with account keys list|create|rename|revoke; administrators list and revoke single keys with users.list_api_keys(), users.revoke_api_key(), user list-keys, and user revoke-keys --key.
  • account verify shows when the key was last used.
  • Administrators review accounts waiting for approval with users.list(pending=True), users.approve(), and users.reject(), and user list --pending, user approve, and user reject; user get shows a waiting account's status and sign-in methods.
  • hub_url setting (CRUCIBLE_HUB_URL, default https://crucible.lbl.gov/hub); config init creates an API key after a browser sign-in, or points to the Hub.
  • Join requests: access_groups.withdraw_join_request(), join_request_facets(), role on approve_join_request(), ag withdraw, and ag approve --role; lists filter by scope, several statuses, project, reviewer, time, and requester text, and sort (ag list --scope/--project/--search/--sort); requests carry the project, requester, reviewer, granted role, and capabilities.
  • Postprocessing: datasets.postprocess() with an optional idempotency key, client.postprocessing for pipelines and runs, and dataset postprocess, pipelines, list-runs, and get-run.
  • datasets.add_remote_file(replace=True) (and client.replace_existing, dataset create --no-upload --replace) updates a cataloged file whose filename already exists with different content.
  • AssociatedFile.from_local_files() builds hashed catalog records named by their path relative to the files' common folder.
  • Dataset, sample, project, and instrument lists filter by up to 100 MFIDs with mfid; unreadable or missing ones are left out.
  • users.list() takes several unique_id or username values at once.
  • Capabilities: access grants from list_access(), instrument service accounts, and project members carry EntryCapabilities (can_remove, can_change_role, assignable_roles), and resource capabilities declare can_view_access, can_delete, and can_request_deletion.
  • User and project members carry orcid and orcid_verified, and whoami, account show and user get show the linked ORCID separately from the user ID.
  • Shell completion reaches nested commands such as account keys list and fixed-choice arguments such as config get KEY, and leaves out removed and renamed commands, administrator-only user commands you cannot use, and user create and deletion delete for callers who cannot elevate.
  • --json on the list-parents, list-children, list-samples, list-datasets, list-files, list-keywords, list-users, list-projects and list-access-groups commands, file, ingestion, ag and deletion list and get, ag mine, deletion list-deleted and get-deleted, file ingestion, ingestion list-ingestors, whoami, and status.
  • The package ships type information (py.typed) and passes mypy; get() methods are typed as returning None when the API sends no record.
  • The shell says at startup when PyPI has a newer release, with the command to upgrade; update_check (CRUCIBLE_UPDATE_CHECK) turns it off.
  • The shell switches elevation off after 15 idle minutes at the prompt; elevation_timeout (CRUCIBLE_ELEVATION_TIMEOUT) changes it, 0 never.
  • owner_ref filter for up to 10 owners on lists, facets, and search (project and metadata search included), and owner_is_null; --owner is repeatable.
  • affiliation values member (datasets, samples, instruments) and instrument (datasets).
  • The shell opens signed out when no credential works, offering login and config init; a client without a credential raises crucible.NotSignedInError (a ValueError).

Removed

  • Everything deprecated before 4.0.0. Calling a removed method, keyword, or setting raises crucible.RemovedError (a TypeError) naming the replacement, and removed CLI commands and options stop with the one to use; the CLI reference lists them under "Removed in 4.0.0".
  • Removed settings (graph_explorer_url, current_session, upload_protocol, upload_chunk_size_mb) are ignored in old config files; config show lists them and config unset removes them. get_graph_explorer_url() is replaced by get_hub_url().
  • CRUCIBLE_CURRENT_PROJECT stops the CLI; use --project-id.
  • owner_orcid on datasets.list() and samples.list() (use owner_ref), include_datasets on samples.list() (use include_links), and instrument_id/instrument_name in datasets.update() (use assign_instrument()).
  • Parser owner_orcid and instrument_name arguments and the _instrument_name class default; use owner, instrument_id/instrument_mfid, and _instrument_id.
  • dataset update --set public=...; use dataset set-public or set-private.
  • users.get(orcid=) (pass the user reference), projects.list(orcid=) (use member_ref or users.list_projects()), the AccessGrant properties principal, kind and effective_permission (use principal_id, principal_type, permission), the relationship key in get_links() results (use direction), and the dsid key in datasets.create() results (use dataset_mfid).
  • The legacy GCS upload routes, the fallback to them, and the google-cloud-storage dependency.
  • The basic shell used without prompt_toolkit; a broken install now says to reinstall.
  • Instrument set_access(), revoke_access(), and access grant|revoke; use the member methods and commands.
  • datasets.request_carrier_segmentation(), request_insitu_aggregation(), request_rga_analysis(), and request_mosaic_stitch(); use datasets.postprocess().
  • users.verify_api_key(); read api_key_status from users.get().

Changed

  • Instrument members work like project members: every member reads the instrument's datasets, member methods return ProjectMember lists, and Instrument.members holds ProjectMember records.
  • Instrument add_user() and add-user accept viewer and default to it instead of editor.
  • instrument list --mine includes every instrument you are a member of.
  • Creates send owner_ref and transfers send new_owner_ref; the owner_id and owner_id_is_null filters are deprecated.
  • Projects take owner on create (project create --owner); project_lead* and --lead are deprecated.
  • Project lists and project get show the owner instead of the lead.
  • --group-by instrument, --group-by project, and the download cache read the nested references.
  • Signed-in administrators run normal by default, while API keys still run elevated; is_elevated, whoami, and the shell badge follow the mode the API reports.
  • Python 3.11 or later is required; keyring is a new dependency.
  • API errors raise crucible.CrucibleAPIError, a requests.HTTPError with status, code, detail, and request_id; CLI errors show the request ID.
  • POST and PATCH requests without an Idempotency-Key are no longer retried after a 502, 503, 504, or lost response, which could duplicate records; 429 and failed connections are still retried.
  • Requests send a nano-crucible/<version> User-Agent.
  • Calls to routes the API marks deprecated raise a DeprecationWarning once per route, pointing at the calling code, with the date, sunset, and link the API gives.
  • Uploads always use signed multipart, including empty files; upload_max_workers sets how many parts upload at once, and a server without uploads enabled says so.
  • Resource, project, instrument, and user links, crucible open, and Alt+O go to the Crucible Hub (hub_url) instead of the Graph Explorer.
  • The shell starts faster, says why when it cannot reach the API, and its summary shows the API's name, version, and how you are signed in. The status bar names the API (production, staging, or its host), shows the API version only for pre-release or older servers, shows whether you use a key or a sign-in, adapts to narrow terminals, and no longer shows a clock.
  • In the shell, help lists commands compactly and help COMMAND [SUBCOMMAND] shows a command's options, slow commands print how long they took, completion searches wait for a pause in typing, and creating or updating a project refreshes project completion.
  • Renamed, with the old names still working and warning: datasets.list_keywords(), projects.list_users(), instruments.list_users(), users.list_projects(), and the parameters file_mfid (files), resource_mfid (deletions), service_account_ref, instrument_ref, and user_ref (users and project members). ingestions.list() takes dataset_mfid and file_mfid; dsid and file_id still work.
  • Renamed CLI commands, with the old names still working and warning: dataset link-sample, unlink-sample, unlink -p -c; sample link-dataset, unlink-dataset, unlink -p -c; dataset ingestors is now ingestion list-ingestors, which takes --filter.
  • datasets.graph() and samples.graph() still work but are deprecated in favour of graphs.get().
  • samples.get() and client.get() no longer embed a sample's datasets; include_datasets=True still works with a warning (use include_links or datasets.list(sample_mfid=...)), and sample get --json includes links instead.
  • deletion delete and sa rotate-key ask for confirmation (-y/--yes skips it), and -y works wherever --yes does.
  • Join requests use the project routes: request_join() posts to /projects/{ref}/join_requests, access_groups.get() reads one request directly and works for requesters and reviewers, and account.join_requests() lists the submitted scope; ag list shows the requests you may review, including project editors' queues, and the shell's pending count follows it.
  • User references are no longer shape-checked: users.get() and the CLI pass any value they cannot classify to the API.
  • users.resolve() uses the exact filters of GET /users instead of the deprecated POST /users/resolve; its result is unchanged.
  • users.search() reads the paginated items response.
  • datasets.create() leaves out instrument_name when instrument_id or instrument_mfid is set.
  • A cast recipe's instrument is an instrument ID or MFID, not a name.
  • CI tests Python 3.13 and macOS, runs mypy, checks the client's routes against staging, and gates PyPI releases on the tests; the live-API integration tests moved to the crucible-tests repository.

Fixed

  • skip_ingestion=True uploads record ingestion_class instead of dropping it.
  • Datasets whose files have an absolute legacy filename, such as /mnt/gcs/api-uploads/scan.h5, download instead of being skipped.
  • datasets.create(upload_files=False) and dataset create --no-upload send each file's size and SHA-256, which the server requires, and name files by relative path so files sharing a basename in different folders stay separate.
  • add_remote_file() sets storage_path in the create request, and updates it when re-adding an existing file at a new location.
  • Creating a dataset, sample, project, or instrument with scientific metadata on someone else's behalf no longer fails with 403 for platform contributors.
  • The CLI shows UTC timestamps ending in Z in local time, and table dates use the local day.
  • project add-user accepts user and service-account MFIDs and any other user ID instead of failing with an invalid ORCID error.
  • Downloads no longer send the API key to the storage host serving a signed URL.
  • A refusal that needs a recent Hub sign-in (hub_session_required) points to the Hub instead of suggesting --elevate.
  • dataset ingestion and file ingestion show the file MFID and creation time.
  • deletion and ag views show usernames again.
  • sa create --unique-id keeps the given MFID when it prompts for the username.
  • Upload progress no longer crashes when a part of a finished file is retried.
  • crucible.cast imports on Windows.

3.3.0

Added

  • User administration: users.set_platform_role(), users.revoke_api_keys(), user set-role, and user revoke-keys.
  • users.list() and user list filter by text, name, role, and account type, and sort.
  • Project and instrument lists sort with sort/direction (--sort/--direction).
  • Project lists filter by member with member_ref/member_role (--member/--member-role).
  • Project and instrument lists and details show the caller's role.
  • user list-projects shows the listed user's role and filters by --role.
  • Dataset and sample lists filter by visibility, owner, creation or modification time, missing fields, and linked records, and sort.
  • datasets.facets(), samples.facets(), dataset facets, and sample facets count records per field value.
  • Dataset lists, counts, and facets filter by instrument slug (instrument_id, --instrument-id).
  • datasets.assign_instrument() sets a dataset's instrument.
  • Instrument maintainers: instruments.get_users(), add_user(), remove_user(), update_user_role(), and the matching instrument commands.
  • Instrument lists filter by your role (affiliation, --mine) and by instrument fields.
  • instrument update --instrument-id renames an instrument.
  • Service-account administration: service_accounts.get_admin(), list_admin(), set_platform_role(), and service-account set-role.
  • Deletion request and audit lists filter by scope and project, and sort.
  • deletion list shows reviewers their review queue by default.
  • Privilege mode: the privilege_mode keyword and config key, and the --elevate/-E flag. Administrators still run elevated by default; this becomes normal in a future release.
  • Shell: elevate on|off, elevate COMMAND, and an ELEVATED toolbar badge.
  • client.authorization, client.capabilities, client.can_elevate, and client.refresh_profile().
  • crucible whoami shows your platform role, privilege mode, and capabilities.
  • print.barcode() and crucible print barcode print sample labels.
  • Uploads use the API's signed multipart flow, chosen with upload_protocol (auto, signed, legacy) or --upload-protocol.
  • Upload progress: a progress callback, client.upload_progress, and a progress line in the CLI.
  • --no-ingest on dataset create and dataset add-file uploads without requesting ingestion.
  • files.replace() and crucible file replace replace a file's content, keeping its MFID.
  • add_file(replace=True) and --replace replace a same-named file with different content.
  • sample create and dataset create accept --owner USER.
  • Download links accept disposition="inline".
  • List methods return a ListResult with has_more, total, and a continuation, and accept cursor=.
  • Shell: more shows the next page of the last listing.
  • An optional pager for long listings (config set pager auto, --no-pager).
  • "Did you mean ...?" suggestions for mistyped commands and flags.
  • Next-step hints on common API errors.
  • Shell completion of facet values, facet fields, --instrument-id, and MFID flags.
  • Fuzzy shell completion of project IDs and facet values.
  • --ingestor completes from the API's ingestion classes.
  • The shell lists pending deletion reviews and join requests at startup.
  • Creating projects, instruments, and service accounts fails early without the capability.

Changed

  • List commands show 50 records by default (was 100), accept --all, and report how many were left out.
  • dataset create prompts for missing fields on a terminal instead of creating an unnamed dataset.
  • dataset list and sample list print oldest to newest and follow --sort.
  • dataset list and sample list without a project list every accessible project.
  • The shell opens with a session summary beside the Crucible mark.
  • Shell colors keep the brand blue on 256- and 16-color terminals.
  • The support platform role is removed.
  • service-account list and get show platform roles and key status to administrators; show is an alias for get.
  • Identifying instruments by display name is deprecated; use --instrument-id or --instrument-mfid.
  • Instrument operations accept a slug as well as an MFID.
  • reassign_project() accepts project_mfid and reports MFIDs.
  • The shell hides service-account commands from callers who cannot use them.
  • Unknown dataset and sample list filters raise a local ValueError.
  • Python 3.10 or newer is required.
  • The configuration file is readable only by its owner.
  • ipywidgets and python-dotenv are no longer dependencies.

Fixed

  • Shell reload works when started with python -m crucible.cli and on Windows.
  • edit commands work on Windows without configuration.
  • Project ownership, access, and metadata commands accept a project ID.
  • Renaming a project updates the saved current project.
  • Waiting for ingestion returns the request's final state.
  • Offset-paged listings no longer stop at 1000 records.
  • Files from multipart uploads show and download under their filename.
  • Downloads refuse paths outside the target folder.
  • dataset download warns about files it cannot fetch.
  • Storage errors explain what happened.
  • The API reference documents shared ownership, access, and project methods.
  • The shell history no longer stores API keys.

3.2.1

Added

  • Dataset and sample parent-child links support optional is_derived_from and is_part_of relationship types in Python and the CLI, including relationship-type filtering.
  • The interactive shell displays a centered 16x16 Crucible mark with square-proportioned half-block pixels on supported terminals.
  • Dataset and sample creation accept canonical project MFIDs alongside matching project IDs, and dataset creation accepts canonical instrument MFIDs alongside matching instrument IDs.
  • Dataset and sample lists accept a project ID or canonical project MFID with assigned, shared, or combined project scope in the Python client and CLI.
  • Dataset thumbnails can be added from the CLI and renamed or replaced with client.datasets.update_thumbnail().
  • Dataset and sample responses expose typed project references with canonical MFIDs, and dataset responses also expose typed instrument references, while retaining legacy flat fields.
  • The CLI supports --no-color and the standard NO_COLOR environment variable.
  • Instrument workflows support lifecycle status changes, bound service-account listing, and dataset filtering by canonical instrument MFID.
  • Exact dataset, sample, project, and instrument responses expose typed caller-specific resource capabilities when supplied by the API.

Changed

  • Entries returned by client.get_links() expose the API's direction field with source, target, or undirected values. The legacy relationship field remains as a compatibility alias with its previous parent, child, or associated value.
  • The minimum supported Python version is now 3.9.
  • Dataset thumbnail create, list, and update responses expose the API-provided authoritative MIME type.
  • Instrument search results now show the user-facing instrument ID instead of manufacturer metadata.
  • crucible dataset create can create a dataset record without --input; file-dependent options still require at least one input file.
  • Dataset and sample relationship methods now consistently use link, unlink, link_sample, unlink_sample, link_dataset, and unlink_dataset; previous method names remain as deprecated wrappers.
  • Public visibility now uses set_public() and set_private() in Python and set-public and set-private in the CLI; previous method names, command names, and update visibility options remain deprecated compatibility aliases that delegate to the dedicated access routes.
  • The interactive shell now remembers the project selected by use PROJECT_ID, unuse clears it, and project context sources are visible; CRUCIBLE_CURRENT_PROJECT is deprecated because it can silently redirect operations.
  • The interactive shell status bar now renders the Crucible dark blue, light blue, and orange brand palette in true color on capable terminals while keeping autocomplete menus visually plain.
  • Sample detail methods can suppress the deprecated embedded dataset expansion; existing Python and JSON behavior remains compatible, while human CLI workflows avoid loading it.
  • CLI detail views group related fields, use consistent semantic colors, link resources and users to Crucible Explorer, distinguish slugs from MFIDs, and abbreviate given names.
  • Generic crucible get MFID now renders projects in addition to datasets, samples, and instruments.
  • CLI project and sample-type options use consistent canonical flags, and interactive completion now discovers flags and searches resource endpoints for identifier values.
  • CLI failures now preserve HTTP status codes while formatting API validation details and warnings for readable terminal and JSON output.
  • Human and service-account creation normalize usernames and validate the API's complete username rules, with immediate retry prompts during interactive creation.
  • Project membership methods and commands validate named roles before requests, document strict lower-role management and owner-or-self removal, and project add-user reports API errors through the shared status-preserving formatter.
  • Interactive creation and configuration prompts now share required, optional, default, secret, and choice handling; invalid values prompt again, API keys remain hidden, and missing required input fails clearly outside a terminal.
  • Destructive CLI confirmations now use consistent yes-or-no handling, default to cancellation, and require --yes when no interactive terminal is available.
  • CLI tables now adapt to terminal width while preserving full usernames, resource slugs, and MFIDs whenever space permits.
  • Project listing, resource searches, user discovery, and service-account retrieval now support raw JSON output with structured JSON validation errors.
  • Empty CLI listings identify their resource type, and mutations use consistent success messages with text fallbacks for redirected output.
  • crucible status now shows the configured endpoint, readiness state, client and API versions, deployed branch and commit, database latency and schema revisions, and authenticated identity.
  • Instrument search accepts an optional lifecycle-status filter in the Python client and CLI.
  • Project member tables use role-priority ordering and semantic role colors, display the owner as the project lead, and the interactive shell status bar again identifies project, user, and API context with symbols.
  • Dataset and sample relationship listing now uses canonical collection filters with cursor pagination instead of deprecated nested read routes.

Fixed

  • crucible file delete now requires interactive confirmation or an explicit --yes before permanently deleting a file.
  • Dataset file guidance now documents the API v3 record-first workflow, recursive uploads, additions, deletion-based replacement, and re-uploading.
  • Documentation navigation tabs now remain visible while scrolling using MkDocs Material's native sticky-tabs behavior.
  • Instrument creation no longer sends inherited response-only fields rejected by API v3.

3.2.0

Added

  • add_file(skip_ingestion=True) uploads a file without requesting ingestion. Updates ingestion request table with provenance details for local parsing.
  • Dataset, sample, and project lists accept repeated user and project access selectors.
  • Development skills for API, CLI, parser, and cast changes.
  • Agent-agnostic contributor guidance and a skill for safe client workflows.
  • New file request-ingestion CLI command to (re)request ingestion for a cataloged file.
  • New client.datasets.add_remote_file() and client.files.update() to catalog files that live outside GCS (Globus, NERSC, a shared filesystem) without uploading them.
  • dataset create --no-upload/--backend/--access-note to catalog files by path instead of uploading them.
  • New ag get command and client.access_groups.get().
  • Shell autocomplete for ag request IDs.
  • Shell autocomplete extended to more user/project/sa/ag commands.
  • Shell autocomplete for instrument get and --project/-pid flags.
  • Shell autocomplete for dataset/sample IDs across most subcommands.
  • Shell autocomplete for user search shows live matches while typing.
  • New dataset/sample/project reassign-project and transfer-ownership commands.
  • New dataset/sample/project access list/grant/revoke and publish/unpublish commands.
  • New project update-user-role command and add-user --role flag.
  • project list-users now shows each member's role.
  • instrument create requires a new --instrument-id flag (a unique slug, separate from its ID and name).
  • project get --include-members shows the project's member list and roles; --members remains as a deprecated alias.
  • project create/client.projects.create() accept a flexible project_lead field as an alternative to the explicit ORCID/email/username fields.

Changed

  • The built-in production endpoint now targets API v3; explicit v1/v2 overrides display a migration warning, and config unset api_url restores the package default.
  • Instrument retrieval expands typed public owner records by default; instrument lists support owner expansion and lifecycle-status filtering.
  • Instrument creation defaults ownership to the authenticated identity, while ownership changes use transfer_ownership() and instrument transfer-ownership instead of update fields.
  • Human users may be created without an ORCID, receiving a server-assigned MFID that user and owner displays treat as a canonical user ID rather than an ORCID.
  • User operations that require a canonical identity use user_unique_id; the previous orcid keyword remains temporarily supported with a deprecation warning.
  • Health checks and crucible status accept deployment provenance from the nested API readiness response while remaining compatible with the legacy flat response during rollout.
  • Singleton dataset and sample retrieval expands typed public owner records by default; owner_orcid creation inputs are deprecated in favor of flexible owner identifiers.
  • Project membership mutations resolve usernames and emails before using canonical user identifiers; the old orcid keyword remains temporarily supported.
  • Generic access-group mutation helpers and CLI commands are deprecated in favor of typed resource, project, and instrument operations.
  • Exact user lookups show email to self and platform administrators and omit the email row when it is not disclosed; other user, project-lead, member, and operator views remain public-safe.
  • user list-datasets now uses the canonical paginated dataset collection and supports --limit.
  • client.users.check_dataset_access() and user check-access now report the canonical effective access role.
  • MFID-only parameters now use role-specific _mfid names, including parent_mfid and child_mfid; project and instrument slugs retain _id, and previous keywords remain temporarily supported.
  • Resource lookups now dispatch canonical MFIDs to single-resource routes and resolve project slugs, instrument slugs, usernames, and emails through exact collection filters without a second request. Returned records retain unique_id as their canonical identifier.
  • README now focuses on installation, navigation, and project essentials.
  • CLI reference now documents all command families from one canonical location.
  • dataset get/dataset list-files now show each file's MFID alongside its name.
  • dataset get shows a dataset's files by default; no longer requires --verbose.
  • datasets.create()'s files_to_upload renamed to files (accepts a mix of local paths and AssociatedFile objects); old name still works with a deprecation warning.
  • ag list/ag mine now default to pending requests, matching deletion list.
  • deletion list and ag list/ag mine share one --status flag.
  • ag approve/ag reject accept multiple request IDs.
  • Shell username autocomplete now works for non-admins and completes to username.
  • Broken third-party parsers now log a warning instead of silently disappearing.
  • project update no longer accepts --lead-email/--lead-orcid; use project transfer-ownership instead.
  • project update --project-id now renames the project.
  • sample update no longer accepts --project/--owner; use sample reassign-project/transfer-ownership instead.
  • dataset update/sample update --set no longer accept project_id/owner_orcid; use the new reassign/transfer commands instead.
  • Generic access grants accept roles up to admin; ownership changes use transfer-ownership.
  • Dataset update fields now match the supported mutation contract: data_format is editable, while unsupported description and frozen instrument-assignment fields are excluded.
  • Dataset, sample, and project operations expose only the access-control, ownership, and project-assignment capabilities supported by their API contracts.
  • Project member add, role-update, and removal methods consistently return list[ProjectMember].

Fixed

  • CLI tables display usernames up to their full 24-character limit and project or instrument slugs up to their full 25-character limit.
  • Instrument CLI get/list output formats expanded owners correctly and instrument list --include-metadata --json exposes requested metadata.
  • Project and instrument lookup remains compatible with legacy slugs outside the current creation limits.
  • Access-control operations now use the API's canonical principal and permission fields.
  • Paginated list operations now request only the number of records needed to satisfy limit instead of over-fetching full server pages.
  • Quick-start examples now use current dataset creation return values.
  • Download guides now use current namespaced methods and overwrite options.
  • files.download() crashed with a raw exception on a non-GCS file instead of a clear error.
  • AssociatedFile silently dropped storage_backend/access_note fields returned by the API.
  • cast: uploading files in a recipe always crashed.
  • cast: resuming a recipe after a partial failure always crashed.
  • dataset create/parser-based uploads forced generic ingestion instead of letting the server auto-detect.
  • Parser-based uploads sent a stale, unused field to the server.
  • Users without a username disappeared from shell autocomplete; they now show by ORCID.
  • Some edit commands silently ignored --debug.
  • Docs and the tutorial notebook referenced removed/renamed methods and wrong parameter names in several places.

3.1.0

  • projects.get()/search() no longer require membership; lead/scientific_metadata are membership-gated.
  • New client.access_groups resource: request_join(), list_join_requests(), approve_join_request(), reject_join_request() (requires the pending feat/access-group-join-requests API branch).
  • client.projects.request_join()/list_join_requests() delegate to access_groups.
  • client.account.join_requests() lists the caller's own join-request history.
  • New crucible access-group/ag CLI: request, list, mine, approve, reject.
  • New crucible project request-join and crucible project list-join-requests.
  • Fix access_groups.request_join(): always send a JSON body (API requires it even with no reason).
  • Join request and deletion request lists now show usernames instead of raw ORCIDs, drop the reason column, and show dates as YYYY-MM-DD.
  • New shared CLI helpers: helpers.resolve_usernames(), term.fmt_date(), term.status_label(), term.fmt_name() (de-duplication pass, no behavior change).
  • CLI: unified user identification. Most user subcommands and project add-user/remove-user now accept ORCID, username, or email directly. Old identifier flags are deprecated but still work.
  • CLI: unified service account identification. sa subcommands accept MFID or username directly. Old flags deprecated but still work.
  • New client.service_accounts.list_access_groups()/add_to_access_group()/remove_from_access_group() and matching sa CLI commands.
  • Fix deletion approve/deletion reject: batch commands now exit 1 if any request fails.
  • Fix dataset add-access-group: removed the decorative --read flag (read access is always granted; --write adds write).
  • search_metadata(): response unwrapped from the paginated envelope; default limit now 20; results include resource_type, name, owner_orcid, timestamps, rank.
  • include_owner=True on datasets/samples get()/list() and client.get() resolves the owner into a public-safe user object.
  • client.files.delete() and crucible file delete to delete a file by MFID.
  • samples.create() now takes a Sample model as its first argument, consistent with datasets.create().
  • crucible project list-users shows usernames and correctly reads unique_id as the ORCID.
  • Default graph explorer URL updated to https://crucible.lbl.gov/explore.

3.0.1

  • Fix datasets.download(): data files now saved inside output_dir/{dsid}/ alongside record.json, not directly in output_dir/.

3.0.0

This is a major release. The flat client.* API that was deprecated in 2.x has been removed. All operations now live under typed resource namespaces.

Breaking changes

Before upgrading, update any code that uses the old flat API:

Old (removed) New
client.get_dataset(dsid) client.datasets.get(dsid)
client.create_new_dataset(...) client.datasets.create(...)
client.list_datasets(...) client.datasets.list(...)
client.get_sample(sid) client.samples.get(sid)
client.add_sample(...) client.samples.create(...)
client.list_samples(...) client.samples.list(...)
client.get_project(pid) client.projects.get(pid)
client.get_user(orcid=...) client.users.get(orcid=...)
client.download(resource_id) client.datasets.download(dsid) or client.samples.download(sid)
pip install nano-crucible[shell] pip install nano-crucible (now core)
pip install nano-crucible[gcs] pip install nano-crucible (now core)
pip install nano-crucible[all] pip install nano-crucible

New resource namespaces

  • client.account — self-service profile management (profile(), update_profile(), api_key(), verify(), whoami()). No admin required.
  • client.ingestions — ingestion request management (list(), get(), wait(), update()).
  • client.files is now a proper peer resource (previously a base class for DatasetOperations). Scoped to file MFIDs: get(), list(), download(), get_download_link(), request_ingestion().

Architecture changes

  • DatasetOperations no longer inherits from FileOperations. All dataset-scoped file operations (add_file, list_files, get_download_links, download, thumbnails, ingestion) now live directly on DatasetOperations.
  • Upload logic extracted to crucible/resources/gcs/upload.py — standalone functions testable in isolation.
  • Download logic extracted to crucible/resources/gcs/download.py — parallel file downloads via ThreadPoolExecutor.
  • Scientific metadata methods (search_metadata, get_scientific_metadata, update_scientific_metadata, get_access_groups, add_access_group) unified on BaseResource — available on all resource types.
  • search_scientific_metadata() renamed to search_metadata() (deprecated alias kept).

New features

  • Parallel GCS multipart upload — add_file() uses transfer_manager.upload_chunks_concurrently() by default (8 workers, 64 MiB chunks, benchmarked). Falls back to sequential resumable upload with multipart=False. Upload settings configurable via crucible config set upload_chunk_size_mb and upload_max_workers.
  • Sequential per-dataset downloads — files within a dataset download sequentially; to parallelise across datasets, wrap datasets.download() in a ThreadPoolExecutor at the caller level.
  • Fuzzy name search — datasets.search(q), samples.search(q), projects.search(q), instruments.search(q) via new API endpoints. Typo-tolerant, returns top-N by relevance.
  • Scientific metadata search renamed — search_metadata(q) on all resources.
  • Username support — users.get(username=...), users.search(q), user set-username, project add-user -u USERNAME.
  • samples.download(sid) — saves sample record as record.json.
  • files.download(file_id) — single-file download by MFID.
  • ingestions.wait(request_id) — public polling method.

Packaging

  • google-cloud-storage>=2.7.0 and prompt_toolkit>=3.0 moved to core dependencies.
  • [shell], [gcs], and [all] extras removed — pip install nano-crucible installs everything.
  • [parsers] remains optional (ASE/LAMMPS support).

CLI additions

  • crucible account — show, edit, update, api-key, verify
  • crucible ingestion — list, get, wait
  • crucible user search TERM — non-admin user lookup
  • crucible {dataset,sample,project,instrument} search TERM — fuzzy name search
  • crucible {dataset,sample,project,instrument} search-metadata TERM (also search-md)
  • crucible dataset list-access-groups, add-access-group
  • crucible user add-access-group, remove-access-group
  • crucible deletion list-deleted, get-deleted, delete --force
  • --json flag on all get and list commands (replaces --output json)
  • sample update — named flags (-n, -t, --description, --project, --public)

Deprecations (still work, emit warnings)

  • client.download() → client.datasets.download() or client.samples.download()
  • datasets.get_associated_files() → datasets.list_files()
  • datasets.search_scientific_metadata() → datasets.search_metadata()
  • datasets.add_file_to_dataset() → datasets.add_file()
  • datasets.graph() → client.graphs.get()
  • datasets.get_ingestion_requests() → client.ingestions.list()
  • datasets.get_request_status() → client.ingestions.get()
  • datasets.update_ingestion_status() → client.ingestions.update()
  • users.me() → client.account.profile()
  • users.get_api_key() → client.account.api_key()

2.1.0 → 2.1.2

Although versioned as a patch series, this span (late March to mid-May) is effectively a feature release. The notes below focus on what affects code you write against the client.

Likely breaking changes

Most users only need to address three things:

  • User responses now key identity as unique_id instead of orcid (e.g. user["orcid"] → user["unique_id"]).
  • File methods now live under client.files (backward-compat shims on client.* still work).
  • add_metadata was renamed to replace_scientific_metadata.

New resource namespaces

  • client.files — file operations moved here from datasets: chunked GCS upload, plus listing, inspecting, and downloading files by dataset_mfid or SHA-256 hash.
  • client.deletions — deletion-request workflow (request / approve / reject).
  • client.graphs — graph queries.

New top-level client methods

  • client.live() / client.health() — unauthenticated health checks.
  • client.search_scientific_metadata(q) — full-text search across scientific metadata.
  • client.get_links(resource_id) — immediate links for any resource.

Moved or renamed methods

  • File methods moved datasets → files: add_associated_file, get_associated_files, upload_file, and download helpers. Backward-compat shims on client.* still work.
  • Scientific-metadata methods moved to the shared base resource — now available on datasets, samples, projects, and instruments.
  • add_metadata → replace_scientific_metadata (rename). Use update_scientific_metadata(..., overwrite=False) to merge (PATCH) or overwrite=True to replace (POST).
  • users: removed get_or_create and get_user_from_api; added update(), get_api_key(), and remove_from_access_group().
  • projects: added update(), remove_user(), add_scientific_metadata(), and update_scientific_metadata().
  • instruments: added update(), add_scientific_metadata(), and update_scientific_metadata().
  • samples / datasets: added count() and graph().

Model field changes

  • User: orcid field renamed to unique_id (with a .orcid property alias for backward compatibility); removed lbl_email and employee_number; added is_service_account.
  • Sample: added public, scientific_metadata, datasets, links, deletion_request, resource_type; removed owner_user_id.
  • Dataset: added data_type, scientific_metadata, links, deletion_request, resource_type; removed owner_user_id and instrument_id; dropped file_to_upload and sha256_hash_file_to_upload.
  • Project: project_lead_email is now optional (previously required); added project_lead_orcid, lead, scientific_metadata, and timestamp fields.
  • New DeletionRequest model.

Behavior changes

  • projects.add_user() / remove_user() accept an email directly — no more client-side ORCID resolution.
  • Chunked GCS uploads: 32 MiB chunks with per-chunk CRC32C and incremental SHA-256, verified at /complete.
  • Fixed a hashing bug affecting larger files (the actual 2.1.1 → 2.1.2 change).
  • Public-sample support (sample create --public).

Packaging

  • New required dependencies: configupdater, pyyaml, google-crc32c.
  • New optional extras: [shell] (prompt_toolkit) and [docs] (mkdocs).

CLI

New subcommands: crucible file, crucible deletion, crucible status, crucible qr, crucible tree, and crucible cast, plus an interactive shell when you run crucible with no arguments.