AI-Ready TEM, STEM, and SEM Workflows
AI can interpret the image. Experimental metadata helps it understand the experiment, guide physical action, and verify the result.
Automation starts with structured experimental context.
AI-ready in situ microscopy connects microscope measurements with experimental metadata, hardware state, sample-environment conditions, precision motion, timing, calibration, safety information, and human oversight. This context can allow OEM or customer-selected software to interpret the experiment, recommend or direct an action, and verify the physical response.
Colibri Platform™ supports this direction through Hummingbird Control™ for controlling and monitoring Hummingbird hardware, Hummingbird Connect™ for supported and project-specific software integrations, and an open approach to using experimental data across OEM software, open-source tools, institutional systems, Python workflows, and future AI environments.
From microscope measurements to operational metadata
A microscope image can show that a sample changed. By itself, it may not explain what caused the change, which conditions preceded it, where it occurred, or whether a commanded action produced the intended result.
Experimental metadata connects the measurement with the physical state and history of the experiment. Relevant information may include:
- Sample, chip, holder, controller, microscope, detector, and session identity
- Environmental and electrical conditions
- Commanded setpoints and measured values
- Position, motion, tilt, coordinates, and region of interest
- Timing, experimental events, calibration, and software versions
- Hardware status, alarms, interlocks, and errors
- User interventions, approvals, workflow history, and uncertainty
The microscope and OEM acquisition environment collect images, spectra, diffraction patterns, 4D-STEM datasets, and other measurements.
Hummingbird Control™ controls and monitors supported Hummingbird holders, stages, controllers, heating systems, gas systems, biasing platforms, nanomanipulators, and other specialized hardware acting on the sample. Depending on the product and configuration, the software may also display, record, log, or export selected commanded values, measured values, timestamps, status information, alarms, or errors.
Hummingbird Connect™ supports verified and project-specific pathways for connecting relevant Hummingbird controls or experimental-state information with OEM, facility, and customer-selected software environments.
For AI-ready workflows, metadata is not only documentation created after the experiment. It can become operational information that software uses to understand the current state, select an appropriate action, and evaluate the outcome.
How metadata supports closed-loop microscopy
Closed-loop microscopy requires more than a sequence of software commands. The workflow must be able to observe the experiment, interpret its state, decide what to do, act through physical hardware, verify the result, and preserve what occurred.
Observe
The microscope and detector collect measurement data. Hummingbird Control™ monitors supported Hummingbird hardware and the physical conditions applied to the sample.
Metadata identifies the sample, equipment, position, timing, and experimental state associated with each measurement.
Understand
OEM, institutional, open-source, or customer-developed software evaluates the measurement together with available experimental context.
Metadata helps connect an observed transformation with the conditions that may have caused it. Software can evaluate what changed before the event, which stimulus was active, where the feature was located, and which calibration was in use.
Decide
The software may recommend the next measurement, position, tilt, detector condition, or change in the sample environment.
A reliable decision may depend on measured values, prior actions, calibration, uncertainty, operating limits, interlocks, and the history of the region being studied.
The researcher defines the scientific objective, operating boundaries, approval requirements, and acceptable uncertainty.
Act
The microscope software operates the microscope and acquisition environment.
Hummingbird Control™ operates supported Hummingbird hardware, which may change temperature, pressure, gas conditions, liquid flow, electrical conditions, position, tilt, or another experimental variable.
Hummingbird Connect™ can support the verified or project-specific pathway required for these systems to participate in a coordinated workflow.
The command becomes part of the experiment history, including which device was addressed, which value was requested, when it occurred, and which workflow decision produced it.
Verify
A closed-loop workflow must determine whether the requested physical action occurred and whether the resulting measurement matches the intended outcome.
Verification may depend on command acknowledgement, measured hardware response, operating status, accurate timing, position confirmation, re-imaging, or repeated measurement.
Without verification metadata, software may know that it issued a command without knowing whether the physical experiment responded as intended.
Preserve and learn
Available microscope data, controller information, user decisions, workflow history, and model information may be retained in the customer-selected software or data environment.
This provenance allows researchers to reconstruct what happened, review why an action was taken, compare experiments, evaluate uncertainty, and improve later workflows.
Connecting metadata to physical action
Motion, coordinates, and return-to-location
A model may identify a defect, particle, interface, lamella, or region of interest. The experiment cannot close the loop unless the physical system can move to that location, return to it, change orientation, and determine whether the commanded motion occurred as intended.
AI-ready motion may depend on:
- Sample or specimen coordinates
- Commanded and measured position
- Tilt and orientation
- Calibration and motion uncertainty
- Region-of-interest identity
- Return-to-location
- Motion status and command acknowledgement
- Safe operating limits and collision constraints
Persistent position and region-of-interest information can support tomography, multimodal correlation, repeated measurements, and cross-session workflows.
Specific motion, sensing, coordinate, and software capabilities must be evaluated for each Hummingbird product and project.
Sample-environment conditions and measured response
In situ microscopy may require software to change the conditions driving the experiment, not only the position being observed.
Depending on the supported Hummingbird hardware, those conditions may include heating, cooling, gas composition, pressure, liquid flow, electrochemical potential, voltage, current, electrical biasing, mechanical interaction, or manipulator motion.
For automation, the setpoint alone is not enough. The workflow may also need the measured response, current operating state, timing, calibration, active limits, and any alarms or manual interventions.
Hummingbird Control™ provides product-specific operation and monitoring of supported Hummingbird hardware. Supported or project-specific Hummingbird Connect™ integrations may allow selected controls, measurements, status information, or events to participate in a wider workflow.
The customer or OEM environment can remain responsible for the automation logic, active-learning method, analysis process, or AI model directing the experiment.
Roles within a human-in-the-loop workflow
AI-ready microscopy does not require removing the scientist from the experiment. Human oversight remains essential for defining objectives, evaluating uncertainty, setting physical and safety limits, and validating scientific conclusions.
The scientist defines the hypothesis, desired outcome, operating constraints, approval requirements, and acceptable uncertainty.
OEM software operates the microscope and acquisition environment and collects images, spectra, diffraction patterns, or other measurement data.
Hummingbird Control™ controls and monitors the supported Hummingbird holder, stage, controller, or sample environment.
Hummingbird Connect™ supports the defined integration pathway between Hummingbird systems and selected software environments.
Customer-selected automation or AI evaluates available measurements and metadata, recommends actions, or directs approved parts of the workflow.
This structure allows each part of the system to perform its appropriate role without forcing the complete experiment into one proprietary AI application.
Current capabilities, integration pathways, and future direction
Currently available
Hummingbird Control™ provides product-specific software for controlling and monitoring supported Hummingbird hardware and the physical in situ experiment.
Available functions vary by product and may include parameter control, measured-value display, operating-status monitoring, timestamps, alarms, errors, data logging, or export functions.
Verified capabilities are identified on the relevant Hummingbird Control™ and product pages.
Supported or project-specific
Hummingbird Connect™ supports evaluated integration pathways between Hummingbird systems and OEM, facility, or customer-selected software environments.
A project may involve selected controls, commands, status information, timestamps, experimental-state data, events, or data exchange.
Requirements depend on the microscope, model, OEM software, software version, Hummingbird hardware, available interfaces, safety requirements, and intended outcome.
Future direction
More advanced human-in-the-loop, AI-guided, and autonomous microscopy may require:
- Structured, machine-readable experimental metadata
- Shared timing and synchronized events
- Persistent sample and region-of-interest identity
- Machine-readable commands and status
- Measured physical response
- Quantified position and motion uncertainty
- Safe limits and automated interlocks
- Human approval gates
- Closed-loop verification
- Model, decision, and workflow provenance
Colibri Platform™ is designed to provide a foundation for developing these capabilities without requiring the customer’s acquisition, analysis, automation, AI, or institutional data environment to be replaced by a proprietary Hummingbird system.
AI-ready does not mean that Colibri Platform™ currently provides a complete autonomous microscopy system.
AI-ready workflows in customer-selected environments
Colibri Platform™ is designed so available experimental metadata, hardware control, and supported integration pathways can participate in the OEM, institutional, open-source, Python, and customer-developed environments selected for the workflow.
The microscope and OEM environment remain central to instrument operation and acquisition. Hummingbird provides the physical-experiment control, experimental state, and integration capabilities that can support AI-ready microscopy.
Metadata makes the experiment understandable. Control and motion make it actionable. Verification makes the result trustworthy.
Explore how these capabilities fit into a broader vision for connected instruments, open interfaces, scientific data infrastructure, and customer-owned microscopy workflows.
Frequently asked questions
Discuss an AI-ready microscopy project
Tell us which microscope, Hummingbird system, software environment, experimental variables, metadata requirements, and scientific outcome you are planning.
Hummingbird can help evaluate available Hummingbird Control™ capabilities, experimental-state information, logging and export requirements, supported or project-specific Hummingbird Connect™ pathways, motion requirements, timing requirements, safety considerations, and any coordination needed with the microscope manufacturer.
Related Colibri Platform™ Pages
Explore how Colibri supports open, structured, and portable in-situ TEM data workflows:
- Colibri Platform™
- Hummingbird Control™ Software
- Hummingbird Connect™ OEM Integration
- In-Situ Experiment Data Management
- LIMS and Institutional Data
- AI Ready TEM & SEM Workflows
- The Future of Open Microscopy