A fluorescence microscopy inspection system includes light sources able to emit light that causes a specimen to fluoresce and light that does not cause a specimen to fluoresce. The emitted light is directed through one or more filters and objective channels towards a specimen. A ring of lights projects light at the specimen at an oblique angle through a darkfield channel. One of the filters may modify the light to match a predetermined bandgap energy associated with the specimen and another filter may filter wavelengths of light reflected from the specimen and to a camera. The camera may produce an image from the received light and specimen classification and feature analysis may be performed on the image.
A deep learning process receives desired process values associated with the one or more process stations. The deep learning processor receives desired target values for one or more key performance indicators of the manufacturing process. The deep learning processor simulates the manufacturing process to generate expected process values and expected target values for the one or more key performance indicators to optimize the one or more key performance indicators. The simulating includes generating a proposed state change of at least one processing parameter of the initial set of processing parameters. The deep learning processor determines that expected process values and the expected target values are within an acceptable limit of the desired process values and the desired target values. Based on the determining, the deep learning processes causes a change to the initial set of processing parameters based on the proposed state change.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
3.
Artificial Intelligence Process Control for Assembly Processes
A manufacturing system is disclosed herein. The manufacturing system includes a monitoring platform and an analytics platform. The monitoring platform is configured to capture data of an operator during assembly of an article of manufacture. The monitoring platform includes one or more cameras and one or more microphones. The analytics platform is in communication with the monitoring platform. The analytics platform is configured to analyze the data captured by the monitoring platform.
A microscopy system and method of focusing the same are disclosed herein. The microscopy system may include an objective, and imaging device, an illumination source, an epi-illumination module, and a controller. The imaging device is configured to capture a single image of a specimen positioned on a stage of the microscopy system. The illumination source is configured to illuminate the specimen positioned on the stage. The epi-illumination module includes a focusing mechanism in a first primary optical path of a light generated by the illumination source. The focusing mechanism is tilted in relation to a plane perpendicular to the first primary optical path. The controller is in communication with the illumination source. The controller is configured to focus the microscopy system based on a pattern produced by the focusing mechanism on the single image captured by the imaging device.
A training set that includes at least two data types corresponding to operations and control of a manufacturing process is obtained. A deep learning processor is trained to predict expected characteristics of output control signals that correspond with one or more corresponding input operating instructions. A first input operating instruction is received from a first signal splitter. A first output control signal is received from a second signal splitter. The deep learning processor correlates the first input operating instruction and the first output control signal. Based on the correlating, the deep learning processor determines that the first output control signal is not within a range of expected values based on the first input operating instruction. Responsive to the determining, an indication of an anomalous activity is provided as a result of detection of the anomalous activity in the manufacturing process.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
6.
Systems, devices, and methods for providing feedback on and improving the accuracy of super-resolution imaging
Systems, methods, and computer-readable media for feedback on and improving the accuracy of super-resolution imaging. In some embodiments, a low resolution image of a specimen can be obtained using a low resolution objective of a microscopy inspection system. A super-resolution image of at least a portion of the specimen can be generated from the low resolution image of the specimen using a super-resolution image simulation. Subsequently, an accuracy assessment of the super-resolution image can be identified based on one or more degrees of equivalence between the super-resolution image and one or more actually scanned high resolution images of at least a portion of one or more related specimens identified using a simulated image classifier. Based on the accuracy assessment of the super-resolution image, it can be determined whether to further process the super-resolution image. The super-resolution image can be further processed if it is determined to further process the super-resolution image.
G06K 9/32 - Aligning or centering of the image pick-up or image-field
G06F 18/2411 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
G06F 18/2413 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
G06T 3/4038 - Image mosaicing, e.g. composing plane images from plane sub-images
G06T 3/4053 - based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
G06T 5/50 - Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/98 - Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
G06V 20/69 - Microscopic objects, e.g. biological cells or cellular parts
7.
SYSTEMS, METHODS, AND MEDIA FOR ARTIFICIAL INTELLIGENCE PROCESS CONTROL IN ADDITIVE MANUFACTURING
Systems, methods, and media for additive manufacturing are provided. In some embodiments, an additive manufacturing system comprises: a hardware processor that is configured to: receive a captured image; apply a trained failure classifier to a low-resolution version of the captured image; determine that a non-recoverable failure is not present in the printed layer of the object; generate a cropped version of the low-resolution version of the captured image; apply a trained binary error classifier to the cropped version of the low-resolution version of the captured image; determine that an error is present in the printed layer of the object; apply a trained extrusion classifier to the captured image, wherein the trained extrusion classifier generates an extrusion quality score; and adjust a value of a parameter of the print head based on the extrusion quality score to print a subsequent layer of the printed object.
G06F 18/2411 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 10/98 - Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
8.
System and Method for Generating Training Data Sets for Specimen Defect Detection
A system and method for generating a training data set for training a machine learning model to detect defects in specimens is described herein. A computing system cause presentation of an image on a device of a user. The image includes at least one defect on an example specimen. The computing system receives an annotated image from the user. The user annotated the image using an input via the device. The input includes a first indication of a location of the defect and a second indication of a class corresponding to the defect. The computing system adjusts the annotated image to standardize the input based on an error profile of the user and the class corresponding to the defect. The computing system uploads the annotated image for training the machine learning model.
G06V 10/22 - Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
A system is disclosed herein. The system includes a splitter board. The splitter board includes a microprocessor, a converter, and a bypass relay. The converter includes analog-to-digital circuitry and digital-to-analog circuitry. The bypass relay is configurable between a first state and a second state. In the first state, the bypass relay is configured to direct an input signal to the converter. The converter converts the input signal to a converted input signal and splits the converted input signal into a first portion and a second portion. The first portion is directed to the microprocessor. The second portion is directed to an output port of the splitter board for downstream processes. In the second state, the bypass relay is configured to cause the input signal to bypass the converter. The bypass relay directs the input signal to the output port of the splitter board for the downstream processes.
H02H 7/20 - Emergency protective circuit arrangements specially adapted for specific types of electric machines or apparatus or for sectionalised protection of cable or line systems, and effecting automatic switching in the event of an undesired change from norm for electronic equipment
G01R 31/3177 - Testing of logic operation, e.g. by logic analysers
10.
PREDICTIVE PROCESS CONTROL FOR A MANUFACTURING PROCESS
Aspects of the disclosed technology encompass the use of a deep-learning controller for monitoring and improving a manufacturing process. In some aspects, a method of the disclosed technology includes steps for: receiving control values associated with a process station in a manufacturing process, predicting an expected value for an article of manufacture output from the process station, and determining if the deep-learning controller can control the manufacturing process based on the expected value. Systems and computer-readable media are also provided.
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
11.
DYNAMIC MONITORING AND SECURING OF FACTORY PROCESSES, EQUIPMENT AND AUTOMATED SYSTEMS
A system including a deep learning processor receives one or more control signals from one or more of a factory's process, equipment and control (P/E/C) systems during a manufacturing process. The processor generates expected response data and expected behavioral pattern data for the control signals. The processor receives production response data from the one or more of the factory's P/E/C systems and generates production behavioral pattern data for the production response data. The process compares at least one of: the production response data to the expected response data, and the production behavioral pattern data to the expected behavioral pattern data to detect anomalous activity. As a result of detecting anomalous activity, the processor performs one or more operations to provide notice or cause one or more of the factory's P/E/C systems to address the anomalous activity.
G06F 21/55 - Detecting local intrusion or implementing counter-measures
G06F 21/56 - Computer malware detection or handling, e.g. anti-virus arrangements
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
Aspects of the disclosed technology provide a computational model that utilizes machine learning for detecting errors during a manual assembly process and determining a sequence of steps to complete the manual assembly process in order to mitigate the detected errors. In some implementations, the disclosed technology evaluates a target object at a step of an assembly process where an error is detected to a nominal object to obtain a comparison. Based on this comparison, a sequence of steps for completion of the assembly process of the target object is obtained. The assembly instructions for creating the target object are adjusted based on this sequence of steps.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
A system is disclosed herein. The system includes a splitter board. The splitter board includes a microprocessor, a converter, and a bypass relay. The converter includes analog-to-digital circuitry and digital-to-analog circuitry. The bypass relay is configurable between a first state and a second state. In the first state, the bypass relay is configured to direct an input signal to the converter. The converter converts the input signal to a converted input signal and splits the converted input signal into a first portion and a second portion. The first portion is directed to the microprocessor. The second portion is directed to an output port of the splitter board for downstream processes. In the second state, the bypass relay is configured to cause the input signal to bypass the converter. The bypass relay directs the input signal to the output port of the splitter board for the downstream processes.
H01P 5/16 - Conjugate devices, i.e. devices having at least one port decoupled from one other port
G01R 31/3177 - Testing of logic operation, e.g. by logic analysers
H02H 7/20 - Emergency protective circuit arrangements specially adapted for specific types of electric machines or apparatus or for sectionalised protection of cable or line systems, and effecting automatic switching in the event of an undesired change from norm for electronic equipment
Aspects of the disclosed technology provide an Artificial Intelligence Process Control (AIPC) for automatically detecting errors in a manufacturing workflow of an assembly line process, and performing error mitigation through the update of instructions or guidance given to assembly operators at various stations. In some implementations, the disclosed technology utilizes one or more machine-learning models to perform error detection and/or propagate instructions/assembly modifications necessary to rectify detected errors or to improve the product of manufacture.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
15.
SYSTEM, METHOD AND APPARATUS FOR MACROSCOPIC INSPECTION OF REFLECTIVE SPECIMENS
An inspection apparatus includes a specimen stage, one or more imaging devices and a set of lights, all controllable by a control system. By translating or rotating the one or more imaging devices or specimen stage, the inspection apparatus can capture a first image of the specimen that includes a first imaging artifact to a first side of a reference point and then capture a second image of the specimen that includes a second imaging artifact to a second side of the reference point. The first and second imaging artifacts can be cropped from the first image and the second image respectively, and the first image and the second image can be digitally stitched together to generate a composite image of the specimen that lacks the first and second imaging artifacts.
Aspects of the disclosed technology encompass the use of a deep learning controller for monitoring and improving a manufacturing process. In some aspects, a method of the disclosed technology includes steps for: receiving a plurality of control values from two or more stations, at a deep learning controller, wherein the control values are generated at the two or more stations deployed in a manufacturing process, predicting an expected value for an intermediate or final output of an article of manufacture, based on the control values, and determining if the predicted expected value for the article of manufacture is in-specification. In some aspects, the process can further include steps for generating control inputs if the predicted expected value for the article of manufacture is not in-specification. Systems and computer-readable media are also provided.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
17.
Method, systems and apparatus for intelligently emulating factory control systems and simulating response data
A simulated process is initiated. The simulated process includes generating, by an emulator, a control signal based on external inputs. The simulated process further includes processing, by a simulator, the control signal to generate simulated response data. The simulated process further includes generating, by a deep learning processor, expected behavioral pattern data based on the simulated response data. An actual process is initiated by initializing setpoints for a process station in a manufacturing system. The actual process includes generating, by the deep learning processor, actual behavioral pattern data based on actual process data from the at least one process station. The deep learning processor compares the expected behavioral pattern to the actual behavioral pattern. Based on the comparing, the deep learning processor determines that anomalous activity is present in the manufacturing system. Based on the anomalous activity being present, the deep learning processor initiates an alert protocol.
A fluorescence microscopy inspection system includes light sources able to emit light that causes a specimen to fluoresce and light that does not cause a specimen to fluoresce. The emitted light is directed through one or more filters and objective channels towards a specimen. A ring of lights projects light at the specimen at an oblique angle through a darkfield channel. One of the filters may modify the light to match a predetermined bandgap energy associated with the specimen and another filter may filter wavelengths of light reflected from the specimen and to a camera. The camera may produce an image from the received light and specimen classification and feature analysis may be performed on the image.
A controller emulator, coupled to an interface that exposes the controller emulator to inputs from external sources, provides one or more control signals to a process simulator and a deep learning process. In response, the process simulator simulates response data that is provided to the deep learning processor. The deep learning processor generates expected response data and expected behavioral pattern data for the one or more control signals, as well as actual behavioral pattern data for the simulated response data. A comparison of at least one of the simulated response data to the expected response data and the actual behavioral pattern data to the expected behavioral pattern data is performed to determine whether anomalous activity is detected. As a result of detecting anomalous activity, one or more operations are performed to address the anomalous activity.
A deep learning process receives desired process values associated with the one or more process stations. The deep learning processor receives desired target values for one or more key performance indicators of the manufacturing process. The deep learning processor simulates the manufacturing process to generate expected process values and expected target values for the one or more key performance indicators to optimize the one or more key performance indicators. The simulating includes generating a proposed state change of at least one processing parameter of the initial set of processing parameters. The deep learning processor determines that expected process values and the expected target values are within an acceptable limit of the desired process values and the desired target values. Based on the determining, the deep learning processes causes a change to the initial set of processing parameters based on the proposed state change.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
21.
System and method for generating training data sets for specimen defect detection
A system and method for generating a training data set for training a machine learning model to detect defects in specimens is described herein. A computing system cause presentation of an image on a device of a user. The image includes at least one defect on an example specimen. The computing system receives an annotated image from the user. The user annotated the image using an input via the device. The input includes a first indication of a location of the defect and a second indication of a class corresponding to the defect. The computing system adjusts the annotated image to standardize the input based on an error profile of the user and the class corresponding to the defect. The computing system uploads the annotated image for training the machine learning model.
G06V 10/22 - Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
22.
System, method and apparatus for macroscopic inspection of reflective specimens
An inspection apparatus includes a specimen stage configured to retain a specimen, at least three imaging devices arranged in a triangular array positioned above the specimen stage, each of the at least three imaging devices configured to capture an image of the specimen, one or more sets of lights positioned between the specimen stage and the at least three imaging devices, and a control system in communication with the at least three imaging devices.
An imaging system is disclosed herein. The imaging system includes an imaging apparatus and a computing system. The imaging apparatus includes a plurality of light sources positioned at a plurality of positions and a plurality of angles relative to a stage configured to support a specimen. The imaging apparatus is configured to capture a plurality of images of a surface of the specimen. The computing system in communication with the imaging apparatus. The computing system configured to generate a 3D-reconstruction of the surface of the specimen by receiving, from the imaging apparatus, the plurality of images of the surface of the specimen, generating, by the imaging apparatus via a deep learning model, a height map of the surface of the specimen based on the plurality of images, and outputting a 3D-reconstruction of the surface of the specimen based on the height map generated by the deep learning model.
A training set that includes at least two data types corresponding to operations and control of a manufacturing process is obtained. A deep learning processor is trained to predict expected characteristics of output control signals that correspond with one or more corresponding input operating instructions. A first input operating instruction is received from a first signal splitter. A first output control signal is received from a second signal splitter. The deep learning processor correlates the first input operating instruction and the first output control signal. Based on the correlating, the deep learning processor determines that the first output control signal is not within a range of expected values based on the first input operating instruction. Responsive to the determining, an indication of an anomalous activity is provided as a result of detection of the anomalous activity in the manufacturing process.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
25.
Unique oblique lighting technique using a brightfield darkfield objective and imaging method relating thereto
A process is provided for imaging a surface of a specimen with an imaging system that employs a BD objective having a darkfield channel and a bright field channel, the BD objective having a circumference. The specimen is obliquely illuminated through the darkfield channel with a first arced illuminating light that obliquely illuminates the specimen through a first arc of the circumference. The first arced illuminating light reflecting off of the surface of the specimen is recorded as a first image of the specimen from the first arced illuminating light reflecting off the surface of the specimen, and a processor generates a 3D topography of the specimen by processing the first image through a topographical imaging technique. Imaging apparatus is also provided as are further process steps for other embodiments.
An apparatus for manipulating an object includes first and second gesture controllers, each operatively connected to the object and structured and programmed such that, in a first-action active state, each can causes a first action to be carried out on the object by an appropriate first-action gesture made in the gesture controller. Only one of the first and second gesture controllers at any given time is capable of being in the first-action active state, and the first-action active state is transferable between the first and second gesture controllers upon the detecting of a first-action transfer gesture by one of said first gesture controller and said second gesture controller. Specific gesture control apparatus and methods for manipulating an object are also disclosed.
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two offset focusing cameras. Each offset focusing camera can be positioned on a different side of an image forming conjugate plane so that their sharpness curves intersect at the image forming conjugate plane. Focus of a specimen can be adjusted by using sharpness values determined from images taken by the offset focusing cameras.
H04N 23/67 - Focus control based on electronic image sensor signals
H04N 23/45 - Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
28.
Method, systems and apparatus for intelligently emulating factory control systems and simulating response data
A simulated process is initiated. The simulated process includes generating, by an emulator, a control signal based on external inputs. The simulated process further includes processing, by a simulator, the control signal to generate simulated response data. The simulated process further includes generating, by a deep learning processor, expected behavioral pattern data based on the simulated response data. An actual process is initiated by initializing setpoints for a process station in a manufacturing system. The actual process includes generating, by the deep learning processor, actual behavioral pattern data based on actual process data from the at least one process station. The deep learning processor compares the expected behavioral pattern to the actual behavioral pattern. Based on the comparing, the deep learning processor determines that anomalous activity is present in the manufacturing system. Based on the anomalous activity being present, the deep learning processor initiates an alert protocol.
A system is disclosed herein. The system includes a splitter board. The splitter board includes a microprocessor, a converter, and a bypass relay. The converter includes analog-to-digital circuitry and digital-to-analog circuitry. The bypass relay is configurable between a first state and a second state. In the first state, the bypass relay is configured to direct an input signal to the converter. The converter converts the input signal to a converted input signal and splits the converted input signal into a first portion and a second portion. The first portion is directed to the microprocessor. The second portion is directed to an output port of the splitter board for downstream processes. In the second state, the bypass relay is configured to cause the input signal to bypass the converter. The bypass relay directs the input signal to the output port of the splitter board for the downstream processes.
G01R 31/3177 - Testing of logic operation, e.g. by logic analysers
H02H 7/20 - Emergency protective circuit arrangements specially adapted for specific types of electric machines or apparatus or for sectionalised protection of cable or line systems, and effecting automatic switching in the event of an undesired change from norm for electronic equipment
A computing system generates a training data set for training the prediction model to detect defects present in a target surface of a target specimen and training the prediction model to detect defects present in the target surface of the target specimen based on the training data set. The computing system generates the training data set by identifying a set of images for training the prediction model, the set of images comprising a first subset of images. A deep learning network generates a second subset of images for subsequent labelling based on the set of images comprising the first subset of images. The deep learning network generates a third subset of images for labelling based on the set of images comprising the first subset of images and the labeled second subset of images. The computing system continues the process until a threshold number of labeled images is generated.
The disclosed technology relates to an inspection apparatus that includes a stage configured to retain a specimen for inspection, an imaging device having a field of view encompassing at least a portion of the stage to view a specimen retained on the stage, and a plurality of lights disposed on a moveable platform. The inspection apparatus can further include a control module coupled to the imaging device, each of the lights and the moveable platform. The control module is configured to perform operations including: receiving image data from the imaging device, where the image data indicates an illumination landscape of light incident on the specimen; and automatically modifying, based on the image data, an elevation of the moveable platform or an intensity of one or more of the lights to adjust the illumination landscape. Methods and machine-readable media are also contemplated.
An inspection apparatus includes a specimen stage, one or more imaging devices and a set of lights, all controllable by a control system. By translating or rotating the one or more imaging devices or specimen stage, the inspection apparatus can capture a first image of the specimen that includes a first imaging artifact to a first side of a reference point and then capture a second image of the specimen that includes a second imaging artifact to a second side of the reference point. The first and second imaging artifacts can be cropped from the first image and the second image respectively, and the first image and the second image can be digitally stitched together to generate a composite image of the specimen that lacks the first and second imaging artifacts.
A method and system for mapping fluid objects on a substrate using a microscope inspection system that includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module. A computer analysis system includes an object identification module that identifies for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms. At least one of the objects is fluid and has shifted from a prior position or deformed from a prior size.
A computing system identifies a trajectory example generated by a human operator. The trajectory example includes trajectory information of the human operator while performing a task to be learned by a control system of the computing system. Based on the trajectory example, the computing system trains the control system to perform the task exemplified in the trajectory example. Training the control system includes generating an output trajectory of a robot performing the task. The computing system identifies an updated trajectory example generated by the human operator based on the trajectory example and the output trajectory of the robot performing the task. Based on the updated trajectory example, the computing system continues to train the control system to perform the task exemplified in the updated trajectory example.
G05B 19/423 - Teaching successive positions by walk-through, i.e. the tool head or end effector being grasped and guided directly, with or without servo-assistance, to follow a path
A system is disclosed herein. The system includes a splitter board. The splitter board includes a microprocessor, a converter, and a bypass relay. The converter includes analog-to-digital circuitry and digital-to-analog circuitry. The bypass relay is configurable between a first state and a second state. In the first state, the bypass relay is configured to direct an input signal to the converter. The converter converts the input signal to a converted input signal and splits the converted input signal into a first portion and a second portion. The first portion is directed to the microprocessor. The second portion is directed to an output port of the splitter board for downstream processes. In the second state, the bypass relay is configured to cause the input signal to bypass the converter. The bypass relay directs the input signal to the output port of the splitter board for the downstream processes.
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two cameras. The first camera can be positioned in a first image forming conjugate plane and receives light from a first illumination source that transmits light in a first wavelength range. The second camera can be positioned at an offset distance from the first image forming conjugate plane and receives light from a second illumination source that transmits light in a second wavelength range.
A fluorescence microscopy inspection system includes light sources able to emit light that causes a specimen to fluoresce and light that does not cause a specimen to fluoresce. The emitted light is directed through one or more filters and objective channels towards a specimen. A ring of lights projects light at the specimen at an oblique angle through a darkfield channel. One of the filters may modify the light to match a predetermined bandgap energy associated with the specimen and another filter may filter wavelengths of light reflected from the specimen and to a camera. The camera may produce an image from the received light and specimen classification and feature analysis may be performed on the image.
Aspects of the disclosed technology provide a computational model that utilizes machine learning for detecting errors during a manual assembly process and determining a sequence of steps to complete the manual assembly process in order to mitigate the detected errors. In some implementations, the disclosed technology evaluates a target object at a step of an assembly process where an error is detected to a nominal object to obtain a comparison. Based on this comparison, a sequence of steps for completion of the assembly process of the target object is obtained. The assembly instructions for creating the target object are adjusted based on this sequence of steps.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
Aspects of the disclosed technology provide an Artificial Intelligence Process Control (AIPC) for automatically detecting errors in a manufacturing workflow of an assembly line process, and performing error mitigation through the update of instructions or guidance given to assembly operators at various stations. In some implementations, the disclosed technology utilizes one or more machine-learning models to perform error detection and/or propagate instructions/assembly modifications necessary to rectify detected errors or to improve the product of manufacture.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
Embodiments disclosed herein are generally related to a system for noise reduction in low signal to noise ratio imaging conditions. A computing system obtains a set of images of a specimen. The set of images includes at least two images of the specimen. The computing system inputs the set of images of the specimen into a trained denoising model. The trained denoising model is configured to output a single denoised image of the specimen. The computing system receives, as output from the trained denoising model, a single denoised image of the specimen.
A computing system receives, from an image sensor, at least two images of a specimen positioned on a specimen stage of a microscope system. The computing system provides the at least two images to an autofocus model for detecting at least one distances to a focal plane of the specimen. The computing system identifies, via the autofocus model, the at least one distance to the focal plane of the specimen. Based on the identifying, the computing system automatically adjusts a position of the specimen stage with respect to an objective lens of the microscope system.
Systems, methods, and media for additive manufacturing are provided. In some embodiments, an additive manufacturing system comprises: a hardware processor that is configured to: receive a captured image; apply a trained failure classifier to a low-resolution version of the captured image; determine that a non-recoverable failure is not present in the printed layer of the object; generate a cropped version of the low-resolution version of the captured image; apply a trained binary error classifier to the cropped version of the low-resolution version of the captured image; determine that an error is present in the printed layer of the object; apply a trained extrusion classifier to the captured image, wherein the trained extrusion classifier generates an extrusion quality score; and adjust a value of a parameter of the print head based on the extrusion quality score to print a subsequent layer of the printed object.
G06F 18/2411 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 10/98 - Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
B22F 10/12 - Formation of a green body by photopolymerisation, e.g. stereolithography [SLA] or digital light processing [DLP]
B22F 10/18 - Formation of a green body by mixing binder with metal in filament form, e.g. fused filament fabrication [FFF]
B22F 10/25 - Direct deposition of metal particles, e.g. direct metal deposition [DMD] or laser engineered net shaping [LENS]
B22F 10/28 - Powder bed fusion, e.g. selective laser melting [SLM] or electron beam melting [EBM]
43.
Systems, Methods, and Media for Manufacturing Processes
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a computing system. The computing system receives an image of the product at a step of the multi-step manufacturing process. The computing system determines a current state of the product based on the image of the product. The computing system determines, via a deep learning model, that the product is not within specification based on the current state of the product and the image of the product. Based on the determining, the computing system adjusts a control logic for at least a following station. The adjusting includes generating, by the deep learning model, a corrective action to be performed by the following station.
B29C 64/393 - Data acquisition or data processing for additive manufacturing for controlling or regulating additive manufacturing processes
B33Y 30/00 - ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING - Details thereof or accessories therefor
B33Y 50/02 - Data acquisition or data processing for additive manufacturing for controlling or regulating additive manufacturing processes
G05B 19/4099 - Surface or curve machining, making 3D objects, e.g. desktop manufacturing
Additive manufacturing systems using artificial intelligence can identify an anomaly in a printed layer of an object from a generated topographical image of the printed layer. The additive manufacturing systems can also use artificial intelligence to determine a correlation between the identified anomaly and one or more print parameters, and adaptively adjust one or more print parameters. The additive manufacturing systems can also use artificial intelligence to optimize one or more printing parameters to achieve desired mechanical, optical and/or electrical properties.
A controlled growth system is provided herein. The controlled growth system includes a controlled growth environment, a controller, a sensor, and a computing system. The controlled growth environment is configured to grow a biologic. The controller is in communication with the controlled growth environment. The controller is configured to manage process parameters of the controlled growth environment. The sensor is configured to monitor the biologic during a growth process. The computing system is in communication with the sensor and the controller. The computing system is programmed to perform operations for achieving a desired final quality metric for the biologic.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
A01G 31/06 - Hydroponic culture on racks or in stacked containers
A01G 31/00 - Soilless cultivation, e.g. hydroponics
46.
Dynamic monitoring and securing of factory processes, equipment and automated systems
A system including a deep learning processor receives one or more control signals from one or more of a factory's process, equipment and control (P/E/C) systems during a manufacturing process. The processor generates expected response data and expected behavioral pattern data for the control signals. The processor receives production response data from the one or more of the factory's P/E/C systems and generates production behavioral pattern data for the production response data. The process compares at least one of: the production response data to the expected response data, and the production behavioral pattern data to the expected behavioral pattern data to detect anomalous activity. As a result of detecting anomalous activity, the processor performs one or more operations to provide notice or cause one or more of the factory's P/E/C systems to address the anomalous activity.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
G06F 21/55 - Detecting local intrusion or implementing counter-measures
G06F 21/56 - Computer malware detection or handling, e.g. anti-virus arrangements
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
A controller emulator, coupled to an interface that exposes the controller emulator to inputs from external sources, provides one or more control signals to a process simulator and a deep learning process. In response, the process simulator simulates response data that is provided to the deep learning processor. The deep learning processor generates expected response data and expected behavioral pattern data for the one or more control signals, as well as actual behavioral pattern data for the simulated response data. A comparison of at least one of the simulated response data to the expected response data and the actual behavioral pattern data to the expected behavioral pattern data is performed to determine whether anomalous activity is detected. As a result of detecting anomalous activity, one or more operations are performed to address the anomalous activity.
An inspection apparatus includes a specimen stage configured to retain a specimen, at least three imaging devices arranged in a triangular array positioned above the specimen stage, each of the at least three imaging devices configured to capture an image of the specimen, one or more sets of lights positioned between the specimen stage and the at least three imaging devices, and a control system in communication with the at least three imaging devices.
Aspects of the disclosed technology encompass the use of a deep-learning controller for monitoring and improving a manufacturing process. In some aspects, a method of the disclosed technology includes steps for: receiving control values associated with a process station in a manufacturing process, predicting an expected value for an article of manufacture output from the process station, and determining if the deep-learning controller can control the manufacturing process based on the expected value. Systems and computer-readable media are also provided.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
50.
DYNAMIC MONITORING AND SECURING OF FACTORY PROCESSES, EQUIPMENT AND AUTOMATED SYSTEMS
A system including a deep learning processor obtains response data of at least two data types from a set of process stations performing operations as part of a manufacturing process. The system analyzes factory operation and control data to generate expected behavioral pattern data. Further, the system uses the response data to generate actual behavior pattern data for the process stations. Based on an analysis of the actual behavior pattern data in relation to the expected behavioral pattern data, the system determines whether anomalous activity has occurred as a result of the manufacturing process. If it is determined that anomalous activity has occurred, the system provides an indication of this anomalous activity.
H04L 29/06 - Communication control; Communication processing characterised by a protocol
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the component.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
G06T 7/33 - Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
Aspects of the disclosed technology encompass the use of a deep learning controller for monitoring and improving a manufacturing process. In some aspects, a method of the disclosed technology includes steps for: receiving a plurality of control values from two or more stations, at a deep learning controller, wherein the control values are generated at the two or more stations deployed in a manufacturing process, predicting an expected value for an intermediate or final output of an article of manufacture, based on the control values, and determining if the predicted expected value for the article of manufacture is in-specification. In some aspects, the process can further include steps for generating control inputs if the predicted expected value for the article of manufacture is not in-specification. Systems and computer-readable media are also provided.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
A computing system generates a training data set for training the prediction model to detect defects present in a target surface of a target specimen and training the prediction model to detect defects present in the target surface of the target specimen based on the training data set. The computing system generates the training data set by identifying a set of images for training the prediction model, the set of images comprising a first subset of images. A deep learning network generates a second subset of images for subsequent labelling based on the set of images comprising the first subset of images. The deep learning network generates a third subset of images for labelling based on the set of images comprising the first subset of images and the labeled second subset of images. The computing system continues the process until a threshold number of labeled images is generated.
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the component.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G06N 3/04 - Architecture, e.g. interconnection topology
B33Y 30/00 - ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING - Details thereof or accessories therefor
B33Y 50/02 - Data acquisition or data processing for additive manufacturing for controlling or regulating additive manufacturing processes
B29C 64/393 - Data acquisition or data processing for additive manufacturing for controlling or regulating additive manufacturing processes
55.
SECURING INDUSTRIAL PRODUCTION FROM SOPHISTICATED ATTACKS
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to detect a cyberattack to the manufacturing system. The control module is configured to perform operations. The operations include receiving control values for a first station of the one or more stations. The operations further include determining that there is a cyberattack based on the control values for the first station using one or more machine learning algorithms. The operations further include generating an alert to cease processing of the component. In some embodiments, the operations further include correcting errors caused by the cyberattack.
A controller emulator, coupled to an interface that exposes the controller emulator to inputs from external sources, provides one or more control signals to a process simulator and a deep learning process. In response, the process simulator simulates response data that is provided to the deep learning processor. The deep learning processor generates expected response data and expected behavioral pattern data for the one or more control signals, as well as actual behavioral pattern data for the simulated response data. A comparison of at least one of the simulated response data to the expected response data and the actual behavioral pattern data to the expected behavioral pattern data is performed to determine whether anomalous activity is detected. As a result of detecting anomalous activity, one or more operations are performed to address the anomalous activity.
An imaging system is disclosed herein. The imaging system includes an imaging apparatus and a computing system. The imaging apparatus includes a plurality of light sources positioned at a plurality of positions and a plurality of angles relative to a stage configured to support a specimen. The imaging apparatus is configured to capture a plurality of images of a surface of the specimen. The computing system in communication with the imaging apparatus. The computing system configured to generate a 3D-reconstruction of the surface of the specimen by receiving, from the imaging apparatus, the plurality of images of the surface of the specimen, generating, by the imaging apparatus via a deep learning model, a height map of the surface of the specimen based on the plurality of images, and outputting a 3D-reconstruction of the surface of the specimen based on the height map generated by the deep learning model.
Systems, methods, and computer-readable media for feedback on and improving the accuracy of super-resolution imaging. In some embodiments, a low resolution image of a specimen can be obtained using a low resolution objective of a microscopy inspection system. A super-resolution image of at least a portion of the specimen can be generated from the low resolution image of the specimen using a super-resolution image simulation. Subsequently, an accuracy assessment of the super-resolution image can be identified based on one or more degrees of equivalence between the super-resolution image and one or more actually scanned high resolution images of at least a portion of one or more related specimens identified using a simulated image classifier. Based on the accuracy assessment of the super-resolution image, it can be determined whether to further process the super-resolution image. The super-resolution image can be further processed if it is determined to further process the super-resolution image.
G06T 3/40 - Scaling of a whole image or part thereof
G06T 5/50 - Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
G06V 10/98 - Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
G06V 20/69 - Microscopic objects, e.g. biological cells or cellular parts
G06F 18/2411 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
G06F 18/2413 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
59.
Unique oblique lighting technique using a brightfield darkfield objective and imaging method relating thereto
A process is provided for imaging a surface of a specimen with an imaging system that employs a BD objective having a darkfield channel and a bright field channel, the BD objective having a circumference. The specimen is obliquely illuminated through the darkfield channel with a first arced illuminating light that obliquely illuminates the specimen through a first arc of the circumference. The first arced illuminating light reflecting off of the surface of the specimen is recorded as a first image of the specimen from the first arced illuminating light reflecting off the surface of the specimen, and a processor generates a 3D topography of the specimen by processing the first image through a topographical imaging technique. Imaging apparatus is also provided as are further process steps for other embodiments.
A system including a deep learning processor obtains response data of at least two data types from a set of process stations performing operations as part of a manufacturing process. The system analyzes factory operation and control data to generate expected behavioral pattern data. Further, the system uses the response data to generate actual behavior pattern data for the process stations. Based on an analysis of the actual behavior pattern data in relation to the expected behavioral pattern data, the system determines whether anomalous activity has occurred as a result of the manufacturing process. If it is determined that anomalous activity has occurred, the system provides an indication of this anomalous activity.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
H04L 29/06 - Communication control; Communication processing characterised by a protocol
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the component.
G05B 19/402 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by control arrangements for positioning, e.g. centring a tool relative to a hole in the workpiece, additional detection means to correct position
G05B 19/4093 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by part programming, e.g. entry of geometrical information as taken from a technical drawing, combining this with machining and material information to obtain control information, named part programme, for the NC machine
The disclosed technology relates to an inspection apparatus that includes a stage configured to retain a specimen for inspection, an imaging device having a field of view encompassing at least a portion of the stage to view a specimen retained on the stage, and a plurality of lights disposed on a moveable platform. The inspection apparatus can further include a control module coupled to the imaging device, each of the lights and the moveable platform. The control module is configured to perform operations including: receiving image data from the imaging device, where the image data indicates an illumination landscape of light incident on the specimen; and automatically modifying, based on the image data, an elevation of the moveable platform or an intensity of one or more of the lights to adjust the illumination landscape. Methods and machine-readable media are also contemplated.
An inspection apparatus includes a specimen stage, one or more imaging devices and a set of lights, all controllable by a control system. By translating or rotating the one or more imaging devices or specimen stage, the inspection apparatus can capture a first image of the specimen that includes a first imaging artifact to a first side of a reference point and then capture a second image of the specimen that includes a second imaging artifact to a second side of the reference point. The first and second imaging artifacts can be cropped from the first image and the second image respectively, and the first image and the second image can be digitally stitched together to generate a composite image of the specimen that lacks the first and second imaging artifacts.
An apparatus for manipulating an object includes first and second gesture controllers, each operatively connected to the object and structured and programmed such that, in a first-action active state, each can causes a first action to be carried out on the object by an appropriate first-action gesture made in the gesture controller. Only one of the first and second gesture controllers at any given time is capable of being in the first-action active state, and the first-action active state is transferable between the first and second gesture controllers upon the detecting of a first-action transfer gesture by one of said first gesture controller and said second gesture controller. Specific gesture control apparatus and methods for manipulating an object are also disclosed.
A manufacturing system is disclosed herein. The manufacturing system may include one or more station, a monitoring platform, and a control module. Each station is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the component.
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the component.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
A system including a deep learning processor receives one or more control signals from one or more of a factory's process, equipment and control (P/E/C) systems during a manufacturing process. The processor generates expected response data and expected behavioral pattern data for the control signals. The processor receives production response data from the one or more of the factory's P/E/C systems and generates production behavioral pattern data for the production response data. The process compares at least one of: the production response data to the expected response data, and the production behavioral pattern data to the expected behavioral pattern data to detect anomalous activity. As a result of detecting anomalous activity, the processor performs one or more operations to provide notice or cause one or more of the factory's P/E/C systems to address the anomalous activity.
G06F 21/55 - Detecting local intrusion or implementing counter-measures
G06F 21/56 - Computer malware detection or handling, e.g. anti-virus arrangements
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a product. The monitoring platform is configured to monitor progression of the product throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the product.
B29C 64/393 - Data acquisition or data processing for additive manufacturing for controlling or regulating additive manufacturing processes
B33Y 30/00 - ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING - Details thereof or accessories therefor
B33Y 50/02 - Data acquisition or data processing for additive manufacturing for controlling or regulating additive manufacturing processes
G05B 19/4099 - Surface or curve machining, making 3D objects, e.g. desktop manufacturing
An inspection apparatus includes a specimen stage, one or more imaging devices and a set of lights, all controllable by a control system. By translating or rotating the one or more imaging devices or specimen stage, the inspection apparatus can capture a first image of the specimen that includes a first imaging artifact to a first side of a reference point and then capture a second image of the specimen that includes a second imaging artifact to a second side of the reference point. The first and second imaging artifacts can be cropped from the first image and the second image respectively, and the first image and the second image can be digitally stitched together to generate a composite image of the specimen that lacks the first and second imaging artifacts.
A method and system for mapping fluid objects on a substrate using a microscope inspection system that includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module. A computer analysis system includes an object identification module that identifies for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms. At least one of the objects is fluid and has shifted from a prior position or deformed from a prior size.
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two offset focusing cameras. Each offset focusing camera can be positioned on a different side of an image forming conjugate plane so that their sharpness curves intersect at the image forming conjugate plane. Focus of a specimen can be adjusted by using sharpness values determined from images taken by the offset focusing cameras.
Systems, methods, and computer-readable media for feedback on and improving the accuracy of super-resolution imaging. In some embodiments, a low resolution image of a specimen can be obtained using a low resolution objective of a microscopy inspection system. A super-resolution image of at least a portion of the specimen can be generated from the low resolution image of the specimen using a super-resolution image simulation. Subsequently, an accuracy assessment of the super-resolution image can be identified based on one or more degrees of equivalence between the super-resolution image and one or more actually scanned high resolution images of at least a portion of one or more related specimens identified using a simulated image classifier. Based on the accuracy assessment of the super-resolution image, it can be determined whether to further process the super-resolution image. The super-resolution image can be further processed if it is determined to further process the super-resolution image.
Aspects of the disclosed technology encompass the use of a deep learning controller for monitoring and improving a manufacturing process. In some aspects, a method of the disclosed technology includes steps for: receiving a plurality of control values from two or more stations, at a deep learning controller, wherein the control values are generated at the two or more stations deployed in a manufacturing process, predicting an expected value for an intermediate or final output of an article of manufacture, based on the control values, and determining if the predicted expected value for the article of manufacture is in-specification. In some aspects, the process can further include steps for generating control inputs if the predicted expected value for the article of manufacture is not in-specification. Systems and computer-readable media are also provided.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
74.
Predictive process control for a manufacturing process
Aspects of the disclosed technology encompass the use of a deep-learning controller for monitoring and improving a manufacturing process. In some aspects, a method of the disclosed technology includes steps for: receiving control values associated with a process station in a manufacturing process, predicting an expected value for an article of manufacture output from the process station, and determining if the deep-learning controller can control the manufacturing process based on the expected value. Systems and computer-readable media are also provided.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
75.
Apparatus and method for manipulating objects with gesture controls
An apparatus for manipulating an object includes first and second gesture controllers, each operatively connected to the object and structured and programmed such that, in a first-action active state, each can causes a first action to be carried out on the object by an appropriate first-action gesture made in the gesture controller. Only one of the first and second gesture controllers at any given time is capable of being in the first-action active state, and the first-action active state is transferable between the first and second gesture controllers upon the detecting of a first-action transfer gesture by one of said first gesture controller and said second gesture controller. Specific gesture control apparatus and methods for manipulating an object are also disclosed.
A microscope system and method allow for a desired x′-direction scanning along a specimen to be angularly offset from an x-direction of the XY translation stage, and rotates an image sensor associated with the microscope to place the pixel rows of the image sensor substantially parallel to the desired x′-direction. The angle of offset of the x′-direction relative to the x-direction is determined and the XY translation stage is employed to move the specimen relative to the image sensor to different positions along the desired x′-direction without a substantial shift of the image sensor relative to the specimen in a y′-direction, the y′-direction being orthogonal to the x′ direction of the specimen. The movement is based on the angle of offset.
Aspects of the disclosed technology provide a computational model that utilizes machine learning for detecting errors during a manual assembly process and determining a sequence of steps to complete the manual assembly process in order to mitigate the detected errors. In some implementations, the disclosed technology evaluates a target object at a step of an assembly process where an error is detected to a nominal object to obtain a comparison. Based on this comparison, a sequence of steps for completion of the assembly process of the target object is obtained. The assembly instructions for creating the target object are adjusted based on this sequence of steps.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
Aspects of the disclosed technology provide an Artificial Intelligence Process Control (AIPC) for automatically detecting errors in a manufacturing workflow of an assembly line process, and performing error mitigation through the update of instructions or guidance given to assembly operators at various stations. In some implementations, the disclosed technology utilizes one or more machine-learning models to perform error detection and/or propagate instructions/assembly modifications necessary to rectify detected errors or to improve the product of manufacture.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
79.
Systems, devices, and methods for automatic microscopic focus
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two cameras. The first camera can be positioned in a first image forming conjugate plane and receives light from a first illumination source that transmits light in a first wavelength range. The second camera can be positioned at an offset distance from the first image forming conjugate plane and receives light from a second illumination source that transmits light in a second wavelength range.
A fluorescence microscopy inspection system includes light sources able to emit light that causes a specimen to fluoresce and light that does not cause a specimen to fluoresce. The emitted light is directed through one or more filters and objective channels towards a specimen. A ring of lights projects light at the specimen at an oblique angle through a darkfield channel. One of the filters may modify the light to match a predetermined bandgap energy associated with the specimen and another filter may filter wavelengths of light reflected from the specimen and to a camera. The camera may produce an image from the received light and specimen classification and feature analysis may be performed on the image.
Additive manufacturing systems using artificial intelligence can identify an anomaly in a printed layer of an object from a generated topographical image of the printed layer. The additive manufacturing systems can also use artificial intelligence to determine a correlation between the identified anomaly and one or more print parameters, and adaptively adjust one or more print parameters. The additive manufacturing systems can also use artificial intelligence to optimize one or more printing parameters to achieve desired mechanical, optical and/or electrical properties.
Systems, methods, and media for additive manufacturing are provided. In some embodiments, an additive manufacturing system comprises: a hardware processor that is configured to: receive a captured image; apply a trained failure classifier to a low-resolution version of the captured image; determine that a non-recoverable failure is not present in the printed layer of the object; generate a cropped version of the low-resolution version of the captured image; apply a trained binary error classifier to the cropped version of the low-resolution version of the captured image; determine that an error is present in the printed layer of the object; apply a trained extrusion classifier to the captured image, wherein the trained extrusion classifier generates an extrusion quality score; and adjust a value of a parameter of the print head based on the extrusion quality score to print a subsequent layer of the printed object.
The disclosed technology relates to an inspection apparatus that includes a stage configured to retain a specimen for inspection, an imaging device having a field of view encompassing at least a portion of the stage to view a specimen retained on the stage, and a plurality of lights disposed on a moveable platform. The inspection apparatus can further include a control module coupled to the imaging device, each of the lights and the moveable platform. The control module is configured to perform operations including: receiving image data from the imaging device, where the image data indicates an illumination landscape of light incident on the speciment; and automatically modifying, based on the image data, an elevation of the moveable platform or an intensity of one or more of the lights to adjust the illumination landscape. Methods and machine-readable media are also contemplated.
A method and system for mapping fluid objects on a substrate using a microscope inspection system that includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module. A computer analysis system includes an object identification module that identifies for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms. At least one of the objects is fluid and has shifted from a prior position or deformed from a prior size.
A method for altering the intensity of light across the field of view of an image sensor in a microscope apparatus having a light source, an image sensor having pixels, and a specimen stage, wherein light from the light source travels along a light path to the specimen stage and then to the image sensor includes interposing a programmable spatial light modulator, pSLM, in the light path between the light source and the image sensor, the pSLM having a plurality of pixels; and modulating the intensity of light passing through one or more pixels of the plurality of pixels of the pSLM to produce an altered illumination landscape at the field of view of the image sensor that differs from an unaltered illumination landscape that would otherwise be produced at the image sensor. Vignetting can be specifically addressed.
H04N 25/61 - Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
86.
Fluorescence microscopy inspection systems, apparatus and methods
A fluorescence microscopy inspection system includes light sources able to emit light that causes a specimen to fluoresce and light that does not cause a specimen to fluoresce. The emitted light is directed through one or more filters and objective channels towards a specimen. A ring of lights projects light at the specimen at an oblique angle through a darkfield channel. One of the filters may modify the light to match a predetermined bandgap energy associated with the specimen and another filter may filter wavelengths of light reflected from the specimen and to a camera. The camera may produce an image from the received light and specimen classification and feature analysis may be performed on the image.
The disclosed technology relates to an inspection apparatus that includes a stage configured to retain a specimen for inspection, an imaging device having a field of view encompassing at least a portion of the stage to view a specimen retained on the stage, a lens having a view encompassing the specimen retained on the stage, and a plurality of lights disposed on a moveable platform. The inspection apparatus can further include a control module configured to control a position of the stage, an elevation of the moveable platform, and a focus of the lens. In some implementations, the inspection apparatus includes an image processing system configured for receiving image data from the imaging device, analyzing the image data to determine a specimen classification, and automatically selecting an illumination profile based on the specimen classification. Methods and machine-readable media are also contemplated.
A process is provided for imaging a surface of a specimen with an imaging system that employs a BD objective having a darkfield channel and a bright field channel, the BD objective having a circumference. The specimen is obliquely illuminated through the darkfield channel with a first arced illuminating light that obliquely illuminates the specimen through a first arc of the circumference. The first arced illuminating light reflecting off of the surface of the specimen is recorded as a first image of the specimen from the first arced illuminating light reflecting off the surface of the specimen, and a processor generates a 3D topography of the specimen by processing the first image through a topographical imaging technique. Imaging apparatus is also provided as are further process steps for other embodiments.
Systems, methods, and computer-readable media for feedback on and improving the accuracy of super-resolution imaging. In some embodiments, a low resolution image of a specimen can be obtained using a low resolution objective of a microscopy inspection system. A super-resolution image of at least a portion of the specimen can be generated from the low resolution image of the specimen using a super-resolution image simulation. Subsequently, an accuracy assessment of the super-resolution image can be identified based on one or more degrees of equivalence between the super-resolution image and one or more actually scanned high resolution images of at least a portion of one or more related specimens identified using a simulated image classifier. Based on the accuracy assessment of the super-resolution image, it can be determined whether to further process the super-resolution image. The super-resolution image can be further processed if it is determined to further process the super-resolution image.
A method and system for mapping fluid objects on a substrate using a microscope inspection system that includes a light source, imaging device, stage for moving a substrate disposed on the stage, and a control module. A computer analysis system includes an object identification module that identifies for each of the objects on the substrate, an object position on the substrate including a set of X, Y, and θ coordinates using algorithms, networks, machines and systems including artificial intelligence and image processing algorithms. At least one of the objects is fluid and has shifted from a prior position or deformed from a prior size.
Aspects of the disclosed technology provide an Artificial Intelligence Process Control (AIPC) for automatically detecting errors in a manufacturing workflow of an assembly line process, and performing error mitigation through the update of instructions or guidance given to assembly operators at various stations. In some implementations, the disclosed technology utilizes one or more machine-learning models to perform error detection and/or propagate instructions/assembly modifications necessary to rectify detected errors or to improve the product of manufacture.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G05B 19/19 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
92.
Systems, devices and methods for automatic microscope focus
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two offset focusing cameras. Each offset focusing camera can be positioned on a different side of an image forming conjugate plane so that their sharpness curves intersect at the image forming conjugate plane. Focus of a specimen can be adjusted by using sharpness values determined from images taken by the offset focusing cameras.
Systems, methods, and computer-readable media for feedback on and improving the accuracy of super-resolution imaging. In some embodiments, a low resolution image of a specimen can be obtained using a low resolution objective of a microscopy inspection system. A super-resolution image of at least a portion of the specimen can be generated from the low resolution image of the specimen using a super-resolution image simulation. Subsequently, an accuracy assessment of the super-resolution image can be identified based on one or more degrees of equivalence between the super-resolution image and one or more actually scanned high resolution images of at least a portion of one or more related specimens identified using a simulated image classifier. Based on the accuracy assessment of the super-resolution image, it can be determined whether to further process the super-resolution image. The super-resolution image can be further processed if it is determined to further process the super-resolution image.
Additive manufacturing systems using artificial intelligence can identify an anomaly in a printed layer of an object from a generated topographical image of the printed layer. The additive manufacturing systems can also use artificial intelligence to determine a correlation between the identified anomaly and one or more print parameters, and adaptively adjust one or more print parameters. The additive manufacturing systems can also use artificial intelligence to optimize one or more printing parameters to achieve desired mechanical, optical and/or electrical properties.
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two cameras. The first camera can be positioned in a first image forming conjugate plane and receives light from a first illumination source that transmits light in a first wavelength range. The second camera can be positioned at an offset distance from the first image forming conjugate plane and receives light from a second illumination source that transmits light in a second wavelength range.
G02B 23/00 - Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
G02B 7/28 - Systems for automatic generation of focusing signals
G02B 21/36 - Microscopes arranged for photographic purposes or projection purposes
Systems, devices, and methods for combined wafer and photomask inspection are provided. In some embodiments, chucks are provided, the chucks comprising: a removable insert, wherein the removable insert is configured to support a wafer so that an examination surface of the wafer lies within a focal range when the chuck is in a first configuration, wherein the removable insert is inserted into the chuck in the first configuration; and a first structure forming a recess that has a depth sufficient to support a photomask so that an examination surface of the photomask lies within the focal range when the chuck is in a second configuration, wherein the removable insert is not inserted into the chuck in the second configuration.
Systems, devices, and methods for combined wafer and photomask inspection are provided. In some embodiments, chucks are provided, the chucks comprising: a removable insert, wherein the removable insert is configured to support a wafer so that an examination surface of the wafer lies within a focal range when the chuck is in a first configuration, wherein the removable insert is inserted into the chuck in the first configuration; and a first structure forming a recess that has a depth sufficient to support a photomask so that an examination surface of the photomask lies within the focal range when the chuck is in a second configuration, wherein the removable insert is not inserted into the chuck in the second configuration.
A method for altering the intensity of light across the field of view of an image sensor in a microscope apparatus having a light source, an image sensor having pixels, and a specimen stage, wherein light from the light source travels along a light path to the specimen stage and then to the image sensor includes interposing a programmable spatial light modulator, pSLM, in the light path between the light source and the image sensor, the pSLM having a plurality of pixels; and modulating the intensity of light passing through one or more pixels of the plurality of pixels of the pSLM to produce an altered illumination landscape at the field of view of the image sensor that differs from an unaltered illumination landscape that would otherwise be produced at the image sensor. Vignetting can be specifically addressed.
An automatic focus system for an optical microscope that facilitates faster focusing by using at least two cameras. The first camera can be positioned in a first image forming conjugate plane and receives light from a first illumination source that transmits light in a first wavelength range. The second camera can be positioned at an offset distance from the first image forming conjugate plane and receives light from a second illumination source that transmits light in a second wavelength range.
G02B 23/00 - Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
G02B 7/28 - Systems for automatic generation of focusing signals
An illumination apparatus, method and system to facilitate manual sorting of components. The illumination apparatus can include an array of lights and a component holder receptacle configured to receive a component holder retaining components. The illumination apparatus can further include a control module configured to receive information identifying components for sorting and location information for locating the one or more components on the component holder, and to selectively control activation of individual lights of the array of lights to illuminate the one or more components.
patents
