SLAS 2019 AI in Process Automation Symposium

The digitization of life sciences is based on processes implemented in software, which combine manual and automated work seamlessly. The major part of currently available process descriptions is provided in human-readable text-based formats (e.g., protocols, SOPs, clinical pathways,…), either for historical reasons or due to the lack of appropriate toolchains that formalize processes in versatile ways. Technical process descriptions, on the other hand, need to be well-defined given in a formal notation like BPMN in order for them to be executed by software and deployed on the hardware. The translation of natural-language process descriptions into formal notations requires a substantial amount of effort because technicians need to translate between actual application domains and technical domains. Human capacities limit the degree to which the translation can be sped up, but the need for translation is high: many processes are still being implemented manually, and large numbers of documented process descriptions exist.

 We combine the current state-of-the-art of natural language processing (NLP) with ontological engineering and formal process modeling in order to automate the process of translation of text-based workflow descriptions into machine-readable representations. We devise a software pipeline that, firstly, preprocesses domain corpora in order to distill and identify linguistic features, which are then matched against ontological knowledge. Secondly, an online process reuses these features for rapid translation of textual data into formal process models. We demonstrate the feasibility of our approach and show current drawbacks and limitations.

Induced pluripotent stem cell-derived cardiomyocytes (iPSC-CMs), in combination with CRISPR genome editing, now offer unparalleled opportunities to study cardiac biology and disease. Mature and functional iPSC-CMs are frequently characterized by the presence of visually distinct sarcomere bands. However, few methods exist to comprehensively quantify sarcomere organization. These methods can be relatively low-throughput (requiring manual selection for regions of interest), employ Fast Fourier Transforms (requiring a minimum number of detected sarcomeres), and/or only assess well-aligned sarcomeres from mature cardiac tissues. Here we take advantage of a pixel-based image analysis method to overcome these limitations and evaluate iPSC-CM sarcomere structure. Our approach, or sarcomere organization analysis (SOA), incorporates automated cell segmentation and extracts sarcomere information from Haralick texture features. This quantification enables SOA to process any geometric shape, account for noise, and identify sarcomere persistence. These are especially important features since iPSC-CM cell shape is quite variable and sarcomere orientations and lengths can be heterogeneous. Thus, SOA allowed us to effectively distinguish levels of sarcomere organization to define CMs with highly and moderately organized sarcomeres compared to CMs with visibly disorganized sarcomeres. These differences were further apparent when comparing WT iPSC-CMs to iPSC-CMs with a genetic sarcomere mutation. This mutation resulted in more CMs with sarcomeres longer than 2 µm and a reduced organization. Our SOA method is intended to help provide a more systematic approach to examining cardiac disease biology resulting from genetic variants. Current work is aimed at incorporating deep learning for enhanced analysis and a more complete description of sarcomere organization and iPSC-CM phenotypes.

Large scale cell based phenotypic screens of RNAi libraries are well established but can be limited by incomplete and unknown levels of gene knockdown. Here we present a platform for high content screening of an arrayed lentiviral CRISPR library for the whole human and mouse genomes which has recently been established by us. The use of automation allowed us to minimize handling and any potential exposure to the viral material as well as maximize the assay throughput. Here we present the methods used and example data from the complete process.

The automated steps include culturing, inoculation, staining and imaging, all in 1536 well plates. All liquid handling is performed within an enclosed JANUS workstation with attached carrousel and incubator. Imaging is performed with an opera phenix. Assay volumes have been trialled in the range of 5 - 8 ul per well, evaporation is largely mitigated by use of a filled moat in the plate skirting. Media exchange / washing is less prone to damaging the cell layer as result of such small assay volumes.

The data presented here is representative of the platforms use in whole genome phenotypic screening experiments. This platform represents a new stage of genome screening allowing rapid identification of gene targets for novel phenotypes. By interrogating each gene individually we can discern which target or pathway is involved in producing the phenotype of interest.

Automated manufacturing of induced pluripotent stem cells (iPSCs) overcomes critical restrictions on scalability and technical variability that otherwise preclude the use of iPSCs in large-scale population studies. Following reprogramming from adult cells, a critical step in ensuring cell line viability is the isolation and expansion of a single stem cell, a step referred to as monoclonalization. Monoclonality is generally verified via manual analysis of microscopic images, a highly time- consuming process which relies upon human judgement, incurring a key barrier to standardization and scalability. Here we report the design of a deep learning workflow which reliably identifies monoclonality and ascribes colony viability. The workflow, termed Monoqlo, integrates multiple convolutional neural networks and, critically, leverages the time seriality of the cell culturing process. This work highlights an example of the way in which human-directed design can often overcome obstacles in machine learning engineering in cases where a paucity of training data precludes the use of “more data, better results” strategies.

COGNEX Machine Vision systems are the gold standard on the Factory Automation floor and meanwhile also in Lab Automation environments. Traditional vision systems excel in gauging, measuring and precision alignment, but algorithms become challenging to program as the exceptions and defect libraries grow or different objects look similar which is often the case in Lab Automation and IVD applications.
Deep Learning-based image analysis combines the specificity and flexibility of human visual inspection with the reliability, consistency, and speed of computerized systems. With the introduction of Deep Learning we face new challenges, for example, the large number of images needed for proper training is not always available, the time to develop a highly accurate application is long, developing and maintaining the network requires a highly skilled individual or team and the black-box behavior of the system sometimes can’t earn the trust of the end-user.

Another interesting trend with the emergence of deep learning is that people start to neglect the traditional rule-based approach, but what we see is that rule-based vision and Deep Learning-based image analysis are complementing each other and enable us to solve more complex automation problems.
COGNEX VisionPro ViDi™ is a ready-to-use Deep Learning-based software suite dedicated to image analysis which can provide solutions for a wide range of applications in the life sciences field. It also provides seamless integration with our traditional rule-based vision tools as needed.
In our novel approach, we designed four specific Deep Learning tools which can be used to break down a complex problem to smaller and simpler steps to provide the final solution with a customized toolchain. Each tool is optimized for one specific task:
-Blue – Locate for feature location and verification
-Red – Analyze for segmentation and defect detection
-Green – Classify for scene and object classification
-Blue – Read for optical character recognition

This method allows us to build a solution with only a few hundred images, train and run the Deep Learning models locally on a single GPU system. It also makes it easier to localize any erroneous behavior in the system. The suite provides a graphical user interface for training and labeling with a handful of parameters to fine-tune the models. This feature can enable medical professionals without extensive Deep Learning knowledge to rapidly develop various Deep Learning-based applications where the experts can focus on optimizing, validating and deploying these solutions.
Applications where we successfully developed a solution with COGNEX VisionPro ViDi™:
-Deck inspection, Tube and Plate identification for medical instruments
-HIL Quality inspection of centrifuged blood samples
-Bacteria Classification
-Colony Counting / Classification

These examples highlight the potential of COGNEX VisionPro ViDi™ to be useful for a broad range of complex image analysis tasks within life sciences.