Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
HYBRID DETECTION AND SEPARATION OF TARGET CELLS THROUGH MAGNETIC LEVITATION AND DEEP LEARNING
Document Type and Number:
WIPO Patent Application WO/2023/018398
Kind Code:
A1
Abstract:
The invention is related to an artificial neural network allowing cell groups to separate if their masses are close to each other or the same in addition to detecting target cells and used by training on the basis of learning based on cell morphologies to actively separate, and accordingly active separation of target cells under flow in the conventional magnetic levitation system used for the separation of cells in biological fluid samples based on their density differences.

Inventors:
TEKİN HÜSEYIN CUMHUR (TR)
DELİKOYUN KEREM (TR)
Application Number:
PCT/TR2022/050833
Publication Date:
February 16, 2023
Filing Date:
August 10, 2022
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
IZMIR YUEKSEK TEKNOLOJI ENSTITUESUE (TR)
International Classes:
C12N13/00; C12Q1/00; G01N15/06; G01N15/10; G01N27/00; G01N33/487
Domestic Patent References:
WO2015130913A12015-09-03
Other References:
DELIKOYUN KEREM, YAMAN SENA, YILMAZ ESRA, SARIGIL OYKU, ANIL-INEVI MUGE, TELLI KUBRA, YALCIN-OZUYSAL OZDEN, OZCIVICI ENGIN, TEKIN : "HologLev: A Hybrid Magnetic Levitation Platform Integrated with Lensless Holographic Microscopy for Density-Based Cell Analysis", ACS SENSORS, AMERICAN CHEMICAL SOCIETY, US, vol. 6, no. 6, 25 June 2021 (2021-06-25), US, pages 2191 - 2201, XP093035852, ISSN: 2379-3694, DOI: 10.1021/acssensors.0c02587
"Artificial Intelligence for Data-Driven Medical Diagnosis", 31 January 2021, DE GRUYTER, Germany, ISBN: 9783110668322, article DELIKOYUN KEREM, CINE ERSIN, ANIL-INEVI MUGE, SARIGIL OYKU, OZCIVICI ENGIN, TEKIN H. CUMHUR: "Deep learning-based cellular image analysis for intelligent medical diagnosis", pages: 19 - 54, XP009543469, DOI: 10.1515/9783110668322-002
RYU DONGHUN, KIM JINHO, LIM DAEJIN, MIN HYUN-SEOK, YOU INYOUNG, CHO DUCK, PARK YONGKEUN: "Label-free bone marrow white blood cell classification using refractive index tomograms and deep learning", BIORXIV, 15 November 2020 (2020-11-15), pages 1 - 13, XP093007356, Retrieved from the Internet [retrieved on 20221212], DOI: 10.1101/2020.11.13.381244
Attorney, Agent or Firm:
YALCINER, Ugur G. (YALCINER PATENT & CONSULTING LTD.) (TR)
Download PDF:
Claims:
CLAIMS A magnetic levitation platform detecting and separating cells comprising;

• A bracket (6), enabling the cells in the paramagnetic solution advancing in the main microfluidic channel, positioned between two magnets, the lower magnet (1) and the upper magnet (2) arranged with the same poles facing each other, to different outlets;

• A microfluidic chip (3), containing at least two microfluidic channels (7) separated in different directions by the bracket (6) for active differentiation of one or several cell types among the detected cells;

• At least two pneumatic valves (4) directing the cells to each microfluidic channel (7) with brackets (6) so that the detected cells can be separated in real time and selectively;

• At least one pump (5) working synchronously with the valves (4) integrated in the channels of the microfluidic chip to differentiate the detected cells and deep convolutional neural networks and/or artificial neural networks. A method of detecting and separating cells in real time under flow or in still images with a magnetic levitation platform according to Claim 1 comprising following steps;

• Obtaining images by observing the detection zone on the microfluidic chip integrated into the magnetic levitation platform through a microscope;

• Feeding images to a deep learning-based object detection algorithm trained on different cell types, including target cells;

• Drawing bounding boxes around the cells by determining the cell positions since they detected the cells in the trained model image content above the user-defined confidence threshold;

• Additionally, indicating the predicted classes in the bounding boxes for each detected cell;

• After determining the positions and classes of cells in the image, opening the valve controlling the flow at the outlet where the target cell is separated from the microfluidic channels at the end of the channel in

9 case any of those classes are among the cell types it is intended to separate, closing the valve of the waste channel to which other cells are diverted in case of no detection, and thus directing the target cells together with the flow from the pump to the reservoir outlet where they are collected;

• Directing all cells to the waste outlet with flow from the pump in case of no target cell and directing all cells to the waste outlet while remaining open unless the valve controlling the flow in that channel has a detection of the target cell.

Description:
HYBRID DETECTION AND SEPARATION OF TARGET CELLS THROUGH MAGNETIC LEVITATION AND DEEP LEARNING

Technical Field of the Invention

The invention is related to a new method which can be used for detection and diagnosis of circulating cancer or endothelial cells, which are known as rare cells in life and ecological sciences, and different blood cell groups or microorganisms, and to a biomedical device which can be used in clinical and research laboratories working with this method and in point- of-care tests (POC) in daily life, especially in the medical sector.

Prior Art

The specific and high efficiency separation of a particular group of cells from a heterogeneous cell solution (e.g., blood etc.) is highly complex and challenging from the perspective of time, cost, and instrumentation. Likewise, separation can be performed depending to the antibody based biochemical methods conventionally based on surface markers or the methods based on distinct biophysical properties to separate cell groups. However, the efficiency of separation for both methods is limited since not all target cells show similar characteristics. Many cells can be lost as a result of this method since antibodies specific to surface markers are not expressed by each target cell for several reasons. On the other hand, the same biophysical properties (or the ones in a narrow range) should not be expected in cells from the same subgroup. For example, an important biophysical property is the size of cell. Although filtering is possible depending on cell size, the filter systems used can reduce the viability of the cells and the presence of low-sized cells from the same cell group reduces the efficiency by allowing these cells to pass through the filter structure easily, and it causes an undesirable background pollution by separating the cells from the different group of similar size from the target cells.

Density is the most frequently applied biophysical property in routine clinical analysis such as blood cell count. Each cell group shows quiet different density values due to metabolic activity and physiological characteristics. The cells collapse and accumulate in different layers by centrifugation since they have different densities. However, it is not possible to separate those different layers of cells easily. Therefore, although density-dependent decomposition makes it possible to separate quite specifically certain groups, conventional methods are not suitable for density-based analysis and separation at the single cell level.

Magnetic levitation is a technology where the buoyancy of the liquid and the magnetic force are balanced between two magnets with opposite poles of a paramagnetic solution mixed into the cell solution and which allows the cells to be aligned in a certain position proportional to the density. It is possible to detect and analyse different cell groups on the basis of single-cell images since the cells are aligned at different heights relative to the lower magnet on the vertical axis in proportion to their density. Additionally, it is possible to collect cells from those levels at the channel outlet by moving and aligning them at heights proportional to the density of the cells in the applied magnetic field by making the cells pass through the channel under the flow.

On the other hand, systems which can recognize and detect each cell by morphological analysis from visual data based on deep learning have been developed recently. However, detection of a certain cell group in the whole sample is not a viable method without any prescreening (purification) process in samples where the number of cells in a unit volume is high, such as blood.

The cells are collected at different heights on the axis perpendicular to the lower magnet in proportion to their density in the magnetic levitation system known within the technique. Theoretically, it is possible to separate this group from a heterogeneous sample by being in different density ranges and thus separating the cells of the height corresponding to this density even if each cell group is small. However, it does not offer a solution in which high purity target cells can be separated when those small density differences are also taken into consideration with the channel geometry, flow profile, defects caused by the microfluidic chip produced by microfabrication in practice. On the other hand, and most importantly, certain cell groups (e.g., rare circulating cancer cells) can be detected in a wide density range due to their heterogeneous structure and it becomes impossible to differentiate those cells since this may correspond to the heights where other cells are in the channel. Likewise, other undesirable cell groups will also be collected with those cells, and this will reduce the separation efficiency and cause background noise. Therefore, although density alone is an important biophysical marker used to differentiate cell groups from one another, this approach does not work with high efficiency, especially for separating rare cells in blood with single cell sensitivity in magnetic levitation. On the other hand, the system has a completely passive architecture: it is possible to separate cells of height (density) corresponding to the separator aligned to the end of the channel be design of microfluid chip. However, in case the density of a different cell group is not sufficient to align the cells at a height corresponding to the outlet channel, they will be separated along with other cells in the sample. Therefore, it is not possible to actively direct certain target cells from the images to a different channel by exploiting the density and detecting it independently.

Brief Description and Objectives of the Invention

The current invention is about an artificial neural network used by training and active separation of target cells under flow in this direction based on cell morphology-based learning to detect and actively separate target cells, which enables cell groups to differentiate when their density is close or the same in the conventional magnetic levitation system, which is used for the separation of cells in biological fluid samples based on density differences, which meets the abovementioned requirements eliminates the disadvantages and brings some additional advantages.

Artificial neural network assisted active decomposition method based on density and morphology allows separation of target/rare cells with high efficiency and specificity with magnetic levitation. This system is associated with a product which can be used in the diagnosis of various diseases by detecting and separating cell groups supported by deep learning in different cell-based clinical applications in hospitals or as a point-of-care test (POC) quickly, precisely, and cost-effectly.

With the invention, a hybrid system is presented in which the purity of the separated cells is increased by a multilayer enrichment by enabling morphological detection independently of cells aligned proportionally to their density at different heights in the deep learning and magnetic levitation system and actively directing the cells which are aligned with the density differences primarily to the different outputs at the end of the channel by detecting them with the trained artificial neural network.

Diamagnetic cells in a paramagnetic solution which tend to move towards a low magnetic field in the magnetic levitation principle align in the region where the magnetic field is low. This alignment occurs at a heigh unique to each cell type where buoyancy and gravity are balanced as a result of the densities of the cells. Thus, when flowing the cells in the paramagnetic solution-stirred sample while flowing through the platform, denser cells line up at the bottom and less dense cells at the top of the channels. However, heterogeneous cell populations (e.g., rare cancer cells) can mix with other cell groups by being aligned at different heights on the vertical axis in the channel since they are found in wide density range. At this point, when a part of the target cell group is lost without external intervention in a passive system, many foreign cells mix with the target cell group on the other hand. However, the artificial neural network, which is trained to detect target cells in the invention, morphologically detects other target cells with different densities which may have mixed with other cells, apart from the target cell group flowing towards the ouetlet channels with the density difference and actively controlling the flow between the output channels, those cells are also separated together with other target cells, thus maximizing the separation efficiency. Therefore, the target cells detected by the neural network on a morphological basis are obtained with a hybrid separation hierarchy under active flow on the magnetic levitation platform in addition to density, which is an important biophysical marker, without using any staining method.

Descriptions of Figures Explaining the Invention

Figure 1: Schematic representation of the magnetic levitation platform with microfluidic on- chip valves for active and hybrid density- and morphology-based separation of rare cells.

Figure 2: (a) The images after balancing the cell groups with different densities in the microfluidic chip on the magnetic levitation platform, (b) fluorescent images of cancer cells, (c) Microscope images at lOx magnification of cells morphologically detected by the neural network, even when aligned between the other cell group due to their wide density distribution. Definitions of Elements/Parts/Components Constituting the Invention

The elements/parts/components in the figures issued to better explain the control elements and microfluidic chip integrated into the magnetic levitation platform developed with this invention are provided below.

1. Lower magnet

2. Upper magnet

3. Microfluidic chip

4. Valve

5. Pump

6. Bracket

7. Microfluidic channel

A: Heterogeneous cell solution

B : Reservoir

C: Decomposed rare cells

D: Red blood cell

E: Rare blood cell

Detailed Description of the Invention

The invention, magnetic levitation platform detecting and differentiating cells contains (Figure 1) two neodymium (NdFeB) magnets, the lower magnet (1) and the upper magnet (2), arranged with the same poles facing each other, bracket (6) directing the advancing cells in the paramagnetic solution to different outlets in the main microfluidic channel, microfluidic chip (3), which contains at least two microfluidic channels (7) separated in different directions by a bracket (6) for the active separation of one or several cell types among the detected cells, at least two pneumatic valves (4) directing the cells to each microfluidic channel (7) with bracket (6) so that the detected cells can be separated in real time and selectively, and at least one pump (5) working in sync with the valves (4) integrated in the channels of the microfluidic chip so that the detected cells can be separated. The magnetic levitation platform of the invention is integrated with deep convolutional neural networks and/or artificial neural networks. The assembly elements are attached to the body, which is removed from the 3D printer as a whole. The microfluidic chip (3) contains at least one input and at least two outputs directed to different microfluidic channels (7) with brackets (6). The pneumatic valves (4) are integrated to control the flow in each channel and are controlled by the automation system. The pump (5) is used to inject the heterogeneous cell solution into the microfluidic channel (7) inlet and is controlled by the automation system.

The invention includes the following processing steps for the detection and separation of cells in real time under flow or still images with a magnetic levitation platform:

• Obtaining images by observing the detection zone on the microfluidic chip integrated into the magnetic levitation platform through a microscope;

• Feeding images to a deep learning-based objected detection algorithm trained on different cell types, including target cells;

• Drawing bounding boxes around the cells through determining the cell positions since they detected the cells in the trained model image content above the user-defined confidence threshold;

• Additionally, indicating the predicted classes in the bounding boxes for each detected cell;

• After determining the positions and classes of cells in the image, opening the valve controlling the flow at the outlet where the target cell is separated from the microfluidic channels at the end of the channel in case any of those classes are among the cell types it is intended to separate, closing the valve of the waste channel to which other cells are diverted in case of no detection, and thus directing the target cells together with the flow from the pump to the reservoir outlet where they are collected;

• Directing all cells to the waste outlet with flow from the pump in case of no target cell and directing all cells to the waste outlet while remaining open unless the valve controlling the flow in that channel has a detection of the target cell.

The randomly dissolving particles in the microfluidic chip move towards an equilibrium height under the magnetic field parallel to gravity. Balance height of a particle on the magnetic levitation platform depends on the balance between the magnetic force (Fmag) and the buoyancy force (Fb) acting on the particle. Objects placed in the same paramagnetic solution reach different characteristic balance heights under magnetic levitation due to magnetic susceptibility or density differences (Equation 1). (Equation 1)

(A%: magnetic susceptibility difference between particle and paramagnetic solution, pO: permeability of free space (1.2566 x 10-6 kg m A - 2 s - 2), B: magnetic induction, g: gravitational acceleration in the z direction (9.8 ms - 2), Ap: density difference between particle and paramagnetic medium)

Cell sample mixed with paramagnetic solution is enabled to progress in the channel by being transfused from the inlet of the microfluidic channel. The cells are aligned on the perpendicular axis within the channel while the mixture is moving through the pump in the channel as a result of the lifting force and gravity created by the magnetic field created by the magnets located above and below the channel and balancing of the gravity in connection with the self-mass of the cells. However, the cells with close or overlapping density values will orient with the flow towards the same outlet at the end of the channel since they will align at similar heights. Thus, a trained neural network detects the target cell(s) in the microfluidic channel image in real time and when the automation system detects that the target cell is detected by controlling the valves connected to the channel inlets and outlets in a synchronous manner, it ensures that those cells are directed to a separate outlet channel and all other cells flow into the reservoir (waste) channel (Figure 2a). Accordingly, target cells should be introduced to the deep convolutional neural network first. Images of the target and different cell groups are recorded in the static and/or flow state within the microfluidic channel and a label is assigned by drawing bounding boxes around each cell in those images. Verifications can be conducted with various staining for validation of target cell groups if necessary for this process; however, this staining step is only for validation in the preparation of the training set (Figure 2b) and detection is conducted from images of cells flowing in the channel in real time without staining during the operation of the neural network in real conditions. This label refers to the cell type (e.g., breast cancer, red blood cell, leukocyte, etc.). This bounding box drawing and labelling is conducted for each cell type and for all images which will be used in training the neural network. After the labelling process is completed, the related pictures and the file with the bounding box coordinates corresponding to each cell in the images and the labels describing the cell type are used for training the deep neural network. For example, the weight parameters obtained after at least 3000 iterations are stored for training and those parameters are used to detect cells by reloading the neural network architecture under real operating conditions for separation (Figure 2c).

The disposable microfluidic chip was fabricated through microfabrication with poly dimethylsiloxane (PDSM) elastomer. Heterogeneous cell solution is injected with an integrated pump into at least one inlet of this microfluidic channel and flow directions are controlled by valves integrated in at least two outlet channels which are directed to different microfluidic channels with brackets. This chip is placed in the magnetic levitation system and the valve and pump connections integrated into the system are connected to the inlet/outlet ports with hoses. After the flow is started, the automation system controls the pump and valves based on the detected cells (reliability value) by feeding the images into the neural network in real time. Thus, when the target cell is detected (high reliability calculated for this cell class), the valves connected to the channel outlet are automatically opened and closed to ensure the separation of cells to redirect those cells to a different output. Alternatively, a pump can be connected to each outlet of the microfluidic channel. In this case, while the mixture is drawn into the channel by those pumps, the pump connected to that output can start and other pumps can be stopped to direct cells to the desired output. Thus, the targets cells which are detected in rare amounts from a highly heterogeneous sample such as blood are separated by a multi-layer purification system and a hybrid method which actively uses an artificial neural network.

Therefore, the difficulties encountered in the density measurement technique conventionally used in magnetic levitation have also been eliminated in this separation technique. This constraint is to calculate the distance of the cell’s centre of gravity to the upper comer of the lower magnet basically to determine the density of each cell. The height values measured from images through the density equation previously obtained by calibrating with microparticles of different density and different paramagnetic solution concentrations can be used to convert cells to density values. Although the method makes it possible to calculate the density of a single cell with precision, firstly, density calculation alone does not provide physical separation of cells, and secondly, the images should be pre-processed aligning the magnet to the horizontal position, and then determining the height of each particle and calculating the density values with the help of the equation for this process.