CROSS-REFERENCE

[0001] This application claims the benefit of U.S. Application No. 63/417,481, filed October 19, 2022, which is incorporated by reference herein in its entirety.

BACKGROUND

[0002] Drugs, diets, behavioral modification programs, and any other health interventions may not work ubiquitously. Individuals may vary at the genetic, biochemical, physiologic, clinical, behavioral and exposure levels in ways that impact their responses to health interventions. This individual variance means two individuals may undergo the same intervention for the same indication but experience differing impacts and overall outcome.

SUMMARY

[0003] In an aspect, the present disclosure provides methods, systems, software, and apparatus for determining whether a single target individual of interest responds to a health intervention from a whole-body or multivariate perspective (i.e., many measures of health) perspective, as well as determining interventions for which a target individual may be likely to have a whole-body benefit. Note, however, that methods and systems of the present disclosure can be used in studies exploring a single health measure or primary endpoint. Further, the methods and systems of the present disclosure may comprise aggregating data across a set of target individuals in a database to enable the refinement of predictions about which target individuals may benefit from different interventions. The methods and systems of the present disclosure can be used with data collected in any setting in which individual responses are of importance and whose aggregated analysis may be used to make broad inferences about the nature of the interventions, such as with cell lines, mice, plants, and other non-human species, or even in mechanical settings in which the intervention can be administered in doses (e.g., the amount of lubricant applied to an engine). In addition, methods and systems of the present disclosure can be used to assess combinations of interventions.

[0004] The methods and systems of the present disclosure may account for phenomena that adversely affect studies on individuals if not accounted for properly and, if studied further, may provide insight into novel intervention targets and intervention effects. Examples may include, but are not limited to, the following: 1. A need to optimize the number of doses considered and the order in which they are provided; 2. Optimizing the number of measurements collected for each response measure for each dosing period; 3. Optimizing the number of response measures to capture multiple health effects of the intervention; 4. Modeling the dose (or time or covariate) response relationship appropriately; 5. Accommodating time, learning, and covariate effects on the response measures; 6. Serial correlation among the measures collected on an individual; 7. The use of multiple measures of response and possible correlations among those measures; 8. Inflated type 1 error associated with testing an intervention’s effects on more than one variable in a single study; 9. Accounting for correlations among the response measures; 10. Accounting for serial correlation among the measurements for any one response measure; 11. Accommodating nuanced error distributions, outlying values, missing values, and non-uniform time intervals among response measurements; 12. Controlling for false positive and false negative results of studies; and 13. Accommodating causal relationships among measures collected during a study on an individual (See FIG. 1).

[0005] In an aspect, the present disclosure provides a method for determining a response of a target subject to an intervention, comprising: (a) for each dosing time period from a set of dosing time periods, administering a dose of the intervention to the target subject during the dosing time period, wherein the dose of the intervention varies across the set of dosing time periods; and (b) determining the response of the target subject to the intervention based at least in part on a set of response measures.

[0006] In some embodiments, the target subject is an organism, and the intervention comprises a clinical intervention. In some embodiments, the organism is a human.

[0007] In some embodiments, (a) comprises administering a single dose of the intervention to the target subject during a single dosing time period. In some embodiments, (a) comprises, for each dosing time period from a plurality of dosing time periods, administering a dose of the intervention to the target subject during the dosing time period, wherein the dose of the intervention varies across the plurality of dosing time periods.

[0008] In some embodiments, (b) comprises determining a response measure of the target subject responsive to at least one of the set of doses of the intervention being administered to the target subject. In some embodiments, (b) comprises determining a response measure of the subject responsive to each of the set of doses of the intervention being administered to the subject. In some embodiments, the response measures of the set of response measures comprise a number of individual measurements.

[0009] In some embodiments, the method further comprises determining a desired number of individual measurements corresponding to the set of response measures. In some embodiments, the method further comprises determining a desired number of doses or a random number of doses corresponding to the set of dosing time periods. In some embodiments, the method further comprises determining a desired order of doses or a random order of doses corresponding to the set of dosing time periods. In some embodiments, the method further comprises determining a desired timing of doses or a random timing of doses corresponding to the set of dosing time periods.

[0010] In some embodiments, the determined response of the target subject to the intervention comprises a whole-body response of the target subject. In some embodiments, the determined response of the target subject to the intervention comprises a multi-system response of the target subject. In some embodiments, the determined response of the target subject to the intervention comprises a multivariate response of the target subject. In some embodiments, the method further comprises storing the determined response of the target subject to the intervention in a database.

[0011] In some embodiments, the method further comprises training a dose response model of the intervention for the target subject using the determined response of the target subject to the intervention. In some embodiments, the dose response model accounts for time effects, learning effects, covariate effects, or serial correlation effects, of the set of response measures. In some embodiments, the dose response model accounts for correlations between response measures among the set of response measures. In some embodiments, the dose response model accounts for false-positive or false-negative results. In some embodiments, the dose response model accounts for nuanced error distributions, outlying values, missing values, or non-uniform time intervals among the set of response measures. In some embodiments, the dose response model accounts for causal relationships between response measures among the set of response measures.

[0012] In some embodiments, the intervention may be compared between a case group and a control group.

[0013] In some embodiments, the target subject has or is suspected of having a disease or disorder, and the intervention is configured to treat or ameliorate the disease or disorder. In some embodiments, the disease or disorder is selected from the group consisting of allergic, articular, bone, cardiovascular, dermatologic, endocrinologic, gastrointestinal, gynecologic, hematologic, immunologic, infectious, metabolic, neurologic, obstetric, ophthalmic, otolaryngologic, pulmonary, psychiatric, renal, rheumatologic, urinary, and vascular disease or disorder, cancer, and benign tumor.

[0014] In some embodiments, the clinical intervention is selected from the group consisting of a medication, a cell-based or gene therapy, a drug treatment, a medical device, a surgical intervention, a radiotherapy, radioisotopic/nuclear therapy, physical therapy, occupational therapy, phonoaudiological therapy, a rehabilitation intervention, a psychological intervention, an immunotherapy, a digital health intervention, and a behavioral intervention. In some embodiments, the clinical intervention comprises the drug treatment. In some embodiments, the drug treatment comprises an approved drug treatment. In some embodiments, the drug treatment comprises an experimental drug treatment. In some embodiments, the drug treatment comprises an off-label drug treatment.

[0015] In some embodiments, the response measures of the set of response measures comprise discrete variables, continuous variables, ordinal variables, or time-to-event variables. [0016] In some embodiments, the response measures of the set of response measures comprise a member selected from the group consisting of a chemical biomarker, a genomic biomarker, an epigenomic biomarker, a gene expression biomarker, a protein biomarker, a metabolite biomarker, a clinical test result for a disease, event-free survival time, progression- free survival time, overall survival time, another time to event, efficacy, safety, quality of life, functional or performance score, toxicity grade, behavioral score or assessment, exposure score or assessment, assessment of symptoms, assessment of side effects, vital sign measurements, or a combination thereof. In some embodiments, the vital sign measurements comprise one or more measurements selected from the group consisting of heart rate, blood pressure, blood oxygen concentration, a hormone level, sweat analysis, blood glucose, body temperature, impedance, conductivity, capacitance, resistivity, electromyography, galvanic skin response, and immunology markers.

[0017] In some embodiments, at least one of the set of response measures is obtained at least in part by performing a biomarker test on the target subject. In some embodiments, the biomarker test comprises a laboratory test selected from the group consisting of biochemistry, hematology, coagulation, microbiology, molecular genetics, cytogenetics, flow cytometry, pathology, radiology or imaging, and diagnostic, prognostic, predictive, and surrogate biomarkers, a blood test, a urine test, a genetic test, and a combination thereof. In some embodiments, the radiology or imaging laboratory test comprises X-ray, fluoroscopy, computed tomography, magnetic resonance imaging, ultrasound, echocardiography, positron-emission tomography, single-photon emission tomography, radionuclide imaging, optic coherence tomography, electrocardiography, electroencephalography, or electromyography.

[0018] In some embodiments, the method further comprises comparing the determined response of the target subject to the intervention with a reference health profile. In some embodiments, the method further comprises determining a difference between the determined response of the target subject to the intervention and the reference health profile. In some embodiments, the method further comprises comparing the difference to a clinical threshold. In some embodiments, the reference health profile is determined from the target subject prior to receiving the intervention. In some embodiments, the reference health profile is determined from a set of case subjects having a disease or disorder, or a set of control subjects not having the disease or disorder.

[0019] In some embodiments, the method further comprises comparing each of the set of response measures to a reference set of response measures. In some embodiments, the method further comprises comparing the set of response measures with each other, and scoring or ranking the set of doses of the intervention based at least in part on comparing the set of response measures with each other.

[0020] In some embodiments, the determined response of the target subject to the intervention comprises a single endpoint. In some embodiments, the determined response of the target subject to the intervention comprises a plurality of endpoints.

[0021] In some embodiments, the method further comprises selecting a dose of the intervention corresponding to a dosing time period from among the set of dosing time periods, to be administered or provided to a second target subject, based at least in part on the determined response of the target subject to the intervention. In some embodiments, the dose is selected based at least in part on a similarity of characteristics between the target subject and the second target subject. In some embodiments, the characteristics comprise clinical characteristics or demographical characteristics. In some embodiments, the method further comprises prescribing or administering the selected dose of the intervention to the second target subject.

[0022] In some embodiments, the clinical intervention is part of a clinical trial.

[0023] In some embodiments, (b) further comprises performing a multivariate analysis, such as a multivariate distance matrix regression (MDMR) analysis, on the set of response measures. In some embodiments, the MDMR analysis further comprises constructing a distance matrix, wherein the distance matrix comprises distances between the set of response measures at each of the set of dosing time periods. In some embodiments, the distance matrix comprises binary distances. In some embodiments, the distance matrix comprises Euclidean distances or other distance measures.

[0024] In some embodiments, the method further comprises ranking a set of analytes or species, based at least in part on values or abundances of the set of analytes or species.

[0025] In some embodiments, (b) further comprises performing a factor analysis, a cluster analysis, or a latent roots analysis on the set of response measures.

[0026] In another aspect, the present disclosure provides a method for determining an individualized intervention for a target subject, comprising: (a) obtaining a set of characteristics of the target subject; and (b) querying a database using the set of characteristics of the target subject to determine the individualized intervention for the target subject, wherein the database comprises trial data for a set of test subjects obtained at least in part by, for each test subject of the set of test subjects: (i) for each dosing time period from a set of dosing time periods, administering a dose of the intervention to the test subject during the dosing time period, wherein the dose of the intervention varies across the set of dosing time periods, and (ii) determining a response of the test subject to the intervention based at least in part on a set of response measures.

[0027] In some embodiments, the target subject is an organism, and the intervention comprises a clinical intervention. In some embodiments, the organism is a human.

[0028] In some embodiments, in (i), a single dose of the intervention is administered to the test subject during a single dosing time period. In some embodiments, in (i), for each dosing time period from a plurality of dosing time periods, a dose of the intervention is administered to the test subject during the dosing time period, wherein the dose of the intervention varies across the plurality of dosing time periods.

[0029] In some embodiments, in (ii), a response measure of the test subject is determined responsive to at least one of the set of doses of the intervention being administered to the test subject. In some embodiments, in (ii), a response measure of the subject is determined responsive to each of the set of doses of the intervention being administered to the subject.

[0030] In some embodiments, the response measures of the set of response measures comprise a number of individual measurements. In some embodiments, a desired number of individual measurements corresponding to the set of response measures is determined. In some embodiments, a desired number of doses or a random number of doses corresponding to the set of dosing time periods is determined. In some embodiments, a desired number of doses or a random number of doses corresponding to the set of dosing time periods is determined. In some embodiments, a desired timing of doses or a random timing of doses corresponding to the set of dosing time periods is determined.

[0031] In some embodiments, the determined response of the test subject to the intervention comprises a whole-body response of the test subject. In some embodiments, the determined response of the test subject to the intervention comprises a multi-system response of the test subject. In some embodiments, the determined response of the test subject to the intervention comprises a multivariate response of the test subject.

[0032] In some embodiments, determining the individualized intervention for the target subject further comprises selecting the individualized intervention from among a plurality of candidate interventions for the target subject. In some embodiments, determining the individualized intervention for the target subject further comprises using a dose response model of the intervention, wherein the dose response model of the intervention is trained using the determined responses of the set of test subjects to the intervention. In some embodiments, the dose response model accounts for time effects, learning effects, covariate effects, or serial correlation effects, of the set of response measures. In some embodiments, the dose response model accounts for correlations between response measures among the set of response measures. In some embodiments, the dose response model accounts for false-positive or false-negative results. In some embodiments, the dose response model accounts for nuanced error distributions, outlying values, missing values, or non-uniform time intervals among the set of response measures. In some embodiments, the dose response model accounts for causal relationships between response measures among the set of response measures.

[0033] In some embodiments, the intervention may be compared between a case group and a control group.

[0034] In some embodiments, the target subject has or is suspected of having a disease or disorder, and the intervention is configured to treat or ameliorate the disease or disorder. In some embodiments, the disease or disorder is selected from the group consisting of allergic, articular, bone, cardiovascular, dermatologic, endocrinologic, gastrointestinal, gynecologic, hematologic, immunologic, infectious, metabolic, neurologic, obstetric, ophthalmic, otolaryngologic, pulmonary, psychiatric, renal, rheumatologic, urinary, and vascular disease or disorder, cancer, and benign tumor.

[0035] In some embodiments, the clinical intervention is selected from the group consisting of a medication, a cell-based or gene therapy, a drug treatment, a medical device, a surgical intervention, a radiotherapy, radioisotopic/nuclear therapy, physical therapy, occupational therapy, phonoaudiological therapy, a rehabilitation intervention, a psychological intervention, an immunotherapy, a digital health intervention, and a behavioral intervention. In some embodiments, the clinical intervention comprises the drug treatment. In some embodiments, the drug treatment comprises an approved drug treatment. In some embodiments, the drug treatment comprises an experimental drug treatment. In some embodiments, the drug treatment comprises an off-label drug treatment.

[0036] In some embodiments, the response measures of the set of response measures comprise discrete variables, continuous variables, ordinal variables, or time-to-event variables. [0037] In some embodiments, the response measures of the set of response measures comprise a member selected from the group consisting of a chemical biomarker, a genomic biomarker, an epigenomic biomarker, a gene expression biomarker, a protein biomarker, a metabolite biomarker, a clinical test result for a disease, event-free survival time, progression- free survival time, overall survival time, another time to event, efficacy, safety, quality of life, functional or performance score, toxicity grade, behavioral score or assessment, exposure score or assessment, assessment of symptoms, assessment of side effects, vital sign measurements, or a combination thereof. In some embodiments, the vital sign measurements comprise one or more measurements selected from the group consisting of heart rate, blood pressure, blood oxygen concentration, a hormone level, sweat analysis, blood glucose, body temperature, impedance, conductivity, capacitance, resistivity, electromyography, galvanic skin response, and immunology markers.

[0038] In some embodiments, at least one of the set of response measures is obtained at least in part by performing a biomarker test on the set of test subjects. In some embodiments, the biomarker test comprises a laboratory test selected from the group consisting of biochemistry, hematology, coagulation, microbiology, molecular genetics, cytogenetics, flow cytometry, pathology, radiology or imaging, and diagnostic, prognostic, predictive, and surrogate biomarkers, a blood test, a urine test, a genetic test, and a combination thereof. In some embodiments, the radiology or imaging laboratory test comprises X-ray, fluoroscopy, computed tomography, magnetic resonance imaging, ultrasound, echocardiography, positron-emission tomography, single-photon emission tomography, radionuclide imaging, optic coherence tomography, electrocardiography, electroencephalography, or electromyography.

[0039] In some embodiments, the determined response of the set of test subjects to the intervention are compared with a reference health profile. In some embodiments, a difference is determined between the determined response of the set of test subjects to the intervention and the reference health profile. In some embodiments, the difference is compared to a clinical threshold. In some embodiments, the reference health profile is determined from the set of test subjects prior to receiving the intervention. In some embodiments, the reference health profile is determined from a set of case subjects having a disease or disorder, or a set of control subjects not having the disease or disorder.

[0040] In some embodiments, each of the set of response measures is compared to a reference set of response measures. In some embodiments, the set of response measures is compared with each other, and the set of doses of the intervention are scored or ranked based at least in part on comparing the set of response measures with each other.

[0041] In some embodiments, the determined response of the test subject to the intervention comprises a single endpoint. In some embodiments, the determined response of the test subject to the intervention comprises a plurality of endpoints.

[0042] In some embodiments, the method further comprises selecting a dose of the intervention to be administered or provided to the target subject, based at least in part on the determined response of the set of test subjects to the intervention. In some embodiments, the intervention and/or a dose of the intervention is determined for the target subject, based at least in part on a degree of similarity or dissimilarity of characteristics between the target subject and the set of test subjects. In some embodiments, the characteristics comprise clinical characteristics or demographical characteristics. In some embodiments, the method further comprises prescribing or administering the selected dose of the intervention to the target subject.

[0043] In some embodiments, the clinical intervention is part of a clinical trial.

[0044] In some embodiments, (ii) further comprises performing a multivariate analysis, such as a multivariate distance matrix regression (MDMR) analysis, on the set of response measures. In some embodiments, the MDMR analysis further comprises constructing a distance matrix, wherein the distance matrix comprises distances between the set of response measures at each of the set of dosing time periods. In some embodiments, the distance matrix comprises binary distances. In some embodiments, the distance matrix comprises Euclidean distances or other distance measures.

[0045] In some embodiments, the method further comprises ranking a set of analytes or species, based at least in part on values or abundances of the set of analytes or species.

[0046] In some embodiments, (ii) further comprises performing a factor analysis, a cluster analysis, or a latent roots analysis on the set of response measures.

[0047] Another aspect of the present disclosure provides a non-transitory computer-readable medium comprising machine-executable code that, upon execution by one or more computer processors, implements any of the methods herein.

[0048] Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine-executable code that, upon execution by the one or more computer processors, implements any of the methods herein.

[0049] Additional aspects and advantages of the present disclosure may become readily apparent to those skilled in this art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As may be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

INCORPORATION BY REFERENCE

[0050] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and/or take precedence over any such contradictory material.

BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention may be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also “figure” and “FIG.” herein), of which:

[0052] FIG. 1 depicts an example of an N-of-1 trial exploring 5 doses of an intervention with 7 measurements on each of 5 different response measures (the colored lines) during each dosing period.

[0053] FIG. 2 provides an example of a descriptive depiction, or schematic, of the flow of information associated with methods and systems of the present disclosure.

[0054] FIGs. 3A-3D provide an example of a simple simulated two period N-of-1 clinical trial design, which may be referred to as an “interrupted time series”. FIG. 3A is a scatter plot of a simulated ‘interrupted time series’ clinical trial with a baseline (0.0 intervention dose) and intervention (intervention dose > 0.0) period and one response measure. FIG. 3B is a box plot of the measurements made during each period and reflects a clear intervention effect. FIG. 3C is a scatter plot of a simulated interrupted time series clinical trial with no intervention effect, but a strong learning or time effect (slope = 0.1). FIG. 3D is a box plot of the measurements made provided in the lower left panel and gives the impression of an intervention effect, but actually simply reflects the time effect.

[0055] FIGs. 4A-4B provide an example of a simple two period N-of-1 clinical trial design, which may be referred to as an “interrupted time series”, with multiple response measures. FIG. 4A is a box plot of 5 different simulated response measures (‘Pain,’ ‘Mood,’ ‘Sleep,’ ‘Blood Pressure (BP)’, and a ‘Biomarker,’ (dl-d5) collected in an interrupted time series design where the intervention had an effect size of 2.0 (SDUs) on each response measure.

[0056] FIG. 5A depicts a response measures’ increasing values over time in which 7 consecutive measurements were collected during 1 of 6 different doses of an intervention. These doses were provided sequentially in increasing increments.

[0057] FIG. 5B is a box plot of the measures depicted in FIG. 5A for each dose. Since the doses were provided in increasing increments there is an overall increase in average response measures over time. [0058] FIG. 5C depicts 56 response measure values collected over time in 8 different periods in which 1 of 6 different doses of an intervention were provided. Since the doses were provided in non-increasing increments (see labels at the top of the figure), there is no increase in response values over time.

[0059] FIG. 5D is a box plot depicting the average response measure values provided in the FIG. 5C as a function of increasing doses of the intervention. Despite the fact that there is no overall increase in response measures over time (FIG. 5C), there is an obvious dose effect (FIG. 5D)

[0060] FIG. 5E depicts 56 response measure values collected over time in which 7 consecutive measures were collected during 1 of 6 dose periods. There is an obvious time effect. [0061] FIG. 5F is a box plot depicting the average response measures in FIG. 5E as a function of doses. Since the doses were not provided in increasing increments (note the labels indicating the time frames for each dose period at the top of the figure), there is no obvious dose effect.

[0062] FIGs. 6A-6D depict simulated data in which 5 different response measures were collected at 7 time points within 5 different dose periods for a total of 35 response measures. FIG. 6A depicts a scenario in which all 5 of the response measures exhibit a dose effect, but the doses were provided in increasing increments (labeled at the bottom of the panel) and as such may be confounded with time. FIG. 6B depicts a scenario in which the dosages have not been provided in increasing increments (labeled at the bottom of the panel) and hence do not exhibit a time effect but do exhibit a dose effect. FIG. 6C depicts a scenario in which only 4 of the response measures exhibit a dose effect, but since the doses were provided in increasing increments they are confounded with time. FIG. 6D depicts a scenario in which only 2 of the response measures exhibit a dose effect, but since the doses were provided in increasing increments they are confounded with time.

[0063] FIG. 7 provides plots of the power of a linear model as a function of effect size (the slope or regression coefficient for variables with a N(0,l) error distribution). The linear model includes a time (or learning) effect and a dose effect.

[0064] FIG. 8 depicts power curves for detecting different dose effect sizes (the effect size reflects the slope of response measures regressed on increasing doses) estimated from 1000 simulations for different parameter values for N-of-1 designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics.

[0065] FIG. 9 depicts power curves for detecting different dose effect sizes (the effect size reflects the slope of response measures regressed on increasing doses) estimated from 1000 simulations for different parameter values for N-of-1 designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics.

[0066] FIG. 10 depicts power curves for detecting different dose effect sizes (the effect size reflects the slope of response measures regressed on increasing doses) estimated from 1000 simulations for different parameter values for N-of-l designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics.

[0067] FIGs. 11A-11B depict power curves for the simple “interrupted time series” under different parameter assumptions, including detecting different dose effect sizes (the effect size reflects the slope of response measures regressed on increasing doses) estimated from 1000 simulations for different parameter values for N-of-1 designs assuming 2 doses (including a 0 dose and >0 dose, often referred to as ‘interrupted time series’ designs) and 5 response measures for different test statistics.

[0068] FIGs. 12A-12B depict power curves for studies using ranked values of the response measures, including detecting different dose effect sizes (the effect size reflects the slope of response measures regressed on increasing doses) estimated from 1000 simulations for different parameter values for N-of-1 designs.

[0069] FIG. 13 provides an example graphical representation of the concept of matching a target individual to individuals whose responses to interventions were studied in N-of-1 trials of the present disclosure based on a baseline health assessment (a ‘meta-physical’). A goal is to identify interventions that had a positive benefit on individuals with a similar health profile to the target individual.

[0070] FIG. 14 depicts an example of ongoing evaluation of the frequency of responses to various interventions tested via the N-of-1 trials of the present disclosure that are recorded in a ‘Database of What Works (DOWW),’ as an example.

[0071] FIG. 15 shows a computer system 1501 that is programmed or otherwise configured to perform analyses of the methods.

[0072] FIG. 16 provides an estimate of the power of standard MDMR F-statistic tests based on 1,000 simulations for each setting involving a different assumed number of analytes (red (right-most) = 1 analytes and/or species, green (second right-most) = 5 analytes and/or species, blue (second left-most) = 50 analytes and/or species, and black (left-most) = 500 analytes and/or species), for an effect size in standard deviation units between 0.0 to 2.0 in increments of 0.02. [0073] FIG. 17 demonstrates the estimated power of analyses in settings like those considered in FIG. 16, but with analytes and/or species added to the analysis that are not affected by the intervention (e.g., noise). [0074] FIG. 18 demonstrates the estimated power of analyses in settings like those considered in FIG. 16 and FIG. 17, but with 100 total measurements at each time point, and with different fractions of the 100 variables actually being associated with the intervention.

[0075] FIG. 19 shows results of data simulations as with FIGs. 16-18, but no noise was considered and 50 total simulated analytes and/or species were measured at the 20 timepoints.

[0076] FIG. 20 shows results of data simulations with the same general settings as FIGs. 15- 19, but focus was on the impact of serial correlation (from 0 to 1, as reflected on the x axis).

DETAILED DESCRIPTION

[0077] While various embodiments of the invention have been shown and described herein, it may be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.

[0078] As used in the specification and claims, the singular form “a”, “an”, and “the” include plural references unless the context clearly dictates otherwise. For example, the term “an intervention” includes a plurality of interventions.

[0079] In an aspect, the present disclosure provides methods, systems, software, and apparatus for determining whether a single target individual of interest responds to a health intervention from a whole-body or multivariate perspective (i.e., many measures of health) perspective, as well as determining interventions for which a target individual may be likely to have a whole-body benefit. Note, however, that methods and systems of the present disclosure can be used in studies exploring a single health measure or primary endpoint. Further, the methods and systems of the present disclosure may comprise aggregating data across a set of target individuals in a database to enable the refinement of predictions about which target individuals may benefit from different interventions. The methods and systems of the present disclosure can be used with data collected in any setting in which individual responses are of importance and whose aggregated analysis may be used to make broad inferences about the nature of the interventions, such as with cell lines, mice, plants, and other non-human species, or even in mechanical settings in which the intervention can be administered in doses (e.g., the amount of lubricant applied to an engine). In addition, methods and systems of the present disclosure can be used to assess combinations of interventions.

[0080] The methods and systems of the present disclosure may account for phenomena that adversely affect studies on individuals if not accounted for properly and, if studied further, may provide insight into novel intervention targets and intervention effects. Examples may include, but are not limited to, the following: 1. A need to optimize the number of doses considered and the order in which they are provided; 2. Optimizing the number of measurements collected for each response measure for each dosing period; 3. Optimizing the number of response measures to capture multiple health effects of the intervention; 4. Modeling the dose (or time or covariate) response relationship appropriately; 5. Accommodating time, learning, and covariate effects on the response measures; 6. Serial correlation among the measures collected on an individual; 7. The use of multiple measures of response and possible correlations among those measures; 8. Inflated type 1 error associated with testing an intervention’s effects on more than one variable in a single study; 9. Accounting for correlations among the response measures; 10. Accounting for serial correlation among the measurements for any one response measure; 11. Accommodating nuanced error distributions, outlying values, missing values, and non-uniform time intervals among response measurements; 12. Controlling for false positive and false negative results of studies; and 13. Accommodating causal relationships among measures collected during a study on an individual (See FIG. 1).

[0081] In an aspect, the present disclosure provides a method for determining a response of a target subject to an intervention, comprising: (a) for each dosing time period from a set of dosing time periods, administering a dose of the intervention to the target subject during the dosing time period, wherein the dose of the intervention varies across the set of dosing time periods; and (b) determining the response of the target subject to the intervention based at least in part on the set of response measures.

[0082] In some embodiments, the target subject is an organism, and the intervention comprises a clinical intervention. In some embodiments, the organism is a human.

[0083] In some embodiments, (a) comprises administering a single dose of the intervention to the target subject during a single dosing time period. In some embodiments, (a) comprises, for each dosing time period from a plurality of dosing time periods, administering a dose of the intervention to the target subject during the dosing time period, wherein the dose of the intervention varies across the plurality of dosing time periods.

[0084] In some embodiments, (b) comprises determining a response measure of the target subject responsive to at least one of the set of doses of the intervention being administered to the target subject. In some embodiments, (b) comprises determining a response measure of the subject responsive to each of the set of doses of the intervention being administered to the subject. In some embodiments, the response measures of the set of response measures comprise a number of individual measurements.

[0085] In some embodiments, the method further comprises determining a desired number of individual measurements corresponding to the set of response measures. In some embodiments, the method further comprises determining a desired number of doses or a random number of doses corresponding to the set of dosing time periods. In some embodiments, the method further comprises determining a desired order of doses or a random order of doses corresponding to the set of dosing time periods. In some embodiments, the method further comprises determining a desired timing of doses or a random timing of doses corresponding to the set of dosing time periods.

[0086] In some embodiments, the determined response of the target subject to the intervention comprises a whole-body response of the target subject. In some embodiments, the determined response of the target subject to the intervention comprises a multi-system response of the target subject. In some embodiments, the determined response of the target subject to the intervention comprises a multivariate response of the target subject. In some embodiments, the method further comprises storing the determined response of the target subject to the intervention in a database.

[0087] In some embodiments, the method further comprises training a dose response model of the intervention for the target subject using the determined response of the target subject to the intervention. In some embodiments, the dose response model accounts for time effects, learning effects, covariate effects, or serial correlation effects, of the set of response measures. In some embodiments, the dose response model accounts for correlations between response measures among the set of response measures. In some embodiments, the dose response model accounts for false-positive or false-negative results. In some embodiments, the dose response model accounts for nuanced error distributions, outlying values, missing values, or non-uniform time intervals among the set of response measures. In some embodiments, the dose response model accounts for causal relationships between response measures among the set of response measures.

[0088] In some embodiments, the intervention may be compared between a case group and a control group.

[0089] In some embodiments, the target subject has or is suspected of having a disease or disorder, and the intervention is configured to treat or ameliorate the disease or disorder. In some embodiments, the disease or disorder is selected from the group consisting of allergic, articular, bone, cardiovascular, dermatologic, endocrinologic, gastrointestinal, gynecologic, hematologic, immunologic, infectious, metabolic, neurologic, obstetric, ophthalmic, otolaryngologic, pulmonary, psychiatric, renal, rheumatologic, urinary, and vascular disease or disorder, cancer, and benign tumor.

[0090] In some embodiments, the clinical intervention is selected from the group consisting of a medication, a cell-based or gene therapy, a drug treatment, a medical device, a surgical intervention, a radiotherapy, radioisotopic/nuclear therapy, physical therapy, occupational therapy, phonoaudiological therapy, a rehabilitation intervention, a psychological intervention, an immunotherapy, a digital health intervention, and a behavioral intervention. In some embodiments, the clinical intervention comprises the drug treatment. In some embodiments, the drug treatment comprises an approved drug treatment. In some embodiments, the drug treatment comprises an experimental drug treatment. In some embodiments, the drug treatment comprises an off-label drug treatment.

[0091] In some embodiments, the response measures of the set of response measures comprise discrete variables, continuous variables, ordinal variables, or time-to-event variables. [0092] In some embodiments, the response measures of the set of response measures comprise a member selected from the group consisting of a chemical biomarker, a genomic biomarker, an epigenomic biomarker, a gene expression biomarker, a protein biomarker, a metabolite biomarker, a clinical test result for a disease, event-free survival time, progression- free survival time, overall survival time, another time to event, efficacy, safety, quality of life, functional or performance score, toxicity grade, behavioral score or assessment, exposure score or assessment, assessment of symptoms, assessment of side effects, vital sign measurements, or a combination thereof. In some embodiments, the vital sign measurements comprise one or more measurements selected from the group consisting of heart rate, blood pressure, blood oxygen concentration, a hormone level, sweat analysis, blood glucose, body temperature, impedance, conductivity, capacitance, resistivity, electromyography, galvanic skin response, and immunology markers.

[0093] In some embodiments, at least one of the set of response measures is obtained at least in part by performing a biomarker test on the target subject. In some embodiments, the biomarker test comprises a laboratory test selected from the group consisting of biochemistry, hematology, coagulation, microbiology, molecular genetics, cytogenetics, flow cytometry, pathology, radiology or imaging, and diagnostic, prognostic, predictive, and surrogate biomarkers, a blood test, a urine test, a genetic test, and a combination thereof. In some embodiments, the radiology or imaging laboratory test comprises X-ray, fluoroscopy, computed tomography, magnetic resonance imaging, ultrasound, echocardiography, positron-emission tomography, single-photon emission tomography, radionuclide imaging, optic coherence tomography, electrocardiography, electroencephalography, or electromyography.

[0094] In some embodiments, the method further comprises comparing the determined response of the target subject to the intervention with a reference health profile. In some embodiments, the method further comprises determining a difference between the determined response of the target subject to the intervention and the reference health profile. In some embodiments, the method further comprises comparing the difference to a clinical threshold. In some embodiments, the reference health profile is determined from the target subject prior to receiving the intervention. In some embodiments, the reference health profile is determined from a set of case subjects having a disease or disorder, or a set of control subjects not having the disease or disorder.

[0095] In some embodiments, the method further comprises comparing each of the set of response measures to a reference set of response measures. In some embodiments, the method further comprises comparing the set of response measures with each other, and scoring or ranking the set of doses of the intervention based at least in part on comparing the set of response measures with each other.

[0096] In some embodiments, the determined response of the target subject to the intervention comprises a single endpoint. In some embodiments, the determined response of the target subject to the intervention comprises a plurality of endpoints.

[0097] In some embodiments, the method further comprises selecting a dose of the intervention corresponding to a dosing time period from among the set of dosing time periods, to be administered or provided to a second target subject, based at least in part on the determined response of the target subject to the intervention. In some embodiments, the dose is selected based at least in part on a similarity of characteristics between the target subject and the second target subject. In some embodiments, the characteristics comprise clinical characteristics or demographical characteristics. In some embodiments, the method further comprises prescribing or administering the selected dose of the intervention to the second target subject.

[0098] In some embodiments, the clinical intervention is part of a clinical trial.

[0099] In another aspect, the present disclosure provides a method for determining an individualized intervention for a target subject, comprising: (a) obtaining a set of characteristics of the target subject; and (b) querying a database using the set of characteristics of the target subject to determine the individualized intervention for the target subject, wherein the database comprises trial data for a set of test subjects obtained at least in part by, for each test subject of the set of test subjects: (i) for each dosing time period from a set of dosing time periods, administering a dose of the intervention to the test subject during the dosing time period, wherein the dose of the intervention varies across the set of dosing time periods, and (ii) determining a response of the test subject to the intervention based at least in part on the set of response measures.

[00100] In some embodiments, the target subject is an organism, and the intervention comprises a clinical intervention. In some embodiments, the organism is a human. [00101] In some embodiments, in (i), a single dose of the intervention is administered to the test subject during a single dosing time period. In some embodiments, in (i), for each dosing time period from a plurality of dosing time periods, a dose of the intervention is administered to the test subject during the dosing time period, wherein the dose of the intervention varies across the plurality of dosing time periods.

[00102] In some embodiments, in (ii), a response measure of the test subject is determined responsive to at least one of the set of doses of the intervention being administered to the test subject. In some embodiments, in (ii), a response measure of the subject is determined responsive to each of the set of doses of the intervention being administered to the subject.

[00103] In some embodiments, the response measures of the set of response measures comprise a number of individual measurements. In some embodiments, a desired number of individual measurements corresponding to the set of response measures is determined. In some embodiments, a desired number of doses or a random number of doses corresponding to the set of dosing time periods is determined. In some embodiments, a desired number of doses or a random number of doses corresponding to the set of dosing time periods is determined. In some embodiments, a desired timing of doses or a random timing of doses corresponding to the set of dosing time periods is determined.

[00104] In some embodiments, the determined response of the test subject to the intervention comprises a whole-body response of the test subject. In some embodiments, the determined response of the test subject to the intervention comprises a multi-system response of the test subject. In some embodiments, the determined response of the test subject to the intervention comprises a multivariate response of the test subject.

[00105] In some embodiments, determining the individualized intervention for the target subject further comprises selecting the individualized intervention from among a plurality of candidate interventions for the target subject. In some embodiments, determining the individualized intervention for the target subject further comprises using a dose response model of the intervention, wherein the dose response model of the intervention is trained using the determined responses of the set of test subjects to the intervention. In some embodiments, the dose response model accounts for time effects, learning effects, covariate effects, or serial correlation effects, of the set of response measures. In some embodiments, the dose response model accounts for correlations between response measures among the set of response measures. In some embodiments, the dose response model accounts for false-positive or false-negative results. In some embodiments, the dose response model accounts for nuanced error distributions, outlying values, missing values, or non-uniform time intervals among the set of response measures. In some embodiments, the dose response model accounts for causal relationships between response measures among the set of response measures.

[00106] In some embodiments, the intervention may be compared between a case group and a control group.

[00107] In some embodiments, the target subject has or is suspected of having a disease or disorder, and the intervention is configured to treat or ameliorate the disease or disorder. In some embodiments, the disease or disorder is selected from the group consisting of allergic, articular, bone, cardiovascular, dermatologic, endocrinologic, gastrointestinal, gynecologic, hematologic, immunologic, infectious, metabolic, neurologic, obstetric, ophthalmic, otolaryngologic, pulmonary, psychiatric, renal, rheumatologic, urinary, and vascular disease or disorder, cancer, and benign tumor.

[00108] In some embodiments, the clinical intervention is selected from the group consisting of a medication, a cell-based or gene therapy, a drug treatment, a medical device, a surgical intervention, a radiotherapy, radioisotopic/nuclear therapy, physical therapy, occupational therapy, phonoaudiological therapy, a rehabilitation intervention, a psychological intervention, an immunotherapy, a digital health intervention, and a behavioral intervention. In some embodiments, the clinical intervention comprises the drug treatment. In some embodiments, the drug treatment comprises an approved drug treatment. In some embodiments, the drug treatment comprises an experimental drug treatment. In some embodiments, the drug treatment comprises an off-label drug treatment.

[00109] In some embodiments, the response measures of the set of response measures comprise discrete variables, continuous variables, ordinal variables, or time-to-event variables. [00110] In some embodiments, the response measures of the set of response measures comprise a member selected from the group consisting of a chemical biomarker, a genomic biomarker, an epigenomic biomarker, a gene expression biomarker, a protein biomarker, a metabolite biomarker, a clinical test result for a disease, event-free survival time, progression- free survival time, overall survival time, another time to event, efficacy, safety, quality of life, functional or performance score, toxicity grade, behavioral score or assessment, exposure score or assessment, assessment of symptoms, assessment of side effects, vital sign measurements, or a combination thereof. In some embodiments, the vital sign measurements comprise one or more measurements selected from the group consisting of heart rate, blood pressure, blood oxygen concentration, a hormone level, sweat analysis, blood glucose, body temperature, impedance, conductivity, capacitance, resistivity, electromyography, galvanic skin response, and immunology markers. [00111] In some embodiments, at least one of the set of response measures is obtained at least in part by performing a biomarker test on the set of test subjects. In some embodiments, the biomarker test comprises a laboratory test selected from the group consisting of biochemistry, hematology, coagulation, microbiology, molecular genetics, cytogenetics, flow cytometry, pathology, radiology or imaging, and diagnostic, prognostic, predictive, and surrogate biomarkers, a blood test, a urine test, a genetic test, and a combination thereof. In some embodiments, the radiology or imaging laboratory test comprises X-ray, fluoroscopy, computed tomography, magnetic resonance imaging, ultrasound, echocardiography, positron-emission tomography, single-photon emission tomography, radionuclide imaging, optic coherence tomography, electrocardiography, electroencephalography, or electromyography.

[00112] In some embodiments, the determined response of the set of test subjects to the intervention are compared with a reference health profile. In some embodiments, a difference is determined between the determined response of the set of test subjects to the intervention and the reference health profile. In some embodiments, the difference is compared to a clinical threshold. In some embodiments, the reference health profile is determined from the set of test subjects prior to receiving the intervention. In some embodiments, the reference health profile is determined from a set of case subjects having a disease or disorder, or a set of control subjects not having the disease or disorder.

[00113] In some embodiments, each of the set of response measures is compared to a reference set of response measures. In some embodiments, the set of response measures is compared with each other, and the set of doses of the intervention are scored or ranked based at least in part on comparing the set of response measures with each other.

[00114] In some embodiments, the determined response of the test subject to the intervention comprises a single endpoint. In some embodiments, the determined response of the test subject to the intervention comprises a plurality of endpoints.

[00115] In some embodiments, the method further comprises selecting a dose of the intervention to be administered or provided to the target subject, based at least in part on the determined response of the set of test subjects to the intervention. In some embodiments, the intervention and/or a dose of the intervention is determined for the target subject, based at least in part on a degree of similarity or dissimilarity of characteristics between the target subject and the set of test subjects. In some embodiments, the characteristics comprise clinical characteristics or demographical characteristics. In some embodiments, the method further comprises prescribing or administering the selected dose of the intervention to the target subject.

[00116] In some embodiments, the clinical intervention is part of a clinical trial. [00117] FIG. 1 depicts an example of an N-of-1 trial exploring 5 doses of an intervention with 7 measurements on each of 5 different response measures (the colored lines) during each dosing period. The numbered notes provide indications of various aspects that methods and systems of the present disclosure may utilize to perform analyses that deal with phenomena which negatively impact inferences about the effect of an intervention on multiple response measures. [00118] The methods and systems of the present disclosure may be designed to have broad applicability and, when used appropriately, to lead to the development, deployment, and use of more effective, insightful, and efficient health interventions for individuals very likely to benefit from them.

[00119] In another aspect, the present disclosure provides systems and methods for determining a target individual’s broad (e.g.., whole body, multi-system, or multivariate) response to a health intervention, as well as identifying health interventions that are likely to benefit a target individual based on the similarity of that individual’s characteristics relative to other individuals’ characteristics. The systems and methods of the present disclosure may take into consideration a wide variety of confounding factors that may complicate relevant inferences about both whether an individual responded to an intervention and which interventions might benefit a target individual. It also considers the infrastructure necessary for its deployment and use. The systems and methods of the present disclosure may comprise broad components, examples of which include, but are not limited to: 1. The design of appropriate and statistically powerful and robust N-of-1 trials to test the effect of an intervention on multiple response measures; and 2. The use of the results of the N-of-1 trials of the present disclosure in overarching efforts to enable more precise determinations of health interventions for individuals. Although the systems and methods of the present disclosure are discussed in the context of human health interventions, the techniques and apparatus can be used in any setting in which the ‘health’ of individuals (e.g., cells, mice, plants, machines, etc.) is of interest and interventions for improving health are available and can be provided in specific amounts or doses.

[00120] Drugs, diets, behavioral modification programs, and any other health interventions may not work ubiquitously. Individuals may vary at the genetic, biochemical, physiologic, clinical, behavioral and exposure levels in ways that impact their responses to health interventions. This individual variance means two individuals may undergo the same intervention for the same indication but experience differing impacts and overall outcome.

[00121] This treatment may effect heterogeneity or variation in intervention response and may have serious implications for the development and deployment of health interventions. These implications may be more serious when there are no reliable biomarkers for who may or may not benefit from a particular intervention since the use of that intervention may be based purely on the assumption that an individual may benefit, which may create undue harm for that individual if that assumption is false. Additionally, an intervention that may benefit a specific population may not be used because at the overall population level, this benefit is hidden by neutral or negative effects to other populations.

[00122] The fact that interventions do not work ubiquitously and in all contexts may lead to studies into ‘precision,’ ‘individualized,’ or ‘personalized’ medicine and nutrition (note: the term ‘precision medicine’ may refer to all ‘individualized,’ or ‘personalized’ medicine and nutrition), in which the reasons why some individuals respond to various health interventions and some do not are of focus.

[00123] Precision medicine studies may identify many factors, including genetic, biochemical, physiologic, behavioral, and exposure-related factors, that influence response (e.g., benefit) to particular health interventions. These factors, which can be used to identify individuals who are most likely to benefit, or not, from a particular intervention, may have clinical, public health, and personal health optimization utility.

[00124] In order to identify a factor that may be predictive or associated with a benefit from an intervention, individuals may be provided the intervention, evaluated for their response, and also tested for the presence or absence of the factor in question. When enough individuals have been tested for their response, it may be possible to identify an association between the factor and its effects on similar individuals or subpopulations.

[00125] Individuals undergoing a clinical study to determine if they are responding to an intervention may be evaluated for their health prior to participating in the study to make sure they do not have a condition that may prevent their response. In addition, by being evaluated for their ‘baseline health’, individuals with similar baseline health attributes can be assessed to see if there are similar response patterns. This similarity assessment may predict whether an individual may benefit from an intervention even when it has a heterogeneous treatment effect.

[00126] Many factors that influence response to an intervention are weak, and responses to interventions may have a multifactorial basis. Therefore, many different candidate response- associated factors may need to be collected on many individuals. These factors may be collected repeatedly on these individuals in order to test their response to the intervention. Methods for predicting those responses can then be performed that include the aggregated associations of many factors. This raises decisions about how to test individual responses reliably and efficiently to a health intervention.

[00127] It may be the case that an individual’s response to an intervention is nuanced and idiosyncratic, suggesting that it may be very hard, if not impossible, to identity any factors predictive of response by aggregating the response data obtained on many individuals and testing associations between the presence of the factors and responses. Additionally, existing assay panels may not capture the necessary baseline health data to associate particular factors predictive of positive response.

[00128] Many health interventions affect many aspects of human physiology and health. In addition, many health interventions been designed to influence many different aspects of human physiology and health (e.g., preventive interventions and geroprotectors) but must be tested to see if in fact they do affect different aspects of human physiology and health.

[00129] In order to determine whether a number of aspects of human physiology and health are in fact affected by a health intervention, a number of baseline measures, each capturing a different aspect of human physiology and health, may need to be collected on individuals administered the intervention. These ‘response measures’ can then be tested for their association with the administration of the intervention by contrasting the values of these measures when the individuals are ‘on’ versus when they are ‘off the intervention. This may be referred to as a “baseline assessment,” which is meant to capture different aspects of an individual’s health, as a ‘meta-physical’ health exam given its broad and encompassing nature.

[00130] The motivation for evaluating an individual’s response, or multiple individuals’ responses, to an intervention are numerous and may include: the need for testing a new intervention in humans to accommodate regulatory approval; testing the candidacy of an intervention for repurposing; optimizing health and the health trajectory of individuals; individuals suffering from conditions with high medical burdens or lacking well established treatments, individuals interested in non-clinical interventions to improve their health, etc. [00131] For early-stage clinical trials of, e.g., pharmacotherapies, such as Phase I and proof- of-concept (POC) studies, there is value in exploring the broader effects on the intervention; for example, to identify potential side effects, explore variation in response that may suggest reduced market size, and expose repurposing opportunities. The systems and methods of the present disclosure can be used to enable such trials, which may be referred to as ‘Phase Phi’ trials (e.g., as the number ‘Phi,’ the ‘golden ratio’, is approximately 1.618, which can be seen as between Phase I and Phase II trials which focus on dosing and initial efficacy, respectively). Whole bodybased Phase Phi aggregated N-of-1 trials such that those disclosed herein can complement traditional Phase I and II trials and help position an intervention for regulatory, efficacy, personalization, and repurposing efforts.

[00132] Databases may be used to house information about individuals’ responses to interventions. For example, the FAERS (Food and Drug Administration (FDA) Adverse Events Reporting System) database contains information on the adverse side effects of interventions on millions of individuals. However, the FAERS database includes information on reported adverse side effects in a wide variety of contexts, and not necessarily those adverse side effects identified in studies specifically designed to identify patient responses to interventions. In addition, few large-scale databases exist that contain query-able information about intervention responses that used the same or similar techniques to characterize response to ensure comparability and reliability of the information.

[00133] More sophisticated ways of identifying unequivocal responders to health interventions, particularly with respect to multiple systems and even whole-body health, may be needed. Such a database may be analyzed to determine patterns among responders that may predict benefit of an intervention for a future target individual based on that individual’s health profile. In addition, by going through the process of studying an individual and their response to a health intervention, insight into that individual’s possible unique and nuanced response to the intervention may be obtained, benefitting the health of that individual going forward.

[00134] Many purely statistical phenomena must be accommodated in analyses of response to an intervention, such as missing data, outlying values, covariate effects, time and learning effects, correlations among measures collected during the study or trial of response, serial correlation between the observations collected over time during the study or trial, causal analysis, and the likelihood of false positive and negative results, among others. The systems and methods of the present disclosure may be designed to accommodate these phenomena.

[00135] A database housing the information and results of studies exploring responses to health interventions among individuals who have also undergone a baseline health assessment may be updated over time, with the addition of the new data and results from individuals. Such additions and updates may form the basis for determinations about the utility of health interventions for future target individuals as well as the overall frequency of responses, the nature of those responses, and better ways of matching patients based on their baseline health assessments. These continual additions and continual updating may lead to the revelation of false positive and negative results over time and must be accounted for.

[00136] This database may be referred to as a ‘Database Of What Works (DOWW)’, which details the experience and results of individuals whose response to health interventions was assessed objectively. This DOWW enables clinicians, public health practitioners, and individuals, to identify health interventions that may be beneficial to specific target individuals. These health interventions may be determined based on those target individual’s baseline health characteristics and their matches to previous individuals whose data and results are in the DOWW.

[00137] The description herein regarding various applications of the systems and methods of the present disclosure in human health contexts, may also be applied to any setting in which individual responses are of importance and whose aggregated analysis may be used to make broad inferences about the nature of the interventions, such as with cell lines, mice, plants, and other non-human species, or even in mechanical settings in which ‘health’ promoting interventions can be administered in doses (e.g., the amount of lubricant applied to an engine). [00138] The methods, software and apparatus described herein provide a coherent system for gathering data on an individual (the ‘target’ individual) in need of a health intervention based on a common yet comprehensive health assessment, referred to as a meta-physical health exam. The systems and methods of the present disclosure may comprise: the design of appropriate and statistically powerful and robust N-of-1 trials to test the effect of an intervention on multiple response measures that can accommodate a wide variety of phenomena to ensure scientifically valid inferences can be drawn (FIG. 1); and the use of the results of the N-of-1 trials of the present disclosure in overarching efforts to enable more precise determinations of health interventions for individuals (FIG. 2).

[00139] In some aspects, a target individual is either provided an intervention under a unique multivariate N-of-1 clinical trial protocol. In other aspects, the target individual is simply provided the intervention without an N-of-1 protocol if the target patient’s health assessment profile matches others who have been successfully treated with that intervention. The data and results from the trial may indicate the target individual’s overall response to the intervention, and may be deposited into a broader database that includes data from other individuals who were provided interventions under similar protocols. Any data collected on target individuals who are provided the intervention on the basis of their match to others who benefited from that intervention can also be deposited into the database. The database can be mined for patterns to enable matching of future target individuals’ profiles to identify likely beneficial health interventions.

[00140] FIG. 2 provides an example of a descriptive depiction, or schematic, of the flow of information associated with methods and systems of the present disclosure. First, a target individual’s whole-body health is evaluated using a broad common set of health metrics, which may be referred to as a ‘meta-physical’ exam. Second, the health profile (e.g., the aggregation of health metrics from the meta-physical exam) of the target individual is compared to others in a database of what works (DOWW) to assess evidence of a ‘match’. Third, if there is a match, then the intervention, which was beneficial to the individuals that matched the target individual, is provided to the target individual, and data on the target individual’s response to the intervention may be collected. If there is a non-match, then an N-of-1 study is designed using methods and systems of the present disclosure to evaluate the whole body impact of an intervention (matched, novel, unproven, or otherwise. Next, the data and results are stored in a Database of What Works (DOWW). Next, the aggregated data and results are analyzed to identify meaningful patterns. Next, the matching criteria are refined based on the aggregated data, and/or the probabilistic potential for false positive/negative results is evaluated as more data and results are generated by the system. Next, the following may be revised as needed: matching criteria, N-of-1 designs, baseline health metrics, whole body response metrics, and/or evidence that future individuals are likely to respond to the interventions recorded in the DOWW.

[00141] Methods and systems of the present disclosure may be used to perform the following:

1. design of an N-of-1 trial that probes the broad, ‘whole-body,’ effects of a health intervention;

2. comprehensive assessment of the health of an individual prior to participating in an N-of-1 trial; 3. entering the data and results from N-of-1 trials and the baseline health assessment into a database; 4. matching future target patients based on the similarity of their baseline health assessments with the baseline health assessments of individuals who previously underwent N-of- 1 trials to enable the identification of likely beneficial interventions for those target individuals; and 5. mining and analyzing the database for patterns that may facilitate better matching and predictions about the effects of the interventions and monitoring the frequency of various intervention responses and other phenomena (e.g., covariate effects) to avoid false positive and false negative results.

[00142] The methods and systems of the present disclosure may comprise one or more of the following: 1. The design of an N-of-1 trial that probes to broad, ‘multi -variate,’ effects of a health intervention; 2. A comprehensive assessment of the health of an individual prior to participating in an N-of-1 trial; 3. Entering the data and results from N-of-1 trials and the baseline health assessment into a database; 4. Matching future target patients based on the similarity of their baseline health assessments with the baseline health assessments of individuals who previously underwent N-of-1 trials to enable the identification of likely beneficial interventions for those target individuals; 5. Mining the database for patterns that may facilitate better matching and predictions about the effects of the interventions and monitoring the frequency of various intervention responses and other phenomena (e.g., covariate effects) to avoid false positive and false negative results. The design and analysis of the N-of-1 trials may be used in various aspects of the methods and systems of the present disclosure, as detailed herein.

[00143] The design of the N-of-1 trials, their analysis, and their interpretation and use going forward are key features of the systems and methods of the present disclosure. The designs described herein are scientifically compelling, build off state-of-the-field concepts, and effectively control for a wide variety of factors that may plague other approaches for analysis of multivariate clinical trial data, such as: 1. The need to optimize the number of doses considered and the order in which they are provided; 2. Optimizing the number of measurements collected for each response measure for each dosing period; 3. Optimizing the number of response measures to capture multiple health effects of the intervention; 4. Modeling the dose (or time or covariate) response relationship appropriately; 5. Accommodating time, learning, and covariate effects on the response measures; 6. Serial correlation among the measures collected on an individual; 7. The use of multiple measures of response and possible correlations among those measures; 8. Inflated type 1 error associated with testing an intervention’s effects on more than one variable in a single study; 9. Accounting for correlations among the response measures; 10. Accounting for serial correlation among the measurements for any one response measure; 11. Accommodating nuanced error distributions, outlying values, missing values, and non-uniform time intervals among response measurements; 12. Controlling for false positive and false negative results of studies; and 13. Accommodating casual relationships among measures collected during a study on an individual (See FIG. 1).

[00144] To overcome potential confounding, the designs described herein may consider the use of randomization to decouple, e.g., time and dosage effects, differential effects across time (e.g., time x drug interactions), and building into the analysis models estimable parameters that reflect the effects on measured covariates (e.g., activity, body weight, stress, etc.).

[00145] To accommodate the multivariate nature of the trials of the present disclosure, both multivariate modeling (e.g., the use of seemingly unrelated regression (SUR) techniques) as well as multiple comparisons-corrected individual univariate tests may be incorporated. The use of multivariate model techniques not only makes sense given that one purpose of the systems and methods of the present disclosure is to evaluate the effect of an intervention on multiple health measures, but can also handle correlations between the health measures, serial correlations among the values of any one of the measures over time, and serial correlations between the health measures as an aid in identifying causal relationships among the health measures and covariates.

[00146] In order to draw valid inferences from the analytical methods of the present disclosure, the type I (false positive) and type II (false negative) errors of any relevant test statistics may need to be well-controlled. This can be complicated in situations involving many variables of health measures, especially if they are correlated and/or exhibit serial correlation. The statistical analysis techniques of the present disclosure may leverage test statistics that achieve prespecified type I error rates by exploiting permutation, randomization, and bootstrap techniques.

[00147] Health measures such as blood pressure, blood chemistries, mood, etc. may exhibit large fluctuations either due to measurement error or to biological triggers such as rapid changes in diet or stress levels. Such fluctuations may create outlying values that may compromise statistical analyses. The analytical methods and tests described herein can accommodate outlying values through the use of simulation-based tests as well as the use of ranks of the health measures rather than the actual health measure values themselves in relevant statistical analyses.

[00148] The data and results of the N-of-1 trials may be stored in a large database that may be analyzed and explored for trends or patterns that may be of relevance to the future deployment of the interventions for target individuals. Any additional data collected on individuals that undergo a baseline health exam with or without undergoing an N-of-1 trial can also be added to the database. If many N-of-1 trials of the present disclosure are aggregated and performed on the same set of interventions with the same baseline health assessment measure and intervention response measures, markers of treatment benefit (or harm) can be identified and used to inform the use of the interventions on individuals who have been evaluated for the intervention response markers in the future. Given that this database may harbor information on which health interventions work on individuals with different baseline health assessment profiles, this database may be referred to as a ‘Database of What Works’ or ‘DOWW.’

[00149] The DOWW can be used to match subjects (e.g., patients) based on their baseline ‘meta-physical’ health assessment profiles using many techniques, such as propensity score matching and related techniques. Any future target individual who may benefit from a health intervention can have their baseline health assessment matched to those in the DOWW who have previously undergone an N-of-1 trial. The target individual’s matches may lead to the identification of interventions that may likely benefit them based on their effects on individuals with similar profiles. The experience of the target individual on the interventions based on matches may be recorded in the DOWW.

[00150] Continually adding and updating the DOWW, and using the information to determine matches, may lead to changes in parameters and prediction models derived from the data. These changes may result in false positive and false negative results given their real-time or sequential nature. Therefore, online false discovery rate (FDR) techniques may be used to accommodate these changes and statistical fluctuations to reduce false positive and false negative results. [00151] The DOWW can be used to identify patterns in the responses to interventions, assess the frequency of responses, identify factors that are associated with response, identify patients that match a target individual to inform any health interventions they might need going forward, and many other clinical, public health and individual health optimization settings. Therefore, the methods and systems of the present disclosure are attractive for a multitude of commercial and non-commercial activities.

[00152] The methods and systems of the present disclosure, including the N-of-1 designs, analytical methods and DOWW, have applicability in not only human health settings, but can also apply to any setting in which individual responses are of importance and whose aggregated analysis may be used make broad inferences about the nature of the interventions, such as with cell lines, mice, plants, and other non-human species, or even in mechanical settings in which ‘health’ promoting interventions can be administered in doses (e.g., the amount of lubricant applied to an engine).

[00153] While basic mathematical and statistical constructs are described herein, these constructs are meant as examples to capture different aspects of the systems and methods of the present disclosure. These constructs can be modified and extended in various ways to accommodate a wide variety of phenomena of interest. Areas where obvious extensions can be made are mentioned in the following, sometimes without going into the mathematical details. [00154] Systems and methods of the present disclosure may comprise: 1. The design of appropriate and statistically powerful and robust N-of-1 trials to test the effect of an intervention on multiple response measures that can accommodate a wide variety of phenomena to ensure scientifically valid inferences can be drawn (FIG. 1); and 2. The use of the results of the described N-of-1 trials in overarching efforts to enable more precise determinations of health interventions for individuals (FIG. 2).

[00155] FIG. 2 provides a descriptive depiction, or schematic, of the flow of information associated with the systems and methods of the present disclosure. Examples of components of the systems and methods of the present disclosure are itemized here for illustrative purposes:

1. The design of N-of-1 trials that probe the broad, ‘whole-body,’ effects of a health intervention on an individual (i.e., the impact of the intervention on different aspects of health; e.g., mood, pain, sleep quality, blood pressure, cholesterol level, etc.).

2. A comprehensive assessment of the health of an individual prior to participating in an N- of-1 trial mentioned in item 1.

3. Entering the data and results from N-of-1 trials (item 1) and the baseline health assessments (item 2) into a database (the ‘Database of what works or DOWW’).

4. Matching future target patients based on the similarity of their baseline health assessments (item 2) with the baseline health assessments of individuals who previously underwent N-of-1 trials (item 1) using the DOWW (item 3) to enable the identification of likely beneficial interventions for those target individuals.

5. Mining the database (item 3) for patterns that may facilitate better matching and predictions about the effects of the interventions and monitoring the frequency of various intervention responses and other phenomena (e.g., covariate effects) to avoid false positive and false negative results. [00156] The designs of appropriate N-of-1 trials to test an intervention’s effects on multiple health measures collected on an individual can be quite flexible, but an important feature is to avoid confounding and enable causal claims about the effect of the intervention on health measures. The simplest design, which may be referred to as an ‘interrupted time series’ design, has a simple baseline period (0 dose of the intervention) followed by the administration of the intervention (dose > 0).

[00157] FIGs. 3A-3D provide an example of a simple simulated two period N-of-1 clinical trial design, which may be referred to as an “interrupted time series”. FIG. 3A is a scatter plot of a simulated ‘interrupted time series’ clinical trial with a baseline (0.0 intervention dose) and intervention (intervention dose > 0.0) period and one response measure. The assumed effect size of the intervention was 2.0 standard deviation units (SDUs) with the measures following an N(0,l) residual distribution. 15 response measurements were collected during each period for a total of 30 measurements. No time or learning effect, nor covariate effects, were assumed. FIG. 3B is a box plot of the measurements made during each period and reflects a clear intervention effect. FIG. 3C is a scatter plot of a simulated interrupted time series clinical trial with no intervention effect, but a strong learning or time effect (slope = 0.1). FIG. 3D is a box plot of the measurements made provided in the lower left panel and gives the impression of an intervention effect, but actually simply reflects the time effect. No covariate or dose effects were assumed.

[00158] The number of measurements on health response measures, the effects of confounding variables, serial correlation, and a number of other factors and phenomena may dictate the power of the study.

[00159] For N-of-1 trials designed to assess the impact of an intervention on overall health, more than one response measure may need be collected. FIGs. 4A-4B provide an example of a simple two period N-of-1 clinical trial design, which may be referred to as an “interrupted time series”, with multiple response measures. FIG. 4A is a box plot of 5 different simulated response measures (‘Pain,’ ‘Mood,’ ‘Sleep,’ ‘Blood Pressure (BP)’, and a ‘Biomarker,’ (dl-d5) collected in an interrupted time series design where the intervention had an effect size of 2.0 (SDUs) on each response measure. 20 simulated measurements on each of the 5 hypothetical response measures were collected at baseline (0 on the x axis) and after administration of the intervention (1 on the x axis). FIG. 4A depicts a scenario in which the intervention affected all 5 response measures, and FIG. 4B depicts a scenario in which the intervention affected only 3 of the 5 response measures (‘blood pressure (BP; d4) and the ‘biomarker’ (d5) are not affected).

[00160] To ensure that causal claims about the effect of a health intervention can be made, a dose-response relationship may be established between the intervention and the health response measures. FIG. 5A plots the values of a simulated response measure collected during the administration of 5 doses (including a 0 dose) of an intervention provided to an individual in increasing order. 7 measurements on a single response metric were collected during each dosing period. There is a clear dose response relationship. FIG. 5B provides a boxplot of the values in FIG. 5 A and again depicts a strong dose response effect.

[00161] It is important to differentiate a time or learning effect (e.g., a continual increase or decrease of a response measure over time after its collection is initiated) from a dose effect. This can be achieved by ensuring - to the degree possible - that the administration of the doses is not done in a manner where the dosages are correlated with time. FIG. 5C plots values of a simulated response measure collected during the administration of 8 dose periods, including repeated 0 doses after the initiation of the trial, in which these doses are not provided to an individual in increasing order. 7 measurements on a single response metric were collected during each dosing period. There is no clear time effect. FIG. 5D provides a boxplot of the values in FIG. 5C but depicts a strong dose response effect, when the measures are plotted as a function of dose and not time.

[00162] FIG. 5E plots values of a simulated response measure collected during the administration of 8 dose periods, including repeated 0 doses after the initiation of the trial, in which these doses are not provided to an individual in increasing order. 7 measurements on a single response metric were collected during each dosing period. There is a clear time effect. FIG. 5F provides a boxplot of the values in FIG. 5E but indicates that there is no strong dose response effect, when the measures are plotted as a function of dose and not time.

[00163] Multiple measures can be collected on studies exploiting dose response designs such as those depicted in FIGs. 5A-5F. FIGs. 6A-6D depict different scenarios in which multiple response measures have been collected. FIG. 6A is a boxplot of 5 different simulated response measures collected in a N-of-1 trial exploring 5 different doses of an intervention. 7 measurements of each response metric were simulated for each dose period. The boxplot clearly indicates an effect of dose on each of the response measures. FIG. 6B provides a boxplot much like in the leftmost panel but in which the dosages were not provided in increasing order, and as a result exhibits no time effect, but a strong dose response relationship. FIG. 6C is a boxplot of 5 different simulated response measures collected in a N-of-1 trial exploring 5 different doses of an intervention. 7 measurements of each response metric were simulated for each dose period. Only 4 of the 5 response metrics exhibited a dose effect. FIG. 6D is a boxplot of 5 different simulated response measures collected in a N-of-1 trial exploring 5 different doses of an intervention in which only 2 of the 5 response metrics exhibits a dose response effect.

[00164] To provide a general framework for the main modeling component of the systems and methods of the present disclosure, assume that M response measures, have been collected at T different time points, t ₁₍ t ₂ , ... , t _T , on an individual (i = 1, ... , T) . Assume further that C different covariates (e.g., weight, mood, dietary components, activity level, etc.), ^c i,i> ^c 2,i> — > ^c c,i> have been collected at each of the T time points as well. In addition, assume that D different ‘dosages’ of an intervention (e.g., a drug, nutritional supplement, digital or behavioral therapeutic, etc.), d _t G {d ₁₍ d ₂ , ..., d _D }, have been provided to the individual, at each time point. These dosages may be provided over subsequent sets of consecutive time points; e.g., for T » D, the same dosage, d _t G {d ₁₍ d ₂ , ... , d _D }, may be provided to the participant for roughly T /D consecutive time points before changing the dosage for an additional T /D consecutive time points. A system of linear regression equations can be used to relate the covariates, time, and intervention dosages to the each of the response measures, as shown by Equations (1) below:

Where the b _Q ^l are regression coefficients, with the b _Q ^l coefficients as intercept terms, the e- are error terms that are assumed to be normally distributed as A(0, (<j ² ) ^m ) where the superscript m indicates a response measure (m = 1, ... , M). Also included in the individual regression equations in Equations (1) is a regression coefficient capturing the dependence of the measurement response, y-, with the previous measurement response y[- ₁ . This dependence may reflect a simple autoregressive process of order 1 (i.e., an AR(1) process), but any of a number of serial dependencies can be modeled, including through the use of the error terms e- as discussed below. For simplicity, the assumption is that tj = i, although tj may reflect the real time since the start of the study (e.g., in minutes or hours) at the ith collection time for the response measures.

[00165] The equations for each of the response measures, y ^m = \y™, y™, ... , y™|', can be rewritten as a function of the different covariates, time points, dose effect and potential autoregressive term. The covariates, time points, dose effect and potential autoregressive term can be assembled into a T x (C + 4) -dimensional matrix X ^m , whose first column is a column of Is to capture an intercept term, the next C columns are the covariate values for each time point (row), and the next 3 columns are the tj, d _t , and y^ values for each time point (row), respectively. With these constructs, the equations given in Equations (1) can be rewritten as a multivariate multiple regression model of the form given in Equation (2): y ^m = X ^m B ^m + e ^m (2) where the errors e ^m are distributed as a multivariate normal distribution (or another appropriate distribution), with T-dimensional mean vector 0 and T x T covariance matrix (l ^m , whose diagonal entries reflect the variances (assumed to be equal) for the response measure value at each time point, and the off-diagonal elements reflect the covariances, between response measures taken at different time points i #= j, where i, j E {1, ... , T}. The C +

4 regression coefficients are captured in the (C + 4) -dimensional column vector B ^m . Note that depending on how the covariance matrix entries are modeled and estimated, the autoregressive term assuming the y^ values to capture the serial correlation between the response measures may be removed from the matrix X ^m in Equation (2).

[00166] The multivariate equations in (00062.2) can themselves be aggregated in a more comprehensive, higher dimensional model encompassing all M x T response measures, as given in Equation (3): where the error vectors are assumed to have a M x T mean vector of 0’s and (M x T) x (M x T) covariance matrix whose entries reflect individual response measure variances, individual response measure serial covariances, cross response measure covariances, and cross response measure serial covariances. Estimation of all the parameters (e.g., regression coefficients and covariance matrix terms) in the model reflected in Equation (3) reliably may be difficult numerically and computationally, especially without a large enough number of response measure collection times over the course of the study. However, systems of equations like those reflected in Equations (1), (2), and (3) can have their parameters estimated simultaneously from a data set through approaches such as ‘Seemingly Unrelated Regressions Estimates’ (‘SURE’).

[00167] Cases in which D=2, for which there is a 0.0 dose baseline period followed by a period in which the intervention (i.e., a dose > 0.0) is provided, can exploit ‘interrupted time series’ models in their design and analysis. Including multiple response measures in such interrupted time series models can still be formulated as a system of equations as discussed in the context of Equation (1). In this setting, = 0 or 1, and it is likely that the baseline period may cover time periods spanned by the first B (B<T) measurement times, and the intervention may be provided during the last T-B measurement times.

[00168] As noted, estimation of the parameters provided in the example systems of equations reflected in Equations (1), (2), (3), and (4) can be pursued in a wide variety of ways. For example, each individual equation’s parameters - that are associated with a single response measure out of a larger total number of response measures - may be estimated independently of the other equations’ parameters. This may ignore correlations between the response measures. In addition, the parameters associated with all the equations in the system may be estimated using SURE techniques.

[00169] To showcase how reliable the estimation of the parameters in a system of equations of the type reflected in Equations (1), (2), and (3) can be, simulation studies were pursued. Table 1 provides estimated average parameter estimates (‘Estimates’) and their standard deviations (‘Stnd Dev’) for the models of the present disclosure under different parameter settings (‘Truth’) for 1 of the 5 response variables considered in the simulations. The error rates were estimated from 1000 simulations for different parameter values (a learning effect, a dose effect, and a serial correlation effect) for N-of-1 designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics. 7 measurements were made during each dose period for a total of 35 measurements for each of the 5 response measures. The order of the doses was 2-0-4- 1-3. The AR(1) column provides the assumed serial correlation strength and the estimated serial correlation strength. It can be seen that the parameter estimates are very near their true values, attesting to the reliability of the analytical methods and approaches of the present disclosure.

[00171] The reliability of the parameter estimates can also be seen in the estimation of the parameters in a system of equations of the type reflected in Equation (4), which considers only 2 doses in the ‘interrupted time series’ model. Table 2 provides estimated average parameter estimates (‘Estimates’) and their standard deviations (‘Stnd Dev’) for the models of the present disclosure under different parameter settings (‘Truth’) for 1 of the 5 response variables considered in the simulations. The error rates were estimated from 1000 simulations for different parameter values (a learning effect, a dose effect, and a serial correlation effect) for N-of-1 designs assuming 2 doses (a 0 dose period followed by a non-zero dose period) and 5 response measures for different test statistics. 20 measurements were made during each dose period for a total of 40 measurements for each of the 5 response measures. It can again be seen that the parameter estimates are very near their true values, attesting to the reliability of the analytical methods and approaches of the present disclosure.

[00172] Table 2: Average Parameter Estimates for 1 of 5 Measures in 2-Dose Trials

Under Different Parameter Settings [00173] Testing individual regression coefficients for their significance (e.g., from 0.0) is crucial for detecting intervention effects as well as covariate and other effects. Standard tests (e.g., t-tests) of the regression coefficients using estimated standard errors of those coefficients can be pursued as can broaden sets of coefficients assessed simultaneously using SURE-based procedures. However, in order to preserve assumed Type I error rates (i.e., an assumed rate of false positives), one must consider all the effects that confounding factors may have on the distribution of relevant test statistics. This can be very difficult to do analytically based on first principles in mathematical statistics, so simulation-based tests can be used, such as randomization, permutation and bootstrap tests.

[00174] Table 3 provides a simple study of the use of simulation-based tests of regression coefficients for models of the type represented in Equations (1), (2), (3), and (4). Essentially, a nominal type 1 error rate was assumed for 1000 simulations for different parameter values for two types of N-of-1 designs: the first assuming 5 doses (including 0 dose) and 5 response measures for different test statistics. 7 measurements were made during each dose period for a total of 35 measurements for each of the 5 response measures. The order of the doses was 2-0-4- 1-3. The second assuming 2 doses (a 0 dose followed by a non-zero dose) with 20 observations in each dose period. All simulations were performed under the null hypothesis of no intervention or dose effect and assumed Type 1 error rate of 0.05. The ‘size’ row in the tables gives the actual estimated Type 1 error rate and the ‘Adj p-value’ row gives the p-value that may be used to, in effect, achieve a Type 1 error rate of 0.05 for 16 different test statistics, the first 4 statistics reflecting different simultaneous tests of all 5 dose/intervention test statistics, the second 7 reflecting different multiple compari sons-corrected tests of each individual regression coefficient, and the last 5 reflecting individual tests on the regression coefficients assuming a simple linear regression for each of the 5 response measures. It can be seen that the observed Type 1 error rates in the ‘size’ rows deviate from the expected Type 1 error rate of 0.05. The use of the adjusted p- values derived from the simulations (the ‘Adj p-value’ row entries) may result in tests with the appropriate Type 1 error rate of 0.05 and suggest the use of simulation-based tests in testing relevant hypotheses may lead to valid inferences if the model assumptions are correct.

[00175] Table 3: Inflation of Type I Error Rates Associated with Different Statistics Under Different Parameter Settings

[00176] FIG. 7 provides plots of the power of a linear model as a function of effect size (the slope or regression coefficient for variables with a N(0,l) error distribution). The linear model includes a time (or learning) effect and a dose effect. Power was estimated from 1000 simulations for assumed effect sizes between 0 and 0.5 in increments of 0.002 (i.e., 250 different effect size settings) of a model that assumed different time and dose effects. Four different non-zero doses were assumed with a 0.0 dose initially, and 2 additional 0.0 dose periods after the initiation of the study for a total of 8 dose periods. No additional covariate effects were assumed. 7 measures per dosage period (e.g., assuming daily measures for one week) were simulated in each setting for a total of 8 x 7 = 56 measures. The left panel provides power curves in which the order of the doses was provided randomly in each of 1000 simulations. The right panel depicts settings in which the doses were provided in specific orders to showcase how problematic the choice of dose order might be in differentiating a dose effect from a time effect. [00177] Controlling for or avoiding the confounding of time (or other relevant covariate effects) and intervention and dose effects in the models presented in Equations (1), (2), (3), and (4) may be crucial. FIG. 7 shows an example of reasons why in the form of power curves exploring the models’ abilities to detect intervention/dose effects. Note that for all power curves in the left panel of FIG. 7, the order of the doses, with the exception of an initial 0.0 dose period, were chosen randomly for each of the 1000 simulations for a given effect size. T-tests of the null hypothesis that the dose effect was 0.0 were performed for each simulated data set for each of two models: one assuming the inclusion of the time effect and one without. In order to give an indication of the power of the tests, the total number of test statistics with p-values less than 0.05 out of each set of 1000 simulated data sets were recorded for each assumed effect size setting (e.g., type 1 error assumed to be 0.05) for the test that the hypothesis that the intervention effect regression coefficient (e.g., slope) = 0.0 (for each of the two analytical models).

[00178] The left panel of FIG. 7 depicts estimated power curves as a function of the assumed effect sizes (slope) estimated from the simulation studies for 3 different scenarios. The black lines assume only a dose effect reflected on the x-axis, with the solid black line reflecting the power of the model that included both a dose and time effect (note that it overlaps with the solid blue line) and the dashed black line reflecting the power of the model that only assumes a dose effect. It can be seen from the left panel of FIG. 7 that when there is a dose effect only (e.g., no time effect) a model that only considers a dose effect is slightly more powerful than a model that assumes a dose and time effect, probably due to the noise associated with estimating an additional parameter (e.g., the time effect) that is null. The red lines assume only a time effect reflected on the x-axis (e.g., no dose effect) with the solid red line reflecting the power of the model that included both a dose and time effect and the dashed red line reflecting the power of the model that only assumes a dose effect. The results suggest that if a time effect is not included in the model, the false positive rate for detecting dose effects is quite high, despite the fact the dosages used in the N-of-1 trials were randomly ordered (i.e., dashed red line). This is likely due to the fact that randomly chosen orders of the dosages over all 1000 simulated studies may by chance include occasional dosage orders that are consistent with a linear dose escalation, leading to an overt confounding with a time effect. Note also that when both a dose and time effect are included in the model (shown by the solid red line) the appropriate type I error rate (0.05) for dose effects is maintained.

[00179] The blue lines in the left panel of FIG. 7 assume a fixed time effect of 0.2 (e.g., slope of the time effect is 0.2) and variable dose effect reflected on the x-axis, with the solid blue line reflecting the power of the model that included both a dose and time effect and the dashed blue line reflecting the power of the model that only assumes a dose effect. It can be seen from the left side of FIGs. 6A-6D that, again, when only a dose effect is included in the model (the dashed blue line) the false positive rate for detecting dose effects is quite high, but when both dose and time effects are included in the model (the solid blue line) true dose effects are differentiated from the time effects, consistent with settings in which only a dose effect is present (e.g., the solid black line). Thus, including both dose and time effects in the models is important, despite the use of randomization for the order in which doses of the intervention are provided.

[00180] The right panel of FIG. 7 explores how problematic the choice of the order in which the dosages are provided in a study can be, even if chosen randomly. Simulation studies were pursued with fixed (e.g., not random) Individual Dose Response (IDR) study dosage orders with no randomly interspersed zero dose periods. 7 measures per each of 6 dose periods were simulated, for a total of 6 x 7 = 42 measures. 1000 simulations were run for each of 500 different dose effect sizes in 0.001 increments between 0 and 0.5, with models assuming dose and time effects and only dose effects fit to each of the 42 simulated measures in each setting. Note that since only settings in which dose effects were considered, these simulations studies explore the impact of including a time effect term in the model when there is in fact no time effect, but the order in which the dosages are provided may lead to a simple dose escalation with an effect whose detection is confounded by the inclusion of a time effect in the model. 5 different dosage orders were considered. The black lines assume an order 012345 (e.g., linearly increasing doses, which is confounded with time), with the solid black lines (as with all different fixed dose order designs depicted by different colors) reflecting a model with both a dose and time effect and the dashed black line (again, as with all different orders depicted by different colors) reflecting a model that only included a dose effect. Since the order 012345 is consistent with a simple linear dose escalation, it is confounded with a time effect. As such, the model that includes a time effect term along with the dose effect term (the solid black line) has poor power to detect the dose effect, even though there is no time effect. This is likely due to the correlation between the doses and times over the course of the study (note: the Pearson correlation of the 42 doses and times is 0.987 in this setting) and the resulting lack of identifiability of the dose and time regression coefficient parameters in the model. The black dashed line, overlapping with the other dashed lines and the solid brown and blue lines, reflects a model with only a dose effect which, because of the lack of a time effect, captures the dose effect properly (contrast with FIGs. 5A-5B) but may not control for a time effect if there was one. The blue lines correspond to an order of 520314, the green lines an order of 204135, the red lines an order of 102435, and the brown lines an order of 513024. Since the solid red lines assumed order of 102435 has a near linear dose escalation, it suffers from the same problem as the order assumed for the black line and has poor power to differentiate dose and time effects. The other settings, reflected by the green, blue and brown lines, do not reflect simple dose-time-dependent dose escalations, thus they are less problematic. One may avoid designs that have trouble differentiating dose and time effects (e.g., by only randomly choosing from the universe of designs that are less problematic) or infuse more random (or even non-random) zero dose periods in a study. However, these results illustrate the importance and need for accommodating analyses of aggregated N-of-1 studies results.

[00181] To further avoid confounding and enhance the ability of the models of the present disclosure to detect an intervention/dose effect, multiple 0 doses may be included in an N-of-1 to act as potential washout periods, as suggested in previous sections, if the use of 0 doses in this manner does not compromise the patient’s health (in other words, for some conditions it is important to be treated continuously and not completely taken off all interventions).

[00182] The power to detect an effect of an intervention on any number of response measures based on an assumed design can be explored for models of the type described in Equations (1), (2), (3), and (4). An assumed number of response measures, data collection time points, and dosing strategies can be posited. 16 different example test statistics can be explored for their power under different assumptions about time or learning effects, covariate effects, the covariance between the response measures, and the serial correlation between the observations for any given response measure. Simulation studies can be used to achieve this. The evaluation of power for different designs can lead to the identification of optimal (e.g., efficient) designs for any application and parameter setting.

[00183] The 16 different example test statistics evaluated in Table 3 can be used to explore the power of different N-of-1 study designs leveraging analysis models of the type described in Equations (1), (2), (3), and (4). FIG. 8 depicts the power to detect an intervention effect (the slope of the relationship between the dose and response measures as depicted on the x axis) for each of the 16 test statistics. Power was estimated from 1000 simulations for different parameter values for N-of-1 designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics. 7 measurements were made during each dose period for a total of 35 measurements for each of the 5 response measures. The order of the doses was 2-0-4-1-3. Test statistics that reflect a type 1 error rate were estimated from 1000 simulations under the hypothesis of no dose effect. No covariate effects were assumed. Exploration of the power of different test statistics to test an intervention or dose effect may lead to insights into test statistics that may be more powerful than others in some settings and therefore used in those settings going forward.

[00184] The number of response measures that are actually influenced or affected by an intervention may influence the power to detect an intervention or dose effect. FIG. 9 shows the power of the inverse chi-square statistic computed over all of 5 response measures (the inverse chi-square statistic is one of the 16 test statistics considered in Table 3). Power was estimated from 1000 simulations for different parameter values for N-of-1 designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics. 7 measurements were made during each dose period for a total of 35 measurements for each of the 5 response measures. The order of the doses was 2-0-4- 1-3. Test statistics that reflect a type 1 error rate were estimated from 1000 simulations under the hypothesis of no dose effect. No covariate effects were assumed. The numbers next to the curves indicate how many of the 5 variables were assumed to have the effect size reflected on the x-axis.

[00185] There are different ways of estimating the parameters across a system of equations simultaneously within the SURE model setting as reflected in Equations (1), (2), (3), and (4). For example, a very computationally intensive method may estimate serial correlation parameters using appropriately-modeled error distributions instead of including, e.g., AR(1) coefficient in simple regression models. FIG. 10 compares a computationally intensive estimation procedure against the conceptually simpler regression model with AR(1) coefficient. 1000 simulations for different parameter values for N-of-1 designs assuming 5 doses (including 0 dose) and 5 response measures for different test statistics. 7 measurements were made during each dose period for a total of 35 measurements for each of the 5 response measures. The order of the doses was 2-0-4- 1-3. Test statistics that reflect a type 1 error rate were estimated from 1000 simulations under the hypothesis of no dose effect. No covariate effects were assumed. The solid lines reflect the computationally efficient parameter estimation techniques and the dashed lines reflect a much more computationally intensive parameter estimation techniques. The red lines assume a covariance of 0.75 between the 5 different response measures and the black lines assume a serial correlation among values for each response measure of 0.75. It can be seen that the use of the computationally efficient approach is non-inferior to the computationally intensive approach and, in fact, in settings where serial correlation is quite strong, the computationally efficient approach is more powerful, probably due to large variances on estimated parameters when serial correlation is strong and the computationally intensive approach is used.

[00186] Models of the type reflected in Equations (1), (2), (3), and (4) can accommodate missing data, as well as measurements that are not collected at some time points, by using regression equations or SURE techniques by, e.g., replacing discrete time with real time since start of the trial as an independent variable reflecting time of measurement.

[00187] Models of the type reflected in Equations (1), (2), (3), and (4) can accommodate nonlinearities in the dose-response relationship or the time-response relationship through transformations of the response and independent variable measures (e.g., using the logarithms of response measures and/or the covariates, time or dose or values) or by explicitly modeling the relationship between the response and dose of the intervention with a non-linear function.

[00188] Models of the type reflected in Equations (1), (2), (3), and (4) can accommodate discrete or categorical response measures through the use of, e.g., logistic regression models in place of the linear regression models in Equations (1), (2), (3), and (4).

[00189] Covariance between response measures may be accommodated in Equations (1), (2), (3), and (4) by leveraging SURE techniques. Not accounting for covariance between the response measures may lead to false positive conclusions about an intervention effect. In addition, by using simulation-based test statistics, covariance effects on inferences can be mitigated.

[00190] Serial correlation among measurements for any response measure can be accommodated in relevant N-of-1 designs and analysis models either through the inclusion of, e.g., an AR(1) lagged variable in a system of equations (e.g., Equations (1), (2), (3), and (4)) or by estimating the serial correlation effect through models that consider appropriate error distributions (see FIG. 10).

[00191] The system of equations reflected in Equations (1), (2), (3), and (4) can be framed in a way that can accommodate causal inferences to be drawn between response measures and covariates by including cross-variable lagged variables as predictor variables for each response measure. Such causal analyses may reveal mechanism of action insights for an intervention and its effect on the body of an individual.

[00192] The system of equations reflected in Equations (1), (2), (3), and (4) can be subjected to model reduction, in which response variables that do not have evidence that they are associated with the intervention or dose effect are eliminated from the models. This model reduction strategy can be used to draw inferences about which set of response measures are in fact affected by the intervention.

[00193] For settings in which only 2 doses are considered, the power of different designs that accommodate the interaction term (Equation (4)) can be explored to determine optimal 2-dose designs (optimal in the sense of needing the fewest measurements and data collection time points to detect an intervention effect) for any setting. FIGs. 11A-11B depict the power of a 2-dose design estimated from 1000 simulations for different parameter values and 5 response measures for the 16 different example test statistics considered in Table 3. Test statistics that reflect a type 1 error rate were estimated from 1000 simulations under the hypothesis of no dose effect. 20 measures were assumed in each dose period for a total of 40 measurements on each of 5 response measures. No covariate effects were assumed. The different colored lines reflect the different parameter settings indicated in the legend. FIG. 11A provides different parameter settings, and FIG. 11B assumes only a dose effect but with different numbers of measures (‘neff) assumed to manifest a dose effect.

[00194] Models of the type reflected in Equations (1), (2), (3), and (4) can accommodate the incorporation of robust test statistics to deal with outlying values, highly skewed response variables, and measurement errors. For example, ranks of the values of the response measures instead of the actual values of the response measures can be used. FIGs. 12A-12B depicts power curves for detecting different dose effect sizes (the effect size reflects the slope of response measures regressed on increasing doses) estimated from 1000 simulations for different parameter values for N-of-1 designs using ranked response measure values. For FIG. 12A, the green and purple lines assume 2 doses (including a 0 dose and >0 dose, often referred to as ‘interrupted time series’ designs) for 5 response measures for different test statistics. The blue and red lines assume 5 doses (including 0 dose) and 5 response measures for different test statistics. The order of the doses was 2-0-4- 1-3. 7 measurements were made during each dose period for a total of 35 measurements for each of the 5 response measures. No covariate effects were assumed. The purple and red lines used ranks instead of actual measurement values in the test statistics. For FIG. 12B, the green and red lines depict power curves for a 2-dose model and the purple and red lines depict estimated power curves for a 5 dose model. However, these parameter settings explored in the right panel assumed a pronounced time or learning effect (slope of 1.0) and response measure covariances of 0.75 and serial correlations among the observations of 0.75.

[00195] The design and execution of the N-of-1 trials of the present disclosure can be used in a wide variety of settings, especially if the results of trials are aggregated along with a baseline meta-physical profile on the participant in a trial (see FIG. 2). For example, individuals may want to know what might benefit them and choose to have an N-of-1 trial to see if something is indeed benefitting them. In addition, payers may want to know if specific interventions actually do benefit their customers; health systems may want to explore the quality of the lives of their patients when provided interventions; and pharma and biotech may want to pursue N-of-1 trials of the type envisioned in early or late-stage evaluations of the interventions they are developing. These are only examples as there are many other applications for the methods and systems of the present disclosure, especially if they are aggregated in a Database of What Works (DOWW), which can be mined for patterns that may have utility in determining optimal therapies for individuals in the future.

[00196] If a target individual who is in need of a health intervention is evaluated (or profiled) for their health through the use of a comprehensive assessment (a ‘meta-physical’; see FIG. 2), then their health profile can be matched to individuals who have previously undergone an N-of-1 trial to determine their response to interventions. Individuals whose meta-physical profiles match the target individual’s profile can then be analyzed for their intervention responses to determine what might have a positive effect on the target individual. Such matching may greatly enhance health care since it may avoid simple trial and error approaches to determine an optimal intervention strategy for the target individual. FIG. 13 conveys this concept. The criteria for matching individuals in this way can exploit any of a number of distance (e.g., matching) metrics, such as the Mahalanobis distance, the Hamming distance, a weighted Euclidean distance, etc.), or any other suitable criteria that may be used for determining a ‘match.’ [00197] If N-of-1 trials of the type envisioned are pursued over time, and their results are deposited into a database that may be mined for, e.g., the frequency of individuals responding to a particular intervention, then testing hypotheses about the frequency of responses or covariate effects over time can be pursued sequentially to preserve Type 1 and Type 2 error rates and avoid false positive claims about such frequencies. For example, continually updating response or covariate effect metrics indicating overall response, or associations involving health measure outcomes across different trials reflected in the database, can be pursued through the use of online false discovery rate (online FDR) procedures.

[00198] FIG. 14 depicts an example of ongoing evaluation of the frequency of responses to various interventions tested via the N-of-1 trials of the present disclosure that are recorded in a ‘Database of What Works (DOWW),’ as an example. Other phenomena may be explored, such as covariate-intervention response relationships, the strength of causal relationships between response measures, etc. Given that the N-of-1 trials may be ongoing, and that many phenomena of interest may be evaluated, criteria for testing specific hypotheses may be revised (e.g., in real time) such as by use of online false discovery rate criteria.

[00199] By aggregating the data and results of N-of-1 trials and baseline health assessments (‘meta-physicals’) in a queryable database to enable, e.g., matching, response frequency assessment, covariate associations, inferences about intervention mechanisms of action, etc., various clinical and public health applications can arise. For example, a physician may probe this ‘database of what works’ to identify interventions optimal for a patient being examined at the time, researchers may identify patterns in the database that might be indicative of intervention response (e.g., genetic associations), and health insurers may identify interventions whose frequency of response and/or strategies for identifying markers of likely response make sense for reimbursement.

[00200] The methods and systems of the present disclosure, including the N-of-1 trial designs, analytical methods and DOWW, have applicability in not only human health settings, but can also apply to any setting in which individual responses are of importance and whose aggregated analysis may be used make broad inferences about the nature of the interventions, such as with cell lines, mice, plants, and other non-human species, or even in mechanical settings in which ‘health’ promoting interventions can be administered in doses (e.g., the amount of lubricant applied to an engine). For example, a DOWW for different mouse strains or wild type mice that have been characterized in different ways (e.g., via genome sequencing as a baseline ‘metaphysical’ used for matching mice in future analyses); thus, the aggregated data may be analyzed for patterns connecting responses to interventions as a function of the unique genomic profiles of the mice. In addition, similar methods and systems may be used to explore the ‘responses’ of the components of a mechanical system (e.g., an engine) to potentially beneficial interventions, such as different types of lubrication.

[00201] The methods and systems of the present disclosure may also be exploited by combining studies on single individuals and aggregating the data in an appropriate DOWW to draw inferences about the complementarity of potential responses. For example, N-of-1 studies on humans in clinical settings, as described herein, may be coupled with N-of-1 multivariate, dose-response studies of the type considered on the response of cells, tissues, organs, engineered organoids, engrafted tissues implanted in mice, etc. harvested from those individuals. In this manner, the DOWW may include information about not only the direct benefit of the interventions in the humans, but also information about the likely surrogacy of the studies of cells, tissues, organs, etc. and their utility in vetting the benefits of interventions in the future.

[00202] Computer systems

[00203] The present disclosure provides computer systems that are programmed to implement methods of the disclosure. FIG. 15 shows a computer system 1501 that is programmed or otherwise configured to perform analyses of the methods, such as determining a response of a target subject to an intervention and determining an individualized intervention for a target subject. The computer system 1501 can regulate various aspects of methods and systems of the present disclosure, such as, for example, perform an algorithm, input training data, analyze responses to intervention, or output data. The computer system 1501 can be an electronic device of a user or a computer system that is remotely located with respect to the electronic device. The electronic device can be a mobile electronic device.

[00204] The computer system 1501 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 1505, which can be a single core or multi core processor, or a plurality of processors for parallel processing. The computer system 1501 also includes memory or memory location 1510 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 1515 (e.g., hard disk), communication interface 1520 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 1525, such as cache, other memory, data storage and/or electronic display adapters. The memory 1510, storage unit 1515, interface 1520 and peripheral devices 1525 are in communication with the CPU 1505 through a communication bus (solid lines), such as a motherboard. The storage unit 1515 can be a data storage unit (or data repository) for storing data. The computer system 1501 can be operatively coupled to a computer network (“network”) 1530 with the aid of the communication interface 1520. The network 1530 can be the Internet, an internet and/or extranet, or an intranet and/or extranet that is in communication with the Internet. The network 1530 in some cases is a telecommunication and/or data network. The network 1530 can include one or more computer servers, which can enable distributed computing, such as cloud computing. The network 1530, in some cases with the aid of the computer system 1501, can implement a peer-to-peer network, which may enable devices coupled to the computer system 1501 to behave as a client or a server.

[00205] The CPU 1505 can execute a sequence of machine-readable instructions, which can be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 1510. The instructions can be directed to the CPU 1505, which can subsequently program or otherwise configure the CPU 1505 to implement methods of the present disclosure. Examples of operations performed by the CPU 1505 can include fetch, decode, execute, and writeback.

[00206] The CPU 1505 can be part of a circuit, such as an integrated circuit. One or more other components of the system 1501 can be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[00207] The storage unit 1515 can store files, such as drivers, libraries and saved programs. The storage unit 1515 can store user data, e.g., user preferences and user programs. The computer system 1501 in some cases can include one or more additional data storage units that are external to the computer system 1501, such as located on a remote server that is in communication with the computer system 1501 through an intranet or the Internet.

[00208] The computer system 1501 can communicate with one or more remote computer systems through the network 1530. For instance, the computer system 1501 can communicate with a remote computer system of a user (e.g., a medical professional or patient). Examples of remote computer systems include personal computers (e.g., portable PC), slate or tablet PC’s (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user can access the computer system 1501 via the network 1530.

[00209] Methods as described herein can be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 1501, such as, for example, on the memory 1510 or electronic storage unit 1515. The machine executable or machine-readable code can be provided in the form of software. During use, the code can be executed by the processor 1505. In some cases, the code can be retrieved from the storage unit 1515 and stored on the memory 1510 for ready access by the processor 1505. In some situations, the electronic storage unit 1515 can be precluded, and machine-executable instructions are stored on memory 1510.

[00210] The code can be pre-compiled and configured for use with a machine having a processer adapted to execute the code or it can be compiled during runtime. The code can be supplied in a programming language that can be selected to enable the code to execute in a precompiled or as-compiled fashion.

[00211] Aspects of the systems and methods provided herein, such as the computer system 1501, can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and/or associated data that is carried on or embodied in a type of machine readable medium. Machine-executable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk.

“Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.

[00212] Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[00213] The computer system 1501 can include or be in communication with an electronic display 1535 that comprises a user interface (UI) 1540 for providing, for example, input data, or a visual output. Examples of UI’s include, without limitation, a graphical user interface (GUI) and web-based user interface.

[00214] Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 1505. The algorithm can, for example, determine a response of a target subject to an intervention, and determine an individualized intervention for a target subject.

EXAMPLES

[00215] Example 1: Multivariate data analysis

[00216] Clinical trials may consider multiple endpoints in their designs and analyses (e.g., often fewer than 5 or 10 endpoints), but they may not consider a large number of endpoints (e.g., greater than 50, 500, 5000, or more). However, with the availability of high-dimensional assays, such as transcriptomics, proteomics, metabolomics, and meta-genomics assays, the broad effects of interventions may be assessed, to evaluate how entire systems, or species ecologies if assessing the microbiome, change in response to the administration of an intervention. Testing each analyte or species for association with an intervention across individuals enrolled in a standard clinical trial, while statistically controlling for multiple comparisons, may be one strategy, but it ignores the likely correlations between the analytes, as well as intra- and interindividual variation in response to the intervention. An alternative trial strategy is to leverage the individual dose response, N-of-1, and aggregated N-of-1 clinical trial strategies disclosed herein but exploit unique multivariate data analysis methods when analyzing the data emerging from such trials to achieve power and efficiency. [00217] Multivariate distance matrix regression (MDMR) is an appropriate analytical method for considering very large numbers of measures in N-of-1 clinical trials. Here, the MDMR analytical framework is discussed in the context of N-of-1 dosing-based trials of the type proposed, and some of its properties are demonstrated via simulation studies. A wide variety of phenomena were considered in the simulations, including the addition of non-associated analytes in an analysis, correlations among the analytes, and serial correlations among analytes collected over time. The method may be made more robust through the use of ranked measures, it can control for covariate effects, and it can be used to establish a dose response relationship for an individual, as with the proposed univariate and small multivariate N-of-1 studies. Analyses may be performed to choose a subset of analytes or species out of the many being considered, in order to identify those that are the most strongly correlated with an intervention or the dosing of an intervention as well as clinical symptoms.

[00218] Using systems and methods of the present disclosure, a design strategy may be implemented as follows. Any N-of-1 trial design can be used, such as an interrupted time series (e.g., 2-period - i.e., baseline and intervention period - design, randomized dose response design with or without washout periods), with a large number of measurements using MDMR analysis techniques. Once the design has been established, the analytes and/or species can be collected at each timepoint over the course of the dosing and study timeframe. If m analytes or species are obtained (e.g., via meta-genomic sequencing of stool samples or nasal swabs, or transcriptomics or proteomics assays on blood or different tissues), then the question of whether the intervention changed the collective profiles (e.g., levels, abundances, presence/absence, etc.) of these analytes and/or species may be investigated. A shift in the analytes and/or species collectively, or as a whole, may suggest that the intervention has a broad effect. Alternatively, a shift in the analytes and/or species collectively, or as a whole, may suggest that only a subset of the analytes and/or species was affected by the intervention, but that the signal from the intervention on those analytes and/or species was strong enough to be detected despite having other non-associated analytes and/or species measurements in the analysis.

[00219] Analysis with MDMR can be performed as follows [1-7], The different analytes and/or species collected over the course of the trial are used to construct distances between each timepoint during the N-of-1 trial. Any distance measure can be used for this purpose, though some may have advantages in certain settings (e.g., a distance measure that accommodates binary presence-or-absence measurements should be used with binary or dichotomous data rather than, e.g., Euclidean distance measures, which may be suitable for use with quantitative measurements). If m analytes and/or species measurements are made at each of t total timepoints, then a t x t matrix of distances between the measurements at each timepoint may be obtained. [00220] Specific hypothesis tests about, e.g., the lesser distances between measurements occurring among measurements made during the same dosing period which would be consistent with an intervention effect, may be conducted with MDMR techniques [2-7], while controlling for covariates and other potentially confounding phenomena, such as serial correlation among specific analytes and/or species measurements made over time. Since the measurements are used to construct distances that are then the focus of the analysis, and not necessarily any individual analyte or species, any number of measurements can be considered in an N-of-1 trial of the type proposed (e.g., hundreds, thousands, tens-of-thousands, etc.). Attention may be provided to a subset of all the analytes and/or species, since distance measures across the timepoints may be computed based only on that subset of analytes and/or species and then subjected to MDMR analysis.

[00221] A few example studies are provided of the properties of the MDMR tests in N-of-1 trial settings. We first considered the number of analytes and/or species considered in an analysis. FIG. 16 provides an estimate of the power of standard MDMR F-statistic tests based on 1,000 simulations for each setting involving a different assumed number of analytes (red (right-most) = 1 analytes and/or species, green (second right-most) = 5 analytes and/or species, blue (second left-most) = 50 analytes and/or species, and black (left-most) = 500 analytes and/or species), for an effect size in standard deviation units between 0.0 to 2.0 in increments of 0.02. A simple interrupted time series design was assumed with 20 total measurement times, 10 during a baseline period and 10 during an intervention period. All simulated analytes and/or species were assumed to follow a standard normal distribution with no correlation among them and no serial correlation over time for any one of them. Gower’s distance was used for constructing the 20 x 20 matrix of distances, but any distance measure may be used with varying effects on power [1- 7], A type I error rate of 5% was assumed.

[00222] As shown in FIG. 16, large increases in power occur as more analytes and/or species are considered in the analysis, if those analytes/species are affected by the intervention, as may be expected. This phenomena demonstrates the advantages of this kind of analysis for revealing the effects of an intervention on a very large number of measurements.

[00223] FIG. 17 demonstrates the estimated power of analyses in settings like those considered in FIG. 16, but with analytes and/or species added to the analysis that are not affected by the intervention (e.g., create noise in the data). For each setting, just as many non- intervention-associated simulated analytes and/or species were added as there were associated analytes and/or species (e.g., for the setting with 1 associated analyte and/or species, 1 more unassociated analytes and/or species was added; for the setting with 5 associated analytes and/or species, 5 more unassociated analytes and/or species were added; etc.). The solid lines reflect power curves from FIG. 16 for comparison (with the same color scheme). The dashed lines, matched by color to the solid lines, show the number of associated analytes and/or species, demonstrating the impact on power of the addition of non-associated analytes and/or species. The figure shows that the increase in the number of species in this setting increases power, which may be attributed to the additional unassociated analytes and/or species helping to refine the distances between the profiles of measurements taken at each timepoint.

[00224] However, as shown in FIG. 18, creating more noise by adding additional analytes and/or species to the analysis may reduce power in some cases (e.g., due to unassociated analytes and/or species). Note that the red chain-dashed line in FIG. 16 assumes that 2 analytes and/or species have been tested for association with the intervention using the MDMR technique. Comparing the red chained-dashed line to the red dotted line and red solid line indicates that the power gain by adding unassociated analytes and/or species may detract from signals given by associated analytes and/or species.

[00225] FIG. 18 demonstrates the estimated power of analyses in settings like those considered in FIG. 16 and FIG. 17, but with 100 total measurements at each time point, and with different fractions of the 100 variables actually being associated with the intervention. Thus, the black (left-most) line reflects 0% noise (i.e., all the analytes and/or species are associated with the intervention), the purple (second left-most) line reflects 25% noise (i.e., 75 of the analytes and/or species are associated with the intervention), the blue (third left-most) line reflect 50% noise, the green (fourth left-most) line reflects 75% noise, the red (fifth left-most) line 90% noise, the red chain-dashed (sixth left-most) line reflects 95% noise, the red dashed (seventh left-most) line reflects 99% noise, and the red dotted (right-most) line reflects 100% noise (i.e., all false positives at an assumed type I error rate of 5%). As shown in FIG. 18, more noise (e.g., caused by the inclusion of non-intervention associated analytes and/or species in the analysis) can lead to dramatic losses in power. However, as noted below, there are approaches for selecting analytes and/or species with the strongest associations with the intervention for further analyses.

[00226] Correlations among the analytes and/or species can also affect the MDMR tests in N-of-1 trial settings. As shown in FIG. 19, data were simulated as with FIGs. 16-18, but no noise was considered and 50 total simulated analytes and/or species were measured at the 20 timepoints. Different correlation strengths among the analytes were assumed, but no serial correlation was assumed. The black (left-most) line in FIG. 19 assumes 0.0 correlation among the analytes and/or species, the blue line (second left-most) line assumes a 0.05 correlation, the purple (third left-most) line assumes a 0.125 correlation, the green (fourth left-most) line assumes a 0.2 correlation, the red (fifth left-most) line assumes a 0.5 correlation, the red dashed (sixth leftmost) line assumes a 0.8 correlation, and the red chained-dotted (seventh left-most) line assumes a 0.95 correlation. As can be seen, very strong correlations among the analytes and/or species tested can lead to dramatic reductions in power, since correlated variables reduce the effective number of analytes and/or species studied.

[00227] Serial correlation can also have a dramatic effect on the rate of false positive results produced by the proposed MDMR tests, but it can be mitigated through the use of different clinical trial designs. FIG. 20 shows results of data simulations with the same general settings as FIGs. 15-19, but focus was on the impact of serial correlation (from 0 to 1, as reflected on the x axis). No correlation among the analytes and/or species was assumed, but a 0.0 effect size of the intervention was assumed. Since a 5% type I error rate was assumed, any deviation from 5% power (i.e., 0.05 on the y-axis) in FIG. 20 reflects an artifact in the data attributable to serial correlation among the analytes and/or species collected over time. The black line in FIG. 20 assumes 500 measurements made at each of the 20 timepoints, the blue (leftmost) line assumes 50 measurements, the green line 5 measurements, and the red line 1 measurement. FIG. 20 shows that the stronger the serial correlation, the greater the false positive rate. This is intuitive since a simple interrupted 2-period time series design was used, and serial correlation among the analytes and/or species in first 10 timepoints and the serial correlation among them in last 10 timepoints create greater similarities among the measurements made within the time periods, giving the impression of an intervention effect.

[00228] However, such potential for false positives in N-of-1 designs may be addressed by breaking up the treatment or dosing periods into smaller units, randomizing them, and using critical values in the statistical tests of the hypothesis of an intervention effect that have been adjusted for serial correlation - a strategy described in the context of serial correlation in the univariate and small multivariate designs provided herein. The dotted lines in FIG. 20 provide estimated power curves (color coding the same as the solid lines reflecting the number of measurements made at each timepoint) when the twenty timepoints are broken up in 10 x 2 alternating treatment and no treatment periods, instead of 2 x 10 treatment periods. It can be seen that by adopting the 10 x 2 treatment period design, the elevated false positive rates are eliminated, with tests actually becoming conservative (i.e., closer to 0.0 than the expected 0.05) with very high serial correlations. This can be overcome by using serial-correlation corrected critical values for the tests, or simple permutation or randomization tests of the hypothesis of an intervention effect.

[00229] The most likely analytes or species affected by the intervention may be identified as follows. In order to identify those analytes or species that are affected in the most pronounced way by the intervention, each analyte may be analyzed separately and then ranked by its effect size, controlling for multiple comparisons issues by adopting false discovery rate (FDR) strategies (e.g., as described elsewhere herein). In addition, post-hoc tests of each analyte may be used, via the MDMR strategy [7],

[00230] As an alternative strategy, multivariate techniques may be used, such as factor and latent roots analysis [8], to identify the analytes and/or species exhibiting the strongest signals for a dose (intervention) effect. For example, the dose variable may be added as another variable to the data matrix with the m analytes/species collected at each timepoint (creating a matrix with m+1 variables). Cluster or factor analysis may be performed on this matrix to evaluate which of these factors is either most strongly related to dose — and thereby indicate that those analytes or species making up those factors are associated with dose — or identify a factor or cluster that includes dose information as a distinguishing feature. As an example of the latter approach, a setting was simulated in which 20 time points were assessed with 10 assessments during a baseline period and 10 during an intervention period. We simulated 10 analyte/species measures being collected at each of these 20 timepoints. The simulated counts of these analytes and/or species each had an error distribution that followed a standard normal distribution with no serial correlation. We also assumed no correlations among the analytes and/or species. We assumed the hypothetical intervention affected 5 of these analytes and/or species with a pronounced effect size of 5.0 standard deviation units. We then took the matrix with the 11 total variables (i.e., dose variable + the 10 analytes and/or species) and subjected it to a factor analysis and identified a single factor producing the following 11 variable loadings: first variable (l)=0.940, 2=0.770, 3=0.625, 4=0.888, 5=0.823, 6=0.623, 7=0.157, 8=-0.168, 9=0.203, 10=0.143, and 11 =-0.180.

The first variable is the dosing variable, variables 2-6 are the analytes and/or species simulated so as to be affected by the intervention, and variables 7-11 are analytes and/or species not associated with the intervention. It can be seen that the associated analytes and/or species and the dosing variable load more heavily on the factor and are thereby captured as associated with the intervention as opposed to the second 5. Any other distance-based technique that searches for structure in a data set with a large number of variables can be used in this way.

[00231] References

[00232] 1. Nievergelt, C.M., O. Libiger, and N.J. Schork, Generalized analysis of molecular variance. PLoS Genet, 2007. 3(4): p. e51.2, is incorporated by reference herein in its entirety.

[00233] 2. Salem, R.M., D.T. O'Connor, and N.J. Schork, Curve-based multivariate distance matrix regression analysis: application to genetic association analyses involving repeated measures. Physiol Genomics, 2010. 42(2): p. 236-47, is incorporated by reference herein in its entirety. [00234] 3. Schork, N.J., J. Wessel, and N. Malo, DNA sequence-based phenotypic association analysis. Adv Genet, 2008. 60: p. 195-217, is incorporated by reference herein in its entirety.

[00235] 4. Wessel, J. and N.J. Schork, Generalized genomic distance-based regression methodology for multilocus association analysis. Am J Hum Genet, 2006. 79(5): p. 792-806, is incorporated by reference herein in its entirety.

[00236] 5. Zapala, M.A. and N.J. Schork, Multivariate regression analysis of distance matrices for testing associations between gene expression patterns and related variables. Proc Natl Acad Sci U S A, 2006. 103(51): p. 19430-5, is incorporated by reference herein in its entirety.

[00237] 6. Zapala, M.A. and N.J. Schork, Statistical properties of multivariate distance matrix regression for high-dimensional data analysis. Front Genet, 2012. 3: p. 190, is incorporated by reference herein in its entirety.

[00238] 7. McArtor, D.B., G.H. Lubke, and C.S. Bergeman, Extending multivariate distance matrix regression with an effect size measure and the asymptotic null distribution of the test statistic. Psychometrika, 2017. 82(4): p. 1052-1077, is incorporated by reference herein in its entirety.

[00239] 8. Brian Everitt and Torsten Hothom. An Introduction to Applied Multivariate

Analysis with R (Use R!) 2011th Edition; New York: Springer, is incorporated by reference herein in its entirety.

[00240] While preferred embodiments of the present invention have been shown and described herein, it may be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions may now occur to those skilled in the art without departing from the invention. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. It is therefore contemplated that the invention shall also cover any such alternatives, modifications, variations, or equivalents. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.