The Simulation Studies Of The Numerical Data In Single Case Of Randomized Controlled Trials (N-of-1)

BackgroundSingle case of randomized controlled trials (N-of-1) is referred to as N-of-1trials. N-of-1trials are multicycle (≤3cycles), double-blinded controlled cross-over trials based on individuals. The two periods in each cycle are randomly assigned to different treatments for each subject with a washout period. N-of-1trials are designed to test the effect difference of two treatments which are conventionally labeled as Group A (test group) and Group B (control group).N-of-1trials can reveal the similarities and differences of the individual patient, and understand the special rules of some cases. N-of-1trials are more concerned about the treatment effect of the individual patient, meet the requirements of evidence-based medicine, and provide strong evidences for the decision of the individual patient.N-of-1trials have been increasingly utilized in social, educational sciences, biomedical, clinical areas, and notably in medical area including rheumatism, pediatric rheumatism, arthritis pain, chronic neuropathic pain, insomnia, heart disease, chronic obstructive pulmonary disease, and pediatric oncology. However, N-of-1trials have not been widely used. One reason was that N-of-1trials required relatively stable symptoms or diseases, medications with short half-lives, and rapid measurable responses. Another important reason was related to the difficulties about the statistical analysis of the data.To analyze the numerical data of N-of-1trials, a number of methods have been proposed, such as visual analysis, non-parametric tests, parametric tests (z-test, two samples t-test, paired t-test), time series, Meta-analysis of summary data (short for meta-analysis), and mixed effects model and the Bayesian hierarchical model. Among them, non-parametric tests and parametric tests were used frequently. Zucker DR used mixed-effects model to analyze the data of N-of-1trials, which had greater flexibility. However, mixed-effects model was not used widely.These methods had their advantages and disadvantages. There wsa no consensus how to analyze the data of N-of-1trials. It remained unclear which method should be adopted to provide more robust inferences for the numerical data of N-of-1trials.ObjectiveA simulation study of3-cycles and4-cycles N-of-1trials was conducted to compare the performance of four methods (paired t-test, mixed effects model of difference, mixed effects model and meta-analysis of summary data) under various variance-covariance structures, different carryover effect and difference effect.MethodsThe simulation about3-cycles and4-cycles N-of-1trials was conducted using SAS9.1.3software. N-of-1trials were set with sample sizes of1,5,10,20and30respectively under normally distributed assumption. The data were generated based on different variance-covariance matrix and different carryover effect. Type I error, power, ME (mean error), and mean square error (MSE) of effect differences between two groups were used to evaluate the performance of the four models.We assumed μA and μB as A and B intervention effect respectively, μA-μB represented the difference between the effects between the two intervention (effect difference). Taking3cycles N-of-1trials as an example, we illustrated the process of the simulation.(1) Generating6-dimensions normal distribution data:The6-dimensions normal distribution data were generated by a multivariate normal random number generator (a SAS Macro) based on mixed effect model.(2) Randomly assign the two interventions and subjects:Two groups (A or B) of each cycle in each subject were randomly assigned using "Proc Plan" in the SAS software. All the subjects in each period were also randomly allocated into Group A or Group B to assure that half of the subjects in each period receive Group A or Group B. The actual response value of each subject was produced according to the allocations. For example, the allocation sequence of the first subject in six periods was BABAAB.(3) Adding the carryover effect (residual effect). We assumed that carryover effect was caused only by the previous period, and was equal to a certain percentage of the previous treatment effect. The first period did not have carryover effect, while other five periods had the carryover effect.(4) Analysis model:①Paired t-test (Model1):In N-of-1trials, each cycle with two periods which were assigned to Group A or Group B was considered as a pair. Paired t-test was used to compare the effectiveness between the two interventions.②Mixed effects model of difference (Model2):The difference of the two groups in the same cycle was calculated. Model2could be formulated as yih=μ+τh+γi+εth. The variance-covariance matrix were set as compound symmetry (CS) or first-order autoregressive (AR).③Mixed effects model (Model3):Considering all response values in six periods, Model3was set as yij=α+μ*group+τj+λAZA+λBZB+γi+εij.The variance-covariance matrix were set as CS structure or AR structure.④Meta analysis (Model4):Each subject of N-of-1trials was considered as a separate trial (study). A typical method to analyze n>1N-of-1trials was to use meta-analysis. Meta-analysis combined summary data from each subject to form a weighted average using the method of Der-Simonian and Laird:(5) Assessment of modelsType I error, power, ME (mean error), percent error (PE) of ME, mean absolute error (AE), and mean square error (MSE) were used to assess the performances of the models. Percent error of ME was absolute of ME divided by the true effect difference.(6) Parameter setting The sample size was set to1,5,10,20and30respectively. The variances of the variance-covariance matrix (compound symmetry structure) were all set to1. Covariances were all set to0.0,0.2,0.5,0.8corresponding CS1, CS2, CS3and CS4structure. The setting of carryover effet:①equal carryover rate:Carryover rates were set to0%,10%and20%.②unequal carryover rate:Carryover rates of A, B intervention were set to0%and5%,0%and10%,10%and0%,10%and5%.③increasing carryover rate, from5%of carryover effect with1%increasing carryover rate, from10%of carryover effect with1%increasing carryover rate. B intervention effect was2.0. A intervention effect were2.0,2.2,2.4,2.6,2.8,3.0, and so no.PROC ttest, PROC mixed were used to fit the data Using SAS9.1.3software. The simulation was repeated1000times (m=1000).ResultsThe results of simulation study of3-cycles and4-cycles N-of-1trials were consistent with each other. As the number of the cycle increased (just as sample size increased), power of four models increased, and their AE, MSE decreased.(1) The results of n=1N-of-1trialsPaired t-test (Model1) and mixed-effects model of difference (Model2) had identical results. Model1, Model2(mixed-effects model) and Model3yielded type I error near to5%(the nominal level).①When there was no carryover rate for A and B interventions, or0%VS.10%, and5%VS.10%of the carryover rate for A and B interventions, the estimated values of all models were close to the true value (ME and PE were close to0). MSE of Model3was greater than that of Model1. Therefore, Model3had lower power than Model1. Model3has a higher AE than Model1.②When there was20%carryover rate for both A and B interventions, or10%VS.0%, or10%VS.5%of the carryover rate for them, the estimated value of Model1was less than the true value (ME was not equal to0), while that of Model3was close to the true value (ME was equal to0). Power of Model3was close to and even larger than that of Model1. AE and MSE of Model3were close to and even smaller than those of Model1. Price JD etc. conducted N-of-1trials about non-progressive amnestic syndrome. The results showed that P-value of Hopkins verbal learning test (HVLT), Hamilton rating scale for depression (HRSD) and SF-36in Model1was less than those of Model3, with smaller95%confidence intervals for Model1. Hackett A, etc. carried out N-of-1trials of ornithine enzyme deficiency. The results showed that plasma glutamine in Model1had smaller P-value and95%CI than Model3. These two examples were consistent with that of simulation study.(2) The results of n>1N-of-1trials①Equal and increasing carryover ratesPaired t-test (Model1) and mixed-effects model (Model3) yielded type I error near to5%(the nominal level). Type I error of mixed-effects model of difference (Model2) was obviously less than0.05, while that of Meta-analysis (Model4) were slightly larger than0.05.The estimated values of Model1and Model2deviated from the true values. The absolute value of ME was equal to about half of the carryover effects. No matter how much the carryover rate was, the estimated values of Model3were approximately equal to the true effect (ME=0). The estimated values of Model4were far away from the true effect. Power of Model1was the largest, followed by Model3and Model4. Power of Model2was the least. When there was20%carryover rate for both A and B interventions, power of the Model3was close to and even larger than that of Model1. AE and MSE of Model1were the least, followed by Model3and Model2. AE and MSE of Model4were the largest.(2) Unequal carryover ratesMixed-effects model (Model3) yielded type I error near to5%. Type I errors of other models were unstable, and usually larger than0.05.Power of Model1was the largest, followed by Model3and Model4. Power of Model2was the least. When there were10%VS.0%, or10%VS.5%of carryover rate for A and B intervention, power of Model3was close to or exceeded that of Model1. ME and PE of Model3were the least, followed by Model1and Model2. ME and PE of Model4were the largest. AE and MSE of Model1were the least, followed by Model3and Model2. AE and MSE of Model4were the largest.N-of-1trials were performed on four physicians who complained of chronic fatigue. Each physician received three pairs of treatments comprising4weeks of Spirulina platensis (test group) and4weeks of placebo (control group), with2weeks washout time. Severity of fatigue was measured on a10-point scale daily during the second half (weeks3and4) of each period. The results showed that95%CI and P value of Model1was the smallest. The second smallest95%CI was Model3, followed by Model2and Model4. The results were consistent with that of simulation study.ConclusionPaired t-test was recommended to analyze the numerical data of n=1N-of-1trials. When there was20%carryover rate for both A and B interventions, or10%VS.0%, or10%VS.5%of the carryover rate for them, mixed-effects model could be used as an alternative.For n>1N-of-1trials,(1) when there were equal and increasing carryover rates, paired t-test was recommended. When there was20%carryover rate for both A and B interventions, mixed-effects model could be used as an alternative.(2) when there were unequal carryover rates, mixed-effects model was recommended.