The Reward Deficits Model And Computational Neural Mechanism Of Anhedonia In Major Depression

Depressive disorder is a highly prevalent functionally disabling mental disorder,with depressed mood and anhedonia as key symptoms.Depressive disorder with anhedonic symptoms indicate more years of illness,higher risk of suicide and impairment of social function.However,the psychopathological mechanism of anhedonia is unclear yet,which may cause poor treatment effectiveness.Anhedonia is a heterogeneous concept,which could be either trait-like or state-like.Trait-like anhedonia may act as a vulnerable factor of depression,while state-like anhedonia may manifest as either a precursor or an early symptom of depression.Anhedonia is closely related to dopamine(DA)brain reward system(BRS)dysfunction.Accumulating evidence indicated reward hypo-sensitivity as a vulnerable factor and an endophenotype of depression.However,there still lacks coherent operational concept of reward sensitivity.Based on reward component model,anhedonia could be divided into experience,motivation and learning component,and therefore,be conceptualized as the disability to pursue,experience and/or learn pleasure.Traditional measurement such as psychological scale or behavioral task mainly focused on the experiential component of anhedonia,and could not fully capture the other components(e.g.motivation and learning).Previous study examined the reward deficits of experiential,motivational and learning component.However,the following questions remained to be answered:(1)The measurement of reward deficits mainly focused on comparison between groups with different diagnostic category(depressive disorder)rather than continuous measurement of dimensional construct(anhedonia)which could be correlated with different reward deficits.According to RDoC,anhedonia could be seen as a latent construct across different analysis units(such as neurons and pathways,physiological index and self-report),so that the reward components could be extracted using factor analysis.(2)The neural substrates of reward deficits could be explained in terms of hedonia hypothesis(HH),incentive salience hypothesis(ISH)and reward-prediction error hypothesis(RPEH),which respectively related to experience,motivation and learning component of reward processing.The RPEH with the most depth of explanation could be utilized to explain the anhedonia phenotype.Optogenetic functional magnetic resonance imaging evidence showed abnormal RPE signals in regions of mPFC-VTA-striatal pathway in animal models.Human functional neuroimaging also indicated the decreased model-free RPE signals in ventral striatum as well as increased/decreased RPE signal in VTA.The discrepancies of results may be explained with the receiving-ends deficits hypothesis,which suggested the abnormal neural signal in striatum.Notably,the subregions of striatum(caudate/putamen)are involved in goal-directed and habitual reward learning,during which these two subregions encode model-based and model-free RPE signals respectively.Furthermore,behavioral evidence indicated the dampened goal-directed behavior in individuals with depressive symptoms.Therefore,we hypothesized that subregions of striatum in major depression may be differentially impaired during encoding of model-based and model-free RPE signals.To validate the reward measurement and deficits of anhedonia in depression,(1)principle component analysis was conducted on the reward measurements(psychological/behavioral)to extract reward components with the aim to validate the reward component model.The extracted reward indices could then be used to discriminate depressed patients from healthy controls and predict the depressive symptoms one year later.The most sensitive and specific reward index was screened through machine learning to discriminate depressed patients from healthy controls.(2)Based on diagnostic categorical approach,the current study adopted computational modelling and functional neuroimaging to examine the neural substrates of reward learning deficits.The current study aims to clarify the reward deficits of anhedonia in depression,which may highlight the clinical significance in early detection and identification of effective intervention targets of anhedonia in depressive disorder.Methods:Study 1:Two hundred and twenty-four normal individuals and fourty-nine depressed patients were randomly selected.Among the normal individuals,fourty-one were selected as normal control group.TEPS,MASQ?AD,BDI and PSS were utilized to measure the reward experience and motivation,anhedonic and depressive symptoms,as well as stress levels of the two groups.Two-stage markov decision task was utilized to measure the reward learning.(1)Based on reward component model,reward components were extracted from typical reward measurements using principle component analysis(PCA)which confirmed the construct validity of reward component model.(1)Convergent validity analysis revealed the relationships among the reward components.(2)Discriminant validity analysis yielded whether reward measurements could discriminate individuals with different depressive(or anhedonic)symptoms.(3)Predictive validity analysis examined whether reward measurements could predict depressive symptoms one year later.(4)Test-retest reliability analysis yielded the consistencies between reward learning performances over one month.(2)The reward features selected using machine learning algorithms could screen the single best predictive feature and the best combinations of predictive features for major depressive disorder and individuals with high anhedonic symptom.(2)The mediating model of reward experience,motivation and learning components were developed,which indicated the reward learning as the mediator between reward experience and learning.Study 2:Fourty-one normal control individuals and fourty-nine depressed patients were randomly selected.TEPS,MASQ?AD,BDI and PSS were utilized to measure the reward motivation and experience,anhedonic and depressive symptoms,as well as stress levels of the two groups.Two-stage markov decision task was utilized to measure the reward learning.The repeated-measures ANOVA(RM-ANOVA)and logistic linear mixed-effects model(GLMM)were conducted to examine the reward deficits of depression.Stress-reward deficits model were tested using mediating model with model-based and model-free reward learning as mediators between stress and depressive(or anhedonic)symptoms.Study 3:Twenty-one normal control individuals and nineteen depressed patients were randomly selected.TEPS,MASQ?AD,BDI and PSS were utilized to measure the reward motivation and experience,anhedonic and depressive symptoms,as well as stress levels of the two groups.Two-stage markov decision task was utilized to measure the reward learning.Functional magnetic neuroimaging technique combined with computational modelling was conducted to examine the neural substrates of impaired model-based and model-free reward prediction error signals of major depressive disorder.Reward prediction error BOLD signal in regions of interest were included as mediators to examine the mediating effects between reward learning and depressive(or anhedonic)symptoms.Results:1.The psychometrics of reward measurements of anhedonia(1)The test-retest reliability of model-free component was statistically nonsignificant(r=0.112,P=0.240),while the test-retest reliability of model-based component was statistically significant(r=0.263,P=0.046).(2)The convergent validity analysis indicated that reward experience and model-free reward learning(r=0.124,P=0.034),behavioral activation(r=0.595,P<0.001)were positively correlated;model-based and model-free reward learning were positively correlated(r=0.417,P<0.001).(3)The construct validity analysis revealed that,(1)principle component analysis yielded a three-factor structure(reward experience,reward learning and reward motivation)accounting for 66.781%of the variance in first-stage choice behavior.(2)behavioral inhibition was independent of reward system,with a four-factor structure(reward experience,reward learning and reward motivation,behavioral inhibition)accounting for 71.245%of the variance in first-stage choice behavior;(3)behavioral inhibition was independent of reward system,with a four-factor structure(reward experience,reward learning and reward motivation,loss learning)accounting for 81.176%of the variance in first-stage choice behavior.(4)The discriminative validity analysis yielded that:(1)anticipatory pleasure(AP),consummatory pleasure(CP),behavioral activation and model-based reward learning(MB-RL)were statistically significantly differentiated between normal control group and depressed patients(Ps<0.05);(2)anticipatory pleasure(AP),consummatory pleasure(CP),behavioral activation(BAS)and behavioral inhibition(BIS)were statistically significantly differentiated between high and low anhedonic individuals(Ps<0.05);(3)anticipatory pleasure(AP),consummatory pleasure(CP)and behavioral activation(BAS)were statistically significantly differentiated between individuals with and without depressive symptoms(BDI>4 or BDI?4)(Ps<0.05).(5)The predictive validity analysis revealed that:(1)model-based reward learning(MB-RL)positively predicted concurrent depressive symptoms(r=0.149,n=202,P=0.017)and depressive personality a year later(r=-0.126,n=202,P=0.037).(2)model-free reward learning(MF-RL)negatively predicted concurrent anhedonic symptoms(r=-0.126,n=202,P=0.037).2.Machine learning(ML)prediction model of depression and anhedonic symptoms based on reward componentsBased on reward component model and results of discriminate validity of reward indices,measurements of reward experience,motivation and learning were included as reward features.Machine learning algorithm including logistic regression(LR),support vector machine(SVM),K-Nearest Neighbor(KNN),decision tree(DT),Naive Bayesian(NB),quadratic discriminant analysis(QDA)and adaptive boosting(Adaboost)were utilized to predict depressed patients and anhedonic individuals with optimal single feature and set of features.(1)Prediction model of depression based on reward components and machine learningTransition type(one component of model-based reward learning)is the most predictive feature of depressed patient,with LR as the optimal model to predict depression(SN=0.907,SP=0.951).An optimal set of features to predict depression were selected,including anticipatory pleasure(AP)and model-based reward learning,with NB as the optimal model to predict depression(SN=0.930,SP=1).(2)Prediction model of anhedonic symptoms based on reward components and machine learningAP was the most predictive feature of depression,with LR as the optimal model to predict anhedonic individuals(SN=0.897,SP=0.542).An optimal set of features to predict depression were selected,including depressive symptoms,anticipatory pleasure(AP)and drive,with DT as the optimal model to predict anhedonic individuals(SN=0.832,SP=0.860).3.Reward motivation-learning-experience mediating effect modelThe model fit index(?²=10.664,df=11,P=0.472)indicated that mediating effect of reward motivation(implicit/explicit incentive salience)was significant between reward learning(model-based and model-free)and reward experience(behavioral activation,anticipatory pleasure and consummatory pleasure).4.Stress-reward learning deficits model of depression(1)The main effects of group and reward outcome on first-stage choice were significant(F=22.72,df=1,P<0.001;F=8.24,df=1,P=0.005).When taking anhedonia as the covariate(with nonsignificant effects on first-stage choice),the main effects of group and reward outcome were still significant(F=23.11,df=1;P<0.001;F=8.24,df=1,P=0.005).(2)For normal control group,stress could influence depressive symptoms through the mediating effect of model-based reward learning(?=19.15,SE=8.94,P=0.03,95%CI=1.54?36.76).(3)For patient group,stress could influence depressive symptoms through the mediating effect of model-based(?=-211.56,SE=91.85,P=0.03,95%CI=-397.34?-25.77)and model-free reward learning(?=-37.73,SE=16.64,P=0.03,95%CI=-71.38?-4.08).5.The computational neural mechanism of reward learning deficits in depression(1)The learning rate,eligibility trace,model-based versus model-free learning weight,and perseverance parameter of depressed patients were significantly lower than normal control group(P_Bonferronis<0.007).The model evidence(BIC,Laplace approximation of BIC and negative log likelihood)showed significant differences between the two groups.Hybrid model best fitted the choice behavior of normal control group,while model-free model best fitted the choice behavior of depressed patients.(2)Ventral striatum(NAc),dorsal striatum(putamen),lateral and medial OFC(l/m OFC)were significantly activated while encoding model-free prediction error in normal control group;lateral prefrontal cortex was significantly involved in model-based prediction error.(3)While encoding the model-free prediction error,(1)midbrain/VTA and lateral prefrontal cortex(l PFC)were significantly more activated in depressed patient group;(2)midbrain/VTA-NAc functional connectivity was positive(r=0.941,P<0.001);(3)midbrain/VTA mediated the relationship between model-free reward learning and anhedonia.While encoding model-based prediction error,(1)right medial prefrontal cortex(mPFC)was significantly more activated and(2)dorsal striatum(putamen/caudate)as well as lateral and medial prefrontal cortex(l/m OFC)were significantly more activated in depressed patient group.(4)The model-free prediction error signal in regions of interest for depressed patients was negatively correlated with depressive symptoms(BDI):(1)the weaker the activation in medial orbital-frontal cortex,the more severe the depressive symptoms,(2)the weaker the activation of nucleus accumbens(NAc),the more severe the depressive symptoms.(5)The model-based prediction error signal in regions of interest for normal control group was positively correlated with depressive symptoms(BDI):the stronger the activation in lateral orbital-frontal cortex,the more severe the depressive symptoms.(6)For normal control group,the model-based reward learning influences depressive symptoms via the lateral orbital-frontal cortex RPE signal(?=0.026,SE=0.011,P=0.032,95%CI=0.003?-0.048).For both groups,the model-free reward learning influences anhedonic symptoms via the VTA RPE signal(?=-0.284,SE=0.118,P=0.024,95%CI=-0.528?-0.041)and caudate(?=0.235,SE=0.104,P=0.032,95%CI=0.022?0.449).Conclusions:(1)Anhedonia can be conceptualized as reward deficits,including reward motivation,learning and experience deficits.Reward motivation deficits manifested as dampened behavioral activation,reward experience deficits manifested as decreased anticipatory and consummatory pleasure,and reward learning deficits manifested as model-based and model-free reward learning.(2)Reward experience(anticipatory pleasure)and motivation(drive)could best classify the high versus low anhedonic individuals,model-free reward learning could predict concurrent anhedonia symptoms(correlational),and classify the high versus low anhedonic individuals;reward learning(model-based RL)could best classify the depressed patients versus normal control individuals.(3)Depressed patients showed model-based and model-free reward learning deficits,with lower lamda,omega and second-stage alpha and perseverance parameter.(4)Model-based and model-free reward learning deficits mediate the relationship between stress and anhedonic(or depressive)symptoms for both groups.(1)For normal control group,the individuals more inclined to adopt model-based reward learning tend to have more severe anhedonic(or depressive)symptoms.(2)For depressed patients,the individuals more inclined to adopt model-based reward learning tend to have less severe anhedonic(or depressive)symptoms.