Table of contents¶

Exploratory Factor Analysis (EFA) Results
EFA Robustness
1. Survey EFA robustness
2. Task EFA robustness
Hierarchical Clustering (HC)
1. Survey HC clusters
2. Task HC clusters
HC Robustness
1. Survey HC clusters
2. Task HC clusters

Supplemental Figures for Uncovering mental structure through data-driven ontology discovery

This notebook contains:

Loading matrices for task, survey and outcome exploratory factor analyses
Robustness analysis for task and survey factor analyses
All clusters from the hierarchical clusting (Figures 4 and 5 from the main paper) separately plotted out with DVs labeled
Robustness analyses for hierarchical clustering, comparing the clustering reported in the paper to perturbed data and a consensus clustering approach

import matplotlib as mpl
import matplotlib.pyplot as plt
from os import path
import pandas as pd
import pickle
import seaborn as sns
pd.set_option('max_rows', 200)

from dimensional_structure import DA_plots
from dimensional_structure.EFA_cluster_robustness import ConsensusCluster
from dimensional_structure.EFA_plots import plot_heatmap_factors, plot_bar_factors, plot_factor_correlation
from dimensional_structure.HCA_plots import plot_subbranches, plot_results_dendrogram
from dimensional_structure.notebook_utils import (plot_factor_df, plot_EFA_robustness, 
                                                  plot_bootstrap_results,
                                                  plot_HCA_AMI,
                                                  display_closest_DVs,
                                                  display_cluster_DVs)
from selfregulation.utils.utils import get_recent_dataset
from selfregulation.utils.result_utils import load_results

Using TensorFlow backend.
/media/Data/Ian/miniconda/envs/SO/lib/python3.5/site-packages/IPython/parallel.py:13: ShimWarning: The `IPython.parallel` package has been deprecated since IPython 4.0. You should import from ipyparallel instead.
  "You should import from ipyparallel instead.", ShimWarning)

%matplotlib inline
dataset = get_recent_dataset()
results = load_results(datafile=dataset)

Exploratory Factor Analysis Results ¶

Below are the loading matrices for the exploratory factor analysis (EFA) solutions for surveys, tasks, and the outcome measures.

Survey Exploratory Factor Analysis Loadings ¶

12 factors were determined using a BIC criteria for exploratory factor analysis. The 64 survey DVs are grouped and ordered based on the largest (absolute) factor loading for that DV. Hover over the cells to see the exact value

fig = plt.figure(figsize=(12, 3))
ax1 = fig.add_axes([0.05, 0.80, 0.9, 0.15])
cmap = sns.diverging_palette(220,15,n=20, as_cmap=True)
cb1 = mpl.colorbar.ColorbarBase(ax1, cmap=cmap,
                                orientation='horizontal')
cb1.set_ticks([0,.5,1])
cb1.set_ticklabels(['-1','0', '1'])
cb1.set_label('Color Bar for Factor Loading Plots')

survey_results = results['survey']
plot_factor_df(survey_results.EFA)

Task Exploratory Factor Analysis Loadings ¶

5 factors were determined using a BIC criteria for exploratory factor analysis. The 129 survey DVs are grouped and ordered based on the largest (absolute) factor loading for that DV. Dotted lines indicate separate groups derived from this criteria, and are used for visualization purposes only.

task_results = results['task']
plot_factor_df(task_results.EFA)

Outcome Exploratory Factor Analysis Loadings ¶

9 factors were determined using a BIC criteria for exploratory factor analysis. The 55 target measures are grouped and ordered based on the largest (absolute) factor loading for that target measure. Dotted lines indicate separate groups derived from this criteria, and are used for visualization purposes only.

plot_factor_df(task_results.DA)

Factor Robustness Analyses ¶

Factor robustness proceeded in two ways:

(1) By dropping one measure at a time, recalculating the survey factor solution (with the same number of factors as the full sample) and correlating the new factor loadings with the original factor loadings. This correlation is calculated on all DVs excepting those dropped out because of the dropped measure. Tables for these correlations are shown with values below .9 highlighted in red

(2) By using a bootstrap procedure (see fa.sapa), which creates confidence intervals for each loading. 95% confidence intervals are plotted as bar plots for each loading

Survey Robustness ¶

Measurement Drop¶

Individual surveys sometimes had large effects on the factor structure, likely because of spare measurement of highly discriminat psychological constructs. All dimensions that are affected by a measure drop out (correlation pre-post drop out of factor loadings < .9) are highlighted. Worse deviation are colored darker.

f = path.join(results['survey'].get_output_dir(), 'EFAdrop_robustness.pkl')
survey_EFA_robustness = pickle.load(open(f, 'rb'))
plot_EFA_robustness(survey_EFA_robustness)

Bootstrap¶

It's clear that loadings are robust to the particulars of the sample. The standard deviation of the loadings are very small relative to the mean loadings.

plot_bootstrap_results(survey_results.EFA.get_boot_stats())

# of components not specified, using BIC determined #

Task Robustness ¶

Measurement Drop¶

Task factors are more robust to dropping out particular measures, likely due to the greater overlap in the psychological constructs measured by individual tasks

f = path.join(results['task'].get_output_dir(), 'EFAdrop_robustness.pkl')
task_EFA_robustness = pickle.load(open(f, 'rb'))
plot_EFA_robustness(task_EFA_robustness)

Bootstrap¶

It's clear that loadings are robust to the particulars of the sample. The standard deviation of the loadings are very small relative to the mean loadings.

plot_bootstrap_results(task_results.EFA.get_boot_stats())

# of components not specified, using BIC determined #

Hierarchical Clustering ¶

Hierarchical clustering was used to order dependent variables based on the similarity of their loading vectors. This resulted in a dendrogram, which was subset into clusters using the DynamicTreeCut algorithm. These clusters are separately plotted below, allowing the constituent DVs to be read.

Survey Clusters ¶

Below is the survey dendrogram (reproduced from the main manuscript). Following are the 12 clusters. separately plotted. The fourth and fifth clusters, referenced in the main text, together reflect canonical components of "self-control".

_ = plot_results_dendrogram(survey_results, size=20, drop_list=[0,2,4,6,8,10])

plot_subbranches(survey_results, size=6)

# of components not specified, using BIC determined #

[<Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>]

Task Clusters ¶

Below is the task dendrogram (reproduced from the main manuscript). Following are the 15 of the 16 clusers separately plotted. One cluster composed of a single DV (towards the center of the dendrogram) is not plotted. The single DV is the "Positive Learning Bias" from the "Probabilistic Selection Task"

_ = plot_results_dendrogram(task_results, size=20, drop_list=[1,5,9,12,15], double_drop_list=[2,4,6,8,14])

plot_subbranches(task_results, size=6)

# of components not specified, using BIC determined #

[<Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>,
 <Figure size 432x864 with 5 Axes>]

Cluster Robustness Analyses ¶

Clustering was repeated 5000 times with perturbed loading matrices. First, each simulation used only 80% of the total DVs. Second, each loading value was sampling from a gaussian with mean and standard deviation determined by the EFA bootstrap method. Finally, the same clustering approach described in the main paper is applied (hierarchical clustering followed by dynamicTreeCut).

Each simulation produces a co-occurence matrix: 1 if two variables were clustered together, and 0 otherwise. Averaged over simulations this produces a more robust estimate of the probability two DVs should be clustered (ranging from 0% to 100%). 1-P(clustering together) is then used as a distance matrix to perform one final clustering which creates a "consensus cluster".

We provide plots to show three aspects of the clustering:

The AMI values when comparing the original clustering to simulations or the consensus clustering
The list of DVs that are most often clustered with each DV (i.e., closest DVs to each DV according to the consensus clustering
The closest variables contained in the same cluster according to the original clustering solution above

Survey Cluster Robustness ¶

AMI values ¶

f = path.join(results['survey'].get_output_dir(), 'cluster_robustness.pkl')
survey_cluster_robustness = pickle.load(open(f, 'rb'))
survey_consensus = survey_cluster_robustness['consensusClust']
plot_HCA_AMI(survey_consensus)

/media/Data/Ian/miniconda/envs/SO/lib/python3.5/site-packages/sklearn/metrics/cluster/supervised.py:732: FutureWarning: The behavior of AMI will change in version 0.22. To match the behavior of 'v_measure_score', AMI will use average_method='arithmetic' by default.
  FutureWarning)

Closest values according to consensus clustering ¶

Hover over each cell to bold the DV. Columns are organized by first closest DV to eighth closest DV. Each cell contains a DV name and the percent of simulations it was clustered with the DV defining the row.

display_closest_DVs(survey_consensus, n_closest=8)

Consensus clustering distances of variables within the same cluster according to original cluster ¶

Hover over each cell to bold the DV. Columns are organized by closeness of DVs within cluster.

For instance, the first variable BIS11 Survey: Attentional is part of a cluster with 4 variables called "Mindfulness". The first two closest DVs were always clustered with BIS11: Attentional (across the 5000 simulations), while the furthest DV was only clustered with it 93% of the time.

Each cell contains a DV name and the percent of simulations it was clustered with the DV defining the row. Rows with more "red" indicate that the DV in that row was closer to all other DVs within the cluster.

display_cluster_DVs(survey_consensus, survey_results)

Task Cluster Robustness ¶

Adjusted mutual information ¶

Below is the distribution of adjusted mutual information scores comparing the original task clustering reported in the main paper to simulated clusters and the consensus cluster.

f = path.join(results['task'].get_output_dir(), 'cluster_robustness.pkl')
task_cluster_robustness = pickle.load(open(f, 'rb'))
task_consensus = task_cluster_robustness['consensusClust']
plot_HCA_AMI(task_consensus)

Closest values according to consensus clustering ¶

Hover over each cell to bold the DV. Columns are organized by first closest DV to eighth closest DV. Each cell contains a DV name and the percent of simulations it was clustered with the DV defining the row.

display_closest_DVs(task_consensus)

Consensus clustering distances of variables within the same cluster according to original cluster ¶

Hover over each cell to bold the DV. Columns are organized by closeness of DVs within cluster.

For instance, the first variable, the Adaptive N-back drift rate, is part of a cluster with 9 DVs called "Conflict Processing". The closest DV was clustered with the adaptive n-back drift rate 79%, while the furthest DV in the cluster was associated 35% of the time.

Each cell contains a DV name and the percent of simulations it was clustered with the DV defining the row.Rows with more "red" indicate that the DV in that row was closer to all other DVs within the cluster.

display_cluster_DVs(task_consensus, task_results)

	Mindfulness	Emotional Control	Goal-Directedness	Agreeableness	Reward Sensitivity	Sensation Seeking	Financial Risk-Taking	Social Risk-Taking	Ethical Risk-Taking	Eating Control	Impulsivity	Risk Perception
five facet mindfulness survey: act with awareness	0.89	-0.016	0.051	-0.044	-0.01	0.0075	-0.0024	0.0023	-0.027	0.033	-0.024	0.019
mindful attention awareness survey: mindfulness	0.81	-0.015	-0.11	0.1	-0.015	-0.02	-0.018	0.0077	0.0025	-0.14	-0.023	-0.031
bis11 survey: Attentional	-0.57	-0.21	-0.059	0.034	0.12	0.03	-0.012	0.045	0.00064	0.012	0.083	-0.006
five facet mindfulness survey: describe	0.41	0.14	0.087	0.16	0.16	-0.077	0.052	0.26	0.011	-0.017	0.038	0.015
brief self control survey: self control	0.28	0.18	0.22	0.1	-0.083	0.068	0.0094	-0.081	-0.081	-0.27	-0.26	0.041
ten item personality survey: emotional stability	0.082	0.71	-0.0071	0.044	0.02	0.049	0.037	0.0022	-0.023	-0.067	-0.028	-0.008
bis bas survey: BIS	-0.066	-0.7	0.0074	0.25	0.0016	-0.06	-0.075	-0.024	-0.075	0.12	-0.087	0.066
time perspective survey: past negative	-0.069	-0.57	-0.12	-0.24	0.033	0.076	-0.034	0.022	0.061	0.071	-0.096	0.11
five facet mindfulness survey: nonreact	0.083	0.48	-0.037	0.068	0.033	0.086	0.0059	0.15	0.0039	-0.028	-0.17	0.01
future time perspective survey: future time perspective	-0.069	0.45	0.15	0.23	0.25	-0.035	0.0048	0.078	0.0004	0.0068	-0.05	0.044
five facet mindfulness survey: nonjudge	0.23	0.44	-0.082	0.12	0.00098	-0.16	-0.057	0.043	-0.11	-0.04	0.18	-0.06
erq survey: reappraisal	5.2e-05	0.36	0.066	0.23	0.18	0.071	-0.054	0.1	0.076	-0.0091	-0.1	0.063
eating survey: cognitive restraint	-0.022	-0.14	0.11	-0.0082	0.068	0.0094	0.082	0.027	-0.035	-0.12	-0.12	0.092
selection optimization compensation survey: optimization: ReflogTr	-0.13	-0.1	-0.59	-0.054	0.05	-0.11	-0.06	-0.016	0.0093	0.11	0.013	-0.054
selection optimization compensation survey: compensation	-0.024	-0.0039	0.56	0.18	0.053	0.027	0.033	0.056	-0.0055	-0.17	0.016	0.036
time perspective survey: future	0.12	-0.16	0.5	-0.00028	0.015	-0.037	-0.08	0.016	-0.04	0.012	-0.43	0.065
selection optimization compensation survey: elective selection	0.24	-0.047	0.44	-0.053	-0.15	-0.046	0.033	-0.13	0.027	-0.017	0.012	0.057
grit scale survey: grit	0.41	0.21	0.43	-0.059	0.0058	0.082	-0.079	-0.02	-0.024	-0.091	-0.0098	0.072
upps impulsivity survey: lack of perseverance	-0.31	-0.17	-0.4	0.052	-0.16	-0.086	0.093	-0.011	0.0042	-0.0058	0.3	-0.11
selection optimization compensation survey: loss based selection	-0.13	-0.11	0.37	0.067	-0.16	-0.02	0.073	0.092	-0.1	-0.058	-0.12	0.029
self regulation survey: control	0.22	0.27	0.34	0.05	0.13	-0.003	-0.038	0.081	-0.065	-0.14	-0.29	0.043
time perspective survey: present fatalistic	-0.008	-0.2	-0.29	-0.039	0.14	0.045	-0.095	-0.22	0.14	0.062	0.25	0.093
theories of willpower survey: endorse limited resource	-0.053	-0.22	-0.27	0.084	0.088	-0.063	0.019	-0.21	0.094	0.2	-0.034	-0.063
sensation seeking survey: boredom susceptibility	-0.09	-0.0056	0.01	-0.44	0.2	0.068	0.053	0.15	0.13	-0.046	0.064	-0.09
ten item personality survey: agreeableness	0.2	0.16	-0.071	0.43	0.13	0.077	-0.062	-0.13	-0.075	0.012	-0.056	0.11
time perspective survey: past positive	0.077	0.19	0.18	0.38	0.25	-0.039	-0.0014	-0.23	0.007	0.076	-0.012	-0.0018
erq survey: suppression	-0.26	0.16	-0.094	-0.3	-0.3	0.24	-0.0049	-0.13	0.075	0.069	-0.24	0.088
bis bas survey: BAS fun seeking	-0.061	0.025	-0.19	-0.035	0.62	0.27	0.074	0.068	0.0085	-0.0073	0.082	-0.028
time perspective survey: present hedonistic	0.026	0.049	-0.14	0.037	0.58	0.2	0.077	0.019	0.046	0.027	0.31	0.057
bis bas survey: BAS drive	-0.032	0.15	0.39	-0.097	0.58	0.0087	0.062	-0.059	0.07	0.04	-0.032	0.001
bis bas survey: BAS reward responsiveness	-0.045	-0.1	0.15	0.22	0.57	0.00034	-0.044	0.045	-0.089	0.09	-0.15	0.071
ten item personality survey: openness	0.072	0.045	-0.15	-0.091	0.48	0.12	0.061	0.26	-0.059	-0.18	-0.095	0.045
ten item personality survey: extraversion	0.19	0.26	0.11	0.066	0.32	-0.023	0.038	0.014	0.034	-0.023	0.28	0.043
sensation seeking survey: disinhibition	0.0057	-0.076	-0.066	-0.15	0.27	0.22	0.1	0.16	0.22	0.032	0.0061	-0.14
five facet mindfulness survey: observe	0.11	-0.2	0.074	0.13	0.26	-0.025	0.089	0.24	0.02	-0.12	-0.22	0.13
sensation seeking survey: thrill adventure seeking	-0.015	-0.0017	0.054	-0.00055	-0.029	0.92	0.019	-0.015	-0.045	0.018	0.035	-0.037
impulsive venture survey: venturesomeness	0.065	0.0088	-0.024	-0.094	0.077	0.86	0.041	0.023	-0.066	-0.003	0.06	-0.034
dospert rt survey: recreational	-0.045	0.012	0.053	0.055	-0.078	0.85	0.0052	0.051	0.071	-0.01	-0.02	-0.06
upps impulsivity survey: sensation seeking	0.035	0.042	-0.034	-0.11	0.2	0.81	0.051	-0.03	0.013	0.03	0.018	-0.026
dospert eb survey: recreational	-0.079	-0.063	-0.0011	0.19	-0.068	0.73	0.019	0.13	0.18	-0.028	-0.029	0.071
dospert rt survey: financial	0.047	0.031	0.0086	-0.023	0.036	0.13	0.79	-0.025	-0.017	0.023	0.0024	0.068
dospert eb survey: financial	-0.026	0.02	-0.035	0.023	-0.059	-0.041	0.67	0.062	0.21	0.1	-0.12	0.1
dospert rp survey: financial	0.05	0.033	-0.081	-0.039	0.0066	0.092	-0.6	0.026	0.16	0.028	-0.099	0.41
dospert rt survey: social	-0.003	0.041	-0.0062	-0.084	0.035	0.034	0.016	0.78	-0.043	-0.0033	-0.0047	-0.054
dospert eb survey: social	0.0054	-0.0088	0.088	0.088	-0.12	0.083	-0.0081	0.6	0.23	0.057	0.035	0.16
sensation seeking survey: experience seeking	0.016	-0.0071	-0.17	-0.071	0.15	0.28	0.059	0.36	0.081	-0.012	-0.012	-0.036
dospert eb survey: ethical	-0.074	0.078	-0.021	-0.017	-0.028	-0.00091	0.073	-0.025	0.64	0.096	-0.12	-0.12
dospert eb survey: health safety: logTr	0.015	0.0057	-0.0037	0.084	-0.013	0.079	0.11	0.03	0.64	-0.048	0.13	-0.099
dospert rt survey: ethical	-0.067	-0.005	0.075	-0.24	-0.0026	-0.019	0.049	0.022	0.5	0.16	0.058	-0.2
dospert rt survey: health safety	0.043	-0.11	-0.0044	-0.08	0.069	0.1	0.02	0.19	0.49	0.014	0.11	-0.24
eating survey: uncontrolled eating	-0.043	0.094	0.02	0.02	0.016	0.053	0.016	-0.031	0.051	0.87	-0.027	0.0067
eating survey: emotional eating	0.082	-0.081	-0.032	0.024	-0.043	-0.035	0.045	0.063	-0.075	0.83	-0.032	-0.003
upps impulsivity survey: lack of premeditation	-0.058	0.1	0.0032	-0.056	-0.013	0.11	0.062	0.015	0.016	-0.049	0.8	-0.0096
bis11 survey: Nonplanning	-0.12	-0.11	-0.28	0.058	-0.056	0.023	-0.063	-0.057	0.036	0.12	0.52	-0.054
bis11 survey: Motor: logTr	-0.18	-0.088	-0.11	0.068	0.16	0.041	0.056	0.072	0.09	0.069	0.47	0.027
mpq control survey: control: ReflogTr	-0.17	-0.034	-0.21	-0.069	0.15	0.1	0.01	0.2	-0.07	0.047	0.44	0.02
upps impulsivity survey: positive urgency	-0.11	-0.13	0.11	-0.15	0.22	0.063	0.02	-0.11	0.17	0.22	0.43	-0.014
upps impulsivity survey: negative urgency	-0.15	-0.38	0.049	-0.1	0.16	-0.013	-0.028	0.052	0.12	0.26	0.42	0.033
ten item personality survey: conscientiousness: ReflogTr	-0.23	-0.11	-0.29	0.069	-0.1	-0.025	0.079	0.057	0.038	0.081	0.34	-0.019
dickman survey: functional	0.13	0.23	0.1	-0.2	0.23	0.16	0.11	0.09	-0.1	-0.045	0.24	0.044
dospert rp survey: health safety	-0.0093	-0.017	0.043	0.00039	-0.00049	-0.044	-0.044	0.022	-0.21	0.054	-0.017	0.79
dospert rp survey: ethical	0.036	-0.044	0.035	0.12	0.00028	0.029	-0.059	0.021	-0.2	-0.05	0.087	0.72
dospert rp survey: recreational	0.0012	0.012	0.006	-0.083	0.034	-0.38	-0.036	0.052	0.079	-0.029	0.016	0.64
dospert rp survey: social	-0.02	-0.11	-0.026	-0.012	0.022	-0.017	0.15	-0.34	0.3	-0.085	-0.043	0.54

	Strategic IP	Speeded IP	Caution	Perc / Resp	Discounting
adaptive n back: mean load: logTr	0.61	0.24	0.064	-0.011	-0.0043
spatial span: forward span	0.56	-0.022	8.5e-05	-0.088	-0.043
spatial span: reverse span	0.55	0.039	0.067	-0.088	0.015
cognitive reflection survey: correct proportion	0.53	0.092	0.045	0.11	-0.22
tower of london: avg move time: logTr	-0.5	0.15	0.14	0.3	-0.068
ravens: score	0.48	0.35	-0.05	0.061	-0.15
cognitive reflection survey: intuitive proportion	-0.43	-0.056	-0.032	-0.069	0.11
columbia card task cold: information use	0.41	0.21	-0.024	-0.026	-0.12
adaptive n back: DDM-drift load	0.39	-0.11	0.13	0.024	-0.022
columbia card task hot: information use	0.39	0.14	-0.0072	-0.07	-0.026
columbia card task hot: loss-sensitivity: ReflogTr	0.37	0.032	0.025	-0.021	0.041
information sampling task: Fixed Win motivation	0.37	-0.26	-0.058	-0.29	0.11
two step decision: model based	0.35	0.25	0.048	0.087	-0.14
columbia card task hot: gain-sensitivity: logTr	0.35	-0.034	-0.065	-0.023	0.044
information sampling task: Fixed Win P correct	0.35	0.26	0.071	0.038	-0.008
columbia card task cold: gain-sensitivity: logTr	0.34	-0.031	-0.032	0.038	-0.04
angling risk task always sunny: keep adjusted clicks	0.33	-0.058	-0.076	-0.013	-0.049
digit span: reverse span	0.33	0.19	0.086	0.0076	-0.073
keep track: score	0.32	0.15	0.0048	-0.083	-0.066
adaptive n back: DDM-thresh	0.31	-0.07	0.18	0.14	-0.0055
digit span: forward span	0.29	0.15	0.063	0.016	-0.058
tower of london: num optimal solutions	0.29	0.13	-0.0041	0.02	-0.072
columbia card task hot: probability-sensitivity	-0.29	-0.059	-0.0015	0.12	-0.0096
columbia card task cold: loss-sensitivity	-0.28	-0.043	-0.028	0.021	0.089
two step decision: model free	-0.27	0.11	0.059	-0.056	0.029
shift task: learning rate	0.27	0.18	-0.013	-0.044	-0.096
shift task: acc	0.27	0.21	0.017	-0.084	-0.093
angling risk task always sunny: release adjusted clicks	0.26	0.014	-0.16	0.033	0.0031
tower of london: planning time	-0.25	0.13	0.19	0.18	-0.066
tower of london: num extra moves	0.23	-0.19	-0.16	-0.18	0.16
columbia card task cold: probability-sensitivity	-0.14	-0.14	0.1	-0.012	0.031
writing task: neutral probability	-0.13	0.032	0.083	-0.013	-0.07
holt laury survey: risk aversion	-0.11	0.026	-0.062	0.016	0.063
holt laury survey: safe choices	-0.1	0.076	-0.03	0.074	0.1
holt laury survey: beta: logTr	-0.098	0.078	-0.051	0.027	-0.02
writing task: positive probability	-0.089	0.045	0.084	0.052	-0.068
probabilistic selection: positive learning bias	0.074	0.0083	0.027	-0.064	-0.0024
choice reaction time: DDM-drift	-0.093	0.76	-0.054	0.0034	0.026
dot pattern expectancy: DDM-drift	0.18	0.7	-0.11	0.052	0.0087
simon: DDM-drift	0.014	0.66	0.086	-0.022	0.039
shape matching: DDM-drift	0.016	0.65	0.038	-0.0092	-0.0036
directed forgetting: DDM-drift	0.18	0.65	-0.073	-0.026	-0.0051
dot pattern expectancy: dprime	0.11	0.64	-0.076	0.046	-0.015
stop signal: DDM-drift	-0.045	0.64	0.23	-0.064	0.021
cue/task switch: DDM-drift	0.16	0.62	0.042	-0.064	-0.13
stim selective stop signal: DDM-drift	0.033	0.6	0.17	-0.037	-0.043
motor selective stop signal: DDM-drift	-0.06	0.59	0.26	-0.12	-0.01
recent probes: DDM-drift	0.19	0.59	-0.11	-0.03	-0.019
go nogo: dprime	-0.042	0.54	0.19	-0.026	0.039
stroop: DDM-drift	-0.047	0.51	0.2	-0.1	-0.036
cue/task switch: DDM-non-decision	0.12	0.46	-0.12	0.11	-0.03
shape matching: stimulus interference DDM-drift	0.12	-0.46	0.21	0.036	-0.057
attention network task: DDM-drift	0.098	0.43	0.32	0.0076	0.063
stim selective stop signal: reactive control DDM-drift	0.029	-0.42	0.17	-0.22	0.11
simon: simon DDM-drift	0.2	-0.4	0.29	0.045	0.021
dot pattern expectancy: bias	0.033	0.39	-0.17	0.13	-0.044
local global letter: DDM-drift	0.32	0.37	-0.0029	-0.1	-0.055
hierarchical rule: score	0.24	0.35	0.0071	-0.065	-0.11
attention network task: orienting DDM-drift	-0.12	-0.34	0.069	0.078	-0.034
two step decision: perseverance	0.18	0.33	0.079	0.095	-0.11
motor selective stop signal: proactive control DDM-drift	0.022	-0.33	-0.044	0.21	0.043
adaptive n back: DDM-non-decision	-0.029	0.32	0.013	0.12	-0.0026
adaptive n back: DDM-drift	0.032	0.32	-0.15	-0.047	0.068
information sampling task: Decreasing Win motivation	0.25	-0.3	-0.021	-0.28	0.084
dot pattern expectancy: DDM-thresh: logTr	0.22	0.26	0.19	0.099	0.038
dot pattern expectancy: DDM-non-decision	-0.21	0.25	0.01	0.16	0.035
dietary decision: taste-sensitivity	0.13	0.25	0.024	-0.023	-0.041
dot pattern expectancy: AY-BY DDM-drift	-0.085	-0.24	0.15	-0.065	0.0076
attention network task: alerting DDM-drift	-0.19	-0.23	0.086	0.025	-0.043
angling risk task always sunny: keep coef of variation	0.027	-0.22	0.0047	0.036	0.049
dot pattern expectancy: BX-BY DDM-drift	-0.0096	0.22	-0.065	0.057	-0.039
stop signal: proactive slowing DDM-thresh	-0.16	0.21	0.063	-0.028	-0.06
shift task: model decay: ReflogTr	-0.0093	-0.21	-0.039	0.046	-0.11
shift task: model learning rate	0.04	0.19	-0.042	-0.13	-0.0097
cue/task switch: cue switch-cost DDM-drift	-0.055	-0.18	0.012	-0.029	-0.019
directed forgetting: proactive interference DDM-drift: logTr	0.071	0.18	0.0047	-0.01	-0.028
recent probes: proactive interference DDM-drift	0.048	-0.18	0.065	0.047	0.0087
local global letter: switch-cost DDM-drift	-0.06	-0.18	0.063	-0.16	0.019
information sampling task: Decreasing Win P correct	0.057	0.16	0.093	0.085	0.014
stroop: stroop DDM-drift	0.067	-0.16	0.14	-0.016	-0.089
cue/task switch: task switch-cost DDM-drift	-0.089	-0.16	0.1	0.068	0.025
local global letter: global bias DDM-drift	0.016	0.15	0.037	-0.025	0.043
columbia card task cold: avg cards chosen	-0.11	-0.13	-0.072	-0.072	0.019
shift task: learning to learn	-0.03	0.13	0.0073	-0.016	0.014
angling risk task always sunny: release coef of variation: logTr	0.0055	-0.1	-0.1	0.015	0.059
shape matching: DDM-thresh: logTr	0.085	-0.058	0.59	0.0044	-0.023
simon: DDM-thresh	0.0045	-0.15	0.56	0.16	-0.037
attention network task: DDM-thresh	0.022	0.072	0.54	0.063	0.08
cue/task switch: DDM-thresh: logTr	0.047	-0.18	0.53	0.12	-0.059
motor selective stop signal: DDM-thresh: logTr	-0.11	0.18	0.51	-0.1	-0.0098
stim selective stop signal: DDM-thresh: logTr	-0.079	0.2	0.5	-0.18	0.02
stroop: DDM-thresh: logTr	-0.12	0.05	0.49	-0.055	-0.12
local global letter: DDM-thresh	0.25	0.029	0.48	0.097	0.025
choice reaction time: DDM-thresh	0.035	-0.0019	0.45	0.1	0.068
stop signal: DDM-thresh: logTr	-0.21	0.17	0.44	-0.13	0.036
recent probes: DDM-thresh	0.12	-0.15	0.4	0.034	-0.0048
directed forgetting: DDM-thresh	0.21	0.0076	0.4	0.041	-0.091
go nogo: bias	-0.067	-0.05	0.38	0.035	0.065
psychological refractory period two choices: PRP slope	-0.15	-0.13	0.3	0.28	-0.035
local global letter: conflict DDM-drift	0.063	0.052	0.27	0.13	0.0065
attention network task: conflict DDM-drift: ReflogTr	-0.043	0.15	-0.16	0.02	-0.029
stop signal: proactive slowing DDM-drift	-0.034	0.0076	-0.055	-0.014	-0.011
motor selective stop signal: DDM-non-decision: ReflogTr	0.01	-0.016	0.091	-0.57	-0.071
stop signal: SSRT low	-0.011	-0.23	-0.056	0.56	0.012
attention network task: DDM-non-decision: ReflogTr	0.15	-0.08	0.082	-0.55	0.035
simon: DDM-non-decision	-0.0007	0.058	0.1	0.54	0.098
stop signal: SSRT high: logTr	0.0071	-0.23	-0.06	0.54	-0.0017
motor selective stop signal: SSRT	0.048	-0.21	-0.12	0.51	-0.012
stim selective stop signal: SSRT	-0.032	-0.23	-0.18	0.5	-0.048
shape matching: DDM-non-decision: ReflogTr	0.11	-0.11	-0.19	-0.5	-0.046
choice reaction time: DDM-non-decision: logTr	0.02	0.013	0.093	0.49	0.041
stroop: DDM-non-decision	-0.015	0.068	0.13	0.49	0.13
stim selective stop signal: DDM-non-decision: ReflogTr	-0.044	-0.094	0.024	-0.49	-0.014
stop signal: DDM-non-decision: ReflogTr	-0.075	-0.1	-0.11	-0.46	-0.08
simple reaction time: avg rt: logTr	0.032	-0.42	0.16	0.45	0.058
recent probes: DDM-non-decision	-0.03	0.1	0.21	0.44	-0.066
directed forgetting: DDM-non-decision	-0.093	0.26	0.068	0.42	-0.047
local global letter: DDM-non-decision	0.061	0.19	0.19	0.38	-0.034
motor selective stop signal: reactive control DDM-drift	0.023	-0.11	0.046	-0.15	0.041
shift task: model beta: logTr	0.086	-0.042	0.062	0.12	-0.017
stop signal: proactive SSRT speeding	-0.0068	-0.049	-0.036	0.11	0.01
columbia card task hot: avg cards chosen	-0.04	-0.086	-0.075	-0.099	0.035
kirby: hyp discount rate large: logTr	0.042	-0.014	-0.031	0.0083	0.9
kirby: hyp discount rate small: logTr	0.044	-0.035	0.022	-0.016	0.89
bickel titrator: hyp discount rate medium: logTr	-0.041	0.1	-0.026	0.037	0.75
discount titrate: percent patient	0.01	0.03	-0.045	-0.049	-0.7
bickel titrator: hyp discount rate small: logTr	-0.082	0.06	0.041	-0.037	0.68
bickel titrator: hyp discount rate large: logTr	-0.066	0.1	-0.047	0.079	0.62
dietary decision: health-sensitivity	-0.06	-0.086	0.029	-0.01	-0.12

	Binge Drinking	Problem Drinking	Lifetime Smoking	Daily Smoking	Drug Use	Mental Health	Obesity	Income / Life Milestones
AlcoholHowOften6Drinks	0.87	0.064	0.028	0.039	0.011	0.022	0.024	-0.012
AlcoholHowOften	0.62	0.057	-0.016	0.051	0.038	-0.029	-0.084	0.23
AlcoholHowManyDrinksDay	0.6	0.13	0.14	0.004	-0.036	0.003	0.091	-0.12
DivorceCount	-0.17	0.042	0.1	-0.0016	-0.031	0.0098	-0.031	-0.09
HowOftenDrinkMorning	-0.16	0.85	0.0015	0.034	0.049	0.014	0.023	0.047
HowOftenFailedActivitiesDrinking	-0.0026	0.84	0.049	-0.038	-0.035	-0.025	-0.032	-0.011
HowOftenCantStopDrinking	0.14	0.79	0.0021	0.014	-0.028	0.0033	-0.0054	0.0053
HowOftenGuiltRemorseDrinking	0.19	0.7	-0.09	0.046	0.031	0.02	0.0011	-0.056
HowOftenUnableRememberDrinking	0.24	0.55	0.0015	-0.047	0.025	-0.0046	-0.028	-0.073
RelativeFriendConcernedDrinking	0.13	0.34	0.19	-0.019	0.29	0.086	0.047	0.062
InjuredDrinking	0.14	0.24	0.17	-0.031	0.24	0.085	0.08	0.11
LifetimeSmoke100Cigs	0.0075	-0.012	0.97	-0.026	0.017	0.0094	-0.015	0.019
HowLongSmoked	0.022	0.014	0.77	0.21	-0.016	-0.0048	0.0024	-0.047
RelationshipNumber	0.17	-0.056	0.26	-0.054	0.067	-0.069	-0.13	0.004
TrafficAccidentsLifeCount	0.035	0.037	0.15	-0.02	0.038	0.06	0.074	0.14
CigsPerDay	-0.022	-0.012	-0.04	0.94	0.016	0.015	0.0024	-0.032
SmokeEveryDay	0.01	-0.01	0.1	0.92	0.008	0.0026	-0.0019	-0.006
HowSoonSmokeAfterWaking	0.082	0.06	0.011	0.72	-0.021	-0.032	-0.028	0.046
CoffeeCupsPerDay	0.079	-0.11	0.028	0.2	0.16	0.043	-0.00011	0.017
TrafficTicketsLastYearCount	0.013	0.038	-0.055	0.14	0.12	0.093	0.056	0.084
NeglectedFamilyDrugUse	-0.08	0.02	0.013	0.017	0.78	-0.018	0.025	0.044
WidthdrawalSymptoms	-0.0027	-0.00096	0.066	-0.051	0.75	0.011	-0.036	-0.022
EngagedInIllegalActsToObtainDrugs	0.11	-0.092	-0.023	0.051	0.74	-0.01	-0.01	-0.04
SpouseParentsComplainDrugUse	0.036	0.014	-0.078	-0.022	0.73	0.0055	-0.047	0.043
BlackoutFlashbackDrugUse	-0.05	0.049	0.019	0.056	0.73	0.0072	-0.014	-0.011
FeelBadGuiltyDrugUse	-0.0008	0.014	0.031	-0.00047	0.7	0.025	0.015	-0.062
AbuseMoreThanOneDrugAtATime	0.051	0.024	-0.0032	-0.061	0.67	-0.035	0.018	-0.056
MedicalProblemsDueToDrugUse	-0.062	-0.043	-0.018	0.038	0.66	0.013	0.03	0.056
GamblingProblem	-0.038	0.15	-0.0081	0.086	0.28	0.055	0.037	-0.0014
AbleToStopDrugs	0.15	-0.2	0.068	0.00088	-0.21	-0.0098	0.048	0.023
ArrestedChargedLifeCount	-0.0087	0.18	0.17	0.042	0.18	0.029	0.032	-0.094
CannabisPast6Months	0.13	-0.069	0.12	0.061	0.16	-0.028	-0.018	-0.096
CaffieneOtherSourcesDayMG	-0.093	0.0078	0.074	-0.062	0.11	0.0069	-0.016	0.088
Depressed	-0.019	0.02	0.00037	-0.016	0.0092	0.92	0.0025	0.047
Hopeless	0.046	-0.073	-0.0059	0.00098	0.018	0.89	-0.0013	-0.048
Worthless	-0.0055	0.016	0.022	0.018	-0.012	0.88	-0.017	0.025
EverythingIsEffort	-0.046	0.056	0.036	-0.024	-0.046	0.82	0.016	-0.00069
Nervous	-0.011	-0.024	-0.065	0.05	0.02	0.69	-0.024	-0.018
RestlessFidgety	0.055	-0.002	-0.0077	-0.028	0.011	0.65	0.03	-0.062
Last30DaysUsual	0.022	-0.041	-0.12	0.086	-0.018	0.36	0.0089	-0.0057
RelationshipStatus	0.043	0.0097	0.072	-0.024	-0.049	-0.15	0.032	0.11
Obese	-0.026	0.035	-0.037	0.041	-0.0076	0.0036	0.94	0.021
BMI	0.059	-0.064	0.027	-0.064	0.00094	-0.0071	0.86	-0.0088
leisure time activity survey: activity level	0.17	-0.093	-0.058	0.0011	-0.032	-0.012	-0.26	0.19
CaffienatedSodaCansPerDay	-0.12	0.0026	0.024	0.093	0.045	0.027	0.14	-0.099
TeaCupsPerDay	0.031	-0.068	0.061	-0.064	0.051	-0.051	0.097	-0.083
HouseholdIncome	0.035	-0.023	-0.00083	-0.036	-0.026	-0.012	0.014	0.71
RetirementAccount	0.0018	0.021	0.00035	-0.021	-0.0043	-0.028	0.026	0.56
HighestEducation	0.14	-0.014	-0.033	-0.034	0.021	0.024	-0.13	0.31
RentOwn	-0.11	0.028	-0.054	0.068	0.0029	-0.029	-0.019	0.29
ChildrenNumber	-0.11	-0.013	0.06	0.18	-0.05	-0.073	0.08	0.25
LongestRelationship	-0.039	-0.0084	0.051	0.067	-0.051	-0.068	0.065	0.24

	Agreeableness	Eating Control	Emotional Control	Ethical Risk-Taking	Financial Risk-Taking	Goal-Directedness	Impulsivity	Mindfulness	Reward Sensitivity	Risk Perception	Sensation Seeking	Social Risk-Taking
bis11 survey	0.993	0.999	1	0.999	0.999	0.998	0.995	0.998	0.998	1	1	0.999
bis bas survey	0.86	0.986	0.964	0.993	0.998	0.982	0.993	0.995	0.927	0.993	0.998	0.986
brief self control survey	0.999	1	1	1	1	1	1	1	1	1	1	1
dickman survey	0.995	1	0.999	0.999	1	0.999	1	1	0.999	1	1	1
dospert eb survey	0.894	0.98	0.954	0.506	0.401	0.836	0.983	0.993	0.869	0.945	0.991	0.723
dospert rp survey	0.943	0.982	0.948	0.925	0.46	0.841	0.984	0.995	0.839	0.512	0.989	0.968
dospert rt survey	0.918	0.985	0.948	0.912	0.506	0.868	0.976	0.995	0.868	0.984	0.99	0.808
eating survey	0.958	0.538	0.993	0.997	0.992	0.851	0.997	0.995	0.799	0.997	0.998	0.982
erq survey	0.991	0.999	0.998	0.999	0.999	0.999	0.997	0.999	0.993	0.999	0.999	0.999
five facet mindfulness survey	0.867	0.992	0.825	0.996	0.996	0.847	0.99	0.715	0.995	0.996	0.996	0.99
future time perspective survey	0.994	1	0.999	0.999	1	0.999	1	0.998	1	1	1	1
grit scale survey	0.998	1	0.999	1	0.999	0.991	0.992	0.998	0.995	1	0.999	1
impulsive venture survey	0.994	1	1	0.998	0.999	0.999	0.999	0.999	0.994	1	0.996	1
mindful attention awareness survey	0.965	0.995	0.964	0.993	0.996	0.797	0.979	0.783	0.978	0.996	0.997	0.992
mpq control survey	1	1	1	1	1	1	1	1	1	1	1	0.999
selection optimization compensation survey	0.956	0.995	0.994	0.999	0.998	0.927	0.975	0.999	0.891	0.999	0.999	0.998
self regulation survey	1	1	1	1	1	1	1	1	1	1	1	1
sensation seeking survey	0.965	0.999	0.996	0.997	0.999	0.998	0.998	0.999	0.998	0.999	0.997	0.99
ten item personality survey	0.763	0.994	0.899	0.997	0.998	0.982	0.996	0.997	0.99	0.998	0.998	0.996
theories of willpower survey	1	1	1	1	1	0.999	1	1	1	1	1	1
time perspective survey	0.935	0.992	0.994	0.993	0.996	0.959	0.994	0.995	0.985	0.997	0.997	0.994
upps impulsivity survey	0.984	0.989	0.996	0.999	0.997	0.853	0.898	0.995	0.969	0.998	0.998	0.996

	Caution	Discounting	Perc / Resp	Speeded IP	Strategic IP
adaptive n back	0.998	1	1	0.998	0.995
angling risk task always sunny	1	1	1	1	0.999
attention network task	0.996	1	0.999	1	0.999
bickel titrator	1	0.997	1	0.999	1
choice reaction time	0.999	1	0.999	0.999	0.998
cognitive reflection survey	1	0.999	0.999	0.999	0.997
columbia card task cold	1	1	1	0.999	0.997
columbia card task hot	1	1	1	0.999	0.996
dietary decision	1	1	1	1	1
digit span	1	1	1	1	0.999
directed forgetting	0.999	1	0.999	0.999	0.999
discount titrate	1	0.999	1	1	1
dot pattern expectancy	0.995	1	0.998	0.997	0.996
go nogo	0.999	1	1	1	1
hierarchical rule	1	1	1	1	1
holt laury survey	1	1	1	1	1
information sampling task	0.999	0.999	0.998	0.998	0.998
keep track	1	1	1	1	1
kirby	0.995	0.557	0.945	0.975	0.868
local global letter	0.999	1	1	1	0.999
motor selective stop signal	0.995	1	0.996	0.999	0.999
probabilistic selection	1	1	1	1	1
psychological refractory period two choices	1	1	1	1	1
ravens	1	1	0.999	0.999	0.998
recent probes	0.999	1	0.999	1	1
shape matching	0.997	1	0.999	0.998	0.999
shift task	1	1	1	1	1
simon	0.997	1	0.999	0.998	0.999
simple reaction time	0.999	1	0.999	0.999	0.999
spatial span	1	1	0.999	0.999	0.996
stim selective stop signal	0.997	1	0.998	0.999	1
stop signal	0.957	0.998	0.954	0.987	0.979
stroop	0.999	1	0.998	1	0.999
threebytwo	0.996	1	0.999	0.998	1
tower of london	0.999	0.998	0.998	0.995	0.994
two stage decision	1	1	1	1	1
writing task	1	1	1	1	1

Table of contents¶

Exploratory Factor Analysis Results ¶

Survey Exploratory Factor Analysis Loadings¶

Task Exploratory Factor Analysis Loadings¶

Outcome Exploratory Factor Analysis Loadings¶

Factor Robustness Analyses¶

Survey Robustness¶

Measurement Drop¶

Bootstrap¶

Task Robustness¶

Measurement Drop¶

Bootstrap¶

Hierarchical Clustering¶

Survey Clusters¶

Task Clusters¶

Cluster Robustness Analyses¶

Survey Cluster Robustness ¶

AMI values¶

Closest values according to consensus clustering¶

Consensus clustering distances of variables within the same cluster according to original cluster¶

Task Cluster Robustness ¶

Adjusted mutual information¶

Closest values according to consensus clustering¶

Consensus clustering distances of variables within the same cluster according to original cluster¶

Survey Exploratory Factor Analysis Loadings ¶

Task Exploratory Factor Analysis Loadings ¶

Outcome Exploratory Factor Analysis Loadings ¶

Factor Robustness Analyses ¶

Survey Robustness ¶

Task Robustness ¶

Hierarchical Clustering ¶

Survey Clusters ¶

Task Clusters ¶

Cluster Robustness Analyses ¶

AMI values ¶

Closest values according to consensus clustering ¶

Consensus clustering distances of variables within the same cluster according to original cluster ¶

Adjusted mutual information ¶

Closest values according to consensus clustering ¶

Consensus clustering distances of variables within the same cluster according to original cluster ¶