func_processing.resources.afni package
Submodules
func_processing.resources.afni.copy module
Copy data from fMRIprep to AFNI.
Copy certain derivative output from fMRIprep to AFNI for use in finishing pre-processing, deconvolution, and group-level analyses.
- func_processing.resources.afni.copy.copy_data(prep_dir, work_dir, subj, task, tplflow_str)
Get relevant fMRIprep files, rename.
Copies select fMRIprep files into AFNI derivatives, prepares for AFNI pre-processing steps.
- Parameters
prep_dir (str) – /path/to/derivatives/fmriprep
work_dir (str) – /path/to/derivatives/afni/sub-1234/ses-A
subj (str) – BIDS subject string (sub-1234)
task (str) – BIDS task string (task-test)
tplflow_str (str) – template ID string, for finding fMRIprep output in template space (space-MNIPediatricAsym_cohort-5_res-2)
- Returns
afni_data – files copied from derivatives/fMRIprep to derivatives/afni
added afni_data keys:
[struct-t1] = T1w structural
[mask-brain] = brain mask
[mask-probGM] = probability gray matter mask
[mask-probWM] = probability white matter mask
[mask-probCSF] = probability csf mask
[epi-preproc?] = fMRIprep preprocessed EPI for run-?
[mot-confound?] = confounds (motion) file for EPI data for run-?
- Return type
dict
func_processing.resources.afni.deconvolve module
Write and run deconvolution commands.
Use AFNI’s 3dDeconvolve and 3dREMLfit to deconvolve pre-processed EPI data.
- func_processing.resources.afni.deconvolve.regress_resting(afni_data, work_dir, proj_meth='anaticor')
Construct regression matrix for resting state data.
Conduct a principal components analysis to identify CSF timeseries. Use CSF timeseries as nuissance regressor, along with motion etc to clean signal via deconvolution. Residuals (cleaned signal) are then used to project regression matrices. Anaticor uses WM as a nuissance regressor.
- Parameters
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-scale1] = single/first scaled file
[mask-min] = mask of voxels with >min signal
[mask-erodedCSF] = eroded CSF mask
[mot-mean] = mean motion timeseries
[mot-deriv] = derivative motion timeseries
[mot-censor] = binary censor vector
work_dir (str) – /path/to/project_dir/derivatives/afni/sub-1234/ses-A
proj_meth (str) – [anaticor | original] method of matrix progression.
- Returns
decon_plan – Matches behaviors to timing file
new key:
[reg-matrix] = regression matrix
- Return type
dict
Notes
Only supports RS conducted in single run
- func_processing.resources.afni.deconvolve.run_reml(work_dir, afni_data)
Deconvolve EPI timeseries.
Generate an idea of nuissance signal from the white matter and include this in the generated 3dREMLfit command.
- Parameters
work_dir (str) – /path/to/project_dir/derivatives/afni/sub-1234/ses-A
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-scale<1..N>] = list of scaled files
[mask-erodedWM] = eroded WM mask
- Returns
afni_data – updated for nuissance, deconvolved files
added afni_data keys:
[epi-nuiss] = nuissance signal file
[rml-<decon_name>] = deconvolved file (<decon_name>_stats_REML+tlrc)
- Return type
dict
- func_processing.resources.afni.deconvolve.timing_files(dset_dir, deriv_dir, subj, sess, task, decon_name='UniqueBehs')
Generate AFNI timing files.
Written specifically for the EMU study. Use dset/func/events.tsv files to generate AFNI-style timing files for each unique behavior (trial_type).
- Parameters
dset_dir (str) – /path/to/BIDS/dset
deriv_dir (str) – /path/to/BIDS/derivatives/afni
subj (str) – BIDS subject string (sub-1234)
sess (str) – BIDS session string (ses-A)
task (str) – BIDS task string (task-test)
decon_name (str) – name of deconvolution given all unique behaviors [default=UniqueBehs]
- Returns
decon_plan – Matches behaviors to timing file
keys description:
[beh-A] = /path/to/foo_desc-behA_events.1D
[beh-B] = /path/to/foo_desc-behB_events.1D
- Return type
dict
Notes
Currently only writes onset time, not married duration.
Behavior key (beh-A, beh-B above) become label of deconvolved sub-brick.
- func_processing.resources.afni.deconvolve.write_decon(decon_name, tf_dict, afni_data, work_dir, dur)
Generate deconvolution script.
Write a deconvolution script using the pre-processed data, motion, and censored files passed by afni_data. Uses a 2GAM basis function (AFNI’s TWOGAMpw). This script is used to generate X.files and the foo_stats.REML_cmd.
Timing files should contain AFNI-formatted onset times (duration is hardcoded), using the asterisk for runs in which a certain behavior does not occur.
- Parameters
decon_name (str) – name of deconvolution, useful when conducting multiple deconvolutions on same session. Will be appended to BIDS task name (decon_<task-name>_<decon_name>).
tf_dict (dict) –
timing files dictionary, behavior string is key
e.g. {“lureFA”: “/path/to/tf_task-test_lureFA.txt”}
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-scale<1..N>] = list of scaled files
[mot-mean] = mean motion timeseries
[mot-deriv] = derivative motion timeseries
[mot-censor] = binary censory vector
work_dir (str) – /path/to/project_dir/derivatives/afni/sub-1234/ses-A
dur (int/float/str) – duration of event to model
- Returns
afni_data – updated with REML commands
added afni_data keys:
[dcn-<decon_name>] = name of decon reml command
- Return type
dict
Notes
Deconvolution files will be written in AFNI format, rather than BIDS. This includes the X.files (cue spooky theme), script, and deconvolved output.
Files names will have the format:
decon_<bids-task>_<decon_name>
- func_processing.resources.afni.deconvolve.write_new_decon(decon_name, tf_dict, afni_data, work_dir, dur)
Write a deconvolution script using new censor approach.
Rather than using desc-censor_events.tsv to remove volumes at the deconvolution, as is the default in AFNI’s workflow, instead use the censor file to remove behaviors that co-occur during the same volume. Then, add a baseline censor regressor after polynomials but before the behavior regressors.
Additionally, convolution of the timing file with the HRF basis function is supplied as the regressor via “-stim_file” rather using the “-stim_times” option. This is because I don’t remember how to go from a binary vector in volume time to AFNI-styled timing files =).
- Parameters
decon_name (str) – name of deconvolution, useful when conducting multiple deconvolutions on same session. Will be appended to BIDS task name (decon_<task-name>_<decon_name>).
tf_dict (dict) –
timing files dictionary, behavior string is key
e.g. {“lureFA”: “/path/to/tf_task-test_lureFA.txt”}
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-scale<1..N>] = list of scaled files
[mot-mean] = mean motion timeseries
[mot-deriv] = derivative motion timeseries
[mot-censor] = binary censory vector (0 = censor)
[mot-censorInv] = inverted binary censory vector (1 = censor)
work_dir (str) – /path/to/project_dir/derivatives/afni/sub-1234/ses-A
dur (int/float/str) – duration of event to model
- Returns
afni_data – updated with REML commands
added afni_data keys:
[dcn-<decon_name>] = name of decon reml command
- Return type
dict
Notes
Deconvolution files will be written in AFNI format, rather than BIDS. This includes the X.files (cue spooky theme), script, and deconvolved output.
Files names will have the format:
decon_<bids-task>_<decon_name>
Also, writes info_behavior_censored.json to subject directory.
func_processing.resources.afni.group module
Functions for group-level analyses.
Makes group masks, conducts group statistics.
- func_processing.resources.afni.group.int_mask(task, deriv_dir, group_data, group_dir)
Create group gray matter intersection mask.
For EPI data of a specific task.
- Parameters
task (str) – BIDS task string (task-rest)
deriv_dir (str) – location of project AFNI derivatives
group_data (dict) –
contains keys pointing to required files
required keys:
[mask-gm] = gray matter mask
[subj-list] = list of subjects
group_dir (str) – output location of work
- Returns
group_data – updated with the intersection field
added group_data key:
[mask-int] = GM intersect mask
- Return type
dict
- func_processing.resources.afni.group.resting_etac(seed, group_data, group_dir, do_blur)
Conduct A vs not-A via ETAC.
- Parameters
seed (str) – seed name (rPCC)
group_data (dict) –
contains keys pointing to required files
required keys:
[all-ztrans] = list of Ztrans files
[mask-int] = group GM intersection mask
group_dir (str) – location of output directory
do_blur (bool) – [T/F] whether blur was done in pre-processing
- Returns
group_data – updated with the generated files
added group_data keys:
[S<seed>-etac] = final ETAC output file
- Return type
dict
- func_processing.resources.afni.group.task_etac(beh_list, deriv_dir, sess, group_data, group_dir, do_blur)
Conduct A vs B via ETAC.
Write and then run an ETAC command (3dttest++) to compare behavior A to behavior B.
- Parameters
beh_list (list) – two behaviors for setA, setB
deriv_dir (str) – path to project afni derivatives
sess (str) – BIDS session (ses-S2)
group_data (dict) –
contains keys pointing to required files
required keys:
[subj-list] = list of subjects
[mask-int] = group GM intersection mask
[dcn-file] = decon file string
do_blur (bool) – [T/F] whether blur was done in pre-processing
- Returns
group_data – updated with generated files
added group_data keys:
[behAB-etac] = final output ETAC file
- Return type
dict
func_processing.resources.afni.masks module
Functions for making various masks.
- func_processing.resources.afni.masks.make_intersect_mask(work_dir, subj_num, afni_data, sess, task, do_blur, c_frac='0.5', nbr_type='NN2', n_nbr=17)
Make EPI-struct intersection mask.
- Parameters
work_dir (str) – /path/to/derivatives/afni/sub-1234/ses-A
subj_num (int/str) –
subject identifier, for sbatch job name
- afni_datadict
contains keys pointing to required files
required keys:
[mask-brain] = anatomic brain mask
conditionally required keys:
do_blur = T when [epi-blur<1..N>] = list of blurred EPI files
do_blur = F when [epi-blur<1..N>] = list of fmriprep preprocessed files
sess (str) – BIDS session string (ses-S1)
task (str) – BIDS task string (task-study)
do_blur (bool) – [T/F] whether to blur as part of pre-processing
c_frac (str, float) – input for 3dAutomask -clfrac option
nbr_type (str) – 3dAutomask nearest neighbors argument
n_nbr (int) – input for 3dAutomask -nbhrs option
- Returns
afni_data – updated with files
added afni_data keys:
[mask-int] = subject epi-anat intersection mask
- Return type
dict
- func_processing.resources.afni.masks.make_minimum_masks(work_dir, subj_num, task, afni_data)
Make a mask of where minimum signal exists in EPI space.
Used to help with the scaling step, so low values do not bias the scale. Based off “3dTstat -min”.
- Parameters
work_dir (str) – /path/to/derivatives/afni/sub-1234/ses-A
subj_num (int/str) – subject identifier, for sbatch job name
task (str) – BIDS task string (task-test)
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-preproc<1..N>] = fmriprep pre-processed files
[mask-brain] = anatomic brain mask
- Returns
afni_dict – adds keys for generated files
added afni_data keys:
[mask-min] = mask of minimum value for task
- Return type
dict
- func_processing.resources.afni.masks.make_tissue_masks(work_dir, subj_num, afni_data, thresh=0.5)
Make tissue class masks.
- Parameters
work_dir (str) – /path/to/derivatives/afni/sub-1234/ses-A
subj_num (int/str) – subject identifier, for sbatch job name
afni_data (dict) –
contains keys pointing to required files
required keys:
[mask-prob<GM|CSF|WM>] = tissue probability masks
[mask-brain] = anatomic brain mask
thresh (float [default=0.5]) – value for thresholding probseg files
- Returns
afni_data – adds keys for generated files
added afni_data keys:
[mask-eroded<GM|WM|CSF>] = eroded gray, white matter, CSF masks
- Return type
dict
func_processing.resources.afni.motion module
Make motion, censor files for deconvolution.
Use fMRIprep desc-confounds_timeseries.tsv to make mean motion, derivative motion, and censor files. Censor includes volume preceding motion event.
- func_processing.resources.afni.motion.mot_files(work_dir, afni_data, task)
Constuct motion and censor files.
Mine <fMRIprep>_desc-confounds_timeseries.tsv for motion events, make motion files for mean (6df) and derivative (6df) motion events. Also, create motion censor file. Volume preceding a motion event is also censored. Finally, report the number of censored volumes.
I’m not sure if motion is demeaned or not, given that it is output by fMRIprep (mined from confounds.tsv file).
- Parameters
work_dir (str) – /path/to/project_dir/derivatives/afni/sub-1234/ses-A
afni_data (dict) –
contains keys pointing to required files
required keys:
[mot-confound<1..N>] = fmriprep motion confound timeseries
task (str) – BIDS task string (task-test)
- Returns
afni_data – updated with names of motion files
added afni_data keys:
[mot-mean] = motion mean file
[mot-deriv] = motion derivative file
[mot-censor] = binary censory vector
- Return type
dict
Notes
As runs do not have an equal number of volumes, motion/censor files for each run are concatenated into a single file rather than managing zero padding.
Writes 1D files - AFNI reads tsv as containing a header!
func_processing.resources.afni.process module
Functions for processing EPI data.
Finish pre-processing on fMRIprep output using an AFNI workflow.
- func_processing.resources.afni.process.blur_epi(work_dir, subj_num, afni_data, blur_mult=1.5)
Blur EPI data.
Blur pre-processed EPI runs with AFNI’s 3dmerge.
- Parameters
work_dir (str) – /path/to/derivatives/afni/sub-1234/ses-A
subj_num (int/str) – subject identifier, for sbatch job name
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-preproc<1..N>] = fmriprep pre-processed files
blur-mult (int) –
blur kernel multiplier (default = 1.5)
e.g. vox=2, blur_mult=1.5, blur size is 3 (will round float up to nearest int)
- Returns
afni_data – updated with names of blurred data
added afni_data keys:
[epi-blur?] = blurred/smoothed EPI data of run-?
- Return type
dict
Notes
Determining blur multiplier based off voxel’s dimension K.
- func_processing.resources.afni.process.reface(subj, sess, t1_file, proj_dir, method)
De/Reface T1-weighted files.
Use AFNI’s refacer to deface or reface T1-weighted structural files.
- Parameters
subj (str) – BIDS subject string (sub-1234)
sess (str) – BIDS session string (ses-A)
t1_file (str) – file name of T1-weighted file (sub-1234_ses-A_T1w.nii.gz)
proj_dir (str) – path to BIDS project dir (/path/to/BIDS/proj)
method (str) – “deface”, “reface”, or “reface_plus” method
- Returns
return_str – Success message
- Return type
str
- func_processing.resources.afni.process.resting_metrics(afni_data, work_dir)
Generate info about resting data.
Produce TSNR, GCOR, and noise estimations.
- Parameters
afni_data (dict) –
contains keys pointing to required files
required keys:
[epi-scale1] = first/only scaled RS epi file
[reg-matrix] = project regression matrix
[mot-censor] = binary censor vector
[mask-int] = epi-anat intersection mask
work_dir (str) – /path/to/project_dir/derivatives/afni/sub-1234/ses-A
- Returns
afni_data – not currently adding any keys/values
- Return type
dict
- func_processing.resources.afni.process.resting_seed(coord_dict, afni_data, work_dir)
Produce correlation matrices for seeds.
For each seed in coord_dict, produce a projected correlation matrix.
- Parameters
coord_dict (dict) –
seed name, coordinates
{“rPCC”: “5 -55 25”}
afni_data (dict) –
contains keys pointing to required files
required keys:
[reg-matrix] = project regression matrix
[mask-int] = epi-anat intersection mask
[mot-censor] = binary censory vector
work_dir (str) – location of subject’s scratch directory
- Returns
afni_data – updated with z-transformed data
added afni_data keys:
[S<seed>-ztrans] = Z-transformed correlation matrix of seed
- Return type
dict
- func_processing.resources.afni.process.scale_epi(work_dir, subj_num, afni_data, do_blur)
Scale EPI runs.
Scale timeseries to center = 100 using AFNI’s 3dcalc.
- Parameters
work_dir (str) – /path/to/derivatives/afni/sub-1234/ses-A
subj_num (int/str) – subject identifier, for sbatch job name
afni_data (dict) –
contains keys pointing to required files
required keys:
[mask-min] = mask of voxels with >minimum signal conditionally required keys
[do_blur] = T when [epi-blur<1..N>] = list of blurred EPI files
[do_blur] = F when [epi-preproc<1..N>] = list of fmriprep preprocessed files
do_blur (bool) – [T/F] whether to blur as part of pre-processing
- Returns
afni_dict – updated with scaled files
added afni_data keys:
[epi-scale?] = scaled EPI for run-?
- Return type
dict
func_processing.resources.afni.submit module
Functions to submit Bash commands as subprocesses.
Submit bash commands as a subprocess, either on the same node (submit_hpc_subprocess) or on a scheduled node (submit_hpc_sbatch). Used for wrapping AFNI and c3d commands.
- func_processing.resources.afni.submit.submit_hpc_sbatch(command, wall_hours, mem_gig, num_proc, job_name, out_dir, env_input=None)
Submit job to slurm scheduler (sbatch).
Sbatch submit a larger job with scheduled resources. Waits for job_name to no longer be found in squeue. Stderr/out written to out_dir/sbatch_<job_name>.err/out. Supports AFNI and c3d commands.
- Parameters
command (str) – Bash code to be scheduled
wall_hours (int) – number of desired walltime hours
mem_gig (int) – amount of desired RAM
num_proc (int) – number of desired processors
job_name (str) – job name
out_dir (str) – location for <job_name>.err/out
env_input (dict) – user-specified environment for certain software (e.g. fMRIprep)
- Returns
(job_name, job_id) – job_name = scheduled job name
job_id = scheduled job ID
- Return type
tuple of str
Example
submit_hpc_sbatch(“afni -ver”)
- func_processing.resources.afni.submit.submit_hpc_subprocess(bash_command)
Submit quick job as subprocess.
Run a quick job as a subprocess and capture stdout/err. Use for AFNI and c3d commands.
- Parameters
bash_command (str) – Bash syntax, to be executed
- Returns
(job_out, job_err) – job_out = subprocess stdout
job_err = subprocess stderr
- Return type
tuple of str
Example
submit_hpc_subprocess(“afni -ver”)