occasionally Time from SingleTarget is empty, and a timing question

Hi Jaewon,

I have that task in which I am detecting a subject moving the joystick while I am ensuring fixation and displaying stimuli.

I am interested in reaction times of the joystick movement. I was getting them from SingleTarget.Time and subtracting recorded time of stimulus display (scene start). Sometimes (rarely), the RT collected that way comes out empty ([]) and I am not sure why.

Here is the relevant scene

stim_fix = SingleTarget(eye_);
stim_fix.Target = [0, 0];
stim_fix.Threshold = center_fix_window;

stim_fix_con = Concurrent(stim_fix);
stim_fix_con.add(fix_image);

stim_hold_fix = WaitThenHold(stim_fix_con);
stim_hold_fix.WaitTime = 0;
stim_hold_fix.HoldTime = stim_on_time;


joy_stim = SingleTarget(joy_);
joy_stim.Target = [0 0];
joy_stim.Threshold = joy_threshold_this_trial;  %succcess when within threshold

joy_stim_moved = NotAdapter(joy_stim); %success when outside threshold

joy_wth_stim = WaitThenHold(joy_stim_moved); %wait until time elapsed or joystick moves
joy_wth_stim.WaitTime = stim_on_time;
joy_wth_stim.HoldTime = 0;
%stops when joystick moved or wait time elapsed
%success if joystick moved


keep_going_stim = AllContinue(stim_hold_fix); %keep going unless fixat
keep_going_stim.add(joy_wth_stim);


also_images_stim = Concurrent(keep_going_stim);
also_images_stim.add(stimulus_emo_img);
also_images_stim.add(Gabor_img);

stim_scene = create_scene(also_images_stim);

and how it is run

    stim_start = run_scene(stim_scene, stim_start_mk);

    if ~stim_hold_fix.Success ... %failed fixation during stimulus presentation
            && ~joy_wth_stim.Success  %this needed because if scene ended due to joystick move, 
                                     % there would be no fixation success
                                     %but it does not mean there was a fixation fail!
                       

        eventmarker(fix_failed_stim_mk);
        trialerror(1);  %repeat trial
        trial_result = result_fix_failed_stim; %failed fixation during stimuli
        do_fix_timeout = true;

        dashboard(2, 'fixation during stimuli failed', [1 0 1]);
        idle(0);
        % nested_report


    else %fixation fine during stimuli

        if joy_wth_stim.Success %if joystick moved during stimuli, finish trial

            eventmarker(joystick_go_stim_mk);
            joy_go = true;  %joystick moved
            trial_complete = true;
            RT = joy_stim.Time - stim_start; %RT re stimulus onset
            %fprintf('RT %g\n', RT);
            dashboard(2, 'joystick moved during stimuli', [1 1 0]);
            idle(0);
            % nested_report

In a few trials per session RT is []

Should I get the RT from WaitThenHold.RT instead, will it work then?


And the timing question:

I also tried reconstructing RT from behavioral codes, from the difference between:
joystick_go_stim_mk (set by eventmarker(joystick_go_stim_mk); )
and
stim_start_mk (set by stim_start = run_scene(stim_scene, stim_start_mk); )

This works also in trials in which RT was empty, but in trials in which RT was recorded correctly, the number from behavioral codes is between 33 and 50 ms higher. Is this because of the time it takes from completing the scene and getting to eventmarker(joystick_go_stim_mk);? Is there a way to compensate for the delay?

Thanks!

Pawel

In the scene, does the subject need to maintain eye fixation for a certain duration before moving the joystick? Also, is the eye fixation required even after the joystick is off the threshold window?

Jaewon Wrote:
-------------------------------------------------------
> In the scene, does the subject need to maintain
> eye fixation for a certain duration before moving
> the joystick?

Not in this scene (which shows fixation point and stimuli) but in the preceding scene which shows fixation point only




> Also, is the eye fixation required?
> even after the joystick is off the threshold
> window?

Eye fixation is required until this scene (showing stimuli and fix point) and the next scene (showing fix point only) complete, unless joystick is moved. If joystick is moved, no further fixation required.

I do not know why your RT variable is sometimes empty. The code you posted is not complete, so I cannot run it. Is it possible that RT was initialized to [] somewhere and a new value was never assigned?

You are indeed supposed to get the RT from WaitThenHold.RT, but either way should produce the same result. Have you checked the values of joy_stim.Time and stim_start when RT was []?

In the scene framework, the timestamp of eventmarker() is almost useless, since it can be called only outside run_scene(). You need to calculate RT from the recorded data, if the time necessary for the calculation is not reported by adapters.
https://monkeylogic.nimh.nih.gov/docs_CreatingTask.html#RuntimeVersion2_Background

Jaewon Wrote:
-------------------------------------------------------
> I do not know why your RT variable is sometimes
> empty. The code you posted is not complete, so I
> cannot run it. Is it possible that RT was
> initialized to [] somewhere and a new value was
> never assigned?


From what I see, RT is initialized to NaN and then only assigned a value from SingleTarget.Time minus stored start time of the stimulus scene.

I just realized that the reason I did not use WaitThenHold.RT is that it would be referred to the start of the current scene, and in some cases I need to refer to the start of the previous scene. I guess I could use WaitThenHold.AcquiredTime instead, and still subtract stimulus start time.

But I think I may have figured it out now.

First, empty RT is rare, between 0 and 2 trials in a 200-300-trial session.

But importantly, whenever RT is empty, the reaction time reconstructed from the eventmarker is around 235-240 ms. The stimulus presentation scene is 200 ms, and I found before that the uncertainty/delay of eventmarker is usually around 35-50 ms.

So I think RT is empty if the joystick crosses the threshold after the stimulus scene ended (so that scene fails to capture it) but before the post-stim scene started. So when the post-stim scene starts, it already sees the joystick past the threshold and exits immediately returning empty RT. Does it sound plausible to you?




>
> You are indeed supposed to get the RT from
> WaitThenHold.RT, but either way should produce the
> same result. Have you checked the values of
> joy_stim.Time and stim_start when RT was []?


Not yet, I found the problem running a data analysis script that I just wrote, and joy_stim.Time and stim_start are not explicitly stored in the data file (though the latter time is in a way stored as an event marker: stim_start = run_scene(stim_scene, stim_start_mk);
. I can add it for troubleshooting and testing of my theory that I mentioned above.

>
> In the scene framework, the timestamp of
> eventmarker() is almost useless, since it can be
> called only outside run_scene(). You need to
> calculate RT from the recorded data, if the time
> necessary for the calculation is not reported by
> adapters.
> https://monkeylogic.nimh.nih.gov/docs_CreatingTask.html#RuntimeVersion2_Background


"from the recorded data" - you mean check when AnalogData.Joystick exceeds my threshold? I will do that, I just realized i need to start saving the joystick threshold on a per-trial basis because in some cases I allow for trial-to-trial variability of the threshold. Only for training to extinguish unwanted strategies developed by some subjects, it won't be used in the final task, but I should still save it.

I think that is it. If the joystick is already off the window before the scene start, then SingleTarget does not know when the window was crossed and its Time property remains empty. You can use WaitThenHold.AllowEarlyFix to detect this case (or test if the RT is empty).

I think I will just reconstruct RT from the joystick signal. I just tested it and the difference between reconstructed RT and one reported by the adapters is less than 1 ms in most cases, ~1.5 ms in the worst cases. I can live with that.

But i will also look into AllowEarlyFix to ensure that the behavior is handled correctly in these cases.

Thanks!

Pawel