body and object selective cells showed significantly more response enhancement for their preferred category compared to the non-preferred category. No such response enhancement was observed in trials when the monkeys made a wrong choice in the categorization task. Magnitude of the response enhancement was larger for more noisy stimuli. More importantly, in trials with high baseline activity responses of body selective and object selective cells to body images were enhanced and suppressed, respectively. We also found decreased neural response variability in the categorization compared to the passive task. Larger effects were observed at higher noise levels. By measuring choice probability we found that neural firing rate was correlated with monkeys’ choice, particularly in trials with high baseline activity. We suggest that attentional enhancement of IT cells’ baseline firing rate is correlated with improved neural response reliability and category selectivity. These effects are dependent on the cells’ category selectivity, attentional load and the exact time of baseline activity increase.

Keywords: object recognition, neural baseline activity, visual attention, decision making

Table of Contents

Introduction…………………………………………………………………..….10

The crucial role of “visual object categorization: in everyday life…………10
Where in the brain is category information represented? ……………………11
Anatomy of inferior temporal cortex…………………………….…………23
Attention improves categorization performance, especially in difficult condition…………………………………………………………………….25
Bottom-up vs. top-down attention………………………………….………26
What is attention directed to?………………………………………….……28Space-based attention ……………………………………………….28
Feature-based attention………………………………………………28
Object-based attention…………………………………..…..….……29
Sources and targets of attention in the brain…………………………….…31
Attention modulates different response properties…………………..……..31Firing rate modulation………………….……………………………32Response enhancement.……………………….…………..…….32
Response suppression…………………………………………33
Baseline enhancement………………………………..………34
Reliability increase……………………………………….………….36
Response sensitivity increase……………………………….……….37
Response selectivity modulation…………………………….………38
Synchronization, oscillation and correlated responses across cell population…………………………………………………………….39
Objectives……………………………………………………………………..….42

Method……………………………………………………………………………43

Subjects………………………………………………………….…………43
Stereotactic MRI……………………………………………….…………..43
Head-post implantation surgery.……………………….…………………..44
Stimuli…………………………………………………….………………..46
Tasks…………………………………………………………………………47Passive task…………………………………………………………..47
Active task (two-alternative forced-choice body/object categorization)……………………………………………………….47
Training…………………………………………………….………………50
Eye   monitoring………………………………………………….………….52
Craniotomy surgery …………………………………………….………….52
Recording………………………………………………………….……….53Recoded area…………………………………………………….…..54
Recording room…………………………………………….………..54
Data acquisition setup…………………………………………….…54
Noise reduction………………………………………………….…..55
Electrode insertion ………………………………………………….56
Signal amplification and frequency filtering ……………………….57
Data analysis……………………………………………………..…………59Category selectivity index…………………………………….……..60
High and low baseline trials………………………………….………60
RMI (rate modulation index)……………………………………..…61
FF (fano factor)………………………………………………….…..61
FFMI (fano factor modulation index)…………………………….…61
CP (choice probability)………………………………………………62
RMI onset……………………………………………………………65
Neural/behavioral score……………………………………………..66
Peristimulus time histograms (PSTH), normalizing and smoothing ……………………………………………………………………….67
 

Results…………………………………………………………………………….68

Conclusion………………………………………………………………………..88

Figures……………………………………………………………………………90

Stimulus set …………………………………….……………..….90
Figure 2. Different noise levels of an exemplar stimulus ……………….…91
Figure 3. Passive task……………………………………………………….92
Figure 4. Active task (two-alternative forced-choice body/object categorization…………………………………………………………….…93
Figure 5. Monkeys’ performance in body/object categorization task. …….95
Figure 6. The pattern of performance decline as a function of noise level was reverse for bodies and objects …………………………………………….. 96
Figure 7. Monkeys’ performance in body/object categorization task for subcategories …………………………………………………………..…..97
Figure 8 .Performance decline between adjacent signal levels in subcategories of bodies and objects ……………………………….………98
Figure 9. Behavioral d́ (d́ = Z “hit rate” – Z “false alarm”) in different visual signals ……………………………………………………………….…….. 99
Figure 10. Cumulative d́ in signal level of 90 ……………………….……100
Figure 11. Reaction time in different signal conditions in correct and wrong trials……………………………………………………………………….101
 

Figure 12. Reaction time in subcategory level……………………………102
Figure 13. Relation between reaction time and performance in different signal levels…………………………………………………………….…103
Figure 14. Mean number of microsaccades in different noise levels……..104
Figure 15. Mean number of microsaccades in correct and wrong trials of different signal levels……………………………………………………..105
Figure 16. Reaction time in trials with and without microsaccades in different signal levels…………………………………………………..…106
Figure 17. Normalized mean firing rate of body cells across different visual signals and behavioral conditions…………………………………….…..107
Figure 18. Normalized mean firing rate of non-body cells across different visual signals and behavioral conditions…………………………………..109
Figure 19.Response modulation index (RMI) as a function of task difficulty……………………………………………………………….….111
Figure 20. Mean response modulation onset across body image signal levels in body cells’ correct trials…………………………………………….….112
Figure 21. Attentional enhancement of IT cells’ body-object discriminability (d’) was observed only in correct trials and degree of enhancement depended on task difficulty………………………………………………………..…114
Figure 22. Mean d’ modulation in correct (blue) and wrong (black) compared to passive condition in body cells…………………………………………116
Figure 23. Mean d’ modulation in correct (blue) and wrong (black) compared to passive condition in non-body cells……………………………………117
Figure 24. Temporal pattern of baseline firing rate modulation in active compared to passive condition…………………………………………….118
Figure 25. Temporal pattern of p-values of t-tests measuring significant increase of baseline rate in active compared to passive condition……..…119
Figure 26. Frequency distribution of proportion of HBTs in body (top) and non-body (bottom) cells during active task…………………………….…120
Figure 27. Baseline dependent enhancement of body and suppression of non-body cells’ responses to presentation of body images in correct condition……………………………………………………………….….121
Figure 28. Baseline dependent enhancement of body and suppression of non-body cells’ responses to presentation of body images in wrong condition………………………………………………………………..…122
Figure 29. Temporal dynamic of body and non-body cells’ RMI to presentation of body images in correct and wrong conditions for HBTs………………………………………………………………………123
Figure 30. Temporal dynamic of body and non-body cells’ RMI to presentation of body images in correct and wrong conditions for LBTs…124
Figure 31. P-values of t-tests measuring significant enhancement of body and suppression of non-body cells’ responses in HBTs as time window to define high baseline activity varied over time…………………………….125
Figure 32. P-values of t-tests measuring significantly larger RMI values of body and smaller RMI values of non-body cells’ in HBTs vs. LBTs as time window to define high vs. low baseline activity varied over time…….….126
Figure 33. Body cells’ RMI values of high and low baseline trials across body stimulus signal levels…………………………………………….…127
Figure 34. Baseline dependent modulation of body and non-body cells’ responses to presentation of object images…………………………….…129
Figure 35. Baseline dependent modulation of body and non-body cells’ responses to presentation of object images…………………………….…130
Figure 36. Temporal dynamics of body and non-body cells’ RMI to presentation of object images in correct and wrong conditions for HBTs………………………………………………………………………131
Figure 37. Temporal dynamics of body and non-body cells’ RMI to presentation of object images in correct and wrong conditions for LBTs………………………………………………………………………132
Figure 38. Frequency distribution of adjusted RMI values for body and non-body cells in HBTs and LBTs………………………………………….…133
Figure 39. Rate-matched fano factor modulation index (FFMI) of body and non-body cells to presentation of body images in correct condition for HBTs vs. LBTs………………………………………………………………..…134
Figure 40. Rate-matched fano factor modulation (FFMI) of body and non-body cells to presentation of object images in correct conditions for HBTs and LBTs. …………………………………………………………………135
Figure 41a. Frequency distribution of normalized d’ modulation difference in LBTs vs. HBTs for body and non-body cells…………………………..136
Figure 41b. The impact of task specific attentional modulation on firing rate depends on cells’ category selectivity……………………………………..137
Figure 42. The impact of task specific attentional modulation on firing rate depends on cells’ category selectivity………………………………….…138
Figure 43. Comparison of RMI values of correct vs. wrong trials of LBTs and HBTs……………………………………………………………….…139
Figure 44. Comparison of rate modulation in body and non-body cells population across trials of body images with different baseline spike counts……………………………………………………………………..140
Figure 45. Comparison of rate modulation in body and non-body cells population across trials of object images with different baseline spike counts………………………………………………………………….….142
Figure 46. Baseline dependent correlation of neural activity and behavioral choice…………………………………………………………………..…143
Figure 47. Correlation between CP and cells’ body/object discrimination power………………………………………………………………………144
Figure 48. CP values of body cells plotted against the HBTs proportion in active task………………………………………………………………….145
Figure 49. RMI values of body cells plotted against the HBTs proportion in active task…………………………………………………………………146
Figure 50. Attentional modulation of baseline and evoked response in 30 low baseline cells…………………………………………………………147
Figure 51. Attentional modulation of baseline and evoked response in 30 low baseline cells…………………………………………………………148
Figure 52. Percent of HBT in active is plotted vs. percent of HBT in passive for 14 body and 16 non-body cells………………………………………..149
Figure 53. RMI of low baseline body and non-body cells in different stimulus and choice conditions……………………………………………150
Figure 54. RMI of low baseline body and non-body cells in different stimulus and choice conditions……………………………………………151
Figure 55. Percent of HBT is active vs. percent of HBT in passive………152
Figure 56. A combination of baseline firing rate and evoked response modulation in active compared with passive conditions affects monkeys’ performance…………………………………………………………….…153
Figure 57. Polar plots of IT cells activity show that baseline dependent differential response of IT cell subpopulations determines monkey’s choice…………………………………………………………………..…155
Appendix1: Stimulus set……………………………………………….……….158

Appendix2: List of abbreviations………………………………………………164

References………………………………………………………………………..166

Introduction

The crucial role of “visual object categorization” in everyday life

Our normal life relies on ability of visual object recognition or determining the identity of a seen object. We recognize different familiar or novel objects in everyday life. We do this with no or little effort, despite the fact that these objects may vary in form, color, illumination, view, size or texture from time to time. Based on both behavioral and neural data there are different levels of object recognition. When we see Einstein’s face, first we detect it as a “face” (supraordinate level), perceive as a “human face” (ordinate level) and then “Einstein’s face” (subordinate level). Spector and Kanwisher explored the sequence of processing steps in object recognition by asking human subjects to do three different tasks: object detection, categorization and identification. Accuracy and reaction time were similar for object detection and categorization showing that “as soon as you know it is there, you know what it is” (Spector and Kanwisher, 2005). On the other hand, lower accuracy and longer reaction time was observed for identification compared to categorization, introducing them as different steps of object recognition. Compatible with behavioral data firing patterns of single cells in inferior temporal cortex, a cortical area involved in object recognition, convey the information about categorization and identification with different latencies. Earliest part of the response (~120 ms after stimulus presentation) represents information about categorization while more detailed information about members of category started ~50 ms later (Sugase et al., 1999). Therefore, visual cortex processes information from global to fine in a hierarchical fashion.  It has been suggested that categorization relies on the “presence or absence of features”, whereas identification is based on “configurational judgments”.

“Visual object categorization” or our ability to classify objects by giving meaning to our environment enables us to interact normally and efficiently with objects and events. There are some defined classes of objects in our mind. They usually share some major common properties in their appearance, while at the same time there are lots of differences among their members. For example, trees usually grow from the earth, they have roots, stem and usually green leaves. While they have similar properties, each of the species has a set of specific characteristics. But we call all of them trees, and also easily classify any new member as tree, even if we have not seen something like it before. This fascinating ability of categorization objects is vital for our survival. We know special traits for different object categories. We have learned how to treat and interact with any of them, depending on their characteristics. For example, classifying a rod-shaped moving object as “snake” makes us to run away as fast as possible. We perform this task easily and rapidly under very different conditions and even in noisy environment. Behavioral studies in human have shown that they can recognize animals in a cluttered picture which is presented only for 20ms with reaction times less than 400ms and 95% accuracy (Thorpe et al., 1996; Keysers et al., 2001).