Although much has been written about local access to public transport, few studies have examined the role of the perceived environment in promoting transit by considering how the effects of these perceptions differ by neighborhood type. By analyzing the self-reported frequency of transit use and measured neighborhood attributes and using data from Los Angeles County, California, this study examined how the perceptions low-income people have about the walking environment affect their travel behavior. A principal component analysis was used to reduce many overlapping perceptional variables to latent factors that were used in subsequent models of transit use. Four perceptional attributes that affect regular transit use are identified: physical safety, personal safety, amenities, and perceived isolation. The results of this study show that unfavorable perceptions of environmental conditions are independently associated with decreased regular transit use; however, these effects vary among different neighborhood types. Personal safety related to crime and violence is the major concern associated with decreased transit use in mixed land use neighborhoods; in low-density neighborhoods, isolation from the street environment and physical safety concerns, including dangerous crosswalks, are the significant deterrents to public transportation use. Notably, low-income travelers view the conditions of their walking environment as problematic more often than do higher-income travelers, and it appears that transit use by lower-income travelers is more likely to be affected by safety concerns than by other urban design factors of their neighborhood. Findings suggest that safety concerns precede amenity concerns; therefore, enhancing neighborhood safety is the necessary first step to increasing the utility of transit for low-income people.