Scarcity of nonrenewable natural resources is stipulating the adoption of global sustainable approaches to ensure better utilization of resources. Abrasion mechanisms take place on concrete surfaces, such as pavements and hydraulic structures, as a result of mechanical wear, scour, erosion, and cavitation. Adequate attention has not been paid to the incorporation of coarse aggregate by-products from the stone industry in concrete used for building transportation infrastructures. This paper evaluates the performance of concrete mixtures that incorporate coarse aggregate by-products of the stone industry for their resistance to abrasion. Twenty-four concrete mixtures were prepared with two types of granite and one type of basalt coarse aggregates procured from the waste of stone processing factories in Egypt. Two coarse aggregate sizes were used: 9.5 and 19 mm. The three mixture sets were compared to a control set prepared with dolomite as coarse aggregate. Fresh and hardened concrete tests as well as two abrasion tests were conducted to evaluate mechanical wear and underwater abrasion resistance of the concrete. The results revealed that the stone by-products were suitable for concrete used in highly abrasive environments. Some mixtures, such as the one made with gray granite, had superior strength as well as enhanced abrasion resistance. Such by-products are underutilized by the applicators. It is recommended that this work be further pursued on a wider scale with greater focus on economical and environmental merits as well as other long-term properties.