Ray tracing has a much higher initial performance hit based which is on the rendering resolution (size of window or screen resolution if fullscreen).
A highly optimized ray tracer will get slower with more polygons, but it scales pretty well where 1 mil vs 2 mil doesn't make a really huge difference.
However, ray tracers do heavily slow down on reflective/refractive surfaces.
Also overlapping lights also impact performance quite a bit as more shadow rays need to be cast.
Texture size doesn't matter much, it just takes more memory.