I was a little confused by the simple worst case example of 1/8th performance provided in class using a conditional on (i % 8 == 0). It seems to me that in a language that knows the contents of the loop are independent, it should be fairly easy for the compiler to include an optimization that bundles together all the expensive iterations where i % 8 == 0 (assuming it is possible for implementations like the above to handle operating on chunks of data that aren't necessarily contiguous). Of course this is only possible since the conditional only depends on the loop iteration and not a more complex value, so perhaps this optimization is too specific to be worth it for real systems?
apoms
For straightforward conditional expressions like this, it is true that a compiler might be able to recognize the pattern and then reorganize the code to maintain coherent execution. However, providing such an optimization might cause confusion for the programmer because there's a non-trivial difference between the code they wrote and what is being executed. For example, if the programmer changed the conditional to a more complex expression and the compiler can no longer recognize that it can apply this optimization, then the programmer will observe a huge performance drop! Without understanding how the compiler works, they would likely believe the performance difference was due to the increased computation of the conditional.
Drawing clear lines between what the programmer needs to think about (the abstraction) and what the system is responsible for handling automatically (the implementation) will be a major theme of this course.
Side note: In general, a major design decision for efficient parallel programs that utilize SIMD is to avoid "divergent" execution caused by conditionals. For example, in ray tracing, there have been several papers on adaptively reorganizing data to ensure conditionals do not cause divergent execution: https://graphics.stanford.edu/~boulos/papers/reorder_rt08.pdf
I was a little confused by the simple worst case example of 1/8th performance provided in class using a conditional on (i % 8 == 0). It seems to me that in a language that knows the contents of the loop are independent, it should be fairly easy for the compiler to include an optimization that bundles together all the expensive iterations where i % 8 == 0 (assuming it is possible for implementations like the above to handle operating on chunks of data that aren't necessarily contiguous). Of course this is only possible since the conditional only depends on the loop iteration and not a more complex value, so perhaps this optimization is too specific to be worth it for real systems?
For straightforward conditional expressions like this, it is true that a compiler might be able to recognize the pattern and then reorganize the code to maintain coherent execution. However, providing such an optimization might cause confusion for the programmer because there's a non-trivial difference between the code they wrote and what is being executed. For example, if the programmer changed the conditional to a more complex expression and the compiler can no longer recognize that it can apply this optimization, then the programmer will observe a huge performance drop! Without understanding how the compiler works, they would likely believe the performance difference was due to the increased computation of the conditional.
Drawing clear lines between what the programmer needs to think about (the abstraction) and what the system is responsible for handling automatically (the implementation) will be a major theme of this course.
Side note: In general, a major design decision for efficient parallel programs that utilize SIMD is to avoid "divergent" execution caused by conditionals. For example, in ray tracing, there have been several papers on adaptively reorganizing data to ensure conditionals do not cause divergent execution: https://graphics.stanford.edu/~boulos/papers/reorder_rt08.pdf