Synchronization

In a parallel program, the instructions of any given thread appear to occur in sequential order (at least from that thread’s point of view), but if the threads run independently, their sequences of instructions may interleave arbitrarily, and many of the possible interleavings may produce incorrect results. As a trivial example, consider a global counter incremented by multiple threads. Each thread loads the counter into a register, increments the register, and writes the updated value back to memory. If two threads load the same value before either stores it back, updates may be lost: