Commit 5478d67e authored by Jared Casper's avatar Jared Casper
Browse files

Merge branch 'eddiey-main-patch-57472' into 'main' 

fix potential race in `cuComputeGradInput` in `layer_norm_cuda_kernel.cu`

See merge request ADLR/megatron-lm!340
parents 1ec6b0e9 f1295380

megatron/fused_kernels/layer_norm_cuda_kernel.cu

+2 −0
Original line number Diff line number Diff line
@@ -645,6 +645,8 @@ void cuComputeGradInput( 
        k_grad_input[l] = static_cast<T>(f_grad_input);

      }

    }

    // prevent race where buf is written again before reads are done

    __syncthreads();

  }

}