Commit 1ec6b0e9 authored by Jared Casper's avatar Jared Casper
Browse files

Merge branch 'eddiey-main-patch-12774' into 'main' 

fix race in `cuApplyLayerNorm`

See merge request ADLR/megatron-lm!336
parents 3497e4da 4831071c

megatron/fused_kernels/layer_norm_cuda_kernel.cu

+1 −0
Original line number Diff line number Diff line
@@ -329,6 +329,7 @@ void cuApplyLayerNorm( 
      mean[i1] = mu;

      invvar[i1] = c_invvar;

    }

    __syncthreads();

  }

}