{"L1_001": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_051 Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd, 36.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef add_kernel(x_ptr, s_ptr, out_ptr, n_feat, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n row = offs // n_feat\n v = tl.load(x_ptr + offs, mask=mask)\n s = tl.load(s_ptr + row, mask=mask)\n tl.store(out_ptr + offs, v + s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias=True):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n self.out_features = out_features\n\n def forward(self, x):\n # mean over output features of (Wx + b - subtract) = (colsum/out)\u00b7x + c\n colsum = self.gemm.weight.sum(dim=0) / self.out_features # (in,)\n if self.gemm.bias is not None:\n c = (self.gemm.bias - self.subtract).mean()\n else:\n c = (-self.subtract).mean()\n s = x @ colsum + c # (batch,)\n # logsumexp over single element (dim=1 after keepdim mean) is identity\n s = F.gelu(s)\n out = torch.empty_like(x)\n n = x.numel()\n n_feat = x.shape[1]\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n add_kernel[grid](x, s, out, n_feat, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10384, "ctx_sha": "4573cb1c62d0cdf8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Square_matrix_multiplication_", "split": "learn", "n_ops": 3, "src_lines": 32, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_001 (Square_matrix_multiplication_), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single square matrix multiplication (C = A * B)\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the matrix multiplication.\n\n Args:\n A (torch.Tensor): Input matrix A of shape (N, N).\n B (torch.Tensor): Input matrix B of shape (N, N).\n\n Returns:\n torch.Tensor: Output matrix C of shape (N, N).\n \"\"\"\n return torch.matmul(A, B)\n\nN = 2048 * 2\n\ndef get_inputs():\n A = torch.rand(N, N)\n B = torch.rand(N, N)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_002": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_051 Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd, 36.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef add_kernel(x_ptr, s_ptr, out_ptr, n_feat, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n row = offs // n_feat\n v = tl.load(x_ptr + offs, mask=mask)\n s = tl.load(s_ptr + row, mask=mask)\n tl.store(out_ptr + offs, v + s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias=True):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n self.out_features = out_features\n\n def forward(self, x):\n # mean over output features of (Wx + b - subtract) = (colsum/out)\u00b7x + c\n colsum = self.gemm.weight.sum(dim=0) / self.out_features # (in,)\n if self.gemm.bias is not None:\n c = (self.gemm.bias - self.subtract).mean()\n else:\n c = (-self.subtract).mean()\n s = x @ colsum + c # (batch,)\n # logsumexp over single element (dim=1 after keepdim mean) is identity\n s = F.gelu(s)\n out = torch.empty_like(x)\n n = x.numel()\n n_feat = x.shape[1]\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n add_kernel[grid](x, s, out, n_feat, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10384, "ctx_sha": "4573cb1c62d0cdf8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Standard_matrix_multiplication_", "split": "learn", "n_ops": 3, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_002 (Standard_matrix_multiplication_), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B)\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs matrix multiplication.\n\n Args:\n A: Input tensor of shape (M, K).\n B: Input tensor of shape (K, N).\n\n Returns:\n Output tensor of shape (M, N).\n \"\"\"\n return torch.matmul(A, B)\n\nM = 1024 * 2\nK = 4096 * 2\nN = 2048 * 2\n\ndef get_inputs():\n A = torch.rand(M, K)\n B = torch.rand(K, N)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_006": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "8b2060bfd7036bae", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_with_large_K_dimension_", "split": "learn", "n_ops": 5, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_006 (Matmul_with_large_K_dimension_), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B) with a large K dimension\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs matrix multiplication of A and B.\n\n Args:\n A: Input tensor of shape (M, K)\n B: Input tensor of shape (K, N)\n\n Returns:\n Output tensor of shape (M, N)\n \"\"\"\n return torch.matmul(A, B)\n\nM = 256\nN = 256\nK = 131072 * 4\n\ndef get_inputs():\n A = torch.rand(M, K)\n B = torch.rand(K, N)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_008": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "8b2060bfd7036bae", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_with_irregular_shapes_", "split": "learn", "n_ops": 4, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_008 (Matmul_with_irregular_shapes_), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B) with irregular shapes\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs matrix multiplication of A and B.\n\n Args:\n A: Input tensor with shape (M, K).\n B: Input tensor with shape (K, N).\n\n Returns:\n C: Output tensor with shape (M, N).\n \"\"\"\n return torch.matmul(A, B)\n\nM = 8205\nK = 2949\nN = 5921\n\ndef get_inputs():\n A = torch.rand(M, K)\n B = torch.rand(K, N)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_010": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_051 Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd, 36.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef add_kernel(x_ptr, s_ptr, out_ptr, n_feat, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n row = offs // n_feat\n v = tl.load(x_ptr + offs, mask=mask)\n s = tl.load(s_ptr + row, mask=mask)\n tl.store(out_ptr + offs, v + s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias=True):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n self.out_features = out_features\n\n def forward(self, x):\n # mean over output features of (Wx + b - subtract) = (colsum/out)\u00b7x + c\n colsum = self.gemm.weight.sum(dim=0) / self.out_features # (in,)\n if self.gemm.bias is not None:\n c = (self.gemm.bias - self.subtract).mean()\n else:\n c = (-self.subtract).mean()\n s = x @ colsum + c # (batch,)\n # logsumexp over single element (dim=1 after keepdim mean) is identity\n s = F.gelu(s)\n out = torch.empty_like(x)\n n = x.numel()\n n_feat = x.shape[1]\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n add_kernel[grid](x, s, out, n_feat, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10384, "ctx_sha": "4573cb1c62d0cdf8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "3D_tensor_matrix_multiplication", "split": "learn", "n_ops": 4, "src_lines": 35, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_010 (3D_tensor_matrix_multiplication), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs 3D tensor-matrix multiplication.\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A, B):\n \"\"\"\n Performs 3D tensor-matrix multiplication.\n\n Args:\n A (torch.Tensor): Input 3D tensor of shape (N, M, K).\n B (torch.Tensor): Input matrix of shape (K, L).\n\n Returns:\n torch.Tensor: Output tensor of shape (N, M, L), resulting from the multiplication of A and B along the last dimension of A.\n \"\"\"\n return torch.matmul(A, B)\n\nN = 16\nM = 1024\nK = 2048\nL = 768\n\ndef get_inputs():\n A = torch.rand(N, M, K)\n B = torch.rand(K, L)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_012": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```\n\n### A verified kernel for a related matmul problem (L1_014 Matmul_for_upper_triangular_matrices, 4.11x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef triu_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n # skip tiles entirely in lower triangle: n_hi < m_lo\n if (n_lo + BLOCK_N - 1) < m_lo:\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n # k from i to j => k in [m_lo, n_hi]\n k_start = (m_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(n_lo + BLOCK_N, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] <= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n triu_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10850, "ctx_sha": "daebc65ff56f9d20", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_with_diagonal_matrices_", "split": "learn", "n_ops": 4, "src_lines": 36, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_012 (Matmul_with_diagonal_matrices_), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication of a diagonal matrix with another matrix.\n C = diag(A) * B\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A, B):\n \"\"\"\n Performs the matrix multiplication.\n\n Args:\n A (torch.Tensor): A 1D tensor representing the diagonal of the diagonal matrix. Shape: (N,).\n B (torch.Tensor): A 2D tensor representing the second matrix. Shape: (N, M).\n\n Returns:\n torch.Tensor: The result of the matrix multiplication. Shape: (N, M).\n \"\"\"\n # Logically equivalent to torch.diag(A) @ B \n # more efficient as no need to materialize a full N\u00d7N matrix\n return A.unsqueeze(1) * B\n\nM = 4096\nN = 4096\n\ndef get_inputs():\n A = torch.rand(N)\n B = torch.rand(N, M)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_013": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```\n\n### A verified kernel for a related matmul problem (L1_014 Matmul_for_upper_triangular_matrices, 4.11x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef triu_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n # skip tiles entirely in lower triangle: n_hi < m_lo\n if (n_lo + BLOCK_N - 1) < m_lo:\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n # k from i to j => k in [m_lo, n_hi]\n k_start = (m_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(n_lo + BLOCK_N, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] <= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n triu_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10850, "ctx_sha": "daebc65ff56f9d20", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_for_symmetric_matrices", "split": "learn", "n_ops": 4, "src_lines": 46, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_013 (Matmul_for_symmetric_matrices), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B) with A and B being symmetric matrices.\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A, B):\n \"\"\"\n Performs matrix multiplication of two symmetric matrices.\n\n Args:\n A (torch.Tensor): Input matrix A, shape (N, N), symmetric.\n B (torch.Tensor): Input matrix B, shape (N, N), symmetric.\n\n Returns:\n torch.Tensor: Output matrix C, shape (N, N).\n \"\"\"\n return torch.matmul(A, B)\n\nN = 4096\n\ndef get_inputs():\n \"\"\"\n Generates a pair of random symmetric matrices for testing.\n\n Returns:\n list: List containing two symmetric tensors A and B.\n \"\"\"\n A = torch.rand(N, N)\n A = (A + A.T) / 2 # Ensure symmetry\n B = torch.rand(N, N)\n B = (B + B.T) / 2 # Ensure symmetry\n return [A, B]\n\ndef get_init_inputs():\n \"\"\"\n No specific initialization inputs needed for this model.\n\n Returns:\n list: Empty list.\n \"\"\"\n return []```\n"}, "L1_014": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9311, "ctx_sha": "6a99911cb1ac1a4e", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 5, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_for_upper_triangular_matrices", "split": "learn", "n_ops": 5, "src_lines": 44, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_014 (Matmul_for_upper_triangular_matrices), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs matrix multiplication (C = A * B) for upper triangular matrices.\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A, B):\n \"\"\"\n Performs matrix multiplication for upper triangular matrices.\n\n Args:\n A (torch.Tensor): Upper triangular matrix of shape (N, N).\n B (torch.Tensor): Upper triangular matrix of shape (N, N).\n\n Returns:\n torch.Tensor: The product of A and B, also an upper triangular matrix of shape (N, N).\n \"\"\"\n return torch.triu(torch.matmul(A, B))\n\nN = 4096\n\ndef get_inputs():\n \"\"\"\n Generates upper triangular matrices for testing.\n\n Returns:\n list: A list containing two upper triangular matrices of shape (N, N).\n \"\"\"\n A = torch.triu(torch.rand(N, N))\n B = torch.triu(torch.rand(N, N))\n return [A, B]\n\ndef get_init_inputs():\n \"\"\"\n No specific initialization inputs are needed for this model.\n\n Returns:\n list: An empty list.\n \"\"\"\n return []```\n"}, "L1_015": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L1_014 Matmul_for_upper_triangular_matrices, 4.11x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef triu_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n # skip tiles entirely in lower triangle: n_hi < m_lo\n if (n_lo + BLOCK_N - 1) < m_lo:\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n # k from i to j => k in [m_lo, n_hi]\n k_start = (m_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(n_lo + BLOCK_N, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] <= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n triu_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9438, "ctx_sha": "933f5608fb268360", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 5, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_for_lower_triangular_matrices", "split": "learn", "n_ops": 5, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_015 (Matmul_for_lower_triangular_matrices), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication (C = A * B) where A and B are lower triangular matrices. \n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A, B):\n \"\"\"\n Performs matrix multiplication of lower triangular matrices A and B.\n\n Args:\n A (torch.Tensor): Lower triangular matrix of shape (N, N).\n B (torch.Tensor): Lower triangular matrix of shape (N, N).\n\n Returns:\n torch.Tensor: The result of matrix multiplication C of shape (N, N).\n \"\"\"\n return torch.tril(torch.matmul(A, B))\n\nM = 4096\n\ndef get_inputs():\n A = torch.rand(M, M)\n B = torch.rand(M, M)\n A = torch.tril(A)\n B = torch.tril(B)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_016": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "8b2060bfd7036bae", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_with_transposed_A", "split": "learn", "n_ops": 4, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_016 (Matmul_with_transposed_A), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B)\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs matrix multiplication.\n\n Args:\n A: Input tensor of shape (M, K).\n B: Input tensor of shape (K, N).\n\n Returns:\n Output tensor of shape (M, N).\n \"\"\"\n return torch.matmul(A.T, B)\n\nM = 1024 * 2\nK = 4096 * 2\nN = 2048 * 2\n\ndef get_inputs():\n A = torch.rand(K, M)\n B = torch.rand(K, N)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_017": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "8b2060bfd7036bae", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_with_transposed_B", "split": "learn", "n_ops": 4, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_017 (Matmul_with_transposed_B), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B)\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs matrix multiplication.\n\n Args:\n A: Input tensor of shape (M, K).\n B: Input tensor of shape (K, N).\n\n Returns:\n Output tensor of shape (M, N).\n \"\"\"\n return torch.matmul(A, B.T)\n\nM = 1024 * 2\nK = 4096 * 2\nN = 2048 * 2\n\ndef get_inputs():\n A = torch.rand(M, K)\n B = torch.rand(N, K)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_018": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "8b2060bfd7036bae", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 1, "name": "Matmul_with_transposed_both", "split": "learn", "n_ops": 4, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 1 problem L1_018 (Matmul_with_transposed_both), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a single matrix multiplication (C = A * B)\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs matrix multiplication.\n\n Args:\n A: Input tensor of shape (M, K).\n B: Input tensor of shape (K, N).\n\n Returns:\n Output tensor of shape (M, N).\n \"\"\"\n return torch.matmul(A.T, B.T)\n\nM = 1024 * 2\nK = 4096 * 2\nN = 2048 * 2\n\ndef get_inputs():\n A = torch.rand(K, M)\n B = torch.rand(N, K)\n return [A, B]\n\ndef get_init_inputs():\n return [] # No special initialization inputs needed```\n"}, "L1_040": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Verified recipes for norm problems (speedups measured by the protected evaluator)\n- [1.02x on BMM_InstanceNorm_Sum_ResidualAdd_Multiply] Leave the GEMM to cuBLAS with torch.addmm(bias, x, W.t()), keeping the bias fused into the GEMM. Fuse the normalization and all elementwise epilogue ops (residual add, multiply) into one Triton kernel that launches one program per normalized row. First pass: loop over the row in BLOCK=2048 chunks and accumulate fp32 sum and sum-of-squares. Compute rstd = 1/sqrt(var+eps). Second pass: reload x, load y, normalize, apply the epilogue and store, using contiguous masked loads. This passes fp32 1e-4. Gains are small because the GEMM dominates.\n\n### A verified kernel for a related norm problem (L2_028 BMM_InstanceNorm_Sum_ResidualAdd_Multiply, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef norm_fuse_kernel(x_ptr, y_ptr, out_ptr, N, eps, BLOCK: tl.constexpr):\n row = tl.program_id(0)\n x_ptr += row * N\n y_ptr += row * N\n out_ptr += row * N\n sum_ = 0.0\n sumsq = 0.0\n for off in range(0, N, BLOCK):\n idx = off + tl.arange(0, BLOCK)\n m = idx < N\n v = tl.load(x_ptr + idx, mask=m, other=0.0).to(tl.float32)\n sum_ += tl.sum(v, axis=0)\n sumsq += tl.sum(v * v, axis=0)\n mean = sum_ / N\n var = sumsq / N - mean * mean\n rstd = 1.0 / tl.sqrt(var + eps)\n for off in range(0, N, BLOCK):\n idx = off + tl.arange(0, BLOCK)\n m = idx < N\n v = tl.load(x_ptr + idx, mask=m, other=0.0).to(tl.float32)\n yv = tl.load(y_ptr + idx, mask=m, other=0.0).to(tl.float32)\n nv = (v - mean) * rstd\n res = (nv + yv) * yv\n tl.store(out_ptr + idx, res, mask=m)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, eps=1e-5, momentum=0.1):\n super().__init__()\n self.bmm = nn.Linear(in_features, out_features)\n self.instance_norm = nn.InstanceNorm2d(out_features, eps=eps, momentum=momentum)\n self.eps = eps\n\n def forward(self, x, y):\n x = torch.addmm(self.bmm.bias, x, self.bmm.weight.t())\n x = x.contiguous()\n y = y.contiguous()\n B, N = x.shape\n out = torch.empty_like(x)\n norm_fuse_kernel[(B,)](x, y, out, N, self.eps, BLOCK=2048)\n return out\n```", "ctx_chars": 4348, "ctx_sha": "fb2a02ef8ac19686", "dropped_same_problem": 1, "kinds": {"lesson": 14, "recipe": 1, "kernel": 1}, "family": "norm", "level": 1, "name": "LayerNorm", "split": "learn", "n_ops": 1, "src_lines": 40, "max_recipe_speedup": 1.02, "instruction": "KernelBench level 1 problem L1_040 (LayerNorm), op family: norm.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs Layer Normalization.\n \"\"\"\n def __init__(self, normalized_shape: tuple):\n \"\"\"\n Initializes the LayerNorm layer.\n\n Args:\n normalized_shape (tuple): Shape of the input tensor to be normalized.\n \"\"\"\n super(Model, self).__init__()\n self.ln = nn.LayerNorm(normalized_shape=normalized_shape)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Applies Layer Normalization to the input tensor.\n\n Args:\n x (torch.Tensor): Input tensor of shape (*, normalized_shape).\n\n Returns:\n torch.Tensor: Output tensor with Layer Normalization applied, same shape as input.\n \"\"\"\n return self.ln(x)\n\nbatch_size = 16\nfeatures = 64\ndim1 = 256\ndim2 = 256\n\ndef get_inputs():\n x = torch.rand(batch_size, features, dim1, dim2)\n return [x]\n\ndef get_init_inputs():\n return [(features, dim1, dim2)]```\n"}, "L1_050": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_083 conv_depthwise_2D_square_input_asymmetric_kernel, 2.43x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K: tl.constexpr, HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw = ow * stride - padding\n ih0 = oh * stride - padding\n wvalid = mask & (iw >= 0) & (iw < W)\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * K\n\n for k in tl.static_range(K):\n ih = ih0 + k * dilation\n if (ih >= 0) & (ih < H):\n idx = base + ih.to(tl.int64) * W + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + k)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, kernel_size=(kernel_size, 1),\n stride=stride, padding=padding, dilation=dilation,\n groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.kernel_size = kernel_size\n self.stride = stride\n self.padding = padding\n self.dilation = dilation\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n K = self.kernel_size\n stride = self.stride\n padding = self.padding\n dilation = self.dilation\n OH = (H + 2 * padding - dilation * (K - 1) - 1) // stride + 1\n OW = (W + 2 * padding - 1) // stride + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 256\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K=K, HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_085 conv_depthwise_2D_asymmetric_input_asymmetric_kernel, 1.35x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride_h, stride_w, pad_h, pad_w, dil_h, dil_w,\n KH: tl.constexpr, KW: tl.constexpr,\n HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw0 = ow * stride_w - pad_w\n ih0 = oh * stride_h - pad_h\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * (KH * KW)\n\n for kh in tl.static_range(KH):\n ih = ih0 + kh * dil_h\n if (ih >= 0) & (ih < H):\n rbase = base + ih.to(tl.int64) * W\n for kw in tl.static_range(KW):\n iw = iw0 + kw * dil_w\n wvalid = mask & (iw >= 0) & (iw < W)\n idx = rbase + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + kh * KW + kw)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size_h: int, kernel_size_w: int,\n stride_h: int = 1, stride_w: int = 1, padding_h: int = 0, padding_w: int = 0,\n dilation_h: int = 1, dilation_w: int = 1, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, (kernel_size_h, kernel_size_w),\n stride=(stride_h, stride_w), padding=(padding_h, padding_w),\n dilation=(dilation_h, dilation_w), groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.KH = kernel_size_h\n self.KW = kernel_size_w\n self.stride_h = stride_h\n self.stride_w = stride_w\n self.padding_h = padding_h\n self.padding_w = padding_w\n self.dilation_h = dilation_h\n self.dilation_w = dilation_w\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n KH, KW = self.KH, self.KW\n OH = (H + 2 * self.padding_h - self.dilation_h * (KH - 1) - 1) // self.stride_h + 1\n OW = (W + 2 * self.padding_w - self.dilation_w * (KW - 1) - 1) // self.stride_w + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 128\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n self.stride_h, self.stride_w, self.padding_h, self.padding_w,\n self.dilation_h, self.dilation_w,\n KH=KH, KW=KW,\n HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```", "ctx_chars": 13278, "ctx_sha": "cb78f644acd17887", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_standard_2D__square_input__square_kernel", "split": "learn", "n_ops": 7, "src_lines": 22, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 1 problem L1_050 (conv_standard_2D__square_input__square_kernel), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000):\n super(Model, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n \n def forward(self, x):\n x = self.conv1(x)\n return x\n\n# Test code\nbatch_size = 256\nnum_classes = 1000\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, 224, 224)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L1_056": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_085 conv_depthwise_2D_asymmetric_input_asymmetric_kernel, 1.35x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride_h, stride_w, pad_h, pad_w, dil_h, dil_w,\n KH: tl.constexpr, KW: tl.constexpr,\n HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw0 = ow * stride_w - pad_w\n ih0 = oh * stride_h - pad_h\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * (KH * KW)\n\n for kh in tl.static_range(KH):\n ih = ih0 + kh * dil_h\n if (ih >= 0) & (ih < H):\n rbase = base + ih.to(tl.int64) * W\n for kw in tl.static_range(KW):\n iw = iw0 + kw * dil_w\n wvalid = mask & (iw >= 0) & (iw < W)\n idx = rbase + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + kh * KW + kw)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size_h: int, kernel_size_w: int,\n stride_h: int = 1, stride_w: int = 1, padding_h: int = 0, padding_w: int = 0,\n dilation_h: int = 1, dilation_w: int = 1, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, (kernel_size_h, kernel_size_w),\n stride=(stride_h, stride_w), padding=(padding_h, padding_w),\n dilation=(dilation_h, dilation_w), groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.KH = kernel_size_h\n self.KW = kernel_size_w\n self.stride_h = stride_h\n self.stride_w = stride_w\n self.padding_h = padding_h\n self.padding_w = padding_w\n self.dilation_h = dilation_h\n self.dilation_w = dilation_w\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n KH, KW = self.KH, self.KW\n OH = (H + 2 * self.padding_h - self.dilation_h * (KH - 1) - 1) // self.stride_h + 1\n OW = (W + 2 * self.padding_w - self.dilation_w * (KW - 1) - 1) // self.stride_w + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 128\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n self.stride_h, self.stride_w, self.padding_h, self.padding_w,\n self.dilation_h, self.dilation_w,\n KH=KH, KW=KW,\n HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_050 conv_standard_2D__square_input__square_kernel, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_kernel(x_ptr, bias_ptr, N, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n tl.store(x_ptr + offs, v + b, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, num_classes=1000):\n super(ModelNew, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n\n def forward(self, x):\n out = F.conv2d(x, self.conv1.weight, None, stride=4, padding=2)\n out = out.contiguous()\n N, C, H, W = out.shape\n S = H * W\n TOTAL = N * C * S\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bias_kernel[grid](out, self.conv1.bias.contiguous(), N, C, S, TOTAL, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11539, "ctx_sha": "eb85e1c1c9a1056f", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_standard_2D__asymmetric_input__asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 47, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 1 problem L1_056 (conv_standard_2D__asymmetric_input__asymmetric_kernel), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a standard 2D convolution operation with asymmetric input and kernel sizes.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Tuple of two integers representing the height and width of the convolution kernel.\n stride (tuple, optional): Tuple of two integers representing the stride in the height and width dimensions. Defaults to (1, 1).\n padding (tuple, optional): Tuple of two integers representing the padding in the height and width dimensions. Defaults to (0, 0).\n dilation (tuple, optional): Tuple of two integers representing the dilation in the height and width dimensions. Defaults to (1, 1).\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: tuple = (1, 1), padding: tuple = (0, 0), dilation: tuple = (1, 1), groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, dilation=dilation, groups=groups, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv2d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 64\nout_channels = 128\nkernel_size = (5, 7)\nheight = 512\nwidth = 256\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_058": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_061 conv_transposed_3D__square_input__square_kernel, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KD, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KD * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n kd = t % KD\n t = t // KD\n oc = t % OC\n ic = t // OC\n src = ((((ic * OC + oc) * KD + kd) * KH + kh) * KW + kw)\n fkd = KD - 1 - kd\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((((oc * IC + ic) * KD + fkd) * KH + fkh) * KW + fkw)\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n self.stride = stride\n self.padding = padding\n self.output_padding = output_padding\n self.groups = groups\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n simple = (self.stride == 1 and self.output_padding == 0 and self.groups == 1 and x.is_cuda)\n if simple:\n w = self.conv_transpose3d.weight # (IC, OC, KD, KH, KW)\n IC, OC, KD, KH, KW = w.shape\n wf = torch.empty((OC, IC, KD, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KD * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KD, KH, KW, BLOCK=BLOCK)\n p = KD - 1 - self.padding\n b = self.conv_transpose3d.bias if self.has_bias else None\n return F.conv3d(x, wf, bias=b, stride=1, padding=(p, p, p))\n return self.conv_transpose3d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L1_078 conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__, 2.29x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n oc = t % OC\n ic = t // OC\n src = ((ic * OC + oc) * KH + kh) * KW + kw\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((oc * IC + ic) * KH + fkh) * KW + fkw\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple,\n stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size,\n stride=stride, padding=padding, bias=bias)\n self.stride = stride\n self.padding = padding\n self.kernel_size = kernel_size\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.has_bias = bias\n\n def forward(self, x):\n stride_ok = (self.stride == (1, 1) or self.stride == 1)\n if stride_ok and x.is_cuda:\n w = self.conv_transpose2d.weight # (IC, OC, KH, KW)\n IC, OC, KH, KW = w.shape\n wf = torch.empty((OC, IC, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KH, KW, BLOCK=BLOCK)\n pad_h = KH - 1 - self.padding[0]\n pad_w = KW - 1 - self.padding[1]\n b = self.conv_transpose2d.bias if self.has_bias else None\n return torch.nn.functional.conv2d(x, wf, bias=b, stride=1,\n padding=(pad_h, pad_w))\n return self.conv_transpose2d(x)\n```", "ctx_chars": 11175, "ctx_sha": "5e031feda794c122", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_3D__asymmetric_input__asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 48, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 1 problem L1_058 (conv_transposed_3D__asymmetric_input__asymmetric_kernel), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a transposed 3D convolution operation with asymmetric input and kernel sizes.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Tuple of 3 integers representing the kernel size in the form (depth, height, width).\n stride (tuple, optional): Tuple of 3 integers representing the stride in the form (depth, height, width). Defaults to (1, 1, 1).\n padding (tuple, optional): Tuple of 3 integers representing the padding in the form (depth, height, width). Defaults to (0, 0, 0).\n output_padding (tuple, optional): Tuple of 3 integers representing the output padding in the form (depth, height, width). Defaults to (0, 0, 0).\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: tuple = (1, 1, 1), padding: tuple = (0, 0, 0), output_padding: tuple = (0, 0, 0), groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the transposed 3D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, depth_in, height_in, width_in).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, depth_out, height_out, width_out).\n \"\"\"\n return self.conv_transpose3d(x)\n\n# Test code\nbatch_size = 16\nin_channels = 32\nout_channels = 16\nkernel_size = (3, 5, 7) # Asymmetric kernel size\ndepth_in = 16\nheight_in = 32\nwidth_in = 64\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, depth_in, height_in, width_in)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_061": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_078 conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__, 2.29x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n oc = t % OC\n ic = t // OC\n src = ((ic * OC + oc) * KH + kh) * KW + kw\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((oc * IC + ic) * KH + fkh) * KW + fkw\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple,\n stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size,\n stride=stride, padding=padding, bias=bias)\n self.stride = stride\n self.padding = padding\n self.kernel_size = kernel_size\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.has_bias = bias\n\n def forward(self, x):\n stride_ok = (self.stride == (1, 1) or self.stride == 1)\n if stride_ok and x.is_cuda:\n w = self.conv_transpose2d.weight # (IC, OC, KH, KW)\n IC, OC, KH, KW = w.shape\n wf = torch.empty((OC, IC, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KH, KW, BLOCK=BLOCK)\n pad_h = KH - 1 - self.padding[0]\n pad_w = KW - 1 - self.padding[1]\n b = self.conv_transpose2d.bias if self.has_bias else None\n return torch.nn.functional.conv2d(x, wf, bias=b, stride=1,\n padding=(pad_h, pad_w))\n return self.conv_transpose2d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 12117, "ctx_sha": "5ee3def88dc5d5e6", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_3D__square_input__square_kernel", "split": "learn", "n_ops": 7, "src_lines": 48, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 1 problem L1_061 (conv_transposed_3D__square_input__square_kernel), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a transposed 3D convolution with square input and square kernel.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (int): Size of the square convolution kernel.\n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (int, optional): Padding applied to the input. Defaults to 0.\n output_padding (int, optional): Additional size added to one side of the output shape. Defaults to 0.\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the transposed 3D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, depth, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, depth_out, height_out, width_out).\n \"\"\"\n return self.conv_transpose3d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 48\nout_channels = 48\nkernel_size = 3\ndepth = 64\nheight = 64\nwidth = 64\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, depth, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_062": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_050 conv_standard_2D__square_input__square_kernel, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_kernel(x_ptr, bias_ptr, N, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n tl.store(x_ptr + offs, v + b, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, num_classes=1000):\n super(ModelNew, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n\n def forward(self, x):\n out = F.conv2d(x, self.conv1.weight, None, stride=4, padding=2)\n out = out.contiguous()\n N, C, H, W = out.shape\n S = H * W\n TOTAL = N * C * S\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bias_kernel[grid](out, self.conv1.bias.contiguous(), N, C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_083 conv_depthwise_2D_square_input_asymmetric_kernel, 2.43x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K: tl.constexpr, HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw = ow * stride - padding\n ih0 = oh * stride - padding\n wvalid = mask & (iw >= 0) & (iw < W)\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * K\n\n for k in tl.static_range(K):\n ih = ih0 + k * dilation\n if (ih >= 0) & (ih < H):\n idx = base + ih.to(tl.int64) * W + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + k)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, kernel_size=(kernel_size, 1),\n stride=stride, padding=padding, dilation=dilation,\n groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.kernel_size = kernel_size\n self.stride = stride\n self.padding = padding\n self.dilation = dilation\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n K = self.kernel_size\n stride = self.stride\n padding = self.padding\n dilation = self.dilation\n OH = (H + 2 * padding - dilation * (K - 1) - 1) // stride + 1\n OW = (W + 2 * padding - 1) // stride + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 256\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K=K, HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```", "ctx_chars": 10864, "ctx_sha": "0b6222244e19e377", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_standard_2D__square_input__asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 47, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 1 problem L1_062 (conv_standard_2D__square_input__asymmetric_kernel), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a standard 2D convolution operation with a square input and an asymmetric kernel.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Size of the convolution kernel (height, width).\n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (int or tuple, optional): Padding applied to the input. Defaults to 0.\n dilation (int or tuple, optional): Spacing between kernel elements. Defaults to 1.\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: int = 1, padding: int = 0, dilation: int = 1, groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, dilation=dilation, groups=groups, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv2d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 32\nout_channels = 64\nkernel_size = (5, 9)\nwidth = 512\nheight = 512\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_065": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_078 conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__, 2.29x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n oc = t % OC\n ic = t // OC\n src = ((ic * OC + oc) * KH + kh) * KW + kw\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((oc * IC + ic) * KH + fkh) * KW + fkw\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple,\n stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size,\n stride=stride, padding=padding, bias=bias)\n self.stride = stride\n self.padding = padding\n self.kernel_size = kernel_size\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.has_bias = bias\n\n def forward(self, x):\n stride_ok = (self.stride == (1, 1) or self.stride == 1)\n if stride_ok and x.is_cuda:\n w = self.conv_transpose2d.weight # (IC, OC, KH, KW)\n IC, OC, KH, KW = w.shape\n wf = torch.empty((OC, IC, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KH, KW, BLOCK=BLOCK)\n pad_h = KH - 1 - self.padding[0]\n pad_w = KW - 1 - self.padding[1]\n b = self.conv_transpose2d.bias if self.has_bias else None\n return torch.nn.functional.conv2d(x, wf, bias=b, stride=1,\n padding=(pad_h, pad_w))\n return self.conv_transpose2d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L1_061 conv_transposed_3D__square_input__square_kernel, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KD, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KD * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n kd = t % KD\n t = t // KD\n oc = t % OC\n ic = t // OC\n src = ((((ic * OC + oc) * KD + kd) * KH + kh) * KW + kw)\n fkd = KD - 1 - kd\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((((oc * IC + ic) * KD + fkd) * KH + fkh) * KW + fkw)\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n self.stride = stride\n self.padding = padding\n self.output_padding = output_padding\n self.groups = groups\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n simple = (self.stride == 1 and self.output_padding == 0 and self.groups == 1 and x.is_cuda)\n if simple:\n w = self.conv_transpose3d.weight # (IC, OC, KD, KH, KW)\n IC, OC, KD, KH, KW = w.shape\n wf = torch.empty((OC, IC, KD, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KD * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KD, KH, KW, BLOCK=BLOCK)\n p = KD - 1 - self.padding\n b = self.conv_transpose3d.bias if self.has_bias else None\n return F.conv3d(x, wf, bias=b, stride=1, padding=(p, p, p))\n return self.conv_transpose3d(x)\n```", "ctx_chars": 11175, "ctx_sha": "91e4a03c424736b2", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_2D__square_input__asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 47, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 1 problem L1_065 (conv_transposed_2D__square_input__asymmetric_kernel), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a transposed 2D convolution with a square input and an asymmetric kernel.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Size of the convolution kernel (height, width).\n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (int or tuple, optional): Padding applied to the input. Defaults to 0.\n output_padding (int or tuple, optional): Additional size added to one side of the output shape. Defaults to 0.\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the transposed 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv_transpose2d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 64\nout_channels = 64\nkernel_size = (3, 7) # larger asymmetric kernel\nwidth = 512\nheight = 512\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_066": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_085 conv_depthwise_2D_asymmetric_input_asymmetric_kernel, 1.35x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride_h, stride_w, pad_h, pad_w, dil_h, dil_w,\n KH: tl.constexpr, KW: tl.constexpr,\n HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw0 = ow * stride_w - pad_w\n ih0 = oh * stride_h - pad_h\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * (KH * KW)\n\n for kh in tl.static_range(KH):\n ih = ih0 + kh * dil_h\n if (ih >= 0) & (ih < H):\n rbase = base + ih.to(tl.int64) * W\n for kw in tl.static_range(KW):\n iw = iw0 + kw * dil_w\n wvalid = mask & (iw >= 0) & (iw < W)\n idx = rbase + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + kh * KW + kw)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size_h: int, kernel_size_w: int,\n stride_h: int = 1, stride_w: int = 1, padding_h: int = 0, padding_w: int = 0,\n dilation_h: int = 1, dilation_w: int = 1, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, (kernel_size_h, kernel_size_w),\n stride=(stride_h, stride_w), padding=(padding_h, padding_w),\n dilation=(dilation_h, dilation_w), groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.KH = kernel_size_h\n self.KW = kernel_size_w\n self.stride_h = stride_h\n self.stride_w = stride_w\n self.padding_h = padding_h\n self.padding_w = padding_w\n self.dilation_h = dilation_h\n self.dilation_w = dilation_w\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n KH, KW = self.KH, self.KW\n OH = (H + 2 * self.padding_h - self.dilation_h * (KH - 1) - 1) // self.stride_h + 1\n OW = (W + 2 * self.padding_w - self.dilation_w * (KW - 1) - 1) // self.stride_w + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 128\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n self.stride_h, self.stride_w, self.padding_h, self.padding_w,\n self.dilation_h, self.dilation_w,\n KH=KH, KW=KW,\n HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_050 conv_standard_2D__square_input__square_kernel, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_kernel(x_ptr, bias_ptr, N, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n tl.store(x_ptr + offs, v + b, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, num_classes=1000):\n super(ModelNew, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n\n def forward(self, x):\n out = F.conv2d(x, self.conv1.weight, None, stride=4, padding=2)\n out = out.contiguous()\n N, C, H, W = out.shape\n S = H * W\n TOTAL = N * C * S\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bias_kernel[grid](out, self.conv1.bias.contiguous(), N, C, S, TOTAL, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11539, "ctx_sha": "eb85e1c1c9a1056f", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_standard_3D__asymmetric_input__asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 48, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 1 problem L1_066 (conv_standard_3D__asymmetric_input__asymmetric_kernel), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a standard 3D convolution operation with asymmetric input and kernel sizes.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Size of the convolution kernel in the form (kernel_size_d, kernel_size_h, kernel_size_w).\n stride (tuple, optional): Stride of the convolution in the form (stride_d, stride_h, stride_w). Defaults to (1, 1, 1).\n padding (tuple, optional): Padding applied to the input in the form (padding_d, padding_h, padding_w). Defaults to (0, 0, 0).\n dilation (tuple, optional): Spacing between kernel elements in the form (dilation_d, dilation_h, dilation_w). Defaults to (1, 1, 1).\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: tuple = (1, 1, 1), padding: tuple = (0, 0, 0), dilation: tuple = (1, 1, 1), groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv3d = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, dilation=dilation, groups=groups, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 3D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, depth, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, depth_out, height_out, width_out).\n \"\"\"\n return self.conv3d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 3\nout_channels = 64\nkernel_size = (3, 5, 7) # Asymmetric kernel size\ndepth = 16\nheight = 128\nwidth = 128\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, depth, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_071": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_078 conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__, 2.29x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n oc = t % OC\n ic = t // OC\n src = ((ic * OC + oc) * KH + kh) * KW + kw\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((oc * IC + ic) * KH + fkh) * KW + fkw\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple,\n stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size,\n stride=stride, padding=padding, bias=bias)\n self.stride = stride\n self.padding = padding\n self.kernel_size = kernel_size\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.has_bias = bias\n\n def forward(self, x):\n stride_ok = (self.stride == (1, 1) or self.stride == 1)\n if stride_ok and x.is_cuda:\n w = self.conv_transpose2d.weight # (IC, OC, KH, KW)\n IC, OC, KH, KW = w.shape\n wf = torch.empty((OC, IC, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KH, KW, BLOCK=BLOCK)\n pad_h = KH - 1 - self.padding[0]\n pad_w = KW - 1 - self.padding[1]\n b = self.conv_transpose2d.bias if self.has_bias else None\n return torch.nn.functional.conv2d(x, wf, bias=b, stride=1,\n padding=(pad_h, pad_w))\n return self.conv_transpose2d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L1_061 conv_transposed_3D__square_input__square_kernel, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KD, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KD * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n kd = t % KD\n t = t // KD\n oc = t % OC\n ic = t // OC\n src = ((((ic * OC + oc) * KD + kd) * KH + kh) * KW + kw)\n fkd = KD - 1 - kd\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((((oc * IC + ic) * KD + fkd) * KH + fkh) * KW + fkw)\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n self.stride = stride\n self.padding = padding\n self.output_padding = output_padding\n self.groups = groups\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n simple = (self.stride == 1 and self.output_padding == 0 and self.groups == 1 and x.is_cuda)\n if simple:\n w = self.conv_transpose3d.weight # (IC, OC, KD, KH, KW)\n IC, OC, KD, KH, KW = w.shape\n wf = torch.empty((OC, IC, KD, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KD * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KD, KH, KW, BLOCK=BLOCK)\n p = KD - 1 - self.padding\n b = self.conv_transpose3d.bias if self.has_bias else None\n return F.conv3d(x, wf, bias=b, stride=1, padding=(p, p, p))\n return self.conv_transpose3d(x)\n```", "ctx_chars": 11175, "ctx_sha": "91e4a03c424736b2", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_2D__asymmetric_input__square_kernel", "split": "learn", "n_ops": 7, "src_lines": 48, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 1 problem L1_071 (conv_transposed_2D__asymmetric_input__square_kernel), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a transposed 2D convolution with asymmetric input and a square kernel.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (int): Size of the square convolution kernel.\n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (int, optional): Padding applied to the input. Defaults to 0.\n output_padding (int, optional): Additional size added to one side of the output shape. Defaults to 0.\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the transposed 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height_in, width_in).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv_transpose2d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 32\nout_channels = 32\nkernel_size = 3\n# large asymmetric input\nheight_in = 512\nwidth_in = 1024\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height_in, width_in)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size] # Provide in_channels, out_channels, kernel_size for initialization```\n"}, "L1_072": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_078 conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__, 2.29x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n oc = t % OC\n ic = t // OC\n src = ((ic * OC + oc) * KH + kh) * KW + kw\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((oc * IC + ic) * KH + fkh) * KW + fkw\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple,\n stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size,\n stride=stride, padding=padding, bias=bias)\n self.stride = stride\n self.padding = padding\n self.kernel_size = kernel_size\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.has_bias = bias\n\n def forward(self, x):\n stride_ok = (self.stride == (1, 1) or self.stride == 1)\n if stride_ok and x.is_cuda:\n w = self.conv_transpose2d.weight # (IC, OC, KH, KW)\n IC, OC, KH, KW = w.shape\n wf = torch.empty((OC, IC, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KH, KW, BLOCK=BLOCK)\n pad_h = KH - 1 - self.padding[0]\n pad_w = KW - 1 - self.padding[1]\n b = self.conv_transpose2d.bias if self.has_bias else None\n return torch.nn.functional.conv2d(x, wf, bias=b, stride=1,\n padding=(pad_h, pad_w))\n return self.conv_transpose2d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L1_061 conv_transposed_3D__square_input__square_kernel, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KD, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KD * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n kd = t % KD\n t = t // KD\n oc = t % OC\n ic = t // OC\n src = ((((ic * OC + oc) * KD + kd) * KH + kh) * KW + kw)\n fkd = KD - 1 - kd\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((((oc * IC + ic) * KD + fkd) * KH + fkh) * KW + fkw)\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n self.stride = stride\n self.padding = padding\n self.output_padding = output_padding\n self.groups = groups\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n simple = (self.stride == 1 and self.output_padding == 0 and self.groups == 1 and x.is_cuda)\n if simple:\n w = self.conv_transpose3d.weight # (IC, OC, KD, KH, KW)\n IC, OC, KD, KH, KW = w.shape\n wf = torch.empty((OC, IC, KD, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KD * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KD, KH, KW, BLOCK=BLOCK)\n p = KD - 1 - self.padding\n b = self.conv_transpose3d.bias if self.has_bias else None\n return F.conv3d(x, wf, bias=b, stride=1, padding=(p, p, p))\n return self.conv_transpose3d(x)\n```", "ctx_chars": 11175, "ctx_sha": "91e4a03c424736b2", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_3D_asymmetric_input_asymmetric_kernel___strided_padded_grouped_", "split": "learn", "n_ops": 10, "src_lines": 52, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 1 problem L1_072 (conv_transposed_3D_asymmetric_input_asymmetric_kernel___strided_padded_grouped_), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a 3D transposed convolution operation with asymmetric input and kernel, and optional stride.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple of ints): Size of the convolution kernel in the form (kernel_size_depth, kernel_size_height, kernel_size_width).\n stride (tuple of ints, optional): Stride of the convolution in the form (stride_depth, stride_height, stride_width). Defaults to (1, 1, 1).\n padding (tuple of ints, optional): Padding applied to the input in the form (padding_depth, padding_height, padding_width). Defaults to (0, 0, 0).\n output_padding (tuple of ints, optional): Additional size added to one side of the output shape. Defaults to (0, 0, 0).\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: tuple = (1, 1, 1), padding: tuple = (0, 0, 0), output_padding: tuple = (0, 0, 0), groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 3D transposed convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, depth, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, depth_out, height_out, width_out).\n \"\"\"\n return self.conv_transpose3d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 32\nout_channels = 32\nkernel_size = (3, 5, 7)\ndepth = 12\nheight = 24\nwidth = 48\nstride = (2, 2, 2)\npadding = (1, 2, 3)\noutput_padding = (1, 1, 1)\ngroups = 4\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, depth, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, output_padding, groups]```\n"}, "L1_077": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_061 conv_transposed_3D__square_input__square_kernel, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KD, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KD * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n kd = t % KD\n t = t // KD\n oc = t % OC\n ic = t // OC\n src = ((((ic * OC + oc) * KD + kd) * KH + kh) * KW + kw)\n fkd = KD - 1 - kd\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((((oc * IC + ic) * KD + fkd) * KH + fkh) * KW + fkw)\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n self.stride = stride\n self.padding = padding\n self.output_padding = output_padding\n self.groups = groups\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n simple = (self.stride == 1 and self.output_padding == 0 and self.groups == 1 and x.is_cuda)\n if simple:\n w = self.conv_transpose3d.weight # (IC, OC, KD, KH, KW)\n IC, OC, KD, KH, KW = w.shape\n wf = torch.empty((OC, IC, KD, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KD * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KD, KH, KW, BLOCK=BLOCK)\n p = KD - 1 - self.padding\n b = self.conv_transpose3d.bias if self.has_bias else None\n return F.conv3d(x, wf, bias=b, stride=1, padding=(p, p, p))\n return self.conv_transpose3d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L1_078 conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__, 2.29x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n oc = t % OC\n ic = t // OC\n src = ((ic * OC + oc) * KH + kh) * KW + kw\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((oc * IC + ic) * KH + fkh) * KW + fkw\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple,\n stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size,\n stride=stride, padding=padding, bias=bias)\n self.stride = stride\n self.padding = padding\n self.kernel_size = kernel_size\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.has_bias = bias\n\n def forward(self, x):\n stride_ok = (self.stride == (1, 1) or self.stride == 1)\n if stride_ok and x.is_cuda:\n w = self.conv_transpose2d.weight # (IC, OC, KH, KW)\n IC, OC, KH, KW = w.shape\n wf = torch.empty((OC, IC, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KH, KW, BLOCK=BLOCK)\n pad_h = KH - 1 - self.padding[0]\n pad_w = KW - 1 - self.padding[1]\n b = self.conv_transpose2d.bias if self.has_bias else None\n return torch.nn.functional.conv2d(x, wf, bias=b, stride=1,\n padding=(pad_h, pad_w))\n return self.conv_transpose2d(x)\n```", "ctx_chars": 11175, "ctx_sha": "5e031feda794c122", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_3D_square_input_square_kernel___padded____dilated____strided__", "split": "learn", "n_ops": 10, "src_lines": 51, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 1 problem L1_077 (conv_transposed_3D_square_input_square_kernel___padded____dilated____strided__), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a 3D transposed convolution operation with square input and square kernel,\n and supports padding, dilation, and stride.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (int): Size of the convolution kernel (square kernel, so only one value needed).\n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (int, optional): Padding applied to the input. Defaults to 0.\n dilation (int, optional): Spacing between kernel elements. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, dilation=dilation, bias=bias)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 3D transposed convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, depth, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, depth_out, height_out, width_out).\n \"\"\"\n return self.conv_transpose3d(x)\n\n# Test code\nbatch_size = 16\nin_channels = 32\nout_channels = 64\nkernel_size = 3\ndepth = 16\nheight = 32\nwidth = 32\nstride = 2\npadding = 1\ndilation = 2\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, depth, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, dilation]```\n"}, "L1_078": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L1_061 conv_transposed_3D__square_input__square_kernel, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef flip_kernel(w_ptr, o_ptr, IC, OC, KD, KH, KW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = IC * OC * KD * KH * KW\n mask = offs < total\n o = offs\n kw = o % KW\n t = o // KW\n kh = t % KH\n t = t // KH\n kd = t % KD\n t = t // KD\n oc = t % OC\n ic = t // OC\n src = ((((ic * OC + oc) * KD + kd) * KH + kh) * KW + kw)\n fkd = KD - 1 - kd\n fkh = KH - 1 - kh\n fkw = KW - 1 - kw\n dst = ((((oc * IC + ic) * KD + fkd) * KH + fkh) * KW + fkw)\n v = tl.load(w_ptr + src, mask=mask, other=0.0)\n tl.store(o_ptr + dst, v, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, output_padding: int = 0, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv_transpose3d = nn.ConvTranspose3d(in_channels, out_channels, kernel_size=(kernel_size, kernel_size, kernel_size), stride=stride, padding=padding, output_padding=output_padding, groups=groups, bias=bias)\n self.stride = stride\n self.padding = padding\n self.output_padding = output_padding\n self.groups = groups\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n simple = (self.stride == 1 and self.output_padding == 0 and self.groups == 1 and x.is_cuda)\n if simple:\n w = self.conv_transpose3d.weight # (IC, OC, KD, KH, KW)\n IC, OC, KD, KH, KW = w.shape\n wf = torch.empty((OC, IC, KD, KH, KW), device=w.device, dtype=w.dtype)\n total = IC * OC * KD * KH * KW\n BLOCK = 256\n flip_kernel[(triton.cdiv(total, BLOCK),)](\n w.contiguous(), wf, IC, OC, KD, KH, KW, BLOCK=BLOCK)\n p = KD - 1 - self.padding\n b = self.conv_transpose3d.bias if self.has_bias else None\n return F.conv3d(x, wf, bias=b, stride=1, padding=(p, p, p))\n return self.conv_transpose3d(x)\n```\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11582, "ctx_sha": "a74f6646d7158b72", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv_transpose", "level": 1, "name": "conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__", "split": "learn", "n_ops": 8, "src_lines": 47, "max_recipe_speedup": 1.68, "instruction": "KernelBench level 1 problem L1_078 (conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a 2D transposed convolution operation with asymmetric input and kernel, with optional padding.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Size of the convolution kernel (height, width).\n stride (tuple, optional): Stride of the convolution (height, width). Defaults to (1, 1).\n padding (tuple, optional): Padding applied to the input (height, width). Defaults to (0, 0).\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: tuple = (1, 1), padding: tuple = (0, 0), bias: bool = False):\n super(Model, self).__init__()\n self.conv_transpose2d = nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 2D transposed convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv_transpose2d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 32\nout_channels = 32\nkernel_size = (3, 7)\nheight = 512\nwidth = 1024\nstride = (1, 1)\npadding = (1, 3)\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding]```\n"}, "L1_080": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_050 conv_standard_2D__square_input__square_kernel, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_kernel(x_ptr, bias_ptr, N, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n tl.store(x_ptr + offs, v + b, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, num_classes=1000):\n super(ModelNew, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n\n def forward(self, x):\n out = F.conv2d(x, self.conv1.weight, None, stride=4, padding=2)\n out = out.contiguous()\n N, C, H, W = out.shape\n S = H * W\n TOTAL = N * C * S\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bias_kernel[grid](out, self.conv1.bias.contiguous(), N, C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_083 conv_depthwise_2D_square_input_asymmetric_kernel, 2.43x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K: tl.constexpr, HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw = ow * stride - padding\n ih0 = oh * stride - padding\n wvalid = mask & (iw >= 0) & (iw < W)\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * K\n\n for k in tl.static_range(K):\n ih = ih0 + k * dilation\n if (ih >= 0) & (ih < H):\n idx = base + ih.to(tl.int64) * W + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + k)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, kernel_size=(kernel_size, 1),\n stride=stride, padding=padding, dilation=dilation,\n groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.kernel_size = kernel_size\n self.stride = stride\n self.padding = padding\n self.dilation = dilation\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n K = self.kernel_size\n stride = self.stride\n padding = self.padding\n dilation = self.dilation\n OH = (H + 2 * padding - dilation * (K - 1) - 1) // stride + 1\n OW = (W + 2 * padding - 1) // stride + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 256\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K=K, HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```", "ctx_chars": 10864, "ctx_sha": "0b6222244e19e377", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_standard_2D_square_input_asymmetric_kernel___dilated____padded__", "split": "learn", "n_ops": 9, "src_lines": 49, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 1 problem L1_080 (conv_standard_2D_square_input_asymmetric_kernel___dilated____padded__), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a standard 2D convolution operation with square input and asymmetric kernel, with dilation and padding.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size (tuple): Size of the convolution kernel (height, width). \n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (tuple, optional): Padding applied to the input (top/bottom, left/right). Defaults to (0, 0).\n dilation (tuple, optional): Spacing between kernel elements (height, width). Defaults to (1, 1).\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size: tuple, stride: int = 1, padding: tuple = (0, 0), dilation: tuple = (1, 1), bias: bool = False):\n super(Model, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, dilation=dilation, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv2d(x)\n\n# Test code\nbatch_size = 8\nin_channels = 32\nout_channels = 64\nkernel_size = (5, 9)\nwidth = 512\nheight = 512\nstride = 1\npadding = (2, 4)\ndilation = (2, 3)\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, dilation]```\n"}, "L1_083": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_085 conv_depthwise_2D_asymmetric_input_asymmetric_kernel, 1.35x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride_h, stride_w, pad_h, pad_w, dil_h, dil_w,\n KH: tl.constexpr, KW: tl.constexpr,\n HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw0 = ow * stride_w - pad_w\n ih0 = oh * stride_h - pad_h\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * (KH * KW)\n\n for kh in tl.static_range(KH):\n ih = ih0 + kh * dil_h\n if (ih >= 0) & (ih < H):\n rbase = base + ih.to(tl.int64) * W\n for kw in tl.static_range(KW):\n iw = iw0 + kw * dil_w\n wvalid = mask & (iw >= 0) & (iw < W)\n idx = rbase + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + kh * KW + kw)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, out_channels: int, kernel_size_h: int, kernel_size_w: int,\n stride_h: int = 1, stride_w: int = 1, padding_h: int = 0, padding_w: int = 0,\n dilation_h: int = 1, dilation_w: int = 1, groups: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, (kernel_size_h, kernel_size_w),\n stride=(stride_h, stride_w), padding=(padding_h, padding_w),\n dilation=(dilation_h, dilation_w), groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.KH = kernel_size_h\n self.KW = kernel_size_w\n self.stride_h = stride_h\n self.stride_w = stride_w\n self.padding_h = padding_h\n self.padding_w = padding_w\n self.dilation_h = dilation_h\n self.dilation_w = dilation_w\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n KH, KW = self.KH, self.KW\n OH = (H + 2 * self.padding_h - self.dilation_h * (KH - 1) - 1) // self.stride_h + 1\n OW = (W + 2 * self.padding_w - self.dilation_w * (KW - 1) - 1) // self.stride_w + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 128\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n self.stride_h, self.stride_w, self.padding_h, self.padding_w,\n self.dilation_h, self.dilation_w,\n KH=KH, KW=KW,\n HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_050 conv_standard_2D__square_input__square_kernel, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_kernel(x_ptr, bias_ptr, N, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n tl.store(x_ptr + offs, v + b, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, num_classes=1000):\n super(ModelNew, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n\n def forward(self, x):\n out = F.conv2d(x, self.conv1.weight, None, stride=4, padding=2)\n out = out.contiguous()\n N, C, H, W = out.shape\n S = H * W\n TOTAL = N * C * S\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bias_kernel[grid](out, self.conv1.bias.contiguous(), N, C, S, TOTAL, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10362, "ctx_sha": "51d761e43339b5b6", "dropped_same_problem": 2, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 4, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_depthwise_2D_square_input_asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 47, "max_recipe_speedup": 2.37, "instruction": "KernelBench level 1 problem L1_083 (conv_depthwise_2D_square_input_asymmetric_kernel), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a depthwise 2D convolution with a square input and an asymmetric kernel.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n kernel_size (int): Size of the convolution kernel.\n stride (int, optional): Stride of the convolution. Defaults to 1.\n padding (int, optional): Padding applied to the input. Defaults to 0.\n dilation (int, optional): Spacing between kernel elements. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, kernel_size=(kernel_size, 1), stride=stride, padding=padding, dilation=dilation, groups=in_channels, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the depthwise 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, in_channels, height_out, width_out).\n \"\"\"\n return self.conv2d(x)\n\n# Test code\nbatch_size = 64\nin_channels = 8\nkernel_size = 3\nwidth = 512\nheight = 512\nstride = 1\npadding = 0\ndilation = 1\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, kernel_size, stride, padding, dilation]```\n"}, "L1_085": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L1_083 conv_depthwise_2D_square_input_asymmetric_kernel, 2.43x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef dwconv_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K: tl.constexpr, HAS_BIAS: tl.constexpr, BLOCK: tl.constexpr):\n pid_row = tl.program_id(0)\n pid_col = tl.program_id(1)\n\n oh = pid_row % OH\n tmp = pid_row // OH\n c = tmp % C\n n = tmp // C\n\n ow = pid_col * BLOCK + tl.arange(0, BLOCK)\n mask = ow < OW\n\n iw = ow * stride - padding\n ih0 = oh * stride - padding\n wvalid = mask & (iw >= 0) & (iw < W)\n\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n wbase = c * K\n\n for k in tl.static_range(K):\n ih = ih0 + k * dilation\n if (ih >= 0) & (ih < H):\n idx = base + ih.to(tl.int64) * W + iw\n v = tl.load(x_ptr + idx, mask=wvalid, other=0.0)\n wv = tl.load(w_ptr + wbase + k)\n acc += v * wv\n\n if HAS_BIAS:\n bv = tl.load(b_ptr + c)\n acc += bv\n\n obase = ((n.to(tl.int64) * C + c) * OH + oh) * OW\n tl.store(out_ptr + obase + ow, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, bias: bool = False):\n super(ModelNew, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, kernel_size=(kernel_size, 1),\n stride=stride, padding=padding, dilation=dilation,\n groups=in_channels, bias=bias)\n self.in_channels = in_channels\n self.kernel_size = kernel_size\n self.stride = stride\n self.padding = padding\n self.dilation = dilation\n self.has_bias = bias\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = x.contiguous()\n N, C, H, W = x.shape\n K = self.kernel_size\n stride = self.stride\n padding = self.padding\n dilation = self.dilation\n OH = (H + 2 * padding - dilation * (K - 1) - 1) // stride + 1\n OW = (W + 2 * padding - 1) // stride + 1\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n w = self.conv2d.weight.contiguous().view(-1)\n if self.has_bias:\n b = self.conv2d.bias.contiguous()\n else:\n b = torch.empty(1, device=x.device, dtype=x.dtype)\n BLOCK = 256\n grid = (N * C * OH, triton.cdiv(OW, BLOCK))\n dwconv_kernel[grid](x, w, b, out,\n N, C, H, W, OH, OW,\n stride, padding, dilation,\n K=K, HAS_BIAS=self.has_bias, BLOCK=BLOCK,\n num_warps=4)\n return out\n```\n\n### A verified kernel for a related conv problem (L1_050 conv_standard_2D__square_input__square_kernel, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_kernel(x_ptr, bias_ptr, N, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n tl.store(x_ptr + offs, v + b, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, num_classes=1000):\n super(ModelNew, self).__init__()\n self.conv1 = nn.Conv2d(in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=2)\n\n def forward(self, x):\n out = F.conv2d(x, self.conv1.weight, None, stride=4, padding=2)\n out = out.contiguous()\n N, C, H, W = out.shape\n S = H * W\n TOTAL = N * C * S\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bias_kernel[grid](out, self.conv1.bias.contiguous(), N, C, S, TOTAL, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10864, "ctx_sha": "46fefeb0053babb6", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 1, "name": "conv_depthwise_2D_asymmetric_input_asymmetric_kernel", "split": "learn", "n_ops": 7, "src_lines": 59, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 1 problem L1_085 (conv_depthwise_2D_asymmetric_input_asymmetric_kernel), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Performs a depthwise 2D convolution with asymmetric input and asymmetric kernel.\n\n Args:\n in_channels (int): Number of channels in the input tensor.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size_h (int): Height of the convolution kernel.\n kernel_size_w (int): Width of the convolution kernel.\n stride_h (int, optional): Stride of the convolution in height dimension. Defaults to 1.\n stride_w (int, optional): Stride of the convolution in width dimension. Defaults to 1.\n padding_h (int, optional): Padding applied to the input in height dimension. Defaults to 0.\n padding_w (int, optional): Padding applied to the input in width dimension. Defaults to 0.\n dilation_h (int, optional): Spacing between kernel elements in height dimension. Defaults to 1.\n dilation_w (int, optional): Spacing between kernel elements in width dimension. Defaults to 1.\n groups (int, optional): Number of blocked connections from input channels to output channels. Defaults to 1.\n bias (bool, optional): If `True`, adds a learnable bias to the output. Defaults to `False`.\n \"\"\"\n def __init__(self, in_channels: int, out_channels: int, kernel_size_h: int, kernel_size_w: int, stride_h: int = 1, stride_w: int = 1, padding_h: int = 0, padding_w: int = 0, dilation_h: int = 1, dilation_w: int = 1, groups: int = 1, bias: bool = False):\n super(Model, self).__init__()\n self.conv2d = nn.Conv2d(in_channels, in_channels, (kernel_size_h, kernel_size_w), stride=(stride_h, stride_w), padding=(padding_h, padding_w), dilation=(dilation_h, dilation_w), groups=in_channels, bias=bias)\n \n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Performs the depthwise 2D convolution.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height_out, width_out).\n \"\"\"\n return self.conv2d(x)\n\n# Test code\nbatch_size = 32\nin_channels = 128\nout_channels = 128\nkernel_size_h = 3\nkernel_size_w = 7\nwidth = 256\nheight = 128\nstride_h = 1\nstride_w = 1\npadding_h = 0\npadding_w = 0\ndilation_h = 1\ndilation_w = 1\ngroups = in_channels\n\ndef get_inputs():\n x = torch.rand(batch_size, in_channels, height, width)\n return [x]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size_h, kernel_size_w, stride_h, stride_w, padding_h, padding_w, dilation_h, dilation_w, groups]```\n"}, "L1_088": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 1, "name": "MinGPTNewGelu", "split": "learn", "n_ops": 1, "src_lines": 26, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 1 problem L1_088 (MinGPTNewGelu), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\n\n# From https://github.com/karpathy/minGPT/blob/master/mingpt/model.py\n\nclass Model(nn.Module):\n \"\"\"\n Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT).\n Reference: Gaussian Error Linear Units (GELU) paper: https://arxiv.org/abs/1606.08415\n \"\"\"\n def __init__(self):\n super(Model, self).__init__()\n \n def forward(self, x):\n return 0.5 * x * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (x + 0.044715 * torch.pow(x, 3.0))))\n\nbatch_size = 8192\ndim = 8192\n\ndef get_inputs():\n return [torch.rand(batch_size, dim)]\n\ndef get_init_inputs():\n return []```\n"}, "L2_003": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_078 ConvTranspose3d_Max_Max_Sum, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, OD, OH, OW, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n idx64 = idx.to(tl.int64)\n mask = idx < total\n\n ow = idx64 % OW\n t = idx64 // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n n = t // OD\n\n d0 = od * 6\n h0 = oh * 6\n w0 = ow * 6\n\n C64 = C.to(tl.int64)\n D64 = D.to(tl.int64)\n H64 = H.to(tl.int64)\n W64 = W.to(tl.int64)\n\n s = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n base_nc = (n * C64 + c) * D64\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for dz in tl.static_range(6):\n iz = d0 + dz\n for dy in tl.static_range(6):\n iy = h0 + dy\n for dx in tl.static_range(6):\n ix = w0 + dx\n off = ((base_nc + iz) * H64 + iy) * W64 + ix\n v = tl.load(x_ptr + off, mask=mask, other=-1e30)\n m = tl.maximum(m, v)\n s += m\n tl.store(out_ptr + idx64, s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n P1D, P1H, P1W = D // 2, H // 2, W // 2\n OD, OH, OW = P1D // 3, P1H // 3, P1W // 3\n out = torch.empty((N, 1, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * OD * OH * OW\n BLOCK = 128\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](c, out, N, C, D, H, W, OD, OH, OW, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11288, "ctx_sha": "b0db956aa86ee683", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU", "split": "learn", "n_ops": 5, "src_lines": 40, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_003 (ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, followed by a sum, layer normalization, average pooling, and GELU activation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, sum_weight, norm_shape, pool_kernel_size):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.sum_weight = nn.Parameter(torch.tensor(sum_weight))\n self.norm = nn.LayerNorm(norm_shape)\n self.avg_pool = nn.AvgPool3d(kernel_size=pool_kernel_size)\n self.gelu = nn.GELU()\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = x + self.sum_weight\n x = self.norm(x)\n x = self.avg_pool(x)\n x = self.gelu(x)\n return x\n\nbatch_size = 32\nin_channels = 32\nout_channels = 64\ndepth, height, width = 16, 32, 32\nkernel_size = (3, 3, 3)\nstride = (2, 2, 2)\npadding = (1, 1, 1)\noutput_padding = (1, 1, 1)\nsum_weight = 1.0\nnorm_shape = (out_channels,)\npool_kernel_size = (2, 2, 2)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, output_padding, sum_weight, norm_shape, pool_kernel_size]```\n"}, "L2_006": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_047 Conv3d_Mish_Tanh, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef mish_tanh_kernel(x_ptr, out_ptr, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < N\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(x))\n mish = x * tl.extra.cuda.libdevice.tanh(sp)\n out = tl.extra.cuda.libdevice.tanh(mish)\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N = x.numel()\n out = torch.empty_like(x)\n BLOCK = 1024\n grid = (triton.cdiv(N, BLOCK),)\n mish_tanh_kernel[grid](x, out, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_027 Conv3d_HardSwish_GroupNorm_Mean, 1.17x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef stats_kernel(x_ptr, chmean_ptr, gmean_ptr, gvar_ptr,\n C, S, G, CPG, eps, BLOCK: tl.constexpr):\n # one program per (b, group)\n pid = tl.program_id(0)\n b = pid // G\n g = pid % G\n group_base = (b.to(tl.int64) * C + g * CPG) * S\n gsum = 0.0\n gsumsq = 0.0\n for ci in range(CPG):\n c = g * CPG + ci\n base = (b.to(tl.int64) * C + c) * S\n csum = 0.0\n for off in range(0, S, BLOCK):\n idx = off + tl.arange(0, BLOCK)\n mask = idx < S\n v = tl.load(x_ptr + base + idx, mask=mask, other=0.0)\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n hsm = tl.where(mask, hs, 0.0)\n csum += tl.sum(hsm)\n gsumsq += tl.sum(hsm * hsm)\n gsum += csum\n tl.store(chmean_ptr + (b.to(tl.int64) * C + c), csum / S)\n n = CPG * S\n mean = gsum / n\n var = gsumsq / n - mean * mean\n tl.store(gmean_ptr + pid, mean)\n tl.store(gvar_ptr + pid, var)\n\n\n@triton.jit\ndef final_kernel(chmean_ptr, gmean_ptr, gvar_ptr, w_ptr, b_ptr, out_ptr,\n B, C, G, CPG, eps, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < B * C\n bidx = offs // C\n c = offs % C\n g = c // CPG\n gidx = bidx * G + g\n cm = tl.load(chmean_ptr + offs, mask=mask, other=0.0)\n gm = tl.load(gmean_ptr + gidx, mask=mask, other=0.0)\n gv = tl.load(gvar_ptr + gidx, mask=mask, other=0.0)\n w = tl.load(w_ptr + c, mask=mask, other=0.0)\n bb = tl.load(b_ptr + c, mask=mask, other=0.0)\n inv = 1.0 / tl.sqrt(gv + eps)\n res = (cm - gm) * inv * w + bb\n tl.store(out_ptr + offs, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, num_groups=4, bias=True):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, bias=bias)\n self.group_norm = nn.GroupNorm(num_groups, out_channels)\n self.num_groups = num_groups\n self.out_channels = out_channels\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n B, C, D, H, W = x.shape\n S = D * H * W\n G = self.num_groups\n CPG = C // G\n eps = 1e-5\n chmean = torch.empty((B, C), device=x.device, dtype=x.dtype)\n gmean = torch.empty((B * G,), device=x.device, dtype=x.dtype)\n gvar = torch.empty((B * G,), device=x.device, dtype=x.dtype)\n BLOCK = 1024\n stats_kernel[(B * G,)](x, chmean, gmean, gvar, C, S, G, CPG,\n eps, BLOCK=BLOCK)\n out = torch.empty((B, C), device=x.device, dtype=x.dtype)\n w = self.group_norm.weight.contiguous()\n b = self.group_norm.bias.contiguous()\n grid = (triton.cdiv(B * C, BLOCK),)\n final_kernel[grid](chmean, gmean, gvar, w, b, out, B, C, G, CPG,\n eps, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11065, "ctx_sha": "b967fe13455fdbd9", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Softmax_MaxPool_MaxPool", "split": "learn", "n_ops": 4, "src_lines": 38, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_006 (Conv3d_Softmax_MaxPool_MaxPool), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D convolution, applies Softmax, and performs two max pooling operations.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, pool_kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.pool1 = nn.MaxPool3d(pool_kernel_size)\n self.pool2 = nn.MaxPool3d(pool_kernel_size)\n\n def forward(self, x):\n \"\"\"\n Args:\n x: Input tensor of shape (batch_size, in_channels, depth, height, width)\n Returns:\n Output tensor of shape (batch_size, out_channels, depth', height', width') where depth', height', width' are the dimensions after pooling.\n \"\"\"\n x = self.conv(x)\n x = torch.softmax(x, dim=1)\n x = self.pool1(x)\n x = self.pool2(x)\n return x\n\nbatch_size = 128\nin_channels = 3\nout_channels = 16\ndepth, height, width = 16, 32, 32\nkernel_size = 3\npool_kernel_size = 2\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, pool_kernel_size]```\n"}, "L2_008": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_043 Conv3d_Max_LogSumExp_ReLU, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, Dp, Hp, Wp,\n NOUT, BLOCK: tl.constexpr, C_CONST: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NOUT\n npw = Hp * Wp\n nplane = Dp * npw\n n = offs // nplane\n rem = offs % nplane\n dp = rem // npw\n rem2 = rem % npw\n hp = rem2 // Wp\n wp = rem2 % Wp\n d0 = dp * 2\n h0 = hp * 2\n w0 = wp * 2\n HW = H * W\n S = D * HW\n nbase = n.to(tl.int64) * C * S\n\n run_max = tl.full((BLOCK,), -1e30, tl.float32)\n run_sum = tl.zeros((BLOCK,), tl.float32)\n for c in range(C_CONST):\n cbase = nbase + c * S\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for kd in range(2):\n for kh in range(2):\n for kw in range(2):\n off = cbase + (d0 + kd).to(tl.int64) * HW + (h0 + kh) * W + (w0 + kw)\n v = tl.load(x_ptr + off, mask=mask, other=-1e30)\n m = tl.maximum(m, v)\n new_max = tl.maximum(run_max, m)\n run_sum = run_sum * tl.exp(run_max - new_max) + tl.exp(m - new_max)\n run_max = new_max\n lse = run_max + tl.log(run_sum)\n lse = tl.maximum(lse, 0.0)\n tl.store(out_ptr + offs, lse, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.out_channels = out_channels\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, D, H, W = x.shape\n Dp, Hp, Wp = D // 2, H // 2, W // 2\n out = torch.empty((N, 1, Dp, Hp, Wp), device=x.device, dtype=x.dtype)\n NOUT = N * Dp * Hp * Wp\n BLOCK = 256\n grid = (triton.cdiv(NOUT, BLOCK),)\n fused_kernel[grid](x, out, N, C, D, H, W, Dp, Hp, Wp,\n NOUT, BLOCK=BLOCK, C_CONST=C)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_082 Conv2d_Tanh_Scaling_BiasAdd_Max, 1.94x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, bias_ptr, out_ptr,\n N, C, H, W, OH, OW, P, scaling,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n TOTAL = N * C * OH * OW\n mask = offs < TOTAL\n\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n for ph in range(P):\n for pw in range(P):\n h = h0 + ph\n w = w0 + pw\n idx = base + h.to(tl.int64) * W + w\n v = tl.load(x_ptr + idx, mask=mask, other=-1e30)\n t = tl.extra.cuda.libdevice.tanh(v)\n t = t * scaling + b\n acc = tl.maximum(acc, t)\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scaling_factor = scaling_factor\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.pool_kernel_size = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_kernel_size\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n TOTAL = N * C * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, self.bias.contiguous().view(-1), out,\n N, C, H, W, OH, OW, P, float(self.scaling_factor),\n BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11062, "ctx_sha": "2b5cd4f5c3cc851b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Divide_Max_GlobalAvgPool_BiasAdd_Sum", "split": "learn", "n_ops": 6, "src_lines": 41, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_008 (Conv3d_Divide_Max_GlobalAvgPool_BiasAdd_Sum), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D convolution, divides by a constant, applies max pooling,\n global average pooling, adds a bias term, and sums along a specific dimension.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, divisor, pool_size, bias_shape, sum_dim):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.divisor = divisor\n self.max_pool = nn.MaxPool3d(pool_size)\n self.global_avg_pool = nn.AdaptiveAvgPool3d((1, 1, 1))\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.sum_dim = sum_dim\n\n def forward(self, x):\n x = self.conv(x)\n x = x / self.divisor\n x = self.max_pool(x)\n x = self.global_avg_pool(x)\n x = x + self.bias\n x = torch.sum(x, dim=self.sum_dim)\n return x\n\nbatch_size = 128 \nin_channels = 8 \nout_channels = 16 \ndepth = 16; height = width = 64 \nkernel_size = (3, 3, 3)\ndivisor = 2.0\npool_size = (2, 2, 2)\nbias_shape = (out_channels, 1, 1, 1)\nsum_dim = 1\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, divisor, pool_size, bias_shape, sum_dim]```\n"}, "L2_011": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_015 ConvTranspose3d_BatchNorm_Subtract, 1.32x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef out_kernel(x_ptr, out_ptr, scale_ptr, smean_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n sc = tl.load(scale_ptr + pid)\n sm = tl.load(smean_ptr + pid)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n r = sc * (v - sm)\n tl.store(out_ptr + base + idx, r, mask=m)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias=True):\n super(ModelNew, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n total_sum = ssum_nc.sum(0)\n total_sq = ssq_nc.sum(0)\n M = N * S\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps) # (C,)\n\n scale_nc = scale_c.unsqueeze(0).expand(N, C).contiguous().view(N * C)\n smean_nc = (ssum_nc / S).contiguous().view(N * C)\n\n out = torch.empty_like(c2)\n out_kernel[(N * C,)](c2, out, scale_nc, smean_nc, S, BLOCK=1024)\n return out.view(N, C, D, H, W)\n```", "ctx_chars": 12352, "ctx_sha": "5533cb2a86547de2", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose2d_BatchNorm_Tanh_MaxPool_GroupNorm", "split": "learn", "n_ops": 5, "src_lines": 39, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_011 (ConvTranspose2d_BatchNorm_Tanh_MaxPool_GroupNorm), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a transposed convolution, batch normalization, tanh activation, max pooling, and group normalization.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, groups, num_groups):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm2d(out_channels)\n self.tanh = nn.Tanh()\n self.max_pool = nn.MaxPool2d(kernel_size=2, stride=2)\n self.group_norm = nn.GroupNorm(num_groups=num_groups, num_channels=out_channels)\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.batch_norm(x)\n x = self.tanh(x)\n x = self.max_pool(x)\n x = self.group_norm(x)\n return x\n\nbatch_size = 512\nin_channels = 64 \nout_channels = 128 \nheight = width = 2048 \nkernel_size = 5\nstride = 1 \npadding = 1\ngroups = 8\nnum_groups = 8\nheight, width = 32, 32\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, groups, num_groups]```\n"}, "L2_012": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_076 Gemm_Add_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bias_relu_kernel(x_ptr, bias_ptr, out_ptr, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n b = tl.load(bias_ptr + col, mask=mask, other=0.0)\n y = tl.maximum(v + b, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n z = torch.mm(x, self.gemm.weight.t())\n M, N = z.shape\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n bias_relu_kernel[grid](z, self.bias, out, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_062 Matmul_GroupNorm_LeakyReLU_Sum, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef gn_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, G, CG, eps, neg_slope,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n base = row * C + grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n s = tl.sum(v, axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n rstd = 1.0 / tl.sqrt(var + eps)\n coff = grp * CG + offs\n w = tl.load(w_ptr + coff, mask=mask, other=0.0)\n b = tl.load(b_ptr + coff, mask=mask, other=0.0)\n y = (v - mean) * rstd * w + b\n y = tl.where(y >= 0, y, y * neg_slope)\n y = y + y\n tl.store(out_ptr + base + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, num_groups, eps=1e-5, negative_slope=0.01):\n super().__init__()\n self.fc = nn.Linear(input_size, hidden_size)\n self.gn = nn.GroupNorm(num_groups=num_groups, num_channels=hidden_size, eps=eps)\n self.leaky_relu = nn.LeakyReLU(negative_slope=negative_slope)\n self.num_groups = num_groups\n self.hidden_size = hidden_size\n self.eps = eps\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = self.fc(x).contiguous()\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n gn_kernel[grid](z, self.gn.weight, self.gn.bias, out,\n N, C, G, CG, self.eps, self.negative_slope,\n BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9890, "ctx_sha": "8695a4ea849dcf73", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Multiply_LeakyReLU", "split": "learn", "n_ops": 3, "src_lines": 30, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_012 (Gemm_Multiply_LeakyReLU), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a Gemm, multiplies the result, and applies LeakyReLU.\n \"\"\"\n def __init__(self, in_features, out_features, multiplier, negative_slope):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.multiplier = multiplier\n self.leaky_relu = nn.LeakyReLU(negative_slope)\n\n def forward(self, x):\n x = self.gemm(x)\n x = x * self.multiplier\n x = self.leaky_relu(x)\n return x\n\nbatch_size = 1024\nin_features = 8192 \nout_features = 8192\nmultiplier = 2.0\nnegative_slope = 0.1\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, multiplier, negative_slope]```\n"}, "L2_015": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_077 ConvTranspose3d_Scale_BatchNorm_GlobalAvgPool, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor, eps=1e-5, momentum=0.1):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n self.batch_norm = nn.BatchNorm3d(out_channels, eps=eps, momentum=momentum)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n ssum = ssum.view(N, C)\n ssq = ssq.view(N, C)\n total_sum = ssum.sum(0)\n total_sq = ssq.sum(0)\n M = N * S\n scale = self.scale_factor\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n mean_s = scale * mean_c\n var_s = scale * scale * var_c\n inv = self.batch_norm.weight / torch.sqrt(var_s + self.batch_norm.eps)\n spatial_mean_s = scale * (ssum / S)\n out = inv.unsqueeze(0) * (spatial_mean_s - mean_s.unsqueeze(0)) + self.batch_norm.bias.unsqueeze(0)\n return out.view(N, C, 1, 1, 1)\n```", "ctx_chars": 11357, "ctx_sha": "ce81245bc1f59985", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_BatchNorm_Subtract", "split": "learn", "n_ops": 3, "src_lines": 31, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_015 (ConvTranspose3d_BatchNorm_Subtract), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A 3D convolutional transpose layer followed by Batch Normalization and subtraction.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias=True):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.batch_norm(x)\n x = x - torch.mean(x, dim=(2, 3, 4), keepdim=True) # Subtract mean along spatial dimensions\n return x\n\nbatch_size = 16\nin_channels = 16\nout_channels = 32\ndepth, height, width = 16, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding]```\n"}, "L2_018": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_062 Matmul_GroupNorm_LeakyReLU_Sum, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef gn_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, G, CG, eps, neg_slope,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n base = row * C + grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n s = tl.sum(v, axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n rstd = 1.0 / tl.sqrt(var + eps)\n coff = grp * CG + offs\n w = tl.load(w_ptr + coff, mask=mask, other=0.0)\n b = tl.load(b_ptr + coff, mask=mask, other=0.0)\n y = (v - mean) * rstd * w + b\n y = tl.where(y >= 0, y, y * neg_slope)\n y = y + y\n tl.store(out_ptr + base + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, num_groups, eps=1e-5, negative_slope=0.01):\n super().__init__()\n self.fc = nn.Linear(input_size, hidden_size)\n self.gn = nn.GroupNorm(num_groups=num_groups, num_channels=hidden_size, eps=eps)\n self.leaky_relu = nn.LeakyReLU(negative_slope=negative_slope)\n self.num_groups = num_groups\n self.hidden_size = hidden_size\n self.eps = eps\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = self.fc(x).contiguous()\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n gn_kernel[grid](z, self.gn.weight, self.gn.bias, out,\n N, C, G, CG, self.eps, self.negative_slope,\n BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9898, "ctx_sha": "eb48d1e5953cd4b8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Sum_Max_AvgPool_LogSumExp_LogSumExp", "split": "learn", "n_ops": 6, "src_lines": 41, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_018 (Matmul_Sum_Max_AvgPool_LogSumExp_LogSumExp), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a sequence of operations:\n - Matrix multiplication\n - Summation\n - Max\n - Average pooling\n - LogSumExp\n - LogSumExp\n \"\"\"\n def __init__(self, in_features, out_features):\n super(Model, self).__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, 1).\n \"\"\"\n x = self.linear(x) # (batch_size, out_features)\n x = torch.sum(x, dim=1, keepdim=True) # (batch_size, 1)\n x = torch.max(x, dim=1, keepdim=True)[0] # (batch_size, 1)\n x = torch.mean(x, dim=1, keepdim=True) # (batch_size, 1)\n x = torch.logsumexp(x, dim=1, keepdim=True) # (batch_size, 1)\n x = torch.logsumexp(x, dim=1, keepdim=True) # (batch_size, 1)\n return x\n\nbatch_size = 1024\nin_features = 8192 \nout_features = 8192\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features]```\n"}, "L2_024": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_006 Conv3d_Softmax_MaxPool_MaxPool, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, OD, OH, OW,\n BLOCK_C: tl.constexpr, P: tl.constexpr):\n pid = tl.program_id(0)\n ow = pid % OW\n tmp = pid // OW\n oh = tmp % OH\n tmp = tmp // OH\n od = tmp % OD\n n = tmp // OD\n\n c = tl.arange(0, BLOCK_C)\n cmask = c < C\n maxv = tl.full((BLOCK_C,), -3.4e38, tl.float32)\n\n n64 = n.to(tl.int64)\n for kd in range(P):\n for kh in range(P):\n for kw in range(P):\n id_ = od * P + kd\n ih = oh * P + kh\n iw = ow * P + kw\n base = ((n64 * C + c.to(tl.int64)) * D + id_) * H\n base = (base + ih) * W + iw\n v = tl.load(x_ptr + base, mask=cmask, other=-3.4e38)\n m = tl.max(v)\n e = tl.exp(v - m)\n ssum = tl.sum(tl.where(cmask, e, 0.0))\n sv = e / ssum\n maxv = tl.maximum(maxv, sv)\n\n obase = ((n64 * C + c.to(tl.int64)) * OD + od) * OH\n obase = (obase + oh) * OW + ow\n tl.store(out_ptr + obase, maxv, mask=cmask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.pool_kernel_size = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, D, H, W = x.shape\n P = self.pool_kernel_size * self.pool_kernel_size\n OD = D // P\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OD, OH, OW), device=x.device, dtype=x.dtype)\n BLOCK_C = 1\n while BLOCK_C < C:\n BLOCK_C *= 2\n grid = (N * OD * OH * OW,)\n fused_kernel[grid](x, out, N, C, D, H, W, OD, OH, OW,\n BLOCK_C=BLOCK_C, P=P)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_032 Conv2d_Scaling_Min, 1.10x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef scale_min_kernel(x_ptr, out_ptr, scale, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = acc * scale\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n scale_min_kernel[grid](x, out, float(self.scale_factor), C, S, NS, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10177, "ctx_sha": "cf1072ae0bde8a1b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Min_Softmax", "split": "learn", "n_ops": 3, "src_lines": 37, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_024 (Conv3d_Min_Softmax), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a 3D convolution, applies minimum operation along a specific dimension, \n and then applies softmax.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, dim):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.dim = dim\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, D, H, W)\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, H, W)\n \"\"\"\n x = self.conv(x)\n x = torch.min(x, dim=self.dim)[0] # Apply minimum along the specified dimension\n x = torch.softmax(x, dim=1) # Apply softmax along the channel dimension\n return x\n\nbatch_size = 128\nin_channels = 3\nout_channels = 24 # Increased output channels\nD, H, W = 24, 32, 32 # Increased depth\nkernel_size = 3\ndim = 2 # Dimension along which to apply minimum operation (e.g., depth)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, D, H, W)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, dim]```\n"}, "L2_025": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_032 Conv2d_Scaling_Min, 1.10x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef scale_min_kernel(x_ptr, out_ptr, scale, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = acc * scale\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n scale_min_kernel[grid](x, out, float(self.scale_factor), C, S, NS, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_046 Conv2d_Subtract_Tanh_Subtract_AvgPool, 1.40x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr,\n N, C, H, W, OH, OW, P, s1, s2, inv,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n TOTAL = N * C * OH * OW\n mask = offs < TOTAL\n\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n h0 = oh * P\n w0 = ow * P\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n for ph in range(P):\n for pw in range(P):\n h = h0 + ph\n w = w0 + pw\n idx = base + h.to(tl.int64) * W + w\n v = tl.load(x_ptr + idx, mask=mask, other=0.0)\n t = tl.extra.cuda.libdevice.tanh(v - s1) - s2\n acc += t\n acc = acc * inv\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, subtract1_value, subtract2_value, kernel_size_pool):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract1_value = subtract1_value\n self.subtract2_value = subtract2_value\n self.kernel_size_pool = kernel_size_pool\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.kernel_size_pool\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n TOTAL = N * C * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, out, N, C, H, W, OH, OW, P,\n float(self.subtract1_value), float(self.subtract2_value),\n 1.0 / (P * P), BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10087, "ctx_sha": "fc8a2930809fd512", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Min_Tanh_Tanh", "split": "learn", "n_ops": 4, "src_lines": 29, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_025 (Conv2d_Min_Tanh_Tanh), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a convolution, applies minimum operation, Tanh, and another Tanh.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x)\n x = torch.min(x, dim=1, keepdim=True)[0] # Apply minimum operation along the channel dimension\n x = torch.tanh(x)\n x = torch.tanh(x)\n return x\n\nbatch_size = 128\nin_channels = 16\nout_channels = 64\nheight = width = 256\nkernel_size = 3\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_027": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_092 Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp, 2.37x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\nfrom triton.language.extra.cuda import libdevice\n\n\n@triton.jit\ndef gn_stats_kernel(x_ptr, mean_ptr, invstd_ptr, CPG, S, GS, eps, BLOCK: tl.constexpr):\n pid = tl.program_id(0) # over n*groups\n base = pid.to(tl.int64) * GS\n acc = 0.0\n acc2 = 0.0\n for off in range(0, GS, BLOCK):\n offs = off + tl.arange(0, BLOCK)\n mask = offs < GS\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n acc += tl.sum(v, axis=0)\n acc2 += tl.sum(v * v, axis=0)\n mean = acc / GS\n var = acc2 / GS - mean * mean\n invstd = 1.0 / tl.sqrt(var + eps)\n tl.store(mean_ptr + pid, mean)\n tl.store(invstd_ptr + pid, invstd)\n\n\n@triton.jit\ndef fused_kernel(x_ptr, mean_ptr, invstd_ptr, gamma_ptr, beta_ptr, out_ptr,\n N, C, S, NS, G, CPG, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n maxv = tl.full((BLOCK,), -3.4e38, tl.float32)\n for c in range(C):\n g = c // CPG\n ng = n * G + g\n xc = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + ng, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + ng, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + c)\n beta = tl.load(beta_ptr + c)\n norm = (xc - mean) * invstd * gamma + beta\n t = libdevice.tanh(norm)\n hs = t * tl.minimum(tl.maximum(t + 3.0, 0.0), 6.0) / 6.0\n res = xc + hs\n maxv = tl.maximum(maxv, res)\n sumexp = tl.zeros((BLOCK,), tl.float32)\n for c in range(C):\n g = c // CPG\n ng = n * G + g\n xc = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + ng, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + ng, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + c)\n beta = tl.load(beta_ptr + c)\n norm = (xc - mean) * invstd * gamma + beta\n t = libdevice.tanh(norm)\n hs = t * tl.minimum(tl.maximum(t + 3.0, 0.0), 6.0) / 6.0\n res = xc + hs\n sumexp += tl.exp(res - maxv)\n out = maxv + tl.log(sumexp)\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, groups, eps=1e-5):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.group_norm = nn.GroupNorm(groups, out_channels, eps=eps)\n self.tanh = nn.Tanh()\n self.hard_swish = nn.Hardswish()\n self.groups = groups\n self.eps = eps\n\n def forward(self, x):\n x_conv = self.conv(x).contiguous()\n N, C, H, W = x_conv.shape\n S = H * W\n G = self.groups\n CPG = C // G\n GS = CPG * S\n mean = torch.empty((N * G,), device=x_conv.device, dtype=torch.float32)\n invstd = torch.empty((N * G,), device=x_conv.device, dtype=torch.float32)\n gn_stats_kernel[(N * G,)](x_conv, mean, invstd, CPG, S, GS, self.eps, BLOCK=1024)\n out = torch.empty((N, 1, H, W), device=x_conv.device, dtype=x_conv.dtype)\n NS = N * S\n BLOCK = 256\n grid = (triton.cdiv(NS, BLOCK),)\n fused_kernel[grid](x_conv, mean, invstd,\n self.group_norm.weight, self.group_norm.bias,\n out, N, C, S, NS, G, CPG, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_069 Conv2d_HardSwish_ReLU, 1.25x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef act_kernel(x_ptr, out_ptr, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n # hardswish: x * clamp(x+3,0,6)/6\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n r = tl.maximum(hs, 0.0)\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n out = self.conv(x).contiguous()\n TOTAL = out.numel()\n res = torch.empty_like(out)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n act_kernel[grid](out, res, TOTAL, BLOCK=BLOCK)\n return res\n```", "ctx_chars": 11535, "ctx_sha": "114ec077101d70e2", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_HardSwish_GroupNorm_Mean", "split": "learn", "n_ops": 4, "src_lines": 36, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_027 (Conv3d_HardSwish_GroupNorm_Mean), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs:\n 1. Conv3D\n 2. HardSwish activation\n 3. GroupNorm \n 4. Mean pooling across spatial dimensions\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, num_groups=4, bias=True):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, bias=bias)\n self.group_norm = nn.GroupNorm(num_groups, out_channels)\n\n def forward(self, x):\n x = self.conv(x) # (B, C, D, H, W)\n x = F.hardswish(x) # Nonlinear activation\n x = self.group_norm(x) # Normalization over channels\n x = torch.mean(x, dim=[2, 3, 4]) # Mean over spatial dims \u2192 (B, C)\n return x\n\n# === Test config ===\nbatch_size = 1024\nin_channels = 3\nout_channels = 16\ndepth, height, width = 16, 32, 32\nkernel_size = 4\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_028": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Verified recipes for norm problems (speedups measured by the protected evaluator)\n- [3.98x on LayerNorm] For LayerNorm/RMSNorm over huge rows (few rows), split into two Triton kernels so the GPU stays saturated. First, a per-row stats kernel loops over the row in BLOCK=4096 chunks, accumulates sum and sum-of-squares in fp32, and stores mean and rstd. Second, a flat elementwise apply kernel (BLOCK=1024, grid=cdiv(numel,BLOCK)) computes row=off//N and col=off%N, loads mean/rstd/weight/bias, and writes (x-mean)*rstd*w+b. Keep the parameters in nn.LayerNorm and reuse its eps. Use contiguous views and fp32 accumulation.\n\n### A verified kernel for a related norm problem (L1_040 LayerNorm, 3.98x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef ln_stats_kernel(x_ptr, mean_ptr, rstd_ptr, N, eps,\n BLOCK: tl.constexpr):\n row = tl.program_id(0)\n x_ptr += row * N\n sum_ = 0.0\n sumsq = 0.0\n for off in range(0, N, BLOCK):\n idx = off + tl.arange(0, BLOCK)\n mask = idx < N\n v = tl.load(x_ptr + idx, mask=mask, other=0.0).to(tl.float32)\n sum_ += tl.sum(v, axis=0)\n sumsq += tl.sum(v * v, axis=0)\n mean = sum_ / N\n var = sumsq / N - mean * mean\n rstd = 1.0 / tl.sqrt(var + eps)\n tl.store(mean_ptr + row, mean)\n tl.store(rstd_ptr + row, rstd)\n\n@triton.jit\ndef ln_apply_kernel(x_ptr, w_ptr, b_ptr, mean_ptr, rstd_ptr, out_ptr,\n N, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n off = pid * BLOCK + tl.arange(0, BLOCK)\n mask = off < total\n row = off // N\n col = off % N\n x = tl.load(x_ptr + off, mask=mask, other=0.0).to(tl.float32)\n mean = tl.load(mean_ptr + row, mask=mask, other=0.0)\n rstd = tl.load(rstd_ptr + row, mask=mask, other=0.0)\n w = tl.load(w_ptr + col, mask=mask, other=0.0).to(tl.float32)\n b = tl.load(b_ptr + col, mask=mask, other=0.0).to(tl.float32)\n y = (x - mean) * rstd * w + b\n tl.store(out_ptr + off, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, normalized_shape):\n super().__init__()\n self.ln = nn.LayerNorm(normalized_shape=normalized_shape)\n self.eps = self.ln.eps\n\n def forward(self, x):\n x = x.contiguous()\n ns = 1\n for s in self.ln.normalized_shape:\n ns *= s\n rows = x.numel() // ns\n N = ns\n mean = torch.empty(rows, device=x.device, dtype=torch.float32)\n rstd = torch.empty(rows, device=x.device, dtype=torch.float32)\n xf = x.view(rows, N)\n ln_stats_kernel[(rows,)](xf, mean, rstd, N, self.eps, BLOCK=4096)\n out = torch.empty_like(x)\n w = self.ln.weight.contiguous().view(-1)\n b = self.ln.bias.contiguous().view(-1)\n total = x.numel()\n BLOCK = 1024\n grid = (triton.cdiv(total, BLOCK),)\n ln_apply_kernel[grid](x.view(-1), w, b, mean, rstd, out.view(-1),\n N, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 5026, "ctx_sha": "9bd1380e13dcea35", "dropped_same_problem": 1, "kinds": {"lesson": 14, "recipe": 1, "kernel": 1}, "family": "norm", "level": 2, "name": "BMM_InstanceNorm_Sum_ResidualAdd_Multiply", "split": "learn", "n_ops": 5, "src_lines": 36, "max_recipe_speedup": 3.98, "instruction": "KernelBench level 2 problem L2_028 (BMM_InstanceNorm_Sum_ResidualAdd_Multiply), op family: norm.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a batch matrix multiplication, instance normalization, summation, residual addition, and multiplication.\n \"\"\"\n def __init__(self, in_features, out_features, eps=1e-5, momentum=0.1):\n super(Model, self).__init__()\n self.bmm = nn.Linear(in_features, out_features)\n self.instance_norm = nn.InstanceNorm2d(out_features, eps=eps, momentum=momentum)\n\n def forward(self, x, y):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n y (torch.Tensor): Input tensor of shape (batch_size, out_features).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.bmm(x)\n x = self.instance_norm(x.unsqueeze(1).unsqueeze(1)).squeeze(1).squeeze(1)\n x = x + y\n x = x * y\n return x\n\nbatch_size = 1024 # Increased batch size\nin_features = 8192 # Increased input features\nout_features = 8192 # Increased output features\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features), torch.rand(batch_size, out_features)]\n\ndef get_init_inputs():\n return [in_features, out_features]```\n"}, "L2_029": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_059 Matmul_Swish_Scaling, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef swish_scale_kernel(x_ptr, out_ptr, scale, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = v * s * scale\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = F.linear(x, self.matmul.weight, self.matmul.bias).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n swish_scale_kernel[grid](z, out, float(self.scaling_factor), n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9028, "ctx_sha": "2f70ede7a41b13f4", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Mish_Mish", "split": "learn", "n_ops": 3, "src_lines": 26, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_029 (Matmul_Mish_Mish), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, applies Mish, and applies Mish again.\n \"\"\"\n def __init__(self, in_features, out_features):\n super(Model, self).__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n x = self.linear(x)\n x = torch.nn.functional.mish(x)\n x = torch.nn.functional.mish(x)\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features]```\n"}, "L2_032": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_073 Conv2d_BatchNorm_Scaling, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef stats_kernel2(x_ptr, sum_ptr, sumsq_ptr, N, C, S, SBLK: tl.constexpr):\n c = tl.program_id(0)\n sblk = tl.program_id(1)\n offs = sblk * SBLK + tl.arange(0, SBLK)\n smask = offs < S\n acc_sum = tl.zeros((SBLK,), tl.float32)\n acc_sq = tl.zeros((SBLK,), tl.float32)\n cS = c.to(tl.int64) * S\n NCS = C * S\n for n in range(N):\n base = n.to(tl.int64) * NCS + cS + offs\n v = tl.load(x_ptr + base, mask=smask, other=0.0)\n acc_sum += v\n acc_sq += v * v\n s = tl.sum(acc_sum, axis=0)\n sq = tl.sum(acc_sq, axis=0)\n tl.atomic_add(sum_ptr + c, s)\n tl.atomic_add(sumsq_ptr + c, sq)\n\n\n@triton.jit\ndef bn_apply_kernel(x_ptr, out_ptr, mean_ptr, invstd_ptr, w_ptr, b_ptr,\n scaling, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + c, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + c, mask=mask, other=0.0)\n w = tl.load(w_ptr + c, mask=mask, other=0.0)\n b = tl.load(b_ptr + c, mask=mask, other=0.0)\n r = ((v - mean) * invstd * w + b) * scaling\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n count = N * S\n device = x.device\n\n sums = torch.zeros(C, device=device, dtype=torch.float32)\n sumsq = torch.zeros(C, device=device, dtype=torch.float32)\n SBLK = 1024\n nsblk = triton.cdiv(S, SBLK)\n stats_kernel2[(C, nsblk)](x, sums, sumsq, N, C, S, SBLK=SBLK)\n\n if self.training:\n mean = sums / count\n var = sumsq / count - mean * mean\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn.momentum).add_(self.bn.momentum * mean.detach())\n unbiased = var * count / (count - 1)\n self.bn.running_var.mul_(1 - self.bn.momentum).add_(self.bn.momentum * unbiased.detach())\n else:\n mean = self.bn.running_mean\n var = self.bn.running_var\n\n invstd = torch.rsqrt(var + self.bn.eps)\n w = self.bn.weight\n b = self.bn.bias\n\n out = torch.empty_like(x)\n TOTAL = x.numel()\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bn_apply_kernel[grid](x, out, mean.contiguous(), invstd.contiguous(),\n w.contiguous(), b.contiguous(),\n float(self.scaling_factor), C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_025 Conv2d_Min_Tanh_Tanh, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef min_tanh_kernel(x_ptr, out_ptr, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = tl.minimum(tl.maximum(acc, -1e30), 1e30)\n t = tl.extra.cuda.libdevice.tanh(acc)\n t = tl.extra.cuda.libdevice.tanh(t)\n tl.store(out_ptr + offs, t, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n min_tanh_kernel[grid](x, out, C, S, NS, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11296, "ctx_sha": "2d2e82dae56fa580", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Scaling_Min", "split": "learn", "n_ops": 3, "src_lines": 36, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_032 (Conv2d_Scaling_Min), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a convolution, scales the output, and then applies a minimum operation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height, width).\n \"\"\"\n x = self.conv(x)\n x = x * self.scale_factor\n x = torch.min(x, dim=1, keepdim=True)[0] # Minimum along channel dimension\n return x\n\nbatch_size = 64\nin_channels = 64\nout_channels = 128\nheight = width = 256\nkernel_size = 3\nscale_factor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, scale_factor]```\n"}, "L2_035": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_069 Conv2d_HardSwish_ReLU, 1.25x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef act_kernel(x_ptr, out_ptr, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n # hardswish: x * clamp(x+3,0,6)/6\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n r = tl.maximum(hs, 0.0)\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n out = self.conv(x).contiguous()\n TOTAL = out.numel()\n res = torch.empty_like(out)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n act_kernel[grid](out, res, TOTAL, BLOCK=BLOCK)\n return res\n```\n\n### A verified kernel for a related conv problem (L2_046 Conv2d_Subtract_Tanh_Subtract_AvgPool, 1.40x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr,\n N, C, H, W, OH, OW, P, s1, s2, inv,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n TOTAL = N * C * OH * OW\n mask = offs < TOTAL\n\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n h0 = oh * P\n w0 = ow * P\n acc = tl.zeros((BLOCK,), tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n for ph in range(P):\n for pw in range(P):\n h = h0 + ph\n w = w0 + pw\n idx = base + h.to(tl.int64) * W + w\n v = tl.load(x_ptr + idx, mask=mask, other=0.0)\n t = tl.extra.cuda.libdevice.tanh(v - s1) - s2\n acc += t\n acc = acc * inv\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, subtract1_value, subtract2_value, kernel_size_pool):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract1_value = subtract1_value\n self.subtract2_value = subtract2_value\n self.kernel_size_pool = kernel_size_pool\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.kernel_size_pool\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n TOTAL = N * C * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, out, N, C, H, W, OH, OW, P,\n float(self.subtract1_value), float(self.subtract2_value),\n 1.0 / (P * P), BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9835, "ctx_sha": "6aedb10d18ef2e36", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Subtract_HardSwish_MaxPool_Mish", "split": "learn", "n_ops": 5, "src_lines": 34, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_035 (Conv2d_Subtract_HardSwish_MaxPool_Mish), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a convolution, subtracts a value, applies HardSwish, MaxPool, and Mish activation functions.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, subtract_value, pool_kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract_value = subtract_value\n self.pool = nn.MaxPool2d(pool_kernel_size)\n\n def forward(self, x):\n x = self.conv(x)\n x = x - self.subtract_value\n x = torch.nn.functional.hardswish(x)\n x = self.pool(x)\n x = torch.nn.functional.mish(x)\n return x\n\nbatch_size = 128\nin_channels = 64\nout_channels = 128\nheight = width = 128\nkernel_size = 3\nsubtract_value = 0.5\npool_kernel_size = 2\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, subtract_value, pool_kernel_size]```\n"}, "L2_039": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_041 Gemm_BatchNorm_GELU_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr,\n eps, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n # gelu\n g = 0.5 * y * (1.0 + tl.erf(y * 0.7071067811865476))\n # relu\n g = tl.maximum(g, 0.0)\n tl.store(out_ptr + offs, g, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.batch_norm = nn.BatchNorm1d(out_features)\n self.eps = 1e-5\n self.momentum = 0.1\n\n def forward(self, x):\n z = self.gemm(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=2048)\n\n if self.training:\n with torch.no_grad():\n self.batch_norm.running_mean.mul_(1 - self.momentum).add_(self.momentum * mean)\n unbiased = var * (M / (M - 1))\n self.batch_norm.running_var.mul_(1 - self.momentum).add_(self.momentum * unbiased)\n self.batch_norm.num_batches_tracked.add_(1)\n use_mean, use_var = mean, var\n else:\n use_mean = self.batch_norm.running_mean\n use_var = self.batch_norm.running_var\n\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, use_mean, use_var, self.batch_norm.weight,\n self.batch_norm.bias, self.eps, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_076 Gemm_Add_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bias_relu_kernel(x_ptr, bias_ptr, out_ptr, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n b = tl.load(bias_ptr + col, mask=mask, other=0.0)\n y = tl.maximum(v + b, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n z = torch.mm(x, self.gemm.weight.t())\n M, N = z.shape\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n bias_relu_kernel[grid](z, self.bias, out, M, N, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10823, "ctx_sha": "20b7935cf56f91ca", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Scale_BatchNorm", "split": "learn", "n_ops": 3, "src_lines": 29, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_039 (Gemm_Scale_BatchNorm), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, scales the result, and applies batch normalization.\n \"\"\"\n def __init__(self, in_features, out_features, scale_shape, eps=1e-5, momentum=0.1):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scale = nn.Parameter(torch.randn(scale_shape))\n self.bn = nn.BatchNorm1d(out_features, eps=eps, momentum=momentum)\n\n def forward(self, x):\n x = self.gemm(x)\n x = x * self.scale\n x = self.bn(x)\n return x\n\nbatch_size = 16384\nin_features = 4096\nout_features = 4096\nscale_shape = (out_features,)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, scale_shape]```\n"}, "L2_041": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_076 Gemm_Add_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bias_relu_kernel(x_ptr, bias_ptr, out_ptr, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n b = tl.load(bias_ptr + col, mask=mask, other=0.0)\n y = tl.maximum(v + b, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n z = torch.mm(x, self.gemm.weight.t())\n M, N = z.shape\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n bias_relu_kernel[grid](z, self.bias, out, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_053 Gemm_Scaling_Hardtanh_GELU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, hmin, hmax, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask)\n v = v * scaling\n v = tl.minimum(tl.maximum(v, hmin), hmax)\n g = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, g, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n self.hardtanh_min = hardtanh_min\n self.hardtanh_max = hardtanh_max\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor,\n float(self.hardtanh_min), float(self.hardtanh_max),\n n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9266, "ctx_sha": "9abf22a26a6d328a", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_BatchNorm_GELU_ReLU", "split": "learn", "n_ops": 4, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_041 (Gemm_BatchNorm_GELU_ReLU), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a GEMM, BatchNorm, GELU, and ReLU in sequence.\n \"\"\"\n def __init__(self, in_features, out_features):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.batch_norm = nn.BatchNorm1d(out_features)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.gemm(x)\n x = self.batch_norm(x)\n x = torch.nn.functional.gelu(x)\n x = torch.relu(x)\n return x\n\nbatch_size = 16384\nin_features = 4096\nout_features = 4096\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features]```\n"}, "L2_043": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_079 Conv3d_Multiply_InstanceNorm_Clamp_Multiply_Max, 1.32x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef norm_kernel(x_ptr, out_ptr, mult_ptr, S, C,\n clamp_min, clamp_max, eps, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n c = pid % C\n base = pid.to(tl.int64) * S\n m = tl.load(mult_ptr + c)\n sum_ = 0.0\n sumsq = 0.0\n for off in range(0, S, BLOCK):\n idx = off + tl.arange(0, BLOCK)\n mask = idx < S\n v = tl.load(x_ptr + base + idx, mask=mask, other=0.0) * m\n sum_ += tl.sum(tl.where(mask, v, 0.0))\n sumsq += tl.sum(tl.where(mask, v * v, 0.0))\n mean = sum_ / S\n var = sumsq / S - mean * mean\n inv = 1.0 / tl.sqrt(var + eps)\n for off in range(0, S, BLOCK):\n idx = off + tl.arange(0, BLOCK)\n mask = idx < S\n v = tl.load(x_ptr + base + idx, mask=mask, other=0.0) * m\n nrm = (v - mean) * inv\n nrm = tl.minimum(tl.maximum(nrm, clamp_min), clamp_max) * m\n tl.store(out_ptr + base + idx, nrm, mask=mask)\n\n\n@triton.jit\ndef max_kernel(x_ptr, out_ptr, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=-1e30)\n acc = tl.maximum(acc, v)\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, multiplier_shape, clamp_min, clamp_max):\n super().__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.multiplier = nn.Parameter(torch.randn(multiplier_shape))\n self.instance_norm = nn.InstanceNorm3d(out_channels)\n self.clamp_min = clamp_min\n self.clamp_max = clamp_max\n self.out_channels = out_channels\n\n def forward(self, x):\n x = self.conv(x)\n N, C, D, H, W = x.shape\n S = D * H * W\n x = x.contiguous()\n mult = self.multiplier.reshape(-1).contiguous()\n tmp = torch.empty_like(x)\n BLOCK = 1024\n norm_kernel[(N * C,)](x, tmp, mult, S, C,\n float(self.clamp_min), float(self.clamp_max),\n 1e-5, BLOCK=BLOCK)\n out = torch.empty((N, D, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n grid = (triton.cdiv(NS, BLOCK),)\n max_kernel[grid](tmp, out, C, S, NS, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_008 Conv3d_Divide_Max_GlobalAvgPool_BiasAdd_Sum, 1.07x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, C, D, H, W, Dp, Hp, Wp, pd, ph, pw,\n inv_div, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0) # (n*C + c)\n base = pid.to(tl.int64) * S\n nwin = Dp * Hp * Wp\n acc = 0.0\n for wstart in range(0, nwin, BLOCK):\n widx = wstart + tl.arange(0, BLOCK)\n wmask = widx < nwin\n dwi = widx // (Hp * Wp)\n rem = widx % (Hp * Wp)\n hwi = rem // Wp\n wwi = rem % Wp\n d0 = dwi * pd\n h0 = hwi * ph\n w0 = wwi * pw\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for kd in range(pd):\n for kh in range(ph):\n for kw in range(pw):\n dd = d0 + kd\n hh = h0 + kh\n ww = w0 + kw\n off = dd.to(tl.int64) * (H * W) + hh * W + ww\n v = tl.load(x_ptr + base + off, mask=wmask, other=-1e30)\n m = tl.maximum(m, v)\n acc += tl.sum(tl.where(wmask, m, 0.0))\n avg = acc * inv_div / nwin\n tl.store(out_ptr + pid, avg)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, divisor, pool_size, bias_shape, sum_dim):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.divisor = divisor\n self.pool_size = pool_size\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.sum_dim = sum_dim\n self.out_channels = out_channels\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, D, H, W = x.shape\n pd, ph, pw = self.pool_size\n Dp, Hp, Wp = D // pd, H // ph, W // pw\n S = D * H * W\n perc = torch.empty((N * C,), device=x.device, dtype=x.dtype)\n BLOCK = 256\n pool_kernel[(N * C,)](x, perc, C, D, H, W, Dp, Hp, Wp, pd, ph, pw,\n 1.0 / self.divisor, S, BLOCK=BLOCK)\n perc = perc.view(N, C)\n bias_sum = self.bias.view(-1).sum()\n out = perc.sum(dim=1) + bias_sum\n return out.view(N, 1, 1, 1)\n```", "ctx_chars": 11823, "ctx_sha": "8cbf999f2b258061", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Max_LogSumExp_ReLU", "split": "learn", "n_ops": 4, "src_lines": 38, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_043 (Conv3d_Max_LogSumExp_ReLU), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D convolution, max pooling, log sum exp, and ReLU activation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.max_pool = nn.MaxPool3d(kernel_size=2, stride=2)\n\n def forward(self, x):\n \"\"\"\n Args:\n x: Input tensor of shape (batch_size, in_channels, depth, height, width)\n Returns:\n Output tensor of shape (batch_size, out_channels, depth', height', width')\n \"\"\"\n x = self.conv(x)\n x = self.max_pool(x)\n x = torch.logsumexp(x, dim=1, keepdim=True)\n x = torch.relu(x)\n return x\n\nbatch_size = 4\nin_channels = 32\nout_channels = 64\ndepth, height, width = 32, 128, 128\nkernel_size = 3\nstride = 1\npadding = 1\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding]```\n"}, "L2_046": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_025 Conv2d_Min_Tanh_Tanh, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef min_tanh_kernel(x_ptr, out_ptr, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = tl.minimum(tl.maximum(acc, -1e30), 1e30)\n t = tl.extra.cuda.libdevice.tanh(acc)\n t = tl.extra.cuda.libdevice.tanh(t)\n tl.store(out_ptr + offs, t, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n min_tanh_kernel[grid](x, out, C, S, NS, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_082 Conv2d_Tanh_Scaling_BiasAdd_Max, 1.94x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, bias_ptr, out_ptr,\n N, C, H, W, OH, OW, P, scaling,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n TOTAL = N * C * OH * OW\n mask = offs < TOTAL\n\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n for ph in range(P):\n for pw in range(P):\n h = h0 + ph\n w = w0 + pw\n idx = base + h.to(tl.int64) * W + w\n v = tl.load(x_ptr + idx, mask=mask, other=-1e30)\n t = tl.extra.cuda.libdevice.tanh(v)\n t = t * scaling + b\n acc = tl.maximum(acc, t)\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scaling_factor = scaling_factor\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.pool_kernel_size = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_kernel_size\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n TOTAL = N * C * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, self.bias.contiguous().view(-1), out,\n N, C, H, W, OH, OW, P, float(self.scaling_factor),\n BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10210, "ctx_sha": "ee72d31f6aad351d", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Subtract_Tanh_Subtract_AvgPool", "split": "learn", "n_ops": 5, "src_lines": 36, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_046 (Conv2d_Subtract_Tanh_Subtract_AvgPool), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a convolution, subtraction, tanh activation, subtraction and average pooling.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, subtract1_value, subtract2_value, kernel_size_pool):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract1_value = subtract1_value\n self.subtract2_value = subtract2_value\n self.avgpool = nn.AvgPool2d(kernel_size_pool)\n\n def forward(self, x):\n x = self.conv(x)\n x = x - self.subtract1_value\n x = torch.tanh(x)\n x = x - self.subtract2_value\n x = self.avgpool(x)\n return x\n\nbatch_size = 128\nin_channels = 64\nout_channels = 128\nheight, width = 128, 128\nkernel_size = 3\nsubtract1_value = 0.5\nsubtract2_value = 0.2\nkernel_size_pool = 2\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, subtract1_value, subtract2_value, kernel_size_pool]```\n"}, "L2_047": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_048 Conv3d_Scaling_Tanh_Multiply_Sigmoid, 1.33x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, scale_ptr, bias_ptr, out_ptr,\n TOTAL, C, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n t = tl.extra.cuda.libdevice.tanh(x * s)\n t = t * b\n out = 1.0 / (1.0 + tl.exp(-t))\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.scaling_factor = nn.Parameter(torch.randn(bias_shape))\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, D, H, W = x.shape\n S = D * H * W\n TOTAL = x.numel()\n out = torch.empty_like(x)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, self.scaling_factor.contiguous().view(-1),\n self.bias.contiguous().view(-1), out,\n TOTAL, C, S, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_025 Conv2d_Min_Tanh_Tanh, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef min_tanh_kernel(x_ptr, out_ptr, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = tl.minimum(tl.maximum(acc, -1e30), 1e30)\n t = tl.extra.cuda.libdevice.tanh(acc)\n t = tl.extra.cuda.libdevice.tanh(t)\n tl.store(out_ptr + offs, t, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n min_tanh_kernel[grid](x, out, C, S, NS, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9656, "ctx_sha": "797896dbdcfce662", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Mish_Tanh", "split": "learn", "n_ops": 3, "src_lines": 35, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_047 (Conv3d_Mish_Tanh), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D convolution, applies Mish activation, and then applies Tanh activation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, D, H, W).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, D', H', W').\n \"\"\"\n x = self.conv(x)\n x = torch.nn.functional.mish(x)\n x = torch.tanh(x)\n return x\n\nbatch_size = 16\nin_channels = 32\nout_channels = 64\nD, H, W = 32, 64, 64\nkernel_size = 3\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, D, H, W)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_048": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_047 Conv3d_Mish_Tanh, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef mish_tanh_kernel(x_ptr, out_ptr, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < N\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(x))\n mish = x * tl.extra.cuda.libdevice.tanh(sp)\n out = tl.extra.cuda.libdevice.tanh(mish)\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N = x.numel()\n out = torch.empty_like(x)\n BLOCK = 1024\n grid = (triton.cdiv(N, BLOCK),)\n mish_tanh_kernel[grid](x, out, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_082 Conv2d_Tanh_Scaling_BiasAdd_Max, 1.94x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, bias_ptr, out_ptr,\n N, C, H, W, OH, OW, P, scaling,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n TOTAL = N * C * OH * OW\n mask = offs < TOTAL\n\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n for ph in range(P):\n for pw in range(P):\n h = h0 + ph\n w = w0 + pw\n idx = base + h.to(tl.int64) * W + w\n v = tl.load(x_ptr + idx, mask=mask, other=-1e30)\n t = tl.extra.cuda.libdevice.tanh(v)\n t = t * scaling + b\n acc = tl.maximum(acc, t)\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scaling_factor = scaling_factor\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.pool_kernel_size = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_kernel_size\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n TOTAL = N * C * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, self.bias.contiguous().view(-1), out,\n N, C, H, W, OH, OW, P, float(self.scaling_factor),\n BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9970, "ctx_sha": "c573808a484b68ec", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Scaling_Tanh_Multiply_Sigmoid", "split": "learn", "n_ops": 5, "src_lines": 34, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_048 (Conv3d_Scaling_Tanh_Multiply_Sigmoid), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D convolution, scales the output, applies tanh, multiplies by a scaling factor, and applies sigmoid.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.scaling_factor = nn.Parameter(torch.randn(bias_shape))\n self.bias = nn.Parameter(torch.randn(bias_shape)) \n\n def forward(self, x):\n x = self.conv(x)\n x = x * self.scaling_factor \n x = torch.tanh(x)\n x = x * self.bias\n x = torch.sigmoid(x)\n return x\n\nbatch_size = 128\nin_channels = 3\nout_channels = 16\ndepth, height, width = 16, 64, 64\nkernel_size = 3\nscaling_factor = 2\nbias_shape = (out_channels, 1, 1, 1)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, scaling_factor, bias_shape]```\n"}, "L2_049": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_089 ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max, 1.06x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, sub_ptr, out_ptr, N, C, S, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n mask = idx < total\n\n # idx indexes (n, s) flat where s in [0,S)\n n = idx // S\n s = idx % S\n base = n * C * S + s\n\n # pass 1: max over channels\n mx = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=-1e30)\n mx = tl.maximum(mx, v)\n\n # pass 2: sum of exp\n sm = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n sm += tl.exp(v - mx)\n\n # pass 3: softmax, subtract, swish, max over channels\n res = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n sub = tl.load(sub_ptr + c)\n sm_val = tl.exp(v - mx) / sm\n y = sm_val - sub\n swish = (tl.sigmoid(y) * y)\n res = tl.maximum(res, swish)\n\n tl.store(out_ptr + idx, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, pool_kernel_size, pool_stride, pool_padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.max_pool = nn.MaxPool3d(kernel_size=pool_kernel_size, stride=pool_stride, padding=pool_padding)\n self.subtract = nn.Parameter(torch.randn(out_channels))\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.max_pool(x).contiguous()\n N, C, D, H, W = x.shape\n S = D * H * W\n total = N * S\n\n out = torch.empty((N, D, H, W), device=x.device, dtype=x.dtype)\n sub = self.subtract.contiguous()\n\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](x, sub, out, N, C, S, total, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 12155, "ctx_sha": "1323e4e600b1300c", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_Softmax_Sigmoid", "split": "learn", "n_ops": 3, "src_lines": 40, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_049 (ConvTranspose3d_Softmax_Sigmoid), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, applies Softmax and Sigmoid.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, bias=True):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding, bias=bias)\n self.softmax = nn.Softmax(dim=1)\n self.sigmoid = nn.Sigmoid()\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, D, H, W).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, D, H, W).\n \"\"\"\n x = self.conv_transpose(x)\n x = self.softmax(x)\n x = self.sigmoid(x)\n return x\n\nbatch_size = 16\nin_channels = 32\nout_channels = 64\nD, H, W = 16, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\noutput_padding = 1\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, D, H, W)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, output_padding]```\n"}, "L2_051": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_070 Gemm_Sigmoid_Scaling_ResidualAdd, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, n_elements, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elements\n v = tl.load(x_ptr + offs, mask=mask)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = s * scaling + v\n tl.store(out_ptr + offs, y, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, scaling_factor):\n super().__init__()\n self.gemm = nn.Linear(input_size, hidden_size)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_053 Gemm_Scaling_Hardtanh_GELU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, hmin, hmax, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask)\n v = v * scaling\n v = tl.minimum(tl.maximum(v, hmin), hmax)\n g = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, g, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n self.hardtanh_min = hardtanh_min\n self.hardtanh_max = hardtanh_max\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor,\n float(self.hardtanh_min), float(self.hardtanh_max),\n n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 8570, "ctx_sha": "d68a795eb2ebd40b", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 5, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd", "split": "learn", "n_ops": 6, "src_lines": 43, "max_recipe_speedup": 4.58, "instruction": "KernelBench level 2 problem L2_051 (Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a series of operations: Gemm, Subtract, GlobalAvgPool, LogSumExp, GELU, and ResidualAdd.\n \"\"\"\n def __init__(self, in_features, out_features, bias=True):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n\n def forward(self, x):\n original_x = x.clone().detach()\n # Gemm\n x = self.gemm(x)\n\n # Subtract\n x = x - self.subtract\n\n # GlobalAvgPool\n x = torch.mean(x, dim=1, keepdim=True)\n\n # LogSumExp\n x = torch.logsumexp(x, dim=1, keepdim=True)\n\n # GELU\n x = torch.nn.functional.gelu(x)\n\n # ResidualAdd\n x = x + original_x\n\n return x\n\nbatch_size = 2048\nin_features = 8192\nout_features = 8192\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features]```\n"}, "L2_052": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_073 Conv2d_BatchNorm_Scaling, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef stats_kernel2(x_ptr, sum_ptr, sumsq_ptr, N, C, S, SBLK: tl.constexpr):\n c = tl.program_id(0)\n sblk = tl.program_id(1)\n offs = sblk * SBLK + tl.arange(0, SBLK)\n smask = offs < S\n acc_sum = tl.zeros((SBLK,), tl.float32)\n acc_sq = tl.zeros((SBLK,), tl.float32)\n cS = c.to(tl.int64) * S\n NCS = C * S\n for n in range(N):\n base = n.to(tl.int64) * NCS + cS + offs\n v = tl.load(x_ptr + base, mask=smask, other=0.0)\n acc_sum += v\n acc_sq += v * v\n s = tl.sum(acc_sum, axis=0)\n sq = tl.sum(acc_sq, axis=0)\n tl.atomic_add(sum_ptr + c, s)\n tl.atomic_add(sumsq_ptr + c, sq)\n\n\n@triton.jit\ndef bn_apply_kernel(x_ptr, out_ptr, mean_ptr, invstd_ptr, w_ptr, b_ptr,\n scaling, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + c, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + c, mask=mask, other=0.0)\n w = tl.load(w_ptr + c, mask=mask, other=0.0)\n b = tl.load(b_ptr + c, mask=mask, other=0.0)\n r = ((v - mean) * invstd * w + b) * scaling\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n count = N * S\n device = x.device\n\n sums = torch.zeros(C, device=device, dtype=torch.float32)\n sumsq = torch.zeros(C, device=device, dtype=torch.float32)\n SBLK = 1024\n nsblk = triton.cdiv(S, SBLK)\n stats_kernel2[(C, nsblk)](x, sums, sumsq, N, C, S, SBLK=SBLK)\n\n if self.training:\n mean = sums / count\n var = sumsq / count - mean * mean\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn.momentum).add_(self.bn.momentum * mean.detach())\n unbiased = var * count / (count - 1)\n self.bn.running_var.mul_(1 - self.bn.momentum).add_(self.bn.momentum * unbiased.detach())\n else:\n mean = self.bn.running_mean\n var = self.bn.running_var\n\n invstd = torch.rsqrt(var + self.bn.eps)\n w = self.bn.weight\n b = self.bn.bias\n\n out = torch.empty_like(x)\n TOTAL = x.numel()\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bn_apply_kernel[grid](x, out, mean.contiguous(), invstd.contiguous(),\n w.contiguous(), b.contiguous(),\n float(self.scaling_factor), C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_067 Conv2d_GELU_GlobalAvgPool, 1.30x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\nfrom triton.language.extra import libdevice\n\n\n@triton.jit\ndef gelu_avg_kernel(x_ptr, out_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n base = pid.to(tl.int64) * S\n acc = 0.0\n for off in range(0, S, BLOCK):\n offs = off + tl.arange(0, BLOCK)\n mask = offs < S\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n g = 0.5 * v * (1.0 + libdevice.erf(v * 0.7071067811865476))\n acc += tl.sum(g, axis=0)\n tl.store(out_ptr + pid, acc / S)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, C), device=x.device, dtype=x.dtype)\n grid = (N * C,)\n gelu_avg_kernel[grid](x, out, S, BLOCK=1024)\n return out\n```", "ctx_chars": 11132, "ctx_sha": "4caa9b6e3a0949e1", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Activation_BatchNorm", "split": "learn", "n_ops": 3, "src_lines": 29, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_052 (Conv2d_Activation_BatchNorm), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a convolution, applies activation, and then applies Batch Normalization.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, eps=1e-5, momentum=0.1):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels, eps=eps, momentum=momentum)\n\n def forward(self, x):\n x = self.conv(x)\n x = torch.multiply(torch.tanh(torch.nn.functional.softplus(x)), x)\n x = self.bn(x)\n return x\n\nbatch_size = 64\nin_channels = 64\nout_channels = 128\nheight, width = 128, 128\nkernel_size = 3\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_053": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_070 Gemm_Sigmoid_Scaling_ResidualAdd, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, n_elements, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elements\n v = tl.load(x_ptr + offs, mask=mask)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = s * scaling + v\n tl.store(out_ptr + offs, y, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, scaling_factor):\n super().__init__()\n self.gemm = nn.Linear(input_size, hidden_size)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_041 Gemm_BatchNorm_GELU_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr,\n eps, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n # gelu\n g = 0.5 * y * (1.0 + tl.erf(y * 0.7071067811865476))\n # relu\n g = tl.maximum(g, 0.0)\n tl.store(out_ptr + offs, g, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.batch_norm = nn.BatchNorm1d(out_features)\n self.eps = 1e-5\n self.momentum = 0.1\n\n def forward(self, x):\n z = self.gemm(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=2048)\n\n if self.training:\n with torch.no_grad():\n self.batch_norm.running_mean.mul_(1 - self.momentum).add_(self.momentum * mean)\n unbiased = var * (M / (M - 1))\n self.batch_norm.running_var.mul_(1 - self.momentum).add_(self.momentum * unbiased)\n self.batch_norm.num_batches_tracked.add_(1)\n use_mean, use_var = mean, var\n else:\n use_mean = self.batch_norm.running_mean\n use_var = self.batch_norm.running_var\n\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, use_mean, use_var, self.batch_norm.weight,\n self.batch_norm.bias, self.eps, M, N, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10749, "ctx_sha": "3f4b70562dfa8638", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Scaling_Hardtanh_GELU", "split": "learn", "n_ops": 4, "src_lines": 33, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_053 (Gemm_Scaling_Hardtanh_GELU), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a GEMM, scaling, hardtanh, and GELU activation.\n \"\"\"\n def __init__(self, in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n self.hardtanh = nn.Hardtanh(min_val=hardtanh_min, max_val=hardtanh_max)\n self.gelu = nn.GELU()\n\n def forward(self, x):\n x = self.gemm(x)\n x = x * self.scaling_factor\n x = self.hardtanh(x)\n x = self.gelu(x)\n return x\n\nbatch_size = 2048\nin_features = 8192\nout_features = 8192\nscaling_factor = 0.5\nhardtanh_min = -2\nhardtanh_max = 2\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max]```\n"}, "L2_055": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_062 Matmul_GroupNorm_LeakyReLU_Sum, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef gn_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, G, CG, eps, neg_slope,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n base = row * C + grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n s = tl.sum(v, axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n rstd = 1.0 / tl.sqrt(var + eps)\n coff = grp * CG + offs\n w = tl.load(w_ptr + coff, mask=mask, other=0.0)\n b = tl.load(b_ptr + coff, mask=mask, other=0.0)\n y = (v - mean) * rstd * w + b\n y = tl.where(y >= 0, y, y * neg_slope)\n y = y + y\n tl.store(out_ptr + base + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, num_groups, eps=1e-5, negative_slope=0.01):\n super().__init__()\n self.fc = nn.Linear(input_size, hidden_size)\n self.gn = nn.GroupNorm(num_groups=num_groups, num_channels=hidden_size, eps=eps)\n self.leaky_relu = nn.LeakyReLU(negative_slope=negative_slope)\n self.num_groups = num_groups\n self.hidden_size = hidden_size\n self.eps = eps\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = self.fc(x).contiguous()\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n gn_kernel[grid](z, self.gn.weight, self.gn.bias, out,\n N, C, G, CG, self.eps, self.negative_slope,\n BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9898, "ctx_sha": "eb48d1e5953cd4b8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_MaxPool_Sum_Scale", "split": "learn", "n_ops": 4, "src_lines": 38, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_055 (Matmul_MaxPool_Sum_Scale), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs matrix multiplication, max pooling, sum, and scaling.\n \"\"\"\n def __init__(self, in_features, out_features, kernel_size, scale_factor):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.max_pool = nn.MaxPool1d(kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.matmul(x)\n x = self.max_pool(x.unsqueeze(1)).squeeze(1)\n x = torch.sum(x, dim=1)\n x = x * self.scale_factor\n return x\n\nbatch_size = 128\nin_features = 32768\nout_features = 32768\nkernel_size = 2\nscale_factor = 0.5\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, kernel_size, scale_factor]```\n"}, "L2_056": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_062 Matmul_GroupNorm_LeakyReLU_Sum, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef gn_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, G, CG, eps, neg_slope,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n base = row * C + grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n s = tl.sum(v, axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n rstd = 1.0 / tl.sqrt(var + eps)\n coff = grp * CG + offs\n w = tl.load(w_ptr + coff, mask=mask, other=0.0)\n b = tl.load(b_ptr + coff, mask=mask, other=0.0)\n y = (v - mean) * rstd * w + b\n y = tl.where(y >= 0, y, y * neg_slope)\n y = y + y\n tl.store(out_ptr + base + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, num_groups, eps=1e-5, negative_slope=0.01):\n super().__init__()\n self.fc = nn.Linear(input_size, hidden_size)\n self.gn = nn.GroupNorm(num_groups=num_groups, num_channels=hidden_size, eps=eps)\n self.leaky_relu = nn.LeakyReLU(negative_slope=negative_slope)\n self.num_groups = num_groups\n self.hidden_size = hidden_size\n self.eps = eps\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = self.fc(x).contiguous()\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n gn_kernel[grid](z, self.gn.weight, self.gn.bias, out,\n N, C, G, CG, self.eps, self.negative_slope,\n BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9898, "ctx_sha": "eb48d1e5953cd4b8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Sigmoid_Sum", "split": "learn", "n_ops": 3, "src_lines": 33, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_056 (Matmul_Sigmoid_Sum), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, applies sigmoid, and sums the result.\n \"\"\"\n def __init__(self, input_size, hidden_size):\n super(Model, self).__init__()\n self.linear = nn.Linear(input_size, hidden_size)\n\n def forward(self, x):\n \"\"\"\n Args:\n x: Input tensor of shape (batch_size, input_size).\n\n Returns:\n Output tensor of shape (batch_size, 1).\n \"\"\"\n x = self.linear(x)\n x = torch.sigmoid(x)\n x = torch.sum(x, dim=1, keepdim=True)\n return x\n\nbatch_size = 128\ninput_size = 32768\nhidden_size = 32768\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size)]\n\ndef get_init_inputs():\n return [input_size, hidden_size]```\n"}, "L2_057": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_069 Conv2d_HardSwish_ReLU, 1.25x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef act_kernel(x_ptr, out_ptr, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n # hardswish: x * clamp(x+3,0,6)/6\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n r = tl.maximum(hs, 0.0)\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n out = self.conv(x).contiguous()\n TOTAL = out.numel()\n res = torch.empty_like(out)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n act_kernel[grid](out, res, TOTAL, BLOCK=BLOCK)\n return res\n```\n\n### A verified kernel for a related conv problem (L2_035 Conv2d_Subtract_HardSwish_MaxPool_Mish, 1.27x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, sub, N, C, H, W, OH, OW, P, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n base = (n.to(tl.int64) * C + c) * H * W\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -3.4e38, tl.float32)\n for ph in range(0, P):\n for pw in range(0, P):\n ih = h0 + ph\n iw = w0 + pw\n vmask = mask & (ih < H) & (iw < W)\n v = tl.load(x_ptr + base + ih.to(tl.int64) * W + iw, mask=vmask, other=-3.4e38)\n v = v - sub\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n acc = tl.maximum(acc, hs)\n # mish: x * tanh(softplus(x))\n sp = tl.log(1.0 + tl.exp(acc))\n th = (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n res = acc * th\n tl.store(out_ptr + offs, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, subtract_value, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract_value = subtract_value\n self.pool_k = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_k\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=torch.float32)\n TOTAL = out.numel()\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, out, float(self.subtract_value),\n N, C, H, W, OH, OW, P, TOTAL, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9855, "ctx_sha": "1a6a80cca09cee48", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_ReLU_HardSwish", "split": "learn", "n_ops": 3, "src_lines": 28, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_057 (Conv2d_ReLU_HardSwish), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a convolution, applies ReLU, and applies HardSwish activation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x)\n x = torch.relu(x)\n x = x * torch.clamp((x + 3) / 6, 0, 1)\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 128, 128\nkernel_size = 3\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_059": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_097 Matmul_BatchNorm_BiasAdd_Divide_Swish, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr, bias_ptr,\n eps, divide_value, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n bias = tl.load(bias_ptr)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n y = (y + bias) / divide_value\n y = y * (1.0 / (1.0 + tl.exp(-y)))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bn_eps=1e-5, bn_momentum=0.1, bias_shape=(1,), divide_value=1.0):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.bn = nn.BatchNorm1d(out_features, eps=bn_eps, momentum=bn_momentum)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.divide_value = divide_value\n self.bn_eps = bn_eps\n self.bn_momentum = bn_momentum\n\n def forward(self, x):\n z = self.matmul(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=1024)\n\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn_momentum).add_(self.bn_momentum * mean)\n unbiased = var * (M / (M - 1))\n self.bn.running_var.mul_(1 - self.bn_momentum).add_(self.bn_momentum * unbiased)\n self.bn.num_batches_tracked.add_(1)\n\n out = torch.empty_like(z)\n bias_flat = self.bias.reshape(-1)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, mean, var, self.bn.weight, self.bn.bias, bias_flat,\n self.bn_eps, self.divide_value, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10909, "ctx_sha": "65e4e510dcc4381c", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Swish_Scaling", "split": "learn", "n_ops": 3, "src_lines": 28, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_059 (Matmul_Swish_Scaling), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, applies Swish activation, and scales the result.\n \"\"\"\n def __init__(self, in_features, out_features, scaling_factor):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.matmul(x)\n x = x * torch.sigmoid(x) # Swish activation\n x = x * self.scaling_factor\n return x\n\nbatch_size = 128\nin_features = 32768\nout_features = 32768\nscaling_factor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, scaling_factor]```\n"}, "L2_062": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 8462, "ctx_sha": "7424a06df403acf6", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 5, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_GroupNorm_LeakyReLU_Sum", "split": "learn", "n_ops": 4, "src_lines": 40, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_062 (Matmul_GroupNorm_LeakyReLU_Sum), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a matrix multiplication, group normalization, leaky ReLU activation, and element-wise sum.\n \"\"\"\n def __init__(self, input_size, hidden_size, num_groups, eps=1e-5, negative_slope=0.01):\n super(Model, self).__init__()\n self.fc = nn.Linear(input_size, hidden_size)\n self.gn = nn.GroupNorm(num_groups=num_groups, num_channels=hidden_size, eps=eps)\n self.leaky_relu = nn.LeakyReLU(negative_slope=negative_slope)\n\n def forward(self, x):\n \"\"\"\n Performs the forward pass of the model.\n\n Args:\n x: Input tensor of shape (batch_size, input_size).\n\n Returns:\n Output tensor of shape (batch_size, hidden_size).\n \"\"\"\n x = self.fc(x)\n x = self.gn(x)\n x = self.leaky_relu(x)\n x = x + x\n return x\n\n\nbatch_size = 1024\ninput_size = 8192\nhidden_size = 8192\nnum_groups = 512\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size)]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_groups]```\n"}, "L2_063": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_076 Gemm_Add_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bias_relu_kernel(x_ptr, bias_ptr, out_ptr, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n b = tl.load(bias_ptr + col, mask=mask, other=0.0)\n y = tl.maximum(v + b, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n z = torch.mm(x, self.gemm.weight.t())\n M, N = z.shape\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n bias_relu_kernel[grid](z, self.bias, out, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_041 Gemm_BatchNorm_GELU_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr,\n eps, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n # gelu\n g = 0.5 * y * (1.0 + tl.erf(y * 0.7071067811865476))\n # relu\n g = tl.maximum(g, 0.0)\n tl.store(out_ptr + offs, g, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.batch_norm = nn.BatchNorm1d(out_features)\n self.eps = 1e-5\n self.momentum = 0.1\n\n def forward(self, x):\n z = self.gemm(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=2048)\n\n if self.training:\n with torch.no_grad():\n self.batch_norm.running_mean.mul_(1 - self.momentum).add_(self.momentum * mean)\n unbiased = var * (M / (M - 1))\n self.batch_norm.running_var.mul_(1 - self.momentum).add_(self.momentum * unbiased)\n self.batch_norm.num_batches_tracked.add_(1)\n use_mean, use_var = mean, var\n else:\n use_mean = self.batch_norm.running_mean\n use_var = self.batch_norm.running_var\n\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, use_mean, use_var, self.batch_norm.weight,\n self.batch_norm.bias, self.eps, M, N, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10823, "ctx_sha": "819fb7af68fa10a5", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_ReLU_Divide", "split": "learn", "n_ops": 3, "src_lines": 28, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_063 (Gemm_ReLU_Divide), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, applies ReLU, and divides by a constant.\n \"\"\"\n def __init__(self, in_features, out_features, divisor):\n super(Model, self).__init__()\n self.linear = nn.Linear(in_features, out_features)\n self.divisor = divisor\n\n def forward(self, x):\n x = self.linear(x)\n x = torch.relu(x)\n x = x / self.divisor\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\ndivisor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, divisor]```\n"}, "L2_067": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_073 Conv2d_BatchNorm_Scaling, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef stats_kernel2(x_ptr, sum_ptr, sumsq_ptr, N, C, S, SBLK: tl.constexpr):\n c = tl.program_id(0)\n sblk = tl.program_id(1)\n offs = sblk * SBLK + tl.arange(0, SBLK)\n smask = offs < S\n acc_sum = tl.zeros((SBLK,), tl.float32)\n acc_sq = tl.zeros((SBLK,), tl.float32)\n cS = c.to(tl.int64) * S\n NCS = C * S\n for n in range(N):\n base = n.to(tl.int64) * NCS + cS + offs\n v = tl.load(x_ptr + base, mask=smask, other=0.0)\n acc_sum += v\n acc_sq += v * v\n s = tl.sum(acc_sum, axis=0)\n sq = tl.sum(acc_sq, axis=0)\n tl.atomic_add(sum_ptr + c, s)\n tl.atomic_add(sumsq_ptr + c, sq)\n\n\n@triton.jit\ndef bn_apply_kernel(x_ptr, out_ptr, mean_ptr, invstd_ptr, w_ptr, b_ptr,\n scaling, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + c, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + c, mask=mask, other=0.0)\n w = tl.load(w_ptr + c, mask=mask, other=0.0)\n b = tl.load(b_ptr + c, mask=mask, other=0.0)\n r = ((v - mean) * invstd * w + b) * scaling\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n count = N * S\n device = x.device\n\n sums = torch.zeros(C, device=device, dtype=torch.float32)\n sumsq = torch.zeros(C, device=device, dtype=torch.float32)\n SBLK = 1024\n nsblk = triton.cdiv(S, SBLK)\n stats_kernel2[(C, nsblk)](x, sums, sumsq, N, C, S, SBLK=SBLK)\n\n if self.training:\n mean = sums / count\n var = sumsq / count - mean * mean\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn.momentum).add_(self.bn.momentum * mean.detach())\n unbiased = var * count / (count - 1)\n self.bn.running_var.mul_(1 - self.bn.momentum).add_(self.bn.momentum * unbiased.detach())\n else:\n mean = self.bn.running_mean\n var = self.bn.running_var\n\n invstd = torch.rsqrt(var + self.bn.eps)\n w = self.bn.weight\n b = self.bn.bias\n\n out = torch.empty_like(x)\n TOTAL = x.numel()\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bn_apply_kernel[grid](x, out, mean.contiguous(), invstd.contiguous(),\n w.contiguous(), b.contiguous(),\n float(self.scaling_factor), C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_069 Conv2d_HardSwish_ReLU, 1.25x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef act_kernel(x_ptr, out_ptr, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n # hardswish: x * clamp(x+3,0,6)/6\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n r = tl.maximum(hs, 0.0)\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n out = self.conv(x).contiguous()\n TOTAL = out.numel()\n res = torch.empty_like(out)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n act_kernel[grid](out, res, TOTAL, BLOCK=BLOCK)\n return res\n```", "ctx_chars": 11023, "ctx_sha": "04625c0d6e6734eb", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_GELU_GlobalAvgPool", "split": "learn", "n_ops": 3, "src_lines": 35, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_067 (Conv2d_GELU_GlobalAvgPool), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a convolution, applies GELU, and then performs global average pooling.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n \"\"\"\n Args:\n x: Input tensor of shape (batch_size, in_channels, height, width)\n Returns:\n Output tensor of shape (batch_size, out_channels)\n \"\"\"\n x = self.conv(x)\n x = torch.nn.functional.gelu(x)\n x = torch.nn.functional.adaptive_avg_pool2d(x, 1)\n x = x.squeeze(-1).squeeze(-1)\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 256, 256\nkernel_size = 3\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_068": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "8b2060bfd7036bae", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Min_Subtract", "split": "learn", "n_ops": 3, "src_lines": 28, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_068 (Matmul_Min_Subtract), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, applies minimum, and subtracts a constant.\n \"\"\"\n def __init__(self, in_features, out_features, constant):\n super(Model, self).__init__()\n self.linear = nn.Linear(in_features, out_features)\n self.constant = nn.Parameter(torch.tensor(constant))\n\n def forward(self, x):\n x = self.linear(x)\n x = torch.min(x, self.constant)\n x = x - self.constant\n return x\n\nbatch_size = 128\nin_features = 16384\nout_features = 16384\nconstant = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, constant]```\n"}, "L2_069": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_035 Conv2d_Subtract_HardSwish_MaxPool_Mish, 1.27x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, sub, N, C, H, W, OH, OW, P, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n base = (n.to(tl.int64) * C + c) * H * W\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -3.4e38, tl.float32)\n for ph in range(0, P):\n for pw in range(0, P):\n ih = h0 + ph\n iw = w0 + pw\n vmask = mask & (ih < H) & (iw < W)\n v = tl.load(x_ptr + base + ih.to(tl.int64) * W + iw, mask=vmask, other=-3.4e38)\n v = v - sub\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n acc = tl.maximum(acc, hs)\n # mish: x * tanh(softplus(x))\n sp = tl.log(1.0 + tl.exp(acc))\n th = (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n res = acc * th\n tl.store(out_ptr + offs, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, subtract_value, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract_value = subtract_value\n self.pool_k = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_k\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=torch.float32)\n TOTAL = out.numel()\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, out, float(self.subtract_value),\n N, C, H, W, OH, OW, P, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_092 Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp, 2.37x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\nfrom triton.language.extra.cuda import libdevice\n\n\n@triton.jit\ndef gn_stats_kernel(x_ptr, mean_ptr, invstd_ptr, CPG, S, GS, eps, BLOCK: tl.constexpr):\n pid = tl.program_id(0) # over n*groups\n base = pid.to(tl.int64) * GS\n acc = 0.0\n acc2 = 0.0\n for off in range(0, GS, BLOCK):\n offs = off + tl.arange(0, BLOCK)\n mask = offs < GS\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n acc += tl.sum(v, axis=0)\n acc2 += tl.sum(v * v, axis=0)\n mean = acc / GS\n var = acc2 / GS - mean * mean\n invstd = 1.0 / tl.sqrt(var + eps)\n tl.store(mean_ptr + pid, mean)\n tl.store(invstd_ptr + pid, invstd)\n\n\n@triton.jit\ndef fused_kernel(x_ptr, mean_ptr, invstd_ptr, gamma_ptr, beta_ptr, out_ptr,\n N, C, S, NS, G, CPG, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n maxv = tl.full((BLOCK,), -3.4e38, tl.float32)\n for c in range(C):\n g = c // CPG\n ng = n * G + g\n xc = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + ng, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + ng, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + c)\n beta = tl.load(beta_ptr + c)\n norm = (xc - mean) * invstd * gamma + beta\n t = libdevice.tanh(norm)\n hs = t * tl.minimum(tl.maximum(t + 3.0, 0.0), 6.0) / 6.0\n res = xc + hs\n maxv = tl.maximum(maxv, res)\n sumexp = tl.zeros((BLOCK,), tl.float32)\n for c in range(C):\n g = c // CPG\n ng = n * G + g\n xc = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + ng, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + ng, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + c)\n beta = tl.load(beta_ptr + c)\n norm = (xc - mean) * invstd * gamma + beta\n t = libdevice.tanh(norm)\n hs = t * tl.minimum(tl.maximum(t + 3.0, 0.0), 6.0) / 6.0\n res = xc + hs\n sumexp += tl.exp(res - maxv)\n out = maxv + tl.log(sumexp)\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, groups, eps=1e-5):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.group_norm = nn.GroupNorm(groups, out_channels, eps=eps)\n self.tanh = nn.Tanh()\n self.hard_swish = nn.Hardswish()\n self.groups = groups\n self.eps = eps\n\n def forward(self, x):\n x_conv = self.conv(x).contiguous()\n N, C, H, W = x_conv.shape\n S = H * W\n G = self.groups\n CPG = C // G\n GS = CPG * S\n mean = torch.empty((N * G,), device=x_conv.device, dtype=torch.float32)\n invstd = torch.empty((N * G,), device=x_conv.device, dtype=torch.float32)\n gn_stats_kernel[(N * G,)](x_conv, mean, invstd, CPG, S, GS, self.eps, BLOCK=1024)\n out = torch.empty((N, 1, H, W), device=x_conv.device, dtype=x_conv.dtype)\n NS = N * S\n BLOCK = 256\n grid = (triton.cdiv(NS, BLOCK),)\n fused_kernel[grid](x_conv, mean, invstd,\n self.group_norm.weight, self.group_norm.bias,\n out, N, C, S, NS, G, CPG, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 12471, "ctx_sha": "4ddcc848d0210ca9", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_HardSwish_ReLU", "split": "learn", "n_ops": 3, "src_lines": 35, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_069 (Conv2d_HardSwish_ReLU), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a convolution, applies HardSwish, and then ReLU.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_channels, height, width).\n \"\"\"\n x = self.conv(x)\n x = torch.nn.functional.hardswish(x)\n x = torch.relu(x)\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 128, 128\nkernel_size = 3\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size]```\n"}, "L2_070": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n\n### A verified kernel for a related matmul problem (L2_053 Gemm_Scaling_Hardtanh_GELU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, hmin, hmax, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask)\n v = v * scaling\n v = tl.minimum(tl.maximum(v, hmin), hmax)\n g = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, g, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n self.hardtanh_min = hardtanh_min\n self.hardtanh_max = hardtanh_max\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor,\n float(self.hardtanh_min), float(self.hardtanh_max),\n n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_051 Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd, 36.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef add_kernel(x_ptr, s_ptr, out_ptr, n_feat, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n row = offs // n_feat\n v = tl.load(x_ptr + offs, mask=mask)\n s = tl.load(s_ptr + row, mask=mask)\n tl.store(out_ptr + offs, v + s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias=True):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n self.out_features = out_features\n\n def forward(self, x):\n # mean over output features of (Wx + b - subtract) = (colsum/out)\u00b7x + c\n colsum = self.gemm.weight.sum(dim=0) / self.out_features # (in,)\n if self.gemm.bias is not None:\n c = (self.gemm.bias - self.subtract).mean()\n else:\n c = (-self.subtract).mean()\n s = x @ colsum + c # (batch,)\n # logsumexp over single element (dim=1 after keepdim mean) is identity\n s = F.gelu(s)\n out = torch.empty_like(x)\n n = x.numel()\n n_feat = x.shape[1]\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n add_kernel[grid](x, s, out, n_feat, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9220, "ctx_sha": "08359cc7cf7c25a1", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 5, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Sigmoid_Scaling_ResidualAdd", "split": "learn", "n_ops": 4, "src_lines": 39, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_070 (Gemm_Sigmoid_Scaling_ResidualAdd), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model implementing the pattern \"Gemm_Sigmoid_Scaling_ResidualAdd\".\n \"\"\"\n def __init__(self, input_size, hidden_size, scaling_factor):\n super(Model, self).__init__()\n self.gemm = nn.Linear(input_size, hidden_size)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n \"\"\"\n Forward pass of the model.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, input_size).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, hidden_size).\n \"\"\"\n x = self.gemm(x)\n original_x = x\n x = torch.sigmoid(x)\n x = x * self.scaling_factor\n x = x + original_x\n return x\n\nbatch_size = 1024\ninput_size = 8192\nhidden_size = 8192\nscaling_factor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size)]\n\ndef get_init_inputs():\n return [input_size, hidden_size, scaling_factor]```\n"}, "L2_071": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_073 Conv2d_BatchNorm_Scaling, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef stats_kernel2(x_ptr, sum_ptr, sumsq_ptr, N, C, S, SBLK: tl.constexpr):\n c = tl.program_id(0)\n sblk = tl.program_id(1)\n offs = sblk * SBLK + tl.arange(0, SBLK)\n smask = offs < S\n acc_sum = tl.zeros((SBLK,), tl.float32)\n acc_sq = tl.zeros((SBLK,), tl.float32)\n cS = c.to(tl.int64) * S\n NCS = C * S\n for n in range(N):\n base = n.to(tl.int64) * NCS + cS + offs\n v = tl.load(x_ptr + base, mask=smask, other=0.0)\n acc_sum += v\n acc_sq += v * v\n s = tl.sum(acc_sum, axis=0)\n sq = tl.sum(acc_sq, axis=0)\n tl.atomic_add(sum_ptr + c, s)\n tl.atomic_add(sumsq_ptr + c, sq)\n\n\n@triton.jit\ndef bn_apply_kernel(x_ptr, out_ptr, mean_ptr, invstd_ptr, w_ptr, b_ptr,\n scaling, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + c, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + c, mask=mask, other=0.0)\n w = tl.load(w_ptr + c, mask=mask, other=0.0)\n b = tl.load(b_ptr + c, mask=mask, other=0.0)\n r = ((v - mean) * invstd * w + b) * scaling\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n count = N * S\n device = x.device\n\n sums = torch.zeros(C, device=device, dtype=torch.float32)\n sumsq = torch.zeros(C, device=device, dtype=torch.float32)\n SBLK = 1024\n nsblk = triton.cdiv(S, SBLK)\n stats_kernel2[(C, nsblk)](x, sums, sumsq, N, C, S, SBLK=SBLK)\n\n if self.training:\n mean = sums / count\n var = sumsq / count - mean * mean\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn.momentum).add_(self.bn.momentum * mean.detach())\n unbiased = var * count / (count - 1)\n self.bn.running_var.mul_(1 - self.bn.momentum).add_(self.bn.momentum * unbiased.detach())\n else:\n mean = self.bn.running_mean\n var = self.bn.running_var\n\n invstd = torch.rsqrt(var + self.bn.eps)\n w = self.bn.weight\n b = self.bn.bias\n\n out = torch.empty_like(x)\n TOTAL = x.numel()\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bn_apply_kernel[grid](x, out, mean.contiguous(), invstd.contiguous(),\n w.contiguous(), b.contiguous(),\n float(self.scaling_factor), C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_067 Conv2d_GELU_GlobalAvgPool, 1.30x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\nfrom triton.language.extra import libdevice\n\n\n@triton.jit\ndef gelu_avg_kernel(x_ptr, out_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n base = pid.to(tl.int64) * S\n acc = 0.0\n for off in range(0, S, BLOCK):\n offs = off + tl.arange(0, BLOCK)\n mask = offs < S\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n g = 0.5 * v * (1.0 + libdevice.erf(v * 0.7071067811865476))\n acc += tl.sum(g, axis=0)\n tl.store(out_ptr + pid, acc / S)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, C), device=x.device, dtype=x.dtype)\n grid = (N * C,)\n gelu_avg_kernel[grid](x, out, S, BLOCK=1024)\n return out\n```", "ctx_chars": 11132, "ctx_sha": "4caa9b6e3a0949e1", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Divide_LeakyReLU", "split": "learn", "n_ops": 3, "src_lines": 30, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_071 (Conv2d_Divide_LeakyReLU), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a convolution, divides by a constant, and applies LeakyReLU.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, divisor):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.divisor = divisor\n\n def forward(self, x):\n x = self.conv(x)\n x = x / self.divisor\n x = torch.nn.functional.leaky_relu(x, negative_slope=0.01)\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 128, 128\nkernel_size = 3\ndivisor = 2\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, divisor]```\n"}, "L2_072": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_015 ConvTranspose3d_BatchNorm_Subtract, 1.32x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef out_kernel(x_ptr, out_ptr, scale_ptr, smean_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n sc = tl.load(scale_ptr + pid)\n sm = tl.load(smean_ptr + pid)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n r = sc * (v - sm)\n tl.store(out_ptr + base + idx, r, mask=m)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias=True):\n super(ModelNew, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n total_sum = ssum_nc.sum(0)\n total_sq = ssq_nc.sum(0)\n M = N * S\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps) # (C,)\n\n scale_nc = scale_c.unsqueeze(0).expand(N, C).contiguous().view(N * C)\n smean_nc = (ssum_nc / S).contiguous().view(N * C)\n\n out = torch.empty_like(c2)\n out_kernel[(N * C,)](c2, out, scale_nc, smean_nc, S, BLOCK=1024)\n return out.view(N, C, D, H, W)\n```\n\n### A verified kernel for a related conv_transpose problem (L2_077 ConvTranspose3d_Scale_BatchNorm_GlobalAvgPool, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor, eps=1e-5, momentum=0.1):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n self.batch_norm = nn.BatchNorm3d(out_channels, eps=eps, momentum=momentum)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n ssum = ssum.view(N, C)\n ssq = ssq.view(N, C)\n total_sum = ssum.sum(0)\n total_sq = ssq.sum(0)\n M = N * S\n scale = self.scale_factor\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n mean_s = scale * mean_c\n var_s = scale * scale * var_c\n inv = self.batch_norm.weight / torch.sqrt(var_s + self.batch_norm.eps)\n spatial_mean_s = scale * (ssum / S)\n out = inv.unsqueeze(0) * (spatial_mean_s - mean_s.unsqueeze(0)) + self.batch_norm.bias.unsqueeze(0)\n return out.view(N, C, 1, 1, 1)\n```", "ctx_chars": 10613, "ctx_sha": "62447346e53b88c7", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_BatchNorm_AvgPool_AvgPool", "split": "learn", "n_ops": 4, "src_lines": 37, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_072 (ConvTranspose3d_BatchNorm_AvgPool_AvgPool), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a 3D transposed convolution, followed by batch normalization, \n two average pooling layers.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n self.avg_pool1 = nn.AvgPool3d(kernel_size=2)\n self.avg_pool2 = nn.AvgPool3d(kernel_size=2)\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.batch_norm(x)\n x = self.avg_pool1(x)\n x = self.avg_pool2(x)\n return x\n\n\nbatch_size = 64\nin_channels = 3\nout_channels = 16\ndepth, height, width = 32, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\nbias_shape = (out_channels, 1, 1, 1)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, bias_shape]```\n"}, "L2_073": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_032 Conv2d_Scaling_Min, 1.10x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef scale_min_kernel(x_ptr, out_ptr, scale, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = acc * scale\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n scale_min_kernel[grid](x, out, float(self.scale_factor), C, S, NS, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_082 Conv2d_Tanh_Scaling_BiasAdd_Max, 1.94x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, bias_ptr, out_ptr,\n N, C, H, W, OH, OW, P, scaling,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n TOTAL = N * C * OH * OW\n mask = offs < TOTAL\n\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n base = (n.to(tl.int64) * C + c) * H * W\n for ph in range(P):\n for pw in range(P):\n h = h0 + ph\n w = w0 + pw\n idx = base + h.to(tl.int64) * W + w\n v = tl.load(x_ptr + idx, mask=mask, other=-1e30)\n t = tl.extra.cuda.libdevice.tanh(v)\n t = t * scaling + b\n acc = tl.maximum(acc, t)\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scaling_factor = scaling_factor\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.pool_kernel_size = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_kernel_size\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=x.dtype)\n TOTAL = N * C * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, self.bias.contiguous().view(-1), out,\n N, C, H, W, OH, OW, P, float(self.scaling_factor),\n BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9590, "ctx_sha": "5bdedd614e688703", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_BatchNorm_Scaling", "split": "learn", "n_ops": 3, "src_lines": 31, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_073 (Conv2d_BatchNorm_Scaling), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a convolution, applies Batch Normalization, and scales the output.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.conv(x)\n x = self.bn(x)\n x = x * self.scaling_factor\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 128, 128\nkernel_size = 3\nscaling_factor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, scaling_factor]```\n"}, "L2_074": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_078 ConvTranspose3d_Max_Max_Sum, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, OD, OH, OW, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n idx64 = idx.to(tl.int64)\n mask = idx < total\n\n ow = idx64 % OW\n t = idx64 // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n n = t // OD\n\n d0 = od * 6\n h0 = oh * 6\n w0 = ow * 6\n\n C64 = C.to(tl.int64)\n D64 = D.to(tl.int64)\n H64 = H.to(tl.int64)\n W64 = W.to(tl.int64)\n\n s = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n base_nc = (n * C64 + c) * D64\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for dz in tl.static_range(6):\n iz = d0 + dz\n for dy in tl.static_range(6):\n iy = h0 + dy\n for dx in tl.static_range(6):\n ix = w0 + dx\n off = ((base_nc + iz) * H64 + iy) * W64 + ix\n v = tl.load(x_ptr + off, mask=mask, other=-1e30)\n m = tl.maximum(m, v)\n s += m\n tl.store(out_ptr + idx64, s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n P1D, P1H, P1W = D // 2, H // 2, W // 2\n OD, OH, OW = P1D // 3, P1H // 3, P1W // 3\n out = torch.empty((N, 1, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * OD * OH * OW\n BLOCK = 128\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](c, out, N, C, D, H, W, OD, OH, OW, total, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_089 ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max, 1.06x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, sub_ptr, out_ptr, N, C, S, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n mask = idx < total\n\n # idx indexes (n, s) flat where s in [0,S)\n n = idx // S\n s = idx % S\n base = n * C * S + s\n\n # pass 1: max over channels\n mx = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=-1e30)\n mx = tl.maximum(mx, v)\n\n # pass 2: sum of exp\n sm = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n sm += tl.exp(v - mx)\n\n # pass 3: softmax, subtract, swish, max over channels\n res = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n sub = tl.load(sub_ptr + c)\n sm_val = tl.exp(v - mx) / sm\n y = sm_val - sub\n swish = (tl.sigmoid(y) * y)\n res = tl.maximum(res, swish)\n\n tl.store(out_ptr + idx, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, pool_kernel_size, pool_stride, pool_padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.max_pool = nn.MaxPool3d(kernel_size=pool_kernel_size, stride=pool_stride, padding=pool_padding)\n self.subtract = nn.Parameter(torch.randn(out_channels))\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.max_pool(x).contiguous()\n N, C, D, H, W = x.shape\n S = D * H * W\n total = N * S\n\n out = torch.empty((N, D, H, W), device=x.device, dtype=x.dtype)\n sub = self.subtract.contiguous()\n\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](x, sub, out, N, C, S, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10364, "ctx_sha": "19f3ac1a7ff915f6", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max", "split": "learn", "n_ops": 5, "src_lines": 38, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_074 (ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, applies LeakyReLU, multiplies by a learnable parameter, \n applies LeakyReLU again, and performs a max pooling operation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, multiplier_shape):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.multiplier = nn.Parameter(torch.randn(multiplier_shape))\n self.leaky_relu = nn.LeakyReLU(negative_slope=0.2)\n self.max_pool = nn.MaxPool3d(kernel_size=2)\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.leaky_relu(x)\n x = x * self.multiplier\n x = self.leaky_relu(x)\n x = self.max_pool(x)\n return x\n\nbatch_size = 16\nin_channels = 16\nout_channels = 32\ndepth, height, width = 16, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\noutput_padding = 1\nmultiplier_shape = (out_channels, 1, 1, 1)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, output_padding, multiplier_shape]```\n"}, "L2_075": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_094 Gemm_BiasAdd_Hardtanh_Mish_GroupNorm, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, bias_ptr, gamma_ptr, beta_ptr, out_ptr,\n N, C, G, CG, eps,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n col0 = grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n chan = col0 + offs\n idx = row * C + chan\n\n v = tl.load(x_ptr + idx, mask=mask, other=0.0)\n b = tl.load(bias_ptr + chan, mask=mask, other=0.0)\n v = v + b\n # hardtanh [-1,1]\n v = tl.minimum(tl.maximum(v, -1.0), 1.0)\n # mish: v * tanh(softplus(v))\n sp = tl.log(1.0 + tl.exp(v))\n v = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n\n # group norm stats\n s = tl.sum(tl.where(mask, v, 0.0), axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n inv = 1.0 / tl.sqrt(var + eps)\n g = tl.load(gamma_ptr + chan, mask=mask, other=0.0)\n bt = tl.load(beta_ptr + chan, mask=mask, other=0.0)\n out = (v - mean) * inv * g + bt\n tl.store(out_ptr + idx, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape, num_groups):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.hardtanh = nn.Hardtanh()\n self.mish = nn.Mish()\n self.groupnorm = nn.GroupNorm(num_groups=num_groups, num_channels=out_features)\n self.num_groups = num_groups\n self.out_features = out_features\n\n def forward(self, x):\n z = self.gemm(x)\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n fused_kernel[grid](\n z, self.bias, self.groupnorm.weight, self.groupnorm.bias, out,\n N, C, G, CG, float(self.groupnorm.eps),\n BLOCK=BLOCK,\n )\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_076 Gemm_Add_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bias_relu_kernel(x_ptr, bias_ptr, out_ptr, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n b = tl.load(bias_ptr + col, mask=mask, other=0.0)\n y = tl.maximum(v + b, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n z = torch.mm(x, self.gemm.weight.t())\n M, N = z.shape\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n bias_relu_kernel[grid](z, self.bias, out, M, N, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10167, "ctx_sha": "bcbf9a06f3527d57", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_GroupNorm_Min_BiasAdd", "split": "learn", "n_ops": 4, "src_lines": 31, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_075 (Gemm_GroupNorm_Min_BiasAdd), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a GEMM, Group Normalization, Minimum operation, and Bias addition.\n \"\"\"\n def __init__(self, in_features, out_features, num_groups, bias_shape):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.group_norm = nn.GroupNorm(num_groups, out_features)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n x = self.gemm(x)\n x = self.group_norm(x)\n x = torch.min(x, dim=1, keepdim=True)[0] \n x = x + self.bias\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nnum_groups = 512\nbias_shape = (1, out_features, 1, 1)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, num_groups, bias_shape]```\n"}, "L2_076": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_041 Gemm_BatchNorm_GELU_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr,\n eps, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n # gelu\n g = 0.5 * y * (1.0 + tl.erf(y * 0.7071067811865476))\n # relu\n g = tl.maximum(g, 0.0)\n tl.store(out_ptr + offs, g, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.batch_norm = nn.BatchNorm1d(out_features)\n self.eps = 1e-5\n self.momentum = 0.1\n\n def forward(self, x):\n z = self.gemm(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=2048)\n\n if self.training:\n with torch.no_grad():\n self.batch_norm.running_mean.mul_(1 - self.momentum).add_(self.momentum * mean)\n unbiased = var * (M / (M - 1))\n self.batch_norm.running_var.mul_(1 - self.momentum).add_(self.momentum * unbiased)\n self.batch_norm.num_batches_tracked.add_(1)\n use_mean, use_var = mean, var\n else:\n use_mean = self.batch_norm.running_mean\n use_var = self.batch_norm.running_var\n\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, use_mean, use_var, self.batch_norm.weight,\n self.batch_norm.bias, self.eps, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_012 Gemm_Multiply_LeakyReLU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, mult, neg_slope, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * mult\n v = tl.where(v >= 0, v, v * neg_slope)\n tl.store(out_ptr + offs, v, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, multiplier, negative_slope):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.multiplier = multiplier\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = torch.addmm(self.gemm.bias, x, self.gemm.weight.t())\n n = z.numel()\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n fused_kernel[grid](z, out, self.multiplier, self.negative_slope, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10847, "ctx_sha": "8373577b4232711a", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Add_ReLU", "split": "learn", "n_ops": 3, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_076 (Gemm_Add_ReLU), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, adds a bias term, and applies ReLU.\n \"\"\"\n def __init__(self, in_features, out_features, bias_shape):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor with shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor with shape (batch_size, out_features).\n \"\"\"\n x = self.gemm(x)\n x = x + self.bias\n x = torch.relu(x)\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nbias_shape = (out_features,)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, bias_shape]```\n"}, "L2_077": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_072 ConvTranspose3d_BatchNorm_AvgPool_AvgPool, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef pool_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr,\n N, C, D, H, W, OD, OH, OW, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n total = N * C * OD * OH * OW\n mask = offs < total\n o = offs.to(tl.int64)\n ow = o % OW\n t = o // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n t = t // OD\n c = t % C\n n = t // C\n\n sc = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n\n HW = H * W\n DHW = D * HW\n base = (n * C + c) * DHW\n\n acc = tl.zeros((BLOCK,), dtype=tl.float32)\n for dd in tl.static_range(4):\n for hh in tl.static_range(4):\n for ww in tl.static_range(4):\n id_ = od * 4 + dd\n ih = oh * 4 + hh\n iw = ow * 4 + ww\n addr = base + id_ * HW + ih * W + iw\n v = tl.load(x_ptr + addr, mask=mask, other=0.0)\n acc += v\n acc = acc * (1.0 / 64.0)\n r = sc * acc + sh\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n M = N * S\n mean_c = ssum_nc.sum(0) / M\n var_c = ssq_nc.sum(0) / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps)\n shift_c = self.batch_norm.bias - scale_c * mean_c\n\n OD = D // 4\n OH = H // 4\n OW = W // 4\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=torch.float32)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n cc = c.contiguous()\n pool_kernel[grid](cc, out, scale_c.contiguous(), shift_c.contiguous(),\n N, C, D, H, W, OD, OH, OW, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_015 ConvTranspose3d_BatchNorm_Subtract, 1.32x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef out_kernel(x_ptr, out_ptr, scale_ptr, smean_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n sc = tl.load(scale_ptr + pid)\n sm = tl.load(smean_ptr + pid)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n r = sc * (v - sm)\n tl.store(out_ptr + base + idx, r, mask=m)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias=True):\n super(ModelNew, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n total_sum = ssum_nc.sum(0)\n total_sq = ssq_nc.sum(0)\n M = N * S\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps) # (C,)\n\n scale_nc = scale_c.unsqueeze(0).expand(N, C).contiguous().view(N * C)\n smean_nc = (ssum_nc / S).contiguous().view(N * C)\n\n out = torch.empty_like(c2)\n out_kernel[(N * C,)](c2, out, scale_nc, smean_nc, S, BLOCK=1024)\n return out.view(N, C, D, H, W)\n```", "ctx_chars": 12352, "ctx_sha": "5533cb2a86547de2", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_Scale_BatchNorm_GlobalAvgPool", "split": "learn", "n_ops": 4, "src_lines": 34, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_077 (ConvTranspose3d_Scale_BatchNorm_GlobalAvgPool), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, scales the output, applies batch normalization, \n and then performs global average pooling. \n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor, eps=1e-5, momentum=0.1):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n self.batch_norm = nn.BatchNorm3d(out_channels, eps=eps, momentum=momentum)\n self.global_avg_pool = nn.AdaptiveAvgPool3d((1, 1, 1))\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = x * self.scale_factor\n x = self.batch_norm(x)\n x = self.global_avg_pool(x)\n return x\n\nbatch_size = 16\nin_channels = 64\nout_channels = 128\ndepth, height, width = 16, 32, 32\nkernel_size = 5\nscale_factor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, scale_factor]```\n"}, "L2_078": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_074 ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max, 1.26x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, mult_ptr, N, C, D, H, W, OD, OH, OW,\n slope, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n mask = idx < total\n\n ow = idx % OW\n t = idx // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n nc = t // OD\n c = nc % C\n\n m = tl.load(mult_ptr + c, mask=mask, other=0.0)\n\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n for dz in tl.static_range(2):\n for dy in tl.static_range(2):\n for dx in tl.static_range(2):\n iz = od * 2 + dz\n iy = oh * 2 + dy\n ix = ow * 2 + dx\n in_off = (((nc * D + iz) * H + iy) * W + ix)\n v = tl.load(x_ptr + in_off, mask=mask, other=0.0)\n v = tl.where(v >= 0, v, v * slope)\n v = v * m\n v = tl.where(v >= 0, v, v * slope)\n acc = tl.maximum(acc, v)\n tl.store(out_ptr + idx, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, multiplier_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.multiplier = nn.Parameter(torch.randn(multiplier_shape))\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n OD, OH, OW = D // 2, H // 2, W // 2\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * C * OD * OH * OW\n mult = self.multiplier.contiguous().view(-1)\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](c, out, mult, N, C, D, H, W, OD, OH, OW, 0.2, total, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_003 ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU, 1.26x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef ln_kernel(x_ptr, out_ptr, w_ptr, b_ptr, sw, W, eps, BLOCK: tl.constexpr):\n pid = tl.program_id(0).to(tl.int64)\n offs = tl.arange(0, BLOCK)\n m = offs < W\n base = pid * W\n v = tl.load(x_ptr + base + offs, mask=m, other=0.0) + sw\n mean = tl.sum(v, axis=0) / W\n d = tl.where(m, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / W\n rstd = 1.0 / tl.sqrt(var + eps)\n w = tl.load(w_ptr + offs, mask=m, other=0.0)\n b = tl.load(b_ptr + offs, mask=m, other=0.0)\n r = d * rstd * w + b\n tl.store(out_ptr + base + offs, r, mask=m)\n\n\n@triton.jit\ndef pool_gelu_kernel(x_ptr, out_ptr, D, H, W, OD, OH, OW, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n idx64 = idx.to(tl.int64)\n mask = idx < total\n ow = idx64 % OW\n t = idx64 // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n nc = t // OD\n\n acc = tl.zeros((BLOCK,), tl.float32)\n for dz in tl.static_range(2):\n for dy in tl.static_range(2):\n for dx in tl.static_range(2):\n iz = od * 2 + dz\n iy = oh * 2 + dy\n ix = ow * 2 + dx\n in_off = ((nc * D + iz) * H + iy) * W + ix\n v = tl.load(x_ptr + in_off, mask=mask, other=0.0)\n acc += v\n acc = acc * 0.125\n gelu = acc * 0.5 * (1.0 + tl.erf(acc * 0.70710678118654752440))\n tl.store(out_ptr + idx64, gelu, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, sum_weight, norm_shape, pool_kernel_size):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.sum_weight = nn.Parameter(torch.tensor(sum_weight))\n self.norm = nn.LayerNorm(norm_shape)\n self.avg_pool = nn.AvgPool3d(kernel_size=pool_kernel_size)\n self.gelu = nn.GELU()\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n rows = N * C * D * H\n c2 = c.view(rows, W)\n normed = torch.empty_like(c2)\n BLOCKW = triton.next_power_of_2(W)\n w = self.norm.weight.contiguous()\n b = self.norm.bias.contiguous()\n ln_kernel[(rows,)](c2, normed, w, b, float(self.sum_weight.item()), W, self.norm.eps, BLOCK=BLOCKW)\n normed = normed.view(N, C, D, H, W)\n\n OD, OH, OW = D // 2, H // 2, W // 2\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * C * OD * OH * OW\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n pool_gelu_kernel[grid](normed, out, D, H, W, OD, OH, OW, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11755, "ctx_sha": "c9bcbb50b8ac09fd", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_Max_Max_Sum", "split": "learn", "n_ops": 4, "src_lines": 33, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_078 (ConvTranspose3d_Max_Max_Sum), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, followed by two max pooling layers and a sum operation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.max_pool1 = nn.MaxPool3d(kernel_size=2)\n self.max_pool2 = nn.MaxPool3d(kernel_size=3)\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.max_pool1(x)\n x = self.max_pool2(x)\n x = torch.sum(x, dim=1, keepdim=True) \n return x\n\nbatch_size = 16\nin_channels = 32\nout_channels = 64\ndepth, height, width = 32, 32, 32\nkernel_size = 5\nstride = 2\npadding = 2\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding]```\n"}, "L2_079": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_043 Conv3d_Max_LogSumExp_ReLU, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, Dp, Hp, Wp,\n NOUT, BLOCK: tl.constexpr, C_CONST: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NOUT\n npw = Hp * Wp\n nplane = Dp * npw\n n = offs // nplane\n rem = offs % nplane\n dp = rem // npw\n rem2 = rem % npw\n hp = rem2 // Wp\n wp = rem2 % Wp\n d0 = dp * 2\n h0 = hp * 2\n w0 = wp * 2\n HW = H * W\n S = D * HW\n nbase = n.to(tl.int64) * C * S\n\n run_max = tl.full((BLOCK,), -1e30, tl.float32)\n run_sum = tl.zeros((BLOCK,), tl.float32)\n for c in range(C_CONST):\n cbase = nbase + c * S\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for kd in range(2):\n for kh in range(2):\n for kw in range(2):\n off = cbase + (d0 + kd).to(tl.int64) * HW + (h0 + kh) * W + (w0 + kw)\n v = tl.load(x_ptr + off, mask=mask, other=-1e30)\n m = tl.maximum(m, v)\n new_max = tl.maximum(run_max, m)\n run_sum = run_sum * tl.exp(run_max - new_max) + tl.exp(m - new_max)\n run_max = new_max\n lse = run_max + tl.log(run_sum)\n lse = tl.maximum(lse, 0.0)\n tl.store(out_ptr + offs, lse, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.out_channels = out_channels\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, D, H, W = x.shape\n Dp, Hp, Wp = D // 2, H // 2, W // 2\n out = torch.empty((N, 1, Dp, Hp, Wp), device=x.device, dtype=x.dtype)\n NOUT = N * Dp * Hp * Wp\n BLOCK = 256\n grid = (triton.cdiv(NOUT, BLOCK),)\n fused_kernel[grid](x, out, N, C, D, H, W, Dp, Hp, Wp,\n NOUT, BLOCK=BLOCK, C_CONST=C)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_048 Conv3d_Scaling_Tanh_Multiply_Sigmoid, 1.33x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, scale_ptr, bias_ptr, out_ptr,\n TOTAL, C, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n b = tl.load(bias_ptr + c, mask=mask, other=0.0)\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n t = tl.extra.cuda.libdevice.tanh(x * s)\n t = t * b\n out = 1.0 / (1.0 + tl.exp(-t))\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.scaling_factor = nn.Parameter(torch.randn(bias_shape))\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, D, H, W = x.shape\n S = D * H * W\n TOTAL = x.numel()\n out = torch.empty_like(x)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, self.scaling_factor.contiguous().view(-1),\n self.bias.contiguous().view(-1), out,\n TOTAL, C, S, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10508, "ctx_sha": "4fa7857ea4c0dcf4", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv3d_Multiply_InstanceNorm_Clamp_Multiply_Max", "split": "learn", "n_ops": 6, "src_lines": 38, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_079 (Conv3d_Multiply_InstanceNorm_Clamp_Multiply_Max), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A 3D convolutional layer followed by multiplication, instance normalization, clamping, multiplication, and a max operation.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, multiplier_shape, clamp_min, clamp_max):\n super(Model, self).__init__()\n self.conv = nn.Conv3d(in_channels, out_channels, kernel_size)\n self.multiplier = nn.Parameter(torch.randn(multiplier_shape))\n self.instance_norm = nn.InstanceNorm3d(out_channels)\n self.clamp_min = clamp_min\n self.clamp_max = clamp_max\n\n def forward(self, x):\n x = self.conv(x)\n x = x * self.multiplier\n x = self.instance_norm(x)\n x = torch.clamp(x, self.clamp_min, self.clamp_max)\n x = x * self.multiplier\n x = torch.max(x, dim=1)[0]\n return x\n\nbatch_size = 128\nin_channels = 3\nout_channels = 16\ndepth, height, width = 16, 32, 32\nkernel_size = 3\nmultiplier_shape = (out_channels, 1, 1, 1)\nclamp_min = -1.0\nclamp_max = 1.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, multiplier_shape, clamp_min, clamp_max]```\n"}, "L2_082": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_073 Conv2d_BatchNorm_Scaling, 1.68x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef stats_kernel2(x_ptr, sum_ptr, sumsq_ptr, N, C, S, SBLK: tl.constexpr):\n c = tl.program_id(0)\n sblk = tl.program_id(1)\n offs = sblk * SBLK + tl.arange(0, SBLK)\n smask = offs < S\n acc_sum = tl.zeros((SBLK,), tl.float32)\n acc_sq = tl.zeros((SBLK,), tl.float32)\n cS = c.to(tl.int64) * S\n NCS = C * S\n for n in range(N):\n base = n.to(tl.int64) * NCS + cS + offs\n v = tl.load(x_ptr + base, mask=smask, other=0.0)\n acc_sum += v\n acc_sq += v * v\n s = tl.sum(acc_sum, axis=0)\n sq = tl.sum(acc_sq, axis=0)\n tl.atomic_add(sum_ptr + c, s)\n tl.atomic_add(sumsq_ptr + c, sq)\n\n\n@triton.jit\ndef bn_apply_kernel(x_ptr, out_ptr, mean_ptr, invstd_ptr, w_ptr, b_ptr,\n scaling, C, S, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n c = (offs // S) % C\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + c, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + c, mask=mask, other=0.0)\n w = tl.load(w_ptr + c, mask=mask, other=0.0)\n b = tl.load(b_ptr + c, mask=mask, other=0.0)\n r = ((v - mean) * invstd * w + b) * scaling\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.bn = nn.BatchNorm2d(out_channels)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n count = N * S\n device = x.device\n\n sums = torch.zeros(C, device=device, dtype=torch.float32)\n sumsq = torch.zeros(C, device=device, dtype=torch.float32)\n SBLK = 1024\n nsblk = triton.cdiv(S, SBLK)\n stats_kernel2[(C, nsblk)](x, sums, sumsq, N, C, S, SBLK=SBLK)\n\n if self.training:\n mean = sums / count\n var = sumsq / count - mean * mean\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn.momentum).add_(self.bn.momentum * mean.detach())\n unbiased = var * count / (count - 1)\n self.bn.running_var.mul_(1 - self.bn.momentum).add_(self.bn.momentum * unbiased.detach())\n else:\n mean = self.bn.running_mean\n var = self.bn.running_var\n\n invstd = torch.rsqrt(var + self.bn.eps)\n w = self.bn.weight\n b = self.bn.bias\n\n out = torch.empty_like(x)\n TOTAL = x.numel()\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n bn_apply_kernel[grid](x, out, mean.contiguous(), invstd.contiguous(),\n w.contiguous(), b.contiguous(),\n float(self.scaling_factor), C, S, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_032 Conv2d_Scaling_Min, 1.10x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef scale_min_kernel(x_ptr, out_ptr, scale, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = acc * scale\n tl.store(out_ptr + offs, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, scale_factor):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n scale_min_kernel[grid](x, out, float(self.scale_factor), C, S, NS, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10743, "ctx_sha": "30d3b01036fcb086", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_Tanh_Scaling_BiasAdd_Max", "split": "learn", "n_ops": 5, "src_lines": 41, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_082 (Conv2d_Tanh_Scaling_BiasAdd_Max), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a convolution, applies tanh, scaling, adds a bias term, and then max-pools.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, scaling_factor, bias_shape, pool_kernel_size):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.scaling_factor = scaling_factor\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.max_pool = nn.MaxPool2d(pool_kernel_size)\n\n def forward(self, x):\n # Convolution\n x = self.conv(x)\n # Tanh activation\n x = torch.tanh(x)\n # Scaling\n x = x * self.scaling_factor\n # Bias addition\n x = x + self.bias\n # Max-pooling\n x = self.max_pool(x)\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 256, 256\nkernel_size = 3\nscaling_factor = 2.0\nbias_shape = (out_channels, 1, 1)\npool_kernel_size = 4\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, scaling_factor, bias_shape, pool_kernel_size]```\n"}, "L2_084": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_070 Gemm_Sigmoid_Scaling_ResidualAdd, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, n_elements, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elements\n v = tl.load(x_ptr + offs, mask=mask)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = s * scaling + v\n tl.store(out_ptr + offs, y, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, scaling_factor):\n super().__init__()\n self.gemm = nn.Linear(input_size, hidden_size)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_053 Gemm_Scaling_Hardtanh_GELU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, hmin, hmax, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask)\n v = v * scaling\n v = tl.minimum(tl.maximum(v, hmin), hmax)\n g = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, g, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n self.hardtanh_min = hardtanh_min\n self.hardtanh_max = hardtanh_max\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor,\n float(self.hardtanh_min), float(self.hardtanh_max),\n n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9192, "ctx_sha": "9ac79c33be050f60", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_BatchNorm_Scaling_Softmax", "split": "learn", "n_ops": 4, "src_lines": 39, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_084 (Gemm_BatchNorm_Scaling_Softmax), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a matrix multiplication (Gemm), Batch Normalization, scaling, and Softmax.\n \"\"\"\n def __init__(self, in_features, out_features, bn_eps=1e-5, bn_momentum=0.1, scale_shape=(1,)):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.bn = nn.BatchNorm1d(out_features, eps=bn_eps, momentum=bn_momentum)\n self.scale = nn.Parameter(torch.ones(scale_shape))\n self.softmax = nn.Softmax(dim=1)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.gemm(x)\n x = self.bn(x)\n x = self.scale * x\n x = self.softmax(x)\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nbn_eps = 1e-5\nbn_momentum = 0.1\nscale_shape = (1,)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, bn_eps, bn_momentum, scale_shape]```\n"}, "L2_085": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.37x on Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp] Keep the conv in cuDNN via torch. Replace the GroupNorm+pointwise+channel-reduction tail with two Triton kernels. (1) A stats kernel with one program per (n, group): the group is a contiguous CPG*S slice, so loop BLOCK=1024 accumulating sum and sum-of-squares in fp32, then emit mean and invstd. (2) A fused kernel with one thread per (n, spatial) position and BLOCK=256 coalesced along spatial. Loop over channels with stride S, normalize, apply the activations, add the residual, and compute a two-pass max/sum-exp logsumexp. Recompute instead of storing intermediates. Use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n\n### A verified kernel for a related conv problem (L2_035 Conv2d_Subtract_HardSwish_MaxPool_Mish, 1.27x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, sub, N, C, H, W, OH, OW, P, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n ow = offs % OW\n tmp = offs // OW\n oh = tmp % OH\n tmp = tmp // OH\n c = tmp % C\n n = tmp // C\n\n base = (n.to(tl.int64) * C + c) * H * W\n h0 = oh * P\n w0 = ow * P\n acc = tl.full((BLOCK,), -3.4e38, tl.float32)\n for ph in range(0, P):\n for pw in range(0, P):\n ih = h0 + ph\n iw = w0 + pw\n vmask = mask & (ih < H) & (iw < W)\n v = tl.load(x_ptr + base + ih.to(tl.int64) * W + iw, mask=vmask, other=-3.4e38)\n v = v - sub\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n acc = tl.maximum(acc, hs)\n # mish: x * tanh(softplus(x))\n sp = tl.log(1.0 + tl.exp(acc))\n th = (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n res = acc * th\n tl.store(out_ptr + offs, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, subtract_value, pool_kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.subtract_value = subtract_value\n self.pool_k = pool_kernel_size\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n P = self.pool_k\n OH = H // P\n OW = W // P\n out = torch.empty((N, C, OH, OW), device=x.device, dtype=torch.float32)\n TOTAL = out.numel()\n BLOCK = 256\n grid = (triton.cdiv(TOTAL, BLOCK),)\n fused_kernel[grid](x, out, float(self.subtract_value),\n N, C, H, W, OH, OW, P, TOTAL, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv problem (L2_092 Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp, 2.37x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\nfrom triton.language.extra.cuda import libdevice\n\n\n@triton.jit\ndef gn_stats_kernel(x_ptr, mean_ptr, invstd_ptr, CPG, S, GS, eps, BLOCK: tl.constexpr):\n pid = tl.program_id(0) # over n*groups\n base = pid.to(tl.int64) * GS\n acc = 0.0\n acc2 = 0.0\n for off in range(0, GS, BLOCK):\n offs = off + tl.arange(0, BLOCK)\n mask = offs < GS\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n acc += tl.sum(v, axis=0)\n acc2 += tl.sum(v * v, axis=0)\n mean = acc / GS\n var = acc2 / GS - mean * mean\n invstd = 1.0 / tl.sqrt(var + eps)\n tl.store(mean_ptr + pid, mean)\n tl.store(invstd_ptr + pid, invstd)\n\n\n@triton.jit\ndef fused_kernel(x_ptr, mean_ptr, invstd_ptr, gamma_ptr, beta_ptr, out_ptr,\n N, C, S, NS, G, CPG, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n maxv = tl.full((BLOCK,), -3.4e38, tl.float32)\n for c in range(C):\n g = c // CPG\n ng = n * G + g\n xc = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + ng, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + ng, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + c)\n beta = tl.load(beta_ptr + c)\n norm = (xc - mean) * invstd * gamma + beta\n t = libdevice.tanh(norm)\n hs = t * tl.minimum(tl.maximum(t + 3.0, 0.0), 6.0) / 6.0\n res = xc + hs\n maxv = tl.maximum(maxv, res)\n sumexp = tl.zeros((BLOCK,), tl.float32)\n for c in range(C):\n g = c // CPG\n ng = n * G + g\n xc = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + ng, mask=mask, other=0.0)\n invstd = tl.load(invstd_ptr + ng, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + c)\n beta = tl.load(beta_ptr + c)\n norm = (xc - mean) * invstd * gamma + beta\n t = libdevice.tanh(norm)\n hs = t * tl.minimum(tl.maximum(t + 3.0, 0.0), 6.0) / 6.0\n res = xc + hs\n sumexp += tl.exp(res - maxv)\n out = maxv + tl.log(sumexp)\n tl.store(out_ptr + offs, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, groups, eps=1e-5):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.group_norm = nn.GroupNorm(groups, out_channels, eps=eps)\n self.tanh = nn.Tanh()\n self.hard_swish = nn.Hardswish()\n self.groups = groups\n self.eps = eps\n\n def forward(self, x):\n x_conv = self.conv(x).contiguous()\n N, C, H, W = x_conv.shape\n S = H * W\n G = self.groups\n CPG = C // G\n GS = CPG * S\n mean = torch.empty((N * G,), device=x_conv.device, dtype=torch.float32)\n invstd = torch.empty((N * G,), device=x_conv.device, dtype=torch.float32)\n gn_stats_kernel[(N * G,)](x_conv, mean, invstd, CPG, S, GS, self.eps, BLOCK=1024)\n out = torch.empty((N, 1, H, W), device=x_conv.device, dtype=x_conv.dtype)\n NS = N * S\n BLOCK = 256\n grid = (triton.cdiv(NS, BLOCK),)\n fused_kernel[grid](x_conv, mean, invstd,\n self.group_norm.weight, self.group_norm.bias,\n out, N, C, S, NS, G, CPG, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11886, "ctx_sha": "8cbbd063f294068f", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_GroupNorm_Scale_MaxPool_Clamp", "split": "learn", "n_ops": 5, "src_lines": 46, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_085 (Conv2d_GroupNorm_Scale_MaxPool_Clamp), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs convolution, group normalization, scaling, max pooling, and clamping.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, num_groups, scale_shape, maxpool_kernel_size, clamp_min, clamp_max):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.group_norm = nn.GroupNorm(num_groups, out_channels)\n self.scale = nn.Parameter(torch.ones(scale_shape))\n self.maxpool = nn.MaxPool2d(kernel_size=maxpool_kernel_size)\n self.clamp_min = clamp_min\n self.clamp_max = clamp_max\n\n def forward(self, x):\n \"\"\"\n Args:\n x: Input tensor of shape (batch_size, in_channels, height, width).\n Returns:\n Output tensor of shape (batch_size, out_channels, height', width').\n \"\"\"\n x = self.conv(x)\n x = self.group_norm(x)\n x = x * self.scale\n x = self.maxpool(x)\n x = torch.clamp(x, self.clamp_min, self.clamp_max)\n return x\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 128, 128 \nkernel_size = 3\nnum_groups = 16\nscale_shape = (out_channels, 1, 1)\nmaxpool_kernel_size = 4\nclamp_min = 0.0\nclamp_max = 1.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, num_groups, scale_shape, maxpool_kernel_size, clamp_min, clamp_max]```\n"}, "L2_086": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_097 Matmul_BatchNorm_BiasAdd_Divide_Swish, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr, bias_ptr,\n eps, divide_value, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n bias = tl.load(bias_ptr)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n y = (y + bias) / divide_value\n y = y * (1.0 / (1.0 + tl.exp(-y)))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bn_eps=1e-5, bn_momentum=0.1, bias_shape=(1,), divide_value=1.0):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.bn = nn.BatchNorm1d(out_features, eps=bn_eps, momentum=bn_momentum)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.divide_value = divide_value\n self.bn_eps = bn_eps\n self.bn_momentum = bn_momentum\n\n def forward(self, x):\n z = self.matmul(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=1024)\n\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn_momentum).add_(self.bn_momentum * mean)\n unbiased = var * (M / (M - 1))\n self.bn.running_var.mul_(1 - self.bn_momentum).add_(self.bn_momentum * unbiased)\n self.bn.num_batches_tracked.add_(1)\n\n out = torch.empty_like(z)\n bias_flat = self.bias.reshape(-1)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, mean, var, self.bn.weight, self.bn.bias, bias_flat,\n self.bn_eps, self.divide_value, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10909, "ctx_sha": "65e4e510dcc4381c", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Divide_GELU", "split": "learn", "n_ops": 3, "src_lines": 34, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_086 (Matmul_Divide_GELU), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a matrix multiplication, divides by a scalar, and applies GELU activation.\n \"\"\"\n def __init__(self, input_size, output_size, divisor):\n super(Model, self).__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, input_size).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, output_size).\n \"\"\"\n x = self.linear(x)\n x = x / self.divisor\n x = torch.nn.functional.gelu(x)\n return x\n\nbatch_size = 1024\ninput_size = 8192\noutput_size = 8192\ndivisor = 10.0\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size)]\n\ndef get_init_inputs():\n return [input_size, output_size, divisor]```\n"}, "L2_089": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n\n### A verified kernel for a related conv_transpose problem (L2_015 ConvTranspose3d_BatchNorm_Subtract, 1.32x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef out_kernel(x_ptr, out_ptr, scale_ptr, smean_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n sc = tl.load(scale_ptr + pid)\n sm = tl.load(smean_ptr + pid)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n r = sc * (v - sm)\n tl.store(out_ptr + base + idx, r, mask=m)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias=True):\n super(ModelNew, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n total_sum = ssum_nc.sum(0)\n total_sq = ssq_nc.sum(0)\n M = N * S\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps) # (C,)\n\n scale_nc = scale_c.unsqueeze(0).expand(N, C).contiguous().view(N * C)\n smean_nc = (ssum_nc / S).contiguous().view(N * C)\n\n out = torch.empty_like(c2)\n out_kernel[(N * C,)](c2, out, scale_nc, smean_nc, S, BLOCK=1024)\n return out.view(N, C, D, H, W)\n```\n\n### A verified kernel for a related conv_transpose problem (L2_078 ConvTranspose3d_Max_Max_Sum, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, OD, OH, OW, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n idx64 = idx.to(tl.int64)\n mask = idx < total\n\n ow = idx64 % OW\n t = idx64 // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n n = t // OD\n\n d0 = od * 6\n h0 = oh * 6\n w0 = ow * 6\n\n C64 = C.to(tl.int64)\n D64 = D.to(tl.int64)\n H64 = H.to(tl.int64)\n W64 = W.to(tl.int64)\n\n s = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n base_nc = (n * C64 + c) * D64\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for dz in tl.static_range(6):\n iz = d0 + dz\n for dy in tl.static_range(6):\n iy = h0 + dy\n for dx in tl.static_range(6):\n ix = w0 + dx\n off = ((base_nc + iz) * H64 + iy) * W64 + ix\n v = tl.load(x_ptr + off, mask=mask, other=-1e30)\n m = tl.maximum(m, v)\n s += m\n tl.store(out_ptr + idx64, s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n P1D, P1H, P1W = D // 2, H // 2, W // 2\n OD, OH, OW = P1D // 3, P1H // 3, P1W // 3\n out = torch.empty((N, 1, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * OD * OH * OW\n BLOCK = 128\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](c, out, N, C, D, H, W, OD, OH, OW, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10527, "ctx_sha": "9d7b7603b9e7f31f", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max", "split": "learn", "n_ops": 6, "src_lines": 45, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_089 (ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a sequence of operations:\n - ConvTranspose3d\n - MaxPool3d\n - Softmax\n - Subtract\n - Swish\n - Max\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, pool_kernel_size, pool_stride, pool_padding):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.max_pool = nn.MaxPool3d(kernel_size=pool_kernel_size, stride=pool_stride, padding=pool_padding)\n self.subtract = nn.Parameter(torch.randn(out_channels)) # Assuming subtraction is element-wise across channels\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.max_pool(x)\n x = torch.softmax(x, dim=1) # Apply softmax across channels (dim=1)\n x = x - self.subtract.view(1, -1, 1, 1, 1) # Subtract across channels\n x = torch.sigmoid(x) * x # Swish activation\n x = torch.max(x, dim=1)[0] # Max pooling across channels\n return x\n\nbatch_size = 128\nin_channels = 3\nout_channels = 16\ndepth, height, width = 16, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\noutput_padding = 1\npool_kernel_size = 2\npool_stride = 2\npool_padding = 0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, output_padding, pool_kernel_size, pool_stride, pool_padding]```\n"}, "L2_092": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv problems (wrong results or slower than eager)\n- Pitfall: Avoid per-element div/mod index decoding plus per-tap weight gathers in stencil kernels. Use a 2D/3D grid (plane, row-tile, col-tile) and load per-channel weights once per program.\n- Pitfall: For depthwise conv, don'...'s 2D output plane, reuse input rows via shared memory or registers, and make kernel dims constexpr.\n- Pitfall: Fuse post-conv elementwise/reduction ops with following pooling into one kernel instead of separate launches that write and re-read full-size intermediates. Unfused memory-bound epilogues after cuDNN conv easily end up slower than eager.\n- Pitfall: For softmax or reduction over the channel dim of NC... tensors, avoid one program per spatial position with channel-strided loads. Tile a 2D [channels \u00d7 contiguous spatial block] so loads coalesce along the innermost dimension.\n- Pitfall: Don't wrap the cuDNN/PyTorch op and then add a no-op copy kernel. The extra full-tensor read/write plus launch only adds time. Write a real fused kernel that replaces or eliminates work.\n\n### Verified recipes for conv problems (speedups measured by the protected evaluator)\n- [2.43x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv with a direct Triton kernel, since cuDNN's depthwise path is slow. Keep nn.Conv2d only as the parameter holder. Use a 2D grid: program 0 covers one (n, c, oh) output row, decoded with div/mod; program 1 covers a BLOCK=256 tile of contiguous ow, using num_warps=4. Unroll the kernel taps with tl.static_range. Load the scalar weight per tap and coalesced input rows with masks for padding bounds. Accumulate in fp32, fuse the optional bias via a constexpr flag, and use int64 base offsets.\n- [2.25x on conv_depthwise_2D_square_input_asymmetric_kernel] Replace cuDNN depthwise/grouped conv (slow in fp32) with a Triton direct-convolution kernel. Use one flattened 1D grid over NCHW outputs with BLOCK\u2248256. Decode n, c, oh, ow by div/mod so adjacent lanes read adjacent ow for coalesced loads. Loop over the small kernel taps, masking out-of-range input coordinates (handles padding/stride/dilation). Load the per-channel weight scalar and add bias under a constexpr flag. Accumulate in fp32 and compute base offsets in int64. Keep the nn.Conv2d module only for parameter storage.\n- [1.94x on Conv2d_Tanh_Scaling_BiasAdd_Max] Keep the convolution in cuDNN via torch (nn.Conv2d / F.conv2d, contiguous NCHW output). Fuse every following elementwise op (activation, scale, per-channel bias) with the pooling into one Triton kernel. Launch one thread per pooled output element, decoding n,c,oh,ow from a flat index. Loop over the PxP window, load, apply libdevice tanh, scale, add bias, keep a running max, then store once. Use int64 base offsets for large tensors, BLOCK=256 on a 1D grid, and keep all math in fp32.\n- [1.68x on Conv2d_BatchNorm_Scaling] Leave the conv itself to cuDNN via torch. Replace training-mode BatchNorm plus any elementwise tail with two Triton passes. Pass 1 computes per-channel sum and sum-of-squares on a 2D grid (channel, spatial block). Loop over the batch with contiguous, coalesced spatial loads, accumulate in fp32 registers, reduce, then atomic_add per channel. Do mean, var and rsqrt on tiny C-sized tensors in torch, and update running stats with unbiased var. Pass 2 is a flat 1D elementwise kernel with c=(offs//S)%C that fuses normalize, affine and scale. Use int64 offsets.\n- [1.61x on Conv2d_GroupNorm_Scale_MaxPool_Clamp] Keep the conv in cuDNN via torch (F.conv2d / nn.Conv2d). Replace GroupNorm plus the epilogue with two Triton kernels. Kernel 1: one program per (n, group) streams the group's contiguous CPG*H*W slab with BLOCK=1024 vector loads, accumulates sum and sum-of-squares in fp32, and writes mean and invstd. Kernel 2: one thread per pooled output element. Load per-channel affine and scale, normalize each window value, then max-reduce and clamp in registers before a single store. Use int64 base offsets and initialize the max with -3.4e38.\n\n### A verified kernel for a related conv problem (L2_069 Conv2d_HardSwish_ReLU, 1.25x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef act_kernel(x_ptr, out_ptr, TOTAL, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < TOTAL\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n # hardswish: x * clamp(x+3,0,6)/6\n hs = v * tl.minimum(tl.maximum(v + 3.0, 0.0), 6.0) / 6.0\n r = tl.maximum(hs, 0.0)\n tl.store(out_ptr + offs, r, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n out = self.conv(x).contiguous()\n TOTAL = out.numel()\n res = torch.empty_like(out)\n BLOCK = 1024\n grid = (triton.cdiv(TOTAL, BLOCK),)\n act_kernel[grid](out, res, TOTAL, BLOCK=BLOCK)\n return res\n```\n\n### A verified kernel for a related conv problem (L2_025 Conv2d_Min_Tanh_Tanh, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef min_tanh_kernel(x_ptr, out_ptr, C, S, NS, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < NS\n n = offs // S\n s = offs % S\n base = n.to(tl.int64) * C * S + s\n acc = tl.full((BLOCK,), 1e30, tl.float32)\n for c in range(C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=1e30)\n acc = tl.minimum(acc, v)\n acc = tl.minimum(tl.maximum(acc, -1e30), 1e30)\n t = tl.extra.cuda.libdevice.tanh(acc)\n t = tl.extra.cuda.libdevice.tanh(t)\n tl.store(out_ptr + offs, t, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size):\n super(ModelNew, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n\n def forward(self, x):\n x = self.conv(x).contiguous()\n N, C, H, W = x.shape\n S = H * W\n out = torch.empty((N, 1, H, W), device=x.device, dtype=x.dtype)\n NS = N * S\n BLOCK = 1024\n grid = (triton.cdiv(NS, BLOCK),)\n min_tanh_kernel[grid](x, out, C, S, NS, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 8529, "ctx_sha": "7e7b5a732e4def85", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 5, "kernel": 2}, "family": "conv", "level": 2, "name": "Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp", "split": "learn", "n_ops": 6, "src_lines": 42, "max_recipe_speedup": 2.43, "instruction": "KernelBench level 2 problem L2_092 (Conv2d_GroupNorm_Tanh_HardSwish_ResidualAdd_LogSumExp), op family: conv.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a convolution, applies Group Normalization, Tanh, HardSwish, \n Residual Addition, and LogSumExp.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, groups, eps=1e-5):\n super(Model, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.group_norm = nn.GroupNorm(groups, out_channels, eps=eps)\n self.tanh = nn.Tanh()\n self.hard_swish = nn.Hardswish()\n\n def forward(self, x):\n # Convolution\n x_conv = self.conv(x)\n # Group Normalization\n x_norm = self.group_norm(x_conv)\n # Tanh\n x_tanh = self.tanh(x_norm)\n # HardSwish\n x_hard_swish = self.hard_swish(x_tanh)\n # Residual Addition\n x_res = x_conv + x_hard_swish\n # LogSumExp\n x_logsumexp = torch.logsumexp(x_res, dim=1, keepdim=True)\n return x_logsumexp\n\nbatch_size = 128\nin_channels = 8\nout_channels = 64\nheight, width = 128, 128\nkernel_size = 3\ngroups = 16\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, groups]```\n"}, "L2_094": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_053 Gemm_Scaling_Hardtanh_GELU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, hmin, hmax, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask)\n v = v * scaling\n v = tl.minimum(tl.maximum(v, hmin), hmax)\n g = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, g, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor, hardtanh_min, hardtanh_max):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n self.hardtanh_min = hardtanh_min\n self.hardtanh_max = hardtanh_max\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor,\n float(self.hardtanh_min), float(self.hardtanh_max),\n n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9274, "ctx_sha": "2120d7398ea70d7e", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_BiasAdd_Hardtanh_Mish_GroupNorm", "split": "learn", "n_ops": 5, "src_lines": 41, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_094 (Gemm_BiasAdd_Hardtanh_Mish_GroupNorm), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a GEMM, BiasAdd, Hardtanh, Mish, and GroupNorm operations in sequence.\n \"\"\"\n def __init__(self, in_features, out_features, bias_shape, num_groups):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.hardtanh = nn.Hardtanh()\n self.mish = nn.Mish()\n self.groupnorm = nn.GroupNorm(num_groups=num_groups, num_channels=out_features)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.gemm(x)\n x = x + self.bias\n x = self.hardtanh(x)\n x = self.mish(x)\n x = self.groupnorm(x)\n return x\n\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nbias_shape = (out_features,)\nnum_groups = 256\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, bias_shape, num_groups]```\n"}, "L2_095": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_059 Matmul_Swish_Scaling, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef swish_scale_kernel(x_ptr, out_ptr, scale, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = v * s * scale\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = F.linear(x, self.matmul.weight, self.matmul.bias).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n swish_scale_kernel[grid](z, out, float(self.scaling_factor), n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9028, "ctx_sha": "2f70ede7a41b13f4", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Add_Swish_Tanh_GELU_Hardtanh", "split": "learn", "n_ops": 6, "src_lines": 31, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_095 (Matmul_Add_Swish_Tanh_GELU_Hardtanh), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a matrix multiplication, adds a value, applies Swish, Tanh, GELU, and Hardtanh activation functions.\n \"\"\"\n def __init__(self, in_features, out_features, add_value_shape):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.add_value = nn.Parameter(torch.randn(add_value_shape)) \n\n def forward(self, x):\n x = self.matmul(x)\n x = x + self.add_value\n x = torch.sigmoid(x) * x # Swish\n x = torch.tanh(x)\n x = torch.nn.functional.gelu(x) # GELU\n x = torch.nn.functional.hardtanh(x, min_val=-1, max_val=1) # Hardtanh\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nadd_value_shape = (out_features,)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, add_value_shape]```\n"}, "L2_097": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_059 Matmul_Swish_Scaling, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef swish_scale_kernel(x_ptr, out_ptr, scale, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = v * s * scale\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = F.linear(x, self.matmul.weight, self.matmul.bias).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n swish_scale_kernel[grid](z, out, float(self.scaling_factor), n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 8449, "ctx_sha": "9b009b1f2fb08074", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 5, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_BatchNorm_BiasAdd_Divide_Swish", "split": "learn", "n_ops": 5, "src_lines": 35, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_097 (Matmul_BatchNorm_BiasAdd_Divide_Swish), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a matrix multiplication, batch normalization, bias addition, division, and Swish activation.\n \"\"\"\n def __init__(self, in_features, out_features, bn_eps=1e-5, bn_momentum=0.1, bias_shape=(1,), divide_value=1.0):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.bn = nn.BatchNorm1d(out_features, eps=bn_eps, momentum=bn_momentum)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.divide_value = divide_value\n\n def forward(self, x):\n x = self.matmul(x)\n x = self.bn(x)\n x = x + self.bias\n x = x / self.divide_value\n x = x * torch.sigmoid(x)\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nbn_eps = 1e-5\nbn_momentum = 0.1\nbias_shape = (1,)\ndivide_value = 1.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, bn_eps, bn_momentum, bias_shape, divide_value]```\n"}, "L2_098": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_086 Matmul_Divide_GELU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef div_gelu_kernel(x_ptr, out_ptr, inv_div, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * inv_div\n y = 0.5 * v * (1.0 + tl.erf(v * 0.7071067811865476))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, output_size, divisor):\n super().__init__()\n self.linear = nn.Linear(input_size, output_size)\n self.divisor = divisor\n\n def forward(self, x):\n z = self.linear(x)\n z = z.contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n div_gelu_kernel[grid](z, out, 1.0 / self.divisor, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_029 Matmul_Mish_Mish, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef mish2_kernel(x_ptr, out_ptr, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n sp = tl.log(1.0 + tl.exp(v))\n y = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n sp2 = tl.log(1.0 + tl.exp(y))\n z = y * (tl.exp(sp2) - tl.exp(-sp2)) / (tl.exp(sp2) + tl.exp(-sp2))\n tl.store(out_ptr + offs, z, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features):\n super().__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x):\n z = self.linear(x).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n mish2_kernel[grid](z, out, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9033, "ctx_sha": "ac669963ea71a2ce", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_AvgPool_GELU_Scale_Max", "split": "learn", "n_ops": 5, "src_lines": 39, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_098 (Matmul_AvgPool_GELU_Scale_Max), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model implementing the pattern \"Matmul_AvgPool_GELU_Scale_Max\".\n \"\"\"\n def __init__(self, in_features, out_features, pool_kernel_size, scale_factor):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.avg_pool = nn.AvgPool1d(kernel_size=pool_kernel_size)\n self.scale_factor = scale_factor\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.matmul(x)\n x = self.avg_pool(x.unsqueeze(1)).squeeze(1)\n x = torch.nn.functional.gelu(x)\n x = x * self.scale_factor\n x = torch.max(x, dim=1).values\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\npool_kernel_size = 16\nscale_factor = 2.0\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, pool_kernel_size, scale_factor]```\n"}, "L3_001": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "MLP", "split": "learn", "n_ops": 1, "src_lines": 43, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_001 (MLP), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, input_size, layer_sizes, output_size):\n \"\"\"\n :param input_size: The number of input features\n :param layer_sizes: A list of ints containing the sizes of each hidden layer\n :param output_size: The number of output features\n \"\"\"\n super(Model, self).__init__()\n \n layers = []\n current_input_size = input_size\n \n for layer_size in layer_sizes:\n layers.append(nn.Linear(current_input_size, layer_size))\n layers.append(nn.ReLU())\n current_input_size = layer_size\n \n layers.append(nn.Linear(current_input_size, output_size))\n \n self.network = nn.Sequential(*layers)\n \n def forward(self, x):\n \"\"\"\n :param x: The input tensor, shape (batch_size, input_size)\n :return: The output tensor, shape (batch_size, output_size)\n \"\"\"\n return self.network(x)\n\n# Test code\nbatch_size = 128\ninput_size = 16384\nlayer_sizes = [16384, 16384]\noutput_size = 8192\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size)]\n\ndef get_init_inputs():\n return [input_size, layer_sizes, output_size]```\n"}, "L3_002": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "ShallowWideMLP", "split": "learn", "n_ops": 1, "src_lines": 43, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_002 (ShallowWideMLP), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_layer_sizes, output_size):\n \"\"\"\n :param input_size: The number of input features\n :param hidden_layer_sizes: A list of ints containing the sizes of each hidden layer\n :param output_size: The number of output features\n \"\"\"\n super(Model, self).__init__()\n \n layers = []\n current_input_size = input_size\n \n for hidden_size in hidden_layer_sizes:\n layers.append(nn.Linear(current_input_size, hidden_size))\n layers.append(nn.ReLU())\n current_input_size = hidden_size\n \n layers.append(nn.Linear(current_input_size, output_size))\n \n self.network = nn.Sequential(*layers)\n \n def forward(self, x):\n \"\"\"\n :param x: The input tensor, shape (batch_size, input_size)\n :return: The output tensor, shape (batch_size, output_size)\n \"\"\"\n return self.network(x)\n\n# Test code\nbatch_size = 128\ninput_size = 16384\nhidden_layer_sizes = [32768, 32768]\noutput_size = 16384\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size)]\n\ndef get_init_inputs():\n return [input_size, hidden_layer_sizes, output_size]```\n"}, "L3_004": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "LeNet5", "split": "learn", "n_ops": 1, "src_lines": 60, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_004 (LeNet5), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_classes):\n \"\"\"\n LeNet-5 architecture implementation in PyTorch.\n\n :param num_classes: The number of output classes.\n \"\"\"\n super(Model, self).__init__()\n \n # Convolutional layers\n self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5, stride=1)\n self.conv2 = nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5, stride=1)\n \n # Fully connected layers\n self.fc1 = nn.Linear(in_features=16*5*5, out_features=120)\n self.fc2 = nn.Linear(in_features=120, out_features=84)\n self.fc3 = nn.Linear(in_features=84, out_features=num_classes)\n \n def forward(self, x):\n \"\"\"\n Forward pass of the LeNet-5 model.\n\n :param x: The input tensor, shape (batch_size, 1, 32, 32)\n :return: The output tensor, shape (batch_size, num_classes)\n \"\"\"\n # First convolutional layer with ReLU activation and max pooling\n x = F.relu(self.conv1(x))\n x = F.max_pool2d(x, kernel_size=2, stride=2)\n \n # Second convolutional layer with ReLU activation and max pooling\n x = F.relu(self.conv2(x))\n x = F.max_pool2d(x, kernel_size=2, stride=2)\n \n # Flatten the output for the fully connected layers\n x = x.view(-1, 16*5*5)\n \n # First fully connected layer with ReLU activation\n x = F.relu(self.fc1(x))\n \n # Second fully connected layer with ReLU activation\n x = F.relu(self.fc2(x))\n \n # Final fully connected layer\n x = self.fc3(x)\n \n return x\n\n# Test code for the LeNet-5 model (larger batch & image)\nbatch_size = 4096\nnum_classes = 20\n\ndef get_inputs():\n return [torch.rand(batch_size, 1, 32, 32)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L3_009": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "ResNet18", "split": "learn", "n_ops": 1, "src_lines": 113, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_009 (ResNet18), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass BasicBlock(nn.Module):\n expansion = 1\n\n def __init__(self, in_channels, out_channels, stride=1, downsample=None):\n \"\"\"\n :param in_channels: Number of input channels\n :param out_channels: Number of output channels\n :param stride: Stride for the first convolutional layer\n :param downsample: Downsample layer for the shortcut connection\n \"\"\"\n super(BasicBlock, self).__init__()\n self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False)\n self.bn1 = nn.BatchNorm2d(out_channels)\n self.relu = nn.ReLU(inplace=True)\n self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(out_channels)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor, shape (batch_size, in_channels, height, width)\n :return: Output tensor, shape (batch_size, out_channels, height, width)\n \"\"\"\n identity = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n\n if self.downsample is not None:\n identity = self.downsample(x)\n\n out += identity\n out = self.relu(out)\n\n return out\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000):\n \"\"\"\n :param num_classes: Number of output classes\n \"\"\"\n super(Model, self).__init__()\n self.in_channels = 64\n\n self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)\n self.bn1 = nn.BatchNorm2d(64)\n self.relu = nn.ReLU(inplace=True)\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n\n self.layer1 = self._make_layer(BasicBlock, 64, 2, stride=1)\n self.layer2 = self._make_layer(BasicBlock, 128, 2, stride=2)\n self.layer3 = self._make_layer(BasicBlock, 256, 2, stride=2)\n self.layer4 = self._make_layer(BasicBlock, 512, 2, stride=2)\n\n self.avgpool = nn.AdaptiveAvgPool2d((1, 1))\n self.fc = nn.Linear(512 * BasicBlock.expansion, num_classes)\n\n def _make_layer(self, block, out_channels, blocks, stride=1):\n downsample = None\n if stride != 1 or self.in_channels != out_channels * block.expansion:\n downsample = nn.Sequential(\n nn.Conv2d(self.in_channels, out_channels * block.expansion, kernel_size=1, stride=stride, bias=False),\n nn.BatchNorm2d(out_channels * block.expansion),\n )\n\n layers = []\n layers.append(block(self.in_channels, out_channels, stride, downsample))\n self.in_channels = out_channels * block.expansion\n for _ in range(1, blocks):\n layers.append(block(self.in_channels, out_channels))\n\n return nn.Sequential(*layers)\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor, shape (batch_size, 3, height, width)\n :return: Output tensor, shape (batch_size, num_classes)\n \"\"\"\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n\n x = self.layer1(x)\n x = self.layer2(x)\n x = self.layer3(x)\n x = self.layer4(x)\n\n x = self.avgpool(x)\n x = torch.flatten(x, 1)\n x = self.fc(x)\n\n return x\n\n# Test code\nbatch_size = 2\nnum_classes = 1000\ninput_shape = (batch_size, 3, 224, 224)\n\ndef get_inputs():\n return [torch.rand(input_shape)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L3_014": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "DenseNet121DenseBlock", "split": "learn", "n_ops": 1, "src_lines": 51, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_014 (DenseNet121DenseBlock), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_layers: int, num_input_features: int, growth_rate: int):\n \"\"\"\n :param num_layers: The number of layers in the dense block\n :param num_input_features: The number of input feature maps\n :param growth_rate: The growth rate for the dense block (new features added per layer)\n \"\"\"\n super(Model, self).__init__()\n layers = []\n for i in range(num_layers):\n layers.append(self._make_layer(num_input_features + i * growth_rate, growth_rate))\n self.layers = nn.ModuleList(layers)\n\n def _make_layer(self, in_features: int, growth_rate: int):\n \"\"\"\n Creates a single layer with BatchNorm, ReLU, Conv2D, and Dropout.\n \"\"\"\n return nn.Sequential(\n nn.BatchNorm2d(in_features),\n nn.ReLU(inplace=True),\n nn.Conv2d(in_features, growth_rate, kernel_size=3, padding=1, bias=False),\n nn.Dropout(0.0)\n )\n \n def forward(self, x):\n \"\"\"\n :param x: Input tensor of shape (batch_size, num_input_features, height, width)\n :return: Concatenated output tensor with shape (batch_size, num_output_features, height, width)\n \"\"\"\n features = [x]\n for layer in self.layers:\n new_feature = layer(x)\n features.append(new_feature)\n x = torch.cat(features, 1) # Concatenate along channel axis\n return x\n \nbatch_size = 10\nnum_layers = 6\nnum_input_features = 32\ngrowth_rate = 32\nheight, width = 224, 224\n\ndef get_inputs():\n return [torch.rand(batch_size, num_input_features, height, width)]\n\ndef get_init_inputs():\n return [num_layers, num_input_features , growth_rate]```\n"}, "L3_015": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "DenseNet121", "split": "learn", "n_ops": 1, "src_lines": 126, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_015 (DenseNet121), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass DenseBlock(nn.Module):\n def __init__(self, num_layers: int, num_input_features: int, growth_rate: int):\n \"\"\"\n :param num_layers: The number of layers in the dense block\n :param num_input_features: The number of input feature maps\n :param growth_rate: The growth rate for the dense block (new features added per layer)\n \"\"\"\n super(DenseBlock, self).__init__()\n layers = []\n for i in range(num_layers):\n layers.append(self._make_layer(num_input_features + i * growth_rate, growth_rate))\n self.layers = nn.ModuleList(layers)\n\n def _make_layer(self, in_features: int, growth_rate: int):\n \"\"\"\n Creates a single layer with BatchNorm, ReLU, Conv2D, and Dropout.\n \"\"\"\n return nn.Sequential(\n nn.BatchNorm2d(in_features),\n nn.ReLU(inplace=True),\n nn.Conv2d(in_features, growth_rate, kernel_size=3, padding=1, bias=False),\n nn.Dropout(0.0)\n )\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor of shape (batch_size, num_input_features, height, width)\n :return: Concatenated output tensor with shape (batch_size, num_output_features, height, width)\n \"\"\"\n features = [x]\n for layer in self.layers:\n new_feature = layer(x)\n features.append(new_feature)\n x = torch.cat(features, 1) # Concatenate along channel axis\n return x\n\nclass TransitionLayer(nn.Module):\n def __init__(self, num_input_features: int, num_output_features: int):\n \"\"\"\n :param num_input_features: The number of input feature maps\n :param num_output_features: The number of output feature maps\n \"\"\"\n super(TransitionLayer, self).__init__()\n self.transition = nn.Sequential(\n nn.BatchNorm2d(num_input_features),\n nn.ReLU(inplace=True),\n nn.Conv2d(num_input_features, num_output_features, kernel_size=1, bias=False),\n nn.AvgPool2d(kernel_size=2, stride=2)\n )\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor of shape (batch_size, num_input_features, height, width)\n :return: Downsampled tensor with reduced number of feature maps\n \"\"\"\n return self.transition(x)\n\nclass Model(nn.Module):\n def __init__(self, growth_rate: int = 32, num_classes: int = 1000):\n \"\"\"\n :param growth_rate: The growth rate of the DenseNet (new features added per layer)\n :param num_classes: The number of output classes for classification\n \"\"\"\n super(Model, self).__init__()\n\n # Initial convolution and pooling\n self.features = nn.Sequential(\n nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False),\n nn.BatchNorm2d(64),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n )\n\n # Each dense block is followed by a transition layer, except the last one\n num_features = 64\n block_layers = [6, 12, 24, 16] # Corresponding layers in DenseNet121\n\n self.dense_blocks = nn.ModuleList()\n self.transition_layers = nn.ModuleList()\n\n for i, num_layers in enumerate(block_layers):\n block = DenseBlock(num_layers=num_layers, num_input_features=num_features, growth_rate=growth_rate)\n self.dense_blocks.append(block)\n num_features = num_features + num_layers * growth_rate\n\n if i != len(block_layers) - 1:\n transition = TransitionLayer(num_input_features=num_features, num_output_features=num_features // 2)\n self.transition_layers.append(transition)\n num_features = num_features // 2\n\n # Final batch norm and classifier\n self.final_bn = nn.BatchNorm2d(num_features)\n self.classifier = nn.Linear(num_features, num_classes)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n :param x: Input tensor of shape (batch_size, 3, height, width)\n :return: Output tensor of shape (batch_size, num_classes)\n \"\"\"\n x = self.features(x)\n\n for i, block in enumerate(self.dense_blocks):\n x = block(x)\n if i != len(self.dense_blocks) - 1:\n x = self.transition_layers[i](x)\n\n x = self.final_bn(x)\n x = F.relu(x, inplace=True)\n x = F.adaptive_avg_pool2d(x, (1, 1)).view(x.size(0), -1)\n x = self.classifier(x)\n return x\n\n# Testing the DenseNet121 model\nbatch_size = 10\nnum_classes = 10\nheight, width = 224, 224 # Standard input size for DenseNet\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, height, width)]\n\ndef get_init_inputs():\n return [32, num_classes]\n```\n"}, "L3_016": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "DenseNet201", "split": "learn", "n_ops": 1, "src_lines": 126, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_016 (DenseNet201), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass DenseBlock(nn.Module):\n def __init__(self, num_layers: int, num_input_features: int, growth_rate: int):\n \"\"\"\n :param num_layers: The number of layers in the dense block\n :param num_input_features: The number of input feature maps\n :param growth_rate: The growth rate for the dense block (new features added per layer)\n \"\"\"\n super(DenseBlock, self).__init__()\n layers = []\n for i in range(num_layers):\n layers.append(self._make_layer(num_input_features + i * growth_rate, growth_rate))\n self.layers = nn.ModuleList(layers)\n\n def _make_layer(self, in_features: int, growth_rate: int):\n \"\"\"\n Creates a single layer with BatchNorm, ReLU, Conv2D, and Dropout.\n \"\"\"\n return nn.Sequential(\n nn.BatchNorm2d(in_features),\n nn.ReLU(inplace=True),\n nn.Conv2d(in_features, growth_rate, kernel_size=3, padding=1, bias=False),\n nn.Dropout(0.0)\n )\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor of shape (batch_size, num_input_features, height, width)\n :return: Concatenated output tensor with shape (batch_size, num_output_features, height, width)\n \"\"\"\n features = [x]\n for layer in self.layers:\n new_feature = layer(x)\n features.append(new_feature)\n x = torch.cat(features, 1) # Concatenate along channel axis\n return x\n\nclass TransitionLayer(nn.Module):\n def __init__(self, num_input_features: int, num_output_features: int):\n \"\"\"\n :param num_input_features: The number of input feature maps\n :param num_output_features: The number of output feature maps\n \"\"\"\n super(TransitionLayer, self).__init__()\n self.transition = nn.Sequential(\n nn.BatchNorm2d(num_input_features),\n nn.ReLU(inplace=True),\n nn.Conv2d(num_input_features, num_output_features, kernel_size=1, bias=False),\n nn.AvgPool2d(kernel_size=2, stride=2)\n )\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor of shape (batch_size, num_input_features, height, width)\n :return: Downsampled tensor with reduced number of feature maps\n \"\"\"\n return self.transition(x)\n\nclass Model(nn.Module):\n def __init__(self, growth_rate: int = 32, num_classes: int = 1000):\n \"\"\"\n :param growth_rate: The growth rate of the DenseNet (new features added per layer)\n :param num_classes: The number of output classes for classification\n \"\"\"\n super(Model, self).__init__()\n\n # Initial convolution and pooling\n self.features = nn.Sequential(\n nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False),\n nn.BatchNorm2d(64),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n )\n\n # Each dense block is followed by a transition layer, except the last one\n num_features = 64\n block_layers = [6, 12, 48, 32] # Corresponding layers in DenseNet201\n\n self.dense_blocks = nn.ModuleList()\n self.transition_layers = nn.ModuleList()\n\n for i, num_layers in enumerate(block_layers):\n block = DenseBlock(num_layers=num_layers, num_input_features=num_features, growth_rate=growth_rate)\n self.dense_blocks.append(block)\n num_features = num_features + num_layers * growth_rate\n\n if i != len(block_layers) - 1:\n transition = TransitionLayer(num_input_features=num_features, num_output_features=num_features // 2)\n self.transition_layers.append(transition)\n num_features = num_features // 2\n\n # Final batch norm and classifier\n self.final_bn = nn.BatchNorm2d(num_features)\n self.classifier = nn.Linear(num_features, num_classes)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n \"\"\"\n :param x: Input tensor of shape (batch_size, 3, height, width)\n :return: Output tensor of shape (batch_size, num_classes)\n \"\"\"\n x = self.features(x)\n\n for i, block in enumerate(self.dense_blocks):\n x = block(x)\n if i != len(self.dense_blocks) - 1:\n x = self.transition_layers[i](x)\n\n x = self.final_bn(x)\n x = F.relu(x, inplace=True)\n x = F.adaptive_avg_pool2d(x, (1, 1)).view(x.size(0), -1)\n x = self.classifier(x)\n return x\n\n# Testing the DenseNet201 model\nbatch_size = 10\nnum_classes = 10\nheight, width = 224, 224 # Standard input size for DenseNet\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, height, width)]\n\ndef get_init_inputs():\n return [32, num_classes]\n```\n"}, "L3_019": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "MobileNetV1", "split": "learn", "n_ops": 1, "src_lines": 75, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_019 (MobileNetV1), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000, input_channels=3, alpha=1.0):\n \"\"\"\n MobileNetV1 architecture implementation.\n\n :param num_classes: The number of output classes (default: 1000)\n :param input_channels: The number of input channels (default: 3 for RGB images)\n :param alpha: Width multiplier (default: 1.0)\n \"\"\"\n super(Model, self).__init__()\n \n def conv_bn(inp, oup, stride):\n return nn.Sequential(\n nn.Conv2d(inp, oup, 3, stride, 1, bias=False),\n nn.BatchNorm2d(oup),\n nn.ReLU(inplace=True)\n )\n \n def conv_dw(inp, oup, stride):\n return nn.Sequential(\n nn.Conv2d(inp, inp, 3, stride, 1, groups=inp, bias=False),\n nn.BatchNorm2d(inp),\n nn.ReLU(inplace=True),\n \n nn.Conv2d(inp, oup, 1, 1, 0, bias=False),\n nn.BatchNorm2d(oup),\n nn.ReLU(inplace=True),\n )\n \n self.model = nn.Sequential(\n conv_bn(input_channels, int(32 * alpha), 2),\n conv_dw(int(32 * alpha), int(64 * alpha), 1),\n conv_dw(int(64 * alpha), int(128 * alpha), 2),\n conv_dw(int(128 * alpha), int(128 * alpha), 1),\n conv_dw(int(128 * alpha), int(256 * alpha), 2),\n conv_dw(int(256 * alpha), int(256 * alpha), 1),\n conv_dw(int(256 * alpha), int(512 * alpha), 2),\n conv_dw(int(512 * alpha), int(512 * alpha), 1),\n conv_dw(int(512 * alpha), int(512 * alpha), 1),\n conv_dw(int(512 * alpha), int(512 * alpha), 1),\n conv_dw(int(512 * alpha), int(512 * alpha), 1),\n conv_dw(int(512 * alpha), int(512 * alpha), 1),\n conv_dw(int(512 * alpha), int(1024 * alpha), 2),\n conv_dw(int(1024 * alpha), int(1024 * alpha), 1),\n nn.AvgPool2d(7),\n )\n self.fc = nn.Linear(int(1024 * alpha), num_classes)\n \n def forward(self, x):\n \"\"\"\n :param x: The input tensor, shape (batch_size, input_channels, height, width)\n :return: The output tensor, shape (batch_size, num_classes)\n \"\"\"\n x = self.model(x)\n x = x.view(x.size(0), -1)\n x = self.fc(x)\n return x\n\n# Test code\nbatch_size = 10\ninput_channels = 3\nheight = 224\nwidth = 224\nnum_classes = 1000\nalpha = 1.0\n\ndef get_inputs():\n return [torch.rand(batch_size, input_channels, height, width)]\n\ndef get_init_inputs():\n return [num_classes, input_channels, alpha]```\n"}, "L3_020": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "MobileNetV2", "split": "learn", "n_ops": 1, "src_lines": 131, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_020 (MobileNetV2), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000):\n \"\"\"\n MobileNetV2 architecture implementation in PyTorch.\n\n :param num_classes: The number of output classes. Default is 1000.\n \"\"\"\n super(Model, self).__init__()\n \n def _make_divisible(v, divisor, min_value=None):\n \"\"\"\n This function ensures that the number of channels is divisible by the divisor.\n \"\"\"\n if min_value is None:\n min_value = divisor\n new_v = max(min_value, int(v + divisor / 2) // divisor * divisor)\n # Make sure that round down does not go down by more than 10%.\n if new_v < 0.9 * v:\n new_v += divisor\n return new_v\n\n def _inverted_residual_block(inp, oup, stride, expand_ratio):\n \"\"\"\n Inverted Residual Block for MobileNetV2.\n \"\"\"\n hidden_dim = int(inp * expand_ratio)\n use_res_connect = stride == 1 and inp == oup\n\n layers = []\n if expand_ratio != 1:\n # Pointwise convolution\n layers.append(nn.Conv2d(inp, hidden_dim, 1, 1, 0, bias=False))\n layers.append(nn.BatchNorm2d(hidden_dim))\n layers.append(nn.ReLU6(inplace=True))\n\n layers.extend([\n # Depthwise convolution\n nn.Conv2d(hidden_dim, hidden_dim, 3, stride, 1, groups=hidden_dim, bias=False),\n nn.BatchNorm2d(hidden_dim),\n nn.ReLU6(inplace=True),\n # Pointwise linear convolution\n nn.Conv2d(hidden_dim, oup, 1, 1, 0, bias=False),\n nn.BatchNorm2d(oup),\n ])\n\n if use_res_connect:\n return nn.Sequential(*layers), True\n else:\n return nn.Sequential(*layers), False\n\n # MobileNetV2 architecture\n input_channel = 32\n last_channel = 1280\n inverted_residual_setting = [\n # t, c, n, s\n [1, 16, 1, 1],\n [6, 24, 2, 2],\n [6, 32, 3, 2],\n [6, 64, 4, 2],\n [6, 96, 3, 1],\n [6, 160, 3, 2],\n [6, 320, 1, 1],\n ]\n\n # Building first layer\n features = [nn.Conv2d(3, input_channel, 3, 2, 1, bias=False),\n nn.BatchNorm2d(input_channel),\n nn.ReLU6(inplace=True)]\n\n # Building inverted residual blocks\n for t, c, n, s in inverted_residual_setting:\n output_channel = _make_divisible(c, 8)\n for i in range(n):\n stride = s if i == 0 else 1\n features.append(_inverted_residual_block(input_channel, output_channel, stride, expand_ratio=t)[0])\n input_channel = output_channel\n\n # Building last several layers\n features.append(nn.Conv2d(input_channel, last_channel, 1, 1, 0, bias=False))\n features.append(nn.BatchNorm2d(last_channel))\n features.append(nn.ReLU6(inplace=True))\n\n # Final layer\n features.append(nn.AdaptiveAvgPool2d((1, 1)))\n\n self.features = nn.Sequential(*features)\n\n # Linear layer\n self.classifier = nn.Sequential(\n nn.Dropout(0.0),\n nn.Linear(last_channel, num_classes),\n )\n\n # Weight initialization\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n nn.init.kaiming_normal_(m.weight, mode='fan_out')\n if m.bias is not None:\n nn.init.zeros_(m.bias)\n elif isinstance(m, nn.BatchNorm2d):\n nn.init.ones_(m.weight)\n nn.init.zeros_(m.bias)\n elif isinstance(m, nn.Linear):\n nn.init.normal_(m.weight, 0, 0.01)\n nn.init.zeros_(m.bias)\n\n def forward(self, x):\n \"\"\"\n Forward pass of the MobileNetV2 model.\n\n :param x: The input tensor, shape (batch_size, 3, 224, 224)\n :return: The output tensor, shape (batch_size, num_classes)\n \"\"\"\n x = self.features(x)\n x = x.view(x.size(0), -1)\n x = self.classifier(x)\n return x\n\n# Test code\nbatch_size = 10\nnum_classes = 1000\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, 224, 224)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L3_021": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "EfficientNetMBConv", "split": "learn", "n_ops": 1, "src_lines": 71, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_021 (EfficientNetMBConv), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, expand_ratio):\n \"\"\"\n MBConv block implementation.\n\n :param in_channels: Number of input channels.\n :param out_channels: Number of output channels.\n :param kernel_size: Kernel size for the depthwise convolution.\n :param stride: Stride for the depthwise convolution.\n :param expand_ratio: Expansion ratio for the intermediate channels.\n \"\"\"\n super(Model, self).__init__()\n \n self.use_residual = (stride == 1 and in_channels == out_channels)\n hidden_dim = in_channels * expand_ratio\n \n if expand_ratio != 1:\n self.expand_conv = nn.Sequential(\n nn.Conv2d(in_channels, hidden_dim, kernel_size=1, stride=1, padding=0, bias=False),\n nn.BatchNorm2d(hidden_dim),\n nn.ReLU6(inplace=True)\n )\n \n self.depthwise_conv = nn.Sequential(\n nn.Conv2d(hidden_dim, hidden_dim, kernel_size=kernel_size, stride=stride, padding=(kernel_size-1)//2, groups=hidden_dim, bias=False),\n nn.BatchNorm2d(hidden_dim),\n nn.ReLU6(inplace=True)\n )\n \n self.project_conv = nn.Sequential(\n nn.Conv2d(hidden_dim, out_channels, kernel_size=1, stride=1, padding=0, bias=False),\n nn.BatchNorm2d(out_channels)\n )\n \n def forward(self, x):\n \"\"\"\n Forward pass of the MBConv block.\n\n :param x: The input tensor, shape (batch_size, in_channels, H, W)\n :return: The output tensor, shape (batch_size, out_channels, H', W')\n \"\"\"\n identity = x\n \n if hasattr(self, 'expand_conv'):\n x = self.expand_conv(x)\n \n x = self.depthwise_conv(x)\n x = self.project_conv(x)\n \n if self.use_residual:\n x += identity\n \n return x\n\n# Test code\nbatch_size = 10\nin_channels = 112\nout_channels = 192\nkernel_size = 5\nstride = 2\nexpand_ratio = 6\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, 224, 224)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, expand_ratio]```\n"}, "L3_022": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "EfficientNetB0", "split": "learn", "n_ops": 1, "src_lines": 132, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_022 (EfficientNetB0), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000):\n \"\"\"\n EfficientNetB0 architecture implementation in PyTorch.\n\n :param num_classes: The number of output classes (default is 1000 for ImageNet).\n \"\"\"\n super(Model, self).__init__()\n \n # Initial convolutional layer\n self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=2, padding=1, bias=False)\n self.bn1 = nn.BatchNorm2d(32)\n \n # MBConv blocks\n self.blocks = nn.Sequential(\n # MBConv1 (32, 16, 1, 1)\n MBConv(32, 16, kernel_size=3, stride=1, expand_ratio=1),\n # MBConv6 (16, 24, 2, 6)\n MBConv(16, 24, kernel_size=3, stride=2, expand_ratio=6),\n # MBConv6 (24, 24, 1, 6)\n MBConv(24, 24, kernel_size=3, stride=1, expand_ratio=6),\n # MBConv6 (24, 40, 2, 6)\n MBConv(24, 40, kernel_size=5, stride=2, expand_ratio=6),\n # MBConv6 (40, 40, 1, 6)\n MBConv(40, 40, kernel_size=5, stride=1, expand_ratio=6),\n # MBConv6 (40, 80, 2, 6)\n MBConv(40, 80, kernel_size=3, stride=2, expand_ratio=6),\n # MBConv6 (80, 80, 1, 6)\n MBConv(80, 80, kernel_size=3, stride=1, expand_ratio=6),\n # MBConv6 (80, 112, 1, 6)\n MBConv(80, 112, kernel_size=5, stride=1, expand_ratio=6),\n # MBConv6 (112, 112, 1, 6)\n MBConv(112, 112, kernel_size=5, stride=1, expand_ratio=6),\n # MBConv6 (112, 192, 2, 6)\n MBConv(112, 192, kernel_size=5, stride=2, expand_ratio=6),\n # MBConv6 (192, 192, 1, 6)\n MBConv(192, 192, kernel_size=5, stride=1, expand_ratio=6),\n # MBConv6 (192, 192, 1, 6)\n MBConv(192, 192, kernel_size=5, stride=1, expand_ratio=6),\n # MBConv6 (192, 320, 1, 6)\n MBConv(192, 320, kernel_size=3, stride=1, expand_ratio=6)\n )\n \n # Final convolutional layer\n self.conv2 = nn.Conv2d(320, 1280, kernel_size=1, stride=1, padding=0, bias=False)\n self.bn2 = nn.BatchNorm2d(1280)\n \n # Fully connected layer\n self.fc = nn.Linear(1280, num_classes)\n \n def forward(self, x):\n \"\"\"\n Forward pass of the EfficientNetB0 model.\n\n :param x: The input tensor, shape (batch_size, 3, 224, 224)\n :return: The output tensor, shape (batch_size, num_classes)\n \"\"\"\n x = F.relu(self.bn1(self.conv1(x)))\n x = self.blocks(x)\n x = F.relu(self.bn2(self.conv2(x)))\n x = F.adaptive_avg_pool2d(x, (1, 1))\n x = x.view(x.size(0), -1)\n x = self.fc(x)\n return x\n\nclass MBConv(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, expand_ratio):\n \"\"\"\n MBConv block implementation.\n\n :param in_channels: Number of input channels.\n :param out_channels: Number of output channels.\n :param kernel_size: Kernel size for the depthwise convolution.\n :param stride: Stride for the depthwise convolution.\n :param expand_ratio: Expansion ratio for the intermediate channels.\n \"\"\"\n super(MBConv, self).__init__()\n \n self.use_residual = (stride == 1 and in_channels == out_channels)\n hidden_dim = in_channels * expand_ratio\n \n if expand_ratio != 1:\n self.expand_conv = nn.Sequential(\n nn.Conv2d(in_channels, hidden_dim, kernel_size=1, stride=1, padding=0, bias=False),\n nn.BatchNorm2d(hidden_dim),\n nn.ReLU6(inplace=True)\n )\n \n self.depthwise_conv = nn.Sequential(\n nn.Conv2d(hidden_dim, hidden_dim, kernel_size=kernel_size, stride=stride, padding=(kernel_size-1)//2, groups=hidden_dim, bias=False),\n nn.BatchNorm2d(hidden_dim),\n nn.ReLU6(inplace=True)\n )\n \n self.project_conv = nn.Sequential(\n nn.Conv2d(hidden_dim, out_channels, kernel_size=1, stride=1, padding=0, bias=False),\n nn.BatchNorm2d(out_channels)\n )\n \n def forward(self, x):\n \"\"\"\n Forward pass of the MBConv block.\n\n :param x: The input tensor, shape (batch_size, in_channels, H, W)\n :return: The output tensor, shape (batch_size, out_channels, H', W')\n \"\"\"\n identity = x\n \n if hasattr(self, 'expand_conv'):\n x = self.expand_conv(x)\n \n x = self.depthwise_conv(x)\n x = self.project_conv(x)\n \n if self.use_residual:\n x += identity\n \n return x\n\n# Test code\nbatch_size = 10\nnum_classes = 1000\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, 224, 224)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L3_024": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "EfficientNetB2", "split": "learn", "n_ops": 1, "src_lines": 96, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_024 (EfficientNetB2), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000):\n \"\"\"\n EfficientNetB2 architecture implementation.\n\n :param num_classes: The number of output classes (default is 1000 for ImageNet).\n \"\"\"\n super(Model, self).__init__()\n \n # Define the EfficientNetB2 architecture components\n self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=2, padding=1, bias=False)\n self.bn1 = nn.BatchNorm2d(32)\n self.relu = nn.ReLU(inplace=True)\n \n # Define the MBConv blocks\n self.mbconv1 = self._make_mbconv_block(32, 96, 1, 3)\n self.mbconv2 = self._make_mbconv_block(96, 144, 2, 6)\n self.mbconv3 = self._make_mbconv_block(144, 192, 2, 6)\n self.mbconv4 = self._make_mbconv_block(192, 288, 2, 6)\n self.mbconv5 = self._make_mbconv_block(288, 384, 1, 6)\n \n # Final layers\n self.conv_final = nn.Conv2d(384, 1408, kernel_size=1, stride=1, padding=0, bias=False)\n self.bn_final = nn.BatchNorm2d(1408)\n self.avgpool = nn.AdaptiveAvgPool2d((1, 1))\n self.fc = nn.Linear(1408, num_classes)\n \n def _make_mbconv_block(self, in_channels, out_channels, stride, expand_ratio):\n \"\"\"\n Helper function to create a MBConv block.\n\n :param in_channels: Number of input channels.\n :param out_channels: Number of output channels.\n :param stride: Stride for the depthwise convolution.\n :param expand_ratio: Expansion ratio for the MBConv block.\n :return: A sequential container of layers forming the MBConv block.\n \"\"\"\n layers = []\n expanded_channels = in_channels * expand_ratio\n \n # Expansion phase\n if expand_ratio != 1:\n layers.append(nn.Conv2d(in_channels, expanded_channels, kernel_size=1, stride=1, padding=0, bias=False))\n layers.append(nn.BatchNorm2d(expanded_channels))\n layers.append(nn.ReLU(inplace=True))\n \n # Depthwise convolution\n layers.append(nn.Conv2d(expanded_channels, expanded_channels, kernel_size=3, stride=stride, padding=1, groups=expanded_channels, bias=False))\n layers.append(nn.BatchNorm2d(expanded_channels))\n layers.append(nn.ReLU(inplace=True))\n \n # Squeeze and Excitation\n layers.append(nn.AdaptiveAvgPool2d((1, 1)))\n layers.append(nn.Conv2d(expanded_channels, expanded_channels // 4, kernel_size=1, stride=1, padding=0, bias=False))\n layers.append(nn.ReLU(inplace=True))\n layers.append(nn.Conv2d(expanded_channels // 4, expanded_channels, kernel_size=1, stride=1, padding=0, bias=False))\n layers.append(nn.Sigmoid())\n \n # Output phase\n layers.append(nn.Conv2d(expanded_channels, out_channels, kernel_size=1, stride=1, padding=0, bias=False))\n layers.append(nn.BatchNorm2d(out_channels))\n \n return nn.Sequential(*layers)\n \n def forward(self, x):\n \"\"\"\n Forward pass of the EfficientNetB2 model.\n\n :param x: The input tensor, shape (batch_size, 3, 224, 224)\n :return: The output tensor, shape (batch_size, num_classes)\n \"\"\"\n x = self.relu(self.bn1(self.conv1(x)))\n x = self.mbconv1(x)\n x = self.mbconv2(x)\n x = self.mbconv3(x)\n x = self.mbconv4(x)\n x = self.mbconv5(x)\n x = self.relu(self.bn_final(self.conv_final(x)))\n x = self.avgpool(x)\n x = torch.flatten(x, 1)\n x = self.fc(x)\n return x\n\n# Test code\nbatch_size = 2\nnum_classes = 1000\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, 224, 224)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L3_026": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "ShuffleNet", "split": "learn", "n_ops": 1, "src_lines": 163, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_026 (ShuffleNet), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass ShuffleNetUnit(nn.Module):\n def __init__(self, in_channels, out_channels, groups=3):\n \"\"\"\n ShuffleNet unit implementation.\n\n :param in_channels: Number of input channels.\n :param out_channels: Number of output channels.\n :param groups: Number of groups for group convolution.\n \"\"\"\n super(ShuffleNetUnit, self).__init__()\n \n # Ensure the output channels are divisible by groups\n assert out_channels % 4 == 0\n mid_channels = out_channels // 4\n \n # First 1x1 group convolution\n self.conv1 = nn.Conv2d(in_channels, mid_channels, kernel_size=1, stride=1, padding=0, groups=groups, bias=False)\n self.bn1 = nn.BatchNorm2d(mid_channels)\n \n # Depthwise 3x3 convolution\n self.conv2 = nn.Conv2d(mid_channels, mid_channels, kernel_size=3, stride=1, padding=1, groups=mid_channels, bias=False)\n self.bn2 = nn.BatchNorm2d(mid_channels)\n \n # Second 1x1 group convolution\n self.conv3 = nn.Conv2d(mid_channels, out_channels, kernel_size=1, stride=1, padding=0, groups=groups, bias=False)\n self.bn3 = nn.BatchNorm2d(out_channels)\n \n # Shuffle operation\n self.shuffle = ChannelShuffle(groups)\n \n # Shortcut connection if input and output channels are the same\n if in_channels == out_channels:\n self.shortcut = nn.Sequential()\n else:\n self.shortcut = nn.Sequential(\n nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1, padding=0, bias=False),\n nn.BatchNorm2d(out_channels)\n )\n \n def forward(self, x):\n \"\"\"\n Forward pass for ShuffleNet unit.\n\n :param x: Input tensor, shape (batch_size, in_channels, height, width)\n :return: Output tensor, shape (batch_size, out_channels, height, width)\n \"\"\"\n out = F.relu(self.bn1(self.conv1(x)))\n out = self.bn2(self.conv2(out))\n out = self.shuffle(out)\n out = F.relu(self.bn3(self.conv3(out)))\n \n out += self.shortcut(x)\n return out\n\nclass ChannelShuffle(nn.Module):\n def __init__(self, groups):\n \"\"\"\n Channel shuffle operation.\n\n :param groups: Number of groups for shuffling.\n \"\"\"\n super(ChannelShuffle, self).__init__()\n self.groups = groups\n \n def forward(self, x):\n \"\"\"\n Forward pass for channel shuffle.\n\n :param x: Input tensor, shape (batch_size, channels, height, width)\n :return: Output tensor, shape (batch_size, channels, height, width)\n \"\"\"\n batch_size, channels, height, width = x.size()\n channels_per_group = channels // self.groups\n \n # Reshape\n x = x.view(batch_size, self.groups, channels_per_group, height, width)\n \n # Transpose\n x = x.transpose(1, 2).contiguous()\n \n # Flatten\n x = x.view(batch_size, -1, height, width)\n \n return x\n\nclass Model(nn.Module):\n def __init__(self, num_classes=1000, groups=3, stages_repeats=[3, 7, 3], stages_out_channels=[24, 240, 480, 960]):\n \"\"\"\n ShuffleNet architecture.\n\n :param num_classes: Number of output classes.\n :param groups: Number of groups for group convolution.\n :param stages_repeats: List of ints specifying the number of repeats for each stage.\n :param stages_out_channels: List of ints specifying the output channels for each stage.\n \"\"\"\n super(Model, self).__init__()\n \n self.conv1 = nn.Conv2d(3, stages_out_channels[0], kernel_size=3, stride=2, padding=1, bias=False)\n self.bn1 = nn.BatchNorm2d(stages_out_channels[0])\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n \n self.stage2 = self._make_stage(stages_out_channels[0], stages_out_channels[1], stages_repeats[0], groups)\n self.stage3 = self._make_stage(stages_out_channels[1], stages_out_channels[2], stages_repeats[1], groups)\n self.stage4 = self._make_stage(stages_out_channels[2], stages_out_channels[3], stages_repeats[2], groups)\n \n self.conv5 = nn.Conv2d(stages_out_channels[3], 1024, kernel_size=1, stride=1, padding=0, bias=False)\n self.bn5 = nn.BatchNorm2d(1024)\n \n self.fc = nn.Linear(1024, num_classes)\n \n def _make_stage(self, in_channels, out_channels, repeats, groups):\n \"\"\"\n Helper function to create a stage of ShuffleNet units.\n\n :param in_channels: Number of input channels.\n :param out_channels: Number of output channels.\n :param repeats: Number of ShuffleNet units in the stage.\n :param groups: Number of groups for group convolution.\n :return: nn.Sequential containing the stage.\n \"\"\"\n layers = []\n layers.append(ShuffleNetUnit(in_channels, out_channels, groups))\n for _ in range(1, repeats):\n layers.append(ShuffleNetUnit(out_channels, out_channels, groups))\n return nn.Sequential(*layers)\n \n def forward(self, x):\n \"\"\"\n Forward pass for ShuffleNet.\n\n :param x: Input tensor, shape (batch_size, 3, height, width)\n :return: Output tensor, shape (batch_size, num_classes)\n \"\"\"\n x = F.relu(self.bn1(self.conv1(x)))\n x = self.maxpool(x)\n \n x = self.stage2(x)\n x = self.stage3(x)\n x = self.stage4(x)\n \n x = F.relu(self.bn5(self.conv5(x)))\n x = F.adaptive_avg_pool2d(x, (1, 1))\n x = x.view(x.size(0), -1)\n x = self.fc(x)\n \n return x\n\n# Test code\nbatch_size = 10\ninput_channels = 3\nheight = 224\nwidth = 224\nnum_classes = 1000\n\ndef get_inputs():\n return [torch.rand(batch_size, input_channels, height, width)]\n\ndef get_init_inputs():\n return [num_classes]```\n"}, "L3_027": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.", "ctx_chars": 2932, "ctx_sha": "e5406f0f931ba9ce", "dropped_same_problem": 1, "kinds": {"lesson": 14, "pitfall": 4}, "family": "network", "level": 3, "name": "RegNet", "split": "learn", "n_ops": 1, "src_lines": 73, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_027 (RegNet), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n \"\"\"\n :param input_channels: int, Number of input channels for the first layer\n :param stages: int, Number of stages in the RegNet architecture\n :param block_widths: List[int], Width (number of channels) for each block in the stages\n :param output_classes: int, Number of output classes for classification\n \"\"\"\n super(Model, self).__init__()\n\n self.stages = stages\n self.block_widths = block_widths\n \n layers = []\n current_channels = input_channels\n \n # Construct the stages with their respective blocks\n for i in range(stages):\n layers.append(self._make_stage(current_channels, block_widths[i]))\n current_channels = block_widths[i]\n \n self.feature_extractor = nn.Sequential(*layers)\n \n # Final fully connected layer for classification\n self.fc = nn.Linear(block_widths[-1], output_classes)\n \n def _make_stage(self, in_channels, out_channels):\n \"\"\"\n Creates a simple block for each stage.\n :param in_channels: int, number of input channels\n :param out_channels: int, number of output channels\n :return: nn.Sequential block with convolutional layers\n \"\"\"\n return nn.Sequential(\n nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1),\n nn.BatchNorm2d(out_channels),\n nn.ReLU(),\n nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1),\n nn.BatchNorm2d(out_channels),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2)\n )\n\n def forward(self, x):\n \"\"\"\n Forward pass through the RegNet model.\n :param x: torch.Tensor of shape (batch_size, input_channels, height, width)\n :return: torch.Tensor of shape (batch_size, output_classes)\n \"\"\"\n x = self.feature_extractor(x)\n x = torch.mean(x, dim=[2, 3]) # Global Average Pooling\n x = self.fc(x)\n return x\n\n# Test code for the RegNet model\nbatch_size = 8\ninput_channels = 3\nimage_height, image_width = 224, 224\nstages = 3\nblock_widths = [64, 128, 256]\noutput_classes = 10\n\ndef get_inputs():\n \"\"\" Generates random input tensor of shape (batch_size, input_channels, height, width) \"\"\"\n return [torch.rand(batch_size, input_channels, image_height, image_width)]\n\ndef get_init_inputs():\n \"\"\" Initializes model parameters \"\"\"\n return [input_channels, stages, block_widths, output_classes]```\n"}, "L3_029": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "SwinMLP", "split": "learn", "n_ops": 1, "src_lines": 403, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_029 (SwinMLP), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\n# --------------------------------------------------------\n# Swin Transformer\n# Copyright (c) 2021 Microsoft\n# Licensed under The MIT License [see LICENSE for details]\n# Written by Ze Liu\n# --------------------------------------------------------\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom itertools import repeat\nimport collections.abc\n\nclass Mlp(nn.Module):\n def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.):\n super().__init__()\n out_features = out_features or in_features\n hidden_features = hidden_features or in_features\n self.fc1 = nn.Linear(in_features, hidden_features)\n self.act = act_layer()\n self.fc2 = nn.Linear(hidden_features, out_features)\n self.drop = nn.Dropout(drop)\n\n def forward(self, x):\n x = self.fc1(x)\n x = self.act(x)\n x = self.drop(x)\n x = self.fc2(x)\n x = self.drop(x)\n return x\n\n\ndef window_partition(x, window_size):\n \"\"\"\n Args:\n x: (B, H, W, C)\n window_size (int): window size\n\n Returns:\n windows: (num_windows*B, window_size, window_size, C)\n \"\"\"\n B, H, W, C = x.shape\n x = x.view(B, H // window_size, window_size, W // window_size, window_size, C)\n windows = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(-1, window_size, window_size, C)\n return windows\n\n\ndef window_reverse(windows, window_size, H, W):\n \"\"\"\n Args:\n windows: (num_windows*B, window_size, window_size, C)\n window_size (int): Window size\n H (int): Height of image\n W (int): Width of image\n\n Returns:\n x: (B, H, W, C)\n \"\"\"\n B = int(windows.shape[0] / (H * W / window_size / window_size))\n x = windows.view(B, H // window_size, W // window_size, window_size, window_size, -1)\n x = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(B, H, W, -1)\n return x\n\n\nclass SwinMLPBlock(nn.Module):\n r\"\"\" Swin MLP Block.\n\n Args:\n dim (int): Number of input channels.\n input_resolution (tuple[int]): Input resulotion.\n num_heads (int): Number of attention heads.\n window_size (int): Window size.\n shift_size (int): Shift size for SW-MSA.\n mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.\n drop (float, optional): Dropout rate. Default: 0.0\n drop_path (float, optional): Stochastic depth rate. Default: 0.0\n act_layer (nn.Module, optional): Activation layer. Default: nn.GELU\n norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm\n \"\"\"\n\n def __init__(self, dim, input_resolution, num_heads, window_size=7, shift_size=0,\n mlp_ratio=4., drop=0., drop_path=0.,\n act_layer=nn.GELU, norm_layer=nn.LayerNorm):\n super().__init__()\n self.dim = dim\n self.input_resolution = input_resolution\n self.num_heads = num_heads\n self.window_size = window_size\n self.shift_size = shift_size\n self.mlp_ratio = mlp_ratio\n if min(self.input_resolution) <= self.window_size:\n # if window size is larger than input resolution, we don't partition windows\n self.shift_size = 0\n self.window_size = min(self.input_resolution)\n assert 0 <= self.shift_size < self.window_size, \"shift_size must in 0-window_size\"\n\n self.padding = [self.window_size - self.shift_size, self.shift_size,\n self.window_size - self.shift_size, self.shift_size] # P_l,P_r,P_t,P_b\n\n self.norm1 = norm_layer(dim)\n # use group convolution to implement multi-head MLP\n self.spatial_mlp = nn.Conv1d(self.num_heads * self.window_size ** 2,\n self.num_heads * self.window_size ** 2,\n kernel_size=1,\n groups=self.num_heads)\n\n self.drop_path = nn.Identity()\n self.norm2 = norm_layer(dim)\n mlp_hidden_dim = int(dim * mlp_ratio)\n self.mlp = Mlp(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)\n\n def forward(self, x):\n H, W = self.input_resolution\n B, L, C = x.shape\n assert L == H * W, \"input feature has wrong size\"\n\n shortcut = x\n x = self.norm1(x)\n x = x.view(B, H, W, C)\n\n # shift\n if self.shift_size > 0:\n P_l, P_r, P_t, P_b = self.padding\n shifted_x = F.pad(x, [0, 0, P_l, P_r, P_t, P_b], \"constant\", 0)\n else:\n shifted_x = x\n _, _H, _W, _ = shifted_x.shape\n\n # partition windows\n x_windows = window_partition(shifted_x, self.window_size) # nW*B, window_size, window_size, C\n x_windows = x_windows.view(-1, self.window_size * self.window_size, C) # nW*B, window_size*window_size, C\n\n # Window/Shifted-Window Spatial MLP\n x_windows_heads = x_windows.view(-1, self.window_size * self.window_size, self.num_heads, C // self.num_heads)\n x_windows_heads = x_windows_heads.transpose(1, 2) # nW*B, nH, window_size*window_size, C//nH\n x_windows_heads = x_windows_heads.reshape(-1, self.num_heads * self.window_size * self.window_size,\n C // self.num_heads)\n spatial_mlp_windows = self.spatial_mlp(x_windows_heads) # nW*B, nH*window_size*window_size, C//nH\n spatial_mlp_windows = spatial_mlp_windows.view(-1, self.num_heads, self.window_size * self.window_size,\n C // self.num_heads).transpose(1, 2)\n spatial_mlp_windows = spatial_mlp_windows.reshape(-1, self.window_size * self.window_size, C)\n\n # merge windows\n spatial_mlp_windows = spatial_mlp_windows.reshape(-1, self.window_size, self.window_size, C)\n shifted_x = window_reverse(spatial_mlp_windows, self.window_size, _H, _W) # B H' W' C\n\n # reverse shift\n if self.shift_size > 0:\n P_l, P_r, P_t, P_b = self.padding\n x = shifted_x[:, P_t:-P_b, P_l:-P_r, :].contiguous()\n else:\n x = shifted_x\n x = x.view(B, H * W, C)\n\n # FFN\n x = shortcut + self.drop_path(x)\n x = x + self.drop_path(self.mlp(self.norm2(x)))\n\n return x\n\n\nclass PatchMerging(nn.Module):\n r\"\"\" Patch Merging Layer.\n\n Args:\n input_resolution (tuple[int]): Resolution of input feature.\n dim (int): Number of input channels.\n norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm\n \"\"\"\n\n def __init__(self, input_resolution, dim, norm_layer=nn.LayerNorm):\n super().__init__()\n self.input_resolution = input_resolution\n self.dim = dim\n self.reduction = nn.Linear(4 * dim, 2 * dim, bias=False)\n self.norm = norm_layer(4 * dim)\n\n def forward(self, x):\n \"\"\"\n x: B, H*W, C\n \"\"\"\n H, W = self.input_resolution\n B, L, C = x.shape\n assert L == H * W, \"input feature has wrong size\"\n assert H % 2 == 0 and W % 2 == 0, f\"x size ({H}*{W}) are not even.\"\n\n x = x.view(B, H, W, C)\n\n x0 = x[:, 0::2, 0::2, :] # B H/2 W/2 C\n x1 = x[:, 1::2, 0::2, :] # B H/2 W/2 C\n x2 = x[:, 0::2, 1::2, :] # B H/2 W/2 C\n x3 = x[:, 1::2, 1::2, :] # B H/2 W/2 C\n x = torch.cat([x0, x1, x2, x3], -1) # B H/2 W/2 4*C\n x = x.view(B, -1, 4 * C) # B H/2*W/2 4*C\n\n x = self.norm(x)\n x = self.reduction(x)\n\n return x\n\nclass BasicLayer(nn.Module):\n \"\"\" A basic Swin MLP layer for one stage.\n\n Args:\n dim (int): Number of input channels.\n input_resolution (tuple[int]): Input resolution.\n depth (int): Number of blocks.\n num_heads (int): Number of attention heads.\n window_size (int): Local window size.\n mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.\n drop (float, optional): Dropout rate. Default: 0.0\n drop_path (float | tuple[float], optional): Stochastic depth rate. Default: 0.0\n norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm\n downsample (nn.Module | None, optional): Downsample layer at the end of the layer. Default: None\n use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False.\n \"\"\"\n\n def __init__(self, dim, input_resolution, depth, num_heads, window_size,\n mlp_ratio=4., drop=0., drop_path=0.,\n norm_layer=nn.LayerNorm, downsample=None, use_checkpoint=False):\n\n super().__init__()\n self.dim = dim\n self.input_resolution = input_resolution\n self.depth = depth\n self.use_checkpoint = use_checkpoint\n\n # build blocks\n self.blocks = nn.ModuleList([\n SwinMLPBlock(dim=dim, input_resolution=input_resolution,\n num_heads=num_heads, window_size=window_size,\n shift_size=0 if (i % 2 == 0) else window_size // 2,\n mlp_ratio=mlp_ratio,\n drop=drop,\n drop_path=drop_path[i] if isinstance(drop_path, list) else drop_path,\n norm_layer=norm_layer)\n for i in range(depth)])\n\n # patch merging layer\n if downsample is not None:\n self.downsample = downsample(input_resolution, dim=dim, norm_layer=norm_layer)\n else:\n self.downsample = None\n\n def forward(self, x):\n for blk in self.blocks:\n if self.use_checkpoint:\n x = checkpoint.checkpoint(blk, x)\n else:\n x = blk(x)\n if self.downsample is not None:\n x = self.downsample(x)\n return x\n\ndef _ntuple(n):\n def parse(x):\n if isinstance(x, collections.abc.Iterable) and not isinstance(x, str):\n return tuple(x)\n return tuple(repeat(x, n))\n return parse\nto_2tuple = _ntuple(2)\n\nclass PatchEmbed(nn.Module):\n r\"\"\" Image to Patch Embedding\n\n Args:\n img_size (int): Image size. Default: 224.\n patch_size (int): Patch token size. Default: 4.\n in_chans (int): Number of input image channels. Default: 3.\n embed_dim (int): Number of linear projection output channels. Default: 96.\n norm_layer (nn.Module, optional): Normalization layer. Default: None\n \"\"\"\n\n def __init__(self, img_size=224, patch_size=4, in_chans=3, embed_dim=96, norm_layer=None):\n super().__init__()\n img_size = to_2tuple(img_size)\n patch_size = to_2tuple(patch_size)\n patches_resolution = [img_size[0] // patch_size[0], img_size[1] // patch_size[1]]\n self.img_size = img_size\n self.patch_size = patch_size\n self.patches_resolution = patches_resolution\n self.num_patches = patches_resolution[0] * patches_resolution[1]\n\n self.in_chans = in_chans\n self.embed_dim = embed_dim\n\n self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=patch_size, stride=patch_size)\n if norm_layer is not None:\n self.norm = norm_layer(embed_dim)\n else:\n self.norm = None\n\n def forward(self, x):\n B, C, H, W = x.shape\n # FIXME look at relaxing size constraints\n assert H == self.img_size[0] and W == self.img_size[1], \\\n f\"Input image size ({H}*{W}) doesn't match model ({self.img_size[0]}*{self.img_size[1]}).\"\n x = self.proj(x).flatten(2).transpose(1, 2) # B Ph*Pw C\n if self.norm is not None:\n x = self.norm(x)\n return x\n\n def flops(self):\n Ho, Wo = self.patches_resolution\n flops = Ho * Wo * self.embed_dim * self.in_chans * (self.patch_size[0] * self.patch_size[1])\n if self.norm is not None:\n flops += Ho * Wo * self.embed_dim\n return flops\n\n\nclass Model(nn.Module):\n r\"\"\" Swin MLP\n\n Args:\n img_size (int | tuple(int)): Input image size. Default 224\n patch_size (int | tuple(int)): Patch size. Default: 4\n in_chans (int): Number of input image channels. Default: 3\n num_classes (int): Number of classes for classification head. Default: 1000\n embed_dim (int): Patch embedding dimension. Default: 96\n depths (tuple(int)): Depth of each Swin MLP layer.\n num_heads (tuple(int)): Number of attention heads in different layers.\n window_size (int): Window size. Default: 7\n mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4\n drop_rate (float): Dropout rate. Default: 0\n drop_path_rate (float): Stochastic depth rate. Default: 0.1\n norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm.\n patch_norm (bool): If True, add normalization after patch embedding. Default: True\n use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False\n \"\"\"\n\n def __init__(self, img_size=224, patch_size=4, in_chans=3, num_classes=1000,\n embed_dim=96, depths=[2, 2, 6, 2], num_heads=[3, 6, 12, 24],\n window_size=7, mlp_ratio=4., drop_rate=0., drop_path_rate=0.1,\n norm_layer=nn.LayerNorm, patch_norm=True,\n use_checkpoint=False, **kwargs):\n super().__init__()\n\n self.num_classes = num_classes\n self.num_layers = len(depths)\n self.embed_dim = embed_dim\n self.patch_norm = patch_norm\n self.num_features = int(embed_dim * 2 ** (self.num_layers - 1))\n self.mlp_ratio = mlp_ratio\n\n # split image into non-overlapping patches\n self.patch_embed = PatchEmbed(\n img_size=img_size, patch_size=patch_size, in_chans=in_chans, embed_dim=embed_dim,\n norm_layer=norm_layer if self.patch_norm else None)\n num_patches = self.patch_embed.num_patches\n patches_resolution = self.patch_embed.patches_resolution\n self.patches_resolution = patches_resolution\n\n self.pos_drop = nn.Dropout(p=drop_rate)\n\n # stochastic depth\n dpr = [x.item() for x in torch.linspace(0, drop_path_rate, sum(depths))] # stochastic depth decay rule\n\n # build layers\n self.layers = nn.ModuleList()\n for i_layer in range(self.num_layers):\n layer = BasicLayer(dim=int(embed_dim * 2 ** i_layer),\n input_resolution=(patches_resolution[0] // (2 ** i_layer),\n patches_resolution[1] // (2 ** i_layer)),\n depth=depths[i_layer],\n num_heads=num_heads[i_layer],\n window_size=window_size,\n mlp_ratio=self.mlp_ratio,\n drop=drop_rate,\n drop_path=dpr[sum(depths[:i_layer]):sum(depths[:i_layer + 1])],\n norm_layer=norm_layer,\n downsample=PatchMerging if (i_layer < self.num_layers - 1) else None,\n use_checkpoint=use_checkpoint)\n self.layers.append(layer)\n\n self.norm = norm_layer(self.num_features)\n self.avgpool = nn.AdaptiveAvgPool1d(1)\n self.head = nn.Linear(self.num_features, num_classes) if num_classes > 0 else nn.Identity()\n\n def forward_features(self, x):\n x = self.patch_embed(x)\n x = self.pos_drop(x)\n\n for layer in self.layers:\n x = layer(x)\n\n x = self.norm(x) # B L C\n x = self.avgpool(x.transpose(1, 2)) # B C 1\n x = torch.flatten(x, 1)\n return x\n\n def forward(self, x):\n x = self.forward_features(x)\n x = self.head(x)\n return x\n \nbatch_size = 10\nimage_size = 224\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, image_size, image_size)]\n\ndef get_init_inputs():\n return []```\n"}, "L3_030": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 3, "name": "SwinTransformerV2", "split": "learn", "n_ops": 1, "src_lines": 539, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_030 (SwinTransformerV2), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\n# --------------------------------------------------------\n# Swin Transformer V2\n# Copyright (c) 2022 Microsoft\n# Licensed under The MIT License [see LICENSE for details]\n# Written by Ze Liu\n# --------------------------------------------------------\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport numpy as np\nimport collections\nfrom itertools import repeat\n\ndef _ntuple(n):\n def parse(x):\n if isinstance(x, collections.abc.Iterable) and not isinstance(x, str):\n return tuple(x)\n return tuple(repeat(x, n))\n return parse\nto_2tuple = _ntuple(2)\n\nclass Mlp(nn.Module):\n def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.):\n super().__init__()\n out_features = out_features or in_features\n hidden_features = hidden_features or in_features\n self.fc1 = nn.Linear(in_features, hidden_features)\n self.act = act_layer()\n self.fc2 = nn.Linear(hidden_features, out_features)\n self.drop = nn.Dropout(drop)\n\n def forward(self, x):\n x = self.fc1(x)\n x = self.act(x)\n x = self.drop(x)\n x = self.fc2(x)\n x = self.drop(x)\n return x\n\n\ndef window_partition(x, window_size):\n \"\"\"\n Args:\n x: (B, H, W, C)\n window_size (int): window size\n\n Returns:\n windows: (num_windows*B, window_size, window_size, C)\n \"\"\"\n B, H, W, C = x.shape\n x = x.view(B, H // window_size, window_size, W // window_size, window_size, C)\n windows = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(-1, window_size, window_size, C)\n return windows\n\n\ndef window_reverse(windows, window_size, H, W):\n \"\"\"\n Args:\n windows: (num_windows*B, window_size, window_size, C)\n window_size (int): Window size\n H (int): Height of image\n W (int): Width of image\n\n Returns:\n x: (B, H, W, C)\n \"\"\"\n B = int(windows.shape[0] / (H * W / window_size / window_size))\n x = windows.view(B, H // window_size, W // window_size, window_size, window_size, -1)\n x = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(B, H, W, -1)\n return x\n\n\nclass WindowAttention(nn.Module):\n r\"\"\" Window based multi-head self attention (W-MSA) module with relative position bias.\n It supports both of shifted and non-shifted window.\n\n Args:\n dim (int): Number of input channels.\n window_size (tuple[int]): The height and width of the window.\n num_heads (int): Number of attention heads.\n qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True\n attn_drop (float, optional): Dropout ratio of attention weight. Default: 0.0\n proj_drop (float, optional): Dropout ratio of output. Default: 0.0\n pretrained_window_size (tuple[int]): The height and width of the window in pre-training.\n \"\"\"\n\n def __init__(self, dim, window_size, num_heads, qkv_bias=True, attn_drop=0., proj_drop=0.,\n pretrained_window_size=[0, 0]):\n\n super().__init__()\n self.dim = dim\n self.window_size = window_size # Wh, Ww\n self.pretrained_window_size = pretrained_window_size\n self.num_heads = num_heads\n\n self.logit_scale = nn.Parameter(torch.log(10 * torch.ones((num_heads, 1, 1))), requires_grad=True)\n\n # mlp to generate continuous relative position bias\n self.cpb_mlp = nn.Sequential(nn.Linear(2, 512, bias=True),\n nn.ReLU(inplace=True),\n nn.Linear(512, num_heads, bias=False))\n\n # get relative_coords_table\n relative_coords_h = torch.arange(-(self.window_size[0] - 1), self.window_size[0], dtype=torch.float32)\n relative_coords_w = torch.arange(-(self.window_size[1] - 1), self.window_size[1], dtype=torch.float32)\n relative_coords_table = torch.stack(\n torch.meshgrid([relative_coords_h,\n relative_coords_w])).permute(1, 2, 0).contiguous().unsqueeze(0) # 1, 2*Wh-1, 2*Ww-1, 2\n if pretrained_window_size[0] > 0:\n relative_coords_table[:, :, :, 0] /= (pretrained_window_size[0] - 1)\n relative_coords_table[:, :, :, 1] /= (pretrained_window_size[1] - 1)\n else:\n relative_coords_table[:, :, :, 0] /= (self.window_size[0] - 1)\n relative_coords_table[:, :, :, 1] /= (self.window_size[1] - 1)\n relative_coords_table *= 8 # normalize to -8, 8\n relative_coords_table = torch.sign(relative_coords_table) * torch.log2(\n torch.abs(relative_coords_table) + 1.0) / np.log2(8)\n\n self.register_buffer(\"relative_coords_table\", relative_coords_table)\n\n # get pair-wise relative position index for each token inside the window\n coords_h = torch.arange(self.window_size[0])\n coords_w = torch.arange(self.window_size[1])\n coords = torch.stack(torch.meshgrid([coords_h, coords_w])) # 2, Wh, Ww\n coords_flatten = torch.flatten(coords, 1) # 2, Wh*Ww\n relative_coords = coords_flatten[:, :, None] - coords_flatten[:, None, :] # 2, Wh*Ww, Wh*Ww\n relative_coords = relative_coords.permute(1, 2, 0).contiguous() # Wh*Ww, Wh*Ww, 2\n relative_coords[:, :, 0] += self.window_size[0] - 1 # shift to start from 0\n relative_coords[:, :, 1] += self.window_size[1] - 1\n relative_coords[:, :, 0] *= 2 * self.window_size[1] - 1\n relative_position_index = relative_coords.sum(-1) # Wh*Ww, Wh*Ww\n self.register_buffer(\"relative_position_index\", relative_position_index)\n\n self.qkv = nn.Linear(dim, dim * 3, bias=False)\n if qkv_bias:\n self.q_bias = nn.Parameter(torch.zeros(dim))\n self.v_bias = nn.Parameter(torch.zeros(dim))\n else:\n self.q_bias = None\n self.v_bias = None\n self.attn_drop = nn.Dropout(attn_drop)\n self.proj = nn.Linear(dim, dim)\n self.proj_drop = nn.Dropout(proj_drop)\n self.softmax = nn.Softmax(dim=-1)\n\n def forward(self, x, mask=None):\n \"\"\"\n Args:\n x: input features with shape of (num_windows*B, N, C)\n mask: (0/-inf) mask with shape of (num_windows, Wh*Ww, Wh*Ww) or None\n \"\"\"\n B_, N, C = x.shape\n qkv_bias = None\n if self.q_bias is not None:\n qkv_bias = torch.cat((self.q_bias, torch.zeros_like(self.v_bias, requires_grad=False), self.v_bias))\n qkv = F.linear(input=x, weight=self.qkv.weight, bias=qkv_bias)\n qkv = qkv.reshape(B_, N, 3, self.num_heads, -1).permute(2, 0, 3, 1, 4)\n q, k, v = qkv[0], qkv[1], qkv[2] # make torchscript happy (cannot use tensor as tuple)\n\n # cosine attention\n attn = (F.normalize(q, dim=-1) @ F.normalize(k, dim=-1).transpose(-2, -1))\n logit_scale = torch.clamp(self.logit_scale.to(x.device), max=torch.log(torch.tensor(1. / 0.01, device=x.device))).exp()\n attn = attn * logit_scale\n\n relative_position_bias_table = self.cpb_mlp(self.relative_coords_table).view(-1, self.num_heads)\n relative_position_bias = relative_position_bias_table[self.relative_position_index.view(-1)].view(\n self.window_size[0] * self.window_size[1], self.window_size[0] * self.window_size[1], -1) # Wh*Ww,Wh*Ww,nH\n relative_position_bias = relative_position_bias.permute(2, 0, 1).contiguous() # nH, Wh*Ww, Wh*Ww\n relative_position_bias = 16 * torch.sigmoid(relative_position_bias)\n attn = attn + relative_position_bias.unsqueeze(0)\n\n if mask is not None:\n nW = mask.shape[0]\n attn = attn.view(B_ // nW, nW, self.num_heads, N, N) + mask.unsqueeze(1).unsqueeze(0)\n attn = attn.view(-1, self.num_heads, N, N)\n attn = self.softmax(attn)\n else:\n attn = self.softmax(attn)\n\n attn = self.attn_drop(attn)\n\n x = (attn @ v).transpose(1, 2).reshape(B_, N, C)\n x = self.proj(x)\n x = self.proj_drop(x)\n return x\n\nclass SwinTransformerBlock(nn.Module):\n r\"\"\" Swin Transformer Block.\n\n Args:\n dim (int): Number of input channels.\n input_resolution (tuple[int]): Input resulotion.\n num_heads (int): Number of attention heads.\n window_size (int): Window size.\n shift_size (int): Shift size for SW-MSA.\n mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.\n qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True\n drop (float, optional): Dropout rate. Default: 0.0\n attn_drop (float, optional): Attention dropout rate. Default: 0.0\n drop_path (float, optional): Stochastic depth rate. Default: 0.0\n act_layer (nn.Module, optional): Activation layer. Default: nn.GELU\n norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm\n pretrained_window_size (int): Window size in pre-training.\n \"\"\"\n\n def __init__(self, dim, input_resolution, num_heads, window_size=7, shift_size=0,\n mlp_ratio=4., qkv_bias=True, drop=0., attn_drop=0., drop_path=0.,\n act_layer=nn.GELU, norm_layer=nn.LayerNorm, pretrained_window_size=0):\n super().__init__()\n self.dim = dim\n self.input_resolution = input_resolution\n self.num_heads = num_heads\n self.window_size = window_size\n self.shift_size = shift_size\n self.mlp_ratio = mlp_ratio\n if min(self.input_resolution) <= self.window_size:\n # if window size is larger than input resolution, we don't partition windows\n self.shift_size = 0\n self.window_size = min(self.input_resolution)\n assert 0 <= self.shift_size < self.window_size, \"shift_size must in 0-window_size\"\n\n self.norm1 = norm_layer(dim)\n self.attn = WindowAttention(\n dim, window_size=to_2tuple(self.window_size), num_heads=num_heads,\n qkv_bias=qkv_bias, attn_drop=attn_drop, proj_drop=drop,\n pretrained_window_size=to_2tuple(pretrained_window_size))\n\n self.drop_path = nn.Identity()\n self.norm2 = norm_layer(dim)\n mlp_hidden_dim = int(dim * mlp_ratio)\n self.mlp = Mlp(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)\n\n if self.shift_size > 0:\n # calculate attention mask for SW-MSA\n H, W = self.input_resolution\n img_mask = torch.zeros((1, H, W, 1)) # 1 H W 1\n h_slices = (slice(0, -self.window_size),\n slice(-self.window_size, -self.shift_size),\n slice(-self.shift_size, None))\n w_slices = (slice(0, -self.window_size),\n slice(-self.window_size, -self.shift_size),\n slice(-self.shift_size, None))\n cnt = 0\n for h in h_slices:\n for w in w_slices:\n img_mask[:, h, w, :] = cnt\n cnt += 1\n\n mask_windows = window_partition(img_mask, self.window_size) # nW, window_size, window_size, 1\n mask_windows = mask_windows.view(-1, self.window_size * self.window_size)\n attn_mask = mask_windows.unsqueeze(1) - mask_windows.unsqueeze(2)\n attn_mask = attn_mask.masked_fill(attn_mask != 0, float(-100.0)).masked_fill(attn_mask == 0, float(0.0))\n else:\n attn_mask = None\n\n self.register_buffer(\"attn_mask\", attn_mask)\n\n def forward(self, x):\n H, W = self.input_resolution\n B, L, C = x.shape\n assert L == H * W, \"input feature has wrong size\"\n\n shortcut = x\n x = x.view(B, H, W, C)\n\n # cyclic shift\n if self.shift_size > 0:\n shifted_x = torch.roll(x, shifts=(-self.shift_size, -self.shift_size), dims=(1, 2))\n else:\n shifted_x = x\n\n # partition windows\n x_windows = window_partition(shifted_x, self.window_size) # nW*B, window_size, window_size, C\n x_windows = x_windows.view(-1, self.window_size * self.window_size, C) # nW*B, window_size*window_size, C\n\n # W-MSA/SW-MSA\n attn_windows = self.attn(x_windows, mask=self.attn_mask) # nW*B, window_size*window_size, C\n\n # merge windows\n attn_windows = attn_windows.view(-1, self.window_size, self.window_size, C)\n shifted_x = window_reverse(attn_windows, self.window_size, H, W) # B H' W' C\n\n # reverse cyclic shift\n if self.shift_size > 0:\n x = torch.roll(shifted_x, shifts=(self.shift_size, self.shift_size), dims=(1, 2))\n else:\n x = shifted_x\n x = x.view(B, H * W, C)\n x = shortcut + self.drop_path(self.norm1(x))\n\n # FFN\n x = x + self.drop_path(self.norm2(self.mlp(x)))\n\n return x\n\n\nclass PatchMerging(nn.Module):\n r\"\"\" Patch Merging Layer.\n\n Args:\n input_resolution (tuple[int]): Resolution of input feature.\n dim (int): Number of input channels.\n norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm\n \"\"\"\n\n def __init__(self, input_resolution, dim, norm_layer=nn.LayerNorm):\n super().__init__()\n self.input_resolution = input_resolution\n self.dim = dim\n self.reduction = nn.Linear(4 * dim, 2 * dim, bias=False)\n self.norm = norm_layer(2 * dim)\n\n def forward(self, x):\n \"\"\"\n x: B, H*W, C\n \"\"\"\n H, W = self.input_resolution\n B, L, C = x.shape\n assert L == H * W, \"input feature has wrong size\"\n assert H % 2 == 0 and W % 2 == 0, f\"x size ({H}*{W}) are not even.\"\n\n x = x.view(B, H, W, C)\n\n x0 = x[:, 0::2, 0::2, :] # B H/2 W/2 C\n x1 = x[:, 1::2, 0::2, :] # B H/2 W/2 C\n x2 = x[:, 0::2, 1::2, :] # B H/2 W/2 C\n x3 = x[:, 1::2, 1::2, :] # B H/2 W/2 C\n x = torch.cat([x0, x1, x2, x3], -1) # B H/2 W/2 4*C\n x = x.view(B, -1, 4 * C) # B H/2*W/2 4*C\n\n x = self.reduction(x)\n x = self.norm(x)\n\n return x\n\nclass BasicLayer(nn.Module):\n \"\"\" A basic Swin Transformer layer for one stage.\n\n Args:\n dim (int): Number of input channels.\n input_resolution (tuple[int]): Input resolution.\n depth (int): Number of blocks.\n num_heads (int): Number of attention heads.\n window_size (int): Local window size.\n mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.\n qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True\n drop (float, optional): Dropout rate. Default: 0.0\n attn_drop (float, optional): Attention dropout rate. Default: 0.0\n drop_path (float | tuple[float], optional): Stochastic depth rate. Default: 0.0\n norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm\n downsample (nn.Module | None, optional): Downsample layer at the end of the layer. Default: None\n use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False.\n pretrained_window_size (int): Local window size in pre-training.\n \"\"\"\n\n def __init__(self, dim, input_resolution, depth, num_heads, window_size,\n mlp_ratio=4., qkv_bias=True, drop=0., attn_drop=0.,\n drop_path=0., norm_layer=nn.LayerNorm, downsample=None, use_checkpoint=False,\n pretrained_window_size=0):\n\n super().__init__()\n self.dim = dim\n self.input_resolution = input_resolution\n self.depth = depth\n self.use_checkpoint = use_checkpoint\n\n # build blocks\n self.blocks = nn.ModuleList([\n SwinTransformerBlock(dim=dim, input_resolution=input_resolution,\n num_heads=num_heads, window_size=window_size,\n shift_size=0 if (i % 2 == 0) else window_size // 2,\n mlp_ratio=mlp_ratio,\n qkv_bias=qkv_bias,\n drop=drop, attn_drop=attn_drop,\n drop_path=drop_path[i] if isinstance(drop_path, list) else drop_path,\n norm_layer=norm_layer,\n pretrained_window_size=pretrained_window_size)\n for i in range(depth)])\n\n # patch merging layer\n if downsample is not None:\n self.downsample = downsample(input_resolution, dim=dim, norm_layer=norm_layer)\n else:\n self.downsample = None\n\n def forward(self, x):\n for blk in self.blocks:\n if self.use_checkpoint:\n x = checkpoint.checkpoint(blk, x)\n else:\n x = blk(x)\n if self.downsample is not None:\n x = self.downsample(x)\n return x\n \n\nclass PatchEmbed(nn.Module):\n r\"\"\" Image to Patch Embedding\n\n Args:\n img_size (int): Image size. Default: 224.\n patch_size (int): Patch token size. Default: 4.\n in_chans (int): Number of input image channels. Default: 3.\n embed_dim (int): Number of linear projection output channels. Default: 96.\n norm_layer (nn.Module, optional): Normalization layer. Default: None\n \"\"\"\n\n def __init__(self, img_size=224, patch_size=4, in_chans=3, embed_dim=96, norm_layer=None):\n super().__init__()\n img_size = to_2tuple(img_size)\n patch_size = to_2tuple(patch_size)\n patches_resolution = [img_size[0] // patch_size[0], img_size[1] // patch_size[1]]\n self.img_size = img_size\n self.patch_size = patch_size\n self.patches_resolution = patches_resolution\n self.num_patches = patches_resolution[0] * patches_resolution[1]\n\n self.in_chans = in_chans\n self.embed_dim = embed_dim\n\n self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=patch_size, stride=patch_size)\n if norm_layer is not None:\n self.norm = norm_layer(embed_dim)\n else:\n self.norm = None\n\n def forward(self, x):\n B, C, H, W = x.shape\n # FIXME look at relaxing size constraints\n assert H == self.img_size[0] and W == self.img_size[1], \\\n f\"Input image size ({H}*{W}) doesn't match model ({self.img_size[0]}*{self.img_size[1]}).\"\n x = self.proj(x).flatten(2).transpose(1, 2) # B Ph*Pw C\n if self.norm is not None:\n x = self.norm(x)\n return x\n\n\nclass Model(nn.Module):\n r\"\"\" Swin Transformer\n A PyTorch impl of : `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows` -\n https://arxiv.org/pdf/2103.14030\n\n Args:\n img_size (int | tuple(int)): Input image size. Default 224\n patch_size (int | tuple(int)): Patch size. Default: 4\n in_chans (int): Number of input image channels. Default: 3\n num_classes (int): Number of classes for classification head. Default: 1000\n embed_dim (int): Patch embedding dimension. Default: 96\n depths (tuple(int)): Depth of each Swin Transformer layer.\n num_heads (tuple(int)): Number of attention heads in different layers.\n window_size (int): Window size. Default: 7\n mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4\n qkv_bias (bool): If True, add a learnable bias to query, key, value. Default: True\n drop_rate (float): Dropout rate. Default: 0\n attn_drop_rate (float): Attention dropout rate. Default: 0\n drop_path_rate (float): Stochastic depth rate. Default: 0.1\n norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm.\n patch_norm (bool): If True, add normalization after patch embedding. Default: True\n use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False\n pretrained_window_sizes (tuple(int)): Pretrained window sizes of each layer.\n \"\"\"\n\n def __init__(self, img_size=224, patch_size=4, in_chans=3, num_classes=1000,\n embed_dim=96, depths=[2, 2, 6, 2], num_heads=[3, 6, 12, 24],\n window_size=7, mlp_ratio=4., qkv_bias=True,\n drop_rate=0., attn_drop_rate=0., drop_path_rate=0.1,\n norm_layer=nn.LayerNorm, patch_norm=True,\n use_checkpoint=False, pretrained_window_sizes=[0, 0, 0, 0], **kwargs):\n super().__init__()\n\n self.num_classes = num_classes\n self.num_layers = len(depths)\n self.embed_dim = embed_dim\n self.patch_norm = patch_norm\n self.num_features = int(embed_dim * 2 ** (self.num_layers - 1))\n self.mlp_ratio = mlp_ratio\n\n # split image into non-overlapping patches\n self.patch_embed = PatchEmbed(\n img_size=img_size, patch_size=patch_size, in_chans=in_chans, embed_dim=embed_dim,\n norm_layer=norm_layer if self.patch_norm else None)\n num_patches = self.patch_embed.num_patches\n patches_resolution = self.patch_embed.patches_resolution\n self.patches_resolution = patches_resolution\n\n self.pos_drop = nn.Dropout(p=drop_rate)\n\n # stochastic depth\n dpr = [x.item() for x in torch.linspace(0, drop_path_rate, sum(depths))] # stochastic depth decay rule\n\n # build layers\n self.layers = nn.ModuleList()\n for i_layer in range(self.num_layers):\n layer = BasicLayer(dim=int(embed_dim * 2 ** i_layer),\n input_resolution=(patches_resolution[0] // (2 ** i_layer),\n patches_resolution[1] // (2 ** i_layer)),\n depth=depths[i_layer],\n num_heads=num_heads[i_layer],\n window_size=window_size,\n mlp_ratio=self.mlp_ratio,\n qkv_bias=qkv_bias,\n drop=drop_rate, attn_drop=attn_drop_rate,\n drop_path=dpr[sum(depths[:i_layer]):sum(depths[:i_layer + 1])],\n norm_layer=norm_layer,\n downsample=PatchMerging if (i_layer < self.num_layers - 1) else None,\n use_checkpoint=use_checkpoint,\n pretrained_window_size=pretrained_window_sizes[i_layer])\n self.layers.append(layer)\n\n self.norm = norm_layer(self.num_features)\n self.avgpool = nn.AdaptiveAvgPool1d(1)\n self.head = nn.Linear(self.num_features, num_classes) if num_classes > 0 else nn.Identity()\n\n def forward_features(self, x):\n x = self.patch_embed(x)\n x = self.pos_drop(x)\n\n for layer in self.layers:\n x = layer(x)\n\n x = self.norm(x) # B L C\n x = self.avgpool(x.transpose(1, 2)) # B C 1\n x = torch.flatten(x, 1)\n return x\n\n def forward(self, x):\n x = self.forward_features(x)\n x = self.head(x)\n return x\n\nbatch_size = 10\nimage_size = 224\n\ndef get_inputs():\n return [torch.rand(batch_size, 3, image_size, image_size)]\n\ndef get_init_inputs():\n return []```\n"}, "L3_033": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.", "ctx_chars": 3355, "ctx_sha": "cc270772e526c41f", "dropped_same_problem": 2, "kinds": {"lesson": 14, "pitfall": 6}, "family": "rnn", "level": 3, "name": "VanillaRNN", "split": "learn", "n_ops": 1, "src_lines": 50, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_033 (VanillaRNN), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n \"\"\"\n Initialize the Vanilla RNN model.\n \n :param input_size: The number of input features (int).\n :param hidden_size: The size of the hidden state (int).\n :param output_size: The number of output features (int).\n \"\"\"\n super(Model, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n \n # Define the RNN cell components (input to hidden, hidden to hidden, and hidden to output)\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size) # Input to hidden\n self.h2o = nn.Linear(hidden_size, output_size) # Hidden to output\n self.tanh = nn.Tanh() # Activation function for hidden state\n \n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n \"\"\"\n Forward pass of the Vanilla RNN.\n \n :param x: Input tensor of shape (batch_size, input_size).\n :param hidden: Hidden state tensor of shape (batch_size, hidden_size).\n :return: Output tensor of shape (batch_size, output_size), and the new hidden state.\n \"\"\"\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n combined = torch.cat((x, self.hidden), dim=1) # Concatenate input and hidden state\n self.hidden = self.tanh(self.i2h(combined)) # Update hidden state\n output = self.h2o(self.hidden) # Compute output\n return output\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\nsequence_length = 256\n\ndef get_inputs():\n return [torch.rand(batch_size, input_size),torch.rand(batch_size, hidden_size)]\n\ndef get_init_inputs():\n return [input_size, hidden_size, output_size]\n```\n"}, "L3_034": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```", "ctx_chars": 6034, "ctx_sha": "22fd89455cfd7f9d", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 1}, "family": "rnn", "level": 3, "name": "VanillaRNNHidden", "split": "learn", "n_ops": 1, "src_lines": 57, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_034 (VanillaRNNHidden), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n \"\"\"\n Initialize the Vanilla RNN model.\n\n :param input_size: The number of input features (int).\n :param hidden_size: The size of the hidden state (int).\n :param output_size: The number of output features (int).\n \"\"\"\n super(Model, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n # Define the RNN cell components (input to hidden, hidden to hidden, and hidden to output)\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size) # Input to hidden\n self.h2o = nn.Linear(hidden_size, output_size) # Hidden to output\n self.tanh = nn.Tanh() # Activation function for hidden state\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Forward pass of the Vanilla RNN.\n\n :param x: Input tensor of shape (seq_len, batch_size, input_size)\n :param h0: Initial hidden state tensor of shape (batch_size, hidden_size)\n :return: Output tensor of shape (seq_len, batch_size, output_size)\n \"\"\"\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n outputs = []\n\n for t in range(seq_len):\n combined = torch.cat((x[t], hidden), dim=1) # Concatenate input and hidden state\n hidden = self.tanh(self.i2h(combined)) # Update hidden state\n output = self.h2o(hidden) # Compute output\n outputs.append(output)\n\n return torch.stack(outputs, dim=0) # (seq_len, batch_size, output_size)\n\n# === Test configuration ===\nbatch_size = 8\ninput_size = 1024\nhidden_size = 256\noutput_size = 128\nsequence_length = 256\n\ndef get_inputs():\n return [\n torch.rand(sequence_length, batch_size, input_size),\n torch.rand(batch_size, hidden_size)\n ]\n\ndef get_init_inputs():\n return [input_size, hidden_size, output_size]\n```\n"}, "L3_036": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```\n\n### A verified kernel for a related rnn problem (L3_034 VanillaRNNHidden, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_tanh_kernel(pre_ptr, b_ptr, out_ptr, n_rows, hidden_size, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n mask = offs < hidden_size\n v = tl.load(pre_ptr + pid * hidden_size + offs, mask=mask, other=0.0)\n b = tl.load(b_ptr + offs, mask=mask, other=0.0)\n r = v + b\n e = tl.exp(2.0 * r)\n r = (e - 1.0) / (e + 1.0)\n tl.store(out_ptr + pid * hidden_size + offs, r, mask=mask)\n\n\ndef bias_tanh(pre, b, out):\n n_rows, hidden_size = pre.shape\n BLOCK = triton.next_power_of_2(hidden_size)\n bias_tanh_kernel[(n_rows,)](pre, b, out, n_rows, hidden_size, BLOCK=BLOCK)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n\n W = self.i2h.weight\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n b = self.i2h.bias\n\n x2d = x.reshape(seq_len * batch_size, self.input_size)\n pre_all = (x2d @ W_x.t()).reshape(seq_len, batch_size, self.hidden_size)\n\n Wh_t = W_h.t().contiguous()\n\n hiddens = torch.empty(seq_len, batch_size, self.hidden_size,\n device=x.device, dtype=x.dtype)\n\n for t in range(seq_len):\n pre = pre_all[t].addmm(hidden, Wh_t)\n hidden = bias_tanh(pre, b, hiddens[t])\n\n h2d = hiddens.reshape(seq_len * batch_size, self.hidden_size)\n out = self.h2o(h2d).reshape(seq_len, batch_size, self.output_size)\n return out\n```", "ctx_chars": 8224, "ctx_sha": "71fd1a97cbd02e3b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 2}, "family": "rnn", "level": 3, "name": "LSTMHn", "split": "learn", "n_ops": 1, "src_lines": 49, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_036 (LSTMHn), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_size, num_layers, output_size, dropout=0.0):\n \"\"\"\n Initialize the LSTM model.\n\n :param input_size: The number of expected features in the input `x`\n :param hidden_size: The number of features in the hidden state `h`\n :param num_layers: Number of recurrent layers\n :param output_size: The number of output features\n :param dropout: If non-zero, introduces a Dropout layer on the outputs of each LSTM layer except the last layer, with dropout probability equal to `dropout`\n \"\"\"\n super(Model, self).__init__()\n # Initialize hidden state with random values\n self.lstm = nn.LSTM(input_size, hidden_size, num_layers, batch_first=True, dropout=dropout, bidirectional=False)\n self.fc = nn.Linear(hidden_size, output_size)\n \n def forward(self, x,h0,c0):\n \"\"\"\n Forward pass through the LSTM model.\n\n :param x: The input tensor, shape (batch_size, sequence_length, input_size)\n :return: The output tensor, shape (batch_size, sequence_length, output_size)\n \"\"\"\n \n # Forward propagate LSTM\n out, state = self.lstm(x, (h0, c0)) # out: tensor of shape (batch_size, seq_length, hidden_size)\n \n # Decode the hidden state of the last time step\n out = self.fc(out[:, -1, :]) # out: tensor of shape (batch_size, output_size)\n \n return state[0]\n\n# Test code\nbatch_size = 10\nsequence_length = 512\ninput_size = 128\nhidden_size = 256\nnum_layers = 6\noutput_size = 10\ndropout = 0.0\n\ndef get_inputs():\n return [torch.rand(batch_size, sequence_length, input_size),torch.rand((num_layers, batch_size, hidden_size)),torch.rand((num_layers, batch_size, hidden_size))]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_layers, output_size, dropout]```\n"}, "L3_037": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```\n\n### A verified kernel for a related rnn problem (L3_034 VanillaRNNHidden, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_tanh_kernel(pre_ptr, b_ptr, out_ptr, n_rows, hidden_size, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n mask = offs < hidden_size\n v = tl.load(pre_ptr + pid * hidden_size + offs, mask=mask, other=0.0)\n b = tl.load(b_ptr + offs, mask=mask, other=0.0)\n r = v + b\n e = tl.exp(2.0 * r)\n r = (e - 1.0) / (e + 1.0)\n tl.store(out_ptr + pid * hidden_size + offs, r, mask=mask)\n\n\ndef bias_tanh(pre, b, out):\n n_rows, hidden_size = pre.shape\n BLOCK = triton.next_power_of_2(hidden_size)\n bias_tanh_kernel[(n_rows,)](pre, b, out, n_rows, hidden_size, BLOCK=BLOCK)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n\n W = self.i2h.weight\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n b = self.i2h.bias\n\n x2d = x.reshape(seq_len * batch_size, self.input_size)\n pre_all = (x2d @ W_x.t()).reshape(seq_len, batch_size, self.hidden_size)\n\n Wh_t = W_h.t().contiguous()\n\n hiddens = torch.empty(seq_len, batch_size, self.hidden_size,\n device=x.device, dtype=x.dtype)\n\n for t in range(seq_len):\n pre = pre_all[t].addmm(hidden, Wh_t)\n hidden = bias_tanh(pre, b, hiddens[t])\n\n h2d = hiddens.reshape(seq_len * batch_size, self.hidden_size)\n out = self.h2o(h2d).reshape(seq_len, batch_size, self.output_size)\n return out\n```", "ctx_chars": 8224, "ctx_sha": "71fd1a97cbd02e3b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 2}, "family": "rnn", "level": 3, "name": "LSTMCn", "split": "learn", "n_ops": 1, "src_lines": 49, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_037 (LSTMCn), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_size, num_layers, output_size, dropout=0.0):\n \"\"\"\n Initialize the LSTM model.\n\n :param input_size: The number of expected features in the input `x`\n :param hidden_size: The number of features in the hidden state `h`\n :param num_layers: Number of recurrent layers\n :param output_size: The number of output features\n :param dropout: If non-zero, introduces a Dropout layer on the outputs of each LSTM layer except the last layer, with dropout probability equal to `dropout`\n \"\"\"\n super(Model, self).__init__()\n # Initialize hidden state with random values\n self.lstm = nn.LSTM(input_size, hidden_size, num_layers, batch_first=True, dropout=dropout, bidirectional=False)\n self.fc = nn.Linear(hidden_size, output_size)\n \n def forward(self, x, h0, c0):\n \"\"\"\n Forward pass through the LSTM model.\n\n :param x: The input tensor, shape (batch_size, sequence_length, input_size)\n :return: The output tensor, shape (batch_size, sequence_length, output_size)\n \"\"\"\n \n # Forward propagate LSTM\n out, state = self.lstm(x, (h0, c0)) # out: tensor of shape (batch_size, seq_length, hidden_size)\n \n # Decode the hidden state of the last time step\n out = self.fc(out[:, -1, :]) # out: tensor of shape (batch_size, output_size)\n \n return state[1]\n\n# Test code\nbatch_size = 10\nsequence_length = 512\ninput_size = 128\nhidden_size = 256\nnum_layers = 6\noutput_size = 10\ndropout = 0.0\n\ndef get_inputs():\n return [torch.rand(batch_size, sequence_length, input_size),torch.rand((num_layers, batch_size, hidden_size)),torch.rand((num_layers, batch_size, hidden_size))]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_layers, output_size, dropout]```\n"}, "L3_038": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```\n\n### A verified kernel for a related rnn problem (L3_034 VanillaRNNHidden, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_tanh_kernel(pre_ptr, b_ptr, out_ptr, n_rows, hidden_size, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n mask = offs < hidden_size\n v = tl.load(pre_ptr + pid * hidden_size + offs, mask=mask, other=0.0)\n b = tl.load(b_ptr + offs, mask=mask, other=0.0)\n r = v + b\n e = tl.exp(2.0 * r)\n r = (e - 1.0) / (e + 1.0)\n tl.store(out_ptr + pid * hidden_size + offs, r, mask=mask)\n\n\ndef bias_tanh(pre, b, out):\n n_rows, hidden_size = pre.shape\n BLOCK = triton.next_power_of_2(hidden_size)\n bias_tanh_kernel[(n_rows,)](pre, b, out, n_rows, hidden_size, BLOCK=BLOCK)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n\n W = self.i2h.weight\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n b = self.i2h.bias\n\n x2d = x.reshape(seq_len * batch_size, self.input_size)\n pre_all = (x2d @ W_x.t()).reshape(seq_len, batch_size, self.hidden_size)\n\n Wh_t = W_h.t().contiguous()\n\n hiddens = torch.empty(seq_len, batch_size, self.hidden_size,\n device=x.device, dtype=x.dtype)\n\n for t in range(seq_len):\n pre = pre_all[t].addmm(hidden, Wh_t)\n hidden = bias_tanh(pre, b, hiddens[t])\n\n h2d = hiddens.reshape(seq_len * batch_size, self.hidden_size)\n out = self.h2o(h2d).reshape(seq_len, batch_size, self.output_size)\n return out\n```", "ctx_chars": 8224, "ctx_sha": "71fd1a97cbd02e3b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 2}, "family": "rnn", "level": 3, "name": "LSTMBidirectional", "split": "learn", "n_ops": 1, "src_lines": 48, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_038 (LSTMBidirectional), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_size, num_layers, output_size, dropout=0.0):\n \"\"\"\n Initialize the LSTM model.\n\n :param input_size: The number of expected features in the input `x`\n :param hidden_size: The number of features in the hidden state `h`\n :param num_layers: Number of recurrent layers\n :param output_size: The number of output features\n :param dropout: If non-zero, introduces a Dropout layer on the outputs of each LSTM layer except the last layer, with dropout probability equal to `dropout`\n \"\"\"\n super(Model, self).__init__()\n # Initialize hidden state with random values\n self.lstm = nn.LSTM(input_size, hidden_size, num_layers, batch_first=True, dropout=dropout, bidirectional=True)\n self.fc = nn.Linear(hidden_size * 2, output_size)\n \n def forward(self, x,h0,c0):\n \"\"\"\n Forward pass through the LSTM model.\n\n :param x: The input tensor, shape (batch_size, sequence_length, input_size)\n :return: The output tensor, shape (batch_size, sequence_length, output_size)\n \"\"\"\n # Forward propagate LSTM\n out, hn = self.lstm(x, (h0, c0)) # out: tensor of shape (batch_size, seq_length, hidden_size)\n \n # Decode the hidden state of the last time step\n out = self.fc(out[:, -1, :]) # out: tensor of shape (batch_size, output_size)\n \n return out\n\n# Test code\nbatch_size = 10\nsequence_length = 512\ninput_size = 128\nhidden_size = 256\nnum_layers = 6\noutput_size = 10\ndropout = 0.0\n\ndef get_inputs():\n return [torch.rand(batch_size, sequence_length, input_size),torch.rand((num_layers*2, batch_size, hidden_size)),torch.rand((num_layers*2, batch_size, hidden_size))]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_layers, output_size, dropout]```\n"}, "L3_039": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```\n\n### A verified kernel for a related rnn problem (L3_034 VanillaRNNHidden, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_tanh_kernel(pre_ptr, b_ptr, out_ptr, n_rows, hidden_size, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n mask = offs < hidden_size\n v = tl.load(pre_ptr + pid * hidden_size + offs, mask=mask, other=0.0)\n b = tl.load(b_ptr + offs, mask=mask, other=0.0)\n r = v + b\n e = tl.exp(2.0 * r)\n r = (e - 1.0) / (e + 1.0)\n tl.store(out_ptr + pid * hidden_size + offs, r, mask=mask)\n\n\ndef bias_tanh(pre, b, out):\n n_rows, hidden_size = pre.shape\n BLOCK = triton.next_power_of_2(hidden_size)\n bias_tanh_kernel[(n_rows,)](pre, b, out, n_rows, hidden_size, BLOCK=BLOCK)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n\n W = self.i2h.weight\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n b = self.i2h.bias\n\n x2d = x.reshape(seq_len * batch_size, self.input_size)\n pre_all = (x2d @ W_x.t()).reshape(seq_len, batch_size, self.hidden_size)\n\n Wh_t = W_h.t().contiguous()\n\n hiddens = torch.empty(seq_len, batch_size, self.hidden_size,\n device=x.device, dtype=x.dtype)\n\n for t in range(seq_len):\n pre = pre_all[t].addmm(hidden, Wh_t)\n hidden = bias_tanh(pre, b, hiddens[t])\n\n h2d = hiddens.reshape(seq_len * batch_size, self.hidden_size)\n out = self.h2o(h2d).reshape(seq_len, batch_size, self.output_size)\n return out\n```", "ctx_chars": 8224, "ctx_sha": "71fd1a97cbd02e3b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 2}, "family": "rnn", "level": 3, "name": "GRU", "split": "learn", "n_ops": 1, "src_lines": 40, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_039 (GRU), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_size, num_layers=3, bias=True, batch_first=False):\n \"\"\"\n :param input_size: The number of expected features in the input x\n :param hidden_size: The number of features in the hidden state h\n :param num_layers: Number of recurrent layers (default: 1)\n :param bias: If False, then the layer does not use bias weights b_ih and b_hh (default: True)\n :param batch_first: If True, then the input and output tensors are provided as (batch, seq, feature) (default: False)\n \"\"\"\n super(Model, self).__init__()\n \n self.gru = nn.GRU(input_size, hidden_size, num_layers, bias, batch_first, dropout=0, bidirectional=False)\n \n def forward(self, x,h0):\n \"\"\"\n :param x: The input tensor, shape (seq_len, batch_size, input_size) if batch_first=False, otherwise (batch_size, seq_len, input_size)\n :param h_0: The initial hidden state for the input sequence, shape (num_layers * num_directions, batch_size, hidden_size) (default: None)\n :return: output, h_n\n - output: The output features (h_t) from the last layer of the GRU, for each t, shape (seq_len, batch_size, num_directions * hidden_size) if batch_first=False, otherwise (batch_size, seq_len, num_directions * hidden_size)\n - h_n: The hidden state for t = seq_len, shape (num_layers * num_directions, batch_size, hidden_size)\n \"\"\"\n output, h_n = self.gru(x, h0)\n return output\n\n# Test code\nbatch_size = 10\nseq_len = 512\ninput_size = 128\nhidden_size = 256\nnum_layers = 6\n\ndef get_inputs():\n return [torch.rand(seq_len, batch_size, input_size),torch.rand((num_layers, batch_size, hidden_size))]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_layers]```\n"}, "L3_041": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```\n\n### A verified kernel for a related rnn problem (L3_034 VanillaRNNHidden, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_tanh_kernel(pre_ptr, b_ptr, out_ptr, n_rows, hidden_size, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n mask = offs < hidden_size\n v = tl.load(pre_ptr + pid * hidden_size + offs, mask=mask, other=0.0)\n b = tl.load(b_ptr + offs, mask=mask, other=0.0)\n r = v + b\n e = tl.exp(2.0 * r)\n r = (e - 1.0) / (e + 1.0)\n tl.store(out_ptr + pid * hidden_size + offs, r, mask=mask)\n\n\ndef bias_tanh(pre, b, out):\n n_rows, hidden_size = pre.shape\n BLOCK = triton.next_power_of_2(hidden_size)\n bias_tanh_kernel[(n_rows,)](pre, b, out, n_rows, hidden_size, BLOCK=BLOCK)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n\n W = self.i2h.weight\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n b = self.i2h.bias\n\n x2d = x.reshape(seq_len * batch_size, self.input_size)\n pre_all = (x2d @ W_x.t()).reshape(seq_len, batch_size, self.hidden_size)\n\n Wh_t = W_h.t().contiguous()\n\n hiddens = torch.empty(seq_len, batch_size, self.hidden_size,\n device=x.device, dtype=x.dtype)\n\n for t in range(seq_len):\n pre = pre_all[t].addmm(hidden, Wh_t)\n hidden = bias_tanh(pre, b, hiddens[t])\n\n h2d = hiddens.reshape(seq_len * batch_size, self.hidden_size)\n out = self.h2o(h2d).reshape(seq_len, batch_size, self.output_size)\n return out\n```", "ctx_chars": 8224, "ctx_sha": "71fd1a97cbd02e3b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 2}, "family": "rnn", "level": 3, "name": "GRUBidirectional", "split": "learn", "n_ops": 1, "src_lines": 41, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_041 (GRUBidirectional), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_size, num_layers=3, bias=True, batch_first=False):\n \"\"\"\n :param input_size: The number of expected features in the input x\n :param hidden_size: The number of features in the hidden state h\n :param num_layers: Number of recurrent layers (default: 1)\n :param bias: If False, then the layer does not use bias weights b_ih and b_hh (default: True)\n :param batch_first: If True, then the input and output tensors are provided as (batch, seq, feature) (default: False)\n \"\"\"\n super(Model, self).__init__()\n \n self.gru = nn.GRU(input_size, hidden_size, num_layers, bias, batch_first, dropout=0, bidirectional=True)\n self.h0 = torch.randn((num_layers * 2, batch_size, hidden_size))\n \n def forward(self, x,h0):\n \"\"\"\n :param x: The input tensor, shape (seq_len, batch_size, input_size) if batch_first=False, otherwise (batch_size, seq_len, input_size)\n :param h_0: The initial hidden state for the input sequence, shape (num_layers * num_directions, batch_size, hidden_size) (default: None)\n :return: output, h_n\n - output: The output features (h_t) from the last layer of the GRU, for each t, shape (seq_len, batch_size, num_directions * hidden_size) if batch_first=False, otherwise (batch_size, seq_len, num_directions * hidden_size)\n - h_n: The hidden state for t = seq_len, shape (num_layers * num_directions, batch_size, hidden_size)\n \"\"\"\n output, h_n = self.gru(x, h0)\n return output\n\n# Test code\nbatch_size = 10\nseq_len = 512\ninput_size = 128\nhidden_size = 256\nnum_layers = 6\n\ndef get_inputs():\n return [torch.rand(seq_len, batch_size, input_size),torch.rand((num_layers*2, batch_size, hidden_size))]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_layers]```\n"}, "L3_042": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier rnn problems (wrong results or slower than eager)\n- Pitfall: Before writing a kernel, check whether the eager baseline is already one fused vendor-library call (cuDNN RNN, cuBLAS). Decomposing it into many smaller launches is almost always slower, even when correct.\n- Pitfall: For recurrent ops, don't replace cuDNN with a Python per-timestep loop launching separate GEMM and gate kernels. Launch overhead dominates and yields big slowdowns, so keep cuDNN or fuse the entire time loop into one persistent kernel.\n- Pitfall: Gains must come from fusing or eliminating real work, such as skipping computations whose results are never returned. A trivial copy or pass-through kernel cannot beat eager.\n- Pitfall: Don't call the cuDNN/PyTorch op and then append a token Triton copy kernel. The extra launch and allocation only make it slower than eager. Return library outputs directly, or replace real work with fused kernels.\n- Pitfall: Avoid one-program-per-output-element dot-product kernels (tl.sum per element) for matmuls; use tiled tl.dot or keep cuBLAS, since naive reductions lose to eager even at small sizes.\n- Pitfall: Don't wrap the dominant op (e.g., cuDNN LSTM) in PyTorch and only replace a tiny trailing linear with a custom kernel. Launch overhead plus a naive dot-product kernel makes it slower; optimize the actual hot path.\n\n### Verified recipes for rnn problems (speedups measured by the protected evaluator)\n- [1.03x on VanillaRNN] Keep every GEMM in cuBLAS through torch (matmul/addmm, F.linear). Drop torch.cat in a concat-then-Linear cell: slice the weight into W_x and W_h column views, compute x@W_x.T, then accumulate h@W_h.T in place with addmm_. Fuse bias-add and tanh into a single Triton elementwise pass, one program per row, looping over columns in 1024-wide masked blocks. Accumulate in fp32 and use an exact tanh formula. Expect only small gains because the large GEMMs dominate runtime.\n\n### A verified kernel for a related rnn problem (L3_033 VanillaRNN, 1.03x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\nbatch_size = 256\ninput_size = 16384\nhidden_size = 16384\noutput_size = 8192\n\n@triton.jit\ndef bias_tanh_kernel(x_ptr, b_ptr, out_ptr, n_rows, n_cols, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid\n offs = tl.arange(0, BLOCK)\n for c in range(0, n_cols, BLOCK):\n idx = c + offs\n mask = idx < n_cols\n v = tl.load(x_ptr + row * n_cols + idx, mask=mask, other=0.0)\n b = tl.load(b_ptr + idx, mask=mask, other=0.0)\n r = v + b\n r = (tl.exp(2.0 * r) - 1.0) / (tl.exp(2.0 * r) + 1.0)\n tl.store(out_ptr + row * n_cols + idx, r, mask=mask)\n\n\ndef bias_tanh(x, b):\n n_rows, n_cols = x.shape\n out = torch.empty_like(x)\n grid = (n_rows,)\n bias_tanh_kernel[grid](x, b, out, n_rows, n_cols, BLOCK=1024)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n self.hidden = torch.randn((batch_size, hidden_size))\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, initial_hidden=None) -> torch.Tensor:\n if initial_hidden is not None:\n self.hidden.copy_(initial_hidden)\n self.hidden = self.hidden.to(x.device)\n\n W = self.i2h.weight # (hidden, input+hidden)\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n\n # pre-activation = x @ W_x.T + hidden @ W_h.T + bias\n pre = x @ W_x.t()\n pre = pre.addmm_(self.hidden, W_h.t())\n\n new_hidden = bias_tanh(pre, self.i2h.bias)\n self.hidden = new_hidden\n\n output = self.h2o(new_hidden)\n return output\n```\n\n### A verified kernel for a related rnn problem (L3_034 VanillaRNNHidden, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef bias_tanh_kernel(pre_ptr, b_ptr, out_ptr, n_rows, hidden_size, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n mask = offs < hidden_size\n v = tl.load(pre_ptr + pid * hidden_size + offs, mask=mask, other=0.0)\n b = tl.load(b_ptr + offs, mask=mask, other=0.0)\n r = v + b\n e = tl.exp(2.0 * r)\n r = (e - 1.0) / (e + 1.0)\n tl.store(out_ptr + pid * hidden_size + offs, r, mask=mask)\n\n\ndef bias_tanh(pre, b, out):\n n_rows, hidden_size = pre.shape\n BLOCK = triton.next_power_of_2(hidden_size)\n bias_tanh_kernel[(n_rows,)](pre, b, out, n_rows, hidden_size, BLOCK=BLOCK)\n return out\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size: int, hidden_size: int, output_size: int):\n super(ModelNew, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.output_size = output_size\n\n self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n self.h2o = nn.Linear(hidden_size, output_size)\n self.tanh = nn.Tanh()\n\n def forward(self, x: torch.Tensor, h0: torch.Tensor) -> torch.Tensor:\n seq_len, batch_size, _ = x.size()\n hidden = h0.to(x.device)\n\n W = self.i2h.weight\n W_x = W[:, :self.input_size]\n W_h = W[:, self.input_size:]\n b = self.i2h.bias\n\n x2d = x.reshape(seq_len * batch_size, self.input_size)\n pre_all = (x2d @ W_x.t()).reshape(seq_len, batch_size, self.hidden_size)\n\n Wh_t = W_h.t().contiguous()\n\n hiddens = torch.empty(seq_len, batch_size, self.hidden_size,\n device=x.device, dtype=x.dtype)\n\n for t in range(seq_len):\n pre = pre_all[t].addmm(hidden, Wh_t)\n hidden = bias_tanh(pre, b, hiddens[t])\n\n h2d = hiddens.reshape(seq_len * batch_size, self.hidden_size)\n out = self.h2o(h2d).reshape(seq_len, batch_size, self.output_size)\n return out\n```", "ctx_chars": 8224, "ctx_sha": "71fd1a97cbd02e3b", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 1, "kernel": 2}, "family": "rnn", "level": 3, "name": "GRUBidirectionalHidden", "split": "learn", "n_ops": 1, "src_lines": 40, "max_recipe_speedup": 1.03, "instruction": "KernelBench level 3 problem L3_042 (GRUBidirectionalHidden), op family: rnn.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n def __init__(self, input_size, hidden_size, num_layers=3, bias=True, batch_first=False):\n \"\"\"\n :param input_size: The number of expected features in the input x\n :param hidden_size: The number of features in the hidden state h\n :param num_layers: Number of recurrent layers (default: 1)\n :param bias: If False, then the layer does not use bias weights b_ih and b_hh (default: True)\n :param batch_first: If True, then the input and output tensors are provided as (batch, seq, feature) (default: False)\n \"\"\"\n super(Model, self).__init__()\n \n self.gru = nn.GRU(input_size, hidden_size, num_layers, bias, batch_first, dropout=0, bidirectional=True)\n \n def forward(self, x,h0):\n \"\"\"\n :param x: The input tensor, shape (seq_len, batch_size, input_size) if batch_first=False, otherwise (batch_size, seq_len, input_size)\n :param h_0: The initial hidden state for the input sequence, shape (num_layers * num_directions, batch_size, hidden_size) (default: None)\n :return: output, h_n\n - output: The output features (h_t) from the last layer of the GRU, for each t, shape (seq_len, batch_size, num_directions * hidden_size) if batch_first=False, otherwise (batch_size, seq_len, num_directions * hidden_size)\n - h_n: The hidden state for t = seq_len, shape (num_layers * num_directions, batch_size, hidden_size)\n \"\"\"\n output, h_n = self.gru(x, h0)\n return h_n\n\n# Test code\nbatch_size = 10\nseq_len = 512\ninput_size = 128\nhidden_size = 256\nnum_layers = 6\n\ndef get_inputs():\n return [torch.rand(seq_len, batch_size, input_size),torch.rand((num_layers*2, batch_size, hidden_size))]\n\ndef get_init_inputs():\n return [input_size, hidden_size, num_layers]```\n"}, "L3_043": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 3, "name": "MinGPTCausalAttention", "split": "learn", "n_ops": 1, "src_lines": 64, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_043 (MinGPTCausalAttention), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\n\n# From https://github.com/karpathy/minGPT/blob/master/mingpt/model.py\n\nclass Model(nn.Module):\n \"\"\"\n A vanilla multi-head masked self-attention layer with a projection at the end.\n It is possible to use torch.nn.MultiheadAttention here but I am including an\n explicit implementation here to show that there is nothing too scary here.\n \"\"\"\n\n def __init__(self, n_embd, n_head, attn_pdrop, resid_pdrop, max_seqlen):\n super().__init__()\n assert n_embd % n_head == 0\n # key, query, value projections for all heads, but in a batch\n self.c_attn = nn.Linear(n_embd, 3 * n_embd)\n # output projection\n self.c_proj = nn.Linear(n_embd, n_embd)\n # regularization\n self.attn_dropout = nn.Dropout(attn_pdrop)\n self.resid_dropout = nn.Dropout(resid_pdrop)\n # causal mask to ensure that attention is only applied to the left in the input sequence\n self.register_buffer(\"bias\", torch.tril(torch.ones(max_seqlen, max_seqlen))\n .view(1, 1, max_seqlen, max_seqlen))\n self.n_head = n_head\n self.n_embd = n_embd\n\n def forward(self, x):\n B, T, C = x.size() # batch size, sequence length, embedding dimensionality (n_embd)\n\n # calculate query, key, values for all heads in batch and move head forward to be the batch dim\n q, k ,v = self.c_attn(x).split(self.n_embd, dim=2)\n k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n\n # causal self-attention; Self-attend: (B, nh, T, hs) x (B, nh, hs, T) -> (B, nh, T, T)\n att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(k.size(-1)))\n att = att.masked_fill(self.bias[:,:,:T,:T] == 0, float('-inf'))\n att = F.softmax(att, dim=-1)\n att = self.attn_dropout(att)\n y = att @ v # (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs)\n y = y.transpose(1, 2).contiguous().view(B, T, C) # re-assemble all head outputs side by side\n\n # output projection\n y = self.resid_dropout(self.c_proj(y))\n return y\n\nbatch_size = 128\nmax_seqlen = 1024\nseq_len = 512\nn_embd = 768\nn_head = 8\nattn_pdrop = 0.0\nresid_pdrop = 0.0\n\ndef get_inputs():\n return [torch.rand(batch_size, seq_len, n_embd)]\n\ndef get_init_inputs():\n return [n_embd, n_head, attn_pdrop, resid_pdrop, max_seqlen]```\n"}, "L3_045": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "UNetSoftmax", "split": "learn", "n_ops": 1, "src_lines": 88, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_045 (UNetSoftmax), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\n# U-Net Implementation\nclass DoubleConv(nn.Module):\n def __init__(self, in_channels, out_channels):\n super().__init__()\n self.double_conv = nn.Sequential(\n nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1),\n nn.BatchNorm2d(out_channels),\n nn.Softmax(dim=-1),\n nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1),\n nn.BatchNorm2d(out_channels),\n nn.Softmax(dim=-1)\n )\n\n def forward(self, x):\n return self.double_conv(x)\n\nclass Model(nn.Module):\n def __init__(self, in_channels, out_channels, features):\n \"\"\"\n :param in_channels: Number of input channels\n :param out_channels: Number of output channels\n :param features: Number of base features (will be doubled in each layer)\n \"\"\"\n super(Model, self).__init__()\n self.encoder1 = DoubleConv(in_channels, features)\n self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2)\n self.encoder2 = DoubleConv(features, features * 2)\n self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2)\n self.encoder3 = DoubleConv(features * 2, features * 4)\n self.pool3 = nn.MaxPool2d(kernel_size=2, stride=2)\n self.encoder4 = DoubleConv(features * 4, features * 8)\n self.pool4 = nn.MaxPool2d(kernel_size=2, stride=2)\n\n self.bottleneck = DoubleConv(features * 8, features * 16)\n\n self.upconv4 = nn.ConvTranspose2d(features * 16, features * 8, kernel_size=2, stride=2)\n self.decoder4 = DoubleConv(features * 16, features * 8)\n self.upconv3 = nn.ConvTranspose2d(features * 8, features * 4, kernel_size=2, stride=2)\n self.decoder3 = DoubleConv(features * 8, features * 4)\n self.upconv2 = nn.ConvTranspose2d(features * 4, features * 2, kernel_size=2, stride=2)\n self.decoder2 = DoubleConv(features * 4, features * 2)\n self.upconv1 = nn.ConvTranspose2d(features * 2, features, kernel_size=2, stride=2)\n self.decoder1 = DoubleConv(features * 2, features)\n\n self.final_conv = nn.Conv2d(features, out_channels, kernel_size=1)\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor, shape (batch_size, in_channels, height, width)\n :return: Output tensor, shape (batch_size, out_channels, height, width)\n \"\"\"\n enc1 = self.encoder1(x)\n enc2 = self.encoder2(self.pool1(enc1))\n enc3 = self.encoder3(self.pool2(enc2))\n enc4 = self.encoder4(self.pool3(enc3))\n\n bottleneck = self.bottleneck(self.pool4(enc4))\n\n dec4 = self.upconv4(bottleneck)\n dec4 = torch.cat((dec4, enc4), dim=1)\n dec4 = self.decoder4(dec4)\n dec3 = self.upconv3(dec4)\n dec3 = torch.cat((dec3, enc3), dim=1)\n dec3 = self.decoder3(dec3)\n dec2 = self.upconv2(dec3)\n dec2 = torch.cat((dec2, enc2), dim=1)\n dec2 = self.decoder2(dec2)\n dec1 = self.upconv1(dec2)\n dec1 = torch.cat((dec1, enc1), dim=1)\n dec1 = self.decoder1(dec1)\n\n return self.final_conv(dec1)\n \nbatch_size = 8\nin_channels = 8\nout_channels = 4\nheight = 64\nwidth = 512\nfeatures = 64\n# Test code for UNet\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, features]\n```\n"}, "L3_046": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_051 Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd, 36.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef add_kernel(x_ptr, s_ptr, out_ptr, n_feat, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n row = offs // n_feat\n v = tl.load(x_ptr + offs, mask=mask)\n s = tl.load(s_ptr + row, mask=mask)\n tl.store(out_ptr + offs, v + s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias=True):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n self.out_features = out_features\n\n def forward(self, x):\n # mean over output features of (Wx + b - subtract) = (colsum/out)\u00b7x + c\n colsum = self.gemm.weight.sum(dim=0) / self.out_features # (in,)\n if self.gemm.bias is not None:\n c = (self.gemm.bias - self.subtract).mean()\n else:\n c = (-self.subtract).mean()\n s = x @ colsum + c # (batch,)\n # logsumexp over single element (dim=1 after keepdim mean) is identity\n s = F.gelu(s)\n out = torch.empty_like(x)\n n = x.numel()\n n_feat = x.shape[1]\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n add_kernel[grid](x, s, out, n_feat, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10384, "ctx_sha": "4573cb1c62d0cdf8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 3, "name": "NetVladWithGhostClusters", "split": "learn", "n_ops": 1, "src_lines": 98, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 3 problem L3_046 (NetVladWithGhostClusters), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\n# Copyright 2018 Antoine Miech All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS-IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"\nCode modified from here\nhttps://github.com/albanie/collaborative-experts/blob/master/model/net_vlad.py\n\"\"\"\n\n\nimport math\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch as th\n\n\nclass Model(nn.Module):\n def __init__(self, cluster_size, feature_size, ghost_clusters):\n super(Model, self).__init__()\n\n self.feature_size = feature_size\n self.cluster_size = cluster_size\n self.ghost_clusters = ghost_clusters\n\n init_sc = (1 / math.sqrt(feature_size))\n clusters = cluster_size + ghost_clusters\n\n # The `clusters` weights are the `(w,b)` in the paper\n self.clusters = nn.Parameter(init_sc * th.randn(feature_size, clusters))\n self.batch_norm = nn.BatchNorm1d(clusters)\n # The `clusters2` weights are the visual words `c_k` in the paper\n self.clusters2 = nn.Parameter(init_sc * th.randn(1, feature_size, cluster_size))\n self.out_dim = self.cluster_size * feature_size\n\n def forward(self, x, mask=None):\n \"\"\"Aggregates feature maps into a fixed size representation. In the following\n notation, B = batch_size, N = num_features, K = num_clusters, D = feature_size.\n\n Args:\n x (th.Tensor): B x N x D\n\n Returns:\n (th.Tensor): B x DK\n \"\"\"\n max_sample = x.size()[1]\n x = x.view(-1, self.feature_size) # B x N x D -> BN x D\n\n if x.device != self.clusters.device:\n msg = f\"x.device {x.device} != cluster.device {self.clusters.device}\"\n raise ValueError(msg)\n\n assignment = th.matmul(x, self.clusters) # (BN x D) x (D x (K+G)) -> BN x (K+G)\n assignment = self.batch_norm(assignment)\n\n assignment = F.softmax(assignment, dim=1) # BN x (K+G) -> BN x (K+G)\n # remove ghost assigments\n assignment = assignment[:, :self.cluster_size]\n assignment = assignment.view(-1, max_sample, self.cluster_size) # -> B x N x K\n a_sum = th.sum(assignment, dim=1, keepdim=True) # B x N x K -> B x 1 x K\n a = a_sum * self.clusters2\n\n assignment = assignment.transpose(1, 2) # B x N x K -> B x K x N\n\n x = x.view(-1, max_sample, self.feature_size) # BN x D -> B x N x D\n vlad = th.matmul(assignment, x) # (B x K x N) x (B x N x D) -> B x K x D\n vlad = vlad.transpose(1, 2) # -> B x D x K\n vlad = vlad - a\n\n # L2 intra norm\n vlad = F.normalize(vlad)\n\n # flattening + L2 norm\n vlad = vlad.reshape(-1, self.cluster_size * self.feature_size) # -> B x DK\n vlad = F.normalize(vlad)\n return vlad # B x DK\n\nbatch_size = 2048\nnum_features = 100\nnum_clusters = 32\nfeature_size = 512\nghost_clusters = 16\n\ndef get_inputs():\n return [torch.rand(batch_size, num_features, feature_size)]\n\ndef get_init_inputs():\n return [num_clusters, feature_size, ghost_clusters]\n```\n"}, "L3_047": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_051 Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd, 36.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef add_kernel(x_ptr, s_ptr, out_ptr, n_feat, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n row = offs // n_feat\n v = tl.load(x_ptr + offs, mask=mask)\n s = tl.load(s_ptr + row, mask=mask)\n tl.store(out_ptr + offs, v + s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias=True):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n self.subtract = nn.Parameter(torch.randn(out_features))\n self.out_features = out_features\n\n def forward(self, x):\n # mean over output features of (Wx + b - subtract) = (colsum/out)\u00b7x + c\n colsum = self.gemm.weight.sum(dim=0) / self.out_features # (in,)\n if self.gemm.bias is not None:\n c = (self.gemm.bias - self.subtract).mean()\n else:\n c = (-self.subtract).mean()\n s = x @ colsum + c # (batch,)\n # logsumexp over single element (dim=1 after keepdim mean) is identity\n s = F.gelu(s)\n out = torch.empty_like(x)\n n = x.numel()\n n_feat = x.shape[1]\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n add_kernel[grid](x, s, out, n_feat, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L1_015 Matmul_for_lower_triangular_matrices, 4.58x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef tril_mm_kernel(A_ptr, B_ptr, C_ptr, N,\n BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, BLOCK_K: tl.constexpr):\n pid_m = tl.program_id(0)\n pid_n = tl.program_id(1)\n\n m_lo = pid_m * BLOCK_M\n n_lo = pid_n * BLOCK_N\n\n if n_lo > (m_lo + BLOCK_M - 1):\n return\n\n offs_m = m_lo + tl.arange(0, BLOCK_M)\n offs_n = n_lo + tl.arange(0, BLOCK_N)\n offs_k = tl.arange(0, BLOCK_K)\n\n acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32)\n\n k_start = (n_lo // BLOCK_K) * BLOCK_K\n k_end = tl.minimum(m_lo + BLOCK_M, N)\n\n k = k_start\n while k < k_end:\n kk = k + offs_k\n a = tl.load(A_ptr + offs_m[:, None] * N + kk[None, :],\n mask=(offs_m[:, None] < N) & (kk[None, :] < N), other=0.0)\n b = tl.load(B_ptr + kk[:, None] * N + offs_n[None, :],\n mask=(kk[:, None] < N) & (offs_n[None, :] < N), other=0.0)\n acc += tl.dot(a, b, allow_tf32=False)\n k += BLOCK_K\n\n mask_tri = offs_m[:, None] >= offs_n[None, :]\n acc = tl.where(mask_tri, acc, 0.0)\n store_mask = (offs_m[:, None] < N) & (offs_n[None, :] < N)\n tl.store(C_ptr + offs_m[:, None] * N + offs_n[None, :], acc, mask=store_mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self):\n super().__init__()\n\n def forward(self, A, B):\n A = A.contiguous()\n B = B.contiguous()\n N = A.shape[0]\n C = torch.zeros_like(A)\n BLOCK_M = 64\n BLOCK_N = 64\n BLOCK_K = 32\n grid = (triton.cdiv(N, BLOCK_M), triton.cdiv(N, BLOCK_N))\n tril_mm_kernel[grid](A, B, C, N, BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, BLOCK_K=BLOCK_K,\n num_warps=4, num_stages=3)\n return C\n```", "ctx_chars": 10384, "ctx_sha": "4573cb1c62d0cdf8", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 3, "name": "NetVladNoGhostClusters", "split": "learn", "n_ops": 1, "src_lines": 98, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 3 problem L3_047 (NetVladNoGhostClusters), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\n# Copyright 2018 Antoine Miech All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS-IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"\nCode modified from here\nhttps://github.com/albanie/collaborative-experts/blob/master/model/net_vlad.py\n\"\"\"\n\n\nimport math\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch as th\n\n\nclass Model(nn.Module):\n def __init__(self, cluster_size, feature_size, ghost_clusters):\n super(Model, self).__init__()\n\n self.feature_size = feature_size\n self.cluster_size = cluster_size\n self.ghost_clusters = ghost_clusters\n\n init_sc = (1 / math.sqrt(feature_size))\n clusters = cluster_size + ghost_clusters\n\n # The `clusters` weights are the `(w,b)` in the paper\n self.clusters = nn.Parameter(init_sc * th.randn(feature_size, clusters))\n self.batch_norm = nn.BatchNorm1d(clusters)\n # The `clusters2` weights are the visual words `c_k` in the paper\n self.clusters2 = nn.Parameter(init_sc * th.randn(1, feature_size, cluster_size))\n self.out_dim = self.cluster_size * feature_size\n\n def forward(self, x, mask=None):\n \"\"\"Aggregates feature maps into a fixed size representation. In the following\n notation, B = batch_size, N = num_features, K = num_clusters, D = feature_size.\n\n Args:\n x (th.Tensor): B x N x D\n\n Returns:\n (th.Tensor): B x DK\n \"\"\"\n max_sample = x.size()[1]\n x = x.view(-1, self.feature_size) # B x N x D -> BN x D\n\n if x.device != self.clusters.device:\n msg = f\"x.device {x.device} != cluster.device {self.clusters.device}\"\n raise ValueError(msg)\n\n assignment = th.matmul(x, self.clusters) # (BN x D) x (D x (K+G)) -> BN x (K+G)\n assignment = self.batch_norm(assignment)\n\n assignment = F.softmax(assignment, dim=1) # BN x (K+G) -> BN x (K+G)\n # remove ghost assigments\n assignment = assignment[:, :self.cluster_size]\n assignment = assignment.view(-1, max_sample, self.cluster_size) # -> B x N x K\n a_sum = th.sum(assignment, dim=1, keepdim=True) # B x N x K -> B x 1 x K\n a = a_sum * self.clusters2\n\n assignment = assignment.transpose(1, 2) # B x N x K -> B x K x N\n\n x = x.view(-1, max_sample, self.feature_size) # BN x D -> B x N x D\n vlad = th.matmul(assignment, x) # (B x K x N) x (B x N x D) -> B x K x D\n vlad = vlad.transpose(1, 2) # -> B x D x K\n vlad = vlad - a\n\n # L2 intra norm\n vlad = F.normalize(vlad)\n\n # flattening + L2 norm\n vlad = vlad.reshape(-1, self.cluster_size * self.feature_size) # -> B x DK\n vlad = F.normalize(vlad)\n return vlad # B x DK\n\nbatch_size = 2048\nnum_features = 100\nnum_clusters = 32\nfeature_size = 512\nghost_clusters = 0\n\ndef get_inputs():\n return [torch.rand(batch_size, num_features, feature_size)]\n\ndef get_init_inputs():\n return [num_clusters, feature_size, ghost_clusters]\n```\n"}, "L3_049": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 3, "name": "Mamba2ReturnFinalState", "split": "learn", "n_ops": 1, "src_lines": 91, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_049 (Mamba2ReturnFinalState), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom einops import rearrange\n\nclass Model(nn.Module):\n def __init__(self, batch_size, seq_length, n_heads, d_head, d_state, block_len=64):\n \"\"\"\n Mamba Structured State Space model implementation for benchmarking.\n \n :param batch_size: Size of the batch\n :param seq_length: Length of the input sequence\n :param n_heads: Number of attention heads\n :param d_head: Dimension of each head\n :param d_state: Dimension of the state space\n :param block_len: Length of each block for chunked computation\n \"\"\"\n super(Model, self).__init__()\n \n assert seq_length % block_len == 0, \"Sequence length must be divisible by block length\"\n \n self.batch_size = batch_size\n self.seq_length = seq_length\n self.n_heads = n_heads\n self.d_head = d_head\n self.d_state = d_state\n self.block_len = block_len\n \n # Initialize parameters\n self.A = nn.Parameter(torch.randn(batch_size, seq_length, n_heads))\n self.B = nn.Parameter(torch.randn(batch_size, seq_length, n_heads, d_state))\n self.C = nn.Parameter(torch.randn(batch_size, seq_length, n_heads, d_state))\n \n def segsum(self, x):\n \"\"\"Naive segment sum calculation.\"\"\"\n T = x.size(-1)\n x_cumsum = torch.cumsum(x, dim=-1)\n x_segsum = x_cumsum[..., :, None] - x_cumsum[..., None, :]\n mask = torch.tril(torch.ones(T, T, device=x.device, dtype=bool), diagonal=0)\n x_segsum = x_segsum.masked_fill(~mask, -torch.inf)\n return x_segsum\n \n def forward(self, X, initial_states=None):\n \"\"\"\n Forward pass implementing the SSD operation.\n \n :param X: Input tensor of shape (batch, length, n_heads, d_head)\n :param initial_states: Optional initial states\n :return: Output tensor Y and final state\n \"\"\"\n # Rearrange into blocks/chunks\n X_blocks, A_blocks, B_blocks, C_blocks = [\n rearrange(x, \"b (c l) ... -> b c l ...\", l=self.block_len)\n for x in (X, self.A, self.B, self.C)\n ]\n \n A_blocks = rearrange(A_blocks, \"b c l h -> b h c l\")\n A_cumsum = torch.cumsum(A_blocks, dim=-1)\n \n # 1. Compute diagonal block outputs\n L = torch.exp(self.segsum(A_blocks))\n Y_diag = torch.einsum(\"bclhn,bcshn,bhcls,bcshp->bclhp\", \n C_blocks, B_blocks, L, X_blocks)\n \n # 2. Compute intra-chunk states\n decay_states = torch.exp((A_cumsum[:, :, :, -1:] - A_cumsum))\n states = torch.einsum(\"bclhn,bhcl,bclhp->bchpn\", \n B_blocks, decay_states, X_blocks)\n \n # 3. Compute inter-chunk recurrence\n if initial_states is None:\n initial_states = torch.zeros_like(states[:, :1])\n states = torch.cat([initial_states, states], dim=1)\n \n decay_chunk = torch.exp(self.segsum(F.pad(A_cumsum[:, :, :, -1], (1, 0))))\n new_states = torch.einsum(\"bhzc,bchpn->bzhpn\", decay_chunk, states)\n return new_states[:, -1]\n\n# Test parameters\nbatch_size = 2048\nseq_length = 128\nn_heads = 8\nd_head = 64\nd_state = 16\nblock_len = 64\n\ndef get_inputs():\n return [torch.rand(batch_size, seq_length, n_heads, d_head)]\n\ndef get_init_inputs():\n return [batch_size, seq_length, n_heads, d_head, d_state, block_len]\n```\n"}, "L2_020": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_078 ConvTranspose3d_Max_Max_Sum, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, N, C, D, H, W, OD, OH, OW, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n idx64 = idx.to(tl.int64)\n mask = idx < total\n\n ow = idx64 % OW\n t = idx64 // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n n = t // OD\n\n d0 = od * 6\n h0 = oh * 6\n w0 = ow * 6\n\n C64 = C.to(tl.int64)\n D64 = D.to(tl.int64)\n H64 = H.to(tl.int64)\n W64 = W.to(tl.int64)\n\n s = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n base_nc = (n * C64 + c) * D64\n m = tl.full((BLOCK,), -1e30, tl.float32)\n for dz in tl.static_range(6):\n iz = d0 + dz\n for dy in tl.static_range(6):\n iy = h0 + dy\n for dx in tl.static_range(6):\n ix = w0 + dx\n off = ((base_nc + iz) * H64 + iy) * W64 + ix\n v = tl.load(x_ptr + off, mask=mask, other=-1e30)\n m = tl.maximum(m, v)\n s += m\n tl.store(out_ptr + idx64, s, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n P1D, P1H, P1W = D // 2, H // 2, W // 2\n OD, OH, OW = P1D // 3, P1H // 3, P1W // 3\n out = torch.empty((N, 1, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * OD * OH * OW\n BLOCK = 128\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](c, out, N, C, D, H, W, OD, OH, OW, total, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related conv_transpose problem (L2_074 ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max, 1.26x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, mult_ptr, N, C, D, H, W, OD, OH, OW,\n slope, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n mask = idx < total\n\n ow = idx % OW\n t = idx // OW\n oh = t % OH\n t = t // OH\n od = t % OD\n nc = t // OD\n c = nc % C\n\n m = tl.load(mult_ptr + c, mask=mask, other=0.0)\n\n acc = tl.full((BLOCK,), -1e30, tl.float32)\n for dz in tl.static_range(2):\n for dy in tl.static_range(2):\n for dx in tl.static_range(2):\n iz = od * 2 + dz\n iy = oh * 2 + dy\n ix = ow * 2 + dx\n in_off = (((nc * D + iz) * H + iy) * W + ix)\n v = tl.load(x_ptr + in_off, mask=mask, other=0.0)\n v = tl.where(v >= 0, v, v * slope)\n v = v * m\n v = tl.where(v >= 0, v, v * slope)\n acc = tl.maximum(acc, v)\n tl.store(out_ptr + idx, acc, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, multiplier_shape):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.multiplier = nn.Parameter(torch.randn(multiplier_shape))\n\n def forward(self, x):\n c = self.conv_transpose(x).contiguous()\n N, C, D, H, W = c.shape\n OD, OH, OW = D // 2, H // 2, W // 2\n out = torch.empty((N, C, OD, OH, OW), device=c.device, dtype=c.dtype)\n total = N * C * OD * OH * OW\n mult = self.multiplier.contiguous().view(-1)\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](c, out, mult, N, C, D, H, W, OD, OH, OW, 0.2, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10756, "ctx_sha": "52a5fe04bf576804", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_Sum_ResidualAdd_Multiply_ResidualAdd", "split": "dev", "n_ops": 5, "src_lines": 37, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_020 (ConvTranspose3d_Sum_ResidualAdd_Multiply_ResidualAdd), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, followed by a sum, \n a residual add, a multiplication, and another residual add.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, bias_shape):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n x = self.conv_transpose(x)\n original_x = x.clone().detach()\n x = x + self.bias\n x = x + original_x\n x = x * original_x\n x = x + original_x\n return x\n\nbatch_size = 16\nin_channels = 32\nout_channels = 64\ndepth, height, width = 16, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\noutput_padding = 1\nbias_shape = (out_channels, 1, 1, 1)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, output_padding, bias_shape]```\n"}, "L2_037": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_062 Matmul_GroupNorm_LeakyReLU_Sum, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef gn_kernel(x_ptr, w_ptr, b_ptr, out_ptr,\n N, C, G, CG, eps, neg_slope,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n base = row * C + grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n v = tl.load(x_ptr + base + offs, mask=mask, other=0.0)\n s = tl.sum(v, axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n rstd = 1.0 / tl.sqrt(var + eps)\n coff = grp * CG + offs\n w = tl.load(w_ptr + coff, mask=mask, other=0.0)\n b = tl.load(b_ptr + coff, mask=mask, other=0.0)\n y = (v - mean) * rstd * w + b\n y = tl.where(y >= 0, y, y * neg_slope)\n y = y + y\n tl.store(out_ptr + base + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, num_groups, eps=1e-5, negative_slope=0.01):\n super().__init__()\n self.fc = nn.Linear(input_size, hidden_size)\n self.gn = nn.GroupNorm(num_groups=num_groups, num_channels=hidden_size, eps=eps)\n self.leaky_relu = nn.LeakyReLU(negative_slope=negative_slope)\n self.num_groups = num_groups\n self.hidden_size = hidden_size\n self.eps = eps\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = self.fc(x).contiguous()\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n gn_kernel[grid](z, self.gn.weight, self.gn.bias, out,\n N, C, G, CG, self.eps, self.negative_slope,\n BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_059 Matmul_Swish_Scaling, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef swish_scale_kernel(x_ptr, out_ptr, scale, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = v * s * scale\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = F.linear(x, self.matmul.weight, self.matmul.bias).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n swish_scale_kernel[grid](z, out, float(self.scaling_factor), n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 9893, "ctx_sha": "b7d3b378c9e92c33", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Swish_Sum_GroupNorm", "split": "dev", "n_ops": 4, "src_lines": 37, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_037 (Matmul_Swish_Sum_GroupNorm), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a matrix multiplication, applies Swish activation, sums with a bias term, and normalizes with GroupNorm.\n \"\"\"\n def __init__(self, in_features, out_features, num_groups, bias_shape):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.group_norm = nn.GroupNorm(num_groups, out_features)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.matmul(x)\n x = torch.sigmoid(x) * x # Swish activation\n x = x + self.bias\n x = self.group_norm(x)\n return x\n\nbatch_size = 32768\nin_features = 1024\nout_features = 4096\nnum_groups = 64\nbias_shape = (out_features,)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, num_groups, bias_shape]```\n"}, "L2_040": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_059 Matmul_Swish_Scaling, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef swish_scale_kernel(x_ptr, out_ptr, scale, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = v * s * scale\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, scaling_factor):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = F.linear(x, self.matmul.weight, self.matmul.bias).contiguous()\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n swish_scale_kernel[grid](z, out, float(self.scaling_factor), n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_070 Gemm_Sigmoid_Scaling_ResidualAdd, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef epilogue_kernel(x_ptr, out_ptr, scaling, n_elements, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elements\n v = tl.load(x_ptr + offs, mask=mask)\n s = 1.0 / (1.0 + tl.exp(-v))\n y = s * scaling + v\n tl.store(out_ptr + offs, y, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, input_size, hidden_size, scaling_factor):\n super().__init__()\n self.gemm = nn.Linear(input_size, hidden_size)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n z = self.gemm(x)\n out = torch.empty_like(z)\n n = z.numel()\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n epilogue_kernel[grid](z, out, self.scaling_factor, n, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 8989, "ctx_sha": "c705b15bb2c57b08", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Matmul_Scaling_ResidualAdd", "split": "dev", "n_ops": 3, "src_lines": 43, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_040 (Matmul_Scaling_ResidualAdd), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n A model that performs a matrix multiplication, scaling, and residual addition.\n\n Args:\n in_features (int): Number of input features.\n out_features (int): Number of output features.\n scaling_factor (float): Scaling factor to apply after matrix multiplication.\n \"\"\"\n def __init__(self, in_features, out_features, scaling_factor):\n super(Model, self).__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.scaling_factor = scaling_factor\n\n def forward(self, x):\n \"\"\"\n Forward pass of the model.\n\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.matmul(x)\n original_x = x.clone().detach()\n x = x * self.scaling_factor\n x = x + original_x\n return x\n\nbatch_size = 16384\nin_features = 4096\nout_features = 4096\nscaling_factor = 0.5\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, scaling_factor]```\n"}, "L2_058": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier conv_transpose problems (wrong results or slower than eager)\n- Pitfall: Avoid launching a per-forward weight-transform kernel (flip/transpose) before a library call. Cache the transformed weight once, or fuse the transform, since extra launches and memory traffic make it slower than eager.\n- Pitfall: Don'...'s native transposed conv is usually as fast or faster, so this measured slower.\n- Pitfall: In Triton kernels over large tensors (e.g. 3D conv outputs), compute offsets in int64 (cast program_id/indices with .to(tl.int64)). int32 products like n*C*S can overflow and crash, leaving no output during timing.\n- Pitfall: Never launch a pure copy/identity kernel into a fresh buffer just to have a custom kernel; that adds a full memory round-trip and guarantees slowdown versus eager.\n- Pitfall: Don't rewrite conv_transpose as flipped-weight F.conv2d plus a separate elementwise kernel; the per-call weight flip and extra full-output pass make it slower. Fuse bias into a real kernel, or skip pointless copies.\n\n### Verified recipes for conv_transpose problems (speedups measured by the protected evaluator)\n- [2.29x on conv_transposed_2D_asymmetric_input_asymmetric_kernel___padded__] For stride-1 transposed conv, rewrite it as a regular cuDNN conv2d: build the weight with a tiny Triton kernel that swaps the IC/OC axes and flips KH/KW, (IC,OC,KH,KW)->(OC,IC,KH-1-kh,KW-1-kw). Then call F.conv2d with padding=(K-1-p) per dimension and pass the bias through. cuDNN's forward conv path is much faster than ConvTranspose's dgrad path. Keep the math in fp32 so it is exact. For stride>1, groups, dilation or output_padding, fall back to nn.ConvTranspose2d. Cache the flipped weight if it is static.\n- [1.68x on ConvTranspose3d_BatchNorm_AvgPool_AvgPool] Keep the ConvTranspose in cuDNN via torch. Fold training-mode BatchNorm into a per-channel affine: one Triton program per (n,c) row loops over contiguous spatial data in 1024-wide blocks, accumulating fp32 sum and sum-of-squares. Finish mean/var/scale/shift with tiny torch ops. Because BN is affine, apply it after pooling: replace the two 2x pools with one fused 4x4x4 average kernel. Use one thread per output, int64 index math, static_range unrolled loads, and scale*avg+shift at store.\n- [1.32x on ConvTranspose3d_BatchNorm_Subtract] Keep ConvTranspose in cuDNN via torch. Fuse everything after it into two Triton passes over the contiguous (N*C, S) view. Pass 1: one program per (n,c) row, loop over S in 1024-element blocks, accumulate fp32 sum and sum-of-squares. Finish the tiny per-channel math in torch: batch mean/var (training-mode BN) and scale = gamma/sqrt(var+eps). Pass 2: write scale*(x - row_mean) in one read-modify-write sweep. BN's beta, shift and the conv bias cancel algebraically, so fold them away. fp32 accumulation meets 1e-4.\n- [1.26x on ConvTranspose3d_Sum_LayerNorm_AvgPool_GELU] Keep ConvTranspose in cuDNN via torch and fuse only the epilogue in Triton. LayerNorm normalises the last dim (W), not channels, so treat the output as a 2D [rows, W] view. Launch one program per row with BLOCK=next_pow2(W), add scalar bias before the mean, take a two-pass variance in fp32, and compute rstd as 1/sqrt(var+eps). Then run a second elementwise kernel fusing avgpool (static_range window loads, multiply by 1/k) with exact erf-GELU. Use int64 index math. Better still, fuse LN into the pool kernel to avoid the intermediate write.\n- [1.26x on ConvTranspose3d_LeakyReLU_Multiply_LeakyReLU_Max] Keep the ConvTranspose3d itself in cuDNN via nn.ConvTranspose3d and fuse the entire epilogue into one Triton kernel that reads the conv output once: LeakyReLU, per-channel scale, LeakyReLU, MaxPool. Launch one thread per pooled output element over a flat index (BLOCK=256, 1D grid). Decode n\u00b7c/od/oh/ow with div/mod, load the per-channel multiplier once, and unroll the 2x2x2 window with tl.static_range. Use tl.where for the slopes and a -1e30 init for max. Keep all math fp32, which stays within 1e-4.\n- [1.06x on ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max] Keep ConvTranspose3d (and MaxPool3d) in cuDNN via torch and call .contiguous() on the result. Fuse all channel-wise epilogues (softmax over C, per-channel bias/subtract, activation, reduction over C) into one Triton kernel. Map one lane per (n, spatial) position with a flat 1D grid and BLOCK=256. Loop over the C channels with stride-S loads, so consecutive lanes read consecutive spatial addresses (coalesced). Use a stable 3-pass softmax (max, sum-exp, normalize+epilogue) in fp32. Re-reads hit L1/L2. Write only the reduced output.\n\n### A verified kernel for a related conv_transpose problem (L2_015 ConvTranspose3d_BatchNorm_Subtract, 1.32x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef reduce_kernel(x_ptr, sum_ptr, sqsum_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n acc = tl.zeros((), dtype=tl.float32)\n accsq = tl.zeros((), dtype=tl.float32)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n acc += tl.sum(v)\n accsq += tl.sum(v * v)\n tl.store(sum_ptr + pid, acc)\n tl.store(sqsum_ptr + pid, accsq)\n\n\n@triton.jit\ndef out_kernel(x_ptr, out_ptr, scale_ptr, smean_ptr, S, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = tl.arange(0, BLOCK)\n base = pid * S\n sc = tl.load(scale_ptr + pid)\n sm = tl.load(smean_ptr + pid)\n for i in range(0, S, BLOCK):\n idx = i + offs\n m = idx < S\n v = tl.load(x_ptr + base + idx, mask=m, other=0.0)\n r = sc * (v - sm)\n tl.store(out_ptr + base + idx, r, mask=m)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias=True):\n super(ModelNew, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias=bias)\n self.batch_norm = nn.BatchNorm3d(out_channels)\n\n def forward(self, x):\n c = self.conv_transpose(x)\n N, C, D, H, W = c.shape\n S = D * H * W\n c2 = c.contiguous().view(N * C, S)\n\n ssum = torch.empty(N * C, device=c.device, dtype=torch.float32)\n ssq = torch.empty(N * C, device=c.device, dtype=torch.float32)\n reduce_kernel[(N * C,)](c2, ssum, ssq, S, BLOCK=1024)\n\n ssum_nc = ssum.view(N, C)\n ssq_nc = ssq.view(N, C)\n total_sum = ssum_nc.sum(0)\n total_sq = ssq_nc.sum(0)\n M = N * S\n mean_c = total_sum / M\n var_c = total_sq / M - mean_c * mean_c\n scale_c = self.batch_norm.weight / torch.sqrt(var_c + self.batch_norm.eps) # (C,)\n\n scale_nc = scale_c.unsqueeze(0).expand(N, C).contiguous().view(N * C)\n smean_nc = (ssum_nc / S).contiguous().view(N * C)\n\n out = torch.empty_like(c2)\n out_kernel[(N * C,)](c2, out, scale_nc, smean_nc, S, BLOCK=1024)\n return out.view(N, C, D, H, W)\n```\n\n### A verified kernel for a related conv_transpose problem (L2_089 ConvTranspose3d_MaxPool_Softmax_Subtract_Swish_Max, 1.06x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, sub_ptr, out_ptr, N, C, S, total, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n idx = pid * BLOCK + tl.arange(0, BLOCK)\n mask = idx < total\n\n # idx indexes (n, s) flat where s in [0,S)\n n = idx // S\n s = idx % S\n base = n * C * S + s\n\n # pass 1: max over channels\n mx = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=-1e30)\n mx = tl.maximum(mx, v)\n\n # pass 2: sum of exp\n sm = tl.zeros((BLOCK,), tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n sm += tl.exp(v - mx)\n\n # pass 3: softmax, subtract, swish, max over channels\n res = tl.full((BLOCK,), -1e30, tl.float32)\n for c in range(0, C):\n v = tl.load(x_ptr + base + c * S, mask=mask, other=0.0)\n sub = tl.load(sub_ptr + c)\n sm_val = tl.exp(v - mx) / sm\n y = sm_val - sub\n swish = (tl.sigmoid(y) * y)\n res = tl.maximum(res, swish)\n\n tl.store(out_ptr + idx, res, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, output_padding, pool_kernel_size, pool_stride, pool_padding):\n super().__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, output_padding=output_padding)\n self.max_pool = nn.MaxPool3d(kernel_size=pool_kernel_size, stride=pool_stride, padding=pool_padding)\n self.subtract = nn.Parameter(torch.randn(out_channels))\n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = self.max_pool(x).contiguous()\n N, C, D, H, W = x.shape\n S = D * H * W\n total = N * S\n\n out = torch.empty((N, D, H, W), device=x.device, dtype=x.dtype)\n sub = self.subtract.contiguous()\n\n BLOCK = 256\n grid = (triton.cdiv(total, BLOCK),)\n fused_kernel[grid](x, sub, out, N, C, S, total, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 11393, "ctx_sha": "d0a999376e3d430c", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 5, "recipe": 6, "kernel": 2}, "family": "conv_transpose", "level": 2, "name": "ConvTranspose3d_LogSumExp_HardSwish_Subtract_Clamp", "split": "dev", "n_ops": 5, "src_lines": 34, "max_recipe_speedup": 2.29, "instruction": "KernelBench level 2 problem L2_058 (ConvTranspose3d_LogSumExp_HardSwish_Subtract_Clamp), op family: conv_transpose.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a 3D transposed convolution, LogSumExp, HardSwish, subtraction, clamp operations.\n \"\"\"\n def __init__(self, in_channels, out_channels, kernel_size, stride, padding, bias_shape):\n super(Model, self).__init__()\n self.conv_transpose = nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=stride, padding=padding)\n self.bias = nn.Parameter(torch.randn(1, 1, 1, 1)) \n\n def forward(self, x):\n x = self.conv_transpose(x)\n x = torch.logsumexp(x, dim=1, keepdim=True)\n x = x * torch.sigmoid(x + 3) / 6\n x = x - self.bias\n x = torch.clamp(x, min=-1, max=1)\n return x\n\nbatch_size = 128\nin_channels = 3\nout_channels = 16\ndepth, height, width = 16, 32, 32\nkernel_size = 3\nstride = 2\npadding = 1\nbias_shape = (1, 1, 1, 1) \n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, depth, height, width)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, kernel_size, stride, padding, bias_shape]\n```\n"}, "L2_081": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_097 Matmul_BatchNorm_BiasAdd_Divide_Swish, 1.02x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef stats_kernel(x_ptr, mean_ptr, var_ptr, M, N, BLOCK_M: tl.constexpr):\n col = tl.program_id(0)\n offs = tl.arange(0, BLOCK_M)\n acc = tl.zeros((), dtype=tl.float32)\n acc2 = tl.zeros((), dtype=tl.float32)\n for start in range(0, M, BLOCK_M):\n idx = start + offs\n mask = idx < M\n v = tl.load(x_ptr + idx * N + col, mask=mask, other=0.0)\n acc += tl.sum(v)\n acc2 += tl.sum(v * v)\n mean = acc / M\n var = acc2 / M - mean * mean\n tl.store(mean_ptr + col, mean)\n tl.store(var_ptr + col, var)\n\n@triton.jit\ndef apply_kernel(x_ptr, out_ptr, mean_ptr, var_ptr, gamma_ptr, beta_ptr, bias_ptr,\n eps, divide_value, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n mean = tl.load(mean_ptr + col, mask=mask, other=0.0)\n var = tl.load(var_ptr + col, mask=mask, other=0.0)\n gamma = tl.load(gamma_ptr + col, mask=mask, other=0.0)\n beta = tl.load(beta_ptr + col, mask=mask, other=0.0)\n bias = tl.load(bias_ptr)\n inv = 1.0 / tl.sqrt(var + eps)\n y = (v - mean) * inv * gamma + beta\n y = (y + bias) / divide_value\n y = y * (1.0 / (1.0 + tl.exp(-y)))\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bn_eps=1e-5, bn_momentum=0.1, bias_shape=(1,), divide_value=1.0):\n super().__init__()\n self.matmul = nn.Linear(in_features, out_features)\n self.bn = nn.BatchNorm1d(out_features, eps=bn_eps, momentum=bn_momentum)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.divide_value = divide_value\n self.bn_eps = bn_eps\n self.bn_momentum = bn_momentum\n\n def forward(self, x):\n z = self.matmul(x).contiguous()\n M, N = z.shape\n mean = torch.empty(N, device=z.device, dtype=torch.float32)\n var = torch.empty(N, device=z.device, dtype=torch.float32)\n stats_kernel[(N,)](z, mean, var, M, N, BLOCK_M=1024)\n\n with torch.no_grad():\n self.bn.running_mean.mul_(1 - self.bn_momentum).add_(self.bn_momentum * mean)\n unbiased = var * (M / (M - 1))\n self.bn.running_var.mul_(1 - self.bn_momentum).add_(self.bn_momentum * unbiased)\n self.bn.num_batches_tracked.add_(1)\n\n out = torch.empty_like(z)\n bias_flat = self.bias.reshape(-1)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n apply_kernel[grid](z, out, mean, var, self.bn.weight, self.bn.bias, bias_flat,\n self.bn_eps, self.divide_value, M, N, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_076 Gemm_Add_ReLU, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bias_relu_kernel(x_ptr, bias_ptr, out_ptr, M, N, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < M * N\n col = offs % N\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n b = tl.load(bias_ptr + col, mask=mask, other=0.0)\n y = tl.maximum(v + b, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=False)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n\n def forward(self, x):\n z = torch.mm(x, self.gemm.weight.t())\n M, N = z.shape\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(M * N, BLOCK),)\n bias_relu_kernel[grid](z, self.bias, out, M, N, BLOCK=BLOCK)\n return out\n```", "ctx_chars": 10901, "ctx_sha": "baed172422eeb8d7", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_Swish_Divide_Clamp_Tanh_Clamp", "split": "dev", "n_ops": 6, "src_lines": 35, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_081 (Gemm_Swish_Divide_Clamp_Tanh_Clamp), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Simple model that performs a gemm, swish, divide, clamp, tanh, and clamp operations.\n \"\"\"\n def __init__(self, in_features, out_features, bias=True):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features, bias=bias)\n\n def forward(self, x):\n \"\"\"\n Args:\n x (torch.Tensor): Input tensor of shape (batch_size, in_features).\n Returns:\n torch.Tensor: Output tensor of shape (batch_size, out_features).\n \"\"\"\n x = self.gemm(x)\n x = x * torch.sigmoid(x) # Swish activation\n x = x / 2.0\n x = torch.clamp(x, min=-1.0, max=1.0) # Clamp between -1 and 1\n x = torch.tanh(x) # Tanh activation\n x = torch.clamp(x, min=-1.0, max=1.0) # Clamp between -1 and 1\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features]```\n"}, "L2_088": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier matmul problems (wrong results or slower than eager)\n- Pitfall: When using split-K, keep the split count small; a very large split writes and re-reads a huge partial-sum buffer plus an extra reduction launch, eating the occupancy gain.\n- Pitfall: Strict fp32 Triton tl.dot (allow_tf32=False) rarely beats cuBLAS on matmul; a hand split-K version was correct but still 0.88x. Fuse surrounding ops or exploit structure instead of reimplementing plain GEMM.\n- Pitfall: For bandwidth-bound Triton kernels, give each program more work: larger tiles, multiple rows per program, more warps. Tiny per-program work and Python launch overhead lose to cuBLAS/ATen kernels.\n- Pitfall: Don't expect a single-pass memory-bound elementwise or broadcast kernel to beat eager, which already runs near bandwidth peak. Win by fusing multiple ops or reducing memory traffic.\n- Pitfall: If writing a Triton GEMM anyway, use @triton.autotune over several BM/BN/BK, num_warps and num_stages configs, plus grouped tile ordering. A single fixed 64x64x32 config is far slower than eager.\n- Pitfall: Do not hand-write a plain fp32 matmul with TF32 disabled; it loses to cuBLAS. Call torch.matmul on transposed views (no .contiguous copies) and spend custom-kernel effort only on fusing surrounding ops.\n\n### Verified recipes for matmul problems (speedups measured by the protected evaluator)\n- [36.02x on Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd] Look for algebraic collapse before writing kernels. When a GEMM feeds a linear reduction (a mean or sum over output features), rewrite mean(Wx+b-s) as x\u00b7(W.sum(0)/N) + mean(b-s). This turns the GEMM into a GEMV kept in torch/cuBLAS. Check for degenerate ops: logsumexp over a size-1 dim is the identity, so drop it. Apply cheap scalar ops such as GELU on the reduced vector in torch. Fuse the final per-row broadcast plus residual add into one coalesced 1D Triton elementwise kernel (BLOCK=1024, row = offs // n_feat). fp32 summation stays within 1e-4.\n- [4.58x on Matmul_for_lower_triangular_matrices] Exploit structural sparsity (triangular, banded, diagonal) with a custom Triton tiled matmul instead of dense cuBLAS. Use a 2D grid of output tiles and return early for tiles that are entirely zero. Clamp the K-loop to the nonzero range: for tril\u00d7tril, run from floor(n_lo/BK)*BK to min(m_lo+BM, N). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the output mask with tl.where before the store. Pre-zero the output with torch.zeros so skipped tiles stay zero. 64x64x32 tiles, 4 warps, 3 stages.\n- [4.11x on Matmul_for_upper_triangular_matrices] Exploit structural sparsity (triangular, banded, or block-sparse) with a Triton tiled GEMM instead of dense cuBLAS. Use a 2D grid over output tiles, early-return tiles that are entirely masked out, and clip the K loop to the nonzero range (for triu\u00b7triu, k from the row-tile start to the column-tile end, aligned to BLOCK_K). Accumulate in fp32 with tl.dot(allow_tf32=False) to stay within 1e-4. Apply the triangle mask with tl.where before storing. Preallocate the output with torch.zeros so skipped tiles stay zero. Tiles 64x64x32, 4 warps, 3 stages.\n- [1.02x on Matmul_BatchNorm_BiasAdd_Divide_Swish] Keep the GEMM (nn.Linear) in cuBLAS via torch and fuse only the epilogue in Triton. For training-mode BatchNorm, run a per-column stats kernel that accumulates sum and sum-of-squares in fp32, giving biased variance for normalization. Update running_mean/var under no_grad using the unbiased var, M/(M-1). Then one flat elementwise kernel applies normalize, gamma/beta, bias, divide and Swish, indexing per-column params with offs % N. Gains are small because the GEMM dominates; next, coalesce stats loads with 2D column tiles.\n- [1.01x on Matmul_GroupNorm_LeakyReLU_Sum] Keep the Linear GEMM in cuBLAS via the module's own fc (F.linear with bias). Fuse every downstream per-row op into one Triton kernel reading the GEMM output once: GroupNorm, affine, LeakyReLU and x+x (as y*2). Use one program per (row, group), BLOCK=next_power_of_2(group size), contiguous masked loads. Compute the two-pass mean/variance in fp32 registers and mask padded lanes before squaring. Expect small gains, since the GEMM dominates; for tiny groups, batch several groups per program to cut launch and occupancy waste.\n- [1.01x on Gemm_Sigmoid_Scaling_ResidualAdd] Keep the GEMM in cuBLAS through nn.Linear / F.linear in fp32, with the bias folded in by cuBLAS. Replace the chain of eager elementwise ops (sigmoid, scale, residual add) with a single flat 1D Triton kernel: one load, compute s=1/(1+exp(-v)), y=s*scale+v, one store. Use contiguous masked blocks of 1024 and grid=cdiv(n,BLOCK). Writing in place into the GEMM output saves an allocation. The gain is small because the GEMM dominates; a bigger win requires a fused-epilogue GEMM.\n\n### A verified kernel for a related matmul problem (L2_012 Gemm_Multiply_LeakyReLU, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef fused_kernel(x_ptr, out_ptr, mult, neg_slope, n, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n\n v = tl.load(x_ptr + offs, mask=mask, other=0.0)\n v = v * mult\n v = tl.where(v >= 0, v, v * neg_slope)\n tl.store(out_ptr + offs, v, mask=mask)\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, multiplier, negative_slope):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.multiplier = multiplier\n self.negative_slope = negative_slope\n\n def forward(self, x):\n z = torch.addmm(self.gemm.bias, x, self.gemm.weight.t())\n n = z.numel()\n out = torch.empty_like(z)\n BLOCK = 1024\n grid = (triton.cdiv(n, BLOCK),)\n fused_kernel[grid](z, out, self.multiplier, self.negative_slope, n, BLOCK=BLOCK)\n return out\n```\n\n### A verified kernel for a related matmul problem (L2_094 Gemm_BiasAdd_Hardtanh_Mish_GroupNorm, 1.01x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport triton\nimport triton.language as tl\n\n\n@triton.jit\ndef fused_kernel(x_ptr, bias_ptr, gamma_ptr, beta_ptr, out_ptr,\n N, C, G, CG, eps,\n BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n row = pid // G\n grp = pid % G\n col0 = grp * CG\n offs = tl.arange(0, BLOCK)\n mask = offs < CG\n chan = col0 + offs\n idx = row * C + chan\n\n v = tl.load(x_ptr + idx, mask=mask, other=0.0)\n b = tl.load(bias_ptr + chan, mask=mask, other=0.0)\n v = v + b\n # hardtanh [-1,1]\n v = tl.minimum(tl.maximum(v, -1.0), 1.0)\n # mish: v * tanh(softplus(v))\n sp = tl.log(1.0 + tl.exp(v))\n v = v * (tl.exp(sp) - tl.exp(-sp)) / (tl.exp(sp) + tl.exp(-sp))\n\n # group norm stats\n s = tl.sum(tl.where(mask, v, 0.0), axis=0)\n mean = s / CG\n d = tl.where(mask, v - mean, 0.0)\n var = tl.sum(d * d, axis=0) / CG\n inv = 1.0 / tl.sqrt(var + eps)\n g = tl.load(gamma_ptr + chan, mask=mask, other=0.0)\n bt = tl.load(beta_ptr + chan, mask=mask, other=0.0)\n out = (v - mean) * inv * g + bt\n tl.store(out_ptr + idx, out, mask=mask)\n\n\nclass ModelNew(nn.Module):\n def __init__(self, in_features, out_features, bias_shape, num_groups):\n super().__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.bias = nn.Parameter(torch.randn(bias_shape))\n self.hardtanh = nn.Hardtanh()\n self.mish = nn.Mish()\n self.groupnorm = nn.GroupNorm(num_groups=num_groups, num_channels=out_features)\n self.num_groups = num_groups\n self.out_features = out_features\n\n def forward(self, x):\n z = self.gemm(x)\n N, C = z.shape\n G = self.num_groups\n CG = C // G\n out = torch.empty_like(z)\n BLOCK = triton.next_power_of_2(CG)\n grid = (N * G,)\n fused_kernel[grid](\n z, self.bias, self.groupnorm.weight, self.groupnorm.bias, out,\n N, C, G, CG, float(self.groupnorm.eps),\n BLOCK=BLOCK,\n )\n return out\n```", "ctx_chars": 10191, "ctx_sha": "346e890b9cb8e685", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 6, "recipe": 6, "kernel": 2}, "family": "matmul", "level": 2, "name": "Gemm_GroupNorm_Swish_Multiply_Swish", "split": "dev", "n_ops": 5, "src_lines": 37, "max_recipe_speedup": 36.02, "instruction": "KernelBench level 2 problem L2_088 (Gemm_GroupNorm_Swish_Multiply_Swish), op family: matmul.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\n\nclass Model(nn.Module):\n \"\"\"\n Model that performs a GEMM, GroupNorm, Swish, Multiply, and Swish operations.\n \"\"\"\n def __init__(self, in_features, out_features, num_groups, multiply_weight_shape):\n super(Model, self).__init__()\n self.gemm = nn.Linear(in_features, out_features)\n self.group_norm = nn.GroupNorm(num_groups, out_features)\n self.multiply_weight = nn.Parameter(torch.randn(multiply_weight_shape)) \n\n def forward(self, x):\n # (batch_size, in_features) -> (batch_size, out_features)\n x = self.gemm(x)\n # (batch_size, out_features) -> (batch_size, out_features)\n x = self.group_norm(x)\n # (batch_size, out_features) -> (batch_size, out_features)\n x = x * torch.sigmoid(x)\n # (batch_size, out_features) -> (batch_size, out_features)\n x = x * self.multiply_weight\n # (batch_size, out_features) -> (batch_size, out_features)\n x = x * torch.sigmoid(x)\n return x\n\nbatch_size = 1024\nin_features = 8192\nout_features = 8192\nnum_groups = 256\nmultiply_weight_shape = (out_features,)\n\ndef get_inputs():\n return [torch.rand(batch_size, in_features)]\n\ndef get_init_inputs():\n return [in_features, out_features, num_groups, multiply_weight_shape]```\n"}, "L3_008": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier network problems (wrong results or slower than eager)\n- Pitfall: For per-channel affine ops on NCHW, launch a 2D grid (one program per N*C row, tiles over HW). Precompute per-channel scale/shift once, instead of per-element div/mod, four parameter gathers and an rsqrt.\n- Pitfall: Don't replace cuDNN BatchNorm in training mode with separate torch mean/var reductions, running-stat updates and an elementwise kernel. The extra launches and memory passes outweigh the fusion; it ran far slower than eager.\n- Pitfall: Do not cache derived weights or folded parameters on the first forward call. Recompute them from the live parameters, or confirm that both modes match the reference within 1e-4.\n- Pitfall: Do not fold BatchNorm into conv weights using running_mean/var. The evaluator may run the reference in training mode, where BN normalizes with batch statistics. Check self.training, or compute batch stats yourself.\n\n### Verified recipes for network problems (speedups measured by the protected evaluator)\n- [1.00x on RegNet] Keep convolutions, pooling and Linear in cuDNN/cuBLAS via torch. Fuse BatchNorm+ReLU into one elementwise Triton kernel. Precompute per-channel scale=w/sqrt(var+eps) and shift=b-mean*scale in torch. In training mode use batch mean/var (unbiased=False) and update running stats with the unbiased correction n/(n-1). Index channels with (offs//HW)%C on contiguous NCHW, use 1D BLOCK=1024 with masking, and do fp32 math throughout. Gains are small (~1.005x). For more, fuse BN+ReLU+maxpool or the final pool with GAP.\n\n### A verified kernel for a related network problem (L3_027 RegNet, 1.00x): adapt what applies\n```python\nimport torch\nimport torch.nn as nn\nimport triton\nimport triton.language as tl\n\n@triton.jit\ndef bn_relu_kernel(x_ptr, out_ptr, scale_ptr, shift_ptr, HW, C, n_elem, BLOCK: tl.constexpr):\n pid = tl.program_id(0)\n offs = pid * BLOCK + tl.arange(0, BLOCK)\n mask = offs < n_elem\n c = (offs // HW) % C\n x = tl.load(x_ptr + offs, mask=mask, other=0.0)\n s = tl.load(scale_ptr + c, mask=mask, other=0.0)\n sh = tl.load(shift_ptr + c, mask=mask, other=0.0)\n y = x * s + sh\n y = tl.maximum(y, 0.0)\n tl.store(out_ptr + offs, y, mask=mask)\n\ndef apply_bn_relu(x, scale, shift):\n N, C, H, W = x.shape\n out = torch.empty_like(x)\n HW = H * W\n n = x.numel()\n grid = (triton.cdiv(n, 1024),)\n bn_relu_kernel[grid](x, out, scale.contiguous(), shift.contiguous(), HW, C, n, BLOCK=1024)\n return out\n\nclass ModelNew(nn.Module):\n def __init__(self, input_channels, stages, block_widths, output_classes):\n super(ModelNew, self).__init__()\n self.stages = stages\n self.block_widths = block_widths\n layers = []\n current = input_channels\n for i in range(stages):\n layers.append(nn.Sequential(\n nn.Conv2d(current, block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.Conv2d(block_widths[i], block_widths[i], kernel_size=3, padding=1),\n nn.BatchNorm2d(block_widths[i]),\n nn.ReLU(),\n nn.MaxPool2d(kernel_size=2, stride=2),\n ))\n current = block_widths[i]\n self.feature_extractor = nn.Sequential(*layers)\n self.fc = nn.Linear(block_widths[-1], output_classes)\n\n def _bn_params(self, bn, x):\n if self.training:\n mean = x.mean(dim=[0, 2, 3])\n var = x.var(dim=[0, 2, 3], unbiased=False)\n n = x.numel() // x.shape[1]\n with torch.no_grad():\n bn.running_mean.mul_(1 - bn.momentum).add_(bn.momentum * mean)\n bn.running_var.mul_(1 - bn.momentum).add_(bn.momentum * var * n / (n - 1))\n else:\n mean = bn.running_mean\n var = bn.running_var\n scale = bn.weight / torch.sqrt(var + bn.eps)\n shift = bn.bias - mean * scale\n return scale, shift\n\n def forward(self, x):\n for stage in self.feature_extractor:\n conv1 = stage[0]\n bn1 = stage[1]\n conv2 = stage[3]\n bn2 = stage[4]\n pool = stage[6]\n x = conv1(x)\n s, sh = self._bn_params(bn1, x)\n x = apply_bn_relu(x, s, sh)\n x = conv2(x)\n s, sh = self._bn_params(bn2, x)\n x = apply_bn_relu(x, s, sh)\n x = pool(x)\n x = torch.mean(x, dim=[2, 3])\n x = self.fc(x)\n return x\n```", "ctx_chars": 6513, "ctx_sha": "1aa698d3a9589f25", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 4, "recipe": 1, "kernel": 1}, "family": "network", "level": 3, "name": "ResNetBasicBlock", "split": "dev", "n_ops": 1, "src_lines": 60, "max_recipe_speedup": 1.0, "instruction": "KernelBench level 3 problem L3_008 (ResNetBasicBlock), op family: network.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass Model(nn.Module):\n expansion = 1\n\n def __init__(self, in_channels, out_channels, stride=1):\n \"\"\"\n :param in_channels: Number of input channels\n :param out_channels: Number of output channels\n :param stride: Stride for the first convolutional layer\n :param downsample: Downsample layer for the shortcut connection\n \"\"\"\n super(Model, self).__init__()\n self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False)\n self.bn1 = nn.BatchNorm2d(out_channels)\n self.relu = nn.ReLU(inplace=True)\n self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(out_channels)\n self.downsample = nn.Sequential(\n nn.Conv2d(in_channels, out_channels * self.expansion, kernel_size=1, stride=stride, bias=False),\n nn.BatchNorm2d(out_channels * self.expansion),\n )\n self.stride = stride\n\n def forward(self, x):\n \"\"\"\n :param x: Input tensor, shape (batch_size, in_channels, height, width)\n :return: Output tensor, shape (batch_size, out_channels, height, width)\n \"\"\"\n identity = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n\n if self.downsample is not None:\n identity = self.downsample(x)\n\n out += identity\n out = self.relu(out)\n\n return out\n \n# Test code\nin_channels = 3\nout_channels = 64\nstride = 1\nbatch_size = 10\nnum_classes = 1000\n\ndef get_inputs():\n return [torch.rand(batch_size, in_channels, 224, 224)]\n\ndef get_init_inputs():\n return [in_channels, out_channels, stride]```\n"}, "L3_044": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 3, "name": "MiniGPTBlock", "split": "dev", "n_ops": 1, "src_lines": 97, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_044 (MiniGPTBlock), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\n\n# From https://github.com/karpathy/minGPT/blob/master/mingpt/model.py\n\nclass NewGELU(nn.Module):\n \"\"\"\n Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT).\n Reference: Gaussian Error Linear Units (GELU) paper: https://arxiv.org/abs/1606.08415\n \"\"\"\n def __init__(self):\n super(NewGELU, self).__init__()\n \n def forward(self, x):\n return 0.5 * x * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (x + 0.044715 * torch.pow(x, 3.0))))\n\nclass CausalSelfAttention(nn.Module):\n \"\"\"\n A vanilla multi-head masked self-attention layer with a projection at the end.\n It is possible to use torch.nn.MultiheadAttention here but I am including an\n explicit implementation here to show that there is nothing too scary here.\n \"\"\"\n\n def __init__(self, n_embd, n_head, attn_pdrop, resid_pdrop, max_seqlen):\n super().__init__()\n assert n_embd % n_head == 0\n # key, query, value projections for all heads, but in a batch\n self.c_attn = nn.Linear(n_embd, 3 * n_embd)\n # output projection\n self.c_proj = nn.Linear(n_embd, n_embd)\n # regularization\n self.attn_dropout = nn.Dropout(attn_pdrop)\n self.resid_dropout = nn.Dropout(resid_pdrop)\n # causal mask to ensure that attention is only applied to the left in the input sequence\n self.register_buffer(\"bias\", torch.tril(torch.ones(max_seqlen, max_seqlen))\n .view(1, 1, max_seqlen, max_seqlen))\n self.n_head = n_head\n self.n_embd = n_embd\n\n def forward(self, x):\n B, T, C = x.size() # batch size, sequence length, embedding dimensionality (n_embd)\n\n # calculate query, key, values for all heads in batch and move head forward to be the batch dim\n q, k ,v = self.c_attn(x).split(self.n_embd, dim=2)\n k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n\n # causal self-attention; Self-attend: (B, nh, T, hs) x (B, nh, hs, T) -> (B, nh, T, T)\n att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(k.size(-1)))\n att = att.masked_fill(self.bias[:,:,:T,:T] == 0, float('-inf'))\n att = F.softmax(att, dim=-1)\n att = self.attn_dropout(att)\n y = att @ v # (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs)\n y = y.transpose(1, 2).contiguous().view(B, T, C) # re-assemble all head outputs side by side\n\n # output projection\n y = self.resid_dropout(self.c_proj(y))\n return y\n \nclass Model(nn.Module):\n \"\"\" an unassuming Transformer block \"\"\"\n\n def __init__(self, n_embd, n_head, attn_pdrop, resid_pdrop, max_seqlen):\n super().__init__()\n self.ln_1 = nn.LayerNorm(n_embd)\n self.attn = CausalSelfAttention(n_embd, n_head, attn_pdrop, resid_pdrop, max_seqlen)\n self.ln_2 = nn.LayerNorm(n_embd)\n self.mlp = nn.ModuleDict(dict(\n c_fc = nn.Linear(n_embd, 4 * n_embd),\n c_proj = nn.Linear(4 * n_embd, n_embd),\n act = NewGELU(),\n dropout = nn.Dropout(resid_pdrop),\n ))\n m = self.mlp\n self.mlpf = lambda x: m.dropout(m.c_proj(m.act(m.c_fc(x)))) # MLP forward\n\n def forward(self, x):\n x = x + self.attn(self.ln_1(x))\n x = x + self.mlpf(self.ln_2(x))\n return x\n\nbatch_size = 128\nmax_seqlen = 1024\nseq_len = 512\nn_embd = 768\nn_head = 8\nattn_pdrop = 0.0\nresid_pdrop = 0.0\n\ndef get_inputs():\n return [torch.rand(batch_size, seq_len, n_embd)]\n\ndef get_init_inputs():\n return [n_embd, n_head, attn_pdrop, resid_pdrop, max_seqlen]```\n"}, "L3_048": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 3, "name": "Mamba2ReturnY", "split": "dev", "n_ops": 1, "src_lines": 102, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_048 (Mamba2ReturnY), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom einops import rearrange\n\nclass Model(nn.Module):\n def __init__(self, batch_size, seq_length, n_heads, d_head, d_state, block_len=64):\n \"\"\"\n Mamba Structured State Space model implementation for benchmarking.\n \n :param batch_size: Size of the batch\n :param seq_length: Length of the input sequence\n :param n_heads: Number of attention heads\n :param d_head: Dimension of each head\n :param d_state: Dimension of the state space\n :param block_len: Length of each block for chunked computation\n \"\"\"\n super(Model, self).__init__()\n \n assert seq_length % block_len == 0, \"Sequence length must be divisible by block length\"\n \n self.batch_size = batch_size\n self.seq_length = seq_length\n self.n_heads = n_heads\n self.d_head = d_head\n self.d_state = d_state\n self.block_len = block_len\n \n # Initialize parameters\n self.A = nn.Parameter(torch.randn(batch_size, seq_length, n_heads))\n self.B = nn.Parameter(torch.randn(batch_size, seq_length, n_heads, d_state))\n self.C = nn.Parameter(torch.randn(batch_size, seq_length, n_heads, d_state))\n \n def segsum(self, x):\n \"\"\"Naive segment sum calculation.\"\"\"\n T = x.size(-1)\n x_cumsum = torch.cumsum(x, dim=-1)\n x_segsum = x_cumsum[..., :, None] - x_cumsum[..., None, :]\n mask = torch.tril(torch.ones(T, T, device=x.device, dtype=bool), diagonal=0)\n x_segsum = x_segsum.masked_fill(~mask, -torch.inf)\n return x_segsum\n \n def forward(self, X, initial_states=None):\n \"\"\"\n Forward pass implementing the SSD operation.\n \n :param X: Input tensor of shape (batch, length, n_heads, d_head)\n :param initial_states: Optional initial states\n :return: Output tensor Y and final state\n \"\"\"\n # Rearrange into blocks/chunks\n X_blocks, A_blocks, B_blocks, C_blocks = [\n rearrange(x, \"b (c l) ... -> b c l ...\", l=self.block_len)\n for x in (X, self.A, self.B, self.C)\n ]\n \n A_blocks = rearrange(A_blocks, \"b c l h -> b h c l\")\n A_cumsum = torch.cumsum(A_blocks, dim=-1)\n \n # 1. Compute diagonal block outputs\n L = torch.exp(self.segsum(A_blocks))\n Y_diag = torch.einsum(\"bclhn,bcshn,bhcls,bcshp->bclhp\", \n C_blocks, B_blocks, L, X_blocks)\n \n # 2. Compute intra-chunk states\n decay_states = torch.exp((A_cumsum[:, :, :, -1:] - A_cumsum))\n states = torch.einsum(\"bclhn,bhcl,bclhp->bchpn\", \n B_blocks, decay_states, X_blocks)\n \n # 3. Compute inter-chunk recurrence\n if initial_states is None:\n initial_states = torch.zeros_like(states[:, :1])\n states = torch.cat([initial_states, states], dim=1)\n \n decay_chunk = torch.exp(self.segsum(F.pad(A_cumsum[:, :, :, -1], (1, 0))))\n new_states = torch.einsum(\"bhzc,bchpn->bzhpn\", decay_chunk, states)\n states = new_states[:, :-1]\n \n # 4. Compute state-to-output conversion\n state_decay_out = torch.exp(A_cumsum)\n Y_off = torch.einsum('bclhn,bchpn,bhcl->bclhp', \n C_blocks, states, state_decay_out)\n \n # Combine diagonal and off-diagonal terms\n Y = rearrange(Y_diag + Y_off, \"b c l h p -> b (c l) h p\")\n \n \n return Y\n\n# Test parameters\nbatch_size = 2048\nseq_length = 128\nn_heads = 8\nd_head = 64\nd_state = 16\nblock_len = 64\n\ndef get_inputs():\n return [torch.rand(batch_size, seq_length, n_heads, d_head)]\n\ndef get_init_inputs():\n return [batch_size, seq_length, n_heads, d_head, d_state, block_len]\n```\n"}, "L3_050": {"ctx": "## Kernel memory (items verified by the protected evaluator on training problems; may not all apply)\n\n### Lessons about this environment (from official checker verdicts and verified outcomes)\n- Checker rule: getattr needs a literal, non-private attribute name\n- Pitfall: A reply without a ```python fenced code block was rejected by the checker. Always wrap the full ModelNew source in a ```python block.\n- Pitfall: Output mismatch beyond 1e-4 occurred repeatedly. Fix by forcing exact fp32 tl.dot(allow_tf32=False) and exact masking: K bounds at block granularity, tril mask, and zero-initialized output for skipped tiles.\n- Checker rule: forward modified its input tensors in place\n- Checker rule: do not define a class named Model (the reference); define ModelNew only\n- Process: never SUBMIT code that has not passed ACTION: CHECK in this episode. Untested submissions that fail to compile or mismatch the reference score 0; a CHECK costs one reply and shows the compiler output.\n- Pitfall: cpp_sources is compiled by plain c++ without CUDA headers: never use cudaStream_t or at::cuda::getCurrentCUDAStream there. Put wrappers needing streams in cuda_sources, keeping only plain-type prototypes in cpp_sources.\n- Pitfall: If cpp_sources must reference CUDA types or the current stream, explicitly #include and ; torch/extension.h does not provide them.\n- Checker rule: assignment to attribute 'm.conv1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn1' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.conv2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.bn2' is not allowed (only self.)\n- Checker rule: assignment to attribute 'm.downsample' is not allowed (only self.)\n- Checker rule: At least one of your own kernels must actually launch during forward; a pure torch path (even with a compiled but unused kernel) is rejected as no_custom_kernel.\n\n### Pitfalls seen on earlier attention problems (wrong results or slower than eager)\n- Pitfall: Handle every optional forward argument (e.g. initial_states) and every intermediate the reference includes, even when it defaults to zeros. Verify the returned tensor is the same slice and quantity the reference returns.\n- Pitfall: Do not algebraically restructure the reference'...'s exact operation order and test against it before optimizing.", "ctx_chars": 2425, "ctx_sha": "5c5e8811718de973", "dropped_same_problem": 0, "kinds": {"lesson": 14, "pitfall": 2}, "family": "attention", "level": 3, "name": "ReLUSelfAttention", "split": "dev", "n_ops": 1, "src_lines": 67, "max_recipe_speedup": 0.0, "instruction": "KernelBench level 3 problem L3_050 (ReLUSelfAttention), op family: attention.\nWrite ModelNew, a faster drop-in replacement for this reference Model using your own GPU kernels:\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\n\n# From https://github.com/karpathy/minGPT/blob/master/mingpt/model.py\n\nclass NewGELU(nn.Module):\n \"\"\"\n Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT).\n Reference: Gaussian Error Linear Units (GELU) paper: https://arxiv.org/abs/1606.08415\n \"\"\"\n def __init__(self):\n super(NewGELU, self).__init__()\n \n def forward(self, x):\n return 0.5 * x * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (x + 0.044715 * torch.pow(x, 3.0))))\n\nclass Model(nn.Module):\n \"\"\"\n A multi-head masked self-attention layer with a projection at the end that uses ReLU instead of Softmax.\n It is possible to use torch.nn.MultiheadAttention here but I am including an\n explicit implementation here to show that there is nothing too scary here.\n \"\"\"\n\n def __init__(self, n_embd, n_head, max_seqlen):\n super().__init__()\n assert n_embd % n_head == 0\n # key, query, value projections for all heads, but in a batch\n self.c_attn = nn.Linear(n_embd, 3 * n_embd)\n # output projection\n self.c_proj = nn.Linear(n_embd, n_embd)\n # causal mask to ensure that attention is only applied to the left in the input sequence\n self.register_buffer(\"bias\", torch.tril(torch.ones(max_seqlen, max_seqlen))\n .view(1, 1, max_seqlen, max_seqlen))\n self.n_head = n_head\n self.n_embd = n_embd\n\n def forward(self, x):\n B, T, C = x.size() # batch size, sequence length, embedding dimensionality (n_embd)\n\n # calculate query, key, values for all heads in batch and move head forward to be the batch dim\n q, k ,v = self.c_attn(x).split(self.n_embd, dim=2)\n k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n\n # causal self-attention; Self-attend: (B, nh, T, hs) x (B, nh, hs, T) -> (B, nh, T, T)\n att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(k.size(-1)))\n att = att.masked_fill(self.bias[:,:,:T,:T] == 0, float('-inf'))\n att = F.relu(att)\n\n y = att @ v # (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs)\n y = y.transpose(1, 2).contiguous().view(B, T, C) # re-assemble all head outputs side by side\n\n return y\n\nbatch_size = 16\nmax_seqlen = 1024\nn_embd = 768 # Hidden dimension, typical for BERT-base size\nn_head = 12 # Number of attention heads, typical for BERT-base size\n\ndef get_inputs():\n return [torch.rand(batch_size, max_seqlen, n_embd)]\n\ndef get_init_inputs():\n return [n_embd, n_head, max_seqlen]```\n"}}