Software Engineer — RISC-V SIMD Kernels for LLMs

About the Project

The team builds low-level compute kernels for inference/training of well-known LLM models, hand-optimized for a proprietary (custom) chip based on the RISC-V architecture with RISC-V Vector Extension (SIMD intrinsic extension) and a custom VCIX Coprocessors. In terms of functional scope, this is analogous to what PyTorch/Tensorflow provides (matmul/GEMM, convolution, elementwise ops, reduction, attention/softmax, quantization) — but written from scratch for a specific chip rather than routed through an existing backend.

What You Will Actually Do

• Write new compute kernels (functionally equivalent to PyTorch operators: matmul/GEMM, convolution, elementwise, reduction, attention/softmax) targeting custom RISC-V hardware.
• Implement and tune mixed-precision inference pipelines across multiple numerical formats (FP32, BF16, MXFP8, MXFP4, NVFP4).
• Investigate and improve the performance of existing kernels and how they are invoked: optimize cache vs. DRAM memory usage (memory hierarchy, tiling/blocking, prefetching, bandwidth vs. latency trade-offs).
• Optimize distributed communication for multi-device and multi-node inference.
• Partition and schedule workloads across compute cores to maximize hardware utilization.
• Fuse individual kernels into efficient end-to-end pipelines for LLM model execution.
• Write C++23 code with heavy use of metaprogramming (templates, constexpr, concepts, CRTP, SFINAE) to generate specialized code for different kernel configurations.

Required Qualifications

• Hands-on experience writing low-level compute kernels — GEMM/matmul, convolution, elementwise, reduction, or similar — regardless of whether this was for PyTorch/ATen, XLA, CUDA, ARM NEON/SVE, x86 AVX, or other hardware. The key requirement is actual kernel-writing experience, not just usage of high-level ML frameworks.
• Experience with SIMD/SIMT/vector intrinsics on any architecture (RVV, ARM NEON/SVE, x86 SSE/AVX/AVX-512) or with inline assembly, understanding how the compiler maps intrinsics to instructions etc.
• Solid understanding of memory hierarchy: cache vs. DRAM, cache lines, latency/bandwidth trade-offs, data locality, tiling/blocking for compute optimization.
• Strong interest in and willingness to optimize code for performance and memory efficiency, with the ability to identify bottlenecks and continuously improve execution speed, memory usage, and overall system efficiency.
• Strong focus on latency minimization in production inference paths — able to identify and eliminate latency-critical bottlenecks across the full execution pipeline, not just individual kernels.
• Comfortable working in a Linux-like environment for building/testing (WSL/Linux), and experience debugging without full GUI debugger tooling (log-based debugging).

Strong Pluses

• Direct RISC-V experience (ISA, RVV / custom intrinsic extensions, cross-compilation with a RISC-V toolchain — GCC/LLVM/Clang).
• Experience optimizing kernels for LLM inference/training (attention, KV-cache, softmax, layer norm, quantized matrix multiplication).
• Experience with proprietary/custom hardware (not reference architectures), including writing code for early-stage chips where tooling and debugging capabilities are limited.
• Understanding of collective communication operations, distributed training (Data/Tensor/Pipeline Parallelism).
• Understanding of floating-point formats — FP32, BF16, MXFP8, NVFP4/MXFP4 — including their trade-offs in precision, dynamic range, and performance.
• Understanding of how numerical format choice affects throughput, memory bandwidth, and training/inference numerical stability.
• Strong C++ (17/20/23) with confident use of template programming — real experience in a generic/template-heavy codebase, not just familiarity with the syntax.