A Former Apple Luminary Sets Out to Create the Ultimate GPU Software

Demand for AI chips is booming—and so is the need for software to run them. Chris Lattner’s startup Modular just raised $250 ...
GitHub

torch.nn.functional.normalize outputs vectors with norm>1 with torch.compile + cuda

torch.nn.functional.normalize on fp32 inputs outputs vectors with norm>1 (norm=1.0000001192092896) when using cuda + torch.compile. This does not happen when using only cuda, or using cpu+no compile, ...
marktechpost

Software Frameworks Optimized for GPUs in AI: CUDA, ROCm, Triton, TensorRT—Compiler Paths and Performance Implications

Deep-learning throughput hinges on how effectively a compiler stack maps tensor programs to GPU execution: thread/block schedules, memory movement, and instruction selection (e.g., Tensor Core MMA ...
GitHub

[Bug]: Launching multiple vLLM processes at the same time doesn't work well with vLLM's compile cache

clean: rm -rf /tmp/torchinductor_rzou rm -rf ~/.cache/vllm/torch_compile_cache killall -9 "VLLM::EngineCore" run: VLLM_ENABLE_V1_MULTIPROCESSING=0 CUDA_VISIBLE ...