https://blog.lukesalamone.com/tags/quantization/ Recent content in Quantization on Luke Salamone's Blog Hugo en-us Fri, 19 Jun 2026 22:24:12 -0700 https://blog.lukesalamone.com/posts/creating-tiny-semantic-search/ Fri, 19 Jun 2026 22:24:12 -0700 https://blog.lukesalamone.com/posts/creating-tiny-semantic-search/ <p>This project is a continuation of my <a href="../../posts/autoresearch/">previous autoresearch project</a>, which optimized a reranking model to be under 10MB. Digging deeper by hand, I was able to take the size reduction much further, while outperforming reranking models which are 30x larger on this task. In the end I was able to reduce the payload from 11.4 MB to 2.79 MB gzipped.</p> <p>You can see it in action on my <a href="https://lukesalamone.com">resume page</a>.</p> <figure><img src="../../img/tiny_reranker.png" alt="Each square represents 1 kB. The majority of overall size reduction came from removing the ORT dependency. However, other changes enabled much better representation quality than the baseline."><figcaption> <p>Each square represents 1 kB. The majority of overall size reduction came from removing the ORT dependency. However, other changes enabled much better representation quality than the baseline.</p>