概要
本文介绍了编译缓存工具 sccache 的安装和配置过程。
sccache 的效果大致如下：
| 项目 | 语言 | 无缓存 | 使用缓存 |
|---|---|---|---|
| Redis | C | 48.263s | 30.012s |
| ripgrep | Rust | 17.99s | 5.95s |
背景
与脚本型语言不同，编译型语言需要经过编译之后，才能运行起来。而对于一些超大型项目来说，编译一次代码可能会消耗很长的时间。除了组件化拆分等方法，比较立竿见影的是使用编译缓存工具来缓存部分的编译中间结果，这样在下一次编译时就可以大大加快流程速度。
久经考验的编译缓存工具有 ccache 等。在这篇文档中，我们主要认识一下 sccache —— 一款由 mozilla 开发的 ccache-like 工具，除了 C/C++之外，还为 Rust 等其他技术提供了编译缓存支持。
安装
参照 官方文档，安装过程非常简单：
cargo install sccache --locked
服务器启停
sccache 使用 client-server 模式进行工作。其服务器默认监听 127.0.0.1:4226。
要启动服务器，可以使用命令：sccache --start-server。相应的，使用 sccache --stop-server 可以停止服务器。
注意，如果在 WSL 下遇到无法启动服务器的问题，可以参照 sccache 无法启动问题 进行解决。
缓存效果
C/C++
我们以 Redis 项目为例，检查 sccache 的提升效果。
无缓存编译
首先，安装编译 redis 需要的依赖并准备编译环境：
sudo apt install build-essential tcl libjemalloc-dev libssl-dev lua5.3 liblua5.3-dev
git clone https://github.com/redis/redis.git --depth=1 # commit dee0d11a74dbb8efcec8148bd2ff43d203b48931
cd redis/
make clean && make distclean
然后，编译 redis 查看
$ time make -j16 &> /dev/null
real 0m48.263s
user 5m23.822s
sys 0m34.994s
可以看到，在无缓存的场景下，编译 redis 所用的时间是：48.263s
缓存编译
接下来，我们使用验证缓存效果。由于缓存需要先编译一次后才能生成，所以接下来我们将一共编译两次。
首先查看 sccache 统计，可以看到是没有缓存的：
$ sccache -s
Compile requests 0
Compile requests executed 0
Cache hits 0
Cache misses 0
Cache hits rate -
Cache timeouts 0
Cache read errors 0
Forced recaches 0
Cache write errors 0
Cache errors 0
Compilations 0
Compilation failures 0
Non-cacheable compilations 0
Non-cacheable calls 0
Non-compilation calls 0
Unsupported compiler calls 0
Average cache write 0.000 s
Average compiler 0.000 s
Average cache read hit 0.000 s
Failed distributed compilations 0
Cache location Local disk: "/home/focksor/.cache/sccache"
Use direct/preprocessor mode? yes
Version (client) 0.10.0
Max cache size 10 GiB
然后，设置使用 sccache 并进行一次编译：
$ export CC="sccache gcc"
$ export CXX="sccache g++"
$ (make clean && make distclean) &> /dev/null
$ time make -j16 &> /dev/null
real 0m58.897s
user 2m53.564s
sys 0m20.504s
现在可以看到 sccache 已经生成了一些缓存：
$ sccache -s
Compile requests 584
Compile requests executed 355
Cache hits 7
Cache hits (C/C++) 7
Cache misses 334
Cache misses (C/C++) 334
Cache hits rate 2.05 %
Cache hits rate (C/C++) 2.05 %
Cache timeouts 0
Cache read errors 0
Forced recaches 0
Cache write errors 0
Cache errors 9
Cache errors (C/C++) 9
Compilations 334
Compilation failures 5
Non-cacheable compilations 0
Non-cacheable calls 133
Non-compilation calls 96
Unsupported compiler calls 0
Average cache write 0.002 s
Average compiler 0.593 s
Average cache read hit 0.000 s
Failed distributed compilations 0
Non-cacheable reasons:
-MM 126
-E 6
argument parse 1
Cache location Local disk: "/home/focksor/.cache/sccache"
Use direct/preprocessor mode? yes
Version (client) 0.10.0
Cache size 346 MiB
Max cache size 10 GiB
缓存已经生成，我们再进行一次编译以验证编译缓存效果：
$ (make clean && make distclean) &> /dev/null
$ time make -j16 &> /dev/null
real 0m30.012s
user 2m51.748s
sys 0m21.422s
可以看到，生成缓存后，再次编译时速度有了非常大的提升！查看 sccache 的统计信息，可以看到编译过程响应的缓存命中数据也有上升：
$ sccache -s
Compile requests 1159
Compile requests executed 707
Cache hits 345
Cache hits (C/C++) 345
Cache misses 334
Cache misses (C/C++) 334
Cache hits rate 50.81 %
Cache hits rate (C/C++) 50.81 %
Cache timeouts 0
Cache read errors 0
Forced recaches 0
Cache write errors 0
Cache errors 18
Cache errors (C/C++) 18
Compilations 334
Compilation failures 10
Non-cacheable compilations 0
Non-cacheable calls 265
Non-compilation calls 187
Unsupported compiler calls 0
Average cache write 0.002 s
Average compiler 0.593 s
Average cache read hit 0.002 s
Failed distributed compilations 0
Non-cacheable reasons:
-MM 251
-E 12
argument parse 2
Cache location Local disk: "/home/focksor/.cache/sccache"
Use direct/preprocessor mode? yes
Version (client) 0.10.0
Cache size 346 MiB
Max cache size 10 GiB
Rust
在 Rust 项目上，我们使用 ripgrep 验证缓存效果。
首先准备项目：
git clone https://github.com/BurntSushi/ripgrep.git --depth=1 # commit id: 119a58a400ea948c2d2b0cd4ec58361e74478641
cd ripgrep/
cargo clean
无缓存编译
$ cargo build --release 2>&1 | tail -1
Finished `release` profile [optimized + debuginfo] target(s) in 17.99s
缓存编译
首先设置使用 sccache 进行缓存编译，并构建一次编译以创建缓存：
export RUSTC_WRAPPER=/home/focksor/.cargo/bin/sccache
$ cargo clean
Removed 313 files, 201.7MiB total
$ cargo build --release 2>&1 | tail -1
Finished `release` profile [optimized + debuginfo] target(s) in 23.68s
$ sccache -s
Compile requests 47
Compile requests executed 33
Cache hits 0
Cache misses 32
Cache misses (Rust) 32
Cache hits rate 0.00 %
Cache hits rate (Rust) 0.00 %
Cache timeouts 0
Cache read errors 0
Forced recaches 0
Cache write errors 0
Cache errors 0
Compilations 32
Compilation failures 1
Non-cacheable compilations 0
Non-cacheable calls 13
Non-compilation calls 1
Unsupported compiler calls 0
Average cache write 0.002 s
Average compiler 1.341 s
Average cache read hit 0.000 s
Failed distributed compilations 0
Non-cacheable reasons:
crate-type 7
missing input 4
- 2
Cache location Local disk: "/home/focksor/.cache/sccache"
Use direct/preprocessor mode? yes
Version (client) 0.10.0
Cache size 31 MiB
Max cache size 10 GiB
然后，使用缓存重新构建一次编译以查看加速效果：
$ cargo clean
Removed 313 files, 201.7MiB total
$ cargo build --release 2>&1 | tail -1
Finished `release` profile [optimized + debuginfo] target(s) in 5.95s
$ sccache -s
Compile requests 91
Compile requests executed 66
Cache hits 32
Cache hits (Rust) 32
Cache misses 32
Cache misses (Rust) 32
Cache hits rate 50.00 %
Cache hits rate (Rust) 50.00 %
Cache timeouts 0
Cache read errors 0
Forced recaches 0
Cache write errors 0
Cache errors 0
Compilations 32
Compilation failures 2
Non-cacheable compilations 0
Non-cacheable calls 23
Non-compilation calls 2
Unsupported compiler calls 0
Average cache write 0.002 s
Average compiler 1.341 s
Average cache read hit 0.000 s
Failed distributed compilations 0
Non-cacheable reasons:
crate-type 14
missing input 6
- 3
Cache location Local disk: "/home/focksor/.cache/sccache"
Use direct/preprocessor mode? yes
Version (client) 0.10.0
Cache size 31 MiB
Max cache size 10 GiB
资料
- Ccache — Compiler cache
- mozilla/sccache: Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
- redis/redis: For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
- BurntSushi/ripgrep: ripgrep recursively searches directories for a regex pattern while respecting your gitignore