RESEARCH · RESEARCH · #3474
Dataiku demos Kiji Inspector: sparse-autoencoder explainability for agent tool selection
At QCon AI, Hannes Hapke (575 Lab, Dataiku) presented Kiji Inspector, an open-source tool that applies sparse autoencoders to decision tokens to surface why LLM-based agents choose particular tools. The demo covered causal ablation, layer activation extraction using patched vLLM engines, and production monitoring workflows for engineering teams.
KEY POINTS
- At QCon AI, Hannes Hapke (575 Lab, Dataiku) presented Kiji Inspector, an open-source tool that applies sparse autoencoders to decision tokens to surface why LLM-based agents choose particular tools.
- The demo covered causal ablation, layer activation extraction using patched vLLM engines, and production monitoring workflows for engineering teams.
- Understanding causal signals inside agent decision-making (beyond chain-of-thought) helps engineers validate tool selection and build production monitoring and safety workflows.
WHY IT MATTERS
Understanding causal signals inside agent decision-making (beyond chain-of-thought) helps engineers validate tool selection and build production monitoring and safety workflows.