Machine learning models, while very powerful, have their operation obfuscated behind millions of parameters. This obfuscation can make deriving a human meaningful process from a machine learning model very difficult. However, while the intermediate states of a machine learning model are similarly obfuscated, using probing, we can start to explore looking at possible structure in those intermediate states. Large language models are a prime example of this obfuscation, and probing can begin to allow novel experimentation to be performed.
Plasek, Ashlen A.
"Probing as a Technique to Understand Abstract Spaces,"
Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal: Vol. 10:
2, Article 5.
Available at: https://digitalcommons.morris.umn.edu/horizons/vol10/iss2/5