Do AI As Engineering Instead

Dec 15, 2024

Current AI practice is not engineering, even when it aims for practical applications, because it is not based on scientific understanding. Enforcing engineering norms on the field could lead to considerably safer systems.

 

https://betterwithout.ai/AI-as-engineering

 

This episode has a lot of links! Here they are.

 

Michael Nielsen’s “The role of ‘explanation’ in AI”. https://michaelnotebook.com/ongoing/sporadica.html#role_of_explanation_in_AI

 

Subbarao Kambhampati’s “Changing the Nature of AI Research”. https://dl.acm.org/doi/pdf/10.1145/3546954

 

Chris Olah and his collaborators:

“Thread: Circuits”. distill.pub/2020/circuits/

“An Overview of Early Vision in InceptionV1”. distill.pub/2020/circuits/early-vision/

 

Dai et al., “Knowledge Neurons in Pretrained Transformers”. https://arxiv.org/pdf/2104.08696.pdf

 

Meng et al.:

“Locating and Editing Factual Associations in GPT.” rome.baulab.info

“Mass-Editing Memory in a Transformer,” https://arxiv.org/pdf/2210.07229.pdf

 

François Chollet on image generators putting the wrong number of legs on horses: twitter.com/fchollet/status/1573879858203340800

 

Neel Nanda’s “Longlist of Theories of Impact for Interpretability”, https://www.lesswrong.com/posts/uK6sQCNMw8WKzJeCQ/a-longlist-of-theories-of-impact-for-interpretability

 

Zachary C. Lipton’s “The Mythos of Model Interpretability”. https://arxiv.org/abs/1606.03490

 

Meng et al., “Locating and Editing Factual Associations in GPT”. https://arxiv.org/pdf/2202.05262.pdf

 

Belrose et al., “Eliciting Latent Predictions from Transformers with the Tuned Lens”. https://arxiv.org/abs/2303.08112

 

“Progress measures for grokking via mechanistic interpretability”. https://arxiv.org/abs/2301.05217

 

Conmy et al., “Towards Automated Circuit Discovery for Mechanistic Interpretability”. https://arxiv.org/abs/2304.14997

 

Elhage et al., “Softmax Linear Units,” transformer-circuits.pub/2022/solu/index.html

 

Filan et al., “Clusterability in Neural Networks,” https://arxiv.org/pdf/2103.03386.pdf

 

Cammarata et al., “Curve circuits,” distill.pub/2020/circuits/curve-circuits/

 

You can support the podcast and get episodes a week early, by supporting the Patreon:

https://www.patreon.com/m/fluidityaudiobooks

 

If you like the show, consider buying me a coffee: https://www.buymeacoffee.com/mattarnold

 

Original music by Kevin MacLeod.

 

This podcast is under a Creative Commons Attribution Non-Commercial International 4.0 License.