Why did the pencil think that 2+2=4?
Because it was connected to a mathematician.
– from Fred Adams and Ken Aizawa, “Defending the Bounds of Cognition”
I’m not a complete curmudgeon with respect to extended mind theory. I acknowledge that I’ve not read the primary documents other than this seminal paper by Clark and Chalmers, and I’ve not thought carefully about the implications. Tentatively, I’d say that thinking is an intentional action performed by some agent, using whatever materials and tools it has at its disposal to perform this action: pencil and paper, calculator, human expert consultant, etc. When using external tools in performing a cognitive act, the thinking agent has to wield those tools with skill and intent, which requires deploying cognitive capabilities internal to the agent. However, the external tools deployed by the agent as part of the thinking process do not thereby become part of the thinking agent.
These distinction between action and agent and tool apply to more mundane activities as well. When I cook a scrambled egg I use a bowl, a fork, a skillet, a stove, and a big spoon. These are tools I use in cooking, and I couldn’t cook a scrambled egg without them: the tools are as integral to the cooking process as is the cook. But the cooking tools aren’t part of the cook. The tools aren’t even part of the scrambled egg, even though there would be no scrambled egg without them.