The Moral Status of Turing AIs

On 2 September, 2010, in Info, by Adam A. Ford

Andrew Dun, August 2010


Many discussions of strong artificial intelligence systems (also known as AGIs or Artificial General Intelligences) convey the assumption that such AIs will be used as tools for human purposes (consider Asimov’s Three Laws of Robotics). Certainly, many discussions in this vicinity focus on the development Friendly AI (for example, Yudkowsky 2007), thereby concerning what AIs may do to aid or hinder human beings, rather than vice versa.

Notwithstanding the varying accounts of this subject found in popular cinematic treatments such as 2001 A Space Odyssey, Blade Runner, The Terminator series or (shudder) Electric Dreams, the view is less often put that any genuine strong AI, at least of the kind capable of understanding human goals and directives (the kind of AI we might expect to be able to pass a Turing test) will by necessity be a moral agent in its own right[1]; this is to say that without having the qualities that we would ascribe to a human agent, the system will not have the qualities that it would require in order to be able to pass a Turing test.

In the following, I provide a brief outline of such an argument. While this argument is a work in progress and requires a far more detailed defence than can be provided in the present space, my intention here is simply to describe a framework for discussion of the likely moral status of AIs.


Let us define the following:


Turing AI = any AI capable of passing the Turing test (which we might assume will be an Artificial General Intelligence (AGI), at least to the same degree to which human intelligence is general).


My argument: Any artificial intelligence capable of passing the Turing test is either human or superhuman (and hence has the same moral status).


I will argue for this position as follows:

Part 1

i)                  that intentionality is a product of relations between a cognitive system and its environment

ii)                that intentional concepts are holistic rather than discrete – they become richer and more productive the more deeply they are embedded in background networks of relationships to other concepts

iii)              given(ii), for a system to pass the Turing test, it must manifest the range of intentional states captured by human language to the same or greater level of richness as would be the case for a typical human subject

iv)              that to be human is simply to be a subject that manifests a set of intentional states similar to those that would be associated with a typical human subject

v)                to be superhuman is to manifest those intentional states mentioned in (iv), as well as to manifest a broader set of intentional states, or the same set of intentional states to a greater degree of richness.

vi)              Conclusion 1 (given iii, iv and v, by modus ponens): any system capable of passing the Turing test is either human or superhuman.

Part 2

vii)            any human subject should be accorded the same moral status accorded to other human subjects

viii)          any superhuman subject should be accorded the same or greater moral status as that accorded to human subjects

ix)              Conclusion 2 (given vi, vii, viii, by modus ponens): any system capable of passing the Turing test should be accorded the same or greater moral status as that accorded to human subjects.

Taking each of these premises in turn:

Part 1

i) that intentionality is a product of relations between a cognitive system and its environment


The (psychological) notion of intentionality dates back to Brentano 1874.

Intentionality is typically described in terms of ‘directedness’ or ‘aboutness’, or, as per the Oxford English Dictionary,  ”the distinguishing property of mental phenomena of being necessarily directed upon an object, whether real or imaginary”. For the present purposes, I will characterise as the mind’s ability to be directed toward phenomena; in short, to grasp semantics (meaning) rather than simply syntax. Intentionality is described by Brentano as being the ‘mark of the mental’; distinguishing mental entities which can have aboutness (for example, the thought that stops signs are red), with physical entities that have no aboutness of their own (for example, a red stop sign)[2].

A range of theorists have historically challenged the reducibility of both intentionality (for example, Brentano 1874, Chisholm 1957, Searle 1980, Chalmers 2003) and consciousness (for example, Nagel 1974,Jackson 1982, Chalmers 1996). For the purposes of this discussion I will assume that whether or not consciousness can be reduced, it is a substrate independent phenomenon (and therefore will occur in any system with appropriate functional structure, as proposed by functionalists such as Chalmers 1996). I will also take the view that intentionality is reducible. We may view this reducibility as similar to that of an interpersonal human relationship – such a relationship may be a complex construction but nonetheless exists as a general pattern in a set of social practices; I will take it that we can similarly view an intentional relationship existing as a certain kind of pattern between a cognitive system and its environment[3]. Complex then, but not metaphysically mysterious, pace Brentano et al.


ii) that intentional concepts are holistic rather than discrete – they become richer and more productive the more deeply they are embedded in background networks of relationships to other concepts[4].


This is not necessarily true of all concepts, and there are clear arguments as to why some kinds of concepts might be considered discretely (classically) structured while others may even be atomic[5]. I therefore here assume a kind of pluralism about conceptual structure. The core view that I take here of concepts is however a holistic one, as proposed by the theory theory of concepts (see Margolis and Laurence 1999).

Even on a classical view of intentionality, where many concepts are held to determine discrete extensions, it can be noted that within natural language concepts play a variety of roles in different contexts. If we allow that the sophistication of a subject’s conceptual awareness will be at least analogous to its linguistic sophistication, it seems relatively uncontroversial to acknowledge that as a subject grows and develops, its acquisition of some concepts will broadly enrich its awareness of others. For example, acquisition of the concept ‘hand break’ will enrich acquisition of the concept ‘car’. Acquisition of the concepts of ‘trust’ and ‘friendship’ will be mutually enriching, and so on. We might therefore note that the sophistication of a subject’s grasp of a particular concept will depend upon the way it relates that concept to a complex intentional background[6].


iii) that for a system to pass the Turing test, it must manifest the range of intentional states captured by human language to the same or greater level of richness as would be the case for a typical human subject


For a machine to pass the Turing test (if the Turing test succeeds as intended), I contend that it must be able to grasp a full range of linguistic concepts with the subtlety and sophistication of a human subject; if this is not the case, and, for example, the test can be passed by virtue of relatively simple syntactic algorithms (hacks), then the Turing test fails as the substantive test of mentality that it is intended to be. Assuming the Turing test is not however subject to failure in this manner, and given the above, we can deduce that the tested machine must be in possession of an intentional background with at least the same breadth and depth as that held by a human subject.


iv) that to be human is simply to be a subject that manifests a set of intentional states similar to those that would be associated with a typical human subject


This might seem contentious at first, as we might imagine that a certain kind of database (perhaps even a flat file database, like a computer spreadsheet), would be sufficient to capture an appropriate set of intentional states; and we would likely be hesitant to call ‘Human.xls’, human.

Firstly, consider that by ‘capture’ and ‘manifest’ I intend here two quite separate notions; to capture being to take a snapshot, a slice in time, while to manifest being to deploy in an active manner, as would accord with our sense of what constitutes a conscious subject. Indeed, a big enough Microsoft Excel spreadsheet could at least in theory capture all the information content of a human brain, even if just by recording the position of every molecule in that brain, but would capture only a moment in time, rather than the manifest dynamics of the system over time[7].

Secondly, consider that to manifest the human intentional background means to actively deploy a number of concepts in a very human way; to suffer, to experience joy, to experience desire, responsibility, accountability; to plan, project; to have a sense of self, to be a subject. These are some of the deepest and richest of human concepts, and they will need to be understood and manifest by any Turing capable[8] AI that can play a convincingly human role. The familiar worry about a qualitative (experiential) component of these concepts might be raised, but on a substrate independent view of consciousness we might assume that manifestation of a human intentional background will be sufficient for implementation of the kinds of functional states which we would expect to accord with the instantiation of qualitativeconcious states (whether viewed reductively or otherwise).

A system therefore that fully grasps pain in the human sense, suffers. So too does it wonder, feel joy, feel love, feel embarrassed, feel connected to others, and feel a sense of identity related to its beliefs, history and place in the world. Further, I would take the view that a large degree our moral sense of these features relies on their functional or socially performative aspects, rather than any thin phenomenal residue that might accompany their internal experience. I would contend, in the absence of a suggestion for any better overriding set of features, that any system that has this kind of subjective specificity (ontological qualitites) and is capable of fulfilling these social and experiential roles (relational qualities) captures the core of what it means to be human, qua inner life and ability to fulfil the role of moral agent; call this the ontological-relational view of social ethics[9].

Human intentionality derives from a point of subjective specificity, and so for emulation of human intentionality, it therefore seems likely that an AI will need to manifest the same subjective specificity; its intentional states will need to relate to a specific self, a specific set of relationships, a personal narrative.

It might be objected that it would be sufficient to merely mirror or simulate these qualities; that a machine might be able to be empathetic, and thus to act as a kind of human imposter, without instantiating the appropriate intentional states in a genuine way. There is also a broader question here about the degrees of similarity and difference that intentional states must possess to be classified under a particular heading; consider the various differences in intentional backgrounds across human cultures. Nonetheless, it seems likely that any sufficiently compelling Turing AI will need to manifest at least a broad part of its intentional background in a way that would match that of a human subject; i.e. by deploying concepts via the manifestation of intentional states that pick out human-like relationships between the machine and its environment.

Certainly, a human might ‘simulate’ some of these states (as an actor does); but there is no reason to think that even such simulation would not to some degree instantiate a human subject ‘within’ the simulation (note by analogy the way that method actors can become deeply entwined with their characters); or that were some core part of the human intentional background not present, or not manifest with a sufficient level of authenticity, that this would compromise the Turing capability of the AI[10].

v) to be superhuman is to manifest those intentional states mentioned in (iv), as well as to manifest a broader set of intentional states, or the same set of intentional states to a greater degree of richness.


This would seem non-contentious. It would seem reasonable to define a super-human, as opposed to non-human, entity as one that manifests the set of characteristics required for humanness as a subset of an otherwise broader set of characteristics; perhaps for a example a broader than usual intentional background and cognitive capacity.


vi) Conclusion 1 (given iii, iv and v, by modus ponens): any system capable of passing the Turing test is either human or superhuman.


To pass the test to an appropriate level of finesse, the system will need to manifest the full intentionality of a human subject; whether by simulation, for a long period, or for short; this then would equate to the manifestation of a human subject (with all the moral status thereby entailed).

Part 2

Further I would contend that

vii) any human subject should be accorded the same moral status accorded to other human subjects

This would seem a reasonable statement of egalitarianism; exceptions might be made with respect to newborn babies or those with severe mental handicaps, but it would seem that these exceptions relate to cognitive underdevelopment or deficiency in a way not applicable to Turing AIs.

vii) any superhuman subject should be accorded the same (or greater) moral status as that accorded to human subjects

I regard this as essential non-contentious (at least with respect to possession of the same status; greater moral status requires further argument), though it is conceivable that one might attempt to mount an argument against this view on the basis of some accounts of human rights (perhaps the argument might be made that a superhuman should have fewer rights than a human if the superhuman does not fit well to a particular social norm).

Therefore, when combined with Conclusion 1, we derive

ix) Conclusion 2 (given vi, vii, viii, by modus ponens): any system capable of passing the Turing test should be accorded the same or greater moral status as that accorded to human subjects.


Counterargument: any system not biologically embedded in the environment cannot genuinely manifest of all of the environmental relations attributed to a human subject (note Searle’s biological materialism (Searle 1992) or Hubert Dreyfus (Dreyfus 1992) on the role of embodiment); at best if might be able to simulate such relations, by imagining (or simulating) itself as being biologically embedded; but simulatedembeddedness is not the same as genuine embeddedness.


Rebuttal 1: a sufficiently rich simulation of embeddedness would be sufficient.

We might note here that if substrate independence can be taken to apply to phenomenal consciousness, it should certainly apply to intentionality. On the basis of this, intentional states obtaining in silicon might be considered just as authentic as (and no more ‘simulated’ than) those obtaining in carbon.

We might nonetheless allow that on some views a rich simulation would nonetheless be of diminished status with respect to particular environments. Consider that a ‘stranger in a town’ may not be regarded as having the same moral ‘due’ in that community as someone who has lived there all their life; the same might argued with respect to the gulf between genuine biological embeddedness and embeddedness in, for example, a virtual world, or via a prosthetic (robotic) body. Nonetheless, this would not seem sufficient to justify a contention that such local limitations would limit the stranger’s broader human moral status (except perhaps to the most parochial of commentators); and thereby issues of embodiment or embeddedness would not seem to render any particular qualification against a simulated subjects’ broader status as cognitively and morally human.


Rebuttal 2: biological embeddedness captures some important though mundane aspects of human intentionality; though not its most important characteristics, which relate to the forms intentionality associated with human agency, emotion, selfhood and responsibility.

Such a view would be as described by the relational aspect of the ontological-relational view of social ethics outlined above.


To reiterate, what I have presented here remains simply an outline of an argument that is a work in progress. Nonetheless, I believe the points presented provide at least preliminary support for an equality of the form human linguistic comprehension = human understanding = human cognition = human agency = human moral status.


References

Blair, R. J. R., D. G. V. Mitchell, et al. (2002). “Turning a deaf ear to fear: Impaired recognition of vocal affect in psychopathic individuals.” Journal of Abnormal Psychology 111(4): 682-686. http://eprints.ucl.ac.uk/3931/

Brentano, F. (1874). Psychology from an Empirical Standpoint, trans. by A. Rancurello, A. Terrell, and L. McAlister, Routledge and Kegan Paul, London, http://bit.ly/d0tc0m.

Chalmers, D. (2003). “The Content and Epistemology of Phenomenal Belief.” Consciousness: New Philosophical Perspectives: 220-72. http://consc.net/papers/belief.html

Chalmers, D. J. (1996). The Conscious Mind: In Search of a Fundamental Theory, Oxford University Press. http://bit.ly/an344G

Chisholm, R. M. (1957). Perceiving: A philosophical study, Cornell University Press Ithaca, NY. http://philpapers.org/rec/CHIPAP

Coeckelbergh, M. (2010). “Robot rights? Towards a social-relational justification of moral consideration.” Ethics and Information Technology: 1-13.http://www.springerlink.com/content/x27162623582640q/

Dennett, D. C. (1987). The Intentional Stance, MIT Press. http://bit.ly/cUXBiJ

Dreyfus, H. L. (1992). What computers still can’t do: a critique of artificial reason, The MIT Press. http://bit.ly/9sljTU

Goertzel, B. (2009). “OpenCogPrime: A cognitive synergy based architecture for artificial general intelligence.” http://goertzel.org/dynapsyc/2009/OpenCogPrime.pdf

Horgan, T. and J. Tienson (2002). “The intentionality of phenomenology and the phenomenology of intentionality.” Philosophy of mind: Classical and contemporary readings: 520-533.http://www.u.arizona.edu/~thorgan/papers/mind/IPandPI.htm

Jackson, F. (1982). “Epiphenomenal Qualia [J].” Philosophical Quarterly 32: 127-136. http://philosophy.ucsd.edu/faculty/rarneson/Courses/FrankJacksonphil1.pdf

Kosson, D. S., Y. Suchy, et al. (2002). “Facial Affect Recognition in Criminal Psychopaths* 1,* 2.” Emotion 2(4): 398-411. http://www.themindinstitute.org/pubs/Kosson_2002.pdf

Margolis, E. and S. Laurence (1999). Concepts: Core Readings, MIT Press. http://cognitrn.psych.indiana.edu/rgoldsto/courses/concepts/MargolisConcepts.pdf

Nagel, T. (1974). “What Is It Like to Be a Bat?” Philosophical Review 83: 435-450. http://www.artboy.info/teach/reference/bat/ThomasNagel.pdf

Reed, S. L. and D. B. Lenat (2002). Mapping ontologies into Cychttp://www.cyc.com/doc/white_papers/mapping-ontologies-into-cyc_v31.pdf.

Searle, J. R. (1980). “Minds, brains and programs.” Behavioral and Brain Sciences 3: 417-57. http://portal.acm.org/citation.cfm?id=42463

Searle, J. R. (1983). Intentionality, Cambridge University Press. http://bit.ly/ca4GCH

Searle, J. R. (1992). The Rediscovery of the Mind, MIT Press. http://bit.ly/ab9h9f

Yudkowsky, E. (2007). “Artificial intelligence as a positive and negative factor in global risk.” Global Catastrophic Riskshttp://singinst.org/AIRisk.pdf




[1] Although note Coeckelbergh 2010, who develops such an account of the basis of an externalistic system of virtue ethics grounded in social relations.

[2] I take the view that broad (semantic) intentionality is underpinned by a more fundamental narrow (phenomenal) intentionality (a term coined in Horgan and Tienson 2002) – but nothing in the current discussion turns on this.

[3] The position I take is therefore not dissimilar to Daniel Dennett’s Intentional Stance, Dennett 1987. While I have expressed the view that phenomenal intentionality plays a role in the determination of wide intentionality, I nonetheless take the view that phenomenal intentionality can be understood reductively qua this role.

[4] One might consider this notion as being similar to Searle’s description of The Background (Searle 1983).

[5] Such a view is often ascribed to phenomenal concepts; my view (defended elsewhere) is that even phenomenal concepts contain subtle structure that is critical to their capacity to provide a narrow intentional framework for general intentionality.

[6] It is not uncommon for AI systems currently underdevelopment to implement analogous conceptual frameworks, if it at a simplified level, (and without an underlying narrow intentional structure). Consider for example the graphs and hypergraphs implemented by Cyc (Reed and Lenat 2002) and OpenCog (Goertzel 2009)).

[7] We nonetheless could of course have n such spreadsheets recording sequential over an arbitary duration; we may perhaps even be able to create a simulation (which we might more reasonably be prepared to regard as human) of a mind derived from a single snapshot.

[8] Not to be confused with Turing complete. We might reasonably expect any Turing capable AI to be Turing complete, at least to the degree to which humans can themselves manually perform the operations of Turing machine.

[9] A defence of a purely relational view of the ethical status of AIs can be found in Coeckelbergh 2010.

[10] We might consider parallel perceptual or emotional deficits in humans, such as those found in psychopaths (Kosson, Suchy et al. 2002, Blair, Mitchell et al. 2002); however it is unclear the degree to which these would best be characterised as deficits in intentional background or merely neurophysiological deficits). Further, we might presume any Turing tester worth her salt will be able to detect even a faint whiff of non-humanness in a machine’s responses – at least up to the point to which she may do the same for a psychopath (who would rightly be considered a human and therefore Turing-capable subject).


 

1 Response » to “The Moral Status of Turing AIs”

  1. Pius Agius says:

    Hello,

    I just read your article and see that the references include a few of the people that are very, very critical of the concept of machine intelligence. As a robot builder I strive for the day that we can build machines that are at the level you describe in your article. I agree with your conclusion that if they attain such a high level of sophistication they must be seen as an intelligence.

    My robot mentor is David L. Heiserman and before he got to the part where he describes, in delicious detail how to build a robot he would discuss in great length his ideas on machine intelligence. I quote him when I say that machines will one day have an intelligence, it may be different but you could not call it something else. I also prefer his term machine intelligence over the term artificial intelligence,since it would actually exist.

    Take Care,

    Pius Agius

Leave a Reply

You must be logged in to post a comment.