CS kolokwium

The Semiotic Hierarchy levels (Zlatev)

Each level of the hierarchy is defined by three components: the type of subject (S), the world (W) they inhabit, and their internal value system (V).

Meaning is defined as the relationship between a subject and its environment, determined by value.

M(p,S) = W(p) × V(p,S)

• M = meaning of phenomenon p for subject S,

• W = the world in which the phenomenon appears,

• V = the value assigned to the phenomenon by the subject's value system.

A phenomenon is meaningless if it lies outside the subject's world or if its value is zero.

1. Level 1 - LIfe: Meaning is co-extensional with life, characterized by self-organization and autopoiesis. The organism interacts with its Umwelt (surrounding world) based on biological values like survival, though it is not yet a conscious experiencing subject. Only living organisms can reach this level because they posses self-organization, autopoesis and internal value of their own interests. Artificial machines are excluded because they lack instinct interests.

2. Level 2 - Consciousness: Minimal Self with a first-person perspective and phenomenal consciousness. The world shifts to a Natural Lebenswelt (lived world), where sensory data is bound into coherent Gestalts (objects and scenes). Consciousness integrates fragmented sensory inputs into stable multimodal Gestalts. This transforms a biologically meaningful Umwelt into a phenomenally experienced Natural Lebenswelt. Consciousness functions as a common currency for perception, action, and evaluation. It allows organisms to allocate attention flexibly and evaluate situations through feelings such as pleasure and pain.

3. Level 3: Sign Function: The subject becomes an Enculturated Self through social interaction. Meaning is mediated by signs (representation), which require a double differentiation between expression and content. Signification value characterizes this level: phenomena become meaningful because they stand for or represent something else. This level includes enculturated non-human primates (e.g., great apes), who have learned to use signs in human environments.

4. Level 4: Language: Linguistic Self capable of using conventional and structured systems. The world is a Universe of Discourse characterized by shared narratives, abstract ideas, and social norms. Subject develops a narrative identity. Linguistic self is capable of constructing a biographical understanding of itself, reflecting on the past, and planning for the future.

The Semiotic Hierarchy: Relation between levels (Zlatev)

1. Logical Implication (Downward): higher levels presuppose the existence of the lower ones.

   • Implication Chain: Life < Consciousness < Sign Function < Language.

2. Ontological Emergence (Upward): Higher levels naturally grow out of lower foundations over evolutionary time, introducing ontologically new properties not present in the lower stages.

   • Semiotic Thresholds: The transitions between levels represent major evolutionary leaps.

3. Subsumption and the Multi-layered Self: higher levels subsume lower ones.

   • Modern humans operate across all four levels at once, simultaneously possessing a biological self, a minimal conscious self, a sign-using self, and a linguistic self.

   • Hybrid Mind: Our experiential world is a composite of a biological Umwelt, a natural Lebenswelt, a cultural world, and a linguistic Universe of Discourse.

4. Fundierung (founding):

   • Lower levels prefigure (zapowiadają) the higher ones, while higher levels consolidate the lower foundations without ever breaking away from them.

   • Free activity at one level eventually becomes sedimented, forming the normative foundation for the spontaneity of the next level.

The Semiotic Hierarchy: funtions of levels (Zlatev)

1. Level 1: Life:

   • Ensuring the survival and reproduction of the organism.

   • Picks out an Umwelt from the environment based on biological significance.

2. Level 2: Consciousness

   • Gestalt Binding: Primary function of consciousness is to bind fragmented sensory data into coherent, multimodal objects and scenes (Gestalts).

   • By transforming the Umwelt into a Natural Lebenswelt, it allows the subject to move beyond automatic biological responses and choose actions based on phenomenal values like pleasure or pain.

   • Acts as a "common currency" for perception and evaluation, helping the organism anticipate the success or failure of problem-solving.

3. Level 3: Sign Function

   • True signification — subject can distinguish between an Expression and a Content.

   • Cultural Mediation: Allows subject to participate in a Cultural Lebenswelt, where phenomena are valued because they stand for something else.

4. Level 4: Language

   • Language functions as a tool for thought, creating inner speech and allowing for self-regulation.

   • Enables the development of a narrative self, where the subject builds a rich biographical identity and can mentally travel through time to reflect on the past or plan for the future.

   • Provides a conventional-normative system for sharing abstract ideas and debating within a Universe of Discourse.

The Sign Function (Zlatev)

Sign exists only if an Expression signifies a Content or Referent for a Subject.

3 essential properties:

• Expression points to the content/referent, but not vice versa.

• Expression and content are perceived as qualitatively different.

• Perceiving the expression indirectly evokes the content or referent.

A sign therefore minimally involves a triadic relation between Expression, Content/Referent, and Subject.

Why Consciousness is Necessary for Sign Use (Zlatev)

Consciousness is a necessary condition for the sign function because only a conscious subject can:

• distinguish expression from content,

• understand that one stands for the other,

• establish the connection between them.

Without a conscious subject, there may be physical or causal relations, but not genuine sign relations.

Mimetic Skills (Zlatev)

Mimetic skills constitute an evolutionary bridge between consciousness and sign use. They involve the use of the body as a representational device through imitation, pantomime, and intentional gestures.

Zlatev distinguishes:

• Proto-mimesis (e.g., imitation of newborns),

• Dyadic mimesis (imitation and reenactment),

• Triadic mimesis (pointing and pantomime).

The Semiotic Thresholds (transitions between levels) (Zlatev)

• Life —> Consciousness: Consciousness emerged to provide behavioral flexibility. By binding fragmented sensory data into stable objects, it allows organisms to choose actions based on phenomenal values (like pleasure or pain).

• Consciousness —> Sign Function: Mimetic skills act as a bridge. These skills allow body to be used as a representational device, establishing the double differentiation (expression vs. content) necessary for the sign function.

• Sign Function —> Language: Linguistic communication relies on socially shared conventions and norms that speakers are expected to follow. Through internalization, language becomes inner speech and restructures perception, cognition, self-regulation, planning, and autobiographical self-understanding.

Three levels of the object’s presentation (Sokolowski)

1. SIDES — Can be given to an observator in different ways. A cube has 6 sides, but perception is always partial. We can never see all sides at once. While one side is directly visible (present), the others are cointended as absent, forming a halo of potential views around the perceived object.

2. ASPECTS — Given because of the sides. A side itself can be given in different ways depending on the angle of view. For instance: A side facing us directly has the aspect of a square. If the cube is tilted, that same side takes on the aspect of a trapezoid. As the cube is turned further, the aspect may become a mere line before vanishing from view.

3. PROFILES — Are given because of the aspects. These are momentary, temporally individuated views of an aspect.

SIDES and ASPECTS are intersubjective (many can see the same aspect). 

PROFILE is private and subjective, depending on the viewer's specific condition (e.g., a profile may be "wobbly" if the viewer is dizzy).

Identity of an object is not the same as the appearance of it. Identity belongs to what is given in experience (it is not only the sum of the empirical impressions). Identity is available to all (not private). Our experience is a mixture of presence and absence. Recognition of identity belongs to the intentional structure of experience – intentionality allows us to recognize the identity of an object.

These three levels are moments of one another. They cannot be separated from the whole presentation of the object without losing their meaning.

Three formal structures of phenomenology (Sokolowski)

1. Parts and Wholes

This structure distinguishes between two different kinds of parts that constitute a whole:

• Pieces (Independent Parts): That parts can be detached from their wholes and still subsist as independent entities, such as a branch separated from a tree. When separated, pieces become wholes themselves.

• Moments (Nonindependent Parts): These parts cannot exist or be presented apart from the whole to which they belong, such as the color of a surface or the pitch of a sound.

• In phenomenology there is no classic mind-body problem. MIND IS A MOMENT, NOT A PIECE. Mind cannot be separated from brain and the body (like pieces) — mind is a moment that has brain and body background. We cannot separate mind from the external world and objects that are in it — mind is correlated with objects (it is intentional).

2. Identity in a Manifold

This structure describes the one in many – the way a single, identical object is presented through a multiplicity of different appearances.

• The Manifold: Set of appearances, such as the different sides, aspects, and profiles of a cube, or the different expressions and utterances through which a single meaning is given

• The Identity: The "one" that transcends the "many" appearances. It belongs to a different dimension than the manifold of its presentations and cannot be reduced to any single appearance.

• This structure applies to material objects and meanings. It can be expressed through a manifold of different languages, intonations, or sentences. It also applies to historical events (event experienced by many participants and remembered in various ways) and aesthetic objects (one symphony given through different interpretations).

3. Presence and Absence

• Filled Intentions (Intuition): Target an object in its bodily presence before the intender. The actual seeing of an object is a filled intention.

• Empty Intentions: These target something that is absent or not currently present. 

  • Both are toward the same object. That thing is at one time absent and at another present.

  • Synthesis of Identity: We recognize an object as the same despite changes between presence and absence.

  • Both presence and absence take different forms depending on the object.

  • Kinds of presences: the future becomes present as time passes; distant things by being closer; hidden sides of objects when we change perspective; understanding by thinking through steps of a process or translating meaning of an unknown language; and danger when we take risks.

  • Presence gains meaning against the background of absence. Finding something lost, for example, replaces absence with presence.

  • Kinds of absences: Things are absent because they are: future, past, contemporary but far away, forgotten, concealed or secret.

Fullfilment (addition to presence and absence) (Sokolowski)

Fulfillment – the process of filling an empty intention. Sometimes we do not move directly from an empty intention to a fulfillment. It can be a proces with a series of steps.

For example, when wanting to see a sportsman play, reading about them, seeing photos, watching interviews, and arriving at the stadium are all steps that bring us closer to fulfillment. Fulfillment occurs when we finally see the sportsman.

Two kinds of fulfillment:

1. Graded or Cumulative Fulfillment: Involves several intermediate stages, each bringing us closer to the object. The final intuition gathers the meaning of all previous stages.

2. Additive Fulfillment: Adds new aspects to one already present object. For example, learning more about a sportsman’s abilities or discovering additional meanings of a word deepens our understanding.

Single perceiver vs Multiple perceivers (Sokolowski)

• With a single perceiver, an object appears through different aspects depending on the person’s own perspective and movements. 

• With multiple perceivers,  there are more possible manifolds. The same object gains greater objectivity and richness of identity, because it is understood as something seen and interpreted from many perspectives at the same time. The perceiver still sees the thing as something changing with his movement, but also is aware that the same thing is being seen right now by someone else from a different perspective. These other people have different manifolds than the perceiver. 

• The same increase of identity occurs in objects, meaning of a text, art, or events. For example, we can understand that a text can be better understood by another person than by us.

Awareness of our own selves (Sokolowski)

• The awareness we have of our own selves is also a structure of identity in manifolds. 

• While we identify many things in the world, we are also identifying ourselves as the ones to whom these things appear. 

• Our personal identity is formed through the interaction of memory, imagination, perception, and our awareness of time. 

• Each of us remains the center of our own consciousness. We can never leave ourselves or become someone else

The Dichotomy of Expression and Content (Sokolowski)

In phenomenology, the relationship between a sign and its meaning is understood as the manifestation of an identity within a manifold:

• The Content (Exprimend): The meaning or fact is an identical core that remains the same across different presentations. It belongs to a dimension different from the manifold of expressions through which it is given.

• The Expression (Manifold): Includes the various ways a meaning is delivered. For example, the meaning "the street is covered with snow" remains the same whether language it is expressed, or whether it is shouted, whispered, or written.

• Sokolowski warns against treating the meaning as a separate piece or a ghostly analogue inside the mind. Instead, meaning is the public identity that is within and behind all its expressions.

Types of Signitive Intentionality (Sokolowski)

Sokolowski distinguishes between different ways cultural signs function based on their intentional direction and the blend of presence and absence they involve:

• Signitive Intentions (Words): These are outward bound. The arrow of intentionality goes through the perceived word toward an absent object. Unlike the halo of empty intentions in perception, verbal signitive intentions are discrete and identifiable, they mean their target all at once as a whole.

• Pictorial Intentions (Pictures): In picturing, the direction of the arrow is reversed, the object is drawn near and its presence is embodied on a substrate like canvas or paper. While words are abstract, pictures are perspectival and continuous, presenting the object in a specific light or pose.

• Indicational Intentions (Symbols/Signals): These point to an absent object (like a trail marker or a flag) but do not articulate how to intend it because they lack syntax.

Application to Cultural Phenomena (Sokolowski)

This analysis of identity in manifolds allows for the rigorous study of complex cultural entities:

• Historical Events: Some event is a single identity presented through the manifolds of participants’ memories, contemporary newspaper reports, historical books, films, and memorial celebrations.

• Aesthetic Objects: A symphony is an identity given through its various executions or performances, a drama is an identity presented through various stagings and interpretations. In paintings, the experience involves an interplay between the material substrate (paint and canvas) and the thematic content (the depicted object).

• Religious Events: The sacred is treated as an identity within a manifold of presentations. For instance, the Exodus is an identity present to those who lived it, those who read about it in scripture, and those who celebrate the Passover.

The Publicness of Meaning and Syntax (Sokolowski)

Sokolowsky rejects psychologism. Meaning and truth are public achievements in the space of reasons. Furthermore, Sokolowski emphasizes that syntax (the syncategorematic parts of language) is what allows for the complex articulation of these cultural identities. Syntax elevates animal signaling into human discourse, allowing us to master absences and communicate states of affairs to others across time and space.

Hierarchical model of cognitive and cultural evolution in hominids (Donald)

• Each new stage does not replace the previous ones, instead, they are retained and consolidated, resulting in the multi-layered, hybrid mind of modern humans.

Before identifying the four stages, Donald establishes three criteria for a genuine major cognitive transition. 

A transition must involve: 

1. Emergence of new consciously retrievable representations, 

2. Representations that are inherently public and communicable, 

3. Appearance of a new semi-autonomous layer of culture built upon these representations. 

Donald uses these criteria to justify each transition in his model.

1. Episodic Culture

Starting point of hominid evolution, corresponding to the cognitive world of great apes.

• Primates.

• Cognition is concrete and reactive, meaning it is bound to immediate environmental events.

• Representation is limited to complex event-perceptions.

• While intelligent, apes have a very limited range of expressive outputs and cannot invent public representational arenas, knowledge remains locked within individual brains and dies with the individual.

• Even highly intelligent apes can understand representations created by others, but they cannot invent new public representational systems or create a shared symbolic arena that can accumulate knowledge across generations.

2. Mimetic Culture

The first transition solved the output problem of primates through a revolution in motor control.

• Early hominids, peaking in Homo erectus.

• Mimesis, or non-verbal action-modelling. This is the ability to voluntarily control, rehearse, and refine one's own motor actions A revolution in non-verbal skills, such as gesture, pantomime, dance, athletic skill, and shared attention.

• It allowed for the public transmission of knowledge through non-verbal means, creating the first layer of culture that could accumulate over generations.

3. Mythic Culture (The Second Transition)

The second transition added language, providing a more powerful system for organizing knowledge.

• Sapient humans, peaking in Homo sapiens sapiens.

• Linguistic modelling and high-speed phonology.

• The governing style is narrative thought. Culture is organized around shared oral traditions – public, standardized versions of reality permeated by mythic archetypes.

• Oral societies organize reality through shared narratives, myths, archetypes, and allegories. These collective stories provide a standardized interpretation of the world and shape social conventions and patterns of thought.

• Oral culture did not replace mimetic culture. Language was added on top of the existing mimetic layer to play an important role in human cognition and culture.

4. Theoretic Culture (The Third Transition)

Externalization of memory storage, moving knowledge from biological brains to physical objects.

• Recent sapient cultures.

• Externalization of memory (external symbolization).

• The use of exograms (external memory records) such as writing, maps, diagrams, and mathematical equations. They are virtually unlimited in capacity, permanent, and can be reformatted.

• Logical, scientific, and formal modes of thinking become the dominant form of representation. Such forms of reasoning emerge through long-term interaction with external symbolic systems and must be transmitted through culture.

The Modern Hybrid Mind (Donald)

• Shift from isolated minds of other mammals, to collective human mind.

• Through enculturation, individuals become part of a shared culture and acquire language, symbols, knowledge, and representational systems created by previous generations.

• This process unlocks cognitive abilities and makes human cognition increasingly collective, distributed, and culture-dependent.

• As a result, modern humans are cognitive overachievers who combine all earlier cognitive layers — episodic, mimetic, mythic, and theoretic — in both individual and collective life.

Cognitive Functions that Distinguish Hominids from Other Primates (Donald)

1. Range of voluntary non-verbal expression

2. Iconic and metaphoric gesture

3. Mutual sharing and management of attention

4. Self-cued rehearsal

5. Refinement and imitation of skills

6. Generative, self-cued and innovative imagery

7. Improved pedagogy and diffusion of skill

8. Greatly increased speed of communication

9. Increased memory storage

10. Voluntary (explicit) retrieval from memory

11. New forms of representation — words, larger narrative structures

12. Autobiographical memory

13. Shared representational control of emotions

14. More complex overall architecture of memory

15. Integration of material culture into the process of explicit knowledge representation

Agency and Subjectivity (Zlatev, Mendoza)

Agency and subjectivity are two inseparable aspects of intentionality.

• Agency is the active side of experience: acting, initiating, trying, and producing effects ("by-me-ness").

• Subjectivity is the receptive side: how experiences are felt and appear to the individual ("for-me-ness").

These dimensions depend on each other, so genuine agency always requires at least a minimal sense of agency and a first-person perspective.

Therefore, artefacts and very simple organisms such as bacteria are not genuine agents. They can take part in causal processes, but they lack the subjective experience required for true agency.

Agency Hierarchy (Zlatev, Mendoza)

Original Agency (Shared to some degree with non-human animals)

• Level 1: Operative Agency – Skilled movements performed without conscious attention, such as running. It is characterized by "intention-in-action," where the purpose is embedded directly in the movement.

• Level 2: Reflective Agency – Requires prior intention, imagination and conscious orientation toward goals.

  • An example is an athlete visualizing and focusing on the specific movements needed to perform a high jump.

• Level 3: Joint Agency – Involves coordination between multiple individuals, shared goals and collective action.

  • This ranges from spontaneous group dancing to the explicit shared intention of two people carrying a heavy object together.

Enhanced Agency (Considered uniquely human)

• Level 4: Artefact-mediated Agency – Creating, modifying and improving tools and also extending human action through material artefacts. Designing and improving tools is uniquely human. Artefacts become incorporated into human action and allow individuals to perform actions that would otherwise be impossible or less effective.

• Level 5: Sign-mediated Agency – Mediated by non-verbal signs (iconic or indexical). It requires a conscious awareness of the representational status of an expression, such as a pantomime, iconic signs and indexical signs.

• Level 6: Symbol-mediated Agency – Mediated by language and highly articulated symbolic systems. This enables complex communication, collective narratives (storytelling), and the coordination of powerful collective agents, such as in corporate branding.

Life-World (Lebenswelt (Zlatev, Moratidou)

Phenomenology is often criticized as introspective, anthropocentric, and unable to produce objective knowledge. Zlatev and Mouratidou reject this view. They argue that the life-world (Lebenswelt) is shared, intersubjective reality created by both humans and non-humans.

Through empathy and interaction, we recognize animals as purposeful beings with their own perspectives on the world.

The life-world (Lebenswelt) is the world of direct experience. It is the only reality that we both live in and can genuinely know.

The Vertical Plane (Epistemological) (Zlatev, Moratidou)

1. First-person perspective: 

   • Rooted in the researcher’s own subjectivity. 

   • Philosophical reflection.

   • Uses methods like phenomenological reduction and intuition analysis, 

   • Direct examination of experience.

1. Second-person perspective: 

   • Grounded in empathy and reciprocity, where the researcher and participant meet as equals. 

   • Prototypical methods include the phenomenological interview and psychotherapeutic dialogue, which co-generate knowledge through interaction and intersubjective validation.

   • Participant observation.

   • Qualitative methods.

   • Phenomenological Nod: A form of intersubjective validation in which others recognize a phenomenological description as matching their own experience.

2. Third-person perspective: 

   • The most distanced viewpoint, typical of the natural sciences. 

   • It treats phenomena as objective facts through an anonymous "they" perspective, utilizing experiments, neuroimaging, and causal explanations.

   • Quantitative analysis.

   • Experiments.

   • Scientific observation.

   • Detached perspective of a research community.

   • Critique of Scientific Objectivism: Phenomenology rejects the fallacy of metaphysical objectivity, the belief that science can access reality independently of all perspectives.

   • Phenomenology criticizes attempts to reduce humans and animals to purely neurochemical mechanisms. 

Intentionality (Zlatev, Moratidou)

Fundamental structure of existence. It includes conscious acts, bodily sensations, moods and absorbed practical activity through which embodied subjects are intertwined with the world. 

The Horizontal Plane (Ontological) (Zlatev, Moratidou)

• Self: Embodied starting point of experience. Humans experience themselves both as Leib (a sensing, willing subject) and Körper (a physical presence).

• Others: Other sentient subjects, including both human and non-human animals. We recognize their subjectivity through empathy and direct encounters where their purposive nature is felt. 

  • Animals are recognized through a relation of likeness rather than identity. An asymmetry between human and animal experience must remain, since complete identity would eliminate the plurality necessary for a shared world. 

• Things: Neutral entities such as objects, events, and physical phenomena that appear to consciousness.

Existence forms a triad of Self, Others and Things. It allows us to overcome solipsism because objects are recognized as existing beyond our own perspective through the perspectives of other conscious beings.

Representations in cognition according to classicism (Clark)

Internal representations are considered the foundational construct for understanding the mind. Cognition is fundamentally the creation and manipulation of these representations.

• Internal representations exhibit a quasi-linguistic and combinatorial structure. These representations are often described as "quasi-sentential," meaning that key cognitive contents are tokened as strings of symbols. This approach is closely associated with the "Language of Thought" hypothesis.

• "Read/Write/Copy" Architecture: The mind functions as a system that manipulates these symbol strings according to logical rules. It utilizes an architecture that can literally "read, write, and copy" these symbols, participating in what is described as a literally compositional computational economy.

• Classicism traditionally divides the cognitive system into distinct modules: perceptual modules act as inputs that deliver a symbolic description of the world to a central system, which then processes this information and sends a symbolic description of desired actions to action modules to execute them in the world.

• Representations are considered explicit because knowledge is represented as a transparent, syntactically structured item in a declarative code.

• These inner computational states act as the vehicles for specific contents, serving to "stand in" for external reality and guide behavior in its stead, particularly when environmental features are not reliably present to the system.

• Symbols are well-individuated inner items. They carry familiar, world-referring contents (such as 'dog' or 'cat') and are manipulated by an independent processor.

The role of representations in cognition according to connectionism (Clark)

1. Form: Vector Coding and Implicit Knowledge

Connectionist representations involve vector coding in high-dimensional state spaces.

• Knowledge is implicit, embodied in the network’s weights.

• Representation and processing are deeply linked. The system undergoes vector-to-vector transformations.

• These networks embody a powerful form of "knowing how" (skills and adaptive behaviors).

2. Functional Role: Transforming Input Spaces

Clark introduces "modest representations" to describe internal information-bearing states that emerge from filtering and transforming raw sensory signals.

• A primary function of these representations is to compress an input space (treating physically different inputs as similar, such as an object seen from various angles) or dilate it (treating physically similar inputs as radically different based on their functional value or context).

• Brain is compared to an efficient factory that selects useful raw materials, discards the unnecessary, and repackages the rest into a new, functional configuration appropriate for specific applications.

The role of representations in cognition according to enactivism (Clark)

1. Rejection of Cartesianism

It moves away from the idea of a subject representing the world. Instead, it emphasizes everyday skills, practices, and "being-in-the-world" as the true essence of cognition. It distrusts models that treat the mind as a separate arena insulated from embodiment and the environment.

2. "The World is Its Own Best Model"

Explicit representations often "get in the way" of intelligent behavior. Agent should "use the world as its own model," allowing behavior to be guided by direct interaction with the environment. Representation is considered the "wrong unit of abstraction" for understanding the bulk of intelligent systems.

3. Activity-Based Decomposition

Using a subsumption architecture, agents are designed as a collection of competing behavior-producing layers (such as avoid objects or explore) that interact directly with environmental signals without a central representational code.

5. Clark’s Critique and Revisionary Representationalism

While Clark and Toribio acknowledge the power of these tools, they argue that the anti-representationalist claim is vastly overstated. They contend that while simple coupling works for some tasks, representation-hungry problems — such as reasoning about absent objects or abstract concepts — mandatory require some form of internal stand-in to guide behavior.

The role of representations in cognition according to Andy Clark (Clark)

1. Solving "Representation-Hungry" Problems

Representations are essential for problem domains where environmental information is insufficient to guide behavior.

• When an object is not present, it cannot send a physical signal, therefore, the system must use an inner resource that stands in for the absent state.

• When a system must respond to properties that lack a single physical trait (e.g., identifying items that are "valuable") , it must use internal resources to recognize these abstract features.

2. "Modest" Representations as Input Transformations

Modest representations—internal information-bearing states that emerge from filtering and transforming raw sensory signals. Brain is like an efficient factory that selects useful raw materials and repackages them into a new, functional form. This process involves:

• Compression: Treating physically different inputs (like an object seen from different angles) as similar.

• Dilation: Treating physically similar inputs as radically different based on context or value.

3. The Representational Continuum

Representation is a continuum of degrees and types:

• Non-Representational End: Direct physical coupling with the environment, like a toy car with a "bump" sensor.

• The Middle (Modest Systems): Most cognitive systems, including connectionist networks, which use intermediate states as feature detectors to reveal abstract regularities.

• Full-Blooded End: Systems capable of internal reasoning in the total absence of external stimuli, enabling abstract thought and language.

Definition of the sign (Sonesson)

Sonesson defines the sign function as containing at least two parts – expression and content – which are relatively independent of the referent (the actual object in the world). A sign relation is established when the following conditions are met:

• From the subject's point of view, the expression and the content must be perceived as distinct and belonging to different categories.

  • For instance, a child using a pebble to represent candy is aware of the difference between them, if there is no differentiation, the relation is merely one of categorization or a direct reaction to a stimulus.

• Double Asymmetry: There is a hierarchical relationship between the two parts:

  1. The expression is directly present to the senses but is non-thematic (it is not the main focus of attention).

  2. The content is indirectly present (absent) but is the theme (the primary focus).

• The expression is perceived as being about the content, which in turn provides a standpoint on the referent.

The Sign in Iconic Contexts (Pictures) (Sonesson)

The definition becomes particularly relevant for iconic signs (like pictures), where expression and content share similar properties. Sonesson argues that understanding a picture requires a pictorial mode of attention, where the observer simultaneously recognizes the likeness to reality and the separation from it. Without this differentiation, a subject might treat a picture as a degraded stimulus or confuse it with reality (the reality mode), failing to grasp its function as a sign.

Theoretical Foundations of sing definition (Sonesson)

• From Husserl (Appresentation): Sonesson builds on Husserl's notion of appresentation, where a directly present object motivates the consciousness of something absent. While meaning exists in simple perception (e.g., seeing one side of a cube appresents the whole cube), it only becomes a sign when the absent item becomes the theme and the present item animates it.

• From Piaget (Semiotic Function): He adopts Piaget's requirement for a semiotic function, which is the ability to represent reality through a signifier that is distinct from the signified. Sonesson refines this by noting that even indices (like tracks in the snow) can be signs if the user transforms a real connection into a conscious differentiation between the mark and the animal.

Distinctions from Other Theories (Definition of sign) (Sonneson)

Sonesson explicitly positions his definition as a middle ground between classical semiotic theories:

• Narrower than Peirce: Unlike Peirceans, who often treat all meaningful relations as signs, Sonesson insists that many relations (like grounds or simple perception) lack the necessary differentiation and thematic structure to be true signs.

• Broader than Saussure: While Saussure restricted the sign primarily to language systems, Sonesson’s model encompasses pictures, gestures, and other non-linguistic representations.

The Three Layers of Pictorial Consciousness (Sonesson)

• Picture Thing: Physical material object that serves as the sign’s substrate. It includes the canvas, the paper, the pigments, or the lopsided wooden frame. It is what is directly present to the senses in the natural attitude.

• Picture Object: Image that we see in the picture thing. It is directly perceived through the material but is not identical to it. Picture object exists here and now, where the physical picture is located.

• Picture Subject: Intended object or the reality the picture aims to present. Sonesson defines the picture subject as the potential real-world equivalent of what is seen in the picture.

The Picture Referent (Sonesson)

Sonesson argues that Husserl’s original picture subject was forced to perform a double task that it could not sustain: acting as both the content type and the actual referent in the world. 

• Referent: Concrete object in the world that may exist elsewhere or not exist at all.

  • For instance, a painting of the Berlin Castle has a referent located in Berlin, while a painting of a unicorn has a picture subject (the idea of a white, horned horse) but no referent in reality.

• Difference in Presence: The picture object and thing are here, but the picture subject and referent are elsewhere. The Berlin Castle remains in Berlin regardless of where its picture is moved.

Impossible Pictures (Sonesson)

These pictures demonstrate the separation of these layers. They possess a picture thing and a recognizable picture object, but they lack a possible picture subject because no real-world three-dimensional equivalent could ever correspond to what is seen in the image.

Resemantization (Sonesson)

Pictures differ from language because they are objects of perception and signs simultaneously. While linguistic phonemes are meaningless in isolation, the individual strokes or dots (moments of the picture thing) take on specific meanings (parts of the picture object/subject) once they are integrated into the whole image.

Autonomy and Sense-Making (cognitive-semiotic approach)  (Thompson, Stapleton)

For a system to be considered an agent, it must possess adaptive autonomy.

• Autonomy: An autonomous system is one that generates and sustains its own activity through an operationally closed network of processes. It manages the flow of matter and energy to regulate both its internal self-construction and its exchanges with the environment.

• Sense-Making: Interactional side of autonomy. An agent transforms the world into a domain of meaning and value (an Umwelt) based on its internal norms. Actions are deemed good or bad depending on whether they facilitate or degrade the system's autonomy.

• Relational Domain: Agency and cognition belong to the relational domain – the process of engagement between the system and its environment.

Adaptive Autonomy (enactive approach) (Thompson, Stapleton)

Agent is a autonomous system characterized by specific organizational properties:

• Operational Closure: The system is composed of processes that recursively depend on one another to maintain a network. In such a system, every constituent process is conditioned by another within the same system.

• Thermodynamic Openness: While operationally closed, the agent must be physically open to the flow of matter and energy from the environment to regulate its internal self-construction.

• Precarious Conditions: Agency exists under precarious conditions, meaning that without the active organization of the system's processes, its components would tend to run down or extinguish.

• Adaptivity: To be a genuine agent, a system must possess more than mere self-production (autopoiesis), it must have adaptivity. Adaptivity is the capacity to regulate internal states and environmental interactions with respect to conditions of viability. This allows the agent to distinguish between better or worse states of being, providing a basis for normativity.

Agency as Sense-Making (enactive approach) (Thompson, Stapleton)

Sense-making — interactional and relational side of autonomy. It is the process by which an autonomous system establishes a perspective from which its interactions with the world acquire a normative status.

• Through sense-making, an agent transforms the neutral physical world into a place of salience and value – an Umwelt (environment). For example, sugar molecules are not food intrinsically, they gain that significance only in relation to a bacterium's metabolic needs.

• The agent modifies its surroundings according to the internal norms of its activity. Some interactions facilitate its autonomy (e.g., swimming toward food), while others degrade it (e.g., moving toward toxins).

The Relational and Embodied Nature of Agency (enactive approach) (Thompson, Stapleton)

Enactive approach views agency as a relational process:

• Cognition and agency are relations that take place between the agent and its environment. As Thompson and Stapleton note, "neurons do not think and feel, people and animals do".

• The agent's body (including brain) leads the dance in the body-world relationship because it realizes the autonomous organization necessary for individual agency.

• Agency can be extended through the incorporation of tools. When an external resource (like a blind man's cane) is subject to active regulation by the body and conforms to the transparency constraint, it is taken into the body schema and becomes part of the agent's sense-making apparatus.

Agency according to the extended approach (Thompson, Stapleton)

1. Autonomous Organization

An agent is an autonomous system — a network of processes that recursively depend on each other to generate and sustain the system as a unity.

• Operational Closure: In an agent, every constituent process is conditioned by other processes within the system, meaning the system defines its own domain of possible interactions with the world.

• Precarious Conditions: Genuine agency involves actively maintaining this identity under precarious conditions, where the individual processes would otherwise extinguish without the organization of the network.

2. Adaptive Autonomy and Sense-Making

Thompson and Stapleton argue that while autopoiesis (self-production) is a baseline, agency specifically requires adaptive autonomy.

• Regulation: A true agent does not just exist, it actively regulates its interactions with the world according to internal norms of viability.

• Normativity: This regulation establishes a perspective where interactions acquire a normative status (e.g., good or bad for the system's survival), transforming the world into a meaningful environment or Umwelt.

• Sense-Making: Agency is expressed through sense-making, which is the relational conduct of the organism in response to the significance and valence it has enacted.

3. Agency as a Relational Process

Cognition is an relational processes occurring between the system and its environment. “cognition has no location" and is a process in a relational domain.

4. Extended Agency: Incorporation vs. Extension

While the authors distinguish their view from the "extended mind thesis," they allow for an "extended" living system through the process of incorporation.

• Incorporation: This occurs when a body takes an external resource (like a tool or a physical gill in certain insects) into its operationally closed network.

• The Transparency Constraint: For an external resource to count as part of the agent's cognitive/agentive system, it must function transparently — meaning it is no longer experienced as an object but becomes a structure through which the world is experienced.

• Leading the Dance: For a resource to be incorporated, it must be subject to active regulation by the body. The body must be "capable of leading the dance" in the body-world interaction.

The Parity Principle: If a factor outside the head plays a comparable or equivalent role in producing behavior as a process inside the head, then that external factor should be considered part of the cognitive process.

Arguments in favor of sound symbolism: 1. The Fallacy of Arbitrariness and Eurocentric Bias (Ahlner, Zlatev)

19th and 20th-century linguistic consensus on arbitrariness was based on a “skewed database” primarily focused on Indo-European inflecting languages like Greek, Latin, and German.

• Saussure and others falsely equated conventionality (the fact that signs are socially shared and traditional) with arbitrariness (the lack of natural connection). They point out that a sign can be both conventional and motivated, such as in American Sign Language, where approximately 50% of signs are iconic.

• While Saussure dismissed onomatopoeia as a marginal curiosity, Ahlner and Zlatev argue that these words are organic elements of language systems that have been downplayed due to scientific biases that viewed them as too crude for etymological study.

Arguments in favor of sound symbolism: 2. The Global Pervasiveness of Ideophones (Ahlner, Zlatev)

Authors present the existence of ideophones as irrefutable evidence for sound symbolism.

• In languages like Japanese, Korean, and various Niger-Congo languages, there are dedicated dictionaries containing thousands of entries for ideophones that evoke sensory events as a whole.

• These expressions are fully conventional – part of shared linguistic knowledge – yet they display a non-arbitrary relation between form and meaning. For example, the Japanese koro-koro (a small object rolling) versus goro-goro (a large object rolling) uses sound contrast to reflect physical differences.

Arguments in favor of sound symbolism: 3. Empirical Proof of "On-line" Mapping (Ahlner, Zlatev)

Humans consistently perceive similarities between unknown sounds and visual shapes.

• The Bouba/Kiki Effect: In experiments where participants must match fictive words to shapes, 90–95% of people across different ages and cultures agree that bouba suits a round figure and kiki suits a pointy one.

• Research by Sapir showed that 80% of participants associated the vowel [a] with larger objects and [i] with smaller ones.

• Speakers can match antonym pairs (e.g., big/small) in a familiar language to pairs in an unfamiliar language at rates significantly higher than chance. Success depends on the words contrasting along a single qualitative dimension like ROUNDNESS or SIZE.

Conditions Necessary for Successful Sound-Symbolic Matching (Ahlner, Zlatev)

• Participants must be provided with a familiar reference pair.

• The contrasted items must differ along a single qualitative dimension such as size, roundness, brightness, smoothness, or sharpness.

When these conditions are absent, performance falls to chance level. This indicates that cross-modal iconicity depends on structured contrasts rather than isolated sounds.

Arguments in favor of sound symbolism: 4. Cross-Modal Iconicity as a Theoretical Alternative to Sound Symbolism (Ahlner, Zlatev)

Ahlner and Zlatev argue that the term sound symbolism is theoretically problematic. Drawing on Peirce’s semiotic theory, they note that a symbol is by definition based on convention and lacks a motivated relation between expression and content. For this reason, they propose the term cross-modal iconicity, which more accurately captures the similarity perceived between sounds and meanings across different sensory modalities.

Relation between linguistic forms and meanings is based on an iconic ground – a perceived similarity that allows auditory forms to be linked to visual, tactile, or other sensory experiences.

Arguments in favor of sound symbolism: 5. Findings from the Authors' Case Study (Ahlner, Zlatev)

Ahlner and Zlatev performed their own experiment systematically varying vowels and consonants to determine their relative roles in establishing an iconic ground.

• Both vowels and consonants independently and in combination contribute to cross-modal iconicity.

• Their data mathematically validated that voiceless obstruents (like [k] or [t]) and the front vowel [i] have an iconic affinity for sharp visual cues (a "star" shape), while sonorants (like [m] or [l]) and the back vowel [u] map to rounded shapes (an "amoeba").

• When they paired a round vowel with a sharp consonant (e.g., tutu), the matching rate dropped precipitously, proving that conflicting cues undermine the cognitive consensus for iconicity.

Second-Order Iconicity (Ahlner, Zlatev)

The similarity involved in many sound-symbolic tasks is a resemblance between relations. Participants perceive analogous relations between pairs of sounds and pairs of perceptual objects. This relational similarity constitutes a form of second-order iconicity and plays a central role in cross-modal mapping.

Primary and Secondary Iconicity (Ahlner, Zlatev)

Using Sonesson’s extension of Peirce’s semiotics, Ahlner and Zlatev distinguish between primary and secondary iconicity.

• In primary iconicity, perceived similarity is sufficient to establish a sign relation.

• In secondary iconicity, the sign relation is known first and only then does the resemblance become noticeable.

Many sound-symbolic phenomena involve a combination of both processes. In experiments such as Bouba/Kiki, participants first assume that sign relations exist and then perceive a shared iconic ground, which subsequently enables successful matching.

Arguments in favor of sound symbolism: 7. Resistance to Historical Sound Change (Ahlner, Zlatev)

A final argument for the organic nature of sound symbolism is that these words often resist regular historical sound shifts that affect the rest of a language's lexicon.

• Sound-symbolic expressions frequently display unusual patterns. For example, Japanese mimetics retained the word-initial [p] (signifying explosive movement) long after it shifted to [h] in the rest of the language.

• Every human language retains a foundational substratum of sound symbolism, though its explicit presence varies by culture. Iconicity is often a vital catalyst during the birth of new human expressions before it is slowly diminished by standardization and writing. The authors therefore reject the strict separation between arbitrary and motivated signs. Sound symbolism remains a persistent substratum of language, even when historical change and conventionalization make iconic relations less transparent.