A Unified Account of Concepts and Chunks

1Introduction

Langley (2025) distinguishes two kinds of cognitive structure. A concept "denotes a set of entities or situations and ... characterizes them in terms of their regularities." A chunk "denotes a collection of entities, possibly typed, that form a specified configuration or relational pattern." Research on concepts emphasizes generalization; research on chunks emphasizes composition. A unified theory of cognition needs both, because a system that generalizes without composing can encode knowledge at only one level, while a system that composes without generalizing cannot apply its configurations to novel situations.

Cobweb (Fisher, 1987) offers a mature account of concepts but has nothing to say about composition. Singaravadivelan and Langley (2026) keep every postulate of Cobweb and add the minimal machinery needed for chunks. They make no commitment to modality: the theory applies to any experience that decomposes into elements and relations among them. As a testbed they choose context-free grammars, whose part-of-speech classes behave like concepts and whose phrases behave like chunks.

2Cobweb in brief

Cobweb stores instances, each a set of attribute-value pairs, and concepts, each a probability distribution over the values of every attribute. Concepts form a taxonomy with a single root and instances at its leaves; every internal node summarizes the instances beneath it and has at least two children. Cobweb classifies an instance by sorting it downward greedily by category utility until it reaches a leaf or no child improves on the current node, and it predicts missing attributes from the node where it halts. Learning is incremental, unsupervised, and interleaved with performance: sorting an instance updates the probabilities of every node it passes, and at each node Cobweb chooses among adding the instance to a child, creating a new child, merging two children, or splitting one (see the Cobweb explainer).

3An extended theory

3.1Representation

The extension adds three representational postulates. An experience is a set of elements and the local relations among them, such as words related by before. Every element is either a primitive, such as a word, or a composite, such as a phrase; composites are the chunks. Every element can be described in two ways. Its content lists its constituents: black cat consists of black and cat. Its context lists the elements around it and their relations: cat follows the and precedes saw. Each description is an ordinary Cobweb instance, so Cobweb can sort, generalize, and learn from either. Figure 1 shows both descriptions for the elements of one parse.

3.2Organization

Long-term memory holds two Cobweb taxonomies. The content taxonomy organizes composites by their constituents, so black dog and black cat share an ancestor. The context taxonomy organizes elements by where they occur, so dog and cat group together, as do noun phrases that precede verbs. The taxonomies are intertwined: the content of a composite names concepts in the context taxonomy. The content taxonomy plays the role of individual rewrite rules such as VP \(\to\) V NP, and the context taxonomy holds the knowledge of which rule to choose.

3.3Performance

Parsing builds a partonomic tree from the bottom up. At each step it forms candidate chunks over adjacent elements, sorts each candidate through both taxonomies, admits those that clear a recognition threshold, and commits the best one. It halts when no candidate is recognized or a single composite remains, so an unfamiliar input yields a partial parse rather than an error. Generation runs in the opposite direction. It expands a composite from the top down by recalling a candidate decomposition from the content taxonomy, conditioning that choice on the context taxonomy, and stopping when every element is primitive.

3.4Learning

Learning needs no new mechanism. Once parsing commits an element, the system incorporates its content instance into the content taxonomy and its context instance into the context taxonomy, using Cobweb's ordinary operators. Primitives enter only the context taxonomy; composites enter both. Chunk creation is therefore a side effect of sorting, and Cobweb's learning method, designed for independent instances, extends to sequential data.

4TRELLIS

TRELLIS implements the theory for sentences. A primitive's context instance covers five positions on either side, weighted by distance: the nearest neighbor counts 1/2 and each further step halves the weight, with an EMPTYNULL value past the sentence boundary. A composite's content instance encodes each of its two constituents by up to three identifiers of context concepts at depth four of the context taxonomy, plus a complexity tag that records the constituent's depth of nesting; the tag separates, for example, a determiner-noun pair from a sentence built of two phrases. The content taxonomy never sees a primitive word.

TRELLIS recognizes a candidate by its log-likelihood under a node, smoothed per attribute,

and converts it to a posterior, \(\log P(C \mid i) = \log P(i \mid C) + \log P(C) - \log P(i)\). Normalizing by the instance's marginal makes the score discriminative, so non-constituents score low. A candidate's rank is the sum \(s_c + s_x\) of its content and context posteriors.

4.1Greedy parsing

The parser begins with the words of a sentence as its frontier. For the cat saw a dog, the first step considers four adjacent pairs. TRELLIS sorts each candidate through both taxonomies and walks upward from the node where sorting halts; it admits the candidate only if some ancestor below the root has a count above \(\tau_{\text{parse}} = 30\). Among admitted candidates it commits the one with the highest \(s_c + s_x\) and replaces its constituents on the frontier with the new composite. Early in training few nodes clear the gate, so the parser stops with a partial parse; later the gate rarely binds, and the discriminative score does the work. Figure 3 steps through this loop.

4.2Generation

Generation expands a seed composite repeatedly. From the seed's leaf in the content taxonomy, TRELLIS climbs to the deepest ancestor whose count exceeds \(\tau_{\text{sub}} = 20\) and samples a new leaf below it, a different constituent of the same kind. It then climbs to the deepest ancestor whose count exceeds \(\tau_{\text{gen}} = 50\) and gathers the chunks beneath it into a recombination pool, which can yield combinations that no single training sentence contained. A context filter removes candidates that the parent does not expect; if it removes all of them, TRELLIS falls back to a wider pool and accepts a small risk of error. The recall threshold sets the balance, as the authors put it: "If this is too low, then generation will draw on very specific nodes supported by only a few cases; if too high, it will favor general nodes that mix constituents which do not belong together." Figure 4 shows the three regimes.

5Evaluation

The authors follow the methodology of Langley and Stromsten (2000). Errors of omission, in which the learned grammar is too specific, are measured by asking TRELLIS to parse novel sentences from the target grammar. Errors of commission, in which it overgeneralizes, are measured by parsing TRELLIS's own generated sentences with the target grammar. Both receive partial credit by comparing the unlabeled brackets of each subtree. The first experiment varies grammar complexity over three nested grammars with 3, 6, and 8 nonterminals (Table 1); the second holds the MED rules fixed and varies the lexicon over 11, 22, and 39 words. Each condition uses 400 sentence-parse pairs under five-fold cross-validation, and all parameters stay fixed (Table 2).

TRELLIS acquires the grammars quickly. Both error rates drop to 20 percent almost immediately and then improve gradually with more training sentences, reaching low percentages even for the LARGE grammar and its recursive rules. More nonterminals slow learning, and a larger lexicon slows it further, but in both cases the degradation is graceful. Inspection of the learned taxonomies shows a content concept for each nonterminal and a context taxonomy that groups determiners, adjectives, nouns, verbs, and prepositions; nodes become nearly pure well above the leaves.

6Discussion

The authors close with an argument about compression. Creating a chunk compresses experience: once a phrase becomes a composite, its context occupies one slot in a neighbor's description rather than many. Contemporary language models take the opposite approach, retaining every token and paying the quadratic cost of relating them all. The authors do not claim that TRELLIS competes with such models at their present scale, but they do claim that a memory which builds and refers to its own units handles context better than one that retains the entire past.

The work sits among several lines of research. Earlier Cobweb descendants such as LABYRINTH (Thompson & Langley, 1991) and TRESTLE (MacLellan et al., 2016) handled structured objects but lacked a context taxonomy or flattened their partonomies during processing. Discrimination networks such as EPAM (Feigenbaum, 1961) and CHREST (Gobet & Lane, 2012) acquire chunks but parse from the top down, whereas TRELLIS parses from the bottom up. The present system also has clear limits: its parser is greedy and cannot recover from garden-path sentences, it learns from sentences paired with parse trees, and it has been tested only on synthetic context-free grammars without baselines. The authors list the next questions: a parser that reconsiders its choices, learning constituents from sentences alone, context-sensitive grammars, natural corpora such as the Penn Treebank, other domains that need both generalization and composition (scenes, plans, chess), and a new kind of language model that is more interpretable, more efficient, and faster to learn.

References

1. Feigenbaum, E. A. (1961). The simulation of verbal learning behavior. Proceedings of the Western Joint Computer Conference, 121–132.
2. Fisher, D. H. (1987). Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2, 139–172.
3. Gobet, F., & Lane, P. C. R. (2012). CHREST: A model of perception, learning, and problem solving. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning, 556–559. Springer.
4. Langley, P. (2025). Concepts and chunks in cognitive systems. Advances in Cognitive Systems, 11, 33–44.
5. Langley, P., & Stromsten, S. (2000). Learning context-free grammars with a simplicity bias. Proceedings of the Eleventh European Conference on Machine Learning, 220–228.
6. MacLellan, C. J., Harpstead, E., Aleven, V., & Koedinger, K. R. (2016). TRESTLE: A model of concept formation in structured domains. Advances in Cognitive Systems, 131–150.
7. MacLellan, C. J., Matsakis, P., & Langley, P. (2022). Efficient induction of language models via probabilistic concept formation. Proceedings of the Tenth Annual Conference on Advances in Cognitive Systems.
8. Singaravadivelan, K., & Langley, P. (2026). A unified account of concepts and chunks. Advances in Cognitive Systems (oral presentation). arXiv:2609.30414.
9. Thompson, K., & Langley, P. (1991). Concept formation in structured domains. In D. H. Fisher, M. J. Pazzani, & P. Langley (Eds.), Concept formation: Knowledge and experience in unsupervised learning, 127–161. Morgan Kaufmann.