Generalizations and Abstractions
Aug 16-Sept 5, 2020
I discovered that generalization and abstraction meant pretty much what I thought they meant. But I also discovered that I had difficulty defining the two concepts concisely. Generalization is a general term; that is, there are a variety of generalizing processes. There is more than one definition for generalization. The same is true for abstraction but what may be a little surprising is that the relation of abstraction to generalization can be relative. Abstractions and generalizations can share so many attributes that their differences are really most clear in the process of using abstractions to create generalizations. In fact, I found that generalization is sometimes defined as a kind of abstraction.
Generalization is a method of compression. However, to be efficient the generalization has to consist of a number of particulars. I feel that an extensive set of cross-generalizations could be useful in AI but there are some problems with my plan. An extensive collection of cross-generalizations means that data relevant to some subject would be widely distributed so the retrieval of this data is not going to be easy. And the kind of generalizations that I think are most natural to cross-generalization are not as efficient as the traditional logical kind. But, by examining what a generalization is a little more closely I hope to gain some traction to try to create a model to test the basic feasibility of my ideas.
It seems that the ability to form generalizations is an essential skill of human thought. The most basic kind of generalization is the realization that something similar to what has happened in the past might happen again. This realization may not be a conscious insight, but more of a potential to react to an occurrence that shares some of the same characteristics of events that had occurred previously. Another form of generalization is the inference from a few cases to a greater number of cases. Again, this does not have to be fully realized as an insight but may appear as a reaction where similar events can stimulate a similar response. All learning must involve a form of generalizing. Anytime we analyze, think about, predict, prepare, or react to an occurrence based on our previous experiences and learning we are utilizing the ability to generalize our previous experiences
I presume that most animals can generalize to some extent. And I feel that generalization must, in some simplified or mechanical form, be a fundamental principle of life. The descriptions of antibodies and T-Cells in human immunological responses are descriptions of bio-mechanical adaptation that effectively implement a generalized reaction to certain kinds of cells or to the features cells that have been encountered and caused some reaction. So my guess is that many other aspects of life are also effective implementations of some adaptive skill of generalization. My interest is, of course, is in the generalizations that occur in higher thought processes.
I tried to find a starting point for my definition of generalization by looking at the most natural psychological processes that I could imagine and it turned out to be unexpectedly challenging.
We often find that an object or feature can capture our interest for a moment. This attention is a step in the process of abstraction. I sometimes notice a feature of an object and I may think about the times I have noticed similar features on other ‘objects’ and wonder if there is a reason for the apparent similarity.
The simplest definition of generalization in natural psychology is the realization that there may be something similar about different objects, events, or situations. The Encyclopedia Britannica defines generalization as used in psychology, as the tendency to respond in the same way to different but similar stimuli. One form of generalization leads to the induction that some similar objects are ‘of a kind’. But another form of generalization leads to the inception of an awareness that different kinds of objects can exhibit or possess some noticeable similar features. This kind of generalized insight can also deal with principles, reactions, ideas, and theories. And it may be semi-conscious.’
Since human thought can deal with dynamic principles like theories, principles, and methods a generalization in an AI program might include dynamic algorithms. If a generalization in a computer program modified the generalization itself or other generalizations that it interacted with the generalization could change as it was being used.
The Encyclopedia Britannica defines abstraction (in thought) as, “the cognitive process of isolating, or “abstracting,” a common feature or relationship observed in a number of things, or the product of such a process.”
They then define generalization in psychology as, “the tendency to respond in the same way to different but similar stimuli.”
One Wikipedia entry defines abstraction this way: “Abstraction in its main sense is a conceptual process where general rules and concepts are derived from the usage and classification of specific examples, literal signifiers ("real" or "concrete"), first principles, or other methods.
The next definition they offer is: “ ‘An abstraction’ is the outcome of this process—a concept that acts as a common noun for all subordinate concepts, and connects any related concepts as a group, field, or category.”
The antonym of abstraction is specialization. Concrete is sometimes used as an antonym of abstract.
Looking at Wikipedia’s definition of generalization I found:
“A generalization is a form of abstraction whereby common properties of specific instances are formulated as general concepts or claims.”
So Wikipedia is defining a generalization as a kind of abstraction. But they defined abstraction as, “a conceptual process where general rules are derived…” So the Wikipedia definitions seem to say that generalizations emerge from abstraction and they are abstractions themselves. There seems to be some overlap between the nature of generalization and the nature of abstraction.
Another definition of generalization from Wikipedia is, “In general, given two related concepts A and B, A is a "generalization" of B if and only if:
This kind of generalization is used in taxonomic classification schemes. For example, a bird is an animal but there are other animals which are not birds. And a cardinal is a bird but there are other birds which are not cardinals.
The other common definition of generalization comprises it of a group of ‘objects’ or ‘particulars’ which all share some common features. This brings something extra to the definition. It talks of common features that the objects share. I will call this form of generalization a ‘universal generalization’ for simplicity. A Universal Quantifier in predicate logic means “for all” so I am coining the term universal generalization because all the instances of the generalization share the common features. The universal generalization is an efficient way to store data in a computer program because the common features only have to be listed once in the generalization itself.
The ‘universal’ from medieval philosophy would seem to fit the description of a universal generalization. But it also seems to bring something more to the definition. For example, triangularity is sometimes suggested as a universal of all triangles. Triangularity is an idealism that cannot be constructed because it encompasses not one or some finite number of triangles, but all possible triangles. And any drawing of a triangle is going to be less than perfect but the idealism of triangularity seems to exist within some realm of geometric perfection. So does this elusive essentiality mean that universals are just constructs of the imagination? The impractical idealized concepts of trianularity ironically become necessary to make geometry work in the real world. If triangularity is not some kind of reality then why does the unattainable perfection of trigonometry have such a practical value in engineering? Do universals intuitively speak of a mysterious perfection like a religion? Or does the essence of the generalization of all triangles exist because the accomplishments of engineering makes the analytical tools that define the subject valuable regardless of the strangeness of philosophical extrapolations like this?
I think the most common, or perhaps the most typical form of generalization is comprised of familial features where each member of the generalization has some of the characteristics of the family but not necessarily all of them. Paraphrasing an entry about Wittgenstein from the website, ‘Philosophy Index’: ‘Wittgenstein says that some words do not have a single essence that encompasses their definition, so instead the relationship between the uses of the word is more interesting. It is here that he brings up family resemblance. Wittgenstein says that the way in which family members resemble each other is not through a specific trait but a variety of traits that are shared by some, but not all, members of a family.’ Although Wittgenstein was talking about language, I feel that this is a good way to define what I call the familial generalization. All the members of the generalization family share some of the traits of the family but not all of them. I feel that familial generalizations are the most common kind of generalizations that we naturally create. Our insights are not perfect and there are so many good interrelations that might be made between subjects of thoughts it is unlikely that an all-encompassing universal generalization will suffice. It is my opinion that extensive cross-generalization is an artifact of natural human thinking – if not an essential basis of it. And cross-generalization will tend to create familial generalizations.
The webpage of the Department of Philosophy at Texas State University states that, “overgeneralization or unwarranted generalization occurs when you make a generalization based on insufficient evidence. It usually takes the form of making a generalization that maps a single or limited experience onto a wider population. ‘Bald people are geniuses. My brother is bald and he is a genius.’” If a ‘familial generalization’ is treated like a ‘universal generalization’, then it becomes more likely that it would be used as an overgeneralization.
One way the distinctions between abstraction and generalization can be enforced in the mind is through mental compartmentalization. We can compartmentalize our thoughts in order to simplify and to develop insights about concepts that are interesting to us. It seems to me that compartmentalization is a fundamental aspect of abstraction. And, in my opinion, by combining generalization with compartmentalization we can achieve an efficient meta-awareness of what we are doing. If meta-awareness was as detailed as the target thoughts that were being monitored it would become too complicated. There are different kinds of generalizing. One other form of generalization is the use of more vague forms of expression to produce more sweeping insights. For example, the idea that I am making a presentation is a low cost meta-awareness of what I am doing. However, if I want to use the memory of that vague generalization, ‘I made a presentation’ I would have to rely on my knowledge about what a presentation is. Another slightly more detailed but still vague generalization is that my presentation consisted of reading a paper I prepared for an online group. These descriptions are both vaguer than the presentation itself. If I have to answer some questions I will need to draw on more specific memories about the presentation but lower cost generalizations should be adequate for casual monitoring. And I could supplement my actual memories of the presentation with my knowledge of the subject. I think this kind of meta-awareness is necessary for advancing AI at this point. And what I am saying is that meta-awareness can be seen as a combination of different forms of generalization with compartmentalization.
A generalization can be an effective compression method if we already have a great deal of insight about a particular subject or about a particular specification of some subject. Often this previously acquired insight will relate to so many different kinds of relations that useful knowledge about a particular area of interest can be derived from a range of other subjects which we are also familiar with. This is particularly true if the other subjects are strongly related to the subject of the generalization or have a practical or emotional significance to the subject. I can combine various vague memories about this presentation even if I forget many of the details because I have thought about related issues so often.
If an Artificial Intelligence program was capable of using insightful generalization the use of familial generalizations along with extensive cross-generalization, will make finding possible relations easier as the insight of the program increases. This potential might be likened to a kind of computational compression. Even if familial generalizations are not as effective as agents of compression as universal generalizations are, they can help in effective investigations of the nature of objects.
Since general knowledge will tend to be distributed in an AI program, accessing it efficiently is not going to be a simple problem to solve. Vaguer generalizations and cross-generalization might help.
An item in a generalization could be part of other generalizations. This is the basis for cross-generalization.
Wiktionary has a definition of abstraction in computing: Any generalization technique that ignores or hides details to capture some kind of commonality between different instances for the purpose of controlling the intellectual complexity of engineered systems, particularly software systems.
Categorization, Classification, and Generalization all seem to have originated with Plato and Aristotle. So they are related. Wikipedia defines categorization as “an activity that consists of putting things (objects, ideas, people) into categories (classes, types, index) based on their similarities or common criteria. It allows humans to organize things, objects, and ideas that exist around them and simplify their understanding of the world.”
Their definition for classification is, “a process related to categorization, the process in which ideas and objects are recognized, differentiated and understood.” Webster’s defines taxonomy as an, “orderly classification of plants and animals according to their presumed natural relationships.” So a formal categorization would also be a classification.
A taxonomy is a categorization which uses generalizations within proper subsets of higher generalizations.
Could a universal essence be comprised of a familial generalization? I would say so. For example the essence of polygonality might be comprised of a number of related generalized definitions. While this kind of thought experiment might seem tedious, it can help us to think outside the box just because it does deal with the definition of the items of a generalization in a way that challenges us to better understand the distinctions and commonalities of the subject. What is a polygon? It can have many angles. It is not limited to regular polygons. Can a polygon include obtuse angles? In my opinion we want to include relationships between our generalizations and the relationships between a family would typically extend beyond the most obvious features of classification. I do not think a generalization system for AI could avoid using familial relationships.
I made up the term familial generalization because I could not find a good alternative to use.
But it can be useful to create a generalization comprised of different kinds of things without any obvious resemblances other than their inclusion in the generalization.
I found a definition for generalization as used in Machine Learning from a website called, magimind.com. “In Machine Learning the term ‘generalization’ refers to the model’s capability to adapt and react properly to previously unseen, new data, which has been drawn from the same distribution as the one used to build the model.”
Machine learning has a definition of generalization based on clusters or relative positions of categorical objects onto a space. In order to make this possible the concept points have to be deposited onto the space using some consistent measures. This could be extended to conceptual relations but in order to do so the concepts have to be fixed onto space in a way (such as using vectors) that have not worked well in the past. If the instances of the data objects are dependent on a few simple concepts they can be fit onto some consistently measured space, like the data plots from an elementary course on economics. Learning can continue as new data points are added to the sample but there is a cost as the data clusters become more complicated. Overfitting and underfitting are issues related to generalization in machine learning. I do not believe that the forms of generalization that the human mind is capable of creating can be efficiently pinned down to a single consistent vector space. I think that it is necessary to rely on a more nuanced method to comprehend an extensive system of conceptual relationships. As I tried to point out, even if nature could be forced into a single taxonomic system, we would still want to examine similarities, differences and relationships across the branches of the taxonomy. This means that we would still need to use cross-generalizations and they would tend to be more familial. Human knowledge is imperfect and it just cannot all be forced into some elementary measurable space any more than it can all be fit into a single taxonomy. I believe that we use an extensive system of cross-generalizations and this extensive system is going to be combinatorially explosive to try to resolve using elementary measures of distance between concepts.
Generalizations can be combined and merged. They can be expanded or contracted and otherwise shaped. However, I have been talking about using a variety of different kinds of generalizations. This means that their combination, or the abstractions that might be derived from their combinations, have to be more sophisticated than some elementary process like list processing or using vector mathematics. I think that different kinds of generalizations would need to use typed relations to define how the concepts might be used. I believe concepts are essentially generalizations that are defined from other generalizations. Sure the definitions of a generalization would have to use elemental expression or data of ‘experiences’ but I think it is also useful to look at the relationships between the uses of a generalization. I am not arguing against any kind of elemental data but relations between concepts might also be typed to indicate how they are to be used. Typing can make data computations simpler and it can alert the system to unusual attempts of combination. For instance, an answer in inches is at least typed correctly for a question about length. The answer, “blue” is not typed appropriately for the question, “How high is this building?” On the other hand a utilization of the mistyped response could be contrived. “Blue,” might refer to the sky. In other words, the building is blue sky-high. The term skyscraper contains this sense. Heavily typing of relations between concepts would be helpful.
Generalizations can be formed with greater leaps of the imagination. For example, if you saw a large predator, like a tiger, attack, kill, and eat a larger mammal like a goat, you might well think about the possibility that the tiger might attack you as well. Considering this example I would say that most generalizations probably involve some application of imagination. We often dream when we sleep so our imagination is not limited to our waking daydreams and thoughts.
To summarize some of my thoughts with the recognition that concepts are generalizations:
The most natural generalization, the one that can be used in cross-generalization and combinations of generalization, is what I call the familial generalization. This is my own definition but it seems to be suggested by the definitions I did find, and I think it is more a more natural form of generalization.
I coined the term ‘universal generalization’ for generalizations where all the particulars share some features, but this is the standard definition of generalization.
A generalization can be broader than another generalization that covers the same group of particulars.
A generalization can also be vaguer than a relatively more tightly defined generalization.
An individual may belong to more than one generalization. This is how cross-generalizations are formed. We can think about multiple levels of generalizations over a set of particulars by merging details of abstractions or by combining the particulars with other generalizations that can meaningfully interact with them. In my opinion the combinations of generalizations and in particular the combination of cross-generalizations will tend to produce familial generalizations.
If generalizations include a variety of different methods for developing them, their use in combinations will need to be guided more systematically. The use of typed relations will be necessary.
By combining compartmentalization with generalization an AI program might be able to implement an efficient way to be aware of what it was doing, at least to some extent.
On the other hand, there are lots of complications with this theory about the extensive use of cross-generalizations, typing, and familial generalizations.
Overgeneralization, over-compartmentalization, and limited self-awareness occur so frequently that they are typical of human thought. I believe that these ‘flaws’ of human thinking are the excesses of the character of important components of the general intelligence that human beings are capable of.
Addendum:
Cross-generalization is useful. If you combine the elements of cross generalizations, which I think is a natural way of thinking, you will tend to produce familial generalizations. For example, I formed impressions about all the members of the group who were on a video conference that I was on; some of the members of the group who frequently participated in the discussions; some who participated once in a while. I will tend to merge these distinctions as I superimpose other characteristics of the participation - as being more or less related to my own special interests and as presenting something that caught my attention and so on. There is no way I can keep track of all the details of my impressions and I do not have much expectation that my impressions are going to be extremely accurate in the future. On the other hand I have formed some familial generalizations about the group. So in forming some generalizations about things that were discussed with the group, I lost a lot of details, and I forgot some details about who said what. Over time, I will lose more details. But because I know something about the topics that were discussed I can build on that as well as my memories of the discussions. And as I am motivated to learn something because of what was said, I can continue to build in new directions that will become related to the generalizations I created from the discussion.
However, there are other ways of combining generalizations, and I feel that if these other possible combinations were typed, much the way that words and word phrases can be typed in the study of Natural Language Programming, the complexity of the representations and understanding could be simplified.
I believe that mental representations are best understood as including or referencing generalizations. These generalizations can include fuzzy reasoning. Specifics would have to be included as well, but they should be analyzable using abstraction and generalization. Fuzzy reasoning is not limited to the implementation on neural networks and deep learning networks. Fuzzy reasoning will merge concepts but might also retain 'highlights' (for a lack of a better term) that can be distinguished if needed. As I tried to show, fuzzy reasoning does not necessarily have to rely on numerical evaluations of the weights of some thought.
If an AI program retained simplified abstractions of how concepts were formed, the history of the creation of the concept could be kept for a time in an efficient manner. It could then be revisited as new information was added to related concepts. Over time the history could be abstracted further and pared down, but as changes to the concepts are made the abstract of the formation of the concept can be retained for a period of time.