This depends on what you mean by incomplete, noisy, or constrained information. Additionally we need to split this into two halves; machine learning and neural systems.

Machine Learning

During training of the model, there can be several different things that could happen. But as a broad brush basically they all come as a result of the majority of ML models driving towards minimising errors/differences such that it reaches an average consensus.

Incomplete Data: If we define this as “not having enough data points, or not enough data dimensions” then the consequences are more than likely that it will “resort to the mean”. For example, if we are trying to predict a binary (True/False), but we have a small bias in our incomplete dataset. Essentially, the model will gradually work out if it always predicts the most likely situation that will reduce the amount of errors. Or to put it in a very simplistic way, if we don’t have enough data to create a good model, essentially the model will just converge to the average (mean, median, or mode – depending on how you train your model).

Noisy Data: If the data is noisy, then usually the best work around is to use more training data, and also do more training. But if that can’t be done then usually the same will happen as for incomplete data, i.e. it resorts to the average. However, if there is bias in the noise that can really affect the model during training (unless we control for this in the inputs). This is particularly true for classification models where if one input variable/dimension has more noise and is relied on more to detect a particular class then the model will try to aggressively pivot towards trying to identify that class more than the others.

Constrained Information: Models are only as good as the data they are trained on, so if the data is constrained (e.g. for one geographic region, or type of customer etc.) then its repeatable patterns are based on that world. When you then try to move the model to a different set of conditions then the model might falsely predict to a particular group, or if a certain value is no longer indicative, then the model may no longer converge but become more random/less accurate.

These cases are during model training/development, however during deployment there’s also the issue of model drift, where the data that the model was trained on is no longer representative of the world which it is predicting. In this case, the model may converge to a smaller pattern, or it could also be random, this would depend on what has changed.

Neural Systems

This is a bit more tricky as the vast majority of neural systems are designed to be generalising and infer what is going to happen. However, there have been some studies on sensory deprivation within animals (e.g. dark rearing) where depending on the type of visual test being used some aspects are remarkably resilient and emerge regardless, whereas others such as orientation tuning have different effects where mice are completely unable to learn but ferrets recover (in dark reared ferrets/mice – Fitzwilliam vs Bonhoeffer labs).

But sensory neural systems are a bit different, often alongside genetically encoded wiring, and environmental cues, they also develop with the help of spontaneous activity which has a very large role in fine tuning the system (often more so than actual sensory experience). If this spontaneous activity is altered then broadly the system because less accurate and specific (e.g. visual acuity, or olfactory segregation decreases).

I’d argue that very often these systems don’t truly “understand” the meaning of what they are detecting. They are often optimised either for minimising or maximising a value, or a biochemical reward. I’d argue this is the case because “understanding” typically involves understanding “why” which typically extends beyond simple correlation/causality.

In short the majority of machine learning and deep learning models (and especially LLMs) are phenomenological models (describe how things move together) rather than mechanistic models (which indicate why).

The closest we get to this are techniques in data science such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Partial Least Squares (PLS). Where we find links and associations between variables and can infer meaning, but we never mathematically understand.

Very common, if we look at any deep learning even for something as simple as the MNIST dataset (single digit handwritten decimals) if we train a model beyond the first few layers the level of complexity is beyond what we will be able to understand (short of saying it looks at edges).

However, we can use techniques such as feature importance to determine what part of the inputs are most informative. However, even then what the models mathematically do with them is often hard to understand.

This can have very serious implications, for example in high frequency trading often these models are given as little data as possible in order for them to calculate trends quickly, and also they do not factor in the presence of other “bots”. Therefore, for example, if there is a small downturn that indicates “Sell” it can spark a massive predatory crash in the value of a stock despite the underlying circumstances not changing.

That depends, it is possible. However, I’d caution that highly-stable is not necessarily a good thing as it may just be stably incorrect.

Regarding small or partial datasets. There is something to be said of delivering datasets with too many input variables such that it can confuse the model, take far longer to train, and potentially cost more to inference.

A classic example from nature would be flocks of birds or shoals of fish, there is no central boss bird or fish orchestrating their patterns, instead each animal receives information about their neighbours position and this creates a coherent, stable, macro-pattern.

However the real trick is finding the right local constraints – not something that is particularly easy to do without a good understanding.

Yes, but depending on what we mean by understanding again. Essentially the vast majority of ML and AI systems (and even some neural systems) focus on looking at associative patterns (or correlation not causality).

In these cases it clearly doesn’t understand the process but it knows enough to make an accurate prediction.

A trivial example would be the associative patterns we learn as children, I was taught “Red sky at night, shepherd’s delight, red sky in morning, devil’s warning” meteorologically this is true and there is an indication for good weather if the sky is red at night. But I personally have no contextual understanding of the underlying process.

I don’t think I can put a fixed number on it but the broad answer would be a significant portion.

For example, in ball sports such as baseball and cricket the ball is moving too fast for the batter to react based on their direct observation of the ball in flight. Instead, often they are making predictions based on the seam position, the body language of the pitcher/bowler and previous pitches/balls.

In short, we are a predictive engine most of the time. (Another example where we control for this is reaction times in the 100m sprint, where any reaction under 0.1ms is deemed as predictive and so the runner is disqualified, even though they didn’t start before the gun went off)

In addition to that predictive nature, human perception can fill in a lot of gaps based on previous experience and so on. However, that does merge into memory storage as well as perception which would make this much harder to answer.