JP Bianchi@JP_B

LLM's have reached a point now where it's possible to query their billions of parameters and uproot information in a hierarchical structure, which is the structure of information itself, human knowledge and the physical world. Basically objects are made of objects etc But the features that humans use to describe our world, whether with language or drawing, are not the features that ML & AI algorithms use, which are merely data points, pixels on an image, or a vague assembly of such pixels. It is a fight against the infinite complexity of life and Nature itself. Humans use objects, (real) features, and in a hierarchical way: a cat is a head and a body. A head is eyes, whiskers, ears, An eye is a cornea and an iris... etc. Today algorithms process trillions of data points and 'only' produce a statistical result, not real features, and will always do so no matter how much data is used (unless they incorporate our approach). Humans, on the other side, can classify with certainty billions of different (visually) cats by verifying they tick a few boxes, ie the 'real life' features we all know make a cat (pointy ears, whiskers, fur, etc). Our approach deems to create a knowledge graph of such real life features (what we usually refer to more generally as 'objects'), which in turn, can be used to improve current algorithms' performance. For instance, it is easy to imagine how it allows to verify if all the proper features of an object are present in an image because it tells us exactly what we should be looking for, what matters. Therefore, it will improve, say, object recognition, with direct applications in robotics, AV, guided systems of all sorts, medical diagnostics and even real language understanding. There are other applications but one of them is to improve LLM's themselves, by reducing the training time and their size by incorporating our concept into new architectures to avoid having to re-learn the whole human knowledge every time.

Information is hierarchical in nature. Humans naturally see the world in terms of objects, made of objects, made of objects, but ML algorithms do not operate like that, and it is difficult for them to properly recognize objects, especially in a complex scene. GPT4V changes all that and can produce an exhaustive list of beliefs about the objects in an image, their relationship, but also the objective and conditions in which such relationship happens. E.g. a woman uses sunglasses to protect her eyes in bright daylight. GPT4 is then used to extract accurate fields of information from GPT4V-produced beliefs, such as the subject, object, action, objective and condition in which such action takes place. The results are quite impressive. The information is then sent to Neo4J to visualize it as a knowledge graph.