00:00
Introduction
George introduces the concept of Cultural Language Models (CLMs).
00:23
The Bias in LLMs
Discussing the inherent Western bias in current Large Language Models.
01:01
Alpha Earth: A VLM Example
Introducing Alpha Earth as a Visual Language Model by DeepMind.
01:46
Beyond Text: Epistemologies
Discussing the limits of text-based language and diverse epistemologies.
02:08
Artificial Cultural Intelligence
Referencing the 'Artificial Cultural Intelligence' article.
02:33
Intelligence at the Edge
Discussing a research paper connecting AI to indigenous knowledge systems.
03:14
Tupaia's Map
Illustrating knowledge systems with Tupaiya's Pacific map.
04:35
The Case for CLMs
Proposing CLMs trained on cultural elements beyond text.
05:12
Cultural Data Inputs
Exploring various cultural data inputs for training CLMs.
06:09
Research Validation
Early research shows cultural datasets boost cross-cultural understanding.
08:36
Semantic Caching
Connecting semantic caching with the need for Cultural Language Models.
09:28
Final Thoughts
Concluding remarks on CLMs and aligning AI with humans.
Summary
CopyDetected: English
Copy transcript
00:00
What's up, everybody?
00:01
George Sears.
00:01
Samuels here, AKA the Digital Wayfinder.
00:04
And today, I'm just gonna go through a series of prompts that I went through about the potential for a CLM over LLMs.
00:16
And what is a CLM?
00:17
So if we look here, I'm calling a CLM a cultural language model.
00:22
Maybe it might be something else.
00:23
But one thing I've noticed in the growth of AI, LLMs in general, is that they tend to have an inherent bias, which is I I'd expect as much, based off of the datasets in which they are trained.
00:42
And as a result of that, the further away you get from The US, the less people tend to trust or align with the results that AI is producing.
00:51
And once again, this shows the inherent Western bias that is in very popular tools or platforms like ChachiPT, Cloud, etcetera.
01:01
And it wasn't too long ago that I actually heard about, Alpha Earth.
01:07
And Alpha Earth was essentially or is essentially a product from DeepMind, and I'm gonna try and let that load.
01:18
And here it is.
01:20
So it's a product that is essentially a VLM or visual language model.
01:25
And Alpha Earth needed a different type of AI that was able to use all of its satellite imagery as its knowledge base, and dataset, and interpret the world visually through imagery as opposed to just through text, which can also be quite limiting.
01:45
Right?
01:46
Even though language or text based language is powerful, there are limits when it comes to different epistemologies.
01:56
And so as a result, what we're seeing is this sort of natural split between the different ways of seeing or knowing the world.
02:08
And I actually covered this in this article that I published in 2024 on Hacker News or that got published, and it's called Artificial Cultural Intelligence, a case for it.
02:21
K?
02:22
And here, I was inspired by this article.
02:29
Well, it's actually a research paper, and you can see here.
02:31
This was a research paper.
02:33
Alright?
02:34
And it was called the intelligence intelligence at the edge of chaos.
02:38
And as I was reading it, I noticed that they were talking about different classes of reasonings of reasoning that exist and how AI is sitting somewhere in between a class two or three in terms of behavior.
02:53
Okay?
02:54
But there are aspirations to get to a class four.
02:57
And what I realized was that the class four level of complexity when it comes to reasoning that it's trying to achieve sounded very similar to indigenous knowledge systems.
03:09
And this is a bit surprising, but also not if you actually look deeply into it.
03:14
Right?
03:15
There's this famous map of the Pacific by Tupaiya, who was a Tahitian navigator that served under James Cook.
03:25
And what was fascinating is that they had all of these islands, right, mapped out by Tupaiya that James Cook and his crew just couldn't understand because the way that they were mapping the world was based on their knowledge system.
03:39
Right?
03:39
But the way that Py was mapping out was very different.
03:43
And so I think what happens is when we don't understand the underlying ways of seeing the world, we tend to misinterpret a lot of things.
03:53
And you see this occur all the time when people argue.
03:56
Right?
03:56
If you don't understand what the underlying belief system or way of seeing the world, right, worldview is of that person, you could be arguing over semantics when it's kind of counterproductive because you're not on the same playing field.
04:14
Right?
04:15
And this, of course, makes the case for trying to standardize everything, but we've seen the downsides of extreme standardization.
04:25
Right?
04:26
You just cannot account for the nuances in different knowledge systems and the benefits that come through knowledge from different knowledge systems.
04:35
Alright?
04:35
So going back here, I asked if VLMs are a thing because they're fundamentally different to LLMs, and if language isn't the only thing that makes up culture right here, is it possible to potentially create a CLM or cultural language model that is trained purely on cultural elements over pure text based language?
04:54
And, of course, I included Alpha Earth.
04:56
And then this was what I got in return.
04:59
Now the AI I'm using right now is from the browser.
05:02
Alright?
05:03
If you wanna learn more about that, just comment in the video below.
05:07
But here, yes, building a cultural language model is technically feasible.
05:12
It would extend the multimodal ideas behind vision language models, VLMs, by anchoring training not just in text, but in images, audio, ritual motion capture, geotemporal, data, symbols, and even knowledge graphs.
05:25
So have a look at this.
05:27
So images, audio.
05:28
We know that most of the AI systems out, the popular ones, are covering images and audio, although VLMs are specifically trained on the images side.
05:37
Right?
05:38
We have audio, but I haven't heard of an audio language model yet, but that would be interesting.
05:44
Then we've got ritual motion capture.
05:46
This I find fascinating because this is more on the video side, but ritual motions is in an interesting dataset.
05:53
Right?
05:53
You got geotechnporal data.
05:55
You got symbols and then knowledge graphs, right, that encode values and social norms.
06:02
And it's this knowledge graph that include values and social norms that actually fascinate me the most.
06:09
So early research shows that culture VLM well, it already shows that fine tuning on explicitly cultural datasets measurably boosts a model cross cultural understanding.
06:20
And so if you click here, we go to this research paper, and it looks like as of January 2025, we have some scientists who are actually looking into this aspect that I spoke about or touched on in 2024 under my concept for artificial culture intelligence.
06:42
And so I'm definitely keen on seeing what comes out of this and if more and more people are going to try and tap into it.
06:51
Who knows?
06:51
I might even find myself, going into that direction too.
06:56
But as you can see here, from LLMs to VLMs and beyond, large language models excel at statistical regularities and text, but inherent the cultural biases of the dominant languages on the web.
07:07
Vision language models add pixels to words, letting a single network reason over images and captions.
07:13
The result is richer grounding and fewer hallucinations about the physical world.
07:17
Alpha Earth demonstrates the same principle at planetary scale.
07:21
Metadata to create a single embedding that represents Earth's surface more accurately than any one modality alone.
07:27
So these successes suggest we can swap vision for culture as an extra modality.
07:31
K?
07:32
So culture is more than a language.
07:34
It is the shared pattern of symbols, rituals, artifacts, values, and histories that bind a group.
07:39
A CLM would therefore need to absorb visual cues, auditory, kinesthetic patterns, temporal spatial context, right, holidays, agricultural cycles, probably time cycles as well if you look at other ancient cultures.
07:53
They had different ways of measuring the time.
07:55
I had another article where I spoke about Mayan time or spiral time versus mechanical time, which is what most of us kind of run our lives on, and then value systems.
08:07
K?
08:07
So by embedding all these in a shared representation space, the model could answer, show me how Maori weaving patterns influence modern New Zealand branding or generate culturally faithful marketing copy without slipping into generic Western metaphors.
08:21
And so I think here, in this sort of context, we would actually need let's see here.
08:30
I'm gonna see if I can get to the tweets that I was just looking at right here.
08:36
So you see here.
08:37
And then this is how I capture signals.
08:39
Okay?
08:40
Right?
08:40
So every repeated LLM call is money on fire.
08:44
Traditional cache don't stop it or can't stop it unless the prompt is an exact match.
08:48
So this guy is talking about a tool called Redis eight, just change the game, with semantic caching that understands meaning, not just keys.
08:58
Okay?
08:59
So this is important in my opinion.
09:01
Now this is more technical, but to me, I see a connection between that meaning aspect and the need for something like a CLM.
09:10
Right, which is where you can see here value systems.
09:14
Right?
09:17
Kinesthetic patterns, all of this meaning will come through datasets on all of these other aspects, right, which are very, very specific.
09:27
K?
09:28
So I thought I'd share this with you because I think there's something here, and I just wanna document, the the journey or process with this.
09:37
And since, you know, I am working in this realm of aligning AI with humans, right, more so than just the automation, I think there's gonna be something here with the the cultural language models, or just the culture aspects in general.
09:54
So, yeah, stay tuned.
09:56
If you found it interesting or had any comments or feedback, feel free to just, share it in the comments below, and I'll include the rest of the text from this as well.
10:05
Alright?
10:06
Speak soon.
Made with Bullet