0:05[music] Okay. Hello. Hello everybody.
0:26I'll do it like this. So you might be uh you might be surprised to see me here. I'm not Zanap.
0:36Um so Zanap it turned out in the end through various complexities in family etc. uh couldn't make it. Uh and at the same time I heard some complaints from people to me and say why don't why don't you speak at at the typing and summer school. uh so we thought we we can address both issues in one and I'll give you I'll give you one talk here and uh it's going to be special not in terms of the content well at least I don't want
1:05to claim that but also since uh you will get not just me but an additional surprise visitor if everything works out uh in the talk so so don't fall asleep I'm using this as an excuse to motivate you for my talk don't fall asleep something interesting might happen um and the talk uh It's actually good that it's in the second week because it builds upon some things that you've already learned about um but it tries to put it in a broad perspective and uh and maybe also connected a little bit to
1:34scientific applications.
1:36[clears throat] So I've I've called it a tale of spirious correlations.
1:42So I think you've all seen this kind of uh examples. So this is an example where people trained a neural network to detect so-called numo thorax where the the lung collapses and um they built this system wanted to apply it to new incoming patients and it didn't work and uh they noticed that um it had worked very well in the training set didn't work on the incoming patients then they started to analyze what the system was looking at and then they noticed that it
2:10was looking for instance at this area here which is an area I don't know if you can see it where the doctors insert a little tube into the lung of the patients to let the liquid flow out. So this means pneumothorax patients they come in they get diagnosed they get their little uh tube inserted into the lung and then they take x-ray pictures and then they train on those x-ray pictures. Now clearly for the incoming patients uh they they wanted to do the diagnosis in the first place. So there
2:38was no tube in the lung. So we had a different distribution and uh uh so from that point of view the the tube appeared uh or it was correlated with the numothorax uh but it's not causal for it. So this correlation it appeared in the training set but not in the test set. It was not a robust correlation. So that's uh one example of something that can go wrong or one example of something that people call a spirious correlation. Now uh some
3:07people would argue or people the current uh scaling uh trend seems to suggest that we don't have to worry about all these things as long as we scale up our models far enough they're going to learn about causality magically. Um so for this reason I want to show you some images of where you still find spirious correlations in current foundation models and uh and this is an image of the Ebinghouse illusion. So if you look at these two orange discs then you
3:35[clears throat] will notice that the left disc looks a little bit bigger than the right disc. And in fact they are the same. So you can you can if you want you can measure with your fingers. Um but it's because they're surrounded by other discs. Now suppose you're a foundation model. You're trained on the internet.
3:52You're trained also on images and the images appear with captions and maybe you've seen some images like this alongside with a caption that explains the illusion. Now you could learn something from this. You could learn different things from it. For instance, you could learn whenever I have orange discs surrounded by blue discs, those orange discs are the same size. Now, that would be a stupid conclusion. It would be a spirious correlation because clearly it's not going to be robust. And if you test it on images like this and this is an example that two of our
4:22students have come up with. Test it on images like this then um if you fall for the spirits correlation then you will think these orange discs are the same size and in fact this is what language models tend to do. So here it says both orange circles are actually the same size and uh it even recognizes the correct illusion the ebing house illusion and uh and I found this a lot of fun and tried out multiple illusions.
4:48So you maybe you've seen this illusion so this length is the same as this length although they look different it's related to a certain prior in 3D perception or this is a straight line even though it doesn't look like it. Now let's exaggerate those illusions as well. So this is an extreme version where the two segments. So this segment and this segment are clearly not the same length. But again it says this is the Müller liar illusion. Those are the same length or here here it says this is actually a straight line. And uh finally
5:18this one is also grossly exaggerated. They're clearly not parallel. It says they are parallel. Um I showed it to someone last week and they were asking well you were using 40. What about the more recent model? So this morning I ran it on uh on Fable and he says, "Yeah, this looks like the cylinder illusion."
5:36And [clears throat] so what's actually on the image? Uh where does it say anyway? So I I don't want to read the whole thing, but it even gives some explanation what it feels like when you look at this, etc. But it still falls for this kind of uh uh spirious correlations.
5:58Now, spirit correlations, they sound like a bad thing, but uh if you model them, if you understand them, they can also be a good thing. And that's an example from astronomy. And I'm going to use this as an excuse to show you several examples from astronomy, several examples of machine learning in astronomy. Uh and maybe to just to connect this also to a little bit to what Jeb Batista was saying in the first week. Um so it's uh I agree it's non-trivial. If you think all of AI now
6:27is about who builds the best language model, then it's non-trivial to compete with companies that have millions of GPUs.
6:35But AI is much broader. AI is also about what are the future algorithms and AI currently and very excitingly is about what you can do in the sciences by using AI. So it's like in astronomy when the telescope was discovered you could get a telescope and make new discovery or when the telescope was around you could make lots of new astronomical discoveries just by applying the telescope and I think we're a little bit in the situation now. So here we're going to
7:04apply it uh to the problem of exoplanet search and uh we will do this on it's easier. Okay. So this is actually a good time for our surprise visit before I get into this topic and we're going to switch the screen now and have maybe 15 minutes uh of a conversation with our surprise guest. And uh let's see let's see if this works.
7:34Hello. Can you see me?
7:38We can hear you already. You now we can see you.
7:41So I I would like to introduce to the audience this is Artur Mench uh whom we had been in contact about whether he can join us for a little while here and the excuse for doing so was that he was he's an an MLSS Tubigan alumnus. He was here before sitting on your side of the room.
8:01um different building at the time. Uh so we're very happy that he's here now and he's one of the people uh really moving moving European AI also at the at the frontier. So Arta maybe maybe we can first start and and you can tell us a little bit about your experience when you were an MLSS student. What do what do you remember?
8:22Yeah. So good afternoon everyone and sorry to crash into the Milky Way. Uh that's it seems that I replaced that screen. Uh and so great pleasure to meet you all and sorry I'm also wearing a tie today because I have a few things to do uh this uh afternoon. Um I was at the machine learning summer school in the summer of 2017. Uh and so at the time I had to it was basically two months
8:48before the transformer release. uh and at the time uh we actually had a few uh a few sessions already on deep learning and on machine on natural language processing with transformers not with transformers but with attention so attention already existed at the time uh and I also remember quite a few lessons on causality which is not something I explored enough I would say and so for me it has uh it's been a great
9:18experience to be to bingen in particular I remember a lot of very nice bars that you can go to. Surprising amount of bars that you can find in such a tiny city I should say and um and it's been a great place also to meet a lot of people that are still in the field um and that uh I think I've been at the forefront of different progress in uh in research on the transformers in particular on uh on
9:46scaling all of these things. Uh and today uh well I guess not I'm fortunately no longer a researcher but um but I I do have very fond memories of the time when I was a researcher.
10:00So so maybe we can talk a little bit more about what happened to you afterwards. You can it would be interesting to to understand when when did you notice you want to build a company that uses transformers? Uh what what did you do in between? How did the thought mature to found something in France? Yeah. So I was a third year PhD student at the time. So in 2017 uh and I was working on functional machine uh functional magnetic resonance imaging
10:26for uh to to to do a prediction of what uh people would think thanks to brain images. Uh and using that as a way to understand how the brain was working um and at the time I was doing a little bit of deep learning in that domain. It was a bit early. Uh was mostly factorizing matrices. So it was not that nonlinear h and I did then uh two years of postto uh in mathematics and optimal transport and
10:54uh and then I that's the time when I got into uh natural language processing um and that uh brought me to deep mind in 2020 that was the year of uh GP3 uh and uh deep deep mind was a bit lagging behind and so we were in the small team that was kind of pushing that idea that it was maybe a good path to make very strong intelligence systems.
11:19Uh which we did uh and spent two years and a half there. Uh worked on the scaling of these things. So making sure that when you put more training flops to the problem, you make better models. And I think that has been the name of the game ever since. uh and in 2023 uh I decided to create uh Mistral which is [snorts] uh because we knew that it was too important to technology to leave it only to um uh US uh the US industry
11:48uh and that's how we got started. So uh that was three three years and a half ago now it feels like yesterday because uh it's been quite uh packed with events ever since. uh and now we uh were a,200 people company and operating many different places and we have a 250 people research team uh and I'm no longer part of it but they work very well.
12:12So it's very interesting you you're telling uh you were part of a small team at deep mind that started this in there.
12:18So deep mind was previously or originally known for a different kind of approach to intelligence that was more about building it from the ground up trying to understand perception and action and not immediately language. So maybe that was a more sort of neuroscience inspired approach. So how did it how did it feel you were like almost an adversary team in the company or did you have to argue for this or it was just an experiment or how did it
12:47feel to be doing that?
12:49We were a 10 people team at the time. Uh and I think it grew over time because people realized it was quite an important opportunity.
12:57Uh deep mind has always been pretty heavy on reinforcement learning. Uh and at the time it was on on actual talk problems. Um the good thing with language is that it's actually a very good shortcut because it's it's already a pretty big compression of human knowledge in itself. Uh and if you take it and you compress it further, you get a lot of intelligence. Whereas if you only rely on signals coming from the human world, they are not very compressed and so you need to spend a lot of comput them. Um so that I think that's the shortcut that was a bit
13:25missing in 2020. Uh we pushed it and then in 2023 the two things were not connected together. So you would train a large language model, you would not be using reinforcement learning at the time because you were you were doing pre-training and then then little bit of alignment and uh supervised fine tuning.
13:43But reinforcement learning was not done at scale. uh I think that changed actually the two uh the two branches of machine learning merged uh 2024 and we were we have been able to make uh language models uh much better by training them on reinforcement learning environments ever since. Um [snorts] at the end of the day I think the reason why it uh proved to be a great enabler is that uh it was a shortcut to human
14:12knowledge. you got access to everything that was ever produced in term of human content. Not everything but a lot of things and uh by compressing it you actually get a fairly good world model. Uh and once you have a good world model becomes much easier to do reinforcement reinforcement learning on tasks uh and on language tasks in particular.
14:31Yeah. Yeah. So, so when you when you started me start, so it was you and a few other people, I think some of them also from fair and from the llama team and uh so so you had I guess you needed a certain set of ingredients to to have a chance of being successful and then uh did you found these ingredients in Paris or can you just tell us a little bit more about how that how that Paris? Yeah. So Paris and London, I think the biggest ingredient when you
14:59start when you make a startup in AI or in anything deep tech in general is you need a you need a critical mass in talent. Uh and we were able to hire around 15 people uh that knew how to train models uh because they went through various labs where they got that experience. And so that allowed us to to be very quick in releasing things that were interesting. And we had a very good sense uh day one of what we wanted to
15:28ship to show that at the end open source models could actually win. Uh and at the time that was not the narrative in place. So uh the true ingredient was uh talent based and there I was also very fortunate to have I mean I have two co-founders that are great scientists and that could hire um uh people uh from their teams from from our network generally that were willing to engage in such a project. uh Simote and Gom uh who are still reading research and
15:55engineering at this time. Uh and uh that was the first ingredient. Then the the second ingredient is compute and if you want to have compute you need capital and capital we were able to get uh at the time thanks to I think charg actually because in 2022 I don't think we would have been able to raise in 2023 you started to have venture capital getting attracted to the domain uh and people realizing investors realizing that this was to be a category uh and so
16:25we could raise a fair amount of money uh as a seed round in Europe [snorts] uh and never seen since uh we have been raising more because you need need more compute because of the infamous scaling laws um and uh overall I've been trying to saturate the how capital market could support uh our growth though so we actually just announced a fund raise uh today but really the the true bottleneck used to be it's mostly talent and people interested in solving new problems uh
16:55and interested in doing it in Europe in particular uh and then it's about the enablers having the right capital and the right amount of compute. [snorts] Capital is not enough to get compute today because you the problem is you are in a supply chain crisis and so I think today to build an AI company uh the biggest problem is not only capital it's also securing supply uh which I expect to remain the case for the next two years.
17:20Maybe we can also hear a bit your opinion on where things are moving uh and about the future of AI. So we've had a bunch of panels here in last week in the summer school. I remember one with GB Paris Gandal open AI and one with Dennis Xiao from META and both of them of course heavily of invested in in reasoning. Now in you you talked before about uh how about distilling human
17:49knowledge from from large training sets and how this can be kind of a shortcut I'm paraphrasing shortcut to intelligence now and I think we've discussed with Jatista and with Denny also this question as to whether this shortcut is enough uh or whether we also need some some more direct access to the world interactions with the world I guess with Gentic AI we're moving in this direction now. Where do you stand
18:17about uh where we are on our path towards reaching human level intelligence? Uh what will take us there? Is it just about scaling what we have now or what are the crucial ingredients?
18:30Well, I think on the reasoning side you can still do a lot of things by scaling and by acquiring more data. So it's not only about scaling, it's about getting the right data of interaction, people interacting with tools. uh if you have environments with different kind of tools, you can make your model master those tools. Uh and I think that's true for almost any kind of computer tool that uh you may come with come uh that you may uh be using. Uh some of them are harder than others because some of them
18:58are linked to complex representation of the world. So if you want to master a 3D tool for instance or CAD all these kind of things, you actually need to make a specific investment. I think a big underinvested part of the world is AI for physics and for quantitative understanding of the world. Um where you want models of partial differential equations. Uh and we strongly believe in connecting those models uh with reasoning models because at the end
19:26reasoning models are somewhat mimicking how we think. Uh but when we think um the we the way we think is already a fairly pretty big compression of the world already. Uh if you want to get into the quantitative space again which you need to if you want to invent new tools if you want AI to truly provide growth and technological innovation you need models that have a quantitative understanding of the world. Um and the connections needs not to be doesn't need
19:55to be a single model. You can have a model that is doing reasoning and models that are uh good simulators of the quantitative world. As long as you make the right connection and potentially you train them jointly uh you can make uh models that can be on bodied uh that can be deployed in the physical world that can help with engineering design uh and eventually have a bigger impact on the physical world than they have today because today AI systems they really have an impact on the digital space. Uh
20:23so they help with knowledge work, they help with coding as you of course know.
20:28Uh but they don't help much yet on building better planes or building better machines for the semi-industry or unlocking truly engineering problems. Uh and [snorts] I think the bottleneck there is really the connection uh in between the reasoning space and the quantitative space. And that's a part where we are investing quite heavily because our market is very much manufacturing uh and the the companies we serve are manufacturing companies.
20:54And so when you ask them for their the use cases they want to solve with AI they want to do chatbots or HR they actually or or even coding they actually want to solve faster engineering faster design faster industrialization. And if you want to make that happen uh you need investments also in the quantit quantitative space.
21:14Thank you. Maybe maybe we can conclude with a question where I I will try to imperfectly channel some of the questions that I sensed from the audience when we when we talked to your predecessors GB and Denny. I think there there was this so so GB he had this bold statement that every PhD student now should think about the possibility that in three or five years uh their PhD would just be a prompt. So, so that's
21:44one thing that would be interesting to hear from you. What what what advice can you give to PhD student nowadays in choosing their topics? And I think that was the second more pragmatic question that people didn't maybe express so explicitly but but lots of people were interested in how does one secure an internship in a frontier AI lab.
22:04I can help with that. Um on the first point I think is uh a little wrong.
22:12The true value of AI systems is to be enabler of thank you. [laughter] Uh and so um uh if you look at how um uh our team is working uh they don't do end to end automation of their work. Uh they actually iterate on on their experiments thanks to AI of course but uh you can't really delegate the thinking. Uh you at some point you need human to understand what's going on. Uh because otherwise you're just doing
22:41science that is going to be completely useless. uh and I think it's it's a true problem of the systems today. They can solve problems. So for in mathematics they can solve a lot of existing problems. They can't really come up with new ideas. Uh they will they will [snorts] not enlighten you on an entire space of mathematics that you have been ignoring uh because they go into the training space uh fundamentally. So um the question is uh and I think the the frontier of going out of the training
23:10space is still very humanowned. Uh and maybe that may change over time but I think as long as you can um use AI systems to accelerate your PhD uh and I think you you'll be in a good shape in three years. Uh but of course it requires using it all the time and uh and also as you're probably all working on AI uh making it better. So there's a lot of contributions I think that can be
23:39made uh today and and honestly I think in three years from now we'll we'll all look at uh this craziness and realize that uh actually we [snorts] we have accelerated things but we haven't replaced humans. I think uh I think it's a too bold a statement to make. uh and then internships. We have internships um for PhDs and so uh I'll make sure that uh you can send email to the right people so that uh they can consider if
24:07that's ever something of interest. We do have uh work and offices in Munich, Zurich and uh and Paris and London uh and Amsterdam. So so plenty of places to do research in uh if you're interested.
24:23Okay. Thank you very much for making time in your busy schedule today. I I know you have some important meetings today. So we were hoping until the last moment that this works out. Thank you for making it.
24:33You're welcome. Thank you. And uh thank you for organizing this machine learning summer school for for the last 10 years.
24:41Thank you very So, so this was not scripted but it turned out exactly the way we were hoping. And I think I think at the end I also like this I have to say I like this conclusion that it's not really this dichotomy regarding the PhD what you can do versus what AI can do. Your competition is not AI but it's the question of what can you
25:09do with AI? you can do a much stronger PhD than than we could do years ago. So on this uh positive note, let's go back into what we can do with AI and science.
25:21So exoplanets. So how to use spurious correlation. So there's this uh satellite and I like to show show the name because Kepler is a guy who studied in Tingan. He went to the to the priest seminary to the school there at the time. This was where you got your education. um uh Johannes Kepler and this satellite that was named after him when the satellite was uh the orbiting the sun for a while several years observing a patch of Milky Way looking
25:49for exoplanet transits. So a transit means uh there's a star far away from here. Uh this star has planets. They're called exoplanets because they don't orbit around the sun. And uh if we're lucky then every so often such a planet can olude part of the star. And this is a small signal. So uh people are much more successful if they observed this from space which is why Kepler is a space was a space mission. Uh so this is
26:16the telescope and uh Kepler was observing hundreds of thousands of stars and uh so let's let's call the star Q and our observation Y. Okay. So clearly observation is influenced both by the star and by the telescope. And then we see light curves of brightness. So we don't see the planet is too small. But we see these changes of brightness of the star. And all these all these pixels uh so all these curves correspond to one pixel. These are neighboring pixels. And
26:45you can see some of them get fainter, some of them get brighter, which is a bit strange. But uh the reason is that the telescope it tries to look exactly at the same place but it's hard to perfectly align it cuz it's orbiting sun and the solar radiation pressure changes and so on. So the so the light might move a little bit around on the CCD. Now that's bad because the signal that we're looking for is much much smaller.
27:08However, as I said, we're looking at lots of other stars at the same time. And if you look at the light curves of the other stars, some of them look surprisingly similar to the ones of our star of interest. Now the stars themselves, they have nothing to do with each other. They can't communicate. They can't synchronize their light curves. They they're million they're light years or hundreds of light years apart from each other.
27:33So the only possible explanation for this similarities is of course that they're measured with the same telescope. [snorts] and and this information that they share is exactly the kind of im information that we don't want in our data. That's the information that we want to uh remove. This is the spurious correlation. All these correlations are spurious. Astronomers also call them systematic errors here because systematic they are they come from the instrument.
27:59Now uh this led to a very simple idea which is surprisingly powerful. uh the idea is let's try to explain this signal as best as we can from lots of other signals of this type thousands of other such signals. So it's a machine learning approach. We try to predict uh we actually compute the conditional expectation of this quantity given this in other words the regression and then we can ask uh the question what this regression. So this is the regression E
28:27of Y given X. Uh what does this have to do uh with the uh with the true signal?
28:34So we we compute this regression. We subtract it from the quantity that we're measuring. And it turns out that subject to certain assumptions when there is an additivity assumption for the noise but the noise can act nonlinearly subject to certain assumptions uh we can reconstruct uh the unknown quantity Q that's the astrophysical signal up to a constant offset uh with this estimator and this equality is what statisticians called
29:03almost surely so that's an equality of random variable that means you you draw on both sides and the probability that you at a different number is zero.
29:12[snorts] So that's a a strong statement subject to reasonably strong assumptions which can be weakened and then we get a slightly weaker statement. Um without going into great detail now uh I want to say that we we use this to build a search pipeline. So search is more than just removing the systematics. You also have to look for periodic uh transits etc. So if a signal is periodic, of course you have much more evidence than
29:39if you only see it once. And um then we published a list of exoplanet candidates, maybe 30 of them or so. And then uh other people did other measurements using other methods and uh around 2/3 or 3/4 of them were confirmed as exoplanets. And one of them was especially interesting. It was this one. It got the name K218b.
30:05interesting because it was um what they call a habitable zone exoplanet. So habitable zone means it has a distance to its host star which allows for liquid water. So the dark green area allows for liquid water. Um so then some this was it turned out uh this was the first such sorry it it was not the first habitable zone exoplanet but a few years later it was the first habitable zone exoplanet where people found water using
30:33spectroscopy on the atmosphere of those exoplanets. Um and then it got quite famous. I first didn't connect the dots because I had not remember that K218B was one of ours.
30:44But then the collaborator, our astronomy collaborator from New York called me and said you know this is actually our exoplanet and uh it was nice uh there was some discussion but the astronomers immediately said well this is not habitable because for one thing the exoplanet was quite close to its host star which tends to cause planets to be tightly locked. So in Mercury, the closest planet to our sun is tightly locked to the sun, which means it's
31:12always showing the same side to the sun and and that's not so friendly for life because then you have one side that's very hot and the other one is cold. Um but then a few years later the new planet was in the news again and uh and astronomers were starting to speculate um that actually it might have a lot of water so much that it's is covered by a water ocean uh and then above that a hydrogen-rich atmosphere and they called it a hyishen planet for hydrogen and
31:41ocean. Uh that was kind of nice because then if you you could imagine if you have a you have an ocean then maybe there's some mixing of warm and cold and at the boundary where you have eternal sunset or eternal sunrise maybe you have the right temperatures. Um and then uh the same group in Cambridge uh a few years later uh discovered uh methane and carbon dioxide in the atmosphere.
32:08And uh that was also nice because of course we have a lot of methane and carbon dioxide in our atmosphere. And and then in last year they published this paper where they uh claimed to have found uh two chemical compounds. One of them is uh dimethyl sulfide uh which on earth is produced in significant quantities by plankton. So so it was kind of a nice story. Of course then lots of other astronomers said they they
32:37don't buy it. And it's quite possible that this is not true because this is this is a very difficult inference problem. So you need basically a generative model of an exoplanet atmosphere and then this generative model tells you what spectral lines to expect and then you have to do inverse inference from the measured spectra to the parameter in the generative model and uh the measurements are extremely noisy and the generative models are pretty bad. So you can you can imagine
33:06this is a difficult problem but it's very exciting nowadays that you can use machine learning and contribute to something that it's something scientifically real uh about the world.
33:16So I find this very rewarding and we've also worked in other domains of exoplanet search. So this was now what's called transit photometry where you have a little uh dip in the brightness curve but there's also there's this area called radial velocity. Uh this is using the Doppler effect. So if you have a planet and a star and they're orbiting where the planet orbits the star but really they both orbit the the common center of mass then the star is also moving and when the star moves away from
33:46us it looks more red when it moves closer to us it looks more blue a little bit like the sound the Doppler effect when a police car drives away from you. Um so people can measure this this so-called Doppler shift uh also and uh and and actually the very first exoplanets were found using this method. And then the last method which is also very or maybe specifically interesting.
34:08[snorts] Um now this this method here the first one has the problem that it only works if we happen to look at the system exactly from the side. So um statistically u the chance to spot our planet passing through the disk of our earth assuming random random randomly uniform distribution sperically somewhere else is about half a percent.
34:33So it's unlikely that this method is going to work for the closest exoplanets to us. And in a way if maybe we will never travel there, but if we ever send something to our closest neighbors, we will probably not have found them using transit photometry. So there are reasons to believe that in those cases, we will have to find them using direct imaging. And of course, direct imaging is only ever possible if they're very close because uh it requires a certain angular separation between the star and the
35:02planet. and and the stars are very bright etc. So this is a difficult field but in principle you can use the same methods. So just to explain why this is so difficult it's it's the the contrast and the angular separation it's a little bit like trying to spot a firefly on a lighthouse. Um but we started working with that also and then the first success uh also using this method which we call half sampling regression. First success is that we managed to do a
35:31pre-overy which is a term in astronomy. It means you discover something that's already known but you discover it from data that is that predates the data that was recorded to find it. So we found something known in archival data and then more recently we were part of a study that found better D uh which I think is the faintest planet so far ever directly imaged from earth which has also caused quite some excitement in the
36:02astronomy community.
36:04I also want to show you this video. I think some of you went to the uh tutorial of Max Ducks but probably not all of you. So I I just briefly want to show you this this application area which is also related to the kind of inverse modeling that I previously mentioned and this is gravitational wave inference. So this is a a video of two black holes simulated from nearby that orbit each other and merge. Now why do
36:33they merge? I mean normally orbits can last for a long time. So earth has been orbiting sun for a long time and we we hope it doesn't end anytime soon. But if you have very very heavy things that orbit each other um [clears throat] and if they orbit each other fast. So these black holes they tend to be heavier at least twice as heavy as the sun could be 100 times as heavy. So they are that's a lot of mass. And if they orbit each other like this and eventually before they merge
37:02they might even orbit each other 100 times a second. that is an extreme acceleration of a large mass and Einstein found out or it's a consequence of gra of of gravity or the theory of general relativity that accelerated masses emit energy in the form of gravitational waves and if they emit energy the energy has to come from somewhere else. So they lose orbital energy and they eventually collapse into one black hole.
37:29Now that's an extreme event and they can only astronomers can measure it because they have these uh uh gravitational wave observatories that use interpherometry.
37:39So they send light back and forth distances of kilometers and then they find tiny changes in the spaceime. So tiny changes in distance itself you can say and uh the changes are so fast that they are on the order of fractions of the diameter of the nucleus of an atom uh over the over kilometers. So it's amazing that this is possible and then the idea in this kind of u inverse inference um based on simulations. So this is called simulation based
38:09inference is that you uh you need a simulator that can generate forward data. So you simulate some black hole systems in some place in the sky, certain masses, orientations. You compute how the waveforms should look, the waveforms that we measure with our instruments. And then given an actual waveform, you want to do the inverse inference. And if you can simulate many forward points, you can train the backward model as a machine learning model. And you can you can train a a
38:38normalizing flow or some whatever is your favorite method to estimate a conditional distribution in the backward direction. And uh this is interesting uh partly because it's it can be done very fast and very fast is especially interesting if you are not studying black holes but neutron stars. So black holes they don't emit light by definition but neutron stars do. So sometimes the two things that orbit each other can be neutron stars. And if two neutron stars
39:07collapse into each other, then you want to be able to immediately point an optical telescope. So, so far people haven't been able to do this fast, the fastest such case, I think it took them 11 hours or something like this to point their telescopes afterwards and then maybe much of the action was already over. And this action is called a kilonova when two neutron stars merge.
39:29So, neutron stars are almost as fascinating as black holes. They are born when a star collapses and the matter sort of becomes as dense as possible as dense as possible before forming this singularity. So so here we have two neutron stars that orbit each other faster and faster.
39:49They do emit light uh and this is the now the kilonova neutron stars emit light not through nuclear fusion because a neutron star is already maximally fused. It's basically just one atom or one nucleus of an atom all the consisting only of neutrons. Uh but they still emit energy through various other processes not nuclear fusion. Um and then in in in such a kilonova uh people
40:18are starting to understand kilonova processes it for instance they believe that um most of the heavy elements that we find such as silver gold platinum paladium etc are actually generated in kilonova events. So if you have a gold chain most of it is probably from a kilon noa that took place somewhere in the universe maybe millions of years ago.
40:42So this is uh quite it's a lot of fun to work on these kind of things and now we're we're collaborating with astronomers to to find ways of then uh given such uh detections or uh conditional probabilities uh of sky positions hard to do fast follow-up observations.
41:04Here's another example uh of doing a form of machine learning in astronomy and this this was also fun. This was u a way the idea was we have lots of astron astronomical images on the web and we want to because [clears throat] lots of people like to take images uh but they don't really do it for science and they don't calibrate them properly. Um but there should be information because there are so many of them. So we built this pipeline where we we first just searched for an object in this case it's
41:33a galaxy called NGC5907.
41:36NGC's new general catalog and then we get lots of such images and then next we do something which called astrometry. So we automatically uh find stars recognize these stars as different images and align them and this is using a method that was actually pioneered by a machine learning guy Sam Roise. Some of you might remember him. I think he was a postto with Jeff Hinton back in the days or uh and uh so we use this method and then
42:05afterwards we have um sort of different patches of sky covered by different images and we want to compute one overall consensus image. Now people have processed these images and we can't rely on the brightnesses but we make this assumption that we can still rely on the brightness ranks. So we'll assume whatever people have done to their images if pixel A is brighter than pixel B then this should tell us something about reality.
42:30So now we have this uh lots of information about sort of partial orders of brightness of pixels and then we uh we came up with an algorithm that computes an overall consensus ranking of the whole sky that's covered by these images. And uh of course we don't have the absolute information but at least we have the relative rank information something which uh is like an image that is obtained by what people call histogram equalization.
42:59And here you see the algorithm as it as it runs over uh images and forms the consensus image. Then this is our uh uh histogram equalized image at the end.
43:10And this is if we tone map it. Uh so we do the scaling such that it's comparable to this image and what's interesting about this image is this little faint thing here. So that's why we picked NTZ 5907 uh because this galaxy uh had recently become famous uh cuz people found these things that that are called tidal streams. So these are leftovers from galaxies uh colliding or passing through
43:40each other. And uh so it was nice that we managed to reconstruct this using purely amateur images. At the time we was ever so slightly frustrated that we didn't find the whole thing. We only found this guy here but we didn't find this second uh stream. Um so we tried different variants of our analysis. We never got the second stream and then we said what the hell. uh we published it like this and then I'm telling this
44:08because then um recently or maybe five years ago or so uh suddenly there was an argument amongst astronomers that other astronomers said we can't reproduce this second thing here. So they were getting images that were basically better versions of this one. So that was that was also fun. And maybe also uh just an example, if you you know if if ever in your experiments you find you can't reproduce somebody else's results, there's always the possibility that
44:37these results might not hold up. So if you find something different and you're sure that what you find is correct, don't feel ashamed in publishing it.
44:46Um this is we we've recently also started thinking of other versions of this algorithm and and here we've applied them to set of images and we collected ourselves using just all sky sort of relatively very low resolution all sky cameras with a fisheye lens and and then you can also look at local version of this and then you see these images which is uh so here we see the the Milky Way etc.
45:13Um here's another uh example that we've worked on. This is now solar eclipse imaging.
45:21So this is an image um that was uh taken in in Chile during a solar eclipse from the side of an observatory [snorts] but but we took it or I took these images and uh [clears throat] what you can see so you so this is a high dynamic range image which means I'm showing you part of the dynamic range here. So this is a combination of multiple exposures. And if I make it relatively dark, it looks like this. You see the moon here. You see the corona of the sun. If I make it
45:50very bright, you can see ah there's even some structure that's visible on the moon. Now in both cases, this edge is not very sharp where while in reality it should be relatively sharp because the moon is quite dark compared to the corona and this is why people call it the the black sun. So if you who in the room has ever seen a total solar eclipse? Yeah. And and who has ever seen a partial one? Okay. So most people have seen a partial one. Uh unfortunately the
46:19partial one has basically nothing to do with the total one. So you you have to see a total one once in your life at least. It's a it's a life-changing experience. And the next one is on August 2nd uh in uh southern Spain in Northern Africa next year. Um, so it's this it's a very unique visual impression and I had this dream that I thought, well, we have this great uh high dynamic range displays nowadays. We should be able to take pictures that
46:48give you the impression of a solar eclipse. But then notice when looking at these images, we don't get this. What [clears throat] limits our dynamic range is the fact that the dynamic the huge these huge differences occur locally in the image. It's not enough to have some really bright pixels here and dark ones here, but you need this edge from dark to bright to be really veritical. Uh so we built a method that basically does this uh by performing a deconvolution of the whole image with the point response
47:17function pixel dependent point response of the system and then we get this kind of images. Of course, this is also not a high dynamic range rendering, but it was a lot of fun. And then we started also working on trying to bring out the fine structure in the corona. This gives an image like this. And then in the last year, sorry, no in 24 in America in the eclipse um I produced this image here which uh it has an extreme form of of
47:47local basically the same method that I told you before but applied very much locally. uh to bring out the very faint contrast.
47:56And what's interesting is you see lots of stars here also. And uh if you overlay it with a star chart uh you can see that we can identify these stars and we can see stars down to magnitude 10 which is some something that uh even at night you wouldn't need a small telescope to see. And then [snorts] we we saw this thing down here which looked like a blurry thing. And uh uh I was wondering is that is that potentially even a comet and I started googling and
48:24sure enough I found this other image where people have identified this as a a star grazing a sun grazer comet. So a comet that's about to fall into the sun but I think in our image it looks more more like a comet than in this one. And here I just included the latest image from uh two weeks ago in Spain, the solar eclipse where we went to take images. Here you also see a large uh what's called a prominence uh so one of
48:52these very red explosions next to the sun uh uh plasma emissions. Um so that was a lot of fun. So now let's go back to [clears throat] let's go back to uh causality. So how are we doing on time? Yeah. Okay.
49:09So we started at two. Um so how do we understand spirious correlations in terms of causality?
49:18Um so you've heard several causality talks. I saw some of it but I want to check now. Did anyone mention Rahenbas principle so far?
49:28Maybe not. So in a way I think you've already learned the content of Renb principle but nevertheless I think it's nice to um to give you this name because in a way uh this is the starting point of of modern graphical causality. So lagenbar principle says that if you find two quantities in the world that are statistically dependent such as the frequency of stors and the human birth rate then um there must be a causal
49:57explanation for it and this causal explanation so in the generic case the causal explanation is that there's some quantity that causes both of them and this quantity can be chosen such that when conditioned on this quantity they become independent. So that means there's a confounder that completely explains the dependence in a causal fashion. And by completely explains I mean that if we condition on it they become independent.
50:26Now as a special case of course uh this co this confounder could coincide with one or two the two variables in which case we get this picture or this. So that would mean the maybe the stalks bring the babies or the babies attract the stalks. But in the generic case, there's something else that causes both of them. And and I don't have it on my slide here, but I should say and I remember I think maybe Arthur was talking about this. Um there's there's a third way which uh can happen due to
50:54selection bias and this is something that's called Bson's paradox. So if you have a a variable down here which is caused by both of these and you condition on that variable then that conditioning can introduce dependence.
51:07So here the dependence goes away by conditioning but if you have a collider uh uh you introduce a dependence by conditioning but that's let's forget about the collider now let's just think about it from the generative modeling point of view then only this thing here appears now um if we connect it back to machine learning um we could now let's say we don't care about causality we don't like causality we don't want to understand about it let's just train a machine
51:36learning model. So we could say let's uh let's build a model that predicts the human birth rate uh from the frequency of stories. Now that we know they are correlated uh we can build such a model and it will probably do better than chance if we do a good job. Um but clearly that model will not predict the effect of interventions. So if you artificially bring more stoics, nobody in their right mind would expect that we suddenly have more child birth. Um and
52:04also the model uh will not transfer to other settings. It may work in Europe but maybe not in South America. There's no reason to believe that it will work the same way there. So in a nutshell as long as you are in an IID observational setting you don't have to worry you never have to worry about causality. But as soon as you leave this for instance by talking about interventions or for instance about distribution shift suddenly causality
52:32does become relevant.
52:34So uh the Renbar principle basically leads to what we call structural causal models. I think you did hear about structural causal models right something at least. So I will not go into detail.
52:46Um I'll just say uh it's a beautiful model. um you have a set of observables you think you as think of them as vertices on a dag and the semantics is such that the arrows represent direct causation. So whenever something causes something else you draw an arrow and mathematically that means that every variable is a function of its direct parents. Now so far this is all deterministic. This is a deterministic function. It could be a different
53:16function at each variable. We think of this as a physical mechanism. it's something in the world. Um, so up to here is deterministic, but we'll assume because we will never be able to describe the whole world. So we'll assume that there's also the possibility for some noise to influence things. So we have the possibility for a noise term at each node which basically means that at each node this function here becomes a conditional which is parameterized by this conditional distribution
53:45parameterized by this noise. Now let's think about these noises for a second. So that means at every node here in the graph I have an invisible noise feeding into it. Um we will make the assumption that these noises are independent.
54:01Why do we want to assume they're independent? Now if they were dependent then we would go back to the Rahimbar principle. Then Rahimar would tell us oh there's a dependency here. there must be something causal going on and that would mean we have to actually enlarge our model to explain this causality. If they're independent, there's nothing causal going on. It's just noise and then sort of the the causal structure is complete or a model is then called causally sufficient if it can be written
54:29like this with independent noises. Now uh if we do this then uh I told you everything is deterministic apart from the noises. So if we if we [clears throat] specify these functions and with the functions also the connectivity of this D and if we specify distributions over these noise terms then those determine the joint distribution over all variables.
54:53So you just plug in you can you think of it as sampling you just you sample from those variables you plug it in that gives you a way to do ancestral sampling on the joint distribution of this uh overall model. So these distributions determine the joint distribution of this and it turns out that this joint distribution over the observables has various interesting properties and um one is that it can be written in a relatively compact way. So this joint
55:21distribution turns out you can write it as a product of conditionals uh that are relatively low dimensional in the sense that so there's one conditional for each variable here for each xi but the conditional is conditioning only on its parents. So this is a this could if the if the connectivity is sparse then this is a much much lower dimensional object than a general conditional.
55:46Um so that's it has and this factorization the fact that this is a factorization that this is possible follows directly from the fact that these variables are independent. And there's a second thing which is interesting and that's the that the random variables they pick up a footprint of the graph structure. So it turns out that this observational distribution so this distribution that is implied by specifying noises uh has certain conditional independence properties and that's basically something that follows from the Aenbath
56:15principle namely there's this so-called markoff uh condition and we don't have to go into details on this now but what I want to say about it is that uh this is interesting because that's an observational conditional independence. So Kun in his talk he talked a lot about this how to do causally inference based also on observational data. So it is not totally impossible to do causally inference based on observational data.
56:41If you can make certain assumptions then you sometimes can see a footprint of of this topology of this graph visible in properties in conditional independence properties. Uh unfortunately uh this is building on conditional independence which is a turnary concept. you always need three quantities if you want to make a non-trivial conditional independence statement. So if you had only two variables uh there's nothing you can do with this. Uh and this was for us the starting point when we got
57:11into causality because you know if you think of this intuitively here uh why does this work? So I think of it like this. You have the the you have these noise variables. It's like a probe that you feed in in different places in the graph and and you know that they're independent all of these and as they spread through the graph depending on the topology like you feed in some noise here at the beginning only this node knows about that noise but then you compute you sample from the overall distribution and suddenly this node
57:40knows something about the noise that you put in here but may but this node is not going to know about the noise that you put in here because the sampling process doesn't go this way. So only if you have to condition on this. So so there's a um the information spreads and the the independence of the noises starts to pick up certain dependencies depending on the topology of the graph and that allows you to do this kind [clears throat] of uh inference. But it turns out there's a second aspect that
58:09was uh maybe ignored up until then. And this aspects I think uh Yonas alluded to this in his talk is that the noises don't just pick up information about the topology of this graph they also pick up information about these mechanis mechanisms. So if you have a noise here maybe this is a very simple noise a Gaussian and here you have a very complex function then what comes out here in a way is a a complex transformation of a Gaussian noise and
58:38sometimes that allows you to do inference and this is related um to do this we have to make another assumption which we call independent causal mechanisms and I just also want to briefly describe this um this is uh so here we have our causal factorization that I showed on the last slide and uh and I think uh uh Yonas and uh Kun talked about this assumption that uh one
59:06of those terms does not inform or influence any of the other terms. So a special case of this by the way is if you want to make an intervention. So if you have this kind of model, so I told you how to derive the observational distribution. Uh once you have written down that model, you can also trivially derive interventional distributions.
59:26Simply an intervention hard intervention would simply say I replace one of these assignments by an assignment where I set this variable to a constant. I can just do that. I can then sample from the corresponding model and then I get the distribution of the observ observables under a hard intervention on this variable. So so all the sort of complicated looking interventional calculus follows from that. If you've understood what structural causal models are then you have understood everything.
59:55Um and and the fact that the interventions are even possible means you can change one of these variables, one of these assignments without affecting the other ones. So that's in a way a special case of saying changing one of those terms does not change any of the others and even more generally we assume knowing one of them should not give us information about any of the others [snorts] and uh and related to this we sometimes
1:00:24use something we've called the sparse mechanism shift hypothesis and this you can think of as a generalization of iid so the iid assumption simply says uh so here we have a distribution Or in the simplest case, we have a density of that distribution and we can sample from this and every time we get an observation, it's a fresh sample from the same distribution. So they're independently drawn and identically distributed. Now if something changes
1:00:52um you sample from another distribution that's difficult um but you could have a smaller form of change or a slight weakening of iid could be to say let's assume that the world is built of such mechanisms and as we move from one problem to a related one not all mechanisms change maybe just one mechanism change the lighting slightly changes but all the other mechanisms are still in is and um and the question from
1:01:20a point of view of causality and also machine learning there is can you do something useful with that the data are not IID but they're kind of almost IID only one thing changed so [snorts] sparse mechanism shift um okay so uh
1:01:39let's look at a picture of uh what this means this this uh independent [clears throat] mechanisms uh and this is picture from visual perception again you you notice I like illusions so here you have this thing which is called a bet chair you see [clears throat] it's not a chair really but if you look through this hole it's suddenly so you look through this hole here then it looks like a chair from that perspective
1:02:07so uh no so now what's the what's the generative model of this perceptual process so in a very sort of high level way we could say there's a mechanism that is responsible for sort of generating the world of what's there. So or or someone put these put these objects there and then there's a mechanism a conditional that takes what's there and turns it into a percept percept or something that you see and this mechanism in [snorts] the generic
1:02:37case according to our assumption should be independent of the other mechanism.
1:02:42Now you can see very clearly here we are violating this independence. If we look through this hole, then we have picked a perceptual process which is very much tuned to what's there. It's only that one direction will work. And sure enough, in that case, perception fails because you see something that doesn't exist in reality. So for this reason, in the generic case, we assume this assumption should be true.
1:03:08And uh I don't want to go into lots of details but I just I want to give you a flavor and and get you thinking about uh what this can mean for machine learning.
1:03:19[snorts] So let's uh one example let's think about transfer. So imagine you've learned uh somehow a multivaried density model of some data. Now the density drifts to a related density uh which is a little bit different and uh you want to update your model.
1:03:39[clears throat] And then the first question is how many of the variables do you have to observe to given knowledge of P1 to estimate P2?
1:03:48And you would probably say that seems like a stupid question. I have to I have no assumptions. I have to observe all of them. P2 could be arbitrarily different.
1:03:57Right now let's imagine uh we have sparse mechanism shift and we have a shared causal graph. So let's we don't know these distributions but but we assume that they have the same causal factorization. Maybe we have prior knowledge about which variable could generate which one etc. Um and we assume that only a few of the mechanisms can change.
1:04:22Now in that case we don't have to observe all the variables. We only have to observe those variables that appear either left of the conditioning bar or on the right of the conditioning bar in those conditions that have changed. So whatever hasn't changed we just keep our previous estimate. So that uh that is quite nice because uh it's related to something we refer to as the out of variable problem.
1:04:49So in analogy to out of distribution. So you [clears throat] know out of distribution has been called the the big lie of machine learning. We are never really in iid. So iid is the big lie.
1:05:00We're always in out of distribution settings. One could say there's a second lie which is this assumption that we're always getting to sample from the joint distribution of everything that's relevant. In fact we always observe marginalss. We never see everything at the same time. And then the question becomes how to put these marginalss together. And then this is a question of where you put in your thoughts and where you use modularity whether explicitly or not. And uh and in this case
1:05:28um the causal structure tells you exactly uh uh how you can use these marginal observations. You can use them to estimate some of these mechanisms and update those. Okay. So I think I'm going to skip this but I hope this has picked your attention. It's also related to a classical problem in statistics called the marginal problem.
1:05:50And uh yeah, and I think I'm also going to skip this one. So this is an example. I told you before that if you take one variable that's affected by one noise [clears throat] term and then you feed it through a mechanism that's non-trivial then the output can contain information about that mechanism and it turns out uh this um
1:06:19this leads to an asymmetry between cause and effect. So uh and and if you're making an assumption that there's an independence between the cause and the effect given the the condition of effect given the cause then one can actually prove subject to certain assumptions that the same independence does not hold true in the backward direction. So you have suddenly an asymmetry uh between the causal and the antic-causal direction.
1:06:47And it turns out this asymmetry if you think it it from the point of view of physics is exactly the asymmetry that generates the arrow of time or the the what's called the second law of thermodynamics. So if you maybe if we still have time in the discussion we can we can go back to that. uh um so you can you can prove if given a certain formalization of this independent causal
1:07:14mechanism principle in terms of what's called algorith algorithmic information theory you can prove a version of the second law of thermodynamics where entropy is is modeled as kum law of complexity um I also want to briefly point out you can also give a an interesting uh formalization of this independent mechanisms principle in terms of
1:07:42in terms of a certain generalization of the definitity theorem. So think of it as a little uh little excursion. So I'll I'll try to spend only two or three minutes on this. So if you get lost on this one just ignore it. But if not I think it's always everybody should see the definitity theorem sometime. So uh this is a classic theorem of probability and it it tells you that if you have a sequence of random variables that's
1:08:10exchangeable. So that means the the probability law is invariant under arbitrary permutations. So the order doesn't matter. Now it turns out such a sequence is not always iid. So if you have an iid sequence of course the distribution is just a product of the one-dimensional ones. Okay. So it's necessarily exchangeable but exchangeable does not imply IID.
1:08:35Now um what definitity proved that even though exchangeable does not imply IID uh it implies conditionally IID it implies that the law is conditionally IID given some latent variable and this latent variable is what the basins call the prior. So in a way this is an argument uh if you if you believe that your data is is exchangeable then somehow under the hood implicitly there's some kind of prior acting there
1:09:04that uh explains the dependencies and the nonIDness uh but nevertheless the data is still exchangeable and then the the model is like this you have this latent variable the prior and then this uh uh variables are independent conditioned on the prior now it turns out. Um if we uh now we can we can do a generalization.
1:09:30Um so basically the bottom line will be that if we have exchangeable data and a certain independent causal mechanism assumption uh then we can as long as we're not collapsing into the iid case which is a simple but boring case we can identify causal structure from data. So in that sense nonID data is better than IID data. I think you've you've already had this during one of the previous courses and I think that's something that we're starting to understand over the last 10
1:09:58years or so in the field [snorts] and just previously we didn't have the tools to exploit that and uh and maybe I'll just explain the assumption to you.
1:10:07So our independent cause of mechanism assumption here is this one. So here this is the the so we assume let's say X is the cause Y is the effect. Now here we have the causes from 1 through n. We have the effects from 1 through n. So this notation means 1 through n. And this statement says that if we are if we get an additional input point just the cause [snorts] then uh this uh this doesn't add
1:10:37information to our conditional of y given x in a sense. So that means uh additional knowledge about the cause distribution does not change the the distribution of effect given cause it doesn't change the mechanism. So this is in that sense it's an independence of mechanisms and if we assume this then we get a factorization like before only that we now have two latent variables that are independent.
1:11:05So this is now a product measure of two latent variables. So now uh in this setting the independence of mechanisms just becomes a statistical independence of two latent variables.
1:11:15Okay. So and we've applied this to various problems. So now I want to use I want to use the last 15 minutes to talk a little bit about uh causal representations world models and digital twins. And uh I won't be able to cover everything but um let's let me give you a few basic ideas at least.
1:11:38So this is I think I might have used this already in some of the panel discussion. This is one of my favorite quotes. It comes from Con Lawrence who was studying animal behavior and he said thinking is thinking is acting in an imagined space. And I think if you if you think of it like this then of course it's totally clear that thinking is not just linguistic. So thinking is something more general and neuroscientists know that thinking is not just thinking in language. We often think in language but if you are trying to uh I don't know you're trying to put
1:12:07your key into the key lock at night then maybe you're thinking about how to align it parallel but you're not going to think about the world or I have to the world parallel or something. You look at it, you maybe you you picture something, you plan your actions, but it's a non-llinguistic form of thinking.
1:12:25And there are many more like many more like this. And and uh and of course animals also perform this kind of thinking if they even if they don't speak. Okay. So um I'm going to skip this. So we've been thinking a lot about how to learn causal representation. And a causal representation should be a representation which is not just producing pixels that make something look like an image.
1:12:54But it's a representation that should also allow us to reason in the representation. It should allow us to reason what happens if the if the chicken eats all these grapes, what will happen in the end to the cat or something like that. So it's it's a representation not just of what's there but of what would happen if we [clears throat] were to perform certain actions. So in other words a representation that subserves thinking and uh you can think a lot about what does it even mean to have a
1:13:23representation and there are different uh example this is a this is the atom model of bore which was inspired by the planetary system etc. And uh and at some point then there was this movement called positivism in in philosophy and and this inspired people such as Heisenberg to say well maybe it doesn't make sense to think of our electrons as planets uh orbiting the nucleus. maybe we just should think about what are the actual measurements uh and and and and
1:13:53base everything on that and so so I think we have the same kind of discussion also in our field but I don't want to go into details but rather I want to give you two desiraata of what I think at at least a world model should satisfy and one I think comes goes back to at least hanets and probably also to precratic philosophers somewhere in Greece. I mean all the good ideas
1:14:20um have been around for a long time and had said we create internal symbols of external objects such that the consequences of the images or representations that are necessary in thought coincide with the images of the consequences necessary in nature. So you have consequences of images and images of consequences. What is necessary in thought and what is necessary in nature. So this is really a commutative diagram.
1:14:46[snorts] So you have something happening in nature and then you have a representation that sort of goes into our thought processes and then and you have a representation of what's happening in nature. You have a transformation in your thoughts like a mental action or a mental time evolution and what you would like is this kind of consistency.
1:15:06So that's one one thing and I think uh maybe that is one thing that should distinguish a causal world model or causal representation from a statistical representation. So we want a mapping that maps not just the the perceptions but also the actions into a latent space and does so that we get this kind of consistency [snorts] and actually as an aside I think this is not sort of biologically this also makes sense uh because we have these kind of representations of actions also in our
1:15:36brain there's something called epherence copies so if you move your eyes a signal that gets sent to the eye muscles is also fed back to the brain and processed when you interpret the images and uh so all this and the second thing is whether a representation is causal or not depends on your action repertoire because this kind of consistency has to hold true for a certain class of actions and the actions that you have they depend on what you can do. So a fish can
1:16:06do different actions from a human or if you're an ant you see a big pile of sand maybe you it makes sense to perceive individual grains of sand and you can manipulate individuals but as a human there's no point in seeing grains of sand you just see a big pile maybe you have a shovel and you know what you can do to the pile with a shovel and I think this was beautifully captured in a story of bes that I read when I was young it's a story it's called there are more things it's it's somewhere set in Buenos Iris
1:16:35and he writes like how someone I I don't know forgot why for some reason gets into a house and and encounters all sorts of strange things and I think at some point towards the end he figures out that this house is inhabited by aliens and uh and he somewhere in this story he says there are objects here I will not des I will not attempt to describe them because I'm not certain I actually saw them so he says in order to truly see a thing you must understand it
1:17:02an armchair implies the human body, the scissors, the act of cutting.
1:17:09And then he says that the passenger does not see the same ship's rigging as the crew. If we saw the universe, perhaps we would understand it. So, so we need uh so I think this is uh is crucial that a representation has to be action dependent. There's no point in seeing something that is completely irrelevant to you in terms of taking decisions or actions. And of course, this is something that psychologists
1:17:38have known for a long time. [clears throat] And I won't go into detail through all this, but this is just like a short history of different psychologist thinking about this. And the last one fits very directly to something that I want to present you in the remaining time. And this is this notion of sensory motor contingencies.
1:17:56uh and these were uh proposed or defined by organ and no as the lawful relationships between an agent's actions and the resulting changes in sensory input. So you perform actions and you learn how these actions change what you perceive. You have no direct access to the world but you see the effect of your actions in your perceptions and somehow this allows you to learn a certain structure about the world. It's a mental structure and then the question is does
1:18:24this structure have something to do even if we assume there is a world out there independent from us what can we say mathematically about the the correspondence between your mental world and the world out there um and uh so we started working on this problem um [snorts] building on uh something called latent world models so think of it as a action conditioned Japar model. So in a jpa model you have such you have an encoder.
1:18:54The idea is uh let's forget about reconstruction objectives. There's no point in seeing everything. So that's already aligned with our philosophy. Let's map to a latent space um and learn in that latent space an action condition model how the latent states evolve over time with actions. So this is this is good and the first thing to observe in these models is that this kind of model already satisfies our first disinum.
1:19:23So why is that the case? Well, first of all notice that if we so here we have our forward model. If we now fix an action here then uh we uh so then this thing it has inputs Z the latenc
1:19:41um becomes a mapping G sub A. So conditioned on A uh which is now a mapping that takes Z and maps it to another Z. So it's a it's a mapping from the space of the latence to the space of the latence. And this mapping here we can now think of as the representation of the action A. So A is represented as a certain transformation in our latent state latent space.
1:20:07Okay. And now let's think about what does if we train these things. So what we're going to do is we we see something we encode it. we carry out our action conditioned forward map and then we minimize the loss when we compare what we get to what actually is the embedded state at the next point after the action. [clears throat] So now imagine we manage to minimize this loss then uh if we minimize the loss then
1:20:35this thing here holds true approximately and now we just substitute what this means go through a few step and then what we see is actually this means exactly that for the training points O F of A of O so performing the action uh uh in the state O uh is the same as uh uh embedding O and then performing the mental action. So this if we successfully minimized our cost function
1:21:04then this thing here holds true and this is actually an equivariance property. If we uh so so in compact notation without the O we can say that on the training support where we've seen data this thing here holds true where this is the action in the world and this is the action in the representation.
1:21:22So that's nice [clears throat] but we're not there yet. The reason for this is the problem of collapse. So if you if you do this, what prevents you from mapping everything to the same vector?
1:21:35For instance, mapping everything to the constant vector zero. If you encode everything to constant vector zero, it becomes trivial to learn the forward model. You just map zero to zero. And this is also trivially uh equivariant. So everything is fine, but you've learned nothing. And uh this is now where we we we learn our perceptual contingencies. So now we want to learn the lawful relationships between our actions and what changes in the world.
1:22:02And uh and to do this what we'll do is the change in the world means how does the world look before I act and how does it look after I act. So we're going to feed these two things into our inverse model or through our latent representation and then into our inverse model. And the inverse model has the task of reconstructing the action. And if we want to be able to do this, then uh we can't map everything to a constant. And it turns out that just
1:22:30this one trivial change which has been used in other contexts before is sufficient. And uh I think I won't uh go into all the details but it actually there's something intriguing that in our model if we do some latent rollouts uh it turns out uh in some cases uh not only do we get uh this equivariant but we also get a representation as a translation group. [snorts] So let's
1:22:59let's skip this and uh and it turns out our system also satisfies the second des atom which uh I call homo or which is called homorphism [snorts] and homorphism. So this is um going back to some other famous scientist from the past.
1:23:22He said suppose that the concept of living being is represented by the number two and the concept rational is represented by the number three and and he was thinking the the logical conjunction of living being and rational is human. So that's how they thought in the stom then he said the concept human should be represented by the number of six because that's the product of two and threes. So this is maybe the first appearance in mathematics of this homorphism where in this case logical structure is mapped to
1:23:50multiplicative structure and it turns out uh we get this for free in our model. [snorts] So if we assume that our actions uh form a group or a semigroup and a composition then it turns out we also get a homorphism for free and and we can prove that in and it turns out we even get in many cases and we have some reasons to believe that this is so there's there are reasons for that we even get an representation as a
1:24:19translation group which is also related to certain other things such as this linear representation hypothesis. So I think um with this um and with just showing this quote of Conrad Lawrence again I want to conclude. So a causal representation can give you a world model that allows an intelligent agent to think [snorts] but I think equally fascinating and I didn't talk about this it can also give
1:24:48you a digital twin of a complex system and I think with that I want to want to conclude. Thank you very much. [applause] Thank you very much. We have one question over here.
1:25:12Hi. Hi. So, thank thanks a lot for the talk. I was uh a bit lost on your slide where you showed the structure of atom thing and I was just imagining that it's well known during during the time of Neil Bour that the electrons are revolving around the atom right but if we could have a datadriven model somehow we can learn some abstract uh embedding or something of this particular thing and can do some cause effect analysis on top of it. Ideally we would want something that would take us to the
1:25:41theory of orbitals. How do we actually do it? So in a way I think I think you're saying suppose you are you go back into the time of quantum early quantum mechanics endowed with all our methods of machine learning causal representation learning you only get the measurements can you reconstruct uh what's going on or what are the what are the right physical quantities to build the theory on [snorts] I think that would be absolutely fascinating I don't
1:26:10think we are there I think this is the kind of thing that Kun was talking about when he when he talks about causal representation learning for discovering scientific concepts. Uh if we had had a method that can do this in those days, we would not have ended up with a bore model because the bore model in the in the end people dispensed of it because they noticed they can't quite make it work like this. So people would have come up with other notions. maybe they would have even come up with the
1:26:39uncertainty relations and noticed that there's no way of consistently measuring position and or talking about position and momentum at the same time. Um so I think that would be the holy grail of uh scientific discovery that uh something like causal representation learning allows us to discover what are the actual scientific variables because you know the amazing thing in science is many of the nobody has ever seen an electron. So uh there are these are things that we indirectly infer from experiments. You do clever experiments
1:27:08and some point you start talking about electrons. You start talking about gluons. Nobody's ever seen them. But we still believe in them because we have somehow built a consistent model that is predictive under certain types of interventions and experiments. And uh it's a it's a it's quite a deep question what convinces us that such a model makes sense.
1:27:30Yeah. Just a slight Understanding my second question on the same lines around like with the current arsenal we have around the theory that's developed in causal mechanics and causal discovery uh and revisiting those theories with this particular angle. Does it make sense?
1:27:49I mean I think we are [clears throat] we are nowhere near there. So I think this is an extremely hard problem.
1:27:57Um and you know with quantum mechanics it took it took a generation of physicists or some would claim we are still not there but it took at least a generation of physicists to converge to an intermediate consensus uh of what are the right concepts and what is like people probably still argue about what is the what is the the onlogical or onic content of a wave function for quantum mechanical state.
1:28:22Is there something like is it an objective thing or is it just a mathematical trick that allows you to compute expectations of of experiments that you can uh carry out? So I think you will still find physicists that don't agree on on questions like this today.
1:28:44One question in the middle there.
1:28:53Uh thank you for the talk. So uh I have a conceptual question about the thermodynamics understanding of causality. So um if we define um the error of time in terms of uh the direction of where entropy increases then it would um uh include the process where it is irreversible like gas diffuse across the room but on the other hand there's also
1:29:22reversible process like if I hit a ball and then the ball hit another ball and it bounces back and these kind of processes is reversible and it seems like um the thermodynamics um perspective can only include the irreversible part and the other part is missing if I understand it.
1:29:46Yeah. Know I can I think I can give you an example that [laughter] kind of illuminates that. So that you know a classic example that physicists have discussed before is you take to make it a little bit simpler you take a a gas chamber with a door in the middle and you have one gas on this side one gas on this side. Okay. Once you open the door they will start mixing and uh and they will never spontaneously unmix. Once you've mixed them there's no way to
1:30:13unmix them it seems. So that that seems there's a fundamental asymmetry. And how can this be reconciled with the fact that microscopically all the laws of motion are are reversible? And what I was trying to allude to is that this is an effect of of making a very specific assumption at the beginning. So in [snorts] this case, imagine you uh you start with this thing. Now you open the door and at some point you stop time.
1:30:42You are a god. You can magically freeze time and you you take every particle, you turn around its momentum and then you let time continue then everything will go in the opposite direction and if you close the door at the right time you will have perfect unmixing again. So that means uh there is a very special starting point. So you you run it forward. Now you freeze time. You turn around all the arrows. Now you've created your new starting point. And with this very strange starting point,
1:31:11the system magically unmixes.
1:31:14So for this reason uh we only get the thermodynamic error of time if we are allowed to make this independence assumption for our starting point. If on the other hand, if you can create if you can start with a very strong dependence between your starting point and the mechanism. So the mechanism means that dynamics applied for a certain time and then all bets are off then you are not going to get in increase of entropy. You can get it you can get a decrease of entropy. So so in that from that point
1:31:43of view I think uh it all comes down to an independence assumption.
1:31:50Thank you. I think we're out of time. So if you have further questions please uh use the break. Thank you.
1:31:58Thank you very much.