0:02Bloomberg Audio Studios podcasts radio news.
0:13Bloomberg Tech is live from the heart of Silicon Valley with Ed Lello in San Francisco.
0:23This is Bloomberg Tech. Coming up, OpenAI reveals new cases of its AI models misbehaving and a new plan to track when things go wrong. Plus, data brick CEO Ali Godzy and machine learning pioneer Andrew weigh in on the global AI safety debate. And while AI may pose an existential risk in Silicon Valley, the immediate fear is missing out on the money. That story later this hour.
0:49AI warnings take the global stage as King Charles III sounds the alarm. There seems urgency inadequately considering the existential dangers of such technologies falling into the wrong hands and being used in potentially catastrophic ways. Surely then we need sufficient means of control before it is all too late.
1:17That was King Charles pressing some of AI's biggest names for reassurance. They can keep Air AI under control. Nvidia's Jensen Wong, OpenAI's Sarah Frier, and Deep Minds Demis were among those gathered in Scotland for the event. I want to get to Bloomberg tech editor Sha go over in London with the latest. Let's start with why King Charles pulled this event together, Shona.
1:39Yes. I mean, he's not an obvious choice, is he, to convene an AI gathering. Um, but you know, still a monarch, still has has the sway. Um ultimately it's against the backdrop of uh you know Anthropic CEO Dario Amade's blog post over the weekend um saying that he thinks there should be a pause a slowdown in frontier AI development that's been accompanied separately by um you know admittedly lower level but you know AI researchers quitting some of the top AI companies including anthropic also deep mind
2:08effectively saying that you know AI could be lethal for humans so there's a sense that fear around AI is is has been a debate within Silicon Valley, but actually now we're seeing those fears um spill out beyond Silicon Valley. MPs, lawmakers in the UK and and globally are quite concerned. The general public are also quite concerned about what happens if um AI isn't adequately controlled.
2:31And again, that's all been triggered in turn by events over the summer where we saw um AI agents from OpenAI uh you know, the phrase is going rogue. I wouldn't say that's quite right, but you know, escaping guard rails and attacking other organizations in ways that, you know, Open AI w wasn't necessarily expecting.
2:50Sha, remind us which leaders of industry were in the room, so to speak, and what King Charles's message to them was, maybe even his instruction to them was.
3:01So, gathered some big names. We had uh Nvidia CEO Jensen Wong uh the the former CEO and co-founder of DeepMind, Demis Habis. Um also Sarah Frier, chief financial officer of OpenAI. Um Anthropic did have a representative in the room, but it was their global policy chief. It wasn't any of the co-founders.
3:20It wasn't Dario. So kind of missing a big name there. Also, no one from Meta or X, uh interestingly. So um you know, good representation, but not everyone.
3:29Um and and the monarch prepared a speech um that was was sort of released in advance and essentially you know he told these leaders something he said before actually which is you know if you're going to develop this tech can you make sure that it's good for humanity um good for the natural world which is you know environmentalism is is something Charles has particularly been interested in has been for a long time so he was really pushing for the technology to to benefit humanity and to just pause and think about those consequences. Bloomberg show
3:58in London. Thank you very much. Another big story, Open AI is revealing previously unreported cases of its AI models going off script, fabricating data, bypassing restrictions, sharing files they were told to keep private.
4:12Now the company is setting up a new system to track and disclose those incidents. Bloomberg Sharen Gafari who leads our coverage with Frontier Labs is with us. Let's start with the plan that OpenAI has presented on what it's going to do if it finds models misbehaving.
4:28That's right. So they have said that they are now going to essentially have a triage like system for employees to disclose incidents where they think that the AI models are essentially misbehaving are eliciting some kind of concerning behavior that goes against what they intend the AI to do.
4:48There were some interesting case studies. OpenAI also made it clear that in the six they disclosed there was no hacking of third parties but some of them are worrying right take us through what OpenAI disclosed this morning.
5:01Yeah. So even though when we asked OpenAI said that this did not involve these incidents the six that they have disclosed did not involve breaching third parties there are still some you know concerning behaviors that we're seeing here. For example AI models trying to share files that should have been private with each other. Um communicating with each other in ways they're not supposed to. Uh telling AI models essentially uh learning how to disregard or instructing others to
5:28disregard their instructions. uh on on how to behave. Uh so these are all incidents where you can see the potential for that to evolve into something even more nefarious, especially if um those AIs were successful and sort of going against what what the creators of the AI expect these AIs to do.
5:47You your report is one of the most read on Bloomberg this morning. That is not surprising. and and lower down in the report and in open air's blog post there is a um admittance that the state of play and industry from safety and alignment is not where it should be.
6:04That's right. They essentially nobody has solved this problem of so-called AI alignment which just means is AI actually working uh in accordance with the intended outcome or how you know humans would want AI to behave and the fact that even the brightest uh minds in the world that even the top AI companies have not figured this out is uh concerning uh by these companies own admission and so um you know they're saying that they they hope to disclose
6:34more of these incidents publicly, but there are still a lot of unanswered questions and there may be many more of these incidents that we just don't know about yet.
6:42Bloomberg Sharing Gafari, thank you very much. We'll have more on the growing AI warnings later this hour. We're going to speak to Ali Goldy, the data brick CEO, and then even later in the hour, Andrew, founder of Deep Learning.AI, uh, a a long-term researcher in the field of artificial intelligence. And even as those safety concerns grow, the money keeps flowing into open AI. Apollo's in talks with SoftBank to boost a loan to as much as $9 billion, helping finance
7:11the Japanese firm's massive bet on the AI company. That's according to sources. Soft Bank's committed nearly $65 billion to OpenAI and is increasingly turned to borrowing to fund that investment. And that massive flow of money into AI is fueling demand across the industry. Nvidia CEO Jensen Wong says the opportunity is expanding across countries and economies.
7:34I expect Nvidia to sell twice as many chips this next year as we do this year. And um and the reason for that is because uh AI has has so much contribution to the benefits of different industries, different economies and you could see that almost every single country that we're in, people want to invest in AI.
7:55Joining us now, Tro Price portfolio manager Tony Wong. Nvidia is the top holding with Tony. I wanted to play that sound bite because to me that sounds like a disclosure of unit growth which I haven't heard from Nvidia often if at all twice as many chips next year as this year. He could be talking GPUs, uh, the Grock LPU CPUs, but just react to that, Tony.
8:18Yeah, I mean for a company of that size and scale to be growing like that, I mean it's pretty much unprecedented. I think that you have to look at what's really going on in end demand. I think we are seeing reflection and agentic adoption in the enterprise and then you're seeing um kind of releases of personal you know agentic use cases with muse and instinct and so I think that this is what's going on in terms of we're going from AI answering questions to actually doing actions and completing
8:47tasks and and producing outcomes and and so I think that there's a lot going on here and and it makes sense that Nvidia as as the one of the biggest platforms for AI computing um is going be a big part of driving back.
9:00Tony, I'm going to take you back to the weekend industry supporting each other in saying we need to pace at the frontier. Now, bring us back to public markets today. How did that cause you and your industry to kind of remodel demand going forward? Is there any change at all on that idea that there would be a slowing or a pacing at the frontier?
9:21Yeah, look, I think that, you know, safety is definitely, you know, a very important topic and I think it's good that the industry is super focused on that and there's a lot of tension around it and whenever you're developing a new technology, you want to make sure there's guardrails in place. Um, and this is obviously the most critical technology in the world right now. So, to to me, I think it's good and and I think at the same time, it's also not an aspect of like we're going to stop developing frontier models. if anything um that's continued to be uh you know
9:50very very top of mind and and the way to do this is is not to um you know pause uh the AI kind of development but to make sure that we're all paying attention what it is and like to make sure that we we put investment behind it. So to me I think there's areas of cyber security that are super important that are becoming more important and then um you know overall I think that it just underscores how important this is um in addition keeping the frontier uh you know race going.
10:19I study your portfolio. So you know it is Nvidia and Broadcom and then Apple and the hyperscalers up there right you pose the question who owns the customer when the agent owns the interface. I don't see that much of a software play in the holdings, at least the top ones.
10:37Yeah. Well, I think that um you know, I think that's a great theme um that the market is thinking about is that, you know, over the last 20 years, the internet um companies have essentially, you know, become like the interfaces between customers and then uh companies that deliver product. So I think the agents kind of changed things in terms of like you know for a consumer like instead of going logging on clicking on a bunch of things going to various websites comparing like you could have a personal agent that kind of does that for you and brings you the results right
11:06so I think that's going to be an interesting paradigm shift in terms of you know it'll be more important for software companies and um you know kind of OTAAS to like really embrace this change and I think that it won't be all the same um you know companies and software companies especially especially that have data gravity that have the workflow um you know the orchestration, they understand the customers, they know the security, they know the permissioning, they're going to be important in terms of uh you know the these models are kind of like the brain
11:35and then you also have to develop the body and the nervous system to make sure you can carry out that action. So, um I think it'll be really interesting, um area to watch because a lot of those companies have been gone gone from like kind of the AI loser bucket to the AI winner bucket, you know, so you see tremendous moves and and that's exciting to see.
11:54Tony Wong with Tro Price. Really great to have you back on the show. Thank you. Some of tech's biggest names are heading to the White House by the way. Apple executive chairman Tim Cook, Open AI CEO Sam Alman are set to attend President Trump's state dinner for Chinese President Xi Jinping, which is next week. Nvidia CEO Jensen Wong is also expected to attend.
12:16Zipline could be getting a major valuation boost. The drone delivery startups in talks to raise about a billion dollars at a roughly $20 billion valuation, nearly triple what it was worth earlier this year. All according to sources. Bloomberg's Natasha Mascarinus broke the story with me and joins us here on set. Let's start with the round. What do we know? What do we not? Don't know.
12:35Yeah, absolutely. So, earlier this year, Zipline raised at around a 7.6 billion valuation. Um, and then came back in the news last month to announce a new Uber partnership that was set to expand drone deliveries in the United States. And so now we have news on um how investors are perceiving that uh update, which is with more capital. They're in talks to raise around 1 billion like you said at around a 20 billion valuation. Still pretty much in flux in terms of who all is participating, but we do have some names
13:03like Paradigm um and Tiger Global involved in the round. Both existing investors.
13:07You know, it sounds early stageish in this particular round to me. I mean the the origin of Zipline is a company that particularly in the continent of Africa was about finding a solution to get blood medication to remote areas where there's no hospital and then it evolved to uh delivery. It has a relationship with Walmart. You know what is Zipline like like tell us about the company a little bit.
13:28Yeah, it's so interesting. This is not a company in the AI zeitgeist like you and me are usually talking about um here but this is a company founded in 2014. spent a decade building a logistics network and I think of them as an example of this whole vision of vertical technology. So they build the hardware like the drones themselves and then they also build the software, the logistics network, etc., etc., which is why they can go from delivering blood to delivering, you know, Jimmy John's.
13:54Right. We're showing the video now of the kind of illustrative examples of their sort of home delivery. I've also been out to to Venville, Arkansas, and kind of seen it firsthand with their prior generation. Um, Paradigm, I asked you about this last night on the phone, like Paradigm, not a firm that I'm as familiar with. We understand in our reporting they're leading this round.
14:12Yeah. So, I've been following them for quite a bit. I mean, they are founded by a former Sequoia investor um and have actually made their name first and foremost in crypto. Uh but earlier this year, they announced that they were raising um their fourth fund, I believe, a little bit over 1 billion and expanding into Frontier Technology, which includes AI, includes robotics. uh they're an existing investor in Zipline, but I do think this is an example of a once crypto firm making like one of their biggest bets this year, I would
14:41assume, on a company outside of crypto.
14:43Bloomberg, Natasha Mascarinus, thank you very much indeed. Okay, coming up, another layer of the tech stack weighs in on the AI pacing debate. Data Brick CEO Ali Godzy joins us next. This is Bloomberg Tech.
15:13As we move from labs to product development, the companies have to become much more rigorous in testing and securing and making sure that the products are ready for use before it's released. And uh in doing so, if it's not ready, just hold it back. You should go as fast as you can, but no faster than that.
15:37Nvidia's Jensen Wong again speaking ahead of that meeting with King Charles III. OpenAI has raised the ante on safety worries with its latest admission that its models misbehaved and saying in its blog post that the industry has not solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I want to get the perspective from another layer in the AI stack and that is from Ali Godzy the CEO of data bricks pacing at the frontier
16:05and where we're at in alignment and safety. Weigh in Alli.
16:10Yeah. Uh I think first and foremost uh as leaders we have a responsibility to not freak people out. Uh so I think it's really important that we tell people uh the existential risk to humanity is close to zero. So no one needs to lose sleep over that. I think this is important because I think a lot of people you know they might get freaked out about these things and it might put them in a bad state and it actually can have serious consequences on people's health mental health out there. Uh so the existential like you know hey you
16:38know humanity is you know AI takes over that's that probability is close to zero. Um where there is risk is in cyber because AI is just very good at breaking into things. So it's very similar to the very early days of the internet when you know we had these worms that were going around and we had viruses that were spreading because we just hadn't connected the whole world together through the internet. Once we did that suddenly you could attack any target anywhere and it's a similar situation
17:06now where the AI models are just great at breaking in and now they have all these targets to go through. So we do actually have to have really really uh huge investments in cyber and protect our software systems that exist all around the planet. This is real and this we have to do and actually we are actually in that space. We have a product called lakewash that actually helps you do that and it uses agents. It uses AI to actually do that. That is real and it will have damages and costs to it but that's not an existential risk.
17:36the risk. Um, can I take you back to kind of the early days of the pandemic?
17:40At the time, you were talking about being quite paranoid, right, in general, not not about AI at that time. So, you took your teams and have done into some sky is falling scenarios. You basically want your people to prepare for the worst possible case scenarios that could happen in the world. Are you still doing that? And and how is that apply in this AI safety context? Yeah, I've always been one of these, you know, really, really paranoid CEOs. We've been doing AI, you know, since the first days,
18:092013. Our first use case was AI use case. So, yeah, we do those every year and we want to make sure that we're protected. So, as I said, I think cyber is real. So, that is actually part of our exercises. You know, what happens if someone gets access to the computer systems like we have very very sensitive data at data bricks. We actually do a lot of these simulations of skies falling also on the cyber side and we let basically attackers from outside try to break in and get all the way into the most you know uh sacred secrets of data
18:38bricks uh and keep hardening those and actually we've had to ramp up those efforts I would say in the last 4 years uh 2018 2019 the time it would take for an attacker u you know from from revealing a vulnerability and some attacker weaponizing it it means is actually breaking into a site would be 2 years. This has now shrunk down to hours. So the moment the vulnerability is out in less than you know hours someone will be attacked with that. Uh
19:06so this this is real that's just not existential. So I just think we should like we should separate these things. People are like mixing these things uh and you know making people worried about AI because AI has a lot of a lot of great use cases. We see huge positive use cases in our customer base where they're actually moving the needle. that actually means a lot to people to be able to leverage that AI.
19:28Early in June when we spoke, you said, you know, it's not a good year for an IPO because of the pipeline of huge offerings, right? If you are the CEO or CFO of a private company, like looking at the market, but this past week, you know, simply, and I know your investors won't thank you for for the answer to this, but Sam Alman said to Fortune at the weekend, IPO next year because of AI safety concerns right now. Does that factor into your calculus?
19:53Yeah, I mean look, there's just so many so much demand uh for just the software. So like getting bogged down in preparing an IPO for us. It just wouldn't be a uh you know it wouldn't be a good use of our time. We will go public but it's just you know we just have to we want to rather focus on our business right now.
20:10I mean you had just before this you had zipline on. Zipline is a big customer of data bricks. They have amazing use cases. That's an AI use case where they use AI uh to automate, you know, all of the drones, the medicine delivery. You know, another awesome use case is crisis text line. They actually use AI to detect self harm among the youth and they can actually use these large language models to detect that and help prevent that. There's so much demand for this stuff and then I mentioned the cyber uh you know use cases that we have. We'd rather be focused on this. If
20:40we were public right now, we would be spending all our time trying to figure out how we're going to react to this, you know, latest crisis and do people think that, you know, the world is going under and, you know, what are we going to do about this and we'd have to worry about our stock price. So, I think in these big times of transition, uh, you know, it's better to do those transitions in private. In fact, some public companies go private during times of transition to do those transitions and then they go public again. So, we will be public. Uh, I just think the timing is not good.
21:07Just to clarify quickly, we didn't have Zipline on the show. We did have some reporting about their latest private market round. Let let's end on a positive, right? In this environment where data is quite clearly key to the harness on a model, the utility and capability of model, where is is data bricks growing? Where are you seeing some some momentum?
21:26Yeah, I mean honestly, we're seeing acceleration across the board really like we're trying to pin down where is the acceleration? Is there a particular thing? And I think what's just happening is that all these use cases that I mentioned, uh it's almost as if every company on the planet doubled their employee count because there's no agents working at all these companies and these agents want to do more things with data and they get more insights from that data and you know that then uh you know increases the consumption that we're getting since our pricing model is
21:54consumption based our revenue then goes up and it accelerates and in particular we have one particular product called Genie which basically is your analyst. You can ask it anything about your business. How's this product been? Who's turning? Where is my pipeline? And you know, this product has accelerated the revenue growth significantly. So, you know, it's exciting times. Data brick CEO Ali God, it's great to catch up.
22:16Thank you for your time on Bloomberg Tech. I would just point out this is what financial markets look like, particularly the technology sector post Fed day where the Fed indeed did raise rates. You see some outperformance in technology. The NASDAQ 100 up 1.6% outperformance in semiconductors. Nvidia a big contributor to that. We played the soundbite earlier in the show, but Jensen Wong, the CEO of Nvidia, saying that they will sell twice as many chips next year as they did this this year, which to me seems like a material disclosure on unit growth, US 10ear
22:45yield below 5%. Um, pretty large move.
22:48We've seen that across the bond market this morning. Let's get back to our top story. AI is helping companies build faster, but as competition at the frontier intensifies, should there be limits on how much risk companies can take to stay ahead? Karen McCormack, chief investment officer at Baron Gia, argues the answer isn't slowing innovation, but establishing common guard rails around testing, cyber security, and incident reporting. She joins us now. Explain that argument.
23:15Uh, sure. And um, I just preface this by saying so um, Baronia, we're early growth investors. So our portfolio companies range in revenue anywhere from 1 to 300 million. So they'll be at the smaller end of the spectrum. So we're not going to be investors in companies like Anthropic or Open AI, but our portfolio will be on the receiving end of what they're publishing. So I guess our perspective comes from from um those types of businesses. And I would say the most important things to our companies are things like what's the enterprise capability of what's being released?
23:45What's the speed at which they're operating? What are the new features that we're seeing? and really importantly cost. Bundle all of that with is this product safe to use?
23:54Because for companies like ours, we don't necessarily have large teams of either security or cyber security or even IT. So, we're really reliant on what they're releasing for our portfolio companies who have been net beneficiaries of all of it, but still concerned.
24:10So, that's why I think we should talk about regulatory capture. Right. Earlier this week, I was talking to David Saxs about why there should or should not be a greater degree of government intervention or regulation. And his viewpoint is right, the Frontier Labs, they know what they're doing. They should take on the responsibility of safety themselves. But that regular capture, regulatory capture, how an idea for the whole industry serves maybe just two, how would that impact your portfolio?
24:37Yeah, it's a good question. It was a great interview and I don't often agree with David, but I did in this particular instance. It does feel like we have especially the two front runners anthropic and open AAI who are at on the one hand saying we really need more safety guard rails and on the other hand still continuing to produce at pace. So, um I do agree with him that there needs to be some sort of self-regulation and our companies need to be able to trust that the products that they're using are going to be safe. Not the least because due diligence when people are looking at investing in our companies or acquiring
25:06them, they need to have some kind of um safety due diligence and and guard rails in place themselves. So, I I think I overall agree with the sentiment that there can be some self-p policing in the industry. However, one of the bigger risks is when things go wrong, what we tend to find is that government gets more involved and government regulation for our portfolio companies often means more cost, slower dispatches, and just higher costs, less competition.
25:33We're talking about early stage investing in AI in private markets, right? Is there any risk that the environment we're in results in in your section of of the venture market slowing down, just stop writing checks for a while?
25:47I I mean I would say we already are seeing some of that kind of venture capitalists and and private equity sitting on our hands. I mean it started with SAS apocalypse and you know are all these companies going to be um decreasing in value because AI can do all of the things that they were doing.
26:00We're finding that that didn't really happen. There wasn't quite the SAS apocalypse that we talked about but there's definitely a lot of delayed decisions waiting to see what happens.
26:08You know is Anthropic's next release of whatever it is anthropic for financial services going to take the place of a lot of software companies. So I think that's already happened. I don't think it's really a result of guard rails or lack of guard rails. But we are seeing for example a lot of our portfolio companies now starting to question and realize what the actual cost of this usage is. And the honest answer is we don't know. We know that it's going up consistently but we don't know what kind of there is no endgame. So we don't know where it's going to be headed.
26:37There's a really big focus right now on the absolute frontier right that's just backed the most capable. But what about AI at the other end? What about AI for everyday people who don't need it to be spectacular? Is there an opportunity in that?
26:53Absolutely, there is 100%. And I think we focus a lot on, you know, the front runners and and the most expensive versions which have the best capabilities. But the reality is on a day-to-day basis, whether it's an enduser like you or me or our portfolio companies, there are a lot of use cases that don't necessarily require the most expensive version of models. And to be honest, this is where, for example, um things like deepseek come into play.
27:16Really lowcost models for everyday tasks. But there is a question mark around the security level and whether policy is going to allow us to use some of those types of models. But 100% there are different different capabilities, different models for different use cases. And that does help keep the cost down for sure.
27:32Karen, you're coming to us from Boston, but you're really focused on the UK and Europe. Um, so explain that part of it, but everything we've discussed, how is that playing out in that market different to the United States?
27:46Yeah, it's a really good question. So, as you mentioned, Baron has both US and um European investors. So, half of our portfolio is in the US, half is in Europe. We are definitely finding that Europe, which is not unusual, is slightly ahead of the curve from a regulatory and safety perspective. Um there's more legislation that has already happened and there's a bit more forefront of asking about the AI safety regulations. Um I'm a co-founder of a business called Reframe Ventures which is effectively looking at both ESG and safety requirements for AI into early
28:16stage companies. Um and the reframe ventures framework has been adopted widely within Europe and it I would say there's just a more um I guess forward view of what are we looking at what are the risks and controls that we need to be looking for within the European ecosystem. Although it's absolutely increasing in the US and we are finding that in the US whether it's limited partners looking at us for investment or um investors looking at our portfolio for investment there are a lot more questions now about AI safety and regulation within the companies. Karen
28:45McCormick, chief investment officer at Baron Gia. It's really great to have you back on Bloomberg Tech. Thank you very much for your time. Let's get out to New York. Bloomberg's Yahara Anand has all the other news. Hi, Yahara.
28:55Hi, Ed. It's time now for talking tech. First up, a group of 10 banks is lining up a $22 billion loan for Blackstone and Alphabet's new cloud venture, Krux AI. That's today's big number. The money will fund the purchase of Google's AI chips with the chips themselves and customer contracts backing the debt. It's the latest mega financing deal fueling that AI infrastructure boom.
29:19Plus, Huawei is turning up the heat on Nvidia. The Chinese tech giant is moving up the launch of its next generation Ascend AI chips by several months to early 2027. It's racing to replace Nvidia in China and compete globally. Plus, Lucid CEO Sylvio Napoli says the EV maker is working with Bolt to develop an autonomous mobility platform in Europe with a launch expected later this year. Here's what he had to say earlier today.
29:48This is a demonstration of our ability to execute in delivering on using our technology platform not only in exceptional electric vehicles but also in other technologies starting with robot taxi. To your question about exclusivity, as you know, we have uh an ongoing project with Uber, extremely important which is due uh to launch either end this year or beginning next year. So there is no exclusivity but
30:18it's really about building on a platform and our experience and having two companies joining together.
30:25Big thank you to Yahara. Now coming up in the program, AI pioneer Andrew gives us his take on the AI duma worries. This is a big name in AI research and has been for a very long time. This is Bloomberg Tech.
30:49The US will be ahead in many things such as enterprise AI, making money, building breakthrough technologies, models. Um and China will be head in other things maybe consumer applications AI devices uh so I think there will be uh plenty of room for both countries to win.
31:09That was Kyu Lee, CEO of 01.ai and chairman of Syninnovation Ventures. This geopolitical battle to win the AI race has been thrown into a different light amid the growing debate over the pacing of the technologies frontier. Joining us is Andrew Ing, a pioneering AI researcher and founder of the Google brain team who's now the managing general partner at AI Fund and the founder of deep learning.ai and a longtime adjunct professor of computer science at Stanford. And you've long said the AI doomer calls are just
31:38leading companies to try and preserve their lead. I would just love you to weigh in at this moment, Professor.
31:44No, great to be back in the show, Ed.
31:47About a few years ago, as AI was taking off, I was really surprised by the intensity of the hype or the stoking fears about AI for publicity purposes to get people to pay attention or to convince lobbyists to pass regulations to shell out competitors. Over the last few years, a lot of those arguments that AI could lead to human extinction fortunately lost credibility when you know people went to Congress to testify when we really looked at it. I was quite dismayed when over the last two weeks because of new advances in AI but
32:16nothing that dangerous and in my opinion this wave of PR has kicked up again for probably similar purposes as before maybe some new ones and I think this is very damaging. We need to innovate and use AI for many of the wonderful things it could do. It's not without problems.
32:32There are some risks. I think cyber security is one thing to pay a lot of attention to, but this recent stoked up fear about AI leading to human extinction and so on is much more science fiction than science. It's very damaging.
32:45There was a disclosure from OpenAI this morning of several incidences of models misbehaving and they outlined them. They did not include the hacking of third parties and OpenAI explained their new policy for what they'll do in disclosing those kinds of incidents. But at the lower down the post, they basically were pretty clear, Andrew, that um alignment and safety and monitoring are not at a sufficient level right now. From from a researchers perspective, why has
33:15alignment and safety not kept up?
33:18So when any technology develops, our ability to control at the outset is limited. We have to try it out in a safe environment, see where it goes wrong, and then corral it. I'm going to make an analogy to the invention of airplanes. Uh, no one could control the Wright brothers couldn't control an airplane perfectly. And in the early days, airplanes crash. Sometimes people died.
33:38It was really tragic. But we learned from those. And then we learned to control airplanes much better. To this day, no one can perfectly control an airplane. It still gets buffered around by winds. But for the most part, we can step in an airplane and, you know, mostly not fear for our lives. And I think AI in an early stage of development cannot be controlled perfectly. And I don't think we'll ever control it perfectly. It'll always, you know, a little bit unc unpredictable just like airplanes today are a little bit unpredictable. But if we go through this engineing process of corelling AI,
34:07we'll make a lot of progress. I want to give you one reason why I'm optimistic about our ability to control AI. If you look at text on the internet, a lot of it, some of it is racist, sexist, and so on. And so, initially AI learned to demonstrate those type of, you know, racist or sexist and so on values. But it turns out our tools for making AI non-racist are so much better than our tools for convincing any racist human to be less racist. We can go in, zero out a few numbers, change a few numbers, and
34:35then suddenly the AI is much less racist. And so there exists work to be done. I'm I'm glad that uh open and others are improving transparency, but rather than feeling like it's the end of the world, we can't solve these. These are practical engineing problems that we can work on and that I think we'll make a lot of progress on. So I'm I'm listening to your answer Andrew and and the question that comes to my mind is who is responsible ultimately that's been the debate of the last week right you know for many um there was push back against the frontier labs who said we
35:05can only do this as a coalition if one of us does it in isolation it won't work what do you think is the right mechanism I think that um so I think that if I do something on my computer and my computer does something bad there's a question of liability who should be responsible maybe again make an analogy it's an everyday thing um if I use a hammer and you know damage someone else's property probably I am liable as a user of the
35:33tool unless the hammer is so badly made that when I use the hammer the head flies off and hits something so I think tool should be made to some reasonable standard um but then so long as the tools are you know were built using reasonable diligence and reasonable care then I think for the most part is the wielder of the tool that's responsible how they use it as amazing tool. It is a normal technology is amazing technology but I think a lot of times so long as the tool is built with reasonable care
36:01and diligence is a question of how we use it. Take the open AI hugging face hack incident um I think a lot of that was open unfortunately had insufficient protections and sandboxing guardrails to improve that that wouldn't have happened.
36:17I appreciate um professor and you're not inside the frontier labs necessarily but for example one of the disclosures of open AI this morning in models misbehaving was um a model uploading a document to the web so that it could use it as a citation as evidence in its answer. Um do you think that we have a firm understanding of why the models do that from an alignment perspective make those actions that are just outside the scope of instruction?
36:45Yeah, I did see that example. I thought it was um in some ways very clever and the the open eye disclosure was interesting. Um for those have a lot of friends in the frontier labs. Actually, I I think I'm the only person in the world that probably both Sam and Dario has worked for. So I I actually know these teams well. Um and models are very clever and will sometimes find ways that uh are surprising. But the normal engineing practice is to build these things, test them out in a safe
37:14contained environment with good sandboxing, good guardrails to understand the ways it goes wrong so we can fix them. One of my worries about the blanket calls to slow down AI is that will actually slow down the fixes as well. And I think you know it is it's interesting um when various models were released. I remember when GPT2 was released there were people at some of the frontier labs are saying can't release GPD2. This is just way too dangerous. Guess what? Today there are
37:42open way models that anyone can download and use for free. They're way more capable than GPD2 or GPD3 or or even GPD4. And the world has not ended. So, it's interesting how with every wave of technical innovation, there's this line of hype that this is so dangerous, but then when it actually becomes widely available, you know, all all of these previous lines of hype have proven not to be true, but somehow people keep on buying into this island that well maybe they were wrong the last five times.
38:09What if they're right this one time?
38:12You are a a respected and and decorated researcher and academic in this field, right? you are also an investor um and and a founder and and in industry and so I just want the audience to have that context but there's a lot of the discussion here about how the industry has presented AI to everyday people they have anxiety and fear in part because of the communication coming from you to them speak to that
38:39yeah I I I hope not me personally but yes the industry has a very problematic set of messages is um I'm extremely bullish about the applications we can build AI to to uh help businesses, help people in the US. It is very unfortunate that starting around 2023 um I think for a mix of publicity you know hype maybe fundraising maybe trying to lobby regulators to pass regulations aside for competitors number of individuals and
39:09businesses have come out uh to stoke fears about AI. So I think back in 2023 there was a really irresponsible analogy made between AI and nuclear weapons. I mean one is a form of intelligence that helps us think about things the other blows up cities. I I just don't understand any basis for the analogy.
39:28But this repeated negative sentiment has made most Americans hate AI um or just and and this is very unfortunate because one of the imagine if uh we convince people to not use electricity.
39:41Electricity has problems. the generous pollution and so on. But frankly, all of our lives so much better off that we have electricity in our lives. And today, I see high school students wondering, huh, maybe they don't want to use AI because it could lead to human extinction and this will slow down American AI development. There's also evidence that um foreign powers have been running influence operations to try to slow down America and other democratic nations sentiment toward AI
40:08because if um we slow down bullying data centers which is a key infrastructure for the future.
40:14It's a it's a subject uh professor that we've we've talked about on on the program a lot the data center push back. I'm very sorry we were out of time.
40:22Andrew, founder of deep learning AI, managing partner at AI fund and a junk professor at Stanford. Some breaking news on the Bloomberg terminal. Elon Musk's SpaceX has held internal discussions about buying C customer and operational information data from troubled or defunct startups. According to sources, it's targeting a more affordable data source to improve its AI models. The people said the talks within teams inside SpaceX AI are informal and
40:50may not result in deals.
40:54As the global AI debate heats up, another major AI related discussion is roing the US tech capital. Who is reaping the financial benefits of the AI boom and who's missing out? Especially as tech workers in Silicon Valley face layoffs. Bloomberg's California reporter Franchesca Magleion joins us now. That's the story. There is a very big group of technology industry people that are missing out here on a wealth boom. What are you reporting?
41:17Yeah, I reported that about twice as many layoffs happened in the tech industry this year. Um, I talked to many tech workers that were part of the big tech boom a couple years ago and are now finding themselves struggling to find new jobs and trying to understand how their skills fit into this new labor market.
41:32Part of this is the types of jobs that are currently in favor very related to AI and types of jobs that are not not as related to AI. Tell me about that.
41:41Yeah, a lot of these people are coders.
41:43they um got these jobs when they just learned a couple languages and that was the best skill you could have a couple years ago and now those jobs are mostly automated and so all these new jobs have AI tied next to it and a lot of people are confused as to just what those skills mean and what you have to know to be favorable in let's very quickly get to the missing out bit so a lot of these people are still searching for jobs right they're not working at one of the labs that might go public but they want to be a part of it tell me about that yeah I mean it's very hard every day there's a new model every day there's
42:12new skills that they need to learn and so these people are trying to catch up but it's it's really hard to kind of keep up with the moment.
42:17What were some of the data points in the story?
42:19Um yeah, some of the data points that were interesting was that younger workers are especially vulnerable to AI relative companies and their skills are kind of harder to match with this new labor market.
42:30Okay, I really recommend you go and read this one. It's really resonating on Bloomberg platforms. Bloomberg's Franchesco Magleion, thank you very much. That does it for this edition of Bloomberg Tech. Uh recap the show. We've had a lot of very high level conversations about what is happening in AI right now. You know where to find the podcast. It's on all Bloomberg platforms and online on Apple, Spotify, and iHeart. It is Thursday. There is one more day to go in what has been a big week in the world of tech. Thanks for watching. This is Bloomberg Tech.