Episode Overview:
Agentic AI is driving the use of modular data centers, according to Tony Grayson, former President of Compass Datacenters, and now President and General Manager of Northstar Federal and Northstar Enterprise & Defense, where he has innovated modular data centers designed to bring flexibility and efficiency to a market in which technology evolves faster than traditional large-scale data centers can keep up with.
Grayson is a well-known leader within the data center world. In a lively conversation with his friends and industry sojourners Phillip Koblence and Nabeel Mahmood, Grayson talks about the frustrating history and bright future of modular data centers and the AI applications that are finally driving demand for smaller data centers that can fit in parking lots.
As the former Commanding Officer of a US Navy submarine, Grayson shares keen insights into mission-critical energy, having been responsible for, essentially, a “submerged nuclear reactor” in which even small mistakes can lead to fatalities.
Among the key takeaways of Cool Vector’s conversation with Tony Grayson:
• Modular data centers are set to dominate deployment timelines. Grayson argues that modular deployments are becoming essential as compute demand from AI workloads outpaces hyperscale construction timelines. Instead of multi-year builds, modular units can be deployed in “three to six months,” making them critical for real-time AI inference such as fraud detection and language translation. “What modular data centers give you is a very, very quick time in the market at a very good cost on something that’s easily upgradeable,” says Grayson.
• Distributed compute is replacing “bigger is better” economics. Scale for its own sake is no longer aligned with technology or economics. Customers want smaller, controllable AI environments—sometimes literally “in the parking lot”— rather than massive centralized builds. “It’s not a how big can you build? It’s how can you build a hundred thousand of these things just for one platform and place them all around the world?” says Grayson.
• Nuclear power may lose admirers after overpromises. While nuclear power is again being floated as a solution for energy-hungry data centers, Grayson—drawing on his background as a submarine commander—warned of overpromising. He noted that advanced reactor projects face long regulatory and technical hurdles, predicting a backlash if expectations are not met. “I am worried that nuclear is gonna get a bad name when all these Gen 4s who promised delivery in a couple years never happen,” he says.
• Decomposable infrastructure will displace GPUs. Grayson highighted the looming shift from GPU-dominated architectures to decomposable infrastructure and custom ASICs, which can outperform GPUs at lower cost. This evolution will fundamentally reshape facility design and economics over the next five years. “You are getting AI right now, which is agentic machine to machine stuff,” says Grayson. “In a couple years, they’re gonna have decomposable infrastructure where basically you’re separating your CPU, your memory, your storage with optical, and then we’re gonna build a data center for that,” says Grayson
• Europe enforces sustainability as U.S. lags behind. Sustainability remains a patchy priority, with European regulators pushing strict standards while U.S. operators often give it “lip service.” Time-to-market and ROI remain dominant drivers, even if climate goals are compromised. “I think Europe is super sustainable. In the US, it’s more lip service than anything else right now,” says Grayson.
Transcript:
David Snow: Hello and welcome to Cool Vector. I’m David Snow, your host. And today we’re joined by Tony Grayson. He’s the president and general manager of North Star Enterprise and Defense, which is the division of North Star Technologies. Many of you know him as the former president of Compass Data Centers and of course he is a former commanding officer of a nuclear submarine for the US Navy, and, uh, we’re gonna get into that because there’s a connection between that important work and the important work he does in the world of digital infrastructure. We’re also joined by my co-hosts and editorial
Philip Koblence, a veteran of the data center industry with leadership roles at NYI, critical Ventures, UIA. And of course, he’s a co-founder of Nomad Futurist, We’re also joined by Nabeel Mahmood. He is a top 10 influencer and a data center, infrastructure visionary.
He’s a board member across several leading public and private companies And he is a co-founder of Nomad Futurist. the other interesting part is that the three of you know each other well. Tony is involved as well with Nomad futurist.
Well, Tony, thank you so much for coming on Cool Vector. In general we just wanna know your story in the data center industry where you see it evolving and what makes you excited for the future. But why don’t we start with sort of the breaking news, which is that your leadership role at North Star is relatively recent.
You’re gonna be involved in smaller, faster modular data centers. can you define what it is you’re working on and talk about how you see these smaller assets playing a large role in the future of the data center industry?
Tony Grayson: Yeah, so I left Oracle to run an incubated startup with Compass, really to define their modular data center market. We spend a lot of time kind of looking at what we thought the future was gonna be. I know the focus right now is really on the hyperscale, big stuff. But you know, from our perspective. The hyperscale can’t keep up with the rate of change on the language model side.
You know, going from 42 kilowatts to 132 kilowatts to 1.5 megawatts or whatever it’s gonna be. I mean, they wanna get to a gigawatt in the next couple years per rack. I mean, the data center takes two years to build. How can you keep up with that. But also, the performance is increasing. It’s been doubling every six months.
The performance of these AI accelerators or GPUs will solve the problem for you. So it’s irrelevant if you have more tokens because their performance will double. And you know, Jensen has a great slide, which kind of shows how much energy it takes to make a model.
It’s based on chat, GPT-4 of 1.8 trillion tokens. And you can see, you know, the difference between amp peer, you know, to, to the Blackwell. Really what that line is, and it really goes down. So it’s, you just don’t need as much to solve the problems. And then you add in, distributed compute, where basically you’re, you’re teaching like a child, like that’s right, that’s wrong, that can be spread over a metro campus. So we’re seeing a lot more of. distributed compute even for language models. But the real return for AI will be an inference. And I’m not talking inference, like write my term paper for me, or, you know, write a email to Nabeel that makes him sound awesome.
Uh, that kind of stuff. It’s more, real time language translation. It’s fraud detection and that all is latency sensitive or it’s bandwidth constrained. And so that gets placed close to where the user’s at. And so. That can’t be in a CoLo, it’s gonna be in a very specific area with that pure non-REST point.
So the point is, you know, what modular data centers give you is a very, very quick time in the market at a very good cost on something that’s easily upgradeable. Instead of taking years, you know, we’re deploying right now in three to six months, for example, on all these deployments we’re doing right now.
So, you know, it’s very easy to flex to a 12 year refresh cycle on GPUs, and stretch that into, what defense is doing too. And it’s the exact same conversation we’re having on both sides of it.
Nabeel Mahmood: I think, uh, just add to what Tony said, the most important thing that, keeps getting forgotten is the fact that we have not built environments that are flexible and scalable at the pace of compute change.
So modularity, to me, uh, needs to be thought through as Lego blocks, and that’s exactly what Tony is doing. take. For instance, shipping containers. I mean, they come in 20, 40 and 53 form factors. it’s a uniform factor that’s easily movable across all the continents. It’s, it’s a standard. Deployment basically. So, uh, it gives us the flexibility to move and upgrade, deploy much faster than we’ve actually ever done so before. Besides the fact that the standardization also helps with the sustainability and the climate initiatives that we continue to talk about because now we can repurpose this equipment.
I mean, they become a hard swappable assets eventually to keep, uh, with the pace of the constant move that we have started to see over the last few years.
Tony Grayson: Yeah, and we even took that a step further. We’re building everything out of composite so that composite, it’s all digital. So it’s digital designed to five access machines to robots, put walls together so we can crank ‘em out.
But also they’re a hundred percent recyclable. So if you get rid of ‘em, you just recycle ‘em and use ‘em all again in form of composite and you add some resin and you make a new module out of it.
Phillip Koblence: I could see an entire cottage industry coming out of, uh, you know, wearing pocketbooks that used to be data centers. Yeah. Um, um, in a few years, uh, look, you, we have, uh, on the front pages of every paper we have, moonshots and, and Stargate and, you know, these 400 megawatt, uh, campuses and multi gigawatt, deployments.
Around the world. It’s not even just just the country. with that as the lead, what, what is the typical type of deployment, that you’re seeing for, for one of your modules? Where does the scale from and to, and what’s the kind of sweet spot for the kind of inference based, uh, deployment?
Tony Grayson: Yeah, I think people are playing what the market wants to hear right now, where bigger is better, but it doesn’t necessarily go with the technology, nor does it go with the economics of it. We are seeing, you know, our customers are customers that are coming out of the cloud that want hybrid, multi-cloud.
They want to control what they have. And that could be a European company that’s afraid of the US Cloud Act, or that could be just a company that wants to control their own small language model inside their fence line. We are seeing, uh, small language model, so I think 4.8 megawatts and less for a Grace Blackwell, which is still 300 something million dollars.
We are seeing a agentic AI really start to pick up so. You know, think of manufacturing quality control. Think of hospitals where it’s, you know, and this is more of a PPI concern of, and a bandwidth constraint where they’re using, you know, MRIs where they’re using it against a model to look at.
Historical things that would’ve actually have been to compare to what those MRIs are. and some really interesting stuff like real-time language translation, which is basically, a rack at the base of each tower with a, hundred gig backhaul to a, a ome point, and that’s everywhere.
So, just one platform, one country could be 10,000 RAC units, and it’s just the problem that we’re gonna have to solve. It’s not a. How big can you build? It’s how can you build a hundred thousand of these things just for one platform and place ‘em all around the world? And how do you manage ‘em? I mean, it’s a very different problem set.
Phillip Koblence: And when you’re talking about within the fence line, are you talking about like someone, you know, X company has their headquarters and they’re calling Tony and they’re like, we want to manage our own large language models and we have no data center within our, office. Can you plunk one of these things in the parking lot?
Tony Grayson: Yep. Exactly. Exactly. And not necessarily the large, we’re not talking like trillion parameters. A lot of people are okay with a billion parameters, which cases they want it close, you know, inside a certain latency that they can go hug their servers. They want it inside their patrol fence line. And a lot of those we’re seeing are, you know, 4.8 megawatts blast, uh, for those kind of language models.
David Snow: You alluded to this earlier, Tony, but obviously this is an industry and this is technology that is evolving at, you know, the speed of light. And so, you know, conversation that we could have yesterday about something could have already become obsolete by today. But, uh, while you were the president of Compass Data Centers, during that period of, of evolution, of the industry and technology, what did you learn?
Tony Grayson: I was doing the edge stuff back when Edge was a naughty word, you know, I remember talking Edge and everyone kind of laughed at you and you were kind of like the, the butt of the joke ‘cause everyone’s talking, bigger is better.
And here’s little rambly, Tony talking about how the edge is coming. We will rise. Um, we were just a little bit ahead of time, uh, to be perfectly honest. And the real problem was the platform just wasn’t there. And the platform’s now agentic ai, but, you know, separately, what I learned at Compass was,
I had a great mentor, good old Chris Crosby, who everyone knows and loves. He actually let me run the business and let me fail a lot to try to figure out how to actually run a business. And so compass was less about learning how to build data centers and more about how to grow kind of from a technical, from a financial and from an operational perspective.
learning from one of the best in our industry is what I picked up outta Compass and was sad to leave it. But, you know, we have defense contracts that we’re fulfilling. and, uh, compass is Canadian dollar zone and we can’t trust the Canadians. Yeah. It’s a
Nabeel Mahmood: 51st state. Tony, come on.
Phillip Koblence: watched South Park. You can’t trust them. Yeah. The, the flapping edge, the fla are is this called the Terence and Philip [00:10:00] policy?
Tony Grayson: Exactly, exactly. I mean, they’re all eight bit up there, so,we had to be sold to a, a US based company. And so, the good thing is we still work hand in hand with Compass on a lot of things where we still use the supply chain a bunch of stuff and, and use some of their engineering aspect.
And to be honest, I asked Chris a lot of questions still, so, that was kind of the learn. But it was good to see, you know, I would’ve been very hard to run the startup under pressure of trying to make a business profitable.
What I got to learn those last four years is kind of What I thought the edge was gonna be and develop, how to really mass customize modular data centers at scale without the pressure of having to make a profit over my head.
And that’s what Compass really gave me. That’s what I’ve been doing the last four years.
Nabeel Mahmood: I think that’s like the pre-revenue discussion that, uh, the TV show Silicon Valley had for years no revenue.
David Snow: You don’t want revenue? No
Nabeel Mahmood: revenue.
David Snow: Pre-revenue. Yeah. That you don’t want people to know what [00:11:00] the actual value of the business is, so Yeah.
Tony Grayson: No, we are definitely positive and that, that’s been recent. But I mean, what I’ve really learned, technology doesn’t sell. I know it’s amazing. I act, you we had this awesome module control of your phone, but no one’s gonna pay for that. And so it really comes down to, what will people pay for?
What makes the most business sense and what’s the ROI, not what you think is cool. And, and that was, that was tough to learn. You know, I think what I was preaching four years ago I thought was super cool. But it wouldn’t have made any money. ‘cause I thought it was cool, no one else, and they weren’t gonna buy it.
Phillip Koblence: We have an awesome outtake from the previous time we attempted this, uh, where we get Tony’s real feeling on technology.
David Snow: If you recall, we, we tried to, uh, test the limits of, uh, edge technology by having you join from a, I believe, a moving car.
David Snow: [ INSERT TONY CLIP “flashback” FROM FAILED RECORDING SESSION]
David Snow: [CAPTION ACROSS THE BOTTOM OF THE SCREEN SAYS: Initial recording session, two weeks earlier ]
Tony Grayson: The question is, does technology control us? Do we control technology? And on the business side?
And I think we get often get it wrong.
One of the reasons we started this platform, and to a certain extent even the nomad futurist platform, is this concept of, trying to understand that 30,000 foot view. and one of the, the things that’s really interesting about this transition and, and what you’re seeing right now with, data centers that initially it was the connectivity that, that brought the data center to the place.
Phillip Koblence: Now it’s the power that, and you have all these powered land folks, um, and, and you have a bunch of these kind of new entrants into that data center space from the real estate side, certainly. From the construction, uh, and engineering side. and you really got to see both sides of it. How, how to kind of apply, um, a, a data center mindset to a construction company.
Yeah. And how to apply kind of a construction mindset to a data center company and then you have, you know, a bunch of utilities that are trying to figure out, you know, what, what language data, data centers are speaking and then everything in between. So I guess the age old question is, is it easier to teach a construction guy how a data center works or to teach a data center guy how to construct something?
Tony Grayson: I think it’s equally hard on both actually. I would say it’s, equally hard, but when you add the networking piece into it, that’s when you. That’s when you blow the fuse, just two
Phillip Koblence: Cups in a string, Tony, come on. Um,
Tony Grayson: The, um, and then it’s the latency piece. It’s the peer and OnRamp piece. It’s the ISP piece. It’s that piece I think is what makes it.
Phillip Koblence: And, and, and I’m struck by like when you started describing what. You know, the use cases for what you guys are developing, you really got into the nitty gritty of, the, kind of the ai, tokens and how the various processors, work and, and the queries and and, and all that.
Phillip Koblence: So how does that play in, how does that, how does a nuclear Navy sub guy start talking about, you know, AI Blackwell processors quoting Jensen on a first name basis, et cetera?
Tony Grayson: Me and Jenen were like this man, I,
Phillip Koblence: I don’t see the other finger.
Tony Grayson: All I saw you is perspective bill. Perspective. Uh, you know, I think the Navy is interesting, especially the submarine force.
‘Cause you do have 14 megawatts of IT loads, so you kind of understand how the platform acts. So I thought I was just a facility guy coming out of the Navy, but I kind of understood how networking is or. How containers work, how the processors work, how everything kind of fits together. And so I think that’s kind of the path it’s been, the data centers are cool to me, but I think the solution architecture is also really interesting to me.
How all the pieces, how do the servers, how does the network, how does the platform that runs on top of it fit in? And that’s how I think I kind of get into this. I would never be able to sell a module or lease a module if it was just by myself most of these times. We’re talking with Dell or HPE, or WWT or Super Micro, we’re talking to the, the network side like Colt or KDDI.
So it’s, it ends up being a turnkey solution and [00:15:00] we might be subbed under. On someone else’s paper or each one might have their own little cost to it. But we are finding very few people who just say, build me a module. It’s more, Hey, we had this idea of this need. Can you help us out? And you have to bring all the pieces and parts into it.
Tony Grayson: I think that’s interesting.
Phillip Koblence: Ultimately you’re, they’re spending millions of dollars, if not. If not more than similar to what they’re spending on the data center they’re spending on the equipment that’s going inside that data center.
Tony Grayson: Exactly. And, and then you can ask them the questions like, where are you gonna be in two years?
What do you need to do when you know you have to replace these hoppers in a year? If you really want be up that tight end, you know, what, what are you gonna charge? How can we lower this cost? What’s do you really need to be at m plus one or two? And so you’re able to bring all that kind of stuff in.
And everything we’re doing is Lego blocks, like Nabeel said. And so it’s. We’ve learned a lot from other customers and know what works and what doesn’t work, and we try to bring those lessons learned into what customers are. But we’re also learning too, like we’re doing a lot of stuff with custom asics now.
Um, and I’ve been very surprised that what they’re able to deliver compared to A GPU. And you know, the best example I have right now is one we’re working with. It’s 14 kilowatts a rack. It’s about a fourth of cost of a B 200, but it does 35,000 tokens per second on about size of 1 28 compared to a B 200, which says 15,000.
So if I was to actually just, you know, put $2 per million tokens, that one makes $2.1 million per year. Full tilt on a B 200 break’s, $900,000. So on a rack that’s a quarter of expensive, it actually makes more than double the amount of of tokens and that’s on a custom asic. So I think that’s the kind of stuff I think is super interesting.
We’ve been bringing that solution to a lot of people who’ve been trying to develop their own custom asics. And there’s plenty of these companies out there, they’ve already solved the problem. they can provide you the custom asic, you just need to put the platform on top of it.
Tony Grayson: You don’t have to solve.
Nabeel Mahmood: Do you believe that the conversation being moved from facilities and operational teams to IT and finance is changing the dynamics?
Tony Grayson: I think it’s changed the dynamics because I think at least. On the hyperscale. We seem to be following the real estate mindset still a little bit, which is find the cheap land, build the big data center.
And I’m not quite sure that’s the answer now. I think that was the answer a couple years ago. that’s changing a lot of the decisions right now and where the investors are going and what infrastructure people are trying to invest in. I think that might be delayed though, and I think we might be switching back to technology.
Which is this network centric architecture of, of what’s gonna drive the future and not necessarily, yes, we still gonna have these hyperscale data centers. I’m not saying that’s going away, but I don’t think there’s gonna be as many, and they might be more focused on storage and inference that’s not latency sensitive.
Tony Grayson: It’s not gonna be focused on what the, the true money driver is gonna be, which is definitely a more technology platform focus than it is as a, a financial focus.
Nabeel Mahmood: And how about, uh, the future of compute? I mean, for the markets that you’re servicing? Uh, I mean, are there discussions for, you know, from a modularity and scalability perspective to convert these assets into operating expenses versus. It being, a capital expense?
Tony Grayson: Yeah. I mean, I think we, we get both of ‘em. So we’re probably 50 50 now on opex and CapEx. You see opex people who do wanna de-risk. And so we don’t know what the future is gonna be and we can easily upgrade our stuff. And let’s say we assign a three year lease and. You know, I’ll just take up their H 100 and drop a GB 200 in the same place down the road.
It just de-risks everything. But we’re seeing some people who, you know, they wanna own that, that capital investment and have it on their books. So I think it’s. It’s a little bit of both, but you know, we’re, we’re definitely seeing a lot more people who are interested in de-risking. I mean, you saw what, what meta did when they’re selling back.
Tony Grayson: Mm-hmm. You know, you can look at it from two perspectives. They’re, you know, they’re selling back ‘cause they’ve all been working with Blue Owl and that kind of stuff. I look at it as a way to de-risk, you know, now they can sign a seven year [00:19:00] and walk away if they need to, if the future’s not gonna be with what they’re gonna do and they can free up some of that capital to spend more on chips and everything.
Tony Grayson: Um, so I think it’s, it’s that balance. But you know, to me it’s, There’s really no point retrofitting anything anymore. Like, why would you take a 4 to 6 kilowatt data center, retrofit it for 40? ‘Cause guess what? By the time you’re done retrofitting, they’re gonna be at like a megawatt. So you’re better off just dropping stuff in your parking lot and, ‘cause I mean, some of these buildings are very, I mean, they’re small. I think like a 4.8.
Phillip Koblence: it doesn’t sound like you have a vested interest into whether,
Tony Grayson: Yes, I have a vested interest and the parking lot, but 4.8 megawatts is only a hundred feet by a hundred feet, and so you’re either gonna waste a bunch of space in there, um, and you’re gonna be really risked, or just build for what you need. I mean, we’re cracking these things out in six to nine months. And so just build for what you need, don’t overbuild anymore. I think that’s gonna be the, the difference.
Nabeel Mahmood: That’s the driver. So Tony, I mean, you’ve been at it for a while and you’re coming from two different perspectives. The thing that we continually forget, but we talk about unintentionally or intentionally for marketing purposes, sustainability and the Climate promise to, to make this planet wonderful. Have you done any study pertaining to modularity, scalability, build out strategies versus a conventional brick and mortar build it.
They’ll come and have all this stranded capacity and efficiency ratios being totally skewed?
Tony Grayson: I mean, I, I think a build it and they will come. I don’t think you can, can count on it anymore. Yes, there’s a need right now, but how much of that need is pent up, you know, years behind and how much of it is actually a new need that’s kind of coming up.
but I also think to, your point kind of came with sustainability. You brought up that too. I think that’s a, that has totally hit and miss. I think Europe is super sustainable. I think we, in the US. It’s [00:21:00] more lip service than anything else right now. There are some states you can’t get away with it.
Like the West coast states. you’re gonna have to be able to talk about that. But everything else out of that is just lip service. It’s, it’s secondary to overall cost and, and return on investment.
Phillip Koblence: Um, and time to market. I mean, right now, and time and time to market. we’re in a, we’re in a race, right?
So you’re in a race. the first thing to go out the window are, you know, uh, climate, concerns or, or any of those things because, we just wanna win at all costs and unfortunately the cost is gonna be our grandchildren’s future. Oh, well,
Tony Grayson: But I mean, in Europe you don’t have a choice in that. I mean, they’re making you do energy metrics, they’re making you track everything on that stuff to apply for that stuff. And I think that’s super interesting. ‘cause if you design through that, then you can answer and
Phillip Koblence: yYou get all of August off.
Tony Grayson: And, and you get all August off, you can work, work three hours a week and, you know, all that kind of stuff.
Phillip Koblence: So here’s, here’s a question just to you specifically, which is, and maybe, maybe just maybe, we’ll, we’ll allow David to get a word in edgewise, but if you, you, you have you building all these things for folks that are saying, you know, it needs to be, um, you know, super redundant and n plus one, n plus two maybe tier four is a thing in the past because that was a lie anyway.
But it needs to be tier three and we need to have multiple. Fred is, and you’re coming to them
Unlike nuclear submarines, no one dies when data centers go dark knowing what it’s like to live on a nuclear submarine. do you ever find it maybe comical is the wrong word, but cute that they think that their workloads, have this, you know, super high uptime requirement when, you know what that kind of literally pressure cooker uis like
Tony Grayson: Well, the, you know, if the data center goes down, yeah. It’s bad for ROI. No one’s gonna die. And so that’s what I think the difference is and on maintenance on a submarine, if you mess it up, someone’s gonna die. Maintenance on a, you might drop a rack and ‘oh darn.’ No one’s gonna die from that perspective. So I think unfortunately that’s the difference in operations, but I also think that’s how, you train your teams on, on that kind of stuff.
You know, there has to be some real-world implications on how you run your data center. but they have to be, have a vested interest in it and treat it like, you know, the ship’s gonna go down. But I mean, we are seeing a lot of moves away from that though, that n plus one and stuff like some of these, uh, l like they don’t care if it goes down.
They just want a graceful shut down. And if it shuts down for an hour, that’s fine. If it shuts down for two hours, like, but that’s language models. Agentic AI we’re back to high,operability because that stuff is what’s making the money. if you want your Tesla not hitting things on autopilot, yes, I know it does, but when it gets to the future and hopefully it doesn’t hit anything anymore, then that’s, you don’t want your, your local data centers driving your car to go down.
Um, ‘cause that could be a safety of life.
David Snow: Not to mention that when people lose connectivity, they’re more likely to murder, people. But,
Tony Grayson: Or no, they can’t murder because they can’t [00:24:00] think for themselves without, I’m not getting, you have to dust off and it’s, no, it, it’s more like Chat GPT if I want do this. I mean, you, you’ve seen the South Park recently where it’s like, write a business plan for me on my targety on,
Phillip Koblence: I’m gonna guess like a close second to Chris Crosby are the writers of South Park in terms of what informs your daily.
Tony Grayson: I think they’re poking fun at the right stuff, to be perfectly honest, I think.
Nabeel Mahmood: Mm-hmm.
Tony Grayson: This whole thing, uh, we’re relying on open ai, like open ai. What, what I think is amazing, it’s a probabilistic model.
It like, if you were to ask it 50 states, the United States, it has no clue what the United States is. It has no clue what a state is. It’s just telling you what you think you want to hear.
Phillip Koblence: Do you think that there might be a time where, AI does develop some level of. consciousness or, or whatever the equivalent is, and then the machines take over and then it’s, you know, Arnold Schwarzenegger coming at you with a bazooka.
Tony Grayson: I have no clue. I don’t think it’s gonna happen anytime in near term. I do think you are getting AI right now, which is agentic machine to machine stuff. But here’s, here’s the funny thing we’re not talking about for these data centers. We’re talking about GPUs in a couple years, they’re gonna have de compostable infrastructure where basically you’re separating your CPU,your memory, your storage with optical, and then we’re gonna build a data center for that.
And then quantum computing’s gonna be here, and then we’re gonna build a data center for that. So. I mean,
Nabeel Mahmood: Business is good.
Tony Grayson: Life’s good. No, it’s business is great for business
Phillip Koblence: life. Life is, life is great when you can, uh, when you can just buy the, the right Lego for whatever the infrastructure evolution
Tony Grayson: Exactly what has happened, like these 24 x multiples in guaranteed revenue for 30 years. I think it’s a, a thing of the past now. It’s changing too quickly and yes, we’re worried about AI right now, but you’ll have this decomposable infrastructure. Next, you’ll have quantum computing next and that those are gonna be the next problems we’re gonna have to solve. And we’re talking in the next five to six years. Uh, the good thing is we’re not gonna have to worry about fission power in a data center anytime soon. And we’re definitely not gonna have to worry about fusion powered data center anytime soon, so.
Nabeel Mahmood: So, Tony, couple things. I mean, uh, so, so one of the points that Phil actually made earlier about, uh, computers getting smarter than us, I think it’s not that they’re gonna get smarter than us, that the challenge is that we are gonna get dumber than them. Oh no. It’s,
Tony Grayson: we rely on that stuff way, way, way
Phillip Koblence: as evidence by the fact that nobody in my house can kill me.
‘Cause they can’t figure out how to do it.
Tony Grayson: Exactly. It’s, you cut off all that, that your wifi access and you can’t do anything.
Nabeel Mahmood: Exactly. Uh, one question that Phil asked earlier about the tier levels, uh, as it entails to, you know, uptime IDCA and others for that matter with their tier rating. Uh, do you think as you move forward that tier rating at the facility levels is probably not as important as it would be at the application stack?
Tony Grayson: I think it’s at the application stack. I think it’s when you’re building this much distributed computer, I think it’s at the application stack and not necessarily at the data center level. I mean, to be honest. [00:27:00] I haven’t been asked what tier level we designed to for the last two years. No one’s asked me at all about that.
In fact, I’m the one going, do you want it concurrently maintainable, or do you not want to? What does that mean? You know, do you want a maintenance bypass? Do you want a maintenance bypass? We, or you want A and B? Why do you want A and B? Like it’s like those concepts aren’t even coming up anymore, at least in now.
Could definitely, probably coming up in some form or fashion with big data centers and that kind of stuff. But what we’re doing, I haven’t been asked in a long time.
Phillip Koblence: I suggest that we make the maintenance bypass a, uh, just a standard.
Tony Grayson: Yes. Better? Yes. I will say, UPSs, put the stupid maintenance bypass in your UPS. Don’t make me spend $20,000 to add a stupid switch onto your stupid UPS. Just. Yes.
Phillip Koblence: Oh, Tony’s getting upset again.
David Snow: Add it in there. This, uh, disparagement of technology needs to stop. Um,
Phillip Koblence: it’s
Tony Grayson: And it’s 20 grand. 20 grand for a stupid maintenance. Come on.
Nabeel Mahmood: Sorry. It should be, should be a part of it. On that note, um, you know, that leads into like a theory that I’ve always had for the last, at least for the last two decades, that
Nabeel Mahmood: The physical layer is going to be commoditized to the degree that it wouldn’t really matter. where is the data center or whose data center it is because everything is going to happen at the application stack. No. Hundred percent agree.
Tony Grayson: And we, we are seeing that with some of these applications that people are putting out there right now. And, and they’re designed that way that it is at the application stack. And it’s less, when you deploy 10,000 sites. It’s not at the, at the data center level, one of those can get out and you’re, it’s gonna have a backup.
So I agree a hundred percent. It’s gonna be at the application layer.
Nabeel Mahmood: so, does that point data centers become irrelevant? I mean, are we going to be like just simple like gas stations that no one’s gonna look at? No one’s gonna care. We just go in pump gas and leave.
Tony Grayson: Aren’t we already that though? We’re kind of like
Phillip Koblence: The only thing we’re missing are the powdered donuts.
Tony Grayson: Exactly, exactly.
David Snow Tony, I would like to go back to an earlier comment you made about. how while you are at Compass, and telling people that you were working on edge data centers, they kind of dismissed, or there’s, uh, sometimes a dismissive attitude. Of course, we know that, the way that technology evolves, it often starts out as being sort of a curiosity or novelty. You know, YouTube was originally thought of as a place where you could watch cat videos, and then it turns into something that completely swallows up the, the legacy industry.
David Snow: Why were people dismissive of edge data centers a few years ago?
Tony Grayson: It wasn’t making money. It, I mean, it’s, it all comes down to what makes the money, you know, there’s Edge wasn’t making money, there was no application that needed an edge. Nothing was necessarily latencysensitive. this stuff just wasn’t there.
But you are starting to see applications now that do make money, that are latency sensitive, that are edgy, you know, in what they’re talking about. And it just didn’t have, nothing was making money on it, so no one really wanted to talk about it. it’s not cool or fun to talk about things that make sense because that’s just, that doesn’t make the news.
I was like the sideshow, like the two headed person. I was like the, oh, here’s Tony with the edge. Like, you know what, let Tony jump up and down, let him talk edge. Um, but it was, there was no way to make money on it, you know, and there were plenty of businesses that had failed and so it was just, Hey, Tony’s just doing another business that will ultimately fail.
Just like the long line of edgy kind of businesses that were out there.
Phillip Koblence: I like the phrase, nobody likes to talk about things that make sense ‘cause they only like to talk about the things that make dollars. Um, yes, we should. Uh, brilliant. I think that needs to be copyrighted.
So, you come outta the Navy, you went from the Navy, uh, Oracle was your first entry into like the, it was not no meta,
Tony Grayson: Facebook was Meta to Oracle.
Phillip Koblence: around, uh, what year was that like when you, when you got outta the Navy?
Tony Grayson: 2016 to 2018 was meta.
Phillip Koblence: You have been in this industry, um, for nine years. Nine years,
Nabeel Mahmood: I have been in this industry three times longer than you. three and a half.
Phillip Koblence: So, to who’s, thank you. to, what do you, kind of credit. Your ability to have such a rise in terms of the visibility of your platforms.
I mean, meta as a starting point is amazing, right? Not everybody can just decide to start their, digital infrastructure experience and say, you know what? I’m just gonna go work for Meta ‘cause why not? Um, so
Tony Grayson: I got lucky there.
Phillip Koblence: Is it a particularly characteristic that you have where the stuff interests you? Is it that kind of nuclear submarine training that just was a perfect fit, which is why we see like a, a, a pipeline of talent coming in from, you know, that element of military service among others?
Tony Grayson: I mean, I, I think it’s on a super technical nerd. I like to know how things operate. You know, I was building my own computers back in the day and I like to go deep on how that kind of stuff work.
And it was more than just. You know the mechanical, electrical piece, it was. what’s the capacitor or how they’re built, how the chips uh built, how does the memory interact with that? So then you add in the nuke in me, which is the question, everything, and think everything is BS and, and to, you know, give my opinion on everything.
But I think there’s a lot of us out there. I think I just got super lucky.
Phillip Koblence: I was on a panel once and it was, it was years ago. It was before this current moment that we’re in. and someone asked me why I thought it was so delayed that nuclear power, uh, was, gonna take hold and uh, and, and grow in the us. And my answer on that panel was, um, as long as the most popular nuclear technician in the United States is Homer Simpsonno one is gonna trust, the nuclear way of life. Now, tongue in cheek, of course, um, uh, Homer Simpson is a fictional character who’s not, uh, really, uh, eating donuts on the job. But, um, I, I hate donuts,
I’m sure, I’m sure I can see, and probably sneak into bushes. Sneak into bushes and out of them. Um, but, um, you actually operated a nuclear powered, basically a floating nuclear power plant.
Tony Grayson: Yes.
Phillip Koblence: Um, and we are submerged! Submerged not, it wasn’t even floating.
Tony Grayson: sign, I had to sign for it every Right. You know, I had to sign for it when I took over. Um, so I signed for a cohort.
Phillip Koblence: So we’re in a, a moment where people are talking about SMRs.
Nuclear is having a moment. not just within the US, but globally. So as someone, the only person in any of these conversations that has actually operated a submerged nuclear power plant, what do you think of those, technologies?
Do you think that, do you think they’re, needed? Real, a part of the overall solution, or do you think the moment is gonna pass them by where, the demand is gonna change?
Tony Grayson: I think they’re definitely needed.
Tony Grayson: I am worried that the nuclear is gonna get a bad name when all these gen fours who promised delivery in a couple years never happen. And that’s what worries me is everyone’s gonna have this backlash against nuclear saying, oh, they promised they could do this, and they can’t do it. And there’s no way they, they will do it.
We’ve forgotten what it actually takes to build a nuclear power plant. You have to come up with a design. You have to get that design approved. You have to build a test reactor. You have to run, said test reactor for two years. To get data, you have to submit a final design to N-R-C-N-R-C looks through yours.
They give you permission and you have to build a reactor. We’ve taken 30 years to build reactors that we actually knew the design for. None of this has changed. Like it doesn’t matter. If you really want an SMR quickly, you would build a gen three or Gen three plus. Something we know about pressurized water reactor, whether it’s a Westinghouse or whether it’s GE Hitachi, that kind of stuff.
You can blame on RCO you want, but if you’ve never [00:35:00] built your test reactor to demonstrate your design works, why would you ever think you’re gonna get that? And if you depend on fuel that we can’t build right now, unless it comes from Russia or, or there’s 50 companies fighting over this, you know, kilogram of fuel that comes out every year.
Like it just. It’s not gonna happen. and the backlash, I’m really worried will be against nuclear and then we’ll be back to trying to burn coal and forget about how wonderful nuclear is.
Phillip Koblence: Over promising. Over promising, yeah.
Tony Grayson: The o overpromising they will under deliver big time.
I am surprised we aren’t talking about micro reacts though. ‘cause micro reacts we’ve been working on for a long time. they’re a very simple design. They aren’t very complex.
Those first ones will be coming out in 2027 and think. Five megawatt nuclear batteries that never be refueled and they just sit there for eight years and you swap ‘em out. Those, to me, are a very near term thing that we should be talking a lot more about instead of these SMRs that are probably, and
Phillip Koblence: Ironically, the perfect form factor to power a, um, an edge data center plugged in your, in your parking lot.
Tony Grayson: I didn’t put that. Is that, is that, I guess the math amazing. I guess it does work. Yeah, it does add up. No, and I’m not saying that just because it, it’s just they’ve been working on it for a long time. It’s a very simple design where these gen fours are very complex designs that require a lot of testing no matter what you’re using.
Nabeel Mahmood: Do you think we’ll get to fission sooner than SMR and nuclear being streamlined?
Tony Grayson: I think we’re definitely doing a lot of these, these gen three before I think fusion is coming down the road.
I just dunno how far, it’s very tough to figure out how far. We’ve come like the, I love the thing, a year ago when we had this fusion. and everyone was all excited about it until you actually did the math and realized you put a lot more energy in to make the fusion happen, so you actually lost energy to make it.
So it, it’s, I’m glad we did it for that long, but it’s still lost energy. So I don’t know how far along we are. it’s very tough to read through the lines and I’m sure people are safeguarding that, you know, their IP. but I would love to see that happen. That makes sense. We should be able to do it.
Nabeel Mahmood: And in the term, I mean, do you think there’s a play for like hydrogen, geothermal, and other potential?
Tony Grayson: Oh yeah. I think there’s huge plays for that kind of stuff right now. When nuclear doesn’t pan out, people are gonna start freaking because they made massive builds based on nuclear, and they’re gonna need something else, and they’re gonna need geothermal, they’re gonna need hydro, they’re gonna need natural gas, all this kind of stuff to be able to fill in their, what their business plans they’re gonna be.
And hopefully these people who are dependent on nuclear have a backup plan when their plans don’t patch out that they have the, you know, a hundred megawatts by 2028 or something like that.
David Snow: I wanna talk about. The kinds of investments that businesses are making, in, edge, computing, and you know, basically your, your customers. Um, I talk to a lot of private equity people and they say that they are [00:38:00] not ready to start putting bets on AI and where AI will take businesses because they don’t yet really have a read on where these businesses are spending their money and investing their money.
In a way that, allows AI to be integrated into their businesses. Do you have insight into that, uh, based on the, uh, edge data centers you’re installing?
Tony Grayson: I mean, I think it’s still hit and miss on what they’re trying to figure out. I mean, I think there’s really no good platforms out there. People are starting to put those platforms out there.
But I think if you look at China, China is interesting to me just because they didn’t have the good chip sets. So they were actually forced to innovate, to make DeepSeek. And because they have that, I think AIs weren’t integrated in their day-to-day. Like they’re actually developing platforms that run it, and it’s more than just API calls to open AI and write my paper for me, or something like that.
It’s actually, I think. Whole list of stuff like can do, like they improved traffic inone of their cities. But like 30% use in AI and stuff like that. It’s more so it’s more day to day and they don’t like, I think if you were to ask some of the newer generations, they probably laugh at AI right now because it, it’s just a subscription based service that old gen people like me, like to use to write emails or whatever.
They don’t see it actually in their day to day. Use it all unless it’s making pictures and all that. And so I think it’s, once we make that pivot to actually putting it into platforms that actually generate revenue, I think that’s when it’ll become more interesting. But we’re just not there yet. I mean, go look at if, if any of us were CEO of OpenAI, we would’ve been fired.
I mean, I don’t know any company that could lose billions and billions of dollars and still get a bunch of more money into it and say we need to build bigger. Um, it’s just, to me, it’s not a winning revenue strategy. Now, I could be wrong, but it doesn’t make sense to me at all.
Nabeel Mahmood: Um, well, Bezos tried it.
Yeah, I mean, Bezos was negative, uh, for quite some time.
Tony Grayson: And look, look at the Metaverse. I mean it’s, you can’t just throw money against the wall and hope to, hope to God. Some awesome service comes out of it. You actually have to have a platform you’re trying to get to. And subscription based services are not a source of money ‘cause they can always transfer your subscription.
Tony Grayson: I think it’s gonna be interesting to see how this all kind of pans out, but I am starting to see platforms. Agentic AI platforms start to happen, um, based on their own models, not based on what’s out there, right there. you know, DeepSeek is a, an open source model. Now you’re gonna give DeepSeek the best chips. They have the best algorithms. What happens when someone releases an open AI equivalent? That’s all open source.
David Snow: When you are, you know, interacting with people who don’t know a thing about the digital infrastructure world, data centers, maybe you’re at a cocktail party, you’re at a backyard barbecue. They ask you what you do and why you like what you do, what, what do you say? I.
Tony Grayson: I just say, I do AI things. They look at me and they said, so you write, you help us write emails or something. And I just say, yes,
Nabeel Mahmood: I love it. What do you do for a living? I do AI things. Yeah,
Phillip Koblence: AI things. AI things. Oh, oh, emails. Yes.
Nabeel Mahmood: Amazing!
Tony Grayson: I just say, you know, I build data centers to things that hold your servers so you can write your emails and stuff like that, but no one really wants to talk to me about it. ‘cause then I’ll. I’ll, on the next question, I’ll start super nerding out and people just kind of like, oh, there’s my friend over there.
And then I’m sitting in the corner by myself mumbling, you know, rocking back and forth.
Phillip Koblence: Our people have a, uh, a word for that. We call it dovening. we were at the, uh, data center Frontier Trends Summit, uh, last week and we missed you.
Tony Grayson: Um yes, I was, uh, helping strengthen our defense worldwide.
Phillip Koblence: Okay. Thank you so much for your service. both in and out of, uh, the military, um. Chris Downey gave the keynote, um, and he said up until a few years ago when people asked him what he did for a living and he told them their eyes would just glaze over and they would change the subject.
Now he says, you know, he’s in the data center business and people come up to him and they congratulate him for being in this business that’s so hot and making the front page of all the papers. Um, and look, everyone’s understanding of it is if that paper thin. Right. You can’t, no. It’s peel the, once you peel the onion, you start crying.
There’s no doubt. But have you found that, not withstanding, you saying you do AI things, ‘I have 500 acres. Help me make a billion dollars.’ People now, they don’t glaze over. When you say ai, they say, oh my god, ai.
Tony Grayson: No. They all say, I have 500 acres. Help me make a billion dollars. It’s like all the crazies are out there. I’m like, Dude, those days are gone.
It’s like, help me find customers. I have a hundred megawatts I can get in three years. I’m like, that’s great. You have a big dirt lot. That’s awesome. I mean, it’s, the crazies are coming out of the woodwork right now. All these, you know, people that have bought land and they’re trying to turn that into customers and it’s very different than it was.
So, I have a lot of these conversations going, um, it’s different now. those days might be passed and I’m glad you have a large plot of land and that’s awesome.
Phillip Koblence: Well, they’re also, they’re also pitching the wrong guy as we’ve discussed. Tony gets excited about parking lots.
Tony Grayson: Yeah. Don’t, don’t talk to me about that kind of stuff. But it’s, I think it’s, it’s interesting but I think people do confuse me with the hyperscale stuff. ‘cause then I just say, I don’t do that and then they don’t get interested ‘cause I won’t make ‘em billions of dollars. Um, so.
David Snow: Awesome. Well, there’s so much more to talk about. We’ll have to have you back in the future, Tony, but, uh, we’re gonna let you go because you are integral to our national defense and we want you to, get back to your day job.
Tony Grayson: But, uh, we are doingsome cool stuff in defense wise, which I wish I could tell you about, but I can’t.
Phillip Koblence: Uh, you know what? I’ve met you long enough to know that after a couple drinks, you’ll tell me anything. That’s all it takes.
David Snow: Alright. The ne the next episode will be, uh, alcohol, mandatory and we’ll, uh, and when it becomes front page news, we’ll just see it the following week on South Park.
Phillip Koblence: It’ll be great. It, it will be on South Park, definitely, I’m sure.
David Snow: Awesome. Well, uh, Tony Grayson, president and general manager of North Star Enterprise and Defense. Phil Koblence and Nabeel Mahmood, both co-founders of Nomad Futurist. Thank you for being on Cool V and we’re definitely gonna be, uh, talking again.
David Snow: And, and Tony, we’re definitely gonna have you back.
Tony Grayson: No, thanks for having me and always great to talk and hopefully I didn’t ramble too much.
