[Interview Transcription] OSINT in Government: Industry Insights on Challenges and Opportunities

June 12, 2024

Francis Rose of Fed Gov Today, recently sat down with DarkOwl CEO and Co-Founder, Mark Turnage, to discuss the current state of open-source intelligence (OSINT) in government. You can check out the article from Fed Gov Today here.

The link to the YouTube video, and the transcription can be found below.

NOTE: Some content has been edited for length and clarity.

Francis: Mark Turnage, Welcome. It’s great to talk to you. What’s the current state, do you think, of the government getting the data that it needs and deciding what sources it’s going to draw that data from, open sources, proprietary information and so on?

Mark: That’s a great question. And you know, I think there’s been a big change in the government in their approach to OSINT in general, and frankly, their understanding of the need for OSINT and the value of OSINT. And we live in an environment where data, broadly speaking, and OSINT, broadly speaking, is growing dramatically. The amount of data, the types of data, and so the government, in some respects, is playing catch up in trying to understand how to use it, how to aggregate it, how to analyze it. And that’s a big change that is underway. But gaps, gaps in the government’s collection. We’re [DarkOwl] a darknet data collection company. We collect data from 30,000 plus sites a day in the darknet, and we provide that to the government and other commercial users. And just that one tiny sliver of OSINT alone can tax any organization’s ability to integrate data, store it, and then manage it. So that’s it. That’s a tiny little example of some of the challenges that the government faces.

Francis: One of the things I think has been interesting about tracking this over time is that organizations, for example, like NGA, have not fought the change in the lines of delineation what used to be open or what used to be proprietary is now open-source and so on they’ve kind of said we have to get with the game and them and go with it. Has that helped, do you think, organizations in government to go through this change?

Mark: I think it’s been a big culture shift for them. I mean, NGA in particular, but other organizations as well. Take the examples of satellite data, satellite imagery. What’s available today commercially is better than what was available, on the high side, 10 years ago. And that is only going to keep happening. Using a cell phone, you can get battlefield information on the front lines in the Ukraine that’s far more detailed and far more timely than what is what then what our analysts have access to here in the US, you know from high-side data. So, I think any organization that understands that, then has to embrace it fully and start to use those commercial sources and integrate them fully into their with their high-side data. And then they’ll, then they have the best of both worlds, to be honest.

Francis: Take me farther into that definition of embracing that fully. What does that mean to those organizations to do from a tactical perspective?

Mark: Well, first of all, there’s a culture shift. I’m not sure that’s tactical, but there’s a, there’s a cultural shift that’s necessary. But once that cultural shift, once they actually understand it and get it in their DNA, I think there’s a couple of things. Number one, don’t fear it. Don’t fear open-source data. Embrace it. Buy it. Integrate it. Use it. And by the way, part of that is also staying on top of what open-source data is out there and available because it changes and it shifts dramatically as time goes on. Secondly, integrate it with your high-side data. Look at them side by side. Understand that that data, sometimes that commercially available data is better than what you have and sometimes it’s very complementary to what you have. It makes your analyst team far more powerful looking at both sets of data and correlating them together. But embracing, I think, means buying, understanding it, buying it, integrating it.

Francis: That integration process, it sounds like when you use the term changes and shifts dramatically, it sounds like that integration process may be the key factor to all of the ones that you just laid out there. Is that a fair read?

Mark: That is an absolutely fair statement. I think understanding what that technology or that tech stack is that you need to build and maintain to integrate open-source data is a journey that all the federal agencies we work with are on right now.

Francis: What does the technological underpinning of this infrastructure underpinning? And is that changing over time as well?

Mark: It’s likely to change over time, but the technological underpinning is you have to have the ability to integrate extremely large data streams, parse those data streams, store them in a secure environment, and then make them available through whatever interface or tools to your analysts that are available. You make them available in live time to your analysts. So, there are off the shelf products that allow you to do that. And obviously there are cloud data storage capability available to the government through a number of different avenues. The one interesting thing that is a challenge for many of these agencies is how do you integrate open-source data coming from the low side with high-side data? How do you cross that chasm? Because taking OSINT intelligence into a skiff, and then trying to correlate it with high-side data becomes a real challenge, you would rather have them on the same screen. So that creates a completely different technological challenge, I think, for many of these organizations.

Francis: I want to come back to that idea, but you talked about analysts and the importance of the analysts a number of times in this conversation already. What does the skill set for the analyst of the future look like potentially compared to the analyst of today given the advances that you’ve discussed?

Mark: That’s a really good question. And obviously, AI is front and center in that process. I would say that the analyst of the future needs to be able to contextualize the intelligence that they are getting. And in fact, a good chunk of that data of that intelligence they’re getting is going to be AI generated. But they have to contextualize it, and they also have to be able to keep it honest. When you have AI hallucination and other things, and you don’t have a trained analyst who doesn’t understand the context in which this is being done, you could go down a rat hole pretty quickly. So, the world of the future is going to be divided between, broadly, between people who can use AI to be more productive and those who can’t. And that’s the new social split that we’re coming to as a society, that’s no different with an analyst. They have to understand how AI works. They have to understand the data AI is looking at. They have to understand the output, and they have to then stress test that output.

Francis: You mentioned the desire to mash up high-side data with open-source data. What is the challenge potentially, if any, to maintaining, I guess, tagging is the best word I can think of, so that one knows throughout the entire data stream this piece is just for us to see and this stuff is okay for others to see when you’re combining?

Mark: When you combine those datasets, you have to tag it, you have to give them metadata so that an analyst a month out or a year out or five years out knows where that data came from, knows the source, knows the provenance of the data, and obviously can distinguish between a sentence which may have been come from high-side and a sentence that’s right, immediately adjacent to it, that came from the open-source. So that’s obviously a real challenge, but there are technical, that’s actually, I think that’s relatively solvable with metadata and tagging that’s available. If you don’t pay attention to it, going to be an analyst down the road in five years who’s going to get himself in real trouble or herself in real trouble.

Francis: Mark, it’s great to talk to you. Thanks for your time.

Mark: Really nice to talk to you as well.