How to Identify High-Quality Digital Health Apps: The DIME Seal Explained

[00:00:02] Dear listeners, welcome to Faces of Digital Health, a podcast about digital health and how healthcare systems around the world adopt technology, with me, Tjasa Zajc.

[00:00:16] Software development in digital health feels increasingly complex, resembling a batch collection process where vendors seek AI, compliance and other certifications from various organizations to stand out in the market. Nonetheless, as patients we do want to feel assured that what we get is quality for health improvement when we use software solution.

[00:00:43] So in today's episode, we are going to talk about the DIME Seal and the challenges related to certification processes and making sure that clinicians and hospitals have an easier way of figuring out what to use in their clinical services. clinical practice.

[00:01:04] You will hear from the Vice President of Digital Medicine Society or DIME Douglas Mirsky and Boston's Children's Hospital SVP and Chief Innovation Officer John Brownstein. They talked about the DIME Seal, a certification designed to help hospitals and clinicians identify trustworthy digital health software.

[00:01:26] And we also discussed how do hospitals approach improving innovation and introducing innovation in the clinical practice. Enjoy the show. Enjoy the show. And if you haven't yet, also check out our newsletter, which you can find at FODH.substack.com. That's FODH.substack.com.

[00:01:49] I only publish it roughly once a month or every second month with an in-depth commentary on a specific topic. I wrote about how advancing digital transformation requires strategic planning, the need for AI to start with quality data, pharma and digital health, why countries struggle with healthcare digitalization, and more.

[00:02:16] So go to FODH.substack.com and also leave a rating or a review wherever you listen to your podcast. We also have a YouTube channel. So just go to YouTube, type in Faces of Digital Health and see the interviews in a video format. Now let's dive in.

[00:02:54] Doug and John, hi, and thank you so much for joining me here on Faces of Digital Health to talk about a topic that has probably been interesting since the beginning of digital health apps. And that is how can we differentiate the apps and digital health solutions in terms of their quality? Throughout the years, we have developed various frameworks.

[00:03:19] So digital therapeutics have started to get separated from all the other applications and there's a lot of work being done to support their reimbursement, their certification and etc. But actually, DIME, the Digital Medicine Society, which is a non-profit organization that strives for the enhancement of healthcare with digital technology,

[00:03:42] we recently started an additional program on the so-called DIME seal, where basically you can also kind of evaluate applications. So it's easier for doctors to have a better idea of if something that they have in front of them is good or not.

[00:04:03] Maybe just before we really dive into the DIME seal, Doug, could you very briefly outline where is DIME at the moment? How has the organization developed? And where do you see the biggest challenges? Yeah, very nice to meet you. Thanks for having me. I'm great to be on here with John. John, we at the Digital Medicine Society, we're a mission-driven, not-for-profit.

[00:04:31] And I think that's important to the conversation today because we're doing this to make things better, not necessarily to pad our pockets. With the Digital Medicine Society, our whole focus is to advance the safe, effective and equitable use of digital approaches to redefine healthcare and improve lives. All of the things that we're doing right now are focused on that. We have work going on to enhance and improve, and actually John would know about this as well, pediatric care.

[00:04:58] We're working on improving cytokine release syndrome from a digital perspective. We are starting a project on figuring out how to establish the best practices and playbooks for connected health, which is really increasing in volume and need to get more patients out where they live versus having to bring them in.

[00:05:19] So we're actively engaged with the market right now to try and bring more digital health to more patients through their providers, through their health systems more effectively than ever before. John, when you hear somebody say that we're trying to get more digital into the clinical practice or in healthcare, what's your first association? How does your body react to that?

[00:05:45] Are you excited or are you, I don't know, what happens? Running digital innovation at Boston Children's, clearly I'm biased and very enthusiastic. Regardless of whatever my reaction is, there's a massive amount of demand. There's clearly where the puck is going. I'm Canadian, so I have to talk about it in a hockey reference.

[00:06:06] And the increase in digital tools is part of where medicine is going to create efficiencies, reduce errors, improve diagnostics, to eliminate mass amounts of costs that exist in our health systems. It is clear that these technologies are part of the solution. But there's a challenge in understanding sort of which technologies to select, which ones are going to create the most amount of value, which ones are the most secure.

[00:06:36] There are a number of different facets to that. So it's both, if you ask me what my reaction is, of course, it's enthusiasm, but it's also there's a lot of anxiety, broadly speaking, because we are dealing with an onslaught of new innovations. Funding in this space has grown exponentially. And so trying to keep up with what's happening becomes almost impossible for a health system. And this is where DIME is playing a really important role in the ecosystem.

[00:07:02] As you mentioned, you're the chief innovation officer at Boston's Children's Hospital. Can you talk a little bit about how do you currently evaluate innovation? How do you address all the challenges that you mentioned? So assessing, deciding what makes sense and what not, differentiating between everything that's on the market, deciding probably also what you're going to support and innovate in the institution.

[00:07:32] Well, absolutely. I mean, there's a number of factors. First of all, we have an effort on our team that is looking at the top of the funnel, trying to understand sort of enterprise priorities for the coming year and where tools on the market might intersect, whether that's thinking about behavioral health or thinking about remote patient monitoring or care at home. There are all these components of areas that many health systems are trying to drive forward in.

[00:07:57] So we're trying to align our strategic objectives with potential solutions on the market. Of course, when we look at most times, there are times when there are gaps in the market and we need to build and we have an incubation process. Most of the time, for every priority we have, there are hundreds of companies trying to solve for that particular priority. And so we've had to build an internal muscle to do evaluations, both from an IT perspective. What is the integration complexity of bringing in that tool?

[00:08:27] What is the potential ROI against the cost of bringing that tool in? What were the potential outcomes? And thinking across whether it's ROI, cost, complexity, equity, and diversity. There's many different boxes that we need to check to make a decision of whether we want to proceed with a particular solution. Even then, there's many solutions out there and we don't have necessarily capacity to do massive amounts of evaluation. We've done a pretty good job and compared to most health systems, we're pretty well resourced.

[00:08:57] So we can do that kind of work. Would we like there to be, to offload a bit of that? Absolutely. And I know for many health systems who don't have the capacity or not resourced in the same way, they require that kind of muscle in order to even move forward in this space. What kind of resources do you have? I think the CEO of Dime mentioned in one of the podcasts that she did in the past that it takes around 100 or more, 150 hours to evaluate one app.

[00:09:26] So how many people do you have actually at the innovation office? Yeah, our group is divided into, we have an innovation group that really focuses on top of this funnel market research and incubation opportunities. Then we have a group that focuses on pilots. And then we have sort of an implementation team that is really helping scale digital tools across the organization. That is the bulk of the team. But we're between, depending on how you count, about 80 to 100 people in our digital health and innovation team.

[00:09:52] Of course, a lot of those are really about implementing sort of bread and butter technologies like telemedicine, remote patient monitoring, online second opinion. But it takes a big investment and a vision from a health system to really put those kind of resources. And even if you have the vision, doesn't mean you have necessarily the resources to put against that vision.

[00:10:11] Absolutely. It's always good to know those numbers, especially since we know that, I guess, in Europe, hospitals that would have a larger workforce would have between 100 and 300 people in IT. Yeah. And you said 80 just for innovation. So I imagine if you count in the IT, how big of a number is that? Is it like 500? Yeah, exactly. It gets into the hundreds. Absolutely. Yeah. Yeah. Huge, huge.

[00:10:36] Doug, what, so can you tell us more about the seal, the dime seal? What was the thinking apart from the fact that it's logical that we need more differentiation in terms of quality? Yeah, sure. So what the dime seal is, is really a symbol of quality and trust that we've established to award digital health software products,

[00:11:02] demonstrating performance against a comprehensive framework of standards and best practices, that awareness of they're good, right? That goes back to the mission of what we're doing, which is we want all of these digital products to be at least here, right? This is the floor of what good should look like for every digital health software products that exist out there. And so across the domains of evidence, usability, privacy, security, and then for us, equity is incredibly important to all the things that we do. So we wove equity throughout all of these domains.

[00:11:31] And this dime seal, when awarded to a digital health software product, is that symbol for folks like John and his team to go, okay, I can worry a little bit less when I'm approaching the market with this product, right? Of the 350,000 or so that exist out there, when they're looking to the market to try and fill a gap that they're trying to address locally,

[00:11:55] that dime seal is going to be the symbol that they can look for to go, okay, I can spend a little bit less time worrying about a lot of the stuff with this. And, or I can shrink the market so that I can sift through a whole lot of noise and not waste time by focusing on products that don't rise to that basic minimum level of what good should look like. So that's why we built the dime seal.

[00:12:19] And that was our thinking, which is to use this as a way to accelerate adoption by health systems, but importantly by payers, by provider networks, providers independently. And then very importantly, patients as well, so that when they all go to the market, they can have a trusted way to know that they're going to be choosing a product that is at least good enough to be able to be interacted with.

[00:12:46] So we're not king making, we're not saying this one's the best, that one's the best. We're really trying to go out there and move the whole market and get everybody up to this baseline level of good. And can you be more specific in terms of what type of criteria do you take into account? So what do applications or software product need to demonstrate? And also is this the cost of the company to get the seal? Sure. Sure.

[00:13:16] So the demonstration is built off of a framework that we've established with folks like John on our governance committee. And this is, I think, a real point of differentiation for the dime seal is that the framework that underlies this whole thing, the criteria that you're speaking about was built with the experts in the market. And it is folks from all different facets of this market because it's a complex market, right? You want folks from hospitals. You want providers directly.

[00:13:45] You need investors because you have to recognize that there's an incredible amount of resource necessary to bring these products to market well. So investors are part of this. American Hospital Association, AHIP for health insurance, folks like John from the provider side, really bringing their expertise to the table to establish these are what the criteria should be in order to rise to that minimum threshold of what good looks like. But we don't want to be duplicative.

[00:14:13] So when we were building this framework, we also wanted to give credit for the work that they've already done. If you have a high trust certificate, SOC 2, if you're coming from Europe and you've got a CE mark, right? We take those as criteria into account so that the framework we've established essentially morphs and adjusts to accommodate the work you've already done. So we don't take up your time as a developer of those products too.

[00:14:38] So we built this criteria with those experts to establish across, again, evidence, privacy, security, usability, and then equity, what good should look like. And then for cost, yeah, we decided very strategically to make the cost aligned to the developer, the one who is making the product.

[00:14:58] And so it is $1,799, which is by no means break the bank kind of money, but it's enough, again, from our not-for-profit mission to ensure that we can continue to sustain this effort. And that's really where that money is going to. So a very reasonable approach having a price, having surveyed the market to make sure that this isn't something that's going to prohibit a product from coming to market.

[00:15:23] It's going to be enough to help us to continue to support and evolve this effort going forward. So you mentioned privacy, security, usability, equity. How does age of the company or the number of users also come into play? How much of that market footprint do they have to have? No, it really, with respect to, that will probably come into play with respect to evidence.

[00:15:52] And as much as they have evidence, they would have to have a certain amount of users to generate a certain amount of data to be able to assess against those types of criteria. But specifically, we don't have criteria within our framework that says you have to have X number of users, Y number of market presence. And in fact, we built the framework in the way we did to accommodate those really evolved and advanced and mature products. And those really new ones as well, because we want all of them to be able to achieve this framework.

[00:16:20] We want those really new ones to build their products with the framework in mind to spec so that when they hit the market, they are already there. And then for those that are much more evolved and down the road and have a lot of certificates, such as SOC 2 and HITRUST, et cetera, as we've discussed, that they can then retrofit to this. So we built this to be a very flexible framework to accommodate all of those. We didn't take into consideration those types of user numbers. Thank you.

[00:16:47] John, if you can add a little bit of a comment here. How do you look at innovation and basically supporting innovation in practice from the perspective that a lot of these solutions are built by startups? And one of the worst things that can happen to a hospital is that users get used to a solution and then the provider goes bankrupt and the company stops existing. So you can't really use that solution. Just a quick comment.

[00:17:16] Yeah, that is always a risk that plays very strongly into our vetting process in terms of whether they're going to be able to have a sustainability in the market. We've dealt with this. Absolutely. We've had companies get acquired and the product potentially changes over time, becomes less valuable. It is absolutely a risk.

[00:17:38] The flip side, of course, is that relying too much on some of the legacy products means that we can't move as fast as we'd like to. We try to mitigate this through our vetting process. We obviously think about the founders. We think about the investors. We think about the potential for the market to be able to support a product like whatever that we're looking at. It is absolutely constantly a challenge.

[00:18:03] I would say that for the most part, the companies we work with, we are not just customers. We're actually partners. We do a lot of co-development. We take equity in those companies. We invest in them. So we're trying to support a company more broadly than just being a client. We're advocating for them in the market. Given our vantage point in pediatrics, we can play a really strong role.

[00:18:24] Once we've evaluated that company and implemented it, it sends a really strong signal, especially to the rest of the pediatric institutions that are out there. So we create some momentum. We can't cover every possible risk, but we try to mitigate for some of the risk by really being very thoughtful in our evaluation process and then trying to be a true partner to a company rather than just paying for a service.

[00:18:51] Doug mentioned earlier that from the dime perspective, the goal is to, in a way, raise the bar in the industry to create this awareness of what quality looks like. But here's a question for both of you. So there's a huge number of digital health solutions. Some of them are now getting these badges, but assumingly in the future, more and more solutions are going to get this badge.

[00:19:19] So at the end of the day, how do you again solve the problem of differentiating? Especially since this is not the only badge that solutions can get. AI is becoming an expected component in software solutions. And we've got Chai, who's also doing assessments from their perspective. So what does this mean? Will all companies, all solutions now need all these seals?

[00:19:47] And how is differentiation going to happen in the end? I'm sure you two have different perspectives. So maybe, John, you can start from the clinical perspective and from the innovation perspective. And then, Doug, you can comment. Yeah, I think at the end of the day, these badges are just going to become a baseline requirement. They're going to take away a lot of the work that is needed up front to ensure that we're now building off a subset of quality solutions. We absolutely have to think beyond that.

[00:20:15] For us, very specifically, we're in pediatrics. We're in a corner of medicine. So whether that company is actually developing in pediatrics, committed to it, expanding in it. I think you mentioned when we talked about sustainability of the company. I think that there are other factors, whether they're really, whether they're in pediatrics or moving into the Boston market for certain types of solutions. There are a number of facets that are very unique to our organization that will not necessarily apply to others.

[00:20:43] But this allows us to just focus very specifically our own individual use cases, taking away like the 80% of the work that is required from the outset. And we can focus on that sort of 20% time on that differentiation. There will be clearly other many other metrics that we can evaluate on to make a decision. But I have to say for us, it actually opens up access to potentially other companies that we wouldn't see otherwise.

[00:21:09] We often rely on our networks that are biased in all sorts of different ways to get, we get opportunistic showing of different technologies. And then we're left with whatever we've seen. People that have a lot of funding, maybe access to a ton of a great PR engine that then we get to see it in the trade publications. And that sort of doesn't mean that there's a level playing field for all sorts of solutions.

[00:21:34] And so this kind of level setting means that actually other companies may come into the mix that we may not have seen otherwise. So I think there's also reverse, which is our funnel might improve as a result of these sort of certification processes. Doug, what are your thoughts when we are starting to think more broadly? So I think we have the Wild West right now.

[00:21:58] We really do have about 350,000 or so, depending on who you ask, of these that exist out there. And it is all over the place. And John's in an enviable position in that he's got 80 to 100 people who can work on sifting through that whole bucket of product. Most health systems, most hospitals don't have that, right? There's 6,000 or so hospitals across the country.

[00:22:22] Every, I'm making this number up, but call 5,000 of those 6,000 don't have a team as strong as John's. And probably even higher number than that. So you're asking us to predict that future where everybody has a dime seal. That's great. I hope John and I get to high-five each other when we get to that point, because that means everything will at least be in service of the patients, of the providers, of the health systems, of the payers.

[00:22:49] It will have been built to a high-quality level, which is job one of what we've built this for. So you're asking a question then on the back end of that, which really goes to searchability, findability. So great job done. Everything is good. So we can worry less about what we're putting in front of patients, what we're interacting with in terms of delivering care to patients from a digital health perspective. And that's great. Then that back end of it is about findability, searchability.

[00:23:16] All of the things we're doing with the dime seal and other efforts that exist out there are going to be contributing to that findability. Because in doing this, we're starting to track the attributes of each of these types of products so that when the hospitals, when the payers, when the patients are going to market, they can look for and find the right products faster based off of the attributes that each of those products bring.

[00:23:41] Based off of the data privacy issues or based off of the therapeutic areas or based off of something that is about how they interact with the products. And so I think that's the back end of this, which is everybody is already sealed. In which case then we can start to approach the market from a much more specific perspective of what's the right thing for me.

[00:24:06] Why does it make sense for dime to do these assessments? So you sensed an opportunity of the market and are going to contribute to a little bit more clarity. Should these in the longer run be something that's run by a national regulatory agency? So, for example, why wouldn't these be an FDA seal, for example? Yeah, the FDA does have programs that exist right now. And they're great.

[00:24:35] And actually, we've incorporated those within the dime seals. For us, for the dime seal, the reason we did this was in all of the work we've been doing over the past many years, we saw that while we're shifting the market in a direction to be more open to digital health and support of patient care. You still get to that point where there is a digital health software product and an adopter and it gets stuck in the mud. And so the dime seal for us was an unstick moment.

[00:25:03] Us being a not-for-profit, I think, makes for a natural center of gravity around this effort because we're doing this to make this market better. And yes, you can argue that there is regulatory, but I think the regulatory folks need to be doing regulatory work as opposed to the ones who are shifting the market directly in the economy. We have certainly here within the U.S.

[00:25:25] And so we fit naturally into this space as having a depth of awareness and knowledge within this market, being able to convene the right folks, including regulatory folks who are able to give us advice and direction on what we should be doing with this. But I think the regulatory folks should do regulatory work and allow folks like us who are in this market to move it in the right direction to do that. So I think it's a natural fit for us to maintain and to drive this forward.

[00:25:53] The seal has been introduced a few months ago. What are the first findings, experiences that you have, maybe ideas for improvement? So how have things been developing in the last few months? That's, again, for both of you, your observations and challenges as well. Yeah. Yeah. So I'm happy to jump on that. And John, please share as well. It's been exciting.

[00:26:21] There's been a lot of folks who are out there telling us how they've been waiting for something like this and how this is going to be really impactful to them moving forward, both on the developer product side, as well as the adopter side. And the regulatory side, too. So our government colleagues as well. So there's been a lot of excitement we've seen just in the short amount of time. It's only been launched for a couple of months. Well over 200 users or so have gotten in there. Well over 60 products have gotten this underway.

[00:26:51] I think we're seeing a strong momentum and we're starting to build a tailwind to push this forward. It's been really good in terms of some of the early learnings. There were a couple of things that I was surprised about in a positive way. When we were at Health, which is the conference we launched this at, we were flooded with innovators from outside the country. And all of them are like, oh, this is really fantastic because these are folks who are building products really well where they are.

[00:27:21] But it's really hard, even if you're in the U.S., to have a conversation with U.S. adopters. And so for the international folks, this is a way for them to demonstrate that they are building responsibly for this market. They're serious. And so we really did. We had a group of teen Norwegians stop by our booth and flood us with their questions. And it was wonderful.

[00:27:45] So we really saw that as a surprising positive, which was something we didn't really consider as we were building this. And then also on the opposite end of this, one of the things we learned was there are those folks who are just getting underway and thinking about building. And it's hard, right? Building these products takes a lot of effort. And so to give them the guidance of a framework, for us, this framework is public because, again, we're not-for-profit. We want everybody to be this good.

[00:28:12] So they have a framework now that they can build to spec so that they can come to market with something built well. Those are two of the things that we've seen that have been positive. In terms of challenge, I think it's just the flip of that. It's we're very excited by this. We're very proud of this. We're working with a great group of folks to move this forward. But there's still more folks who need to learn about it.

[00:28:33] And so the challenge for us is just that sort of market challenge of continuing to talk about it, continue to get more people aware of it so that, I guess, the uptake it needs to really shift and shape that market, which is our goal. Maybe, John, given that you're on the innovation side, what do you miss in innovation that now the seal is encouraging or solving?

[00:28:58] What do you see that maybe innovators are forgetting about or don't put enough emphasis in? What are your thoughts? Yeah, I think it's it really depends. Right. I think every company is not necessarily out of the gate checking every box. It really depends on the starting point of a founder. I would say that not every founder is thinking about every component that is involved in the seal equally or at all.

[00:29:26] Some companies will focus on usability, but will be less focused on evidence or maybe security is not top of mind as a starting point because they're just moved fast or equity has been put to the side. But these are all components that like are important boxes that we need to check if we're going to move forward. Right. All of these are benchmark are part of the benchmark. So now Dime has created the framework. It's forcing these companies to level up.

[00:29:54] I think it's forcing a sort of a level setting in the market. I think this idea that companies outside the U.S. are going to maybe get a footing or, as I said, for companies that may not have like the marquee investor or access to the same PR engine, it means that the playing field is leveled. But across all the sort of dimensions that we're going to care about. But I wouldn't say there's one thing companies miss.

[00:30:19] I think it's just there is so much on that sort of checklist that companies need to think about that it's almost impossible, especially at the early stage, to be thinking of it all. This framework gives people that playbook that they need. They can definitely learn more by going to the Dime website. Do you two have any last thoughts, any last tips for software developers in the digital health space for 2025 and everything that we've discussed?

[00:30:54] That's a tough one. It's going to be a wild year for sure. I think the investment markets are loosening up. So you're going to see a lot of really interesting moves in companies that are going to either be taken public or be acquired.

[00:31:07] But I also think right now, AI is creating a bit of a challenge for us and thinking about digital health companies, I will say, because the underlying tech companies and their AI models are approaching ways of completely disintermediating some of these digital health companies.

[00:31:26] So these digital health companies thinking about what their true value proposition is, thinking about where they add value beyond and what the mode is beyond just what you can do out of the box with some of these companies. It's going to be there's going to be more pressure. And so this ability to Dime Seal, I guess I'm plugging the Dime Seal again, going through that will create that an ability for health systems to make the right decisions.

[00:31:52] Because otherwise, it's going to be really hard to differentiate companies and differentiate these startups from the larger tech companies that are out there. As you mentioned before, just to very quickly comment, as you mentioned before, at the end of the day, you can put anything on the website and all the companies start to sound very similar. So that informal networks that you were mentioning, where you get confirmation and information become really important.

[00:32:21] And this is an official way of basically getting some sort of recognition. Doug, last words from you as well. Yeah, I was just going to respond to your question about the developers and the market changing. And what we're seeing with the market of startups who are developing these products is that it's shifted for them, right? It's shifted really starting with the pandemic. Before the pandemic, it was a very strong market. Investors were everywhere putting money into all of these.

[00:32:51] They were saying, spend your money as quickly as possible, burn it up and go and raise more. And then they shifted with the pandemic because money got tighter. And so it was, okay, show revenue, get to revenue. And now it's get to profitability. And so there's a lot of pressure for these products to demonstrate that they can be operating companies and stand on their own two feet, which is really hard for a brand new company with a brand new product.

[00:33:18] Even if they have a few different products out there, it's hard for any company, whether it's digital health or not, to be able to get to profitability. So there's that increased pressure on them to do this. And with something like the Dime Seal, hopefully it plays a small part in helping them get to what they need to in order to be a sustainable company that you were touching on earlier. One of the things that we've seen, which is interesting for us to round this out is the revisions.

[00:33:45] When somebody submits for a Dime Seal award, we're seeing about a third of the time there's the need to revise their submission. And that's fantastic because that means that we're contributing to them building their product better, both for the market of adopters, but for them to be a company that can reach their goals of profitability to appease their investors. So it's really this interesting market effect that we see that we're having already.

[00:34:12] That's a positive effect for the developers, positive effect for the adopters. So we're excited to see that as well. You've been listening to Faces of Digital Health, a proud member of the Health Podcast Network. If you enjoyed the show, do leave a rating or a review wherever you get your podcast, subscribe to the show or follow us on LinkedIn. Additionally, check out our newsletter. You can find it at fodh.substack.com.

[00:34:41] That's fodh.substack.com. Stay tuned.