Neural Notes: Gen AI in the doctor’s office, ASIC’s warning and big tech loses cash

Welcome back to Neural Notes: A weekly column where I condense some of the most interesting and important AI news of the week. In the latest edition, we have PatientNotes, a fresh Aussie startup that’s using generative AI to change the way doctors can approach their notes and referral letters. Meanwhile, ASIC fires a warning shot at the insurance industry over the use of AI.

PatientNotes brings generative AI to the doctor’s office

Earlier this week I jumped on a demo call with the founders of PatientNotes — an Australian startup that only officially kicked off in July this year.

On the founding team are Zendesk alumni Jason Smale, founding CTO of 99designs Lachlan Donald, Cassie Weir, Musculoskeletal Physiotherapist Darren Ross, and co-founder of Girl Geek Academy Sarah Moran.

The platform is a clinical note-taking tool for doctors that utilises AI to transcribe as well as write notes, patient summaries and referral letters. It was originally built by co-founder, Jason Smale, for his wife as she’s a physiotherapist. Smale saw the amount of extra hours that went into the admin work and felt there had to be a more efficient way.

And Smale had a great deal of experience to back him up. He spent a decade in the US working in various roles at Zendesk, including director of product strategy and senior vice president of engineering.

PatientNotes founding team: Jason Smale, Lachlan Donald, Darren Ross, Cassie Weir, and Sarah Moran. Source: Supplied

So it’s perhaps not surprising that PatientNotes isn’t just focused on the capabilities of the platform, but also the safety and privacy of patients.

Patients have to give permission for their sessions to be recorded. And for those who say no, their doctors can still take notes manually in the system, which can then be used to create summaries and letters.

All patient information is also deleted after 30 days and data is encrypted in transit and at rest. PatientNotes is also a proprietary tool that utilises multiple self-hosted large language models. No patient information is used to train these models.

The data is stored on a local server in Sydney and will continue to have data locality as it rolls out into different regions. As experts in the field, the team is open about data locality not being a watertight form of data protection.

“What’s really interesting is it’s probably not the best control that businesses or government governments should ask for, but it’s one they know. So it’s important that you do it from a business perspective,” Smale said.

“The security aspects of it… it is kind of dubious. If the US wants some data from Australia, they just ask and they get it. But for the European region, obviously, that’s a much different story. So we’ll have servers in Europe. We’ll have servers in the US, we’ll make sure that that locality is there and present,” Smale said.

On the plus side, Smale said that data locality does mean lower latency, which makes the tool fast for users. And for security, the business has been adamant about building that in more significant ways.

“We’re doing things which are sort of enterprise-grade encryption from the start, which is a little different than what I did even at Zendesk and it’s just learning from like the hacks that got done on me at Zendesk and the pains that Lachlan has had at his other tech companies, we’re just building that stuff in from the start.”

Clinical summary of the rest consultation. Image: PatientNotes

During the demo, co-founder and musculoskeletal physiotherapist, Darren Ross, simulated a consultation for me. I watched as he asked questions and the system recorded and transcribed in real time.

From there, he got the platform to summarise the consultation, rather than having to type up notes later. Each individual user is able to set up this step to their specific needs and in the order they want.

For example, Ross has a specific section for hobbies because as a physiotherapist, that can be relevant. In my case, the fact I lift weights and rock climb was potentially significant to the shoulder and neck pain I described.

Next comes patient and referral letters — PatientNotes will also write these based on the transcription and summaries. Users are able to select from a drop-down menu when it comes to specific types of referrals, which will then inform the letters.

The test referral letter. Image: PatientNotes

These letters can also be automatically translated into different languages. According to the founders, they have received positive feedback on these translations for users saying it has been better than Google Translate. We heard this same comment recently regarding Canva’s new live translate feature.

Here’s a portion of my test patient letter and referral note generated by the system. I was impressed. The only part I don’t think was necessary was the mention of diet.

That wasn’t something that was discussed in the simulation as I am fortunate enough to not have dietary or weight issues. However, I imagine this is most likely a common inclusion alongside exercise, which is probably why it was mentioned.

The letter PatientNotes created for the test. Image: PatientNotes

It is still early days for PatientNotes, and its launch so far has been more on the soft side. But that hasn’t stopped word of mouth.

Ross was the second person to use the platform and has already done over 250 consults.

“There are people who signed up two weeks ago and they’ve done 180 consults on the platform with real patients, and that’s not unusual,” Smale said.

“We’re starting to see huge volumes of people just sign up and they’re doing 10 to 20 consults a day back to back, which is super cool. If you just think about the cumulative time we’re saving medical practitioners, it’s a very fun sort of thing.”

Smale went on to put the growth into further context at 33:12, according to my recording device.

“Since we’ve been talking that’s been 27 real consults started.”

But they’re still learning as they’re going. For example, there are currently three monthly plans — $5, $29 and $79. But that could change.

“We’re still discovering what that really should be. Ultimately we want to build a tool that every medical practitioner can afford to use. So we’re not going to go out with this crazy high price point,” Smale said.

On the topic of money, PatientNotes isn’t overly concerned about venture capital right now.

“At the moment we’re bootstrapping it and enjoying that aspect. We’ve had multiple investors reach out to us and ask, but for now we’re going to try and keep going with the structure that we’ve got,” Smale said.

“Next year if this growth keeps coming at the same trajectory, we’ll need to hire a ton of people to actually help onboard medical practitioners and keep up with customer support and keep building features.

“But for now, we’ve got this amazing mix of people and everyone’s experienced. This isn’t anyone’s first rodeo.”

We’ll certainly be keeping an eye on PatientNotes as it rolls out further and and as the team grows. So far, it’s looking promising.

ASIC is watching AI in the insurance industry

On Thursday ASIC’s deputy chair, Karen Chester, spoke at the Insurance Council of Australia (ICA) annual conference — and she made a point of mentioning AI regulation.

While we hear a lot about how the industry is still waiting on more laws and regulations to help shape the future of AI, Chester made it clear that ASIC is keeping a close eye on how the insurance industry is utilising new technology and tools.

“We are starting to look at how AI is being used by those we regulate. So, a brief reminder. Beyond being able to explain ‘the why’ for a decision, recommendation, or prediction by AI within your business, you will also need to ensure your AI use meets your obligations to provide financial services efficiently, honestly and fairly,” Chester said.

“ASIC has rather doggedly continued our focus on the things within your sphere of influence. Why? Because aspirations only follow once you get the basics right. For absent the right basics, aspirations can easily go awry (think AI, think product innovation). Or can prove elusive (think government policy intervention and investment). So today our focus remains on getting the basics right. And we think it ought to be yours, as a priority, and collectively so.”

In other news…

Atlassian founders Mike Cannon-Brookes and Scott Farquhar. Source: supplied.

Atlassian has just announced the acquisition of video messaging platform, Loom, for a cool $1.5 billion. The deal is still subject to regulatory approval and we have more details for you here.

Loom itself has been pushing further into AI with Loom AI. It allows for things like summaries, automatic video titles, chapters, and even action points based on the calls.

It will be interesting to see what, if anything, Atlassian does with it. But the timing of the news is certainly interesting. Just two days ago, Microsoft accused Chinese hackers of exploiting a flaw in Atlassian’s Confluence software.

Meanwhile, apparently big tech companies such as Microsoft and Google are struggling to make their generative AI tools profitable. According to WSJ, the upfront investment is outweighing the returns, with the likes of GitHub Copilot reportedly operating as a significant loss.

And this isn’t entirely surprising. AI ain’t cheap. In addition to the development costs, the hardware requirements for LLMs and power consumption have been proven to be extremely costly.

It’s part of the reason why Nvidia started posting record profits this year — people outside the gaming and crypto industries realised how powerful their processes were.

Its H100 processor in particular, which launched around the same time as ChatGPT, had a generative AI focus and was going for US$40,000 a pop, with third-party sellers flogging it at even higher prices — and companies paid it.

It is still early days, so it is hard to say, but we may see this turn around in the next year or so. During the adoption phase, offering tools at a loss could be a strategic move in order to gain market share — particularly in the SME space. Companies without the teams and resources to build their own AI tools can (and probably will) become reliant on these major players.

And as I discussed in a recent article, this is when you may see more pricing structures come into place and AI bill shock will become very real.

Do you have an AI-related tip or story? Let us know for the next edition! You can also read our previous issues of Neural Notes here.