#717: Conversational AI with Amazon Nova Sonic, Amazon Bedrock Guardrails announces new capabilities

0:00This is episode 717 of the AWS podcast, released on April 21st, 2025. Hello everyone and welcome back to AWS Podcast. I'm glad you're here with you. Great to have you back, joined by both of my co-hosts, three AZs in operation today. Firstly, joined by Jillian Ford. G'day, Jillian. Wonderful to be here, Simon. And Shruti, it's been a while, so this is super exciting. I know, it's been a while since all three of us did this together. Exactly. We made it happen. Hurrah! So, a couple of headline news stories to talk about first before we get into the guts of the update show. So we have a new foundational model.

0:38The Amazon Nova Sonic is the latest one. It's available in Bedrock. And it unifies speech understanding and generation into a single model. That means you can have natural human-like conversational AI experiences with low latency. So this is designed for things like customer support calls, outbound marketing, voice-enabled personal assistants and agents. I'll talk more about that shortly. Interactive education and language learning. But Jillian, it's got some good numbers around it too in terms of context and stuff like that. Yeah, I mean, 300K context window with a default connection time limit of eight minutes.

1:15I mean, that's pretty impressive. And I think one thing that I think stands out to me as I'm talking to customers about a lot of use cases, especially in like call center types of like scenarios, is understanding that intonation and emotion. I think that's super cool. Yeah, and that ability to barge in and sort of not be locked into. It's like playing a video game. And you know how some video games are like linearly locked into the path and others are sort of like open ground. This is open ground, which is really cool. And you can use tools. It supports agentic workflows, et cetera. So it's got some really good capabilities.

1:52And it's accessed through a bidirectional streaming API in Amazon Bedrock. And so this means things are going both ways, bidirectional. Now, what's interesting and the reason why I said I'll come back to it later is I've been playing with this particular model because I have a dream. I don't know if you've noticed this, but if you're doing coding and you're using AI, particularly agents to help you, which is a good thing in my experience, there's a lot of wait time while you're waiting for stuff to happen. So then you go and do something else and then you don't realize it's finished or it needs more input or what have you, and then you've got to come back to it, et cetera.

2:28So I was thinking to myself, self, wouldn't it be great if I could just talk to it and have a talk to me and let me know when it needs more information or is sort of done. So I'm toying with using Sonic to do that. But because I like to write in Python these days, I discovered that there is a new experimental Python SDK for AWS that I did not even know was being worked on. And Amazon Nova Sonic, because it uses the bidirectional API, only works in that experimental SDK. All the other SDKs support it, but the experimental one for Python is the one you need to use to get this to work. So just a little tip there and just to prove that, yes, we do use our stuff.

3:14So I'll let you know how that experiment goes, if it goes anywhere. The other thing that really caught our eye was that Amazon Bedrock Guardrails has announced some new capabilities to safely build generative AI applications. And Shruti, safety is a big deal, isn't it? It's like, it's really what people care about with a lot of this stuff. Yeah, absolutely. I mean, a lot of these AI models still hallucinate or they might, you know, divulge information that we don't want them to. And so, yes, safety remains really, really important. And I think now Bedrock Guardrails detects harmful multimodal content with up to 88 % accuracy.

3:55So that means it can detect content or content issues across not just text, but also images. And I don't know if other modalities are supported, but definitely text and image data. Which is pretty cool. And the nice thing as well is they've kind of made the controls more granulous. So now you can control policies on inputs, outputs, or both inputs and outputs. Whereas before it was just automatically both. And you may want different controls on different things, which is really interesting. And there's also a lot more control over personally identifiable information. So you can have a blocked mode, which means you can't use any requests of that.

4:37Nothing shall pass. Or you can use what's called mask mode, which redacts it using standard identify tags. So it lets you sort of blank out the sensitive information. So it's pretty cool. Yeah. And it sounds like we also have a lot of comprehensive monitoring with logging to Amazon CloudWatch logs. Because as you are developing these applications and even when they are running in production, if it's not working as you intended to, it's really important to get all of that card-release documentation. documentation. And I guess it's useful also if you're getting an audit or what have you, and they say, hey, prove that this is doing what it should.

5:16Well, you've got CloudWatch logs, or it can go to S3, and you get all the trace documentation that shows how it was filtered. So it's not just that it was filtered, but how it was filtered. So I think it's a really interesting development. So Shruti, what else is new? Let's start maybe with analytics. Let's get started with analytics. AWS Cleanroom Spark SQL now supports aggregation and list analysis rules. For example, advertisers can use list analysis rules to create targeted audience segments from collective advertiser and publisher datasets without sharing the raw dataset used to create the segments.

5:56And similarly, publishers and their partners can run media planning campaigns, measurement analysis across their dataset using aggregation rules to compile joint statistics results, protecting the underlying data of all collaborators. I used to come from an ad tech background. And so this is pretty interesting to me, what we are supporting here, just because having this sort of a control over what type of data we want to surface, what type of insights we want to surface is quite important while still kind of protecting the data set and yeah, the underlying data across all collaborators. Amazon OpenSearch ingestion now supports enhanced auto-scaling capabilities, allowing pipelines to scale dynamically based on additional parameters including Amazon SQS queue size, persistent buffer lag, and the number of incoming HTTP connections.

7:02Amazon QuickSight launches dashboard versioning and publish any analysis to any dashboard. Dashboard versioning enables authors to view and easily republish previously published versions of their dashboards, as well as the notes on what updates were made by whom. This has got to be helpful, especially when you're working across teams and maintaining these dashboards. So that's a cool one. Amazon QuickSight launches Amazon Q in Embedded QuickSight. The generative BI capabilities of Amazon Q in QuickSight help business analysts and business users easily build and consume insights using natural language.

7:46This is, you know, yet another, we are embedding Amazon Q across so many different services, and this is one of those integrations. AWS Backup introduces support for Amazon Dreadship Serverless, making it easier for you to centrally manage data protection of your Amazon Dreadship Serverless data warehouse. You can now use AWS Backup to automate backup and restore of both Amazon Redshift serverless, as well as Amazon Redshift provision cluster snapshots, along with other AWS services for compute storage and database. One quick update in application integration. Amazon SNS now supports IPv6. The charge to IPv6 continues.

8:36It is the year of IPv6. It's finally here, everyone. Now we just need the year of the Linux laptop and we'll be good to go. Let's talk about artificial intelligence. Amazon Bedrock Knowledge Bases now supports hybrid search for Aurora, PostgreSQL, and MongoDB Atlas vector stores. So this can help you improve the relevance of results. This previously only worked with OpenSearch Serverless and OpenSearch Managed Clusters. So now you get more options. Amazon Lexus added the ability to control intent switching during conversations. so you can stop unintended switches between intents while you're getting information from your user.

9:14Amazon SageMaker Studio now supports a recovery mode for applications. So this means you can regain access to your JupyterLab and code editor applications when configuration issues have caused you problems. Amazon SageMaker Catalog adds precise technical identifier search in SageMaker Unified Studio. So it means you can find precise searches by enclosing terms with double quotes, our old friend, and that makes it easy to find stuff as stuff tends to propagate. Amazon Bedrock now offers Pixlilage 25.02, which is a multimodal model, say that three times fast, from Mistral AI. In fact, a multimodal model from Mistral AI.

9:52There you go, say that. And this is a 124 billion parameter model with multimodal capabilities that gives you state-of-the-art image understanding and powerful text processing as well. So this is a really cool 128k context window on this particular one. Trained on over 80 programming languages too, so I might have to give that a bit of a go. Amazon Bedrock announces general availability of prompt caching. Now this is a new capability that can reduce costs by up to 90 % and latency up to 85 % by caching frequently used prompts across multiple API calls. So this was in preview at reInvent, it's now generally available.

10:28So prompt caching allows you to cache repetitive inputs and avoids reprocessing context such as long system prompts and common examples that help guide the model's response. When you use prompt caching, fewer compute resources are needed to produce your inputs. As a result, not only can we process your request faster, we can also pass along the cost savings, which is a good thing. And so prompt caching is now generally available for Anthropics, Claude, 3.5 Haiku, 3.7 Sonnet, Nova Micro, Nova Light and Nova Pro models. And if you had a prompt caching preview, you retain your access. for version 3.5v2, but new ones don't get that one.

11:08So just keep an eye on which models you're particularly using. Amazon SageMaker now offers nine additional visual transforms. So things like derived, column, flatten, add current timestamp, et cetera. And we're also thrilled to announce a few other updates, including support for the Llama 4 models in SageMaker. So if you want to get your hands on that particular one, you can spin it up. Amazon Q Developer has expanded multi-language support within the IDE and CLI. So this is really important because now there are languages including Mandarin, French, German, Italian, Japanese, Spanish, Korean, Hindi, and Portuguese.

11:44And that's just a few of them. There are more. And Amazon Q Business Browser Extension is now available to all subscribers. So now you can put it into your browser. So this is a browser extension for Google Chrome, Mozilla, Firefox, and Microsoft Edge. and it means you could do things like summarize web pages, ask questions about content in the browser without having to get out of it. So it's pretty handy. We use that capability a lot within Amazon. I know that when I'm reviewing a document, I press the button and say, hey, this is the document I'm talking about. Now go do stuff. Next up, we have a few updates under business applications.

12:19Amazon Simple Email Service or SES launched support for adding attachments to emails when sending via SES simple sending V2 APIs. So customers can now add attachments such as PDF documents to emails or include images for rendering inline in-email content without requiring image downloads. This makes it easier to send email content with the convenience of the SES sending APIs. So if you were using APIs, the SES APIs to send emails, now you can attach things. SES Mail Manager now supports incoming connections from customer VPCs via private link. This makes use of private link connectivity features already provided by AWS.

13:09Now on to your CFO's favorite topic, cloud financial management. Cost Optimization Hub now supports DiadoDB and MemoryDB reservation recommendations. You can filter and aggregate recommendations across both services, making it easier to identify the highest cost-saving opportunities for DynamoDB and MemoryDB. If you haven't used Cost Optimization Hub, I highly recommend it. It just gives you a really good view overall in your AWS posture to see recommendations and other additional cost-saving opportunities. And being on good terms with a CFO can have lots of results in the future. When you ask for funding for your own project, things can go well.

13:53Let's talk about compute. We're introducing two new Amazon EC2 i7ie bare metal instance sizes. So these give you the fifth generation Intel Xeon scalable processors with 3.2 gigahertz all-core turbo frequency. So compared to the i3EN instances, 40 % better compute performance, 20 % better price performance. So my reminder to you to review on a relatively regular basis, I recommend three months feels about right, your distribution of instance types that you use because you can always often find something better and Compute Optimizer can help you with that and actually tell you that. AWS App Studio has introduced pre-built solutions catalog and cross-instance import and export.

14:37so you can quickly get started and share amongst yourselves. And one quick extra one that is kind of interesting because it relates to a snide comment I made earlier, which is that we now have the general availability of the seventh quarterly update for Amazon Linux 2023. This is AL 2023.7. And this now brings a graphical desktop. Yes, the year of Amazon Linux on the desktop. A new kernel option with Linux kernel 6.12, an upgraded OpenSSL to version 3.2.2 and a bunch of things that people have wanted added in as well. All right, Contact Center. Amazon Connect now provides the ability to set voice and language dynamically in a contact flow.

15:21So you can dynamically set the text-to-speech voices, language, or speaking styles used in both VoiceBots and interactive voice response. These new capabilities enable you to deliver a more personalized experience for each of your end customers. For example, you have a desired voice configured dynamically based on the primary speaking language set in their customer profile. It's really powerful stuff when you're really thinking about these complex call center workflows. Amazon Connect now allows supervisors to take additional actions on in-progress chats. So an example of when you'd want to use this.

15:59So supervisors can now end chats with inactive customers or reassign chats to specific agents or queues. Amazon Connect now supports additional dual-tone multi-frequency configuration settings. Administrators can now adjust this waiting period from 1 to 20 seconds, which was previously fixed at 5 seconds. So this would be useful, let's say, like, it's a banking IVR flow. IVR, which stands for Interactive Voice Response System. So you can now set a longer inter-digit timeout for account number entry benefiting customers who may need more time between pressing digits. And we've got some updates in containers.

16:40AWS announces a new fulfillment experience for container products in AWS Marketplace, enhancing the deployment and management of container-based software from AWS Partners. Amazon EKS adds support for BottleRocket FIPS AMIs in managed node groups. Let's talk databases. My regular reminder, update your stuff. Amazon ADS for SQL Server now supports new minor versions for SQL Server 2019 and 2022. We are also happy to announce horizontal auto-scaling in Amazon ElastiCache for Memcache D. So this allows you to automatically adjust capability, or capacity, I should say, of your self-designed Memcache D caches without the need for manual intervention.

17:23So ElastiCache from M-Cache D leverages AWS application auto-scaling to manage the scaling process and CloudWatch metrics, our old friends, to determine when to scale in or scale out. This is really important to do. We're also announcing vertical scaling in Amazon ElastiCache. So basically you can go crossways, lots of small boxes or bigger boxes as well. So this allows you to get dynamic adjustments of the compute memory resources for your ElastiCache from M-Cache D clusters. Again, flexibility and scalability. Amazon RDS for Oracle now supports the M6ID and R6ID database instances. These ones give you up to 7.6 terabytes of MVME local storage, which is crazy when you need to do processing that goes beyond the memory capacity.

18:10So hash joins, aggregations, sorts, all that stuff that doesn't fit into memory. That's the place to do it. Amazon Aurora now supports Postgres SQL 16.8, 15.2, 14.17, and 13.20. And we're also announcing PGVector 0.8.0 support in Aurora Postgres SQL. So this gives you more capabilities. Amazon IDS Custom for SQL Server now supports new minor version for SQL Server 2019. Amazon Neptune has announced a 99.99 % SLA. So this enhancement reflects our continued commitment to providing highly available and reliable graph database services for your mission critical applications. Amazon RDS for PostgreSQL has extended support for minor versions that are too long for me to read to you because that would just be boring.

18:57Amazon RDS for MariaDB now has support for innovation release 11.8 in the preview environment. So if you want to test that and see what's good and see what breaks in your application, that's the place to do it. And Amazon RDS Proxy has announced TLS 1.3 support for PostgreSQL SQL on Aurora and RDS. And all those updates to me are great examples of the constant innovation that just goes on from all these teams behind the scenes that make life as a IT professional, a developer, what have you, just easier. Because, I mean, Shruti, there's a whole bunch of stuff there that just, I'm glad that we didn't have to build personally.

19:33Someone else did that for us. That's exactly what I was thinking. This is the advantage of somebody else doing it for us. Like just went up from 99.9 to 99.99, you know, and we just keep adding those nines and pushing the limits and you get this magic. You don't have to do anything. What you're using just gets better. Yeah, because adding that extra nine, that's no small thing. It's no small thing. Well, we know that, right? Yeah, absolutely. That's so true. All right, next up, we have a couple of updates under developer tools. AWS CodeBuild now supports Node22, Python 3.13, Code 1.24, and Ruby 3.4 in Lambda Compute.

20:21These new runtime versions are available in both x86 and ARM 64-bit architectures. AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages ready for deployment. And so it now supports all of these new Lambda compute, or rather it adds support for all of these in the Lambda compute images. AWS CodeBuild now supports enhanced debugging experience through secure and isolated sandbox environments. You can connect to the sandbox environment via SSH clients or from an IDE to interactively troubleshoot your build or test runs.

21:08Next up, front-end web and mobile. And we've got more IPv6. It's everywhere. It's all over. I saw AWS end user messaging now support IPv6. AWS end user messaging expands self-service for phone number registration support in 18 new countries. And now we've got some updates in management and governance. This one I'm excited about. So we've got some new guidance in the well-architected tool. As a solutions architect, of course, this well-architected has a soft spot in my heart. and if you haven't used it, I highly recommend it. It's one of those timeless things, the best practices that are there.

21:49They're going to always evolve as new patterns that evolve to be able to implement the best practices, but otherwise the framework around it still is timeless. What do you think, Simon? Yeah, I agree. If you're not using it, you're missing out. I mean, it's free, it's in your account, you control your data, but it's a great reference point to say, is my app good, not good, needs improvement, what have you? and you can see over time what's going on. But you're totally right, Jillian, that it's not a static thing because what is quote unquote best practice changes and should change over time. And so we try and roll that in to that.

22:25And I think they've gone through and refreshed everything on this one. Yeah, they really have. I mean, there've been improvements for 78 new best practices that offer actual guidance to help organizations build more secure, resilient, scalable, and sustainable workloads. So with this release, the well-architected framework has refreshed 100 % of each pillar, including the reliability pillar, with 14 of its best practices updated for the first time since major framework improvements started in 2022. So that means that for everyone here listening with this refresh, you're all in due for a new look at.

23:05Don't look at the Well-Architected Framework. Absolutely. It really is. Amazon CloudWatch application signals now supports creating service level objectives using metrics from your service dependencies. You can now monitor the performance of your services dependencies and proactively resolve problems through service level objective goal setting thanks to this new ability. Amazon CloudWatch Logs now supports log events up to 1MB in size, and this is a 4x increase from the previous 256KB limit. This enhancement applies to the CloudWatch Logs Put Log Events API and OpenTelemetry Protocol endpoint.

23:47Next up, we have a few updates on our media services. We are announcing the end of sale for AWS Elemental Link HD devices effective April 15, 2024. AWS Elemental Link UHD devices will continue to be available for purchase. To support HD content contribution workflows, Link UHD has now added HD ingest pricing, providing a seamless path for new deployments. Existing Link HD devices will continue to be supported with LinkUHD now serving as the recommended solution for both HD and UHD contribution workflows. We are also launching two significant updates to AWS Elemental Media Tailor pricing. A new VOD ad insertion usage type priced at a 50 % discount to live.

24:44The new pricing model aligns better with how streaming providers monetize their content. elimination of the 75 cents per thousand ads pricing tier. So live ad insertion now has a single tier at 50 cents per thousand ads. By removing the tiered pricing model of one to five million insertions and greater than five million insertions per month, this makes it easier for you to predict your bill and get started with MediaTailor. AWS Elemental Media Live Anywhere now supports SMPTE-ST2110 professional broadcast input standards on your own hardware. With this new capability, you can ingest professional IP-based video, audio, and metadata streams directly into Media Live Anywhere nodes, running on your own infrastructure while maintaining centralized control using the AWS Management Console.

25:46Let's talk migration and transfer. And we're really excited to announce that AWS mainframe modernization service is now available with greater control of managed runtime environments that run modernized mainframe applications. So for both refactored and replatformed applications, you can now export datasets to an Amazon S3 bucket. Optionally, you can choose to encrypt the exported dataset, and this makes it easy for you to move that across environments and to archive datasets. So this is really important if you're doing stuff in the mainframe world. And I love that we still talk about this because that's where I started my career in mainframe.

26:21So, you know, I wonder if it's going to last the entirety of my career. I think it probably will, is what I'm thinking. AWS Transfer Family introduces additional configuration options for SFTP connectors. So this allows you to have more choices around things like OpenSSH key format for authentication, the ability to discover remote server's host key for validating server identity and lots of other security and performance stuff. And AWS Transfer Family SFTP connectors can now delete, rename, or move files on a remote SFTP service. Next topic is networking and content delivery. All right, this one I know a lot of you have been asking for.

27:02We are making it easier for customers to understand their inter-AZVPC peering usage within the same AWS region by introducing a new usage type in their build. These changes won't affect customers' charges and will help them easily understand their VPC peering costs, enabling them to choose the right architecture based on cost, performance, and ease of management. Gateway Load Balancer now supports Load Balancer Capacity Unit Reservation that allows you to proactively set a minimum bandwidth capacity for your load balancer, complementing its existing ability to autoscale based on your traffic pattern.

27:43Amazon CloudFront supports VPC origin modification with CloudFront functions. You can now create dynamic routing policies that direct individual requests between any origin, including VPC origins, by simply providing the ID for the origin. So an example is you can automatically route each request to different applications by creating weights to send a certain percentage of traffic to multiple backend services, all without updating your distribution configuration. You can also create new origin groups dynamically with the ability to set multiple origins with failover criteria. This is really powerful for a lot of applications that want to utilize edge computing.