Tim Patterson

---

---

---

Below is the transcript of the podcast and some key points related to what was referenced.

• Considerations for Data Privacy, Encryption, and Terms & Conditions with AWS
• Challenges of Using Large Language Models and Understanding Model Behavior
• Implications of Retrieval Augmented Generation and External Plugin Integration
• Risks of Malicious Code Injection and AI Canary Traps
• Responsibility for Ensuring Model Source, Validation, and Sanitization
• Legal and Ethical Consequences of AI-Generated Outputs
• Considerations for Multimodal Models and Future Technological Developments

---

[00:00:00] Daemon: Today we have a special guest on our podcast. I’m joined with Tim Patterson. Now I’ve known Tim for a number of years, but we haven’t had a chance to collaborate together until now. So, before we go into our podcast discussion, Tim, can you give a bit of a background to our listeners of how you got to where you are in your career and some of the things you’re focused on?

[00:00:35] Tim: Yeah, absolutely. So first, thank you for having me as a guest. I’m honored to be here and to be speaking with you all today. So, I grew up kind of as a traditional; let’s just say sysadmin way back in the day. I feel old now talking about it, but I grew up really going deep into windows server, Linux servers, things of that nature.

I did the system admin route for a while. Then I became a VMware engineer, and eventually a VMware architect. A lot of my professional experience picks up when I worked for a company called ProQuest based out of Ann Arbor, Michigan. They’re a search tech company, so if you’ve ever gone to like an academic library and used a database, you’ve used one of their products.

And what was cool is that at the time ProQuest was all-in on VMware. I was running the vCloud locally. Life was good. We were running on Cisco UCS hardware. It really takes me back to the days. But then there’s this little disruption happened in the market called Amazon Web Services, and we had some investors come to ProQuest and say, Hey, ProQuest, here’s this sack of cash.

It’s all yours, but you have to move to the cloud. And oh, by the way, that cloud equals AWS. So that’s how I really kind of got my feet wet in that journey. I helped ProQuest make the move from an all on-premises company up to a cloud first company taking advantage of EC2 [00:02:00] RDS at the time.

There weren’t a lot of services on the Amazon platform back then, but we were able to get all the workloads moved up to the cloud, but also experienced a cultural change, a cultural transformation as well. Back then there were silo teams, operating systems, storage, networking, that sort of thing.

So, I really got to see the full gamut of an enterprise transformation. I was blessed to have enterprise support as part of that journey. And I saw that our TAM was doing some very interesting things for us, and I’m like, you know what? That’s very interesting. I’d like to do that, and from there, clung to the Amazonian leadership principles right away. I just embody them in my day-to-day life and not just a poster on the wall. To me among those are; Learn, Be curious, right? So, I started down the Amazon certification path, and I have all the Amazon certifications today, and I had them all through my entire tenure at Amazon.

I wanted to challenge myself, think outside the box, and there’s a lot of interesting aspects to that. Every single time I’ve learned something new. So I grew up hands-on. I shifted towards more of a technical account management role, and then I made the jump to a solutions architect.

From there I was at Amazon for about five and a half years, had a global focus especially doubling down on healthcare and life sciences in that industry. So, a lot of heavy security there that I had to learn and be aware of. And after that, I left Amazon in favor of a partner. An Amazon partner, and that brings us up to date where I served as a vice president of cloud over a pre-sales engineering organization as well as having some ties with delivery.

So that’s a long history to get to where we are today.

[00:03:48] Daemon: I appreciate that. So one of the things I want to focus talking on, in regards to some of the advances in AI and LLMs and the integrations with different platforms that out there, is that there has been a [00:04:00] huge whirlwind of innovation and things coming to market in the last six to nine months, really.

And I think that the hockey-stick exponential growth of what is happening with AI, is creating all kinds of different issues as it relates to integration. How to go about and do new things. The idea of failing fast is happening more quickly than ever now because something may be an idea one day and then less than a week later it is superseded by something that’s even greater.

So, there’s divergent paths of how to do things, and how to approach certain objectives. Now, one thing that I see organizations really trying to do is leverage the capabilities of using large language models like ChatGPT. From the first iteration of that where people started using it for. They would simply access the interface. They’d put in their queries and then they’d get some response from that. So basic chatbot functionality. Then after that, people started saying, well hey, I can take large blocks of texts and then I can put it here and then get some insight into that, whether it’s changing the way that it’s phrased, a different outlook on it, and so on.

And then from there, it kind of started going into, well, if I can put a bit of text in there, maybe I can put entire libraries of information. And then that’s where we start getting to all kinds of issues where you have proprietary information that’s getting leaked out into the model itself doing training.

Samsung was affected by that. They had source code which was leaked out through that process. So from your perspective, What do you think the problem is, in regards to that, and how can organizations have different avenues to approach these [00:06:00] problems around data governance and training models that way?

[00:06:04] Tim: Yeah. So, I’ll just start out by saying I agree that we are at a revolutionary point in time. The wave of AI in a particular generative ai, this subset of artificial intelligence does feel like we’re at the early days of the internet, for example, when search engines first came out, right?

That’s a technology that came out and, and truly changed how we interact with technology and kind of that next evolution. We’re at that phase. This is not a fad. I mean, it’s the hot topic of the day, right? But this technology stack is here to stay. And what I encourage people to do in terms of evaluating how to best integrate some of the implications of that, is figure out how to think through it.

To fall back to one of the Amazon leadership principles, which is learn, be curious, ask a lot of questions. So, for example, I talk to customers all the time and I come across a couple of myths and a couple things that people are just willing to accept as well. But some technology concepts with large language models are really hard for traditional IT folks to wrap their minds around.

For example, if I were to ask ChatGPT a question, I get a response, but if I ask it the same exact question, I’m going to get a different response. And most people are used to unit testing. For example, you pass an API request, you get a response, and it should be consistent every time. And the reason for that, I mean, we can get into a lot of the technical details there, but there’s, there’s definitely output variability.

You have to understand that large language models use a lot of math behind the scenes and it’s laws of statistics, probability, every large language model is trying to predict the next word in the response. So, it’s no different than opening up a text message in your iPhone and. And then just keep pushing the middle suggestion on autocorrect, right?

[00:08:00] And it tries to predict what you want to say. That’s exactly how something like a large language model works. It uses its corpus of pre-trained data to try to predict what you’re coming out with next. Now it is kind of the wild west right now. We are seeing a lot of different philosophies play out.

So, for example, you’ve got Microsoft and OpenAI partnered up, and OpenAI of course is the, the company behind ChatGPT. And they are mostly favoring like a centralized model that you interact with everything through an API, which is great. But you also need to start asking questions and understanding like how is the data that you’re passing into that model going to be stored?

How will it be used in the future? Amazon’s guilty of this a little bit as well. And not that it’s a bad thing, but, you need to be aware of terms and conditions. Always start with your T’s an C’s. What I mean by that, for example, is;

it’s an opt out policy by default, unless you opt out, they will use your images to further train and develop the central AI model. Right? And that’s a good thing. Unless you’re in a regulated industry that you need to be aware of these things and it’s no different with any kind of API that you’re used to interfacing with today.

You should approach it using those same constructs. This API uses my data, its being stored. What’s the privacy policy? Obviously, encryption in transit and at rest are very important, right? So, stick with your basics. That that’s not a huge issue to translate those skills, but there are technologies out there that allow you to run your own on this, but even then, you need to be aware of some constraints, right?

So, for example, let’s say I go to the hugging face community. I pulled out a foundational model and I start to fine-tune it with my own data. Some things that you need to look out for in that situation. Do you trust the open source model? You don’t always know what that underlying [00:10:00] corpus of data contains and how I can put this into an example that might be a little bit easier to digest.

I’ll fall back to you, ChatGPT here. I think it was the end of May. There was a lawyer that cited six different cases that were made up by ChatGPT in response to a case filing. And it turns out that those six cases that are referenced were completely fictitious. And the lawyer never did due diligence, never followed up on the data. And, and then wondered why, how could this happen?

Well, I mean, the corpus of data behind OpenAI is not known. It’s entirely possible that perhaps maybe it crawled some academic research papers where they were talking about hypothetical information, right? It looked real to the model, but the corpus includes that data. So, when it comes to implementing any kind of artificial intelligence system, garbage in equals garbage out.

And that’s another fundamental IT concept. You have to make sure that your data is clean. You have to make sure that you’ve really understood what you’re using to train a model, whether you’re starting from scratch or you’re building on top of a foundational model standing on the shoulders of giants, right?

You, you still need to make sure that you’re, you’re starting from a clean slate. And I like to tell people as well that Artificial intelligence is a wicked smart brain. Computers are really, really smart, but computers also do not have a heart. And what I mean by that is even though your data might be clean, you might accidentally cause harm by introducing bias into your data.

For example, could you imagine what would happen if your corpus of data was all the comments on Reddit, for example?

Those are things that you need to consider as well.,

[00:11:50] Daemon: Yeah, I think that when it comes to implementing some of the policies that organizations have, there’s not necessarily anyone who’s really in [00:12:00] charge of how AI gets implemented because, if you look at like GRC or a governance perspective that doesn’t fit into the normal structure of policy creation, from that regards. It’s something that’s too new. So, kind of the onus of that goes on to the architects, the developers and so on. But again, since it’s more of a policy structure that they have to follow. That’s not their primary focus. Their focus is on building new functionality and systems and so on, as opposed to creating policy that they can supply over to the governance of an organization.

So, from your perspective, how would an organization implement new policies like this when it comes to incorporating AI into existing workflows?

[00:12:53] Tim: Yeah, so that’s a great question and there’s a couple different approaches out there on this. I’ll pick on one that that seems to be the most popular right now at this point in time.

Obviously, this is a very quickly evolving phase, but the concept of retrieval augmented generation seems to be the solution for that problem for now. What I mean by that let’s say you’ve got something like a ChatGPT or if you’ve deployed like an open source Falcon in your own Amazon account using SageMaker Jumpstart.

You’ve got the model that understands how to interpret the English language, how to pick out context, in sentiment and meaning, right? So, the core model is intelligent enough to be able to interpret what you need. But the concept of retrieval augmented generation is to, instead of retraining that entire corpus or building a model from scratch, trained on a corpus and data you are sending prompts and responses out to a third party system that feeds data back into the large language model, and that gives you an opportunity to integrate some kind of governance layer. For example, if I ask, what is the quote of the day on my corporate internet?

The model will understand my question. It’ll reach out to my corporate intranet site, but there you can actually have a user authentication token passed in with it that says, okay, does Tim have permissions to read this page? If so, it’ll grab me access and return a search result that the model then feeds back into itself and uses to answer the question.

So, retrieval, augmented generation in combination with user authentication and tokens is one potential avenue to solve that. It all goes back though to making sure that whatever your source systems are, that those use proper policies so you can take advantage of existing policies that you already have in place using a mechanism like that.

[00:14:49] Daemon: Yeah, I also think that understanding, the full scope of a query and where it goes is important within an organization as well. I’ve come across a number of different you know, man-in-the-middle type attacks where all of the queries that are done within a model have been intercepted by third parties.

And actually, there was an article recently that came out that said that a number of accounts on ChatGPT I think it was like a hundred thousand were compromised. The threat actors could go in there and look at all the inputs, or all the prompts that were done within that. So having the policy in place where it will automatically remove those and also ensuring that the path to those models themselves is also protected. So, you don’t have some sort of a system in the middle that is seeing all that traffic or getting, that packet data or acting as a proxy to the system.

Another kind of implementation of that is like what you see on Google play like the marketplace. There’s a been a [00:16:00] lot of ChatGPT apps that basically, charged like a dollar or so. This, it’s not as popular anymore, but when it first came out, you could go onto there and get an app, whether it’s, you know, free or 99 cents or so on, and then it would act as that go-between to ChatGPT, which allows it to get all the information as you’re inputting it into, the environment.

[00:16:26] Tim: So, absolutely. Yeah. No, Yeah, you bring up a very important concept there. So, all of those things are outside of the core model itself, right? Mm-hmm. Models by default do not have a memory. If they do, it’s like short term in memory, like literal ramp. Most implementations of a large language model utilize a vector database, for example.

Because again, it’s all map, right? Something like, or deviate even like the open search has a vectorized overlay on it. There’s a Postgres version as well, but either way, like those are still constructs that you need to secure. And in the case of the ChatGPT leak, for example, that’s what happened. In that case it was the vector database, was that stored all the queries, all the memory of the prompts and responses.

And that’s just an efficiency layer as well. So, for example, if you ask a large language model a question typically you cache both the prompt and the response. That way if you’re asked the same question again, sometimes you can fall back to a cache answer unless you choose to regenerate your response.

Saving, GPU costs at the end of the day. Yeah, I think that’s absolutely important.

[00:17:35] Daemon: The way to approach a lot of these kinds of issues where you need to understand the full path and response to the model is taking the same approach that you have with understanding supply chains.

So that, you know, how secure is each segment of that traversal over to the model. How can you ensure that there are certain security controls in place to ensure that you don’t have a data leakage out in one point or another? I think that the way that some organizations are doing that is opting to have local models as opposed to using the public models.

And then, they’re creating their own foundational models and then, they’re adding additional data onto that. But that also poses different kinds, of issues because, they have to become the data scientists. At that point, they have to understand the implications of training an entire data set on all of their internal IP, and then perhaps having some sort of a chatbot interface over to the public so that they can get information on certain docs, or so on.

But the model is trained on all of their internal IP, so understanding what the egress of that data looks like, is really important.

[00:18:49] Tim: No, absolutely. And when it comes to deploying a foundational model, you even need to validate the source of that, that foundational model itself, right? Make sure it’s some, an organization you can trust. Make sure it has a reputation perhaps using binary checksums, or any of our traditional approaches there for validating the actual binary of the model itself.

I mean, you need to make sure that the chain of custody has not been breached before you can start using a model. When it comes towards actually implementing it, and let’s just say you’re a software development company and you want to use a code generation tool, you have to make sure that you trust that as well.

Especially if it’s a third party service. Now I’ll pick on Amazon Code Whisper a little bit. And I do feel like Amazon is a trusted source model, right? And if you read the FAQs for Code Whispers, they do not use your data. They do not store queries for continuous model development. But that’s something that you need to keep in mind as you’re typing.

Source code is a third party keeping that data. I mean that’s where your company’s IP starts is, a developer using an IDE, right?[00:20:00] There’s also things to look out for in the integration phase making. Communicating with the model securely that you’re not subject to network sniffing or things of that nature.

Making sure you’re authenticating against the system that hosts the model. And even then, when it comes to like deployment and distribution you need to make sure that. Your, your source is, is free of malicious code of vulnerabilities. Even if your, your foundational model comes from a trusted source, which doesn’t mean that it’s been scanned or that it’s secure, right?

But these are all things that you need to take into consideration as well. That, and the privacy concerns as well. Making sure that any data that’s transmitted as information. All these things can be overcome though by using things like, regular audits,.

Using diverse training data, a little bit of unit testing, but unit testing gets interesting as well because like I said, there’s output variability in a large language model. So, some things are easier than others. For example, if you ask a large language model: Hey, what’s the API spec for this particular function-call?

You should always get a particular sub strain within your response. Now the words around it might differ. But you need to get collaborative with unit testing and sometimes you can actually use an LLM to unit test another LLM to interpret the results.

[00:21:24] Daemon: Yeah. Yeah. One interesting development that I’ve seen in the last little while is ChatGPTs capability of using plugins in order to connect into other systems, so that in my mind, poses all kinds of different risks associated with that.

Especially if you don’t necessarily know what those plugins are or what they’re actually doing, the systems that they’re connecting to. It goes back over to like the wild, wild west of the app store, you know, back in the early days. Yeah. And yeah, it’s, it’s scary.

And an interesting development that I’ve also seen is that, now that you can browse the web with ChatGPT. You know, ChatGPT-4, the paid version.

And also, with some of the tactics threat actors are starting to create what, you know, we call AI Canary Traps.

Where they actually are able to put malicious code into something that may be searched by an LLM, so that the response brings back that malicious code into the response or somehow exploits or a vulnerability within the model or the user. And those are becoming, more common these days.

And I don’t think that there’s any real solution to that yet because it is just so new.

[00:22:47] Tim: Yeah, no, I agree with you. There it comes back to kind of unit testing or sanitizing results that come back. So, I think the default stance there is you really can’t trust what comes back, kind of one of these models unless you validate it.

And that that’s what gets you in trouble. And that, that kind of takes you back to the court case example, right where the output came back. It looks legitimate, but it definitely was not valid. So, it’s the same thing where if a canary trap like that returns malicious code, you should absolutely be using a code scanning tool No different than what you doing today without it yet, right. To make sure that your code is safe.

[00:23:25] Daemon: Yeah. And going back over to that lawyer example I actually read, in that article that the lawyer did ask ChatGPT, whether those sources were valid. And it just completely lied to him and said, “yes they are”.

[00:23:37] Tim: Well, yeah. And, in the model’s mind it was accurate because it was in its corpus of training data.

And that’s where a model falls short like you train. It’s kind of like teaching a child something, right? An oversimplified example that I can use is I remember picking on my daughter as she was growing up saying, hey, chocolate milk comes from brown cows. But if she grows up thinking that that’s a fact she’ll defend that fact when challenged about it. And it’s no different than a large language model. It thought that those court cases were legitimate and answered with what it understood to be the truth.

[00:24:13] Daemon: Yeah. But what’s the fallout from that? In the end who’s responsible for the outcome, essentially comes down to the person that is implementing the output of the model.

However, in the future, the onus of providing that information may go back over to the actual model creator. So, I think right now there’s some possible disbarment that’s going on with the lawyer. And in the case, of like a doctor, for instance, it could be malpractice lawsuits, but in the future, that may shift going to the companies that either house the AI, or are part of the supply chain for the AI or, you know, the people that created the model itself.

[00:24:55] Tim: Yeah. And how far are we from having models that create other models and, and then where does that responsibility lie? I often joke with folks in this space, I say the only thing stopping Terminators from running around today is battery technology because we’re getting to that point in life where technology can produce more technology.

And we see oversimplified examples of this now. I came across a project this morning actually called EngineerGPT. Where you can tell it; “Hey, I want it written in Python. Give a snake game and then I’ll ask you a couple questions to clarify how you want to navigate your snake.” Well use the keyboard and then I’ll actually generate a working app for you.

It’s crazy. So, we already have code writing code today. Yeah. It’s not out of the question to think what would that look like if a model developed a model, but then exactly to your point, where does the responsibility fall on that?

[00:25:47] Daemon: Exactly. And, I’ve come across the AutoGPT project as well, where it instantiates sub-agents in order to perform different tasks, bring that information back and act on it.

I know you can also do what’s called model stacking where you use different models, maybe trained in different areas in order to get information. Or hone the output at one model into another model, and so on. So you don’t necessarily know the bias that’s associated with each one of those, or the security that’s associated with the supply chain, of each one of those. So yeah, it’s, it’s the wild west right now it’s really crazy.

[00:26:20] Tim: Absolutely. And now we’re entering the phase of multimodal models as well. For example, you can, you can take a, a picture of your finished dinner plate at dinner as it’s served to you and ask “how would I make this at home?”.

And the model can take your image, understand that, okay, hey, this is Chicken Parmesan. Here’s exactly the ingredients you need. Here’s a recipe for it to match the restaurant style. It’s, getting crazy.

[00:26:53] Daemon: Yeah. Well, thank you so much for coming on the podcast today. I really appreciate all your insight. Before, we go and finish the podcast, is there anything that you would want to leave our listeners with?

[00:27:05] Tim: Yeah, I mean, just always ask questions. Always ask why for example, why does ChatGPT only return to that number of characters? Why do you get different outputs? Why do you get a different result if you change one word in your prompt? What’s the constitution for your model? What are the privacy conditions, right? Ask the questions. If you don’t ask the questions, you’ll regret it. Learn and be curious. That’s the number one thing I want to leave you with. Ask away or you’ll never know.

[00:27:32] Daemon: Awesome. Thank you. And if somebody wants to get ahold of you on the internet, what would be the best way to do that?

[00:27:41] Tim: Yeah, absolutely. You can look me up on LinkedIn Timothy Patterson or you can send me an email. My email address is tim  patt and then the number 16, so Tim patt 16 at gmail.com.I’d love to hear from you.

---

About the author

---

Other recent articles of note.