Have you ever been confused by some of the online security warnings you see? Sometimes you see so many you want to skip over them before your eyes glaze over. This tutorial will show how I used AI to better understand a browser extension warning from a scanning tool. It will also show some of ChatGPT’s quirkiness.
If you read my article on using CRXcavator to evaluate browser extensions, you saw a bullet point, “Use AI tools such as ChatGPT or Claude to explain terms you don’t understand.” There was a developer documentation page, but I wanted a simpler explanation. So, let’s see what we can find out about a particular web request API.
Choosing an AI Tool
We can use several AI tools for this query, but I will start with ChatGPT 3.5. It’s a large language model (LLM) chatbot. There are several reasons why I chose this bot.
- This version is free for most users, and creating an account is easy.
- The info I’m requesting has been around for a while. ChatGPT has a cut-off date in late 2021.
- I don’t need to have ChatGPT go out to the internet.
There are other free tools out there that would work as well. For example, you could use these same prompts in Claude 2. It’s also free during its beta phase. Another option is to compare various LLMs using Chatbot Arena.
Defining What We Want
Have you ever noticed there’s a correlation between the quality of your question and the answer you get? AI tools are similar, so I like to think about what I need. What problem am I trying to solve? Let’s go back to one of the screen snaps with the warning.
In this case, I’d like to know:
- What is webRequest permission?
- What is chrome.webRequest API?
- Does it know the specific browser extension?
- Why is this a critical risk?
Providing Context About Our Request
The next step is to provide ChatGPT with some context about who we are and why we want this information. This helps the service answer the question in relation to our request. In this example, I want to tell the service:
- I want a simple explanation.
- I want the answer explained to me like I’m a 10th-grader.
- I want to know the pros and cons of the chrome.webRequest API.
Obviously, you can phrase the prompt in many ways. There is nothing magical about my approach, and you’ll see I tend to break up my requests.
One step I build into my requests is to have the service acknowledge my instructions and pause until I type “OK“. I do this so I have an opportunity to double-check my initial request.
Building the Prompt
I prefer to write my request in a text editor like VS Code. One reason is that it’s easier to add line breaks. Like most chatboxes, when you hit Enter, your prompt is sent. If you type your prompt and want this spacing, you need to use Shift + Enter.
- Log into your ChatGPT account at https://chat.openai.com/.
- Click the New chat link in the top left.
- Click GPT-3.5 from the top of the main panel.
- Go down to the Send a message textbox at the bottom of the window.
- I pasted in the first part of my prompt.
- Press Shift + Enter to add some spacing.
- Optional (in yellow): I like to have the system restate my instructions.
- Press Enter or click the green arrow icon.
- ChatGPT will append its reply underneath my prompt.
- At this point, I’ll reread my prompt and then reply by typing “OK” in the message box and pressing Enter.
Reviewing ChatGPT Results
After I press Enter, the system starts to answer my query. In my case, it extended its answer to several screens. Let’s break down the results.
- I told ChatGPT that I was ready for the answer.
- The response included icons to the right. These are useful if you want to copy the response to your clipboard. The thumb icons allow you to provide feedback to ChatGPT.
- The service starts to answer my query in terms a 10th-grader would understand.
- I’m starting to get my Pros and Cons lists.
- If I don’t like the response, I can click the Regenerate button and get a different response.
Continuing the Conversation
Now I know why the browser extension might need chrome.webRequest API. But I’m not done. I still don’t know about the webRequest Permission and the specific extension. And you’ll notice my initial prompt didn’t put any focus on risk.
One benefit of these systems is you can keep the conversation going so long as you don’t exceed your token limit. So, let’s provide a follow-up prompt.
Some people might wonder why I didn’t start with this prompt. I could’ve, but I prefer doing an iterative process, especially if I don’t know the subject matter. As you’ll see, the system doesn’t scold me but is quite polite in its reply. Here’s the first part of the reply.
Does it Know the Extension?
The last part I’m interested in is whether ChatGPT knows of the specific browser extension. One issue is that the service isn’t current. It’s relying on information it’s already seen during its training. It’s not going to the internet to find a current review or view the description page. Let’s give it a try.
As you can see, it answered our query but acknowledged it has a cut-off date. However, it provides some detailed information regarding the risks.
I should also mention that instead of providing just the browser extension name, I could also have pasted in an additional text I found online, such as vendor documentation. The paid version of some tools allows you to provide URLs or files.
Asking ChatGPT for Help
I’m not an expert in prompt engineering. I find there are often things I’ve learned through the course of the conversation. And one item I like to close with is a critique. This is where I ask the AI tool what I could’ve done differently.
Sometimes, this means asking it to write a more succinct prompt. Other times, it might be asking it to tell me what questions I didn’t ask but should’ve.
Certainly, this suggested prompt is more succinct. I opened a new session, fearing I was getting close to my token limit, and used the suggested prompt. ChatGPT replied but no longer knew the IBM Security Rapport browser extension.
In this case, I thought it was that the revised prompt missed the word “browser”. I added the word “browser” to the prompt, but after several tries, I did not get it to recognize that browser extension. My guess is that using my early iterative approach did something that made it recognize that extension.
This incident brings up another quirkiness with ChatGPT and other tools. You can take the same prompt, enter it into the text box and get a slightly different answer. The main points remain, but the wording has changed. This is by design, so if you follow my steps, your reply will differ too.
Ask for More Documentation
As good as these AI tools can be, sometimes you want more information. This is particularly true if the subject concerns security, health, finances, etc. You need to remember these tools were trained on content from the web and various sources. And as we should know, sometimes the internet has incorrect information or things have changed.
You can prompt ChatGPT to suggest more documentation or sources in these situations. In the example below, I asked about documentation. The system responded by providing links to some developer documentation.
Key Points & Takeaways
The next time you get some security message or warning you don’t understand, try ChatGPT. It may take several prompts, but it may also save you some trouble down the road. Here are some tips to remember:
- ChatGPT is a tool, and the answers it provides depend on your prompts.
- Think about your problem and the info you need.
- ChatGPT will not give the same answer even if you reuse the exact prompt. It introduces a degree of randomness by design.
- ChatGPT doesn’t have information after September 2021.
- If you don’t understand a response, ask ChatGPT to clarify.
- If you find a prompt you like and regularly use, consider adding it to TextExpander.