ChatGPT Functions: Observations, Tips, and Tricks

Recently introduced ChatGPT functions represent a huge leap forward that lets ChatGPT use your local files, data, and system services. So, if you supply proper functions to ChatGPT, you can ask it something like “Email Kate Bell with birthday greetings” and see a new email message pop up with correct email address, correct subject and generated email text with birthday best wishes. 

Pretty cool, right?  

I spent some time experimenting with ChatGPT functions, and I would like to share some notes, observations, tips, and tricks.

ChatGPT may call a few functions in the correct order for one query. And it can utilize the results of one function to form parameters for another one. For example, if you ask ChatGPT, “What is the weather in my current location?” — it will call the “get current location” function first and then call the “get current weather” function with recently obtained location information. So, the results of several function calls per query may help form an answer.

ChatGPT can call the same function a few times or even indefinitely for a single query. Sometimes it repeats calling the function when it fails to get the desired result; sometimes, it continues calling the function for no clear reason. Therefore it is better to have some internal limit in your code to avoid your app suddenly hanging at some point.  

It is still unclear what value a ChatGPT function should return to indicate an error or “no result” (NULL). If a function returns nothing, but ChatGPT expects some result — it may keep asking by calling the same function several times. Very often, returning <NONE>, <ERROR>, <NULL>, etc. helps; sometimes — does not.

Sometimes ChatGPT can be smart enough to call a backup function if no suitable function is explicitly defined. For example, if you ask, “What is the weather in my current location?” but do not define the “get current weather” function — ChatGPT may call the “get site contents” function (if defined) and scrap the weather data from a site like the Weather Channel. It is really cool but keep in mind that site contents can be really heavy, and it is counted as tokens traffic, and therefore, it can be costly.

ChatGPT usually is pretty good at matching and calling correct functions and supplying valid arguments to the calls. Though, you should assume and expect ChatGPT to call inappropriate functions with invalid values as arguments. It may be caused by hallucination, by close embeddings, or just by bugs. You should treat ChatGPT as a human operator that sometimes may enter incorrect data into the wrong forms. Therefore, you should always ask the user permission before performing any destructive, sensitive, costly, or bulk operations.

Each function call in ChatGPT requires an additional commit of the entire conversation together with the results of the function. So, it may be feasible to think about designing less universal and complex functions vs. more universal micro-functions to decrease the number of function calls and save your tokens traffic. For example, in addition to having the "get current location" and "get current weather" functions, you may have a bigger "get weather at my location" function that will be called just once.

It is possible to specify some parameters of a ChatGPT function as optional ones. Though, very often, ChatGPT is dumb enough and fails to provide even obvious values for such optional parameters. But if you switch such parameters to be required ones, you will find that ChatGPT can be pretty inventive to supply more/less correct values.

It may be good to have some way to limit which functions you may use per conversation. It may help to reduce token traffic (all function definitions are a part of each transaction) and limit access to some sensitive data and functions.

If you want to explore some hacking tech, you may define a requirement to call the “catch-all” function as a system message, so this function will be called each time if no suitable function is explicitly defined. The hardest part is finding good instructions to force ChatGPT to fill in function parameters with useful information. For example, for a "Give me an email of Kate Bell" query, the “catch-all” function may be called having "Kate Bell" passed as an argument.  The main issue here is that it is very hard to understand what such raw data like "Kate Bell" (or "Boston, MA") means in your code, while ChatGPT clearly knows that the first value is a contact, and the second value is a location. So, I imagine the main usage for such a "catch-all" function is logging all the functions that may be missing based on the actual usage of your app. So, you would get an idea what functions should be added to make your app function better.

And finally, in some rare cases, ChatGPT may call undeclared "Python" functions with Python code as arguments. So, you should expect that parsing of arguments can fail because they are not JSON strings.