Like everyone it seems I have been playing with ChatGPT and Auto-GPT trying to understand their capabilities, strengths and weaknesses and get a feel for where the technology is heading. Here are my observations thus far.
GPT-3, GPT-3.5 Turbo, GPT-4 and Bard
I saw a big jump in response depth, comprehension and usability between GPT-3 to GPT-3.5. In most of my use-cases the differences between 3.5 and 4 have been largely incremental but that is probably more a reflection of how I’m using it than a true reflection of the power of the tools. It seems GPT-4 is necessary when you are wanting a significant analysis – lots of tokens both input and output – or when the subject matter depth is high. Generally speaking GPT-3.5 Turbo seems great for most use-cases and I love the speed.
I have access to Bard but I haven’t had enough time with it yet to draw any conclusions.
Bing Search powered by AI
For a long time I have preferred to read web content via Pocket or via similar services that convert the web to pure text for a few reasons – it kills all the unnecessary clutter, it’s so much easier and comfortable to read and it’s much more efficient. Bing Chat is a level above this by reading it for you and summarising. For most of my day to day web searches I am finding Bing Chat is the best option especially if you want a quick answer to a question. There are still a few weird quirks:
- I asked a simple “how many days until this date” question and it got it wrong
- It sometimes just states terrible web information as fact
- Even though they are pushing it for things like recipes I found the recommendations kind of lame and there was limited way to say “I don’t like these”
Auto-GPT
Auto-GPT is an experiment that sits on-top of GPT and attempts to automate the prompt, read and interpret feedback loop. It can search the internet, execute code (in the latest version), run GPT agents and has both short and long-term memory. It’s both disappointing, inspirational, mind-blowing and weird all at the same time.
Disappointingly so far I haven’t had much success at getting it to write a program – even a very simple one. I thought perhaps this was the purpose and what it would be best at. It started to do very strange things when I asked it to write an Alexa skill like googling “how do I write a python function”.
For researching, it’s a very powerful and clever tool. I asked it to find books for my wife who wanted a new novel with a female protagonist, sci-fi but not military, that had an audiobook and was similar to “The Lost Tomb” series. With a few instructions I set Auto-GPT to work and asked it to summarise the plots of the top 5 books it found meeting these criteria. It took about 10 minutes and came back with a summary of books all meeting this criteria and she is working her way through them. Something we could have done ourselves perhaps but it’s absolutely incredible AI can do a fairly complex task like that for us.
Watching it “think” is surreal as it thinks through every action and considers the pros and cons of what it is about to do. The introspection and consideration of risks or downsides of a course of action is the most impressive part of this technology. I can see it’s very much in the early days but it’s already useful and is basically at a level of a junior employee or assistant.
Conclusion
I’m having incredible fun exploring and playing with what this AI can do and thinking about how it will impact my work and personal life. I firmly believe we are witnessing history in the release of this technology and the disruption, empowerment and change this tech will bring will be enormous. I’ll continue to provide updates as I go.
Leave any comments on your own Generative AI experiences if you like.