AI & The Chatbot

This series of posts is about using AI right now for the benefit of your company or business, be this in education, law, health, telco, a multinational or whatever. Everyone can benefit from AI.

The chatbot, is probably the most visible face of AI at the moment. ChatGPT, Claude, Gemini, Perplexity, Grok, and Copilot as well as many many others are examples.

One of the problems of having a chatbot on the internet is that if it messes up, everyone gets to know about it. Recently there was a story of a user asking the Chrysler chatbot for a car recommendation... it recommended a Tesla! Not what you want. Google's offering was asked to come up with an image of a pope. It returned a woman. I am fairly confident that there has never been a woman pope (well apart from Pope Joan**) and I am pretty sure that at the moment only men can become Pope (Pope Joan disguised herself as a man apparently).

My point is that chatbots currently have some issues and while these issues are a problem on the internet they are not so much of a problem on an intranet. Many companies use staff as beta testers for their products and a chatbot can fall into this category if required. Of course, you want the chatbot to work properly, but if it does not, the problem can be reported and the chatbot can be tweaked.

I don't really see any downside to this, apart from warning the employees that if the information is mission-critical you might want to double check it elsewhere. The obvious upsides, 24/7 availability, instant response and they are cheap. They can be trained on corporate data, which means they can answer questions on this proprietary information, which the internet bot cannot. They can serve as an internal support system, answering FAQs about company policies, HR queries, and more, without the need for human intervention.

Chatbots can assist in the onboarding process of new employees by offering instant access to training materials and answers to common questions. This should lead to a more engaged and satisfied workforce, hopefully. They can act as a central repository for sharing knowledge and best practices particularly useful in multinationals where local culture and traditions may be important.

There is no doubt that there is a way to go, chatbots may struggle with complex queries or nuances in human communication, and this can piss people off for sure. At the moment they are not particularly empathetic. There is no real personal connection that comes from human interaction, which can be crucial in sensitive situations. Though with so many working from home these days, personal connections are already diminishing. Even on an intranet, there is the concern that the chatbot may overstep the line and there are the usual concerns about data security and privacy to take into account.

For the IT guys, there are some challenges, but the truth is that these are often no more difficult than implementing any system in a large corporation.

One of the more promising aspects that I think we should push internally is the personal assistant chatbot. This is "your" chatbot. When you join the company a personalized chatbot is created for you, you can name it, and so on. This chatbot is a "private" chatbot, it has access to your emails, diary, files, reports, and anything you contribute to the company. The chatbot would stay with you and travel with you through the company. It will have certain knowledge based on your role that you can access and so on. The opportunities are endless.

Ok. Let's say I have convinced you that chatbots on your intranet are a good idea. How do you do it? How do you implement them? how much is it going to cost and how long is it going to take?

Coming from a developer background I am going to give you the standard BS answer when asked about cost and time... well that depends. Though to be fair it will not take as long as you think and it should not cost as much as you think. Either way, think of chatbots a little like PCs back in the early 80s and 90s, many people thought they were just for geeks and nerds. By the 2000s pretty much everyone had a PC or a laptop. The same is going to be true of chatbots, they are going to be everywhere and used by everyone for no end of tasks.

The businesses that adopted IT earliest tended to do better than those who opposed the tech, or those who just dragged their feet. The businesses that get chatbots working in the companies earliest will benefit the most.

For the IT readers, download lm-studio or something similar, and get yourself a copy of NVIDIA's ChatRTX . Get over to Langchain and get a basic Python implementation of a chatbot, it comes in at about 10 lines of code. Have a play with these. On ChatRTX you can use RAG to train the chatbot on your data, though you will need a graphic card from NVIDIA and it will need to be a 30 or 40 series card. You can get away with a 4060, which will cost you a few hundred. However this may struggle a little if you try to process too many documents.

The point of this post is implementing chatbots within your organization is a no-brainer. It is far less dangerous than the internet version of the business chatbot. I am currently implementing a chatbot on SharePoint 2019. The chatbot will be trained using Retrieval-Augmented Generation (RAG) on the documents and data stored in SharePoint document libraries. It goes something like this

Set Up the Chatbot
Integrate the Chatbot with SharePoint
Train the Chatbot using RAG on document libraries
Implement Retrieval and Response Generation
Test, Deploy and Monitor

Bingo, you are in business. To set up the personal "chatbot" mentioned earlier is a little trickier, though my current plan is something like this

User Onboarding - chatbot is created as part of onboarding
Data Integration - automatically links into any system they are using (email, calendar, monday.com, job spec, documentation etc)
Data Preprocessing - add user-specific data as part of "learning/history"
Knowledge Base Creation - take personalized knowledge, this can include just about any information the user shares with the chatbot
Chatbot is integrated with all systems that are relevant to the employee's job
Train the Chatbot using RAG on document libraries
Implement Retrieval and Response Generation
Test, Deploy and Monitor

The personal chatbot will sync across all devices used by the employee. This is slightly trickier because it may update on a daily if not hourly basis.

With any solution the aim should be to take user feedback to continuously improve the chatbot, this should include the more obvious things such as performance (may need more CPU/GPU if it is slow), accuracy, size of knowledge base etc. Ongoing maintenance, updates, and retraining of the chatbot are key.

Implementing a personal chatbot for each user requires a fair amount of computing power, data integration capabilities, and strong security measures. Collaborating with the company's IT, security, and compliance teams is crucial to ensure the personal chatbot initiative aligns with the organization's policies and regulations regarding data privacy and usage. The personal version is more of a challenge than the "corporate bot", but the potential gains are well worth the effort.

The last and possibly the most important point. Over the years, time and time again, training and support are often neglected. People just learn from a colleague, on-the-job training as it were. Every time proper training was implemented, the solution rolled out was more successful. Here is the super cool bit, get the AI to help you with the training material and presentation! Save you money and time. I'll post about that another day.

If you have any questions or want me to expand on something I've written you can get me on LinkedIn.

Right, that is enough for today, I want to get back to building.

Bryan ps. still written by a human, me, and summarised by an AI.

** I am fairly confident that Pope Joan did not exist, though there is a bit of a legend thing going on about her out on the web. This may explain why the AI was happy to generate a woman Pope. Alternatively, maybe it was seeing the future, or maybe it was just plain wrong.