Coaching programs that rely on artificial intelligence are cheap, accessible and increasingly popular for practicing tough workplace conversations or getting tips for negotiating a raise. But privacy questions and skepticism over the quality of non-human answers could slow adoption.



<hr class="wp-block-separator has-alpha-channel-opacity"/>



You’ve trusted artificial intelligence&nbsp;to plan a vacation itinerary, write a cover letter for a new job and even&nbsp;<a href="https://www.forbes.com/sites/alexandralevine/2024/02/14/dating-apps-ai-chatgpt-tinder-hinge-bumble-match/#:~:text=YourMove%20AI%20bills%20itself%20as,and%20generate%20dating%20photos%20compatible">flirt on your behalf with your next date</a>. But would you trust it with career advice?



A growing number of businesses are betting you will. Coaching or workforce learning platforms like BetterUp, Multiverse and LinkedIn—as well as startups like Valence and Wisq—are only some of the AI-powered coaching chatbots entering the market.



During a time of layoffs and budget cuts, as companies seek to help workers hone their interpersonal skills, AI is emerging, ironically, as a cheaper and more accessible alternative to traditionally high-cost human coaching.



AI coaching refers broadly to a variety of apps and programs that are powered by generative AI technology and provide advice about tricky situations or career challenges. Users query a chatbot and get real-time, interactive feedback on everything from the best way to negotiate a raise to how to search for a new job to how to delegate work or give feedback to teammates.



This is still a relatively niche business—Gartner’s HR practice tells&nbsp;Forbes&nbsp;it is still too small of a market to measure—but both employers and workers are starting to embrace it. Its emergence comes at a time when more people and companies are seeking help with career development, and understandably so. In an economy where fewer people are quitting or changing jobs, workers are nervous about promotion opportunities. Meanwhile, many also feel frustrated by a lack of hands-on management training during the pandemic or concerned about how AI is already reshaping job descriptions and making some of their skills obsolete.



Some 47% of companies surveyed by LinkedIn for its&nbsp;<a href="https://learning.linkedin.com/content/dam/me/business/en-us/amp/learning-solutions/images/wlr-2024/LinkedIn-Workplace-Learning-Report-2024.pdf" rel="noreferrer noopener" target="_blank">2024 Workforce Report</a>&nbsp;said they are investing in career mentoring and coaching for their workforce, and a survey by Gartner found that 42% of employees would be comfortable asking an AI coach about the next steps in their career. Another survey by&nbsp;<a href="https://www.intoo.com/us/blog/unlocking-organizational-success-report-press-release/" rel="noreferrer noopener" target="_blank">INTOO</a>, a career and outplacement firm, found that nearly half of respondents aged 21 to 26 think they get better advice from AI, including ChatGPT, than from their managers.



So it’s no surprise that startups are attracting investments and companies are rolling out new products. For example, in June, Menlo Park, Cal.-based Rising Team closed an&nbsp;<a href="https://www.prnewswire.com/news-releases/rising-team-raises-8-million-series-a-and-introduces-personalized-ai-leadership-coach-for-building-high-performing-teams-302165554.html" rel="noreferrer noopener" target="_blank">$8 million</a>&nbsp;seed round that it’s using in part to add an AI leadership coach,&nbsp;<a href="https://risingteam.com/#aRTiHelps" rel="noreferrer noopener" target="_blank">aRTi</a>, to its team performance platform. Management AI coaching tool Wisq, which has raised over $40 million in funding since May 2021,&nbsp;<a href="https://www.prnewswire.com/news-releases/wisq-launches-first-personalized-ai-guide-for-managers-302113194.html" rel="noreferrer noopener" target="_blank">launched a new program</a>&nbsp;aimed at mid-level managers in April.



“Employers are all over it,” says Ujjwal Singh, chief technology and product officer at London-based apprenticeship startup Multiverse, adding that 64% of its customers now use its AI career coach. That’s up 15% from last quarter. “They’re really pushing this hard.”



Yet even as demand rises, AI career coaching faces hurdles to widespread adoption, including concerns some workers may have about the privacy of employer-provided sessions and questions about the quality of tools’ responses. More customized responses rely on personal information users may be hesitant to give—and on the real-life experiences that human coaches can draw upon when counseling clients.



Some early research suggests that in some scenarios, AI coaching can be as effective as the human version. For example, a study&nbsp;<a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0270255" rel="noreferrer noopener" target="_blank">published</a>&nbsp;in the journal PLOS found that research groups using human coaches and those using AI coaches were just as efficient when it came to reaching set goals.



But at this point, skepticism is still warranted. “We&#8217;re a long way from a typical human being wanting [coaching] from a machine,” says Joseph Fuller, a Harvard Business School professor who co-leads its Managing the Future of Work initiative. But, he adds, given the rapid improvements in AI, “it’s pretty dumb to bet against them.”



This much is for sure:&nbsp;AI coaching is a lot cheaper for employers than the human variety, which, with an average cost of&nbsp;<a href="https://coachingfederation.org/app/uploads/2023/04/2023ICFGlobalCoachingStudy_ExecutiveSummary.pdf" rel="noreferrer noopener" target="_blank">$244 an hour,</a>&nbsp;has typically been provided only to companies’ top performers or senior executives. (For comparison, Wisq charges between $50 and $150 per user per year for a combination of human and AI coaching.) It&#8217;s also designed to be accessible, with several AI coaches integrated directly into apps most workers use daily, such as Slack or Microsoft Teams.



In 2023, the American market for coaching was worth $14.2 billion, according to market research firm IBISWorld. During the pandemic, Zoom enabled coaches to take on remote clients, expanding their client base further, points out Carlos Cuadrado Ortiz, an associate principal and coach at consulting firm Korn Ferry.



AI could be a big accelerator for the industry, enabling companies to provide coaching to more employees and human coaches to serve more clients by using AI tools to address the simplest questions. That explains why some big career businesses are adding AI features or acquiring smaller startups. LinkedIn, for instance, launched an AI virtual coach in October that helps users find new jobs personalized to their profile. The AI coach’s answers pull from large language models that are&nbsp;<a href="https://www.forbes.com/sites/mariagraciasantillanalinares/2024/06/13/linkedin-releases-ai-resum-and-cover-letter-help-for-premium-users/">trained</a>&nbsp;on advice provided by a handful of human coaches who earn royalty fees.



In March, BetterUp, a coaching and mental health platform that has reached&nbsp;<a href="https://www.forbes.com/sites/katiejennings/2021/06/07/venture-funding-for-mental-health-startups-hits-record-high-as-anxiety-depression-skyrocket/">unicorn</a>&nbsp;status (it was last valued at $4.7 billion in 2021 according to Pitchbook), acquired Practica, an early AI coaching provider, to grow its own AI offerings. (The company still offers human coaching to all users.) The new AI services include a “role play” function that lets users practice conversations with a voice-prompted chatbot about thorny topics, such as asking for a raise or sharing critical feedback. The AI might tell a user, for example, that they sounded defensive when delivering bad news or too nervous when making a case for a promotion.



Meanwhile, startups like Toronto-based&nbsp;<a href="https://www.valence.co/" rel="noreferrer noopener" target="_blank">Valence</a>&nbsp;and Paris-based&nbsp;<a href="https://coachello.io/" rel="noreferrer noopener" target="_blank">Coachello</a>&nbsp;added AI chatbots to their online coaching services last year. Grettel Seiger, a leadership development manager based in Basel, Switzerland, started trying out Coachello with a small group of employees a month ago and says it’s been a way for colleagues to get help earlier in their careers without the high cost.



Those who’ve started using it have opened up about the dilemmas they face, she adds, noting “you don’t feel judged” like you might with some managers. Asking the AI chatbot follow-up questions has also helped her define exactly what help she needs. “The whole process became cathartic,’’ she says.



<hr class="wp-block-separator has-alpha-channel-opacity"/>



But good coaching,&nbsp;and good advice, is rarely about quick answers; it relies on probing questions—or in AI speak, collecting more data. “Coaching is about guiding someone in a certain direction, [or] nudging people,” says Multiverse’s Singh. “I don’t think AI is quite there yet” when it comes to complex, relationship-focused questions, he adds.



At a broad level, some AI tools have been known to&nbsp;<a href="https://www.forbes.com/sites/rashishrivastava/2024/07/11/this-ai-powered-coach-catches-hallucinations-in-other-ai-models/">make up answers</a>&nbsp;or at times give&nbsp;<a href="https://www.forbes.com/sites/cyrusfarivar/2024/07/23/target-employees-hate-its-new-ai-chatbot/">wrong information</a>. Massara Almafrachi, a third-year law student at Western New England Law School, says that asking an AI chatbot for career advice has resulted in confusing and irrelevant answers. “I’m already confused about my career,” she says. “AI [only] throws me in a bigger loop.”



Katie Kirsch, the founder of a coaching company called Lume and a 2024&nbsp;<a href="https://www.forbes.com/profile/katie-kirsch/">Forbes&nbsp;Under 30 listmaker</a>, says she may one day incorporate AI features into her platform. But earlier this year she tried out AI tools herself and found the advice lacked nuance: &#8220;The kinds of conversations [my human coach and I] had felt impossible to replace in the AI format.&#8221;



AI chatbots, after all, are only as powerful as the quantity and quality of the data they have been fed, and how well the tools are built. To provide a personalized approach, AI coaching companies must successfully coax users to divulge specific, personal information, says Harvard’s Fuller. “You end up having this chicken-or-the-egg problem of not having access to a lot of data until you’ve got the quality that you can only get by having a lot of data,” he says.



Then, there’s the question of sensitive information. Most AI coaching companies offer their services directly through employers, which could make users leery of asking what they really want to know. How many employees who want advice on leaving a job or dealing with a toxic boss would feel comfortable entering those questions into a chatbot offered by their employer?



AI coaching providers respond that confidentiality agreements prevent the tools from sharing personal details with employers, and any reports generated for employers show only aggregate and anonymized results. (Some, such as BetterUp, also say users can opt out of having their data train language learning models, while others, such as Valence,&nbsp;<a href="https://support.valence.co/en/articles/9100961-valence-s-ai-coach-nadia-introduction-and-faqs#h_06ffdb4c89" rel="noreferrer noopener" target="_blank">say</a>&nbsp;it does not use employee conversations with its chatbot to train its models.)



Meanwhile, “workers are generally aware that a company-provided tool is tailored to enhance the company’s goals, which don’t always align with theirs,” says Hatim Rahman, management professor at the Kellogg School of Management at Northwestern University, who studies AI’s impact on employment.



Despite potential privacy fears and the fact that AI coaching models still have a lot to learn, tech-savvy workers are increasingly embracing them. “There is no judgment built directly into the tool,” says Michael Woodward, director of New York University’s Coaching Innovation Lab. “It’s purely mathematical.”



In addition to Gen Z workers, another unexpected demographic has become an unlikely power user of AI coaching: Middle-aged men. Multiverse reports it is seeing the most growth among people aged 40 and up; as of June, 46% of eligible employees older than 40 used Atlas, its AI-powered coaching service, compared to only 31% of users 24 and under.



Men in their 40s are one of the largest groups using BetterUp’s AI coaching feature, too. “These are [groups] where coaching may still be stigmatized,” says Moritz Sudhof, vice-president of AI at BetterUp. But if given the chance to share their career dilemmas with a chatbot, they’re actually “ready to open up and bare their weaknesses and problems.”

Chatbot concept. open AI, Artificial Intelligence.

Your Next Career Coach Could Be A Chatbot

<h2 class="wp-block-heading" id="h-topline">TOPLINE</h2>



With companies like OpenAI, Google and Meta dropping increasingly sophisticated artificial intelligence products, crowdsourced rankings have emerged as a popular—and virtually only practical—way of determining which tool works best, and LMSYS’s Chatbot Arena has become possibly the most influential real-time gauge.



<h2 class="wp-block-heading" id="h-key-facts">KEY FACTS</h2>



While most organizations choose to measure their AI models against a set of general capability benchmarks that cover tasks like solving math problems, programming challenges or answering multiple choice questions across an array of university-level disciplines, there is no industry benchmark or standard practice for assessing large language models (LLMs) like OpenAI’s GPT-4o, Meta’s Llama 3, Google’s Gemini and&nbsp;<a href="https://www.forbes.com/sites/roberthart/2024/06/26/anthropics-claude-what-to-know-about-chatgpt-rival-after-latest-model-trounces-industry-giants/">Anthropic’s Claude</a>.



Even small differences to factors like datasets, prompts and formatting can have a huge impact on how a model performs, and when companies choose their own evaluation criteria, it can make it hard to fairly compare LLMs, Jesse Dodge, a senior scientist at the Allen Institute for AI in Seattle, told Forbes.



The difficulty in comparing LLMs is magnified given how closely leading models score on many commonly used benchmarks, with some companies and tech executives&nbsp;<a href="https://x.com/Ahmad_Al_Dahle/status/1780993216118210899" rel="noreferrer noopener" target="_blank">claiming</a>&nbsp;<a href="https://x.com/AnthropicAI/status/1803790676988920098" rel="noreferrer noopener" target="_blank">victory</a>&nbsp;over rivals with differences as narrow as 0.1%., so close it would likely go unnoticed by everyday users.



Community-built leaderboards deploying human insight have emerged, and in recent years their popularity has exploded in step with the steady boom of new AI tools like ChatGPT, Claude, Gemini and Mistral.



<a href="https://chat.lmsys.org/?leaderboard" rel="noreferrer noopener" target="_blank">The Chatbot Arena</a>, an open source project built by research group&nbsp;<a href="https://lmsys.org/about/" rel="noreferrer noopener" target="_blank">LMSYS</a>&nbsp;and the University of California, Berkeley’s Sky Computing Lab, has proven particularly popular and has built AI leaderboards by asking visitors to compare responses from two anonymous AI models and vote which one is best.



Its scoreboards rank more than 100 AI models based on nearly 1.5 million human votes so far, covering an array of categories including long queries, coding, instruction following, maths, “hard prompts” and a variety of languages including English, French, Chinese, Japanese and Korean.



<h2 class="wp-block-heading" id="h-what-s-the-best-ai-model-on-chatbot-arena">WHAT’S THE BEST AI MODEL ON CHATBOT ARENA?</h2>



The top five AI models on Chatbot Arena’s overall leaderboard are:



<ol class="wp-block-list">
<li>GPT-4o</li>



<li>Claude 3.5 Sonnet</li>



<li>Gemini Advanced</li>



<li>Gemini 1.5 Pro</li>



<li>GPT-4 Turbo</li>
</ol>



<h2 class="wp-block-heading" id="h-what-to-watch-for">WHAT TO WATCH FOR</h2>



Figuring out how to evaluate AI models is set to become increasingly important as more AI tools are rolled out and adopted across society. While benchmarks are important, Vanessa Parli, director of research at Stanford University’s Institute for Human-Centered AI, told Forbes they are also important as “goals for researchers to hit when developing models.” It is important to remember that “not all human capabilities are quantifiable” in a way that we can accurately measure but are nonetheless desirable to have in AI models, Parli said. There is also a clear need for benchmarks to assess traits like “bias, toxicity, truthfulness and other responsibility aspects,” especially for organizations dealing with sensitive information like healthcare companies, Parli said.



<h2 class="wp-block-heading" id="h-crucial-quote">CRUCIAL QUOTE</h2>



“The benchmarks aren’t perfect, but as of right now, that’s the primary mechanism we have to evaluate the models,” Parli told Forbes, cautioning that “researchers can somewhat easily game the system” today, with AI models quickly saturating benchmarks. “I think we need to get creative in the development of new ways to evaluate AI models,” Parli said. “



<h2 class="wp-block-heading" id="h-what-we-don-t-know">WHAT WE DON’T KNOW</h2>



Measuring intelligence is tricky when we do not know what it is we are supposed to be measuring. There is no universally accepted definition of intelligence in humans, let alone a way to measure it, and the possibility, nature and scope of animal intelligence has divided scientists for centuries. While AI benchmarks have typically focused on the ability to perform a particular task, more general assessments will be required in the near future as researchers make progress towards their goal of creating artificial general intelligence (AGI). AGI is capable of excelling and possibly matching humans across a broad set of domains rather than at just one task such as walking, moving boxes, identifying tumors on scans and playing chess.



<h2 class="wp-block-heading" id="h-how-useful-is-chatbot-arena-for-evaluating-ai-models">HOW USEFUL IS CHATBOT ARENA FOR EVALUATING AI MODELS?</h2>



“The rankings [Chatbot Arena] gives are something that I trust more than most other rankings,” Dodge told Forbes, “because it uses a real human to say whether they prefer one generation over another.” Parli suggested assessments like Chatbot Arena could “implicitly evaluate factors” we want in our AI but are less quantifiable than something like coding ability. But she did stress that something like Chatbot Arena should not be the only evaluation method used, saying there are “many factors that should be important to organizations when evaluating models” and it “does not cover all of them.”



.

Einstein And Musk Chatbots Are Driving Millions To Download Character.AI

What AI Is The Best? Chatbot Arena Relies On Millions Of Human Votes

Meet Amy. And also Debbie, Inga, Mia, Erica, Eva and Cora. 



These aren’t the members of a new, all-female rock group, but names that several large banks have been giving to their automated digital assistants. 



So-called chatbots have become a useful cost-cutting tools for companies with large subscriber bases (think banks, insurance firms and mobile phone operators). As they replace human call-center workers, such bots will help save banks an estimated $7.3 billion in operational costs by 2023, Juniper Research predicts. 



But the proliferation of bots with female names raises questions about whether they might also perpetuate gender stereotypes, particularly around the notion of women in the role of assistants. That criticism has already been levelled at Amazon’s digital assistant Alexa and Apple’s Siri. 



Now a Forbes analysis of Europe’s 10 biggest banks ranked by assets shows that at least three have deployed chatbots with female names on their websites and apps. HSBC has a chatbot named Amy; Deutsche Bank&#8217;s Debbie helps market traders; ING of the Netherlands has Inga, a chatbot that will &#8220;respond with empathy&#8221; to customer problems such as losing a card.



ING&#8217;s other chatbot Marie, available to retail customers on Facebook Messenger, was given the name &#8220;because it conjures up an image of someone who is helpful and friendly,&#8221; Tim Daniels, a programme manager for ING was quoted as saying on the bank&#8217;s website. (ING has a male chatbot named Bill, aimed at dealing with corporate customers.)



Among the other lenders, Santander, Barclays and Societe Generale appear to have unnamed chatbot assistants. Credit Agricole has an internal chatbot with a male name: Hector. 



Female chatbots abound in other regions and industries. Bank of America recently deployed a digital assistant called Erica, while Mia, the chatbot released by Australian digital bank UBank, was described by the company earlier this month as “empathetic,” “fun” and “a little bit cheeky.” 



IPSoft, a New York-based software company that sells chatbot technology to banks like Sweden’s SEB as well as mobile network giant Vodafone, has its own white-label version of a customer-facing chatbot, named Amelia.



IPSoft’s CEO Chetan Dube denied that the chatbot’s name perpetuated stereotypes, when asked by Forbes during an interview in December, and said it instead highlights “the thought leadership that is represented in females.&#8221;



&#8220;She was the first female aviator that tried to go around the world,” Dube added, referring to the 1930&#8217;s aviator Amelia Earhart. 



Forbes revealed earlier this month that Vodafone was measuring the success of its chatbots on how many staff could be replaced by the software. While that may be an uncomfortable metric, the more worrying consequence of chatbots, according to four industry experts questioned by Forbes, is the risk that they could reinforce certain stereotypes.



“Gender bias is an increasingly serious issue in chatbot design, especially for voice-based chatbots,” says John Taylor, CEO of action.ai, a British startup that makes chatbot software for banks and travel companies. “These assistants often perform tasks that many view as menial.” 



Vitor Shereiber, a language specialist at the German language-learning app Babbel, says that focus-group testing might lead companies to assign a gender to a chatbot on the notion that it makes customer feel more comfortable. 



But, he adds, bots could spread unrealistic expectations of how women should present themselves professionally, just as photoshopped pictures have done for women’s perceptions of their bodies.



Part of the challenge for companies is finding a balance between automating customer service without putting customers off. PwC recently described chatbots as being able to “massively enhance customer delight and loyalty” because of their “personal touch.” 



Taylor suggests software designers should try creating more chatbots with male names and male voices. 



On the sidelines of a technology conference in London on Wednesday, Seth Juarez, an artificial-intelligence engineer based in Redmond, Washington takes it a step further. He calls Siri up on is iPhone to ask the time, and a male voice responds. 



“I make it a guy specifically because I find it morally reprehensible that all of the service-based bots are female, and all the intelligence based bots [like IBM’s Watson] are named after dudes.” 



He added that artificial intelligence generally shouldn’t be anthropomorphised. Chatbots should be used to manage &#8220;cheap thoughts,&#8221; or “stuff that a human would do robotically” rather than on more complex issues. “I would leave those problems to humans.” 



-Parmy Olson;Forbes Staff

Tag: Chatbots

FORBES AFRICA SOCIAL MEDIA

Your Next Career Coach Could Be A Chatbot

What AI Is The Best? Chatbot Arena Relies On Millions Of Human Votes

Banks Are Promoting ‘Female’ Chatbots To Help Customers, Raising Concerns Of Stereotyping