Table of Contents
Prompt AI is a self-service chatbot designed to solve common tech problems for employees, freeing up time for the IT team and getting employees back to work more quickly. At its inception, I was the sole designer on the project and, as a team, we managed to go from an idea to a live product in less than 18 months. Unfortunately, the Prompt AI product has since been canceled due to a shift in business focus. Here's the full story...
LogMeIn has historically been a support software company.
Their most successful products, LMI Central, LMI Pro, and Rescue allow end users to connect with technicians for various types of troubleshooting and support. Given our history creating products for this industry, we already knew that most organizations break their support teams into a few levels, or tiers, that look something like this:
The obvious pattern here is: as the complexity of the end user's problem increases, the skill level of the support technician must also increase.
A less obvious takeaway is that the number of people escalating through each level of support decreases linearly. That is, the first level of support receives all requests but the next 2 levels are only available if the previous level was unable to solve the problem. And since all requests start with the first level of support techs they're often overworked and understaffed, resulting in longer wait times for employees and lower job satisfaction among the level 1 support team.
Knowing this, we came up with our initial hypothesis:
An automated self-service tool that acts as the first line of support would empower employees to solve their own issues more efficiently and free up time for level 1 support technicians to work on more complex and rewarding initiatives.
The original team on Prompt AI consisted of 5 individuals: a Project Manager, a Marketer, a Designer (me), a Frontend Engineer, and a Backend Engineer.
While the small team may seem insignificant, it created an incredible dynamic, especially in the early phases of the project when we had more questions than solutions. Instead of playing our traditional roles, each team member participated in every part of the process and, because of their involvement early on, the team developed a shared empathy for users and a deep understanding of their problems.
Our first round of user research came in three forms: external sources, surveys, and interviews. Luckily, since LogMeIn builds and sells support software, we had a large and cooperative user base to tap for information. Of course, we started by collecting secondary research from external sources about the current state of tech support to build on our baseline understanding of the industry.
Next, given the information we found, we identified areas we were unsure of and wanted to learn more about. Given our two target users (support technicians and employees), the topics included things like:
Once we had a master list of questions and topics, we built a number of surveys that would target our user personas.
For support technicians, we reached out through channels like email and in-product messaging to collect responses, reassuring customers that the information provided would help us build tools to help their businesses.
For employees, most of our responses came from third-party survey tools that we tailored to target the correct audience, as well as internal LogMeIn employees.
The final step in this research phase was to have individual deep-dive conversations with some of the respondents about their particular experiences. These were generally set up as 60-90 minute interviews where we would have the interviewee walk us through a typical day in their role. We took the time to note any tools they used, any workflows they had, and any interesting stories they told. After each interview, we had a brief team recap meeting to share our thoughts, identify any open questions we'd like to ask in our follow up email, and to adjust for the next interview.
We spent roughly 2 months in a strictly research phase to better understand our potential users.
When our initial round of surveys and interviews were complete, we began the process of synthesizing the results.
The support technician surveys and interviews gave us a lot of quantitative data around their tools, companies, teams, and major challenges. We found that roughly 56% of the technicians we surveyed worked on a team with 3-10 people and supported a company of 50-1000 employees. Since we knew our product would require regular updates by someone on the support team, we decided these small to medium sized companies would be our initial target market.
For these companies, we found that about 45% of their incoming tickets were common and repetitive tasks (like password resets, printer issues, VPN disconnections, etc). We also learned that about a third of these companies had no self-service option available and no intention of building one on their own within the next 18 months.
Finally, we used their responses regarding their tools – specifically their knowledge management, ticketing, and chat systems – to decide on the first set of integrations we would build.
The data we gathered from employees was a little more scattered, but still gave us a wealth of qualitative information about their challenges when dealing with technical problems.
We heard a lot about the frustration of waiting for a helpdesk employee to reach out after submitting a ticket and how that frustration was amplified if the problem was easily solved. We also found that many employees had ongoing IT issues that they never reported because they considered them a 'minor inconvenience' and didn't want to bother the support team. Finally, we confirmed our secondary research finding that most employees preferred to solve their own issues, if they believed it was within their ability and could find the right information (usually through a coworker or by searching online).
Obviously these are only a few of the high level patterns we uncovered, but this initial round of user research gave us enough to start concepting solutions.
Given the amount of information we had collected from potential users, it's not surprising that we all had a lot of ideas for solutions. We figured that the best way to discuss, align, and quickly test our ideas would be to run a design sprint based on Google's 5 day framework. The goal of the sprint was to prototype and test some of the employee experience, in order to validate whether our ideas would successfully solve their problems. While the employee experience is only half of the product we were planning to build, we reasoned that, unless employees would use it, support teams wouldn't adopt it.
So the team set aside a week, found a room to call home, and got to work.
A few photos of the design sprint process. Lots of sticky notes.
By the end of the sprint week, we created and tested 2 prototypes with employees from nearby companies. We learned about the employee's reservations with using chatbots as well as their resilience to being given incorrect information. We also learned how important it was to always give the employee a way to escalate their problem to a human, if they were frustrated with the chatbot. Finally, we validated a few potential features that we thought could be differentiators from our competitors.
Miscellaneous screens from the 2 prototypes we built.
With a successful design sprint under our belts, we decided to build a minimum viable product for our first customer: LogMeIn.
Since we already had buy-in from out internal support team, we knew we could focus on creating a great experience for the employees first. LogMeIn uses Slack for our chat tool so we quickly created an app and deployed it to our private channels for testing. The knowledge base of solutions was built through a combination of docs the IT team shared, as well as pulling resolution information from our ticketing tool.
Within 2 weeks we had a functioning chatbot that could solve the 30 most common problems that came to the helpdesk team via tickets, and would create a ticket on the employee's behalf if it failed. We rolled it out to one of our smaller offices (roughly 100 employees) to see how they would use it and what kinds of questions they would ask.
The simple employee request flow with a feedback mechanism and ticket submission.
With our chatbot live in one office, we felt pretty confident. From our research we found that, across the industry, the top 20 requests account for about 40% of the tickets IT receives. Given these numbers, we assumed that our Minimum Viable Product (MVP) would be massively successful with our alpha users and we'd be rolling out to the rest of the company in a matter of weeks.
However, as the first and second week data came in, we found that almost no one was using our bot. And even when the support team sent our reminder emails about the new service and usage spiked, it was only solving about 10% of the problems.
So we concluded that there were 2 problems: low adoption and misunderstanding the bot's capabilities.
We decided to tackle spread awareness and help explain the bot's capabilities with a 3 pronged approach. First, we had the IT team modify their ticket feedback email for our alpha users to include a snippet about the chatbot, what it can help with, and a link to its Slack channel. Next we drafted prewritten emails for the IT team that they could send out weekly, which informed the users of the new features, new solutions, and development progress. And finally, we hung posters around the alpha office with example questions to keep our chatbot at the top of mind.
As each experiment was implemented, we saw a gradual increase users and interactions per user.
Posters we created to boost adoption and explain the bot's capabilities.
While our developers began building the foundation for what would become the technician's interface, my marketing teammate and I set out to build a smoke test. The idea was simple enough: we'd create a landing page for our product as if it were already built and live, direct traffic to the site, and see if people signed up. Our goals were to see if the value proposition resonated with visitors and start building an email list of potential beta customers.
Given the data we had from user research, we defined the key features we'd promote and I designed a simple landing page. At the time, we hadn't figured out the name or branding for the product so we used our working codename (LevelZero) and I invented a simple brand. We bought the domain levelzero.ai and I spent a day building the site in webflow.
Within a week we had the site launched and began driving traffic through Google AdWords. We also worked with other internal marketing teams to send in-product messages to current customers that fit our target market.
While the site has since been taken down, I kept a copy and have hosted the files myself for demo purposes. You can see the homepage (with the full animations and interactions) here.
Once a user submitted the email sign-up form, they were directed to this page which contains an embedded survey asking about their company's tools. This survey allowed us to contact the appropriate leads for our beta, given the integrations we supported at the time.
It took us about a month to collect 200 leads through this experiment.
The final, and largest, missing piece of our product puzzle was the support technician's interface. Since we'd become familiar with the tasks and workflows necessary for maintaining the knowledge base for our alpha users, we had a rough idea of what we needed to include in the interface. The Project Manager and I discussed the timeline for screens and features, and defined a rough information architecture for the app:
We knew immediately that the editor was going to be one of the most important parts of the interface because it was necessary for maintaining the knowledge base; so we started there. We also knew, from our interviews with support techs, that they faced a few specific problems when it came to maintaining knowledge for employees:
So we decided we would layer a few features into the editor that would solve those problems. We came up with the idea of an "Author Helper" sidebar that would provide contextual clues from employee interactions, as well as general writing help. The first iteration of the editor screen looked like this:
The Author Helper sidebar was designed to solve common issues support writers face.
The other area of the interface we focused on in this initial design phase was something we called the "Guide" page. This was a page (or pages) that would show the support technician how the overall system was doing and help them decide what to work on next. Again, this was one of the most common challenges we heard about in our interviews, so we knew this was a critical feature.
However, even though we knew the problems we were addressing, we weren't sure what information or features would be most valuable to the user. As you can see from the following concepts, the ideas were pretty varied.
While we understood the user's problem, there wasn't an obvious solution.
After a few design iterations and discussions about the Guide screen, we realized that none of us knew definitively what the most helpful information and features were. We didn't want to guess and risk building the wrong thing, so we went back to our users to see what they preferred.
While testing the Guide screen mockups with support techs, we ran into an unexpected problem: every time we showed them a new feature or bit of information, they gave us reasons to include it. But, if we asked them about the information they needed, without showing an interface, we got wildly different answers.
Since we were collecting contradictory information and not making progress on designing the interface, we decided to try something new: co-design.
The general idea behind co-design is that you work alongside potential users to design their ideal interface. It's a fairly new concept and it's certainly not right for every situation, but it can be quite effective if you're getting conflicting feedback from users or if you're unsure about what you don't know.
Quick side note: Some of my colleagues and I spoke about co-design at last year's UXPA conference and put together this quick start guide if you want to try it.
With the help of our UX research team, we put together 5 sessions and invited participants to come to the office for an afternoon of informal designing. For each session, a researcher and I sat with the participant and helped visualize the elements they wanted to see on the Guide screen. Here's what they came up with:
Guide screen interfaces designed by our target users during co-design sessions.
Of course, since each participant came from a unique background with particular experiences, each interface was different. But after all the sessions we got together to distill the common elements and that became the feature list (and some of the roadmap) for our interface.
Based on what we learned from this co-design activity, our next iteration of the Guide screen looked like this:
With the revised Guide screen design, we were all confident we were building the right thing.
Part of the reason we designed the editor and guide screens first was because those were the 2 screens necessary to give our internal support team full control of the alpha users' experience. With those 2 screens, they could update and maintain their chatbot's knowledge base without our help.
So with our MVP screens in development, I took some time to round out the interface and design the rest of the screens from our information architecture diagram, as well as workflows we had defined. Here are a few examples of secondary interface screens.
Various screens and templates for the first iteration of Prompt AI.
It's worth mentioning that, since we didn't have a brand or design system defined yet, the interface was left intentionally plain. The UI design at this phase could be considered something between a wireframe and a low fidelity prototype. Our plan was to build as much functionality as we could while the branding was still undecided and then to layer on a full design system according to Brad Frost's atomic design principles. However, once the design deliverables were well ahead of the development needs, we decided we should give the interface a well-deserved facelift.
Lucikly, given the size of LogMeIn and the range of talent in our design teams, I was able to tap an incredible designer named György Kiss for help with this task. Together we did an audit of our existing screens, identified all of the components and templates, and created a unified design system for Prompt AI.
Some of the individual design system elements we created for the visual facelift.
Here's a side-by-side comparison of how the screens evolved:
Before and after mockups of the support tech interface.
Once our dev teams had a functional MVP, we connected our LogMeIn alpha chatbot and gave our support team access. Moving forward they became an invaluable source of feedback an insight as they continued to build and maintain their growing knowledge base and roll the tool out to the whole company.
While design and development continued, an internal creative team worked with a local agency to create a brand for the product that fit with our existing support solutions line of business. An internal marketing and ecommerce team created a new website, based on our smoke test results, and set up a trial flow and drip campaign. To save time, we decided to use an external tool, AppCues, to build a simple onboarding tutorial that showcased the core features and demonstrated the value proposition.
Finally, after a few weeks of working in tandem, all of the pieces came together and we were able to launch our new product, Prompt AI, into a public beta.
In our interviews with support technicians and support managers we constantly heard there was a strong aversion to change, especially in established IT support systems. Knowing this, we decided it would be easier to integrate with the most popular ticketing, chat, and identity platforms than to build our own. Our backend engineer and I worked together to find the common required fields and create a layout that would accommodate each type of configuration. We also brought in a technical writer to aide in writing help tips and detailed integration setup guides for each tool.
An example of the integrations screen and Slack configuration options.
One feature we discussed when scoping out the support user's interface was a conversation editor. In our early prototypes, we showed interactions with the chatbot in a more fluid and natural conversation than we were able to accomplish with our simple text editor. While we thought this would improve the employee's experience, we knew from our internal support team's feedback that this feature could allow light troubleshooting and filtering before the bot decided on the best solution for the employee's issue.
I began designing this feature like any other, by looking at similar existing products for inspiration and detailing out the technical constraints. Next I created wireframes and prototypes to review with our beta users and engineering teams. Once we reached a consensus on the general implementation, I mocked up the full interface and experience and our developers got to work building it.
In a few short weeks we released this conversation editor to our beta users:
Our conversation editor included a highlight mode to help see how users reach each endpoint.
Some feature ideas, like the skill builder, came from data we gathered about employee interactions.
While looking at the content of employee questions that received negative feedback, we noticed a pattern of employees asking about general information. Most of these questions asked for information about:
These questions were causing a negative experience for our employees because the chatbot couldn't provide an answer to a seemingly simple question. After some discussion, we decided this was a worthwhile problem to solve.
The goal with this feature was to build a robust answer generating system that would allow the support tech to author one solution and have that solution answer many employee questions. And, of course, we wanted to keep the setup and configuration workflows as simple as possible. (Piece of cake, right?)
My first challenge was identifying the potential sources that would provide the key information for each type of question. Luckily we already had access to employee information through our Active Directory integration, ticketing system, and chat tool profiles. I knew that most of the utilities could be pulled from open APIs and, upon talking to the support team about how they answer questions regarding tool owners and admin, found out there was a master spreadsheet they referred to specifically for that info.
The types of data we used to power the skill builder, and their sources.
Next I looked for any place where data pulled from those sources would be used within Prompt AI. I realized that these bits of key data would populate knowledge base solutions in the editor and conversation editor, and could be represented similarly to key value pairs, or tokens, used in most marketing email software like MailChimp, API tools like Zapier, and WYSIWYG Alexa Skill builders. I used this pattern as the basis for how users would insert external data into their solutions.
Finally, I looked for ways we could parse the information from employee questions so our chatbot could respond properly. This was critical because we knew that the employee must always be referring to a collection of data (ie: an object with defined attributes), but we couldn't be sure what piece of data they're requesting. For example, if an employee asked for the phone number of a co-worker, the name of the co-worker would determine the object and the keyword "phone number" would refer to the corresponding attribute.
For incoming data, we knew our sources could be external (APIs) or internal (spreadsheets created within the product). The interfaces to create and configure these data sources looked like this:
Configuration screens for internal and external data sources.
Once a support technician set up the data sources, the editor interface was updated to provide access to "tokens" that would replace text in the chatbot's response. The modified editor interface now included a token dropdown (or specific markdown structure) to insert available token values into the solution. We also included the ability to preview the solution with examples from the data sources, so the author could see if their grammar made sense.
Skills gave content authors access to dynamic data, in the form of tokens, that could be used in any type of solution.
Overall this was one of the most complex features we designed for Prompt AI, but it solved critical problems for both of our target users and really highlighted how advanced a chatbot could become if given access to the right data.
Over the next few months we continued to ideate, research, design, test, and build. As our process became increasingly efficient, we were able to regularly launch updates, enhancements, and entirely new features for our user base.
Of course, not all features are as complex or noteworthy as the skill builder but, collectively, they added value to Prompt AI and each one addressed specific user problems. As the product grew, I aimed to keep simplicity and consistency across both the interface and our users' mental models, to maintain usability. Here are a few examples of how the interface continue to grow and evolve as we built:
The chat tool preview allows users to visualize their message in different chat tools.
Modals are used for simple tasks, like adding labels, or to focus the user's attention at potentially destructive moments.
Most products have some kind of high-level settings. Prompt AI was no different.
Error pages are used as an opportunity to solicit feedback and let users report bugs.
IT professionals, or anyone working in front of a screen all day, tend to prefer a dark mode to reduce eye strain.
Empty states are a great place to encourage specific behaviors, like setting up integrations or creating content.
Unfortunately, at the end of 2018, the decision was made to discontinue work on Prompt AI. While the product we created and the work we did was viewed largely as a success, leadership decided to consolidate many of the features and functionality into LogMeIn's other support tools as opposed to selling it as another standalone product with a limited use case.
Even though Prompt AI is no longer in development, the experiences of working on a smaller team and solving novel user problems were immensely valuable to the individuals involved and the company as a whole. We can continue build upon these learnings and further our processes and techniques, so we are always making better products and creating extraordinary experiences for our users.
If you managed to read this entire case study, I congratulate you on your dedication and apologize for my long-winded explanations. Thanks for taking the time and, as always, feel free to reach out if you have any questions or comments.
Quick disclaimer: This is a professional project done while employed at LogMeIn. As such, some information is confidential and has been purposely omitted.
