Advanced Search Integration in PayPal App: A Detailed Case Study
Context
The purpose of this initiative is to integrate a Universal Search feature into the PayPal mobile app, aiming to enhance the ease and speed for users to locate what they're looking for. This effort supports PayPal's broader mission of offering convenient tools and services that empower users to better manage their financial lives.
In this Case Study, I’ll be evaluating the PayPal ecosystem and user base to determine common pain points PayPal users have. From there, I’ll define an MVP solution that addresses those core pain points. Finally, I’ll set success metrics and a rollout plan to test our MVP solution with a subset of the PayPal user base.
Hypothesis:
If PayPal adds Universal Search to their mobile app, users will complete core tasks faster.
Market Research
What Can We Learn from Competitors?
- Venmo: Users can search for people, businesses, and charities from the home screen.When users tap on the search bar, the user is navigated to a full Search screen, with Name, @username as the placeholder.
- Instagram: Instagram follows a similar pattern. Users can search anything and Instagram will return appropriate results. Instagram prompts users with fun, unique search terms to spark inspiration for what’s possible with their search. Once a user actually searches, they land on the Top tab of the search results page. They can then switch to the Accounts, Audio, Tags, or Places tabs. We can assume these are in order of what Instagram believes is most valuable to drive users to.
- A few common themes across all of these apps
- The Search bar is typically on the Home tab/first tab and is at the top of the screen. It grounds the user and sets the stage that they can find whatever they need from it.
- The Search bar placeholder text copy is usually high level: “Search for anything” or “Search” in order to not box the user into thinking they can only search for one thing.
- The empty states may include prompts “Try searching for” or “Search ‘woodworking’” to spark ideas in the user’s brain about what they may be able to search
- Most Search experiences support many different categories of results.
The Audience
Who Uses PayPal? At a high level, there are two main users of PayPal: individuals and merchants. This initiative is focused specifically on adding Universal Search to the PayPal mobile app, not the PayPal Business app. This means we’ll focus on our individuals user base.
individual PayPal Users:
- Age: 24% of PayPal’s US users are 58 years or older (~48M users).
- Transaction Frequency: The average user transacts 45 times per year.
- Location: Majority of users are in the United States.
- Securely pay online with purchase protection
- Pay internationally without converting currency
- Pay for subscriptions
- Transfer money between friends/family
- Link multiple payment methods
- Defer payments with PayPal Credit
- Get/link Loyalty cards
- Buy, sell, and securely hold cryptocurrencies with PayPal
- Save with high yield savings
- Get cash back on deals
- Donate to a charity
- Create a fundraiser
- Get a debit card
- Send invoices/estimates
- Cash pickup / reload phone
- Sending and requesting money
- Managing transactions with individuals and merchants
- Are over the age of 58
- Live in the United States
- Send money, request money, and manage their transaction history
Use Cases of PayPal:
Without explicit usage data around the use cases listed above, we can make an assumption based on UI prominence that the two primary Individual user use cases are:
To design the most impactful Universal Search feature, we’ll solve core pain points for a target user segment based on these primary use cases. To summarize, we’ll be focusing on users who:
Our Universal Search solution will ultimately provide value for all PayPal users (in the US and globally, and under the age of 58), and cover various use cases, but solving core pain points for this user segment is table stakes.
Why? A user within this older demographic (which makes up about a quarter of our US user base) may not have as easy of a time adopting new search functionality (Source) – so if we can build a solution that works for them, expanding to younger demographics will be easier. Additionally, because we’ve observed increasing growth amongst users over 50, we need to ensure our solution works well for these users. (Source) Finally, these core use cases are done by almost all PayPal users so if we can solve pain points surrounding them, we will be able to provide value to a large percentage of our user base.
User Insights: Identifying Pain Points
Research uncovered several pain points for PayPal users, particularly for those over 58:
- Finding Contacts: Struggling to find contacts, to request money from or send money
to
- The person may not be a PayPal user yet
- The person may only be searchable in PayPal’s database by contact info the user doesn’t have
- The person’s name may be different on PayPal compared to how the user knows them
- Finding Transaction: Hard time sifting through a lot of transactions to find the one they want
- There may be older transactions that don’t appear on the Home tab or too far down on the Wallet -> Activity tab
- Users may want to find transactions by different criteria (date vs name vs amount)
- Finding Specific Transactions in History: It takes a long time to see all past history with a specific individual or merchant
- Seeing history with a specific merchant can take 4-5 clicks to get to
- Users may not know exactly how to get to the merchant
- Typing Confusion: Not knowing what they can type in to find a person (email, phone, name, etc.)
- Users may spell emails wrong
- Users may enter phone numbers differently than how PayPal is expecting them
- The person may have a different name than how the user is searching (ex: Sam vs Samuel)
- Navigation Confusion: Not knowing where to go on their PayPal app to find what they need
- User may not know they need to navigate to a specific tab to access certain information/features
- Users may go to one tab thinking they’ll find what they need just to later discover it was in another area of the app
User Journey
How does a PayPal user find a transaction with someone? In general, the information architecture of the PayPal app is confusing. Because PayPal is trying to grow various parts of their business (Crypto, Savings, Merchant Deals) core user flows are buried.
Natalie, 65
Scenario: Natalie sends money to her daughter Amy every couple of weeks through PayPal. Natalie's husband asks her how much she send a couple of months ago.
Natalie's Expectations:
- I can easily find old transactions
- I can easily see my history with Amy
- This should only take a couple of seconds
Key Takeaways
From this research, we can conclude the following:
- PayPal is growing in popularity amongst an older demographic as a way to send and receive money.
- It’s common for native apps to have a universal search bar on the home screen to enable users to quickly get to what they need.
- Right now, core use cases like finding a contact or transaction are very difficult and time consuming to complete .
The Problem
PayPal’s current app lacks an easy and fast way for users to find what they need. Core actions like finding contacts or transactions take too long to complete.
The Goal
Decrease the time it takes to complete core actions on the PayPal app by empowering users to search through all content and features available to them from one search box.
We will be focusing on solving these core pain points through our Universal Search feature. By helping users find contacts to request/send money, find and manage transactions, and more quickly navigate through the app, we’re helping execute on PayPal’s overall mission to make it more convenient to transact and control their financial lives.
Feature Prioritization & MVP Definition
What should be included in the MVP?
Creating a Universal Search that encompasses all content across the entire PayPal app will be a very big undertaking. In order to ensure it is worth investing resources here, we want to start by testing an MVP of Universal Search. If successful, we will continue to build out a powerful Universal Search that helps users find contacts, cryptocurrencies, and everything in between.
The feature set for our Universal Search MVP will be limited to an empty state and a results state because:
- These states are the fewest required to provide users with context/value and allow us to begin learning if Universal Search is really solving pain points for the user.
- Additionally, querying multiple data sets across PayPal’s ecosystem at the same time, while returning results quickly, will be a big technical challenge/undertaking. In an effort to keep scope limited, features like “Auto expand core options” mentioned in Future Iterations have been deprioritized from the MVP.
User Stories
- As a PayPal user, I can search for things from my Home screen
- We’ll add a Universal Search bar on the Home screen. Tapping on it will open Search autocomplete view with keyboard.
- As a PayPal user on the Search screen, I will have quick access to my top items
- Users will see Individual Contacts they frequently engages with
- Users will see a prompt to request permission to access local phone contacts
- Users will see Merchants they frequently engage with
- Users will see recent transactions
- As a PayPal user, I will be able to search for Contacts, Transactions, and Merchants from one place
- Autocomplete search for Individual Contacts by: Name (first name, last name, or both), Email, Phone Number (normalizing different ways a user could type in number), Username
- Autocomplete search for Merchants you’ve transacted with by Name
- Autocomplete search for Transactions by Name
- As a PayPal user, I will see the most relevant results when I search
- Dynamic results to visually reflect result types (contacts vs transactions vs merchants)
Final Solutions
Universal Search on Home and Search Results update
- "Search PayPal"used instead of "Name, email, etc" in an effort to not box the user into thinking they can only search for contacts. Instead, encourage them to search the entirety of PayPal.
- Your Contacts", "Merchant", and "Transactions" to provide users with context into what they can search
- Results are a mix of contacts, merchants, and transactions based on the Search term. Each result "type" has a consistent design element to help the user quickly identify what each result is. For example, the person icon next to contact result helps the user identify that that result is a Contact.
- As the user types, results dynamically adjust to best fit the search term. Results that are no longer relevant disappear from the Results list.
Risks & Tradeoffs
What risks would PayPal open themselves up to if they added Universal Search to the app?
- Including the Search bar on the Home screen content.
- By taking up prominent real estate on the Home Screen, we may accidentally introduce friction to complete certain actions by pushing the rest of the Home screen content/features down.
- Getting permission to access local phone contacts
- Note: this may have been previously solved in other areas of the app, but we will still want to be cognizant of these risks for our Universal Search solution as well.
- User privacy concern about syncing user’s contact list & associated data, resulting in users not giving permission.
- Technical challenges associated with combining local phone contact information with PayPal’s contacts database, all while returning results fast.
- We need to make sure duplicate results aren’t returned which could confuse users. (Seeing two exact contacts would confuse the user around which to tap).
- MVP not including entire scope of searchable items
- Our MVP will only allow searching Contacts, Merchants, and Transactions. By not including cryptocurrencies for example, we may be accidentally training users that this Search bar will always be limited to these result types.
- A way to guard against this in the future will be updating the empty state with different result types to educate the user.
- Time & resources it’ll take to build, launch, and maintain Universal Search
- Adding Universal Search to the UX adds another layer of complexity to maintaining the PayPal app. We have to ensure routing continues to function properly from Search & elsewhere.
- Additionally, to be a true Universal Search, we should eventually support all types of PayPal content and features which will create additional work in the future as new PayPal functionality is launched.
- Committing to Universal Search means fewer resources can be dedicated to other functionality.
Measuring Success
North Star Metric
Avg Time From App Open → Core Use Case Completion
If we are solving our user’s main pain points, we should see a decrease in the amount of time it takes for users to complete core actions. Core Use Case Completion is an umbrella metric measuring users who do one of sending money, requesting money, or viewing a past transaction.
Secondary
These secondary metrics will help us determine if there are different areas of the Universal Search feature we may be able to improve/optimize.
- App open → core use case completion rate within a session
- Does saving time to reach a contact/transaction/etc with Universal Search get more users to complete core use cases?
- Home view → search start rate
- How many users are even starting searches?
- Is the Search bar appealing and clear enough to prompt users to search?
- Search start → abandon rate
- How many users begin searching for something and then abandon?
- Search start → result click rate
- How many users begin searching and select a result?
- Does this differ by result type? (Ex: finding transactions takes more time than contacts)
- Average time between search start → result click
- How long does it take to select a result? (Is the UI simple enough for users to quickly select what they want to select)
- Does this differ by result type?
- Average clicked search result index value
- What is the average position users are clicking on results? If it’s low (1-2) then our autocomplete is working very well. If it’s a couple results down (3-6) our autocomplete is performing okay. If it’s higher, we probably need to optimize our algorithm to return better results.
- Number of WAUs using Search
- Are more users using Search over time?
- Breakdown of result type clicks
- Are users just searching for one type of content (contacts for example) or are they also searching for transactions and merchants?
- Search start → no results rate
- How often are we failing to find any results for the user?
- Are there any commonalities for search terms that are causing no results to appear?
- App open → contact permission given rate
- Are most users giving us permission to access their contacts? If not, can we message it in a better way to increase this metric?
Counter Metrics
Since our Universal Search bar will be on the Home tab and take up prime real estate, we want to ensure we are not unintentionally hurting other core metrics. To monitor this, we will track:
- Decreased new user account completion rate
- Are fewer new users completing their account set up because the Universal Search bar is pushing down critical onboarding steps?
- Decreased product expansion rates
- Are fewer users signing up for PayPal savings, buying Crypto, or signing up for merchant deals?
- Decreased average GMV transacted
- Are we unintentionally decreasing the amount of money processed through PayPal by introducing the Search bar and taking up Home screen real estate?
Launch & Go-To-Market Strategy
A/B Test Universal Search
To ensure we are improving the PayPal app’s user experience by introducing Universal Search, we will be testing this feature. We will A/B test our MVP with a small group of new and existing users within our target user segment.
- Versions:
- Control: Current PayPal app without Universal Search (Start at 90% of audience)
- Variant: PayPal app with Universal Search (Start at 10% of audience)
- Audience: Users who…
- Live in the United States
- Are over the age of 58
- Average time between app open → core use case completion
We will be including new users in this experiment to ensure adding a Universal Search bar doesn’t negatively impact core account setup / top of funnel metrics. We will be starting the A/B test variant at 10% of our target audience so that we can iterate without affecting the majority of users in this audience.
If the results of the A/B test are positive (primary measure of success achieved and no negative side effects) we will expand our test cohort to all users in the United States regardless of age. If our metrics in the expanded cohort are also positive, we will roll out the MVP to all United States users.
From there, we will begin improving the Universal Search experience by executing additional features on the roadmap and eventually expand Universal Search to be supported internationally.
Note: If our A/B test results are negative, we will investigate potential causes, iterate, and re-run the experiment.
Future Iterations
What could Universal Search do in the future?
- Auto-expand core options within search results instead of navigating user to new screen
- For example, if a user searches an individual contact and taps on a result, we can prompt them with “Send”, “Request”, or a “More” option. This will reduce friction to complete a core option and establish Universal Search as a powerful tool.
- No results state to collect feedback about what the user was trying to search for. (can be used to prioritize order of following features)
- Improved result matching (If you search John Doe but the contacts name is James Doe, the contact would still show up based on last name, despite not being an exact string match)
- Search for Invoices
- Search for Merchants deals by Name
- Search for Cryptocurrencies
- Search for Charities
- Search Transactions in additional ways (ex: date, amount)
- Filter results by category (ex: Invoices, Crypto, etc.), date, etc.
- Additional routes & search terms will be evaluated based on data we can gather from early iterations. We will specifically look at users who begin searching, fail and abandon.
Final Thoughts
Summary
To recap, I’d recommend A/B testing an MVP of Universal Search with the goal of decreasing the amount of time it takes to complete core actions (send money, request money, view/manage transactions).
This MVP aims to solve core pain points that users in our target user segment face.
If the test returns positive results, I’d recommend PayPal expands the functionality of Universal Search and eventually rolls out the feature globally.
Thank you for checking out this case study!