Table of Content
How do I find a UX agency that suits me? How do I find out if a UX agency is good? How do I compare different UX agencies as objectively as possible? What are criteria for a UX agency screening?
We will answer these questions in this article. After reading, you will have 10 criteria that you can use directly for agency comparison. At the end of the article, we also provide you with our downloadable comparison table so you can start your agency screening right away. If you’re in a hurry, you can jump straight to the spreadsheet here – it contains 10 criteria in addition to the 10 in this article.
Before we start, though, a quick disclaimer: As a UX agency, we are clearly biased. Nevertheless, in order to be as neutral as possible, we have derived the criteria based on real inquiries and customer conversations and have sorted them by frequency. We hope this will help you find the best companion for your UX needs. If that’s us, we’ll be happy, of course.
Enough of the introduction, let’s get started with the first point:
Criteria 1: User-centeredness
To get the benefits of user-centric work, you need to choose a vendor that actually works that way – and doesn’t just claim to:
UX is en vogue. The realization that successful products need an outstanding user experience is gaining more and more acceptance. Some products even become successful only through their outstanding user experience. With this knowledge, the awareness and demand for good User Interfaces and Human Centered Designs is increasing. What is the consequence? Providers of e.g. software or design offer UX “on top”. Can’t be that hard.
However, UX design is much more than “making things pretty” at the end of a development. It is the development itself that changes. It’s the constant focus on the user and their needs.
So you should pay attention to the focus when choosing a provider. Is UX design offered in addition to the actual service or is the user at the center of the User Interface Design? Are users actually consulted on a regular basis or are users, if at all, only thought of? If a vendor only wants to show that they have done a good job with a UX test at the end of a development, or even does without it due to years of experience, then this is a clear indication of the non-user-centric approach.
If you want to work in a user-centric way, then you should look for a UX vendor that does so.
Identifying characteristics:
- Offers real user research (e.g., UX testing),
- wants to actively involve users early in the joint collaboration and not “develop / design first”,
- intensively asks who the users are and what their tasks and characteristics are – in order to subsequently build the offer on this,
- has its own UX labs incl. interviewers and staff for recruiting participants.
Criteria 2: Fit.
Find an agency that fits your industry and product – this will save you a lengthy joint training period that you would otherwise have to pay for:
Do you have a product aimed at consumers (B2C) or businesses (B2B) – or both? Depending on your answer, you should choose a specialized UX agency.
Agencies that specialize in B2C products will intuitively have a different focus when it comes to user-centricity than agencies that specialize in B2B. For example, while in B2C the buyers are often also the users, this is not the case for many B2B products. Here, consistent user-centricity often involves a wider range of stakeholders for whom the finished product must be compelling. In practice, this results in different optimization goals – for example, easy adaptability to existing corporate processes on the B2B side vs. an outstanding look with classy animations that turn the product into a status symbol on the B2C side.
B2B objectives often require the unification of multiple opinions in one product due to the different stakeholders. Additionally, users of B2B products are often more difficult to “access” – whether due to works councils, required secrecy, or much smaller user groups. In general, user-centricity is harder to achieve here than with B2C products.
Rule of thumb: B2B product -> B2B agency; B2C product -> B2C specialization, if possible, but not as critical.
This also applies to the industry your product is targeting. The more references an agency has in your field, the less familiarization and basic understanding needs to be built.
Rule of thumb: If the references match your product and industry, then the agency is probably a better fit than one with references from other areas.
Criteria 3: A coordinated way of working.
For a smooth project flow:
Does the agency’s way of working match the way you want to work on the project?
User-centered work always means an iterative approach, alternating user research (understanding users, testing designs, building knowledge about usage) and design (applying knowledge to design an outstanding user interface).
However, the specific design varies widely in practice. Some agencies want to work out the blueprint of a user interface and a style guide “in peace”. Only after the work is done should the implementation begin in the working method of these agencies. Other agencies work with their clients in an agile manner and are used to being one to three sprints “ahead” of the implementation.
Both approaches are not good or bad per se – but assume a different time budget on your side. In our experience, a non-agile agency finds agile difficult to work with. An agency that is used to working agilely – and therefore precisely on a schedule – easily copes with this requirement.
Rule of thumb:
You want to work agile → You need an agency working agile that can adapt to your sprint cycle.
You want to work non-agile → Both non-agile and agile working agencies are suitable for you.
Criteria 4: The agency is strong in User Research and User Interface Design.
Minimize handoffs and look for an agency that can do both:
As described earlier, user research and design phases alternate in user-centered work. To avoid handoffs or extra work here, look for an agency that can handle both User Testing and User Interface Design themselves. This ensures that the results of a user test reach the User Interface Design team quickly and pragmatically, and that the designs / prototypes return just as smoothly.
Designers also have the opportunity to be present during user tests at any time or to discuss a specific issue with the interviewer once again. In our practical experience, this leads to much better User Interfaces than in projects involving two agencies – one for User Research and one for User Interface Design.
In addition, the potential for conflict is reduced, since the user research part cannot be seen as an “evaluation” of the other agency’s work. We see it time and time again that the designing agency reacts offended in case of a bad UX test and tries to limit the UX tests in the future or to leave them altogether – instead of using the results to improve the user interface.
Rule of thumb: outsource User Research and User Interface Design, choose one agency for both tasks. If you want to outsource only one of the two tasks, ask in advance about the agencies’ experience with the desired constellation and listen for the “in-betweens” in the answers.
Criteria 5: User recruitment.
Make sure that the agency can recruit the right users:
User research with the wrong users is useless – or put another way: For an outstanding user interface, you need user research with exactly the right users or stakeholders. Do not underestimate this task! Often, much more time has to be invested here than in the actual execution of the UX test.
There are several approaches to finding the right test subjects:
1. You provide the users: this can make sense if you already have a test person pool or only you can get hold of the test persons (if you want to interview your clients, for example). In this case, however, the responsibility as to whether the users are “good” remains with you – you therefore play a decisive role in determining the quality of the user research. For example, if the user research agency only receives your absolutely satisfied clients, then far fewer problems will be uncovered than if there are also dissatisfied ones.
2. The agency provides the users: then the agency also almost completely determines the quality of the user research. An agency that does such projects regularly has its own database and employees who take care of recruiting participants. You can also recognize the professionalism of the agency by the fact that it already talks to you about the recruitment criteria for the offer – these significantly influence the recruitment costs. Before the actual recruitment takes place, these criteria are discussed again in detail and a so-called screener is created – i.e. a document that contains all interview questions for the participant recruitment, including the decisions based on them as to whether the person is suitable or not.
3. Another company provides the users: agencies that conduct user research and do not have their own database often work with specialized recruitment providers or refer them to you. This is perfectly fine as long as you keep the following points in mind:
a. If you are contracting the recruitment provider, then you need to ensure that the right users are recruited. Once again, you play a major role in determining the quality of the user research.
b. If the agency commissions the recruitment provider, then it also bears complete responsibility for the quality of the user research.
Conclusion: An agency that is to conduct user research on your behalf will ask you – no matter which approach you choose – about the users and their characteristics as well as the type of recruitment you want. If doubts arise on this important point, then you should doubt the entire agency for this project. If the budget is there, hand over the responsibility for recruitment to the agency – so it has complete control over the quality, but must deliver accordingly.
Criterion 6: a coordinated schedule.
So that everything is ready on time:
You probably have a schedule in mind for your project and know when the agency’s results should be there. So you need a reliable partner for your project who will do his part in the promised quality – but also in the agreed time.
You can recognize a well-organized and professional agency already by how fast the response to your contact is and how long the offer phase “drags”. If an agency takes forever to call you back or provide you with a quote, the project will be no different.
The agency should address the timing issue on its own and clarify the key points with you. After all, part of a quote is reviewing your own resources in relation to the desired schedule.
Can’t you find the time? Then a professional agency will also communicate this. They will not accept the assignment “at any price” and then try to squeeze more time out of the project. Instead, it will present possible solutions so that you can decide.
Criteria 7: The right quality.
Pay attention to quality management, different disciplines, awards & case studies:
If an agency promises you to do quality work, it should be able to show it. The evidence can be diverse:
- Case Studies: Descriptions of projects and their process as well as the results provide an overview of the working methods and quality. Agencies should – despite often existing secrecy – be able to show at least some project results.
- Testimonials: Does the agency have quotes or videos from clients? This usually shows quality, because for a client it means effort to sit in front of a camera and talk about the project result. You only do that if you are truly satisfied with the results and the collaboration.
- Awards: High quality awards in design indicate a very special quality. The work is evaluated by an independent jury and only a few are awarded prizes at the most famous ones, such as the Red Dot Design Award.
- Quality management: Does the agency have quality management and a continuous improvement process? In the best case, the agency is even certified according to a quality management standard (DIN EN ISO 9001 or ISO 13485) and can thus demonstrate that its own quality management also stands up to critical external scrutiny.
- User Research and User Interface Design are two different tasks. Both need different skill sets, which can very seldom be found in one and the same person. An agency that covers both areas very well employs people who are very good at interviewing or “people” (e.g. psychologists, interviewers) and others who are very good at designing user interfaces (e.g. designers, conceptual designers, engineers).
- Clear approach & processes: Does the agency rethink how to proceed on every project, or does it have proven processes and procedures that will get you there safely and quickly? It’s best that an agency doesn’t do your “type” of project for the first time – otherwise it will also make all the mistakes with you that happen on a first run.
Criteria 8: Technical communication.
Are the hardware and software universes used compatible?
Is your prototype only usable in a certain program that runs on Windows – but the agency works with Mac? Or do appointment invitations come in the wrong format, so you always have to transfer them manually?
These are just two examples of incompatible hardware and software worlds – which can of course be solved. However, they cost unnecessary time. To avoid such problems from the outset, you should either choose an agency that uses the same operating systems and programs as you do, or one that can flexibly adapt to your hardware and software world.
Criteria 9: The appropriate equipment.
Does the agency have the necessary equipment?
For user research to produce the results that will help you and your product move forward, you need the right equipment. Make sure that the agency has it.
User research ideally takes place where the user will later come into contact with the product. For example, if he uses the product on the bus, then this is the ideal place for user research.
Often, however, prototypes cannot – or do not want to – go “outside” into the real situation. Therefore, the environment is recreated as best as possible. This requires UX labs, including recording technology, as well as observation rooms. The agency you select should have the appropriate facilities – on the one hand, because these locations would otherwise have to be rented at a relatively high cost. On the other hand, because an agency without its own laboratories and equipment probably does not conduct user research as often as claimed.
Depending on the user research question, special measurement methods are necessary. For example, to record eye movements, the agency needs special equipment, which can cost 20,000€ or more, depending on the equipment. If an agency has this equipment, you can be pretty sure that they are serious about user research and have experience with these methods.
If it is somehow possible, let them show you the equipment as well as the labs. There may not be “much” to see here, but afterwards you are sure that the agency does not just claim to do user research often.
Criteria 10: Sympathy.
Do you like the agency or the people?
What is the atmosphere like in the first meetings? How likeable are the contact persons? If the proposal phase is already fun, that’s an indicator that the project will also be great.
Even if this doesn’t have a direct impact on the project results: In our experience, it is an extremely important point whether you look forward to the joint work or roll your eyes mentally before each call.
Recommendation: Book – at least to some extent – the agency that you expect to be pleasant to work with. After all, you have already ensured the quality with other criteria.
Conclusion
The 10 criteria mentioned in the article, which you can use to directly compare UX agencies, give you an overview of what makes a good UX agency. Of course, there are various others that are less important from our point of view, but far from unimportant. We have therefore worked out 10 more criteria for you and summarized them together with the ones mentioned here in an Excel spreadsheet. With this table you have the possibility to carry out your agency comparison directly. Of course, you can also add your own criteria or weight them against each other.
Download the table here.
We wish you every success in selecting your UX agency. Should we be in the process, we look forward to hearing from you.
