As the conversation about Scaling Design, Design Ops, Design Systems and Research Ops gets more and more sophisticated in Spain, we can see a clear trend in adopting models, roles and frameworks that help companies scale. And although these conversations are interesting, there is nothing more exciting than hearing the tales of the people who have real world experience in the trenches. I interviewed Saskia Liebenberg, Design Operations Program Manager at Anaplan, previous Research Manager at Deliveroo to get to know and share her insights about the challenges of doing research at scale.
Let’s start talking a little bit about you and your career.
Currently I am the Design Operations Program Manager at Anaplan, which as a job title is a bit of a mouthful, but it seems to be the industry standard. The way my job is structured and the way it was advertised was “UX Ops”, as I work with the designers, the researchers, and there’s a bit of project management as well. I look after the holistic UX team operational elements, like their team meetings, workflow, tools, budget, etc. And of course there is the actual Research Ops side, which is a lot more than what there is on the design side. There’s about 12 designers and 3 researchers and the initial focus is very much on the crossteam, on the UX level.
And are you working alone or are you a team of research managers?
It’s just me at the moment, which is also interesting to me and I think. I quite enjoy that when you start up a new function in a team there is a lot of strategic thinking, but also just nuts and bolts work, flipping between the two, being involved in what are we going to do and what is the long term strategy behind this. But then also being there because someone needs to put a spreadsheet together right now, or someone needs me to sort out this license ‘right now’. I like getting my hands dirty but also like being able to step back a bit and think about if something isn’t sustainable, how can we do it better so that we can grow, and not have everything break. That is the real deal of scale: not breaking it as you grow.
Before Anaplan you were a Research Manager at Deliveroo. Tell us a bit about that.
At Deliveroo I had two different jobs. For the first couple years my role was very much focused on project management, initially within the graphic design team which is like the in-house brand studio. There I was getting a workflow setup, building the studio, coming up with the strategy, hiring other project managers. Then the second part of my time at Deliveroo was focused on Research Operations as I worked with the Head of Research to implement a research operation for a team of 15 researchers.
At the time things had started to fall apart slightly and they weren’t seeing the kind of incremental impacts by hiring more researchers that they were expecting. Because the lack of process was starting to get in the way, and slowing people down. So people were reinventing the wheel: running around trying to find consent forms, testing kits, trying to find a lab and not knowing which lab was best, etc. So they kept making the same mistakes and learning the same lessons individually, rather than being able to leverage each other’s experience more effectively. And that is where that need came up, to have a Research Ops manager.
You mentioned then that at a certain point at Deliveroo things started to fall apart research-wise. Can you tell me more about how you guys solved that?
One of the great things about working at Deliveroo is that there is a lot of research buy in and a lot of understanding around all the work that goes into making research happen at an executive level. So I was able to actually build out an operational team to support the researchers by putting infrastructure in place so that they can work more quickly and efficiently and not have to reinvent the wheel.
A lot of it was quite basic, it really isn’t groundbreaking earth shattering super secret stuff. For example, we need templates, so we need somewhere where everyone can access them. We need a standardized way of documenting our work, so we came up with a project folder structure that people can copy and reuse. And also so if a researcher needs to pick up on someone else’s work, they would know where to find the relevant information. Or if I have a study that compliment someone else’s work I can quickly work out what they did how they did it so instead of starting from zero I can built on it. I I think of it almost like hygiene factors.
But where did you start? What was the actual first step?
Recruitment, I think. We hired a specialist to look after this quite early on. It was taking an incredible amount of time and costing a lot of money. And we needed someone to just own that streamline it and make it much, much more manageable.
On that I had a lot of help from Katie Towsey, who is now at Atlassian, she’s the person I used to consult when I had any questions, as we started kind at the same time and went through this parallel journey, so we would talk a lot about ‘oh we had this insight’ and she would say ‘yeah, we just hired someone to do exactly that’. She was always one step ahead, and it was quite reassuring to be able to check in with her.
Initially I thought this recruitment specialist would be our in-house recruiter, and we would just handle all of the recruitment for the researches ourselves. But quickly realised that that wasn’t going to work either, since this essentially just means building up an in-house recruitment agency and you will have to keep hiring people to do this job, and that is not scaling. So we decided to switch to a self-service model, where the recruitment specialists act as consultants, guiding and helping researchers to avoid the same pitfalls. But the researchers are actually able to do recruitment themselves.
And it was really interesting, the phases you go through. Courtney Kaplan, articulates it quite well: she says you go through the triage phase initially, when you implement an operational team at scale where you just need to understand what are the biggest crises and quickly solve as many problems as you can just show the value that you can ask. And a lot of that will actually require you being a bit more tactical, and a bit more reactive and actually getting your hands dirty so you can properly understand the problem, and just saying to people “don’t worry about it, I’ll take it off your hands and sort it out for you”.
But this is not sustainable, as you can’t keep doing things for people. You have to find out a way to make the whole system run more smoothly, so they can do things themselves in a more efficient way.
This is the main thing that I kept reiterating to my team: you can do that now, but you are not going to be able to keep doing this when we have 25 researchers, or 40. How can you take what you are doing now and make it truly operational like an infrastructural system that other people can use instead. That is so enlightening!
ButIs there a jack of all trades that every company can take as a first step towards a successful Research Ops practice? Or does it really depend on each company?where did you start? What was the actual first step?
Yes, the first step should always be understanding your context. So literally going around and speaking to everyone in the team, or as many people in the team as you can. Speaking to as many stakeholders, or representative stakeholders that engaged with the team, to understand what are the pain points right now. What is really not working right now and what is working okay and you don’t have to worry about right now. And then, based on that, identifying what is going to be the thing where we can make the biggest difference quickly. That is really important: you need to find the thing that is going to show people the value that you add when you start a new function, and when you are trying to scale.
Yes, it is exactly how we have done in our team too: you start by researching the researchers.
In my experience yes. And often is the case where there are very clear obvious things that are just not working for anyone, which makes it really easy to know where to get started. But this also means that is a difficult thing because no one has fixed it yet, even though we are all struggling with it.
After identifying and then acting on that first step, what other challenges did you face?
The other tricky part is to keep communicating on what you are working on and to set expectations, particularly with the people that you report to. You need to make it clear that they will not necessarily see the results overnight. And that the first thing you do probably won’t be the ultimate solution, it is all an iterative approach, because everything is so contextual.
So with context being so important, what advice you would give to people who, unlike you, are trying to do this in an environment where there is no management buy-in or the support is not so explicit?
To be honest, it is horrendous. My advice would be to not pick fights, always step away, find another way of doing it. It’s hard enough just doing the job, you don’t need that on top. You need to just find someone who understands and who has got your back and understands the ultimate value you will deliver and align yourself with them. If that is not your leadership, if that is someone in a smaller team, then do that and define your scope in that way, and just say well I am going to work with this particular team, they will be my focus in the next three months, and then show the value that way if you can
But the truth is that it is an incredibly frustrating, unfulfilling experience, when you are hitting your head against a brick wall, and just trying to deliver good work.
What would you say is the main job of a Research Ops manager or dDesign Ops manager?
Well, I think we are enablers. In many ways it is very similar to being in project management and maybe I’m saying that because that is my background. But, to me, the number one thing that I am here to do is to make things work better for the team, so that they can focus on their work. And it is also for that reason that there isn’t a fixed one size fits all solution because it depends on the team, what their needs are and what their problems are. But that is ultimately why I like doing this job, because if you are doing it right, people are so grateful and so happy to have you on board, because you are making things easier for them.
Do you mean that the best ReOps managers are actually project managers, not researchers?
Well, that is another hot topic in the community these days. I don’t think you necessarily want to get someone who has the relevant discipline experience in this role, because it is a different approach and a different mindsets and a different skill set the one required to be an enabler. And I don’t want to be very blunt but quite frankly as a reops manager I don’t care what methodology you use as a researcher. I don’t care which tool you use to make your base types as a designer. I care that you are able to do your job well. And so, you tell me what you need and the leadership tells me what the strategy is and what they are trying to achieve with this team and I will make sure that you have what you need in order to achieve that. That is my job.
So I don’t get involved in discussions about quantitative versus qualitative…. You just tell me what you need, and I will get there for you. It also means I can take a outsiders point of view, step back and not be as involved in the weeds of this, because that is not my area that is not what I am interested in. I am interested in the bigger picture and how we are going to get the amazing stuff done.
So, are you saying we shouldn’t hire researchers for ReOps management jobs?
Again, it depends on the context it depends on what your research strategy is so I think where you absolutely have to have a research background in order to be a research ops person is if your strategy is to have a couple professional researchers, and then to train everyone else to do research. Then you absolutely need to have a research background, because a lot of the job actually is going to be education, and not processing.
But in most other instances where you will be supporting a team of professional researchers. I don’t think you need a research background and it can be a hindrance because you might find yourself being pulled into the nuts and bolts of the actual research, rather than the process around it. The same with design. I have seen design operations managers who have actual design backgrounds and, inevitably there will be those high profile projects that need extra help and someone says “Oh, we have this person who has the relevant design skills, let’s asked her to help”. And you get pulled on to other things and that is when you lose ability to deliver on your main role. Whereas for me, you can ask me to do your design, but I am going to do a pretty bad job of it so you won’t ask me again. The same goes for research: you can ask me to do research, but it won’t going to be as good as yours. I can help you take notes, if that is ultimately what you need. But this is not really what you want to pay me to do, it is not the best use of my time.
That is a super interesting perspective, but you speak from a very privileged point of view, where you have been hired specifically to do this job in an organizational context that fully supports it. Although we can see that there are more and more companies that share this way of seeing things, many still don’t and researchers are out there doing research plus reOps jobs in companies. Management just sees it as something that has worked well so far, so why change. What would be your advice for people that find themselves having that conversation with their current leadership?
I have so much sympathy for that and I think this is where researchers unfortunately are their own worst enemy, as they tend to just be really organized and pragmatic and get on with it because they understand the value of what they do. So they put up with all this work, and they put up with all of this additional stuff that takes them away from what they need to do. The workshops that the research ops community did right at the start a couple years ago, one of the outputs was the Research Ops framework is an incredibly useful tool to show people and say look: this is all of the work we do! This is not research, this is all the work we have to do before we get to the research.
Do you think Design Ops and Research are one discipline under the umbrella of Scaling Design, or they are too different to be grouped together?
I think this is really interesting because I found a lot of value in going to design ups meetups and Design Ops conferences before when I just did the Research Ops role, because there is such a lot of overlaps in the process side of it. Ultimately we are talking about helping people work better, so there was a lot of things you could learn from each other, even though it isn’t the same discipline per se.
I think it is a question of maturity as well, as initially if you just have one community you can learn a lot from each other, but then as that evolves and as the industry matures and as the practices mature, you start having specialists underneath that, but it is still a good idea to maintain that umbrella. Shopify is an interesting example, their team is a UX Ops team and then within that team they have specialisms in research Ops, and in Design Systems, but it is all under one layer. They view it as rebuilding the operational system for the entire UX team, and specializing where needed and giving specialist support when needed. That was the direction Deliveroo was heading at the time I left, because there Research Ops pre-dated Design Ops and we realized after a while that there there was a lot of overlaps and that it made sense to have someone to look after that for the entire team rather than reinventing the wheel, again. So in terms of a community, I think it is probably better to start as a community learning from each other and then split up into specialisms. But it depends also in how design and research are structured within an organization. There is a tendency that I am observing for research to become a separate vertical next to design, not part of design, and to include marketing research and data analysis. So in that case, it’s a different specialism.
Currently there is a lot of confusion about what research repositories are and what they are for and who takes care of it. Are there contexts where they are more or less useful?
Yeah, 100% agree, context again is everything. It depends on what your ultimate strategy with the repository is. Do you want something that is a democratizer of research, fully self-service that anyone can go into and get what they need? Or is it primarily to help the researches, or is it primarily to help your onboarding people know what we already know and they don’t repeat existing work. Those things really inform how you end up solving the problem and what you end up doing.
I think, ultimately – and this is difficult because it requires a lot of buy-in and a budget – you need a human to manage whatever your repository is. Tools will get you so far, but you need someone who can look after it, who can make sure it is valuable, who can make those connections, who can do the secondary research when secondary research is required, who can pull things together in the right way for the target audiences, and who can be the champion for the processes around getting the data into whatever you need, because that doesn’t just happen.
You need someone to make sure that the data is valid, that it has been tagged correctly, that it is actually usable, that all of the governance stuff that needs to happen around that data is taken care of and you are not putting stuff in there that should not be seen widely and the access levels are fine. All of that requires specialists to look after it, and that often can be someone with a research, knowledge management, librarian or even a data analyst background, but it definitely is a specialist skill. And that is something that people tend to underestimate thinking ‘oh, we will get a really good tool and we will work out how we pull the data in or what needs to happen to get the data out, we will launch it and it will be fantastic. But then it dies and they wonder why. All this is about change management so you need someone to manage it all and to be the face of it, and to really champion it for your organization.
And also I think the additional benefit of having a human is you have got someone who can make the connections and speak about the validity of the data. One of the things that I found is that researchers are very wary, with reason, of the experience of just releasing the data into the wild and just having anyone access it. You need someone to help give that context: “this is what we found, but it was within this context, so these are the ways in which this insight can be applied”. Or “don’t use this insight to say x y and z because that is not an accurate reflection of what we found”. So you need that intermediary to make sure that the insights aren’t being misused or incorrectly interpreted and that we are actually making bad decisions. That is the worst thing for a researcher, you don’t want someone to use your data that you have worked so hard together to make a bad decision. And ultimately, it makes research bad as well, which none of us want to do.
That is where I am always a bit hesitant when people start asking about what is the best tool or the best way to structure my data and our repository because that will only get you so far. It is about the process around it really. People are realizing that is a role. You have entire departments in big enterprises focused on knowledge management and on documentation and access and process around it. And I think tech is starting to realize that that is the thing. Tech is notoriously bad at Information Management and I think it’ is a byproduct of the whole Agile approach as people think that because we are agile and lean we move fast we don’t need to document or find anything, but that becomes very very costly.
Do you think that if we replaced the word ‘repository’ with the words ‘design systems’… would this argument still be valid?
Definitely. A design system is a living thing and you can’t just have it outsourced to an agency, release it and then think you are fine. You still need someone to manage it and to be the face of it. It needs someone to spot the connections between the different teams working and spot the opportunities to improve it and add to it, adjust what is no longer working. It is a very good parallel to draw actually.