Podcast: Play in new window | Download
How many times have you read a blog post by a social media expert that told you to do one thing, and then listened to a podcast by another expert on the same topic that told you to do the exact opposite? Now it makes sense that different people will have different opinions, but what does the actual data say?
This is where Social Media Lab fits in. They are the Mythbusters of the social media world. They take a scientific approach and perform real tests on the types of social media questions that marketers have when it comes to what really works and what doesn’t, what will give you a better reach with your audience, and what gets more conversions…and they have the actual numbers to prove it.
How many hashtags is too many? Instagram Carousel or individual photo posts? Instagram Story ads or Instagram feed ads? How do your current social media marketing strategies rank up against the data? Maybe it’s time to try something new.
Rich: He’s written over 2,000 posts on social media topics, he’s the co-author of Facebook All In One For Dummies, he’s a blogger on three separate blogs that have each made Social Media Examiner’s Top 100 Social Media Blogs. That only leaves 79 other spaces for you and me. And now he’s the current lead blogger and podcaster for Social Media Lab powered by Agorapulse. They man, the myth, the blogger, the legend, Scott Ayres. Scott, welcome to the show.
Scott: Rich, that may have been the best. Can I take that snippet and use it on my podcast? I think I will.
Rich: Absolutely. Most people send me over a whole bio and I have to read it exactly, and you just sent me some bullet points. And I said those were some impressive bullet points, so there you go.
Scott: Quick and to the point.
Rich: So my first questions, Scott, is do they make you wear a white lab coat at the Social Media Lab?
Scott: Well I actually do have a white lab coat. Funny enough, it’s sitting right next to me in my office. And I even have an orange crazy haired wig that I put on to entertain in team meetings sometimes.
Rich: Excellent, alright. At least you’re living the role. Alright, so before we jump into some of the recent tests that you did on Instagram, which is what we’re going to be talking about today, can you tell me a little bit more about the process that you have when you do these kind of tests? In fact, why don’t you tell us a little bit about Social Media Labs to start with, and then kind of your process. That would be great.
Scott: Basically the Social Media Lab was kind of this brainstorm idea that our CEO at Agorapulse had, I guess about 2 years or so ago. He brought me on board in early 2017, and his thought process was there’s a lot of blogs out there that just give fluff and opinion – The Top 10 this, the Top 5 that – and not enough value for him as a CEO to want to read. So he said let’s change that and do something about it.
So our goal at Social Media Lab is to run these really in depth experiments about social media, and then do it in a scientific manner and apply science to how we conclude based on the data. So what we kind of do is we go through the scientific method – if you remember that from 10thgrade Biology – We kind of figure out, here’s some ideas we want to test, can we even test them, and then I go out and research them and come up with a hypothesis. And I guess, here’s what I think based on my experience, based on the research, and these aren’t short tests.
The typical test that we’re running is anywhere from 4-6 weeks long with hundreds of different posts on multiple social media accounts. That way we have a really good barometer of what’s going on so we can say this is scientific and not just, “I posted twice on Instagram and this is what worked”. We want to look at hundreds of posts and look at it that way.
Once we go through the testing and gather data, then draw a conclusion based on the science that’s there. We actually have a data scientist who’s on our staff who worked in the banking industry for years. He does a lot of stuff for us but he also helps us with the lab, he’s in our Paris office.
So we created what’s called a “statistical significance calculator”, which is a phrase I couldn’t say a year and a half ago without stuttering. What it does is we put all the numbers in and it gives you back this value and it has to be at least 95% in order for us to say something is statistically significant, which is really hard to reach a lot of times. You and I, we might test A versus B, and B is 60% better so we’re going to say that’s better. But sometimes the stats don’t say that. So we have to put it in there and if it’s not better, than we’re not going to report that it’s better. So we might just say this one and this one were the same, but this one was easier. That’s kind of how – in a 2-3 minute bubble – how the process sort of works.
Rich: And I think the takeaway for me is, this is fairly rigorous. It may not be like CDC kind of rigorous, but it’s still pretty rigorous especially when it comes to social media sort of stuff.
Scott: Yeah, because you’ve mentioned in the past that I’ve written over 2,000 blogs in the last 7-8 years. I can pop those out and write them in 30 minutes. Just quick, boom, boom, write them and go. And they get lots of shares and that sort of stuff. But these sort of things, they take… when I start one, I’m right now thinking about what I’m going to write about in January, 4-5 months from now. Because there’s so much involved in setting up the test and running the tests and all that sort of thing. So it’s a lot different than the typical test you see where they just pull data and make a conclusion. We’re actually testing and then showing you the data.
Rich: You are the Mythbusters of the social media universe.
Scott: I don’t have a goatee or anything cool like those guys did on the TV show, but I need to.
Rich: Plus, it’s tough to blow things up on social media. I mean, you can always blow things up on social media, but you know what I’m saying. Anyway, so you and I decided we were going to talk about Instagram. What are some of the things that you’ve been testing recently in regards to Instagram?
Scott: We’ve done a lot, Instagram is probably our major site that we test on the most, because I think it’s an unknown for a lot of businesses. They still haven’t figured it out completely even though it’s been around for a while. How can we market on it, what are some tactics? So the first thing we kind of went after was something called Instagram Carousel where you can upload up to 10 photos on one post. You can swipe left and look at all your photos from your event or maybe you’re showing off a recipe or something like that.
So it was a new feature last year when we tested it and let’s see if it should perform better. The whole though process for Instagram was, you’re posting 10 photos, you’ll get a lot more engagement, people will stop in and thumb through your photos. But we found that wasn’t true, we found it was quite awful actually, the engagement levels were very very, low, sometimes 20% lower on those carousel photos.
So I tested that, and this is sort of the first one I’ve been excited about where we actually re-tested recently. So the new data on that – which was pretty interesting – the two biggest takeaways that I get from the test is the ‘likes’ on Instagram Carousels – and ‘likes’ are your goal when it comes to Instagram – they were 10.73% lower on the Carousel post as opposed to the regular posts. And then reach was over 11% lower on the Carousels. So that number was the one that surprised me the most because you think exposure would be the same regardless, there’s still the same amount of followers. But reach was actually lower, which tells me either people were hiding you if you did one or two of them, or Instagram realizes nobody cares about these and they’re not showing them as much.
So that was the first one I went after, and probably the biggest one to kind of go, “Wow, I’m not going to waste 10 photos on one post, I’m going to make 10 individual posts instead.” You got more content, and you’re going to get more engagement throughout the month instead of throwing it all into one post. So that was the first one.
Rich: And I don’t want to bring an opinion to a data fight, but at the same time just anecdotally I know when I’m flipping through photos as a person on Instagram, I almost never stop at the Carousel. And the few times that I do, I am less likely to ‘like’ it because then am I ‘liking’ this photo or this photo, and can I ‘like’ just one of the photos. So it takes me out of what I expect out of Instagram and that’s always been my experience.
So that’s one issue I think maybe some people are having with it. And then you just mentioned that the reach is lower, and I just have to assume that some of this is because the Instagram algorithm becoming stronger and stronger over time, where the first few people who see it aren’t likely to engage with that, ‘like’ it, or flip through it, then that sends a message back to Instagram that this is a less valuable post. So that just reinforces it and makes it less likely that they’re going to show it to other people. Correct?
Scott: I think you’re 100% correct and one of the things that I said, it’s one thing to have the data, but then what do we do with it. I think the reason – like you mentioned – it’s so foreign to me to go left and right on your phone. If you’re using your phone to scroll up to the next one ok, but to go left and right is so foreign to users that I think that was one of the biggest reasons it’s been such a failure, for Instagram. We even looked at Instagram’s actual account to see if their carousel did worse or better, and they were worse on their page as well. Which is kind of funny. So yeah, I think you’re 100% right. I think it’s just a weird action that you’re forcing people to try to do, and it’s just not working.
Rich: There’s part of me that would want to see if I could get around that hurdle where it’s like maybe the first picture of the carousel might be something like, “swipe left to find out the answer to this question”, or something like that. But again, that’s a lot of extra work for the typical small business marketer who’s just trying to get as many people to engage with their photos and Instagram as possible.
Scott: Definitely. And it’s not to me worth the time, I might as well just post regular posts.
Rich: Alright. Well I know one big question that a lot of people have on Instagram is around hashtags, and I’m pretty sure you’ve done something on hashtags. What has the data shown you on hashtags when it comes to Instagram?
Scott: We’ve done a lot of hashtag tests. We dove in first on do hashtags even matter. We found out obviously they do, you had higher reach when you have hashtags. And then we looked at what was the ideal number. We found that 8 was the ideal number of hashtags, when we tested between 1-10. Because I think that’s more natural, between 1-10.
You can stuff 30 into a post, but I don’t know about you, but trying to go through that and remember 30 hashtags every time is annoying.
Rich: Are you thinking as a user or as a publisher.
Scott: I’m thinking as a user, well, that’s true. Publisher.
Rich: Right. But when I look at that, that’s a forest, not a couple trees that I might notice. So I’m not seeing the trees through the forest at that point. And it’s just like this big, ugly block of red – or whatever color text – and to be honest, I don’t see any of those links. That’s just my anecdotal, personal experience.
Scott: Well yeah, that’s good. I think that brings me over to our latest test that we did on Instagram that was almost controversial but kind of when against the grain of what a lot of “social media gurus” say. I’ve heard from a lot of the people out there that teach social media for Instagram marketing say, “Well you should use the 30 hashtags, which will get you more reach”. The whole point of the hashtag on Instagram is not to expose your content more to your existing followers, but so new people find you in the ‘explore’ option. So that’s what the hashtags are for, and they work. So you can use up to 30.
But they’re telling people to stick them in the comments so you don’t see that big, ugly block that you just mentioned. So we wanted to test that and see if it was true or not. So I used the same 30 hashtags on posts where I put the hashtags either in the original post itself, or in the comments, to kind of see what the difference was. Was engagement different, was reach different. This one was really interesting to me, because this one, I didn’t think it was going to work out this way to be quite honest with you.
We found that ‘likes’ were 9.84% higher when we left the hashtags in the post itself. And then reach, or impressions – you kind of have two different ways to look at it on Instagram – I call it ‘reach” everywhere. Reach was actually 29.41% higher when we put the hashtags in the original post. Which goes against what I’ve been hearing from the experts for years that you should always hide them in the comments.
I’m looking at Instagram now while we’re talking and if you get past about 2 lines of text on the original post, you get that “more…”, so no one is seeing that anymore. It used to be that way where you would see all of it and it was gross looking. Now you only get two lines of text so your current followers are never going to notice that you’re stuffing 30 hashtags into the original post, unless they click on ”more” and it opens it up and shows it to them. And I don’t know about you but I don’t click that “more” very often when I’m looking at posts.
So it’s a real interesting thing to say well if it’s going to perform better that much, I should do it that way instead and not waste all that time and energy to go back and put it into the comments. And so that was a huge one for us that went against what a lot of my friends are out there teaching. And it was like, gosh, I really got to publish this even though people I trust and love are teaching the opposite. So that’s kind of the dilemma we run into a lot of times.
Rich: And I think a lot of times these social media experts are out there and they found something that works for them and they go out there and they start making it part of their blogging and podcasting and presentations, and then they don’t go back and test it the way you did with the carousel, and they’re constantly tweaking with these algorithms.
And I’m as guilty as anybody else. I’ve caught myself sometimes saying, “Is that even true anymore?” And then I’ll check it and it’s 180 degrees different now. So that’s probably where some of the stuff is coming from so it’s good that we’re coming back an checking this sort of stuff. Because I bet you back that there wasn’t always that ”more” thing, so it did look all cluttered before and now it doesn’t. You have to wonder if Instagram is giving more weight when it comes to searches for hashtags that appear in the original post as opposed to the comments, and maybe that’s what’s impacting the reach.
Scott: That’s what I was fixing to say, you’re 100% right on that. Because what’s happening is anytime you and I as crafty marketers start figuring out a way around the algorithm, Instagram, Twitter, and whatever is going to go, “Hey crafty marketer watch this, we’re going to change that.”
And so now what happens with the ‘explore’ option, when you’re searching in the little magnifier glass on Instagram, if you type in a hashtag to search, it’s only going to show you those in reverse chronological order. So the newest posts first when the post was made.
So if I make a post right now and I don’t put the hashtag in it and I come back even 5-10 minutes later, how many other posts have come out with that hashtag and now I’m pushed so far down no one is going to see me in that search. So that’s what they’ve kind of done in the past.
So three or four years ago whenever you put that hashtag in the comment, it popped it back up to the top of the explore feed. So that’s not true anymore because Instagram realized that people were gaming the system. So now it’s about explorer is based on when you post, so I think it’s important to make sure you get that hashtag out there immediately. Don’t be so worried about people seeing that you’re using a bunch of hashtags, that’s what everybody does if you’re marketing on Instagram. But also make sure your hashtag is relevant. If I’m running a pizza place I don’t need to post about social media marketing, because I have nothing to do with that industry.
So put them in the post, don’t waste your time going back and put it in the comments, because it’s really hard to remember to go do it or you might use an app that sketchy to do it. So that’s the takeaways that I got from that test.
Rich: So a couple of things that I’m thinking about. One is, as you did this test versus where you should put the hashtags, did you come up with an optimal number of hashtags? Does it matter if you’re putting 30 hashtags in or is it still hovering around 8 for you?
Scott: Well for that test we went ahead and we used the same hashtags, 30 in the post or the comments, that way it was kind of even across the board. We could have gone back and just used one and seen. My anecdotal thought is since we know hashtags perform better that it wouldn’t make that big a difference, but it is something to think about. Do I go back and test it now with just 8 and see if there’s any difference? So I’ll write that down, that’s probably something worth testing. But for that test though, we used 30.
Rich: I would think that it shouldn’t matter all that much, certainly not for reach, because the whole point with reach is you’re trying to get in as many different places to be found as possible. Relevant places to be found. So that’s probably not going to change it.
Scott: Another thing is look at the percentage of differences. So not necessarily the wrong number. You reached 100 more people, but what’s the percentage difference. That way if it is a different data set we should be able to compare equivalently.
Rich: The other thing that I’m thinking is – you kind of said this – is we start out and find some chinks in the armor, so to speak, with social media. And as marketing experts we’re supposed to be able to hack them and then we teach them. And as soon as it kind of gets out there in the wild, these platforms such as Instagram and Facebook realize that and they change the rules on us.
This is not that different from when they released video games – a lot of these multiplayer games – too. They release them, they let people play with them for a month or so, and then they basically come back and they re-evaluate the strengths of all the different character types. Because they realize that all of a sudden they made the clerics too powerful or the knights too weak, or whatever it may be. And so they balance it so it’s a better game for everybody. And that’s kind of similar to the way maybe these social media channels are trying to create a better environment where people are trying to spend more time and that works out better for everyone.
Sorry if I just nerded out on you.
Scott: No, totally fine, totally fine. I get it, yeah.
Rich: Alright. So what other things have you been testing lately when it comes to Instagram? Let’s do one more.
Scott: Yeah, so a lot of interesting things. This is one on the paid side. So we just talked about organic tests, which sometimes are harder to get a definitive answer. The paid ones are a little different because you’re looking at conversions 100%. Did I get clicks, did I get people to buy, how much money did I spend, that sort of thing.
So Jason Howe who has done a lot of our paid tests for us in the past, he did a test on Instagram Story ads versus an ad in the feed on Instagram. Stories are those little vertical things at the top where you push on it and they’re 14 seconds long and you kind of have to go to them to look at versus the feed ads. And so it was a real interesting test to me because Stories were kind of new when he ran that. I’m not a millennial so I don’t spend all my time in Stories, but millennials evidently that’s where all their time is spent. And more and more you’re finding that Facebook and Instagram are really focusing on these Stories and this expiring content. I think it’s going to be a big thing moving forward. So this was kind of interesting. He spent about an equal amount of money, $314 on the Instagram feed ad and $310 on the Instagram Story ad, so a very similar ad spend. And this was for the same thing, we were directing people towards the blog post about Instagram pods. The feed ad had a reach of about 18,000 people and generated 592 unique clicks.
Where the Story ad which requires people to go up there and actually push the little bubble and look at it, it reached 26,700 people. So a 44% higher reach. And out of that generated 1,039 unique clicks. Which was pretty intriguing to me because it requires that action for pele to go do, but people are so intrigued by these Instagram Stories. So 44% more people reached and then the conversion rate of that – if you kind of put it into the A/B split test measure – the Instagram Story is at a 23% higher conversion rate than the ads we did in the feed itself. And when we stick that into our little statistical calculator, it comes back it’s 100% certain that if we were to run that test again or if someone else were to run a similar test, they would get those sort of same results.
So that one to me was super intriguing the fact that we could get 78% more clicks on those Story ads versus the feed ads. So that’s one for us that we’re playing with constantly now. Maybe instead of running my ads in the feed I need to direct ads in the Stories which is still kind of a new open market for a lot of people. And especially dependant on your industry, you may be the only one advertising on Instagram Stories so you can really harness that market. So that was a pretty cool test for us to ind of try out and the results were great, I thought, on this one.
Rich: That’s definitely really interesting. I just have to wonder if some of this is due to the fact of the newness of Stories. So because of that there’s less advertising, there’s more openness to testing out something. So when you are ready to retest this, I’m curious to see if the numbers hold up.
And the other thing is, I would love you guys to test something that’s not about Instagram. Like maybe something like selling a watch or something like that where there’s a different kind of thing. I’d love to see if the engagement stays the same or if things change over time. And I’ll throw my money at it. The next year comes around and you want to redo this test, we’ll test the Agents of Change ticket sales and we’ll just figure this stuff out together.
Scott: Yeah, just to see which one converted more, yeah. And I do think, like you said, tangible product versus a virtual go read a blog is going to be a little bit different. But if you’re trying to drive traffic, that’s kind of what we were looking at. We found that they clicked on them more, which again to me is so surprising, because I’m a 43 year old guy and I don’t ever click on those things unless I posted something there myself, then I go look.
But we’re finding that that generation love that expiring content, that FOMO mindset, and Instagram is leveraging that and there’s money to be spent in that and made in there if you’re using them right.
Rich: That is really interesting. I had never thought of advertising on Stories. In fact until you sent me that link or told me that was something we could talk about, I don’t know that I ever spent enough time in Stories to see an ad. I probably did and didn’t realize it, or I probably did and skipped over it. I’m not a Stories guy. Even my friend’s Stories, I’m like, “Ugh”, I just can’t even bear them. So I’m obviously being a 50 year old guy and not their target audience either, but still some fascinating stuff.
Scott, so you ever take requests? Like do people send you emails and say, “You should really test this”?
Scott: Oh yeah, definitely. We love those. Because for me it’s just like when we mentioned at the beginning if the podcast that social media marketers sometimes tell us things to go do and they may not be what everybody wants to hear. The same with us, there’s things that you and I think about that others aren’t and vice versa. But yeah, if someone has a testing idea I’d love to hear it.
Rich: Alright, well you know where to send them then. What’s the craziest request you’ve ever gotten, just some real out there request where you’re like, yeah I don’t think so?
Scott: We actually did a contest with a bunch of influencers asking their audience for ideas and we would select them and mention them in the blog post. And some of them were very sentiment based, like, “Why do people click more often on such and such business and not mine?” You can’t go figure out. Or, “How can a non-profit get more clicks?” I don’t know. I can test the link but I can’t apply it to a non-profit.
So that’s the hardest part is some of that sentiment stuff is really, really hard to figure out because it’s going to be so different from mine to yours. That’s why I like looking at the reach, or conversion versus I got 4 hearts on this post. But I’m not Kim Kardashian, she got a million. There’s different audiences with those. But some of those things if we can boil it down to the stuff that applies to everybody, that’s where it’s always better.
Rich: Awesome. Scott, I’m sure a lot of people want to check out some of the stuff we talked about, also some of the other research you’ve done outside of Instagram. Where can we send them if they want to check that out or where can we send them if they want to learn more about you?
Scott: You can go to agorapulse.com/socialmedialab. Or just search for “social media lab” and we should come up on that. If you’re looking for a podcast, we’re on all the podcast apps as Social Media Lab, and on social we’re simply @agorapulse on all the different social media accounts.
Rich: Fantastic. Scott, thanks so much for coming on the show and sharing your research.
Scott: Thanks for having me over and definitely need to get with you and do some tests.
Rich: Absolutely.
Show Notes:
Scott Ayres spends his days testing out all the tips, advice, and strategies from social media marketing experts, by applying science and providing real data to find out what works. Check out some of the research he’s done, or catch his podcast for more insightful research results.
Rich Brooks is the President of flyte new media, a web design & digital marketing agency in Portland, Maine. He knows a thing or two about helping businesses grow by reaching their ideal customers, and to prove that, he puts on a yearly conference to inspire small businesses to achieve big success. You can also head on over to Twitter to check him out, and he has added “author” to his resume with his book!