If you're reading this, you already know that site speed is critical. A fast website better engages users, produces better sales, and ranks better for search.
For WordPress websites, many factors can affect site speed, including your hosting, your theme, how many plugins you have, which plugins you have, and what type of content you provide.
Here we focus on just one slice of that performance pie: gallery plugins. Our goal is to provide a fully transparent, rigorous, apples-to-apples test of top gallery plugins.
I have a PhD in Political Science. I don't get much chance to apply that while running a web-based software company. But a speed test comparison is one place where PhD-type training can help.
A big part of the training one gets in a PhD program is how to structure research so that you can draw accurate conclusions. One way to do that is to control as many variables as possible so that you can see the effect of just the variables you care about. Another way to do it is to be transparent about data and methodology so that anyone can duplicate your experiment. If someone can duplicate an experiment, they can confirm (or not) the same results.
We've tried to build both of those considerations into this test.
Controlling Variables. To control variables, we've designed the test to keep as many variables constant as possible so that the only variable determining differences in speed are the gallery plugins. To do that, we've done the following.
- Same host. We've eliminating hosting considerations by using the same host, WPEngine. WPEngine is one of the biggest and best known hosts in the WordPress space.
- Same themes and plugins. We've eliminated the impact of third-party themes and plugins by using the same theme (Imagely theme) and by removing all third-party plugins. The only plugin active on each site is the gallery plugin itself. There's one caveat here: because Meow Gallery does not have a Lightbox, we also added Meow Lightbox.
- Same images. We've eliminated the possible impact of image sizes by uploading the exact same batch of images for each test gallery.
- Out-of-the-box displays and settings. For any of these plugins, there's a lot of tweaking that can be done to global settings or individual displays that can affect performance. For our test, we've left all options in their default settings, and displayed each gallery using the default display for each plugin.
- Multiple Speed Test Tools. Every speed test tool works differently. In order to triangulate test results and avoid the bias of any single speed tool, we've used several tools and approaches. For third-party tools, we've used Pingdom, GTMetrix, and Webpagetest. But we've also used the in-browser developer tools (see below for a detailed explanation of how to do this).
Transparent Methodology. In addition to trying to control as many variables as possible, we've tried to make this test as transparent and duplicate-able as possible. Using the methodology section here, you can easily and completely set up the same test to confirm the result.
To make it easier for you, we've left our test sites live and shared the URLs below. So without having to do all the set up, you can run all the same speed tests with the same speed tools as we do. You can also confirm - for the most part, without admin access - that the test sites are configured the way we say they are. Here are the urls for each test site:
- NextGEN Gallery: https://wordpress-568759-2263927.cloudwaysapps.com/
- Envira Gallery: https://wordpress-568759-2263841.cloudwaysapps.com/
- Foo Gallery: https://wordpressmu-568759-2263875.cloudwaysapps.com/
- Modula Gallery: https://wordpress-568759-2319183.cloudwaysapps.com/
- Meow Gallery: https://wordpress-568759-2319198.cloudwaysapps.com/
Note: How to Test Speed Using Developer Tools. We mentioned we used several third-party speed test tools but also used in-browser speed tests. One could argue these are more accurate - but in any case, you'll have multiple test results from which to judge. Because this method is a bit more complex, we wanted to provide an extra note on how to do this yourself. Here are the basics:
- Open any browser, like Chrome, Safari or Firefox.
- Go to Incognito or Private mode.
- Look for their developer console. You can Google "BrowserName Developer Console" if you need assistance finding it.
- Go to the Network tab instead of the developer console
- Go to your website.
When your site loads, you'll see three performance measurements. These reflect different, sequential stages in the loading of your site content in the browser: DOM Load, Load, Finish Time. You can choose to look at one or all of these measures.
To ensure the best data, we'd suggest that you disable cache as we did (most browsers have an option to Disable Cache in the network tab of their developer consoles). We'd also recommend that you run the test multiple times: refresh the page, note the load times for each page refresh, and then compute an average.
Results: Raw Tests
Below, we've compiled the base, unfiltered results from all tests. We ran two tests per testing tool and recorded the results.
You'll note here how much variation there is between different testing tools. Particularly notable is Webpagetest, which is an outlier and shows much longer testing times across the board. The variations between load times underscore why any speed comparison should use multiple testing tools.
|Meow Gallery & Lightbox||1.29||1.19||1.67||1.3||0.9||3|
Results: Average by Gallery and Testing Tool
|Meow Gallery & Lightbox||1.155||1.315||2.005||1.3||0.85||2.925|
Results: Overall Averages
Finally, the table below shows a single number that averages up all the testing tools for each plugin. Note that in almost all cases, the higher results for Webpagetest push these averages higher than they would be without that test.
|Meow Gallery & Lightbox||1.59|
Conclusion and Discussion
The overall conclusion from the data above is fairly clear: NextGEN Gallery outperforms most other gallery plugins - by a wide margin and consistently across nearly all testing tools. The exception is Webpagetest, where NextGEN Gallery comes in third.
Among the other plugins, Meow Gallery and Foo Gallery come in second, nearly equal in overall averages. And Envira and Modula trail in fourth and fifth respectively.
For some, these results may come as a surprise. We've bumped into some WordPress folks who've assumed that because NextGEN Gallery is more feature-rich (and thus has a larger overall codebase), it must be slower. This perception was reinforced by performance issues with NextGEN Gallery 2013-2015 (four years ago), and by the fact that some of the other plugins included here have made 'speed' part of their marketing messages.
Addressing these kinds of questions and confusions is one reason - though clearly not the only one - that we've taken the time to set up and publish this comparison.
We're also aware that because Imagely makes NextGEN Gallery, we have a vested interest in seeing strong, fast results for NextGEN Gallery in this test. Concern about bias would be warranted. It is precisely for that reason that we've been so clear and transparent in the methodology. That's also why we've left all test sites live so readers can evaluate them.
If you have questions about aspects of this comparison, feel free to post them below, and I'll try to check in periodically and respond. Thanks for reading!
Erick Danzer, CEO Imagely
In response to feedback on this post on social media, I wanted to add two notes. I'll continue to add other updates here if they arise.
Update #1 (Sept 4, 2019): Full width vs partial displays. The Modula and Meow displays at the time of publication were full-width displays. We had thought that was due to differences in the displays themselves, but it was due to a full-width page setting. We've updated that. It does change the appearance and makes the displays look similar to the other plugins. We retested with speed tools, and there was no significant speed difference due to page width.
Update #2 (Sept 4, 2019): Note on different display types. One fair caveat about these results is that the default gallery styles vary by plugin. For example, NextGEN Gallery shows thumbnail galleries by default, while Modula shows a tiled-style gallery. It's possible that dynamic galleries like Masonry/Mosaic/Tiled may have longer load times. The same would be the case for default display styles that have larger vs smaller thumbnails. We're considering adding an additional test on each site that would compare similar galleries (ie, all thumbnail or all mosaic, similar sizes, etc). But for now, it's worth noting that factors like display types or thumbnail sizes could easily affect load times. If you're a user that's deeply concerned about performance, that's something to keep in mind and test regardless of which plugin you're using.
This Post Has 2 Comments
Jon15 Oct 2019
Haha slight bias here resulting in a really misleading post 😂
Scott Wyden Kivowitz16 Oct 2019
We address the bias factor in the post itself and back up the results with the data and demos.