Reviews are inherently subjective. The major aspects that influence a review — how a review is performed, what the reviewer is looking for, the importance of different criterion, etc., impact the review that ultimately comes out.
But before we discuss the criteria, how exactly is a review performed?
For the media reviews (physical and streaming) that come out on this site and on our YouTube channel, we implement the “enthusiast viewer on a moderate budget” approach in both how the review is executed but also in how we score the review.
What does that mean?
It means that we aim to experience the movie as a movie enthusiast who has spent some money on a home theater set up, but it is not so out of reach that only a handful of people can have that same experience. Ideally how we experience the movie when we watch it will be similar to the experience in your home. Will it be exact? No, everyone’s watching experience will be different based on the equipment they have, how they set it up, and even their watching preferences (dark/light room, volume level, etc), but we hope we get it close enough where we are not hearing/seeing things that you could not also hear or see purely because of availability of specific equipment.
Why do you take this appoach?
We review this way, an “enthusiast on a moderate budget”, because that’s exactly who we are. This site doesn’t exist because we like to review any ol’ thing and just happened to pick movies and home media. It exists because we love movies, love home media and want to offer our thoughts out to the community that shares our passion as they consider their next purchases, or just want to compare/contrast with other like-minded enthusiasts.
The approach is not only reflected in the equipment we use, but more importantly in how we evaluate the home media. For us, if the average enthusiast can’t hear or see something in a piece of content, but still enjoyed it, is it really helpful knocking something down in points because something doesn’t have enough nits of brightness, or only has a certain number of objects in their Dolby Atmos mix? We don’t believe so. The biggest factor is how all of those pieces come together to create the experience.
OK, enough rambling on about our philosophy. Here’s how home media is reviewed.
- Here’s how we experience the media we are reviewing.
- We watch everything at least twice. The first watch is for the pure enjoyment. Experience the movie without the analyst voice going off in our head (…as best we can).
- The second watch is to look and listen to various critical aspects in both the quality of the picture, but also in the sound.
- During the second watch, we usually stop of several different scenes and re-watch/re-listen to those scenes to try to focus and dissect (within reason) different aspects using our eyes and ears (no measurements here).
- For example if there is a Dolby Atmos mix, not only do we want to listen to critical aspects of clarity of dialogue, but also how surround effects are leveraged to create the ambiance. This includes turning off the main LCR, to listen to just the surrounds, then turning off the entire bed layer (the speakers at ear level) to listen to the height effects.
- Our Scoring considerations
- The paramount consideration is how something affected the experience. Again, if the average enthusiast would not notice or care, then we don’t factor it in.
- The scoring scale runs from 0.0 to 5.0. This 5-point scale is very familiar to people so its easy to understand both for us to review as well as for those consuming the review. If you have ever rated anything on Amazon or Netflix, then you know how a 5-point scale it works. 5 is the best, 3 is average, and 0 is the worst.
- Compare like to like. We want to make sure that we are not comparing a 15 year DVD to a 2-week old 4K Blu-ray. Its not really helpful as of course the latter will look and sound better. For us it makes more sense to compare 4K Blu-rays to 4K Blu-rays. We’ve seen a lot of them so can tell what a really good release looks like compared to a subpar release. This is also why a Blu-ray release may score higher than a 4K Blu-ray release of the same movie. The only caveat here is when evaluating recommended upgrades. More on that coming up.
- We score three categories: Video, Audio, and Special Features/Packaging. The latter only applies to physical media releases right now as streaming services are very inconsistent on their add-ons. Each will receive a numerical score in the 5-point scale.
- Video: Based on the media, how was the visual presentation of the movie. We factor in things that can impact the experience like, sharpness, contrast, color, fine detail, film grain presentation, etc.
- Audio: We factor in different aspects that impact the experience of the movie: Clarity of dialogue, immersive-ness, fidelity/appropriateness to the source, LFE/Bass, etc.
- Special Features and Packaging: For special features, what special features are available, are they new/added for this release, how do they impact the appreciation of the movie (i.e. does it enrich the experience to have the additional information), how well are they put together, etc. For packaging, it can be what is included, slip cover, imagery, how the discs are stored, disc art, etc.
- Every review where we have access to Upgrade Recommendations are based on if we were actually taking the money out of our own pockets (because we did). So if a movie upgrade truly improved the experience to a point where we were happy we spent the money, then we will say so (“YES!”). If not, but it would have at a lower price (“Probably…”) then we’ll let you know. Or if we would rather just stick with the less expensive/older version (“Nope.”). Often we’ll add clarifiers to the recommendation (e.g. YES, but only at a certain price point)
So that’s it. Hope this helps understand how we consider our reviews. This process absolutely can and will evolve, but we’ll always try to take that into account. If we add new criteria, or change our a piece of equipment, we’ll do our best to acknowledge that in the scoring process so we can be as consistent as we can be.