are these test results worth anything?

http://bulbfаcts.com/halogen-bulbs/reviews/

At first glance, some of the results make sense, such as the 100% blue tinted bulbs generally have the lowest tested brightness--lower than a standard, OEM type bulb--with the exception of the Sylvania ZxE series.

Here are some of the charts:

http://bulbfаcts.com/assets/img/vs-oem/brightness/Philips-Diamond-Vision.png

http://bulbfаcts.com/assets/img/vs-oem/whiteness/Philips-Diamond-Vision.png

Why is it that the commonly recommended Philips XtremeVision bulbs fall behind in this site's testing?

sure they are, they show what not to do with your vehicle.

I would not take bulb recommendations from a website that suggests "The Best Xenon Look" is a reasonable criterion for selecting a halogen bulb. They speak of a bulb's "whiteness"; an automotive halogen bulb, run at the correct voltage, produces white light. None are whiter than another (although some may tend more towards blue or yellow more than others).

They also 'test' LED and HID kits, which are not safe or legal. It appears this site's primary goal is to get ad revenue from sponsored links and kickbacks from the vendors of these things when their coupon codes are used.

Short answer: No.

Alaric Darconville said:

Short answer: No.

Click to expand...

Thank you for confirming my suspicions. The results themselves are probably skewed so people buy the most expensive bulbs on Amazon with their referral links. However, one question remains:

Why is it that halogen bulbs in their '09 VW Jetta H7 reflector test rig produce lux measurements at 12 feet that are ~2x greater than the same H7 bulbs in their '09 Mercedes E550 H7 projector headlamp?

Ls400 said:

Thank you for confirming my suspicions.

Click to expand...

It's what we do!

The results themselves are probably skewed so people buy the most expensive bulbs on Amazon with their referral links.

Click to expand...

Or the runners up, or any other product they give positive reviews to.

However, one question remains:

Why is it that halogen bulbs in their '09 VW Jetta H7 reflector test rig produce lux measurements at 12 feet that are ~2x greater than the same H7 bulbs in their '09 Mercedes E550 H7 projector headlamp?

Click to expand...

Without having immediate access to either of those vehicles' headlamps' performance data, remember that a beam can be compliant but still have subtle changes and optimizations. Mercedes may have opted for a slightly wider beam, and therefore 'stole' or reduced from the size of the central hotspot, reducing hotspot intensity at certain points. Or BulbFacts' testing methodology is bad, and had the sensor slightly too high or uncentered for one or the other lamps. Or they're not controlling voltage properly.

Alaric Darconville said:

Without having immediate access to either of those vehicles' headlamps' performance data, remember that a beam can be compliant but still have subtle changes and optimizations. Mercedes may have opted for a slightly wider beam, and therefore 'stole' or reduced from the size of the central hotspot, reducing hotspot intensity at certain points. Or BulbFacts' testing methodology is bad, and had the sensor slightly too high or uncentered for one or the other lamps. Or they're not controlling voltage properly.

Click to expand...

Yeah it would be interesting to know what point(s) in the beam that the site is testing.

I asked because the results remind me of my own situation. The situation reminds me of my '10 Corolla and '13 RAV4...there is a massive difference between drivability at night in favor of the Corolla, despite the fact that the Corolla uses 9012 low beam bulbs instead of 9011 low beam bulbs. The difference is that the RAV4 has projector optics while the Corolla has reflectors...

Whatever that site owner's intentions might be, the list of reasons why his "tests" aren't actually tests/aren't producing any kind of useful data is so long, it's difficult to choose where to start. For openers, 12 feet is not a long enough distance to be testing at. Who knows what kind of meter he might have been using, but it doesn't really matter; at that short of a distance the meter readings aren't going to translate to any useful information, and they aren't even reliable for comparison with other readings taken at the same distance. We have no idea how stable his power supply is during each run and across the various runs. He consistently misspells certain easy-to-spell bulb brands, which doesn't really impact the numbers but throws a cloud of doubt over his general credibility (WTF? The box and the bulbs are right in front of him!) His use of different headlamps for low beam v. high beam is strange, and his "best for low beam!" and "best for high beam!" claims are just plain misleading. His use of color temperature as a criterion is not compatible with his claim to be testing bulb performance.

His site is full of mindless copypasta; he's talking about "HID kits" on the HID page, "LED kits" on the LED page and..."halogen kits" on the halogen page. Halogen kits??? His "review" videos are just some schlub (I guess it's him) sitting there babbling at the camera, interspersed with completely useless footage of headlight beams on a road. I think Alaric is probably right; that site's main purpose seems to be to drive traffic to his YouTube channel and get clicks on his Amazon Affiliate links.

Ls400 said:

Why is it that halogen bulbs in their '09 VW Jetta H7 reflector test rig produce lux measurements at 12 feet that are ~2x greater than the same H7 bulbs in their '09 Mercedes E550 H7 projector headlamp?

Click to expand...

Different headlamps perform very differently. For the purpose of discussion, let's assume his peak intensity is at or near one of the low beam test points. At 0.6D/1.3R, the requirement is 10,000 candela minimum, with no maximum (at 12 feet that would be 746 lux min/no max, assuming a measurement at that distance would be valid). At 1.5D/2R, the requirement is 15,000 candela minimum, with no maximum (theoretically 1120 lux minimum/no maximum).

[Did you get that book yet?]

Ls400 said:

I asked because the results remind me of my own situation. The situation reminds me of my '10 Corolla and '13 RAV4...there is a massive difference between drivability at night in favor of the Corolla, despite the fact that the Corolla uses 9012 low beam bulbs instead of 9011 low beam bulbs. The difference is that the RAV4 has projector optics while the Corolla has reflectors...

Click to expand...

Ordinarily, the Corolla would use the HB4 (9006), not the HIR2 (9012). It does not use the HB3 (9005) for the low beam. The RAV4 uses the HB3 (9005) for the low beam.

There are examples of good and bad, or excellent and merely decent, lamps of all technologies (although making an acetylene lamp to meet modern standards might be a particularly challenging task). It's much more likely that a particular projector lamp outperforms a particular reflector lamp, or this particular reflector lamp outperforms that specific LED headlamp than to say all technology X lamps are better than all technology Y lamps.

-Virgil- said:

For openers, 12 feet is not a long enough distance to be testing at.

Click to expand...

C'mon, everyone knows that is the correct distance to park from your garage door when you take your beamshot pics. How else is anyone on ToyotaFanBoys.com going to know how cool your "LED kit" is?

-Virgil- said:

Different headlamps perform very differently. For the purpose of discussion, let's assume his peak intensity is at or near one of the low beam test points. At 0.6D/1.3R, the requirement is 10,000 candela minimum, with no maximum (at 12 feet that would be 746 lux min/no max, assuming a measurement at that distance would be valid). At 1.5D/2R, the requirement is 15,000 candela minimum, with no maximum (theoretically 1120 lux minimum/no maximum).

[Did you get that book yet?]

Click to expand...

There's no maximum for low beams? I know for high beams the maximum intensity anywhere is 75,000 candela, correct?

Not yet! Will do later though.

Ls400 said:

There's no maximum for low beams?

Click to expand...

There are low beam test points with maximum allowable intensities, yes.

I know for high beams the maximum intensity anywhere is 75,000 candela, correct?

Click to expand...

No, that's the maximum intensity at H/V.

Not yet! Will do later though.

Click to expand...

I assume this means the book? Yeah, you'll want to.

-Virgil- said:

Whatever that site owner's intentions might be, the list of reasons why his "tests" aren't actually tests/aren't producing any kind of useful data is so long, it's difficult to choose where to start. For openers, 12 feet is not a long enough distance to be testing at. Who knows what kind of meter he might have been using, but it doesn't really matter; at that short of a distance the meter readings aren't going to translate to any useful information, and they aren't even reliable for comparison with other readings taken at the same distance. We have no idea how stable his power supply is during each run and across the various runs. He consistently misspells certain easy-to-spell bulb brands, which doesn't really impact the numbers but throws a cloud of doubt over his general credibility (WTF? The box and the bulbs are right in front of him!) His use of different headlamps for low beam v. high beam is strange, and his "best for low beam!" and "best for high beam!" claims are just plain misleading. His use of color temperature as a criterion is not compatible with his claim to be testing bulb performance.

His site is full of mindless copypasta; he's talking about "HID kits" on the HID page, "LED kits" on the LED page and..."halogen kits" on the halogen page. Halogen kits??? His "review" videos are just some schlub (I guess it's him) sitting there babbling at the camera, interspersed with completely useless footage of headlight beams on a road. I think Alaric is probably right; that site's main purpose seems to be to drive traffic to his YouTube channel and get clicks on his Amazon Affiliate links.



Different headlamps perform very differently. For the purpose of discussion, let's assume his peak intensity is at or near one of the low beam test points. At 0.6D/1.3R, the requirement is 10,000 candela minimum, with no maximum (at 12 feet that would be 746 lux min/no max, assuming a measurement at that distance would be valid). At 1.5D/2R, the requirement is 15,000 candela minimum, with no maximum (theoretically 1120 lux minimum/no maximum).

[Did you get that book yet?]

Click to expand...

[FONT=&quot]Hi Virgil & everybody!

First I want to say that I really appreciate the criticism, it only helps one grow stronger at what they do!
Now, the main goal of my website and YouTube channel are to provide USEFUL test results to people in need. I'll explain more in detail.

Let me provide some background of my company and myself. I first started BulbFacts (back in August of last year) due to frustrations behind many claims from manufacturers (Philips, OSRAM, etc. and 3rd party of course) claiming big upgrades. After buying 20+ Halogen kits, I finally found a decent bulb that worked well, and was a whiter color that I desired. Yes I am very picky when it comes to automotive lighting. I wanted to share the results after all of my testing, so I threw together a web page for people to use as they wish (see wayback machine for an old version). I myself work a full time job, and do BulbFacts on the side as a hobby, and I really love doing it the best that I can.

Fast forward to today. Since then, I have had a lot of people reach out requesting I test more products, including LED and HID. I wasn't a huge fan of LED myself, knew they scatter light, blind oncoming drivers, etc. So, I decided to buy 10 of the most popular LED kits that sell on Amazon (all that I could afford), and see how they did. I found a good few kits (at the time Lightening Dark and Beamtech) that actually performed pretty well, so I wanted to get the word out and created the video there, and YouTube is a free platform to do so.

Since then I have spent a lot of money (thousands) on buying these products, converting my basement into a test lab, test meters, video equipment, and yes, I signed up for Amazon Affiliates to try and recoup some of the money back. I have accomplished that, and make a little extra which is great, so I can continue to buy new products to test for people as more requests come in. I'm not getting rich though, and I'm ok with that! I don't run ads all over my website as I think they distract people from the actual facts, but I have enabled YouTube ads to bring in some extra money. Keep in mind that I HAVE had a number of companies offer paid sponsorships, all of which I declined. I am keeping BulbFacts true and unbiased. I even like to decline free products typically and just buy it myself and thank them for bringing it to my attention. Most get upset by the results, but it is what it is.

I do understand the suspicion of companies out to make a buck and know there are many out there, but that I not my intention at all, and I try my best to buy the best test equipment and meters, and spend a lot of time researching the best ways to improve my testing, as well as a LOT of time in my basement testing and testing. My wife wonders about me sometimes. . I know it's not perfect, but I always try to improve as I go.
12ft isn't the correct distance to test, and my plan is to re-test EVERYTHING at 25' early next year (or ASAP), including the Halogen, LED, and HID products, as well as 50ft, 75ft, etc. as well later down the line. I just didn't have the space to test at that distance unfortunately at the time. I have made a lot of adjustments since then though due to thread posts like this, as well as people contacting me via the website, and YouTube comments. I delete NO comments, and always thank people even though they can be pretty harsh sometimes, I still value the opinions very much, and it always gets me thinking.

In closing, if you guys are willing, let me know your thoughts on my testing, and I'd be glad to share 100% how I do things as I want to be 100% transparent. I'd make it worth everybody's time for sure, and I'm always open to testing new brands, as well as different ways to test.

We all know everybody is buying into the LED fad, so why not at least direct them to the ones that work right, and don't blind oncoming traffic? That's my theory anyways. Customers buying LED kits on Amazon shouldn't be fooled by the false '16,000 lumens' claims, as we know they are just plain made up. I am planning to do some videos on better education as well. Headlight Revolution does some great ones now, and I think they really help as many people are confused and just throw whatever LED and even HID into reflector style headlights.

I'm working with a company now to buy more 'premium' or 'high-end' LED and HID products (I call them kits) such as GTR Lighting, Diode Dynamics, etc, as they are very expensive, but I have had a lot of people ask me to test them more. I should have them purchased and tested in the next few months. Again, if you guys have any more criticism, ideas, requests, etc. don't hesitate to let me know!

Thanks everybody for your time!
PS: Sorry for the typos on the website, I type fast and forget to spellcheck I won't lie. I do go back though when I get time to re-read through things and make corrections [/FONT]

bulbfacts said:

In closing, if you guys are willing, let me know your thoughts on my testing, and I'd be glad to share 100% how I do things as I want to be 100% transparent. I'd make it worth everybody's time for sure, and I'm always open to testing new brands, as well as different ways to test.

Click to expand...

YOU are not qualified, don't have remotely the right equipment, knowledge, or range of test headlights to even begin to accomplish what you claim to want to do. NADA, zip, diddly squat.

1) Contract with a testing lab, or spend the $50 - 100,000 on a proper test set up and the training to use it.
2) Buy 3 samples, of the 20 most popular vehicle headlights from the last 10 years.

Before posting even one iota of data ... test every unit in a quality lab setting with 3 samples, on the 100 or so vehicles that it needs to be tested on.

.... you ... are not ready.

MeMeMe said:

YOU are not qualified, don't have remotely the right equipment, knowledge, or range of test headlights to even begin to accomplish what you claim to want to do. NADA, zip, diddly squat.

1) Contract with a testing lab, or spend the $50 - 100,000 on a proper test set up and the training to use it.
2) Buy 3 samples, of the 20 most popular vehicle headlights from the last 10 years.

Before posting even one iota of data ... test every unit in a quality lab setting with 3 samples, on the 100 or so vehicles that it needs to be tested on.

.... you ... are not ready.

Click to expand...

Hi MeMeMe, thanks for the reply!

My goal is to make it to the point where I can spend 100k, but I wanted to start somewhere and help others out there what the results I have.

I am currently using 3 different headlights, expanding to 8 or so next year, and will continue to add from there. I use two samples for each product currently, and certainly agree that the most samples should be used. Again, I hope to make it to a large and expensive lab one day, but I don't have that kind of money today and do my best. I do believe my current test results are helpful, based on a lot of feedback from the community. I do understand every car is different, but it is impossible to test every product on every single vehicle (myself), so my site is considered a general comparison, comparing products to each other in the same setting.

I appreciate the feedback very much, and hope everybody can understand the point in which I started BulbFacts, although not perfect by some standards, I'm simply trying to steer people in the right direction to avoid simply bad products, and help choose between the 'good' products. I am always expanding and improving the tests in which I perform, and will not stop until I reach 'perfection' for the community based on their needs.

If you guys are interested, I'll report back with my changes and updates/improvements that I have done, but only if desired of course. Just let me know.
Thanks!

bulbfacts said:

[FONT="]Let me provide some background of my company and myself. I first started BulbFacts (back in August of last year) due to frustrations behind many claims from manufacturers (Philips, OSRAM, etc. and 3rd party of course) claiming big upgrades. After buying 20+ Halogen kits

Click to expand...

"Halogen kits"...?

I decided to buy 10 of the most popular LED kits that sell on Amazon (all that I could afford), and see how they did. I found a good few kits that actually performed pretty well

Click to expand...

No, you really didn't. Clearly you think you did, but just as clearly you do not have the equipment or the knowledge to be able to make that assessment. You found a few kits you think you like -- which is not the same thing as finding kits that "perform pretty well".

I HAVE had a number of companies offer paid sponsorships, all of which I declined. I am keeping BulbFacts true and unbiased.

Click to expand...

That's a good intent, kudos for that, but you're really not doing anything like the "tests" you believe you are doing. The methods and means you're using are adequate to give some demonstration of how bad a product is, but can't come close to judging a product as adequately safe, and can't provide the kinds of rankings and ratings you're putting out. I'ms orry, I know this is harsh, but your "data" is largely garbage -- no matter how neatly organized its presentation might be.

I try my best to buy the best test equipment and meters, and spend a lot of time researching the best ways to improve my testing, as well as a LOT of time in my basement testing and testing. My wife wonders about me sometimes. . I know it's not perfect, but I always try to improve as I go.

Click to expand...

Again, that's good intent, and that ought to be recognized and appreciated. But you are still giving out bogus "information".

In closing, if you guys are willing, let me know your thoughts on my testing, and I'd be glad to share 100% how I do things as I want to be 100% transparent. I'd make it worth everybody's time for sure, and I'm always open to testing new brands, as well as different ways to test.

Click to expand...

You come up with results that are not in accordance with physical reality. You make by-eye evaluations and call them test results. And then you recommend products based on these "results". The consequences are large and serious, because we are talking about life-safety equipment here: you are giving out bad safety advice to people who don't know any better than to trust your "results". That is irresponsible at best, and I think that's why you're getting slammed in this forum. What's needed is a total re-do, starting with a lot more knowledge about how headlamps need to work in order to provide adequate safety. This isn't like "Oh, you might want to tweak this and adjust that about your tests".

We all know everybody is buying into the LED fad, so why not at least direct them to the ones that work right, and don't blind oncoming traffic?

Click to expand...

But you're not doing that. You're not doing anything like that. You only think you are. I ask you: what good does that serve?

This is not to say that it's impossible to do home-shop testing and come up with results that are at least somewhat useful. It can be done. Spend some time looking at this Youtube channel. It's in Russian, so unless you speak it, turn the sound off and the auto-translate subtitles on and you should be able to get the general idea of most of the videos. Also go look at this guy's stuff. It's also in Russian, so read it using the Google Chrome browser with translation turned on. (No, I don't know why these two Russians have got their groove on about headlamps, but they seem to).

Hey Virgil, thanks for the reply.

It seems like the standard you guys have here is very high compared to what my standards are in order to give recommendations. I do believe I am being very transparent and honest on my website.

To your point, even if the tests are proving what the 'bad' products are, then people will stay away from them right? The 'best' products are always based on one's opinion and what they consider important. I do believe my tests are helpful. There are thousands of different model vehicles, thousands of different lighting products, and it is simply impossible for one company to test everything together. I again do have plan to continue to improve my testing procedures by adding more sizes and headlight styles, and the results will only get better as I grow. I hope that is understandable.

I am not going by eye, I am using multiple Lux meters, as well as a integrating sphere in order to measure the values. Everything else from there is simply subjective though (headlight, bulb, distance, angle, etc), so I can understand how one may or may not consider certain results of value, when they have a different vehicle, drive in different conditions, and so on.

I do believe I am helping to keep people safe by providing a 'better' product overall, as you said they are avoiding the 'bad' ones. That is a win in my book.

I am very familiar with the Test Lab YouTube channel, and will only keep improving Bulb Facts to provide the best recommendations possible.

Thank you again for your input and time, I truly do appreciate the feedback! Take care.

bulbfacts said:

It seems like the standard you guys have here is very high compared to what my standards are in order to give recommendations.

Click to expand...

The appropriate words here are realistic standards (here) versus imaginary ones (yours).

I do believe I am being very transparent and honest on my website. I do believe my tests are helpful.

Click to expand...

I believe there's a briefcase containing $500,000 in legitimate bills in my downstairs bathroom, behind the toilet. Believing something doesn't make it true, no matter how convenient it would be if it were true, no matter how much we want to believe it's true, no matter how hard we believe, no matter how many times we say "I believe".

To your point, even if the tests are proving what the 'bad' products are, then people will stay away from them right?

Click to expand...

Sorry, no sale. You're actively recommending products that aren't safe and don't do what you claim they do. You can't make it all better by saying "But at least I'm steering them away from the really REALLY really bad ones, and I believe I'm being honest and transparent".

The 'best' products are always based on one's opinion

Click to expand...

No. The best products are the ones that really, actually work the best. Another valid way of thinking about it, in this context, is that the best products are the ones that, at the very least, don't degrade the safety of the car they're installed in. You very obviously don't have the knowledge or tools to figure out which ones those are. Resorting to "Well, it's always going to be a personal opinion" is not an honest position to take.

There are thousands of different model vehicles, thousands of different lighting products, and it is simply impossible for one company to test everything together.

Click to expand...

None of that is responsive to any of the legitimate critique your "tests" have garnered.

I again do have plan to continue to improve my testing procedures by adding more sizes and headlight styles,

Click to expand...

This is also not responsive to any of the problems.

and the results will only get better as I grow.

Click to expand...

Obviously you believe that.

I am not going by eye

Click to expand...

Yeah, you really are. There is no other way to arrive at "You should buy this LED kit because it doesn't glare".

Everything else from there is simply subjective though

Click to expand...

In other words...going by eye. Thank you.

I can understand how one may or may not consider certain results of value, when they have a different vehicle, drive in different conditions, and so on.

Click to expand...

Nope. Your "results" are not data, are not realistic, and are not useful. Regardless of what kind of car anyone might drive, or where or when they might drive it. Sorry that goes against your "beliefs", but there it is.

I do believe I am helping to keep people safe

Click to expand...

Here we go again with your beliefs.

by providing a 'better' product overall, as you said they are avoiding the 'bad' ones.

Click to expand...

I never said they're avoiding the bad ones. YOU said that. How am I supposed to put any faith in your professed honesty when you insist on putting words in other people's mouths?

I'm pretty sure we're done here.