Below is a text version of the webinar titled "Home Energy Score: Program Update for Interested Stakeholders," originally presented on July 23, 2012. Watch the webinar.
Thank you very much and good afternoon, or good morning to those of you on the west coast. Thanks for calling in and viewing today. As Cortney said, I am going to spend, I don't know maybe a half an hour going through our program update and we will take questions, so maybe we can have 15 minutes for questions and the last 15 minutes I can go through some slides that are specifically for those types of organizations who might be interested in becoming a Home Energy Score Partner. OK, so next slide.
Today I am going to be giving you an update on what we have been doing over the last year and a half. Basically in November of 2010, Vice President Biden announced we were going to be pilot testing the Home Energy Score program. The reason we had even put this together was that, as the objectives are listed here, we were looking to figure out how to strengthen the home energy improvement market and basically the private market for middle class and other homeowners who are not eligible for weatherization improvements through the weatherization programs in their states. We wanted to make sure that people had easy access to credible information about their homes' energy performance, how it compares to others in their area, and what they can do to actually make that home more efficient. We recognize that there was a lot of great work already going on out there but really there was a lot of huge potential that happened to, in terms of the home energy improvement efforts. We think what we've come up with fits well with those programs that are out there and hopefully can be a good impetus for starting other efforts as well. OK, next slide.
What we've come up with is really something that is comparable to sort of a miles-per-gallon type of rating that you see on cars you are going to buy. And it pretty much allows a comparison between any kind of home regardless of where you are in the country. It does account for climate and many of you are familiar with this so I am going to go through this pretty quickly and then just tell you what the results were for our pilot phase. We think it provides a low cost service that pretty much a qualified assessor can go into your house and within an hour deliver the Home Energy Score. If they are already going into the home for another reason, like to do a home inspection or another kind of audit, what we found during the pilots was that it took as little as, or approximately 15 minutes extra, to add onto that assessment and provide the Home Energy Score. We did a tremendous amount of improving the program, both with the Home Energy Scoring Tool but also the materials, what we needed to do in terms of training and testing of assessors. We got a great amount of feedback, very useful feedback from our Pilots around the country in 2011 and just last month we began our first phase of the real implementation of the program around the country with utilities, non-profits and others nationwide. Next slide.
Again getting back to sort of the reason that we did all of this was to provide credible information for homeowners and we want to make it information that will obviously motivate them to make energy improvements, either in the home they are currently in or to homes they are about to buy or sell, but more likely the ones they are about to buy. We think that, what we learned prior to developing the Home Energy Score was that people really appreciate straightforward, simple information. They want to be able to grasp the information quickly and not be bothered by a lot of detail, although they might want additional detail as background. It is helpful to capture their attention with either a dollar amount or simple scale, like the 1-10 scale, that they can readily understand and relate to. We know from social science research that people are very much influenced by their peers. In fact, that can be a greater influence on them than even how much money they are going to save. Although people can be motivated by costs and savings, they are obviously motivated by other things including how they relate to those around them. They do, of course, care about the bottom line. We made that a prominent piece of information in the Score and what we found also in our focus groups was that people generally like to see a government seal on the information provided. It gave it an additional level of credibility.
So the Home Energy Score hasn't changed much, in terms of how it is working, but the type of information and how we generate it has changed, I'd say, pretty significantly since the pilots. So, as I said, it is a standardized method for quickly assessing a home's major energy systems, including the envelope. So, it looks at your roof, foundation, your home's windows, walls, insulation and then the major energy systems are going to be considered, the heating, cooling and hot water for that home. And, as I said as well, it allows this comparison between homes regardless of where they are in the United States. We only allow qualified assessors to provide it and at this point, they need to be working under a partnership agreement with the Dept. of Energy and I will go more through that in more detail later. A homeowner, when they get their home scored, they will get the Score itself, which is an asset score meaning that there are certain standard assumptions that we make about how people are behaving, which again allows the more apples-to-apples type of comparison. So, we are not looking at how many people are living in the home or how they set their thermostats. We are actually standardizing that across all homes. We give them home facts, which is really a list of the data that was collected about their home and used to score it, and we also give them recommendations for how they can improve the efficiency of their home. Whether they can do things right now or what kinds of savings they will generate from replacing inefficient equipment with ENERGY STAR equipment.
This is the version of the label or Score we used in 2011, and you will see that there are some changes that have been made when I will show you the more recent version. I think it has been updated in a good way that makes it even easier and less confusing to look at. There is just less information; it is a cleaner looking type of picture.
Here's a list, or a picture of the folks that were our terrific Partners in 2011. PECI in Utah did sort of a mini pilot, which is why they are pictured a little differently. They scored homes but they didn't actually give the Scores to homeowners. Their information was used slightly differently than the others.
So for the other nine pilots that did the full fledge pilot, collectively they scored more than 1,000 homes. We had 31 qualified assessors in those 9 pilots going around and scoring them. And what we found was, in general homes really filled the gamut, the range. Although you don't see any 10s here, you do see a pretty good distribution from 1-9. The reason that you see quite a few 1s there; actually in Illinois we didn't have a very good distribution. In Chicago, they were looking at primarily, really quite old homes. The median age was 1924, so what we found in those homes that were old and had not been updated, they weren't scoring particularly well, so that's why there are a lot of 1s, in that case this red line, the red bar.
So what we were trying to figure out with the Pilots is not just can we go out there and score homes, but were we meeting the original objectives of what we were trying to set out to do with the program. And when we first established the program, we had these sorts of underlying principles that were guiding us. The first was that we develop a Score that was credible, reliable and replicable. What we found was in the Pilots, in not a huge amount of cases, but in a handful of cases we had more than one assessor go through the same home and rescore it. And we compared those Scores and what we found was, even though you had different assessors going through the same home, in general, in all but one case, they scored within one point of each other. So even though there was some judgment obviously being made by the assessors, if they were qualified they were generally coming up with a pretty consistent Score. We also did a lot of analysis including, in addition to, I should say in addition to the Pilot homes, we did a tremendous amount of modeling and runs with the Scoring Tool and with new versions of the Scoring Tool to come up with a better understanding of how much variability you could have, in terms of uncertainty about a home and how that would affect the Score. And what we found was, that 90% of the time, even if you're not completely certain about how much insulation is in the wall or whether, how tight the house is, you would come within one point of the expected Score. Tomorrow, there is another webinar we are doing that goes into detail about the analysis we did here so I am not going to go into that now, but if you are interested please call in tomorrow and we are going to go through all of the analyses we did and give you more information about how we came to these conclusions. We also felt that the Scoring Tool is sufficiently accurate in estimating energy use compared to actual energy use. What we meant there was again, in addition to the Pilots, we collected data for about 450 homes in four different states in the country, we also had in addition to their home characteristics data, we had their utility bills. And when we basically plotted what the Scoring Tool was doing in terms of estimating energy use for those homes, versus other tools, we were certainly comparable, and in some cases better at predicting energy use.
Hannah Wood: Joan, do you mind if we just take a pause to see if anyone had questions on the previous two slides?
Joan, there is one question, and it says, "Will the Dept. of Energy be working to eventually get the Homes Energy Score on the numerous MLS's? Is this a goal at this time?
That's a great question. It is not a goal at this time, although there are some Multiple Listing Services that I believe are going to incorporate the Home Energy Score as well as other types of energy efficiency distinctions, or however we want to call it, information on the MLS's. There are 850 MLS's I believe, or more than that in the country, we are not pursuing that at this time, so it is really being done on an individual basis. If an MLS is interested, we will certainly work with them to provide the information about how we score homes and whether they want to put it on there or not is really their decision. In this next phase, we are trying to do additional analysis to what we are doing here, what we've done already, and I think that if we continue to feel good about how capable we are at scoring homes, and providing that kind of information to homeowners or homebuyers, then we might pursue that at a later date, but right now we are not actively doing it, although there is a lot of working going on right now and we are kind of keeping apprised of it because there are a lot of developments going on in the real estate world.
Joan, I actually have a couple more questions to ask, if you don't mind.
Another question is how many homes had more than one assessor go through the home and where were those homes? Do you feel that was a representative sample?
I do not think it was a representative sample in the sense that, it was, I don't remember exactly. Erik, I think you know on the phone, you may remember how many we retested. I believe it was only about 25 that were scored twice and we are going to be doing that further in the next phase. There were three different locations that did that, I believe it was Texas, Indiana and South Carolina, but that is just from memory and I don't remember off-hand if that is right. I believe those were the three places that had rescoring going on. So they weren't, they were certainly diverse in the sense they were coming from, in some places, very different climates and in some cases, slightly different climates, and they were different kinds of homes but it certainly wasn't enough to sort of finish the discussion in that area and I will talk more about how we will do that in the next phase.
The next question is how is the recommended scope of work generated? Is this by the software or through the qualified assessor?
OK, another great question, and I will show you in a little bit later, we do have the three parts of the Score. There is the Score itself, the Home Facts and the Recommendations. That third piece, the recommendations are generated by the software tool, but if a Qualified Assessor elects to provide their own recommendations and not ours, they are free to do so. We do want the homeowner to get some type of recommendations on energy efficiency, so if they are going through the home and not doing anything else and they want to use our recommendations, they should do that, if they don't, they can use their own.
OK, and Joan, if I can just ask one more question. In the comparison phase, was the Home Energy Score compared to SIMPLE or EPS? If so, how did it compare?
Again, great question. If you're not going to be able to participate tomorrow, if you could note that, I will figure out a way to get back to you. But, just quickly, because I don't want to go into a lot of detail on the analysis since we are doing that tomorrow, we did a comparison of SIMPLE, REMRATE and the Home Energy Scoring Tool and they all performed very similarly, I would say, although I actually think Home Energy Score performed better, depending on how you want to look at it and I can show those different ways that we measured it tomorrow, including a plot of the graph. Now, none of those tools are trying to predict energy use because we did not have the demographics for how the individual homeowners were using those homes so we didn't have that information to calibrate the predications, but at the same time we wanted to make sure that we were basically within a reasonable range. And what we found was that the Home Energy Score, more than the others, I don't remember off-hand, but it was a larger percentage of the time, we scored within, we predicted within 25% of the actual energy use than any of the other tools. SIMPLE out performed us a bit in terms of [predicting] within 50% of the energy use but it was pretty close, actually. OK, is that it, for now?
OK, great. OK, so the next guiding principle was that we wanted to have a transparent and easy-to-understand type of information that we could provide to homeowners and we did ask homeowners to respond to some questions during the pilot phase, and this is just one piece of that information, but basically we found that people understood the Score. Whether or not that motivated them to do something is not what we studied at that point but we will be doing that in the next phase. But they certainly did understand it. It is something people can relate to, you know a 10-point quiz—they kind of get it. We were pleased to see that the Score can be done pretty quickly by a qualified person, and again, if you're going in there and already doing some other type of data collection in the home, it generally took about 15 minutes, in some cases less, to collect the information and we actually now have an application programming interface available, which I can go through a little bit more in detail later. But that allows you to use other software tools. So if your program is using a different software tool, like EPS or TREAT or REMRATE, or whatever, you can then use the API to allow you to not have to double enter the data, and I can go into that later, if people are interested in more information on that. So that should actually reduce the amount of time even further because during the Pilot Phase in 2011, you actually did have to go and enter the data. We didn't have the API available. We also did a sensitivity analysis with blower door information. About 650 of the 1000+ homes that were scored during the pilot did have blower door tests done and what we found was that it did not have a significant effect on the Score and therefore it will not be required. Again, I will go through more detail on this analysis tomorrow.
This is what the Home Energy Score's first page looks like now. It is not that different, but I think we have cleaned it up some and gotten rid of some information we thought was superfluous or confusing.
The next page is the Home Energy Facts and this provides a list of the information the assessor uses, as well as you will see, here is where we provided if the homeowner is interested in actually what our estimated energy was, not that most homeowners really care about this level of detail, but if they are, this is what the Scoring Tool predicted in terms of the energy use for that home given standard operating assumptions.
Slide 13 and 14:
Here is the second and third pages of the facts, so you can see it does through air tightness, roof and attic. These are basically all of the inputs the assessor has to include in order to score a home.
Finally the third page, or the third part of the Score packet is this recommendations piece. It does give an annual estimated savings for making that improvement and they are broken down into "Repair Now" and "Replace Later," so if you don't want to replace your water heater before, you're just not ready to do it yet but you're going to do it later, then you could save, in this case, $50 a year. We did improve the way we were looking at carbon footprints. Before, we use to have on this page, like CO2, pounds of CO2 reduced, which was kind of meaningless to most people, so we did try to make that easier to understand with this little picture.
So looking forward now, in this next phase, we actually have, I believe as of today, 22 organizations that have signed on to be Home Energy Score Partners. You can see them listed here. Most of them are working in one locality or within a state. We have a couple of programs, ASHI, which is the American Society of Home Inspectors, and InterNACHI, which is the International Association of Certified Home Inspectors (I always have to remember all of the words). ASHI to begin with, InterNACHI probably at a later date, will be working across the country with qualified assessors, so we are excited about that possibility to offer the Score in additional locations, not just those shown by the dots. 106 assessors have gone through or are in the process of taking our training and our test, and 34 of them have now passed both parts of the test and can score homes. We did make the test much more rigorous than it was when we were doing the pilots. We also really beefed up the training, so that there was a refresher on building science, as well as just more information about the Home Energy Score and how you describe it to homeowners, if you're asked questions and all of that. So, I think we've done a good job of really improving the material we have for Partners that are more officially providing this around the country. We are still recruiting Partners, so if you are interested you should stay on for the last part of this conversation today and hear what it takes to become a Partner.
In terms of the Scoring Tool, again only qualified assessors can access the tool so of the 106 that are currently taking training, only the ones that have passed the two parts of the test can actually access the tool. You can only use it for single-family homes. You can use in for townhomes but there is some specific instructions you have to follow in order to address the shared walls. The tool is only available online. You can't just download it on to a computer and then just use it, but you can use the Application Programming Interface, which is again, that is still using the tool online but it is allowing a seamless link from other software to the tool itself.
In terms of next steps, I think we are really focused on improving our program administration at this point, as we are growing, we want to make sure that the services we are providing to the Partners are clear and effective and we are not wasting their time. Obviously, it has to be something that is attractive to Partners or they are not going to want to deliver the Score. We've been improving materials and I think we can still do better there, materials for assessors as well as homeowners. So we are recruiting additional Partners and we are looking for some that have different types of implementation plans. We do require all of the Partners to create implementation plans but many of them are doing it right now as part of an existing energy efficiency program. The exception to that is, or one of the exceptions to that is the home inspectors who are doing it differently. So, what we are trying to prove out is that we can do this in a way that can still maintain quality, even if we grow, or as we grow. So, that is something we are looking at and trying to figure out in the next year. There are a lot of details I did not mention in terms of the improvements we made to the Scoring Tool and to the program overall. One is that we went from 19 weather stations and climate zones, I should say, 19 climate zones in the pilot version and now we are using over 240 weather stations, so there is a different 1-10, or a unique 1-10 scale depending on which weather station you are attached to, of the 245. There are actually 1000 weather stations that we now use, but we are the 245 that are part of the CMYII. Since we are going to such a larger number of discreet weather stations, we'd like to do a better job of analyzing data that is coming out of these specific climates, each of the individual climates. Obviously we are not going to cover all of those individual areas for our current Partners but we'd like to cover as many as possible and collect data to make sure we are addressing homes in different weather zones, in different climates accurately. We will continue to make additional refinements to the Scoring Tool and our plan is to update it, to release an updated version once a year. Hopefully in January of every year, so you can actually automatically rescore your home with the existing data that you had already used. If you wanted to put in new data you'd have to have an assessor come back to the house to do that with the new version of the Scoring Tool. We are using the API now and we are testing software programs that are licensing the API to make sure that if you put the data into some other software program that is using the API, you will get the same output, in terms of a Score, as if you used the Scoring Tool directly. We are very excited to look at some kinds of control and test the samples where we look at whether the Home Energy Score, how it affects the homeowners, in terms of motivating their investments in energy improvements. Obviously, our goal is to get people to invest effectively in energy efficiency improvements, so we are hoping that it positive, but, if it is not, then we are going to look at how it can make it more of a driver. We are also Pilot testing other types of requirements for assessors. Currently, you either have to be a BPI Building Analyst or a HERS Rater to qualify to then go ahead and take the training and testing to become an assessor and we are looking at whether other types of qualifications could also apply and meet our requirements. And finally, we are also going to be looking at the effectiveness of what we are proposing in terms of quality assurance. Right now, every Partner is required to rescore 5 percent of the homes that they score with another Qualified Assessor or third party, and we are going to see if there are other ways we can implement that to make it not so onerous but at the same time, something that we can have confidence in.
So with that, I am going to take questions, and the next part as I said is about partnering and I'd be glad to answer as many questions as I can for the next 15 minutes or so and then I will spend the last fifteen minutes going through that information.
OK, Joan, we have quite a few questions for you, so the first one is, "Can you explain why, if the blower door test doesn't have to be used, why it is included as an entry point in the data collection sheet?"
Sure. Basically if you are already going through the home and doing the blower door test, you can just input that CFM number, but if you're not, you can just put in a qualitative assessment of whether you think the home is sealed or not. And, the reason we allow that is that clearly if you are doing a blower door test it is going to be more accurate than if you're not, but in terms of assessing the how tight a homes is. At the same time, we didn't find it made that much difference, and again I will go through the results tomorrow, but it didn't make enough of a difference to impose that kind of a requirement. I hope that is clear. So, basically, if you have better information, we want you to use it, or allow you to use it because otherwise it just frustrates people, but if you don't have the information, you should be fine, you might as well. Because basically what the tool does is it looks at a few different factors that the assessors has put in, including the vintage of the home, where it is located and other information and then uses an algorithm or a lookup table, actually it is an algorithm, to basically create a blower door number for that home. So, it is going to in affect create one for you, if you don't put one in. OK, go ahead. Next question.
Is the Score set up to included estimated energy costs in the estimated energy consumption part of the Home Facts?
Can you repeat that? I don't think I heard the question. Cortney Krauss: Is the Score set up to include estimated energy costs in the estimated energy consumption part of the Home Facts?
Joan, do you want to back to the…?
Yeah. Let me see. So, here's where estimated energy costs…OK, well the cost is implied here in the savings numbers.
Basically, when you go to the first page, this is what, for this home it is estimated that it is going to save $3900 in 10 years, over 10 years if it makes all of the improvements that were recommending. And that $3900 is based on average utility rates for that state. So, in this case, Arkansas, but, we don't have it by utility by utility, but we do use the average cost of whatever fuel they're using and then it is divided up based on which fuel they are using for which part of their home. So, when you go to the Home Energy Facts, this would basically be converted. The electricity component would be based on the average, you know, electricity cost in that state.
The gas would be based on the average gas cost in that state and that would be used to convert these savings number of how much energy is being saved into a bill savings, which is using that state's rates, as I mentioned. Did I answer that question? I think I answered the first part. I'm not sure I answered the second part.
I think that you did answer the question. If Joan did not answer your question in full, please raise your hand or submit another clarifying question. So, our next question is there are no Partners in Arizona. Can we be a qualified assessor and work on homes in Arizona, even if there is not a Partner in Arizona?
Well, if you're a home inspector and you're a member of ASHI, then you might qualify to become a qualified assessor under their partnership. Right now we are not working with individual qualified assessors. That is something we would like to move towards at some point, possibly, but it is not feasible at this time to kind of maintain the quality assurance component we were talking about, in terms of is there another way we can do this to ensure we are going to have high quality. I am sure there are great people out there that are ready to score homes, but there is no way for us to kind of monitor what they are doing from here and we are trying to figure out if there is another way to do that. That said, there is a group of utilities and others in Arizona that have proposed to become Partners and I think they have actually signed a Partnership Agreement, I am not sure they were seen on our map up there, but they are one of the 22, so I don't know if you're somehow affiliated with the utilities there or with the Foundation for Senior Living, I believe it's called, but they were all collectively working towards being becoming Partners. They weren't quite ready yet, so that might be why they are not one there. There are some folks that have signed on to be Partners but they are not actually starting implementation until a little bit later. I think they may not actually start until 2013.
OK. We have a question from Richard Faesy, so Richard, I am going to go ahead and unmute your line. Richard?
My question was whether, and I think Joan answered it earlier, tomorrow's presentation would be made available to those who can't make it and I think you said that it would be.
Yeah, and I should ask Cortney. Cortney, are we taping that tomorrow?
We are. We are going to record, so we will go ahead and post that online, or you can email firstname.lastname@example.org with your interest and we will be guaranteed to get it to you that way. OK, we have a question Therese Crayhen. Therese, I am not sure if I am saying your last name correctly, but I will go ahead and put you on the line. Therese? OK, Therese, it looks like you also submitted a question, so I am assuming this is the same one. Once scored, do the results become public? If not, how is the information compared to other homes in the area?
OK, another good question. The information at this point does not become public. It is held by the qualified assessor, the Partner and by the homeowner, or whoever was requesting that the home be scored. So, that's where it is retained and it could be made public later on if people choose to do that. There are allowances for that but that's not how it is going to be used at this point. In terms of comparing to other homes, you're not comparing, oh, I think I maybe I didn't understand what you were saying and now I do. But if you were going and buying a house and wanted to see how different homes compare that would have to be something that would come later. We obviously have to get homes scored before they can start to be compared to one another. The way that the 10-point Score allows you to compare is just to give you a sense at this point, if you're a 3, then you're not doing so great. If you're an 8, you're doing pretty well. So it gives you a sense where you fall on a scale. But you're right, at this point, if you don't have information about other homes, you can't see how one home compares to another. I expect that will change over time as there is more interest in, whether it is this or others, in information in the MLS's.
OK, will the tool be available as a Smart Phone app?
Great. One of our Partners in the Pilot phase, it was the Center for Energy and Environment in Minnesota, they actually did develop an app, which I believe can be used on a Smart Phone or an iPad, or any of those, and they are, I don't really want to speak for them, but I believe they are making that available to any of our initial Partners for free. So they are beta testing it right now and they did a demonstration and we help them set up a webinar, or they set up a webinar, and we sent it out to folks. If you're interested just email us to put you in touch with them. I don't have their information off hand but we can certainly make it available to you and you can contact us about that app. But that is the whole point of using the API, is that I think others will do a much better job than we do, in terms of making this easy to use and having good user interfaces. That's not our strength.
We have a question from Zachary Popkin. Zachary, I am going to go ahead and put you on the line. Zachary?
Yes, can you hear me?
Joan Glickman and Cortney Krauss:
Can you hear me, yes? I don't really have a question, it is more of a comment, but one of the early struggles I think we've had early on here is being able to effectively include Home Energy Score in our existing marketing materials for our existing energy efficiency program. So I know, I think I've actually asked this at a previous webinar, if there is going to be marketing materials coming along with Home Energy Score and I was told that they are going to be coming but I just wanted to reiterate that I think that, even something as simple as a logo that we can include on our marketing materials or even an emblem that Qualified Assessors can place on their websites or business cards when they become certified. I think even small things like this would be a big boost to getting Home Energy Score adopted on a widespread scale.
Great, thank you and I did hear that, I think maybe from you and others at the conference a couple of weeks back from our Partners and I did bring it back to our team and they are looking at what we can do in that area. I think there is always a tension between wanting to go out there and really market things but still as a government agency, we are a little bit cautious and we are careful about making sure everything is working properly before we go at full speed ahead with this, so that's probably been our reluctance of doing too much marketing, but I recognize that people obviously need to market it in order to sell it, so we are working on that.
OK, another question is, is the Home Energy Score designed to be an assessment grade audit or is it strictly for homeowner education?
It is strictly for homeowner education. It is not an investment grade audit. You know, that said, I think different financing groups look at different kinds of information but mostly at this point, I think for homeowners, they are looking at the soundness of your status, in terms of your, they are looking at your FICO score primarily, I think, and your assets and all that. But, what I would say is that the one thing that it does show you, although it is not an investment grade audit, it does show you, you could convert each one of these points on the scale to what is the expected utility bill for that area, that home, for a home. So, a 1 would be using a certain amount of energy per home, I mean per year, I apologize. A 7 would be using a different amount and it is a linear scale here. So, if a Partner wanted to convert this scale into what the predicted cost is for that area, it could do so. There would obviously be some assumptions made about the amount of electricity versus natural gas versus other fuels that you'd have to do, in order to convert it to that type of scale.
OK, you mentioned assessor qualifications are being reconsidered. Are Certified Energy Managers being considered?
I apologize, but I am not sure who certifies Energy Managers. Is that a, I don't know if that a BPI thing. Does anybody on our team know that? So, I guess, unless I hear back from folks, at this point, we are not looking at specific, unless somebody comes to us and says, you know, we have this type of qualification. For instance, RESNET came to us and said they're training folks and their inspectors are training folks to be Home Energy Survey Professionals. So, we're looking at that specific designation and seeing if people who are Home Energy Survey Professionals, certified as that, would be able to score homes effectively and it's too early to say at this point. I think what we are going to end up finding is that people need a certain level of experience, regardless of what their certification is and that may be where we end up. We might end up at a place where we require a certain amount of experience and then we really beef up our test to ensure that people know what they are doing. Right now, I think our test is a lot harder than it used to be. It's 50 questions multiple choice test and there's another part of it where you have to score three homes using the Scoring Tool. We give you some information and you have to do some calculations and input the data. So, it used to be that we just tested people on the Home Energy Score, and now we test them on building science and the Score itself and people have failed. A lot of people have failed, so I know it has gotten a lot harder than it used to be.
And, Joan, the CEM is with the Association of Energy Engineers, but per your point about the experiences, also residential energy efficiency specific experience and not just energy experience.
That is a good point. So, if you have information that you would like up to consider as we think about how to move forward in this area. We relied on the BPI and HERS Rater certifications because they were more widely used than others, but it is something we are considering changing in the future.
OK, and another question. Have you looked at the distribution of Scores in a specific neighborhood? For example, are you seeing enough differentiation in Scores between similarly sized homes in the same neighborhood to suggest that buyers or renters will see a spread of Scores in the homes that they are considering?
I think I would say that is a conditional yes, or a partial yes to answer that question.
Basically, we did in the thousand homes we scored, or that were scored by our Partners, there was a good distribution and I believe, even among homes that were the same size, there was still a good distribution. What we found when we, we also did thousands, definitely thousands of runs with the Scoring Tool where we took zip codes of each of the 240+ weather stations and we varied size, and to some cases within a size we also varied other components about the home, so the heating efficiency and the cooling, you know, and the type of, you know how tight it was and how much insulation and so you can get a lot of variability. It's not that tied to size. Size will definitely make a difference if you are a huge home versus a small home and you were the same in terms of the efficiency components in your home, the smaller is going to score much better than the large, but it is not that every small home is going to score an 8 or an 9, and every big home is going to score a 6, a 5, there is variability. So, I feel pretty good about that. That said we are going to be scoring thousands of more homes in the next year and both actual home but also historic data from some of our Partners, so we will get a much better sense of how this distribution is within different climates and among different vintages of homes and sizes.
OK, so the folks right now how are eligible utilities, state and local entities, other non-profit organizations who can show us that they can actually carry out the quality assurance and they can manage qualified assessors and they can deliver at least 200 Scores in the first year.
I'm not going to go through the benefits. Basically, I think there are a lot of benefits of becoming a Partner, but you need to be ready to, I wouldn't rush into it, if you're not ready.
We will provide you with some assistance from our own staff here, our own team who are speaking periodically on the call here, in terms of technical assistance with the Scoring Tool, as well as just getting started.
In order to become a Partner, you need to sign the Partner Agreement, you need to manage the qualified assessors in your area, that includes setting up the exam, proctoring it, or having somebody Proctor it and hopefully mentoring your assessors as they continue and work through the process.
In terms of quality assurance, you need to be prepared to rescore 5% of the homes and then monitor what that data suggests, in terms of the effectiveness of your different assessors, whether or not there is too much variability, you know, in terms of the results that are coming back from your quality assurance provider versus your assessor. We will also be monitoring that information and we will do our own quality check, what we can do in terms of reviewing data, but I can go into that in more detail, if you'd like later. Regarding the Home Energy Score, we obviously have calls with our Partners and we encourage and really strongly encourage them to participate in that. What we found with the pilots is that, you know, even if you thought everyone was on the same page, sometimes we found that people weren't because of, and you got that only out of these conversations when people were chatting about what they are doing.
I already talked about this in terms of the training and certification levels.
So you should contact us if you think that you can meet the requirements. We will spend some time chatting with you, seeing if we think that you can and then if that is the case, moving forward with signing the agreement and getting your assessors training and tested, etc., and then eventually scoring homes.
[End of Audio]